PCI enumeration in ARM linux system, is it done by BIOS or linux? What if I don't do anthing for PCIe before linux starts?

I read somewhere the PCI enumeration (finding the bus and device, functions) is done by BIOS in x86 system.(Today I saw it again in a Korean web document : https://melonicedlatte.com/computerarchitecture/2019/11/27/234300.html ). Then what about arm64 based system? I found from https://patchwork.kernel.org/project/linux-pci/patch/1404422876-1160-1-git-send-email-tinamdar@apm.com/ arm64 uses UEFI for PCIe initialization, and also found u-boot supports PCIe for arm64. Then, if I don’t use UEFI firmware and use u-boot and don’t do anything about PCIe during the u-boot, will OS (linux) do the enumeration? (of course I configure linux for PCI and my PCIe controller).

Asked By: Chan Kim


A lot to unpack here:

PCI enumeration

PCI is not the same as PCIe. The rest of the question is about PCIe, so I’ll assume this is about PCIe, not PCI.

PCIe strictly speaking doesn’t have enumeration. Because it’s not a bus, but a network of point-to-point links, with bridges (in a computer network, you’d call these network switches). At no point needs any point in the network know the full network to function per se. There’s no bus , so there’s no bus enumeration.

There will be a need that the operating system knows which devices are there. That’s doable by querying the root complex for all downstream connected devices, and each of these that is a bridge itself for all downstream connected devices.

This can all be done by the operating system, it can (unlike in PCI, typically) be done at any time without harm, and hence, the operating system itself does that. What it does then is actually set up the found hardware, gives it memory-mapped regions and so on. Linux will do this kind of enumeration.

One exception exists: Devices needed to boot (i.e., network interfaces, NVMe, graphics controllers…), which the firmware needs to even run the operating system. Obviously, these need to be found first. So: Your board firmware can hence do that, as well.

This has nothing to do with the architecture of your CPU – it just needs to be done, no matter whether you’re on x86 or arm. On some devices, what we usually call "embedded", you get to write or configure your firmware yourself so that it won’t have to try and figure out which devices are where. It still needs to talk to, and set up the devices. So, same story: "not-full-network" enumeration, but an enumeration of the pre-configured devices.

is done by BIOS in x86 system

"BIOS" is an API for your system firmware. The actual firmware might do some initialization, but the "BIOS" part of it predates PCIe; it’s not involved with that.

"UEFI" is just another API. More modern.

arm64 uses UEFI for PCIe initialization

What exactly is "PCIe initialization"? You need to be very precise here.

Again, ARM64 has nothing to do with whether you have PCIe or how your firmware will interact with PCIe. Simply different things. You can have a very PC-alike computer architecture and firmware with an ARM, or PowerPC or… CPU, and you can have a very non-PC architecture with x86 inside.

Also, UEFI can be anything; the name literally says "extensible firmware interface". There’s UEFI modules that offer a fully-fledged web server! These of course need other UEFI modules to offer a TCP/IP stack, and these need a network interface driver, and that will need to do a bit of PCIe enumeration to find and talk to the network interface.
You can, however, also build a UEFI that doesn’t contain such things, and hence won’t need to do PCIe enumeration.

BUT: somehow, somewhere, you will need to load your software bootloader / operating system. On many computers, the storage device containing that is only reachable via PCIe. So, some kind of setup needs to happen.

On others, for example, embedded x86 SoC or some ARM SoC, the bootloader is on some eMMC and that’s more or less directly attached to the CPU. It all depends on the design of your computer!

CPU architecture has little to do with peripheral interface design. These are simply separate things.

For example, the Playstation 4 is x86 and has PCIe, but it’s not a PC and does not try to have BIOS, the hardware doesn’t have a PC-compatible timer or interrupt controller.

Then, if I don’t use UEFI firmware and use u-boot a

As the name UEF Interface suggest, UEFI is just the software API that your firmware offers (or not offers). That has little to do with what it internally does to the hardware.

Answered By: Marcus Müller
Categories: Answers Tags:
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.