How does PCIe Endpoint device memory is mapped into the systems memory map (MMIO)?

Question

How does Linux Kernel or BIOS map the PCIe endpoint device memory into systems MMIO space ? Is there any API to achieve it ? Lets assume that when writing a Linux device driver for a PCIe endpoint device, How can we map PCIe device memory into MMIO space ? Or Is it true that the device is already mapped into MMIO by BIOS during enumeration and what I would need to do it just remap the device MMIO into the kernel virtual address space using ioremap() ?

Platform : Linux on x86

Answer 1

There are two parts to this answer

Role of the BIOS

The BIOS (typically UEFI based) will do some sort of Depth-First Search (DFS) and enumerate all the children as PCIe is a self-enumerating bus. Since it has the view of the world (device, buses, processors) it will write an address to the BAR registers (could be BAR0 and or multiple of them). This will be the address the system will use and it will actually route these requests from the Host Agent (HA on x86/Intel platforms) to the Root Port to a PCIe switch all the way to the end point.

Each of these elements track what address ranges belong to themselves or one of their child devices (example a Switch may be the child of a Root Port)

Role of the Device Driver

The OS/Kernel will provide a toolkit of helper routines that the driver authors will use to access the device registers. Typically a driver may follow the folling routines

This is some sample driver pseudo-code, just to help illustrate the idea

1. pci_resource_flags(pdev, 0) & IORESOURCE_MEM

Check if a resource region is valid, here check for BAR 0

2. pci_request_regions(pdev, "region")

Take ownership of the resource/region

3. drv->registers = pci_iomap(pdev, 0, SIZE_YOU_WANT_TO_MAP)

This will give you kernel virtual address to device register mapping

Note : In case the BIOS does not enumerate, through Linux one can rescan the PCIe tree to see if a device can be seen or not.