The original PCI specification defined a mechanism for extending itself based on the idea of adding capability fields with the configuration space of a device.
PCI Express (PCIe) is now supplanting most new PCI-based designs, but from a software perspective, it's just additional functionality that's managed by a set of defined PCIe capability registers within PCI configuration space. This mechanism has allowed the PCI specification to evolve while maintaining backward compatibility with previous software installations.
The PCI server architecture enables the use of new hardware capabilities in an analogous fashion with the ability to deliver access to these new capabilities with independent software modules and associated APIs.
Capability modules reside in ${QNX_TARGET}/processor/lib/dll/pci/ on your development host, and in /lib/dll/pci/ on your target. Their names are prefixed with pci_cap- or pcie_xcap- for PCI or PCIe extended capabilities, respectively. There is “use” information in each module explaining what capability it's for, its supported APIs, and how to use it. See also the Capability Modules and APIs appendix.
Similar to the hardware-dependent and logging modules, capability modules are demand-loaded and automatically managed for users, but unlike those other modules, capability modules aren't loaded until software decides that they're required. As an example, a device might support both MSI and MSI-X capabilities for message signalled interrupts, however a driver for that device can decide which of the two it prefers and load only the selected one. In fact all capability modules are managed this way, even the PCI Express capability.
The name of a capability module is in one of these forms:
where nn and nnnn are the PCISIG-assigned capability IDs. The number of digits is important; it must be two for PCI capability IDs, and four for PCIe extended capability IDs. For example:
To add a capability to the PCI server, put a capability DLL into the /lib/dll/pci/ directory (or into the directory named by the PCI_CAP_MODULE_DIR environment variable). A capability header file (if required) defines the APIs to the specific capability. Capability header files are typically named with the name of the capability and not the ID (e.g., cap_msi.h). This is to allow them to be more easily identified within the files that include them.
The goal of this mechanism is that only the users of the capability (i.e., driver software) and the capability provider (i.e., the pci_cap-0xID.so DLL) will need to be updated. All other modules, the PCI server library, and the PCI server itself typically remain unchanged. (Depending on the capability, it may be necessary to update the PCI server itself to also use a capability.)
In order to allow (and in fact force) drivers to understand, select, and configure which capabilities they want to use, the PCI server architecture pushes capability processing into the driver software. This has the following advantages:
In order for a capability module to be used, it must be known to both the driver wanting to use it and the PCI server process. If a capability module is blacklisted (see the PCI_MODULE_BLACKLIST environment variable) in the PCI server, it can't be used even if it isn't blacklisted in the driver process. Similarly, if you use the PCI_CAP_MODULE_DIR environment variable to specify an alternate directory for capability modules (from the default /lib/dll/pci), you must still make sure that any capability that's to be used in the system is visible to both the PCI server and the driver wishing to use it. You can accomplish this by making sure both have the same PCI_CAP_MODULE_DIR or by ensuring that the directory identified by PCI_CAP_MODULE_DIR and the default directory (/lib/dll/pci) both contain the desired capability module (via a symlink or copy).
Note that the PCI server process must have access to the capability modules required by any driver on the system, not just those of a specific driver, and therefore the PCI server process typically uses the default location for capability modules (i.e., it doesn't have a PCI_CAP_MODULE_DIR environment variable). You can then use PCI_CAP_MODULE_DIR to restrict the availability of capabilities to specific drivers.
You can also use this environment variable to simply modify the default location of capability modules for every process.
Driver software has two choices for discovering the supported capabilities of the device it's managing. It can call pci_device_find_capid() to determine if a specific capability exists, or it can scan for all device capabilities with repeated calls to pci_device_read_capid(). Once the driver determines which capabilities the device supports and which ones it intends to enable, it can call pci_device_read_cap() to obtain a handle (of type pci_cap_t) to the capability. The act of obtaining the handle is what triggers the capability module to be loaded, and therefore the driver has complete control over this process. Only after a successful call to pci_device_read_cap() can the driver use any capability-specific APIs, since it's the pci_cap_t handle that's required by all capability module APIs.
If a device supports a capability for which there's no capability module, pci_device_read_cap() returns an error.
Note that there's currently no way to unload a capability module. Once a module is read, it becomes part of the running process image; therefore in order to minimize the runtime image size, drivers should read in only the capability modules that they intend to use.
The other APIs related to capabilities processing that are available to driver software include:Before a capability can be used, it typically must be enabled. Before enabling a capability, the driver should use the capability APIs to configure it. Enabling the capability causes the current capability configuration to take effect. If a change is to be made to the capabilities configuration, the driver should disable the capability, reconfigure it using the capability APIs, and then re-enable it.
Capability modules are written to comply with specifications and necessarily use the hardware-dependent module (which itself can deal with certain types of device or platform errata), however in order to handle device errata or out-of-spec behavior related to device capabilities, the following mechanism is provided. Since a specific PCI device is uniquely identified by its vendor and device IDs, we allow for a device-specific capability module using the same naming convention with an additional tag that consists of the vendor and device IDs (vid and did): pci_cap-id-viddid.so (e.g., pci_cap-0x10-808610d3.so).
When a capability module is to be loaded (triggered by pci_device_read_cap()), an attempt is first made to load the capability module with a matching viddid tag as part of the filename; if found, then this module is used (discovering an already loaded module with the same vid and did means that this would be the second instance of the same device). If not found, then an attempt to find the generic capability module is made. If one is found that's already loaded, it's used; otherwise the generically named capability module is loaded. If none can be found, the capability isn't supported.
This mechanism allows the inclusion of a capability module for a specific device by simply dropping the appropriately named file into the /lib/dll/pci/ directory (or the directory identified by PCI_CAP_MODULE_DIR) and restarting the device driver and PCI server process. (Vendor and device IDs are all that's used to uniquely identify a specific capability module. If multiple devices with the same vid/did exist and the named module is applicable to only one of the devices, then the specific module must handle both devices.)