Mapping DMA devices and memory regions

The preferred method for mapping DMA devices to memory regions and specifying their access permissions is through the SMMUMAN client API.

After the smmuman service has started, processes such as device drivers or external applications can use the client API to connect to the service as clients and use it to program the system IOMMU/SMMUs. This API is defined in the smmu.h header file and made available in the libsmmu.a library. Clients can use this API to:

add or remove DMA devices
add or remove memory regions that DMA devices are permitted to access, or modify the devices' access permissions
have the smmuman service inform them of illegal DMA device attempts to access memory

CAUTION:

Until the smmuman service programs the IOMMU/SMMUs on a board with a DMA device's memory access permissions, that device has unrestricted access to memory.

Task overview

A smmuman client using the service to map DMA devices and memory regions typically does the following:

Obtains the custom abilities it will need to connect to the smmuman (see “Connecting to the service” below).
Calls smmu_init() to connect to the smmuman service.
Calls smmu_obj_create() to create SMMU objects.
Calls the smmu_device_add_*() functions to add DMA devices to the SMMU objects.
Calls smmu_mapping_add() to add the memory mappings and permissions appropriate for the DMA devices attached to the SMMU objects.
As needed, calls smmu_mapping_add() or the appropriate smmu_device_add_*() function to remove devices or memory mappings that are no longer needed.
Calls smmu_obj_destroy() to destroy SMMU objects that are no longer needed and, finally, calls smmu_fini() to terminate the connection with the smmuman service.

See Monitoring DMA transgressions below for instructions on how to have the smmuman service notify a client of illegal DMA device attempts to access memory.

Connecting to the service

To use the SMMUMAN API, a process in your system must become a client of the smmuman service.

Before initializing contact with the service, the process must obtain the appropriate custom abilities:

SMMU_ABILITY_ATTACH_NAME ("smmu/attach") – required for all connections to the service
SMMU_ABILITY_TARGET_NAME ("smmu/target") – required if the client will use the SMF_TARGET flag when calling smmu_mapping_add() (currently reserved for the QNX Hypervisor)

Use procmgr_ability_lookup() and procmgr_ability() to look up and control these custom abilities. For more information, see smmu_init() in the SMMUMAN Client API Reference chapter, and procmgr_ability_lookup() and procmgr_ability() in the C Library Reference.

After your process has the required abilities, call smmu_init() to check if the smmuman service is running and connect to it as a client.

Checking the service's safety status

If you are running a safety-related system, you must use the smmuman safety variant. A smmuman client can call smmu_safety() to check if the smmuman service to which it is connected is the safety variant.

See safety in “Global options” for information about configuring your system's response to the presence of components that aren't safety-certified.

Creating SMMU objects

The smmuman service manages dynamic memory region mappings, and DMA device mappings and permissions through SMMU objects. Thus, at any time after it has initiated contact with the smmuman service and become its client, your process can call smmu_obj_create() to create a SMMU object to which it can then add memory mappings and DMA devices.

A smmuman client may create as many SMMU objects as it needs; this allows different devices to have different mappings to memory regions. For example, the QNX Hypervisor qvm processes use this capability.

Additionally, at any time while the smmuman service is running, your client can remove a DMA device from a SMMU object by calling the relevant smmu_device_add_*() function with the sop argument set to NULL.

Adding DMA devices

After your smmuman client has created one or more SMMU objects, it can add devices to the objects. Call one of the following for each DMA device that needs to access memory:

smmu_device_add_generic() — add a device of an unspecified type to a SMMU object
smmu_device_add_mmio() — add a Memory-Mapped I/O (MMIO) device to a SMMU object
smmu_device_add_pci() — add a Peripheral Component Interconnect (PCI) device to a SMMU object

If you are adding a device to an object after you have added a memory mapping to that object, you may need to refresh the memory mappings (see “Preferred sequence for adding memory mappings and devices”).

Note:

All devices attached to a SMMU object are granted access to the memory regions mapped to that object, with the same permissisons. If you want to give two devices access to the same memory region, but with different permissions (e.g., one device may only read, the other may only write), then you must create two separate SMMU objects (see smmu_mapping_add()).

A device may have only one owner. Attempting to use one of the smmu_device_add_*() functions when the device has already been added to a SMMU object of a different client will result in an EBUSY error. A smmuman client may move a device from one of its SMMU objects to another one of its SMMU objects, however, because this action doesn't change the device owner.

Adding memory mappings

After your smmuman client has created one or more SMMU objects, it can add memory regions to these objects, specifying the access permissions that will be applied to the devices attached to each object. Call smmu_mapping_add() to add a memory region to an object, so that devices attached to the object can have access to this memory region.

Preferred sequence for adding memory mappings and devices

The preferred sequence for adding memory regions and devices to a SMMU object is to add the devices first, then add the memory regions. These memory regions will apply to all the devices linked to the SMMU object.

This is not always possible, however, and you may need to add the memory region mappings first. For example, if a USB device is added to your system after startup, you may need to add it to a memory region mapping you created earlier.

If you add the mappings first, and then you add a device, the smmu_device_add_*() function may return -1 (failure), 0 (nothing more to do), or +1 (see smmu_device_add_generic() and the other smmu_device_add_*() functions).

A +1 return from these functions means that the hardware module used for the new device has indicated that page tables for this device are needed. You address this by calling smmu_mapping_add() again for all the active mappings on the SMMU object.

Note:

After a +1 return from a smmu_device_add_*() function:

Devices that were previously added to the SMMU object may continue their DMA operations; they don't have to wait on the call to smmu_mapping_add().
You need to add only the active mappings again; you don't need to do anything about mappings you deleted from the SMMU object.

Removing devices and memory regions

If a DMA device is no longer on the system (e.g., USB device has been removed), you can call the appropriate smmu_device_add_*() function with its sop argument set to NULL to Adding DMA devices remove the device from a SMMU object.

If the devices attached to a SMMU object no longer require access to some memory regions mapped to a SMMU object, you can remove these regions from the object by calling smmu_mapping_add() without specifying any of the SMF_READ, SMF_WRITE, or SMF_EXEC permissions for the flags argument (i.e., the flags argument set to SMF_NONE; see smmu_mapping_flags in the “SMMUMAN Client API Reference”).

Disconnecting your smmuman client

If your client no longer needs to interface with the smmuman service, it can disconnect from it. To disconnect from the smmuman service, a client must:

For every SMMU object it has created, call smmu_obj_destroy() to destroy each SMMU object in turn.
After it has destroyed all its SMMU objects, call smmu_fini() to terminate its connection with the smmuman service.

Monitoring DMA transgressions

After the smmuman service programs the memory mappings for DMA devices into the system IOMMU/SMMUs, these units will inform it of any attempts by these devices to access memory outside their assigned regions.

A smmuman client can register to be informed of these transgressions. This client doesn't have to be the client that owns the SMMU objects and has added devices and mapped memory regions to it. You could have some clients (e.g., device drivers) create objects, add devices and map memory regions, and another client (i.e., some other process in your system) monitor transgressions.

Call smmu_xfer_notify() to register for updates.
Call smmu_xfer_status() at any time to query the smmuman service for any records it may have of attempted transgressions.
Call smmu_xfer_notify() to reregister for updates.

When the smmuman service informs a client that an IOMMU/SMMU has refused a DMA device's memory access attempt, it clears that client's registration for notification of DMA device memory access transgressions. The smmu_xfer_notify() call after the smmu_xfer_status() call is required to reregister the client so it will receive notifications of any subsequent attempted transgressions.