Guests in a hypervisor system can be tuned to reduce the number of guest exits they trigger.
Below are brief descriptions of common guest actions that trigger exits; they should help you identify changes you can make to your guest configuration and to applications running in the guest that will improve performance.
In addition to the causes described below, guest-initiated inter-process interrupts (IPIs) may cause guest exits (see “Guest IPIs” in this chapter).
For information about how to modify your guests' behaviors to reduce the number of exits they trigger, consult the user documentation for you guest OSs. For example, for QNX Neutrino OS and QOS guests, see OS Core Components in the QNX Neutrino OS documentation set.
When a vdev (e.g., pl011 or ser8250) emulates a physical device, the guest doesn't know that it is working with a virtual device rather than with a hardware device. This means that when a guest reads or writes to one of the vdev's registers, the hypervisor must interpret the register reference so that, when it has completed the device emulation, it can update the state of the vdev to match what the guest expects from hardware. Of course, this interpretation and update requires the hypervisor to execute, which forces a guest exit.
Para-virtualized devices (e.g., virtio-blk) can't eliminate all the guest exits, but they are designed to minimize the number of times guest exits are required. When designing your system, consider using where possible para-virtualized devices rather than emulation vdevs.
For more information about emulation vdevs and para-virtualized devices, see “Emulation vdevs” and “Para-virtualized devices” in the “Understanding QNX Virtual Environments” chapter.
Boards with virtualization support are designed to minimize the number of times a guest needs to read or write privileged system registers. Nonetheless, both ARM and x86 boards still have some system registers (e.g., OSLSR_EL1 on ARM, IA32_APIC_BASE on X86) that the guest may not directly read or modify.
Thus, the virtualization hardware traps any attempt by a guest to access these registers, forcing a guest exit so that the hypervisor can emulate the operation. If you can reduce the frequency at which a guest needs to access these privileged system registers, you will reduce the frequency of your guest exits.
One of the fundamentals of hardware virtualization is the separation of hypervisor host and guest activities into different privilege levels. On ARM platforms, these privilege levels are now called exception levels (ELs); on x86 platforms, they are called rings (see “CPU privilege levels” in the “Understanding QNX Virtual Environments” chapter, and “Exception Level (EL)” and “Ring” in the “Terminology” appendix).
Guests run with less privileges than the hypervisor. When a guest attempts to perform an action for which it doesn't have sufficient privileges, the virtualization hardware traps the requests, forcing the guest to exit and the hypervisor to complete the operation. For example, on an ARM board a guest isn't permitted to execute the SMC opcode to request a Trustzone service. If a guest requests a Trustzone service it will be forced to exit so that the hypervisor can look after the request, either re-issuing the request to the host Trustzone code or handling the operation itself.
Regardless of how the hypervisor handles the guest's attempt to perform an action for which it has insufficient privileges, the attempt forces a guest exit along with its attendant costs. If you can reduce the incidents of requests for services that require privilege levels the guest doesn't have, you can improve the guest's performance, and reduce the overall guest exit–guest entrance overhead in your system, improving overall system performance.
A QNX hypervisor uses a similar technique to virtualize performance monitoring units (PMUs) and floating point units (FPUs). A guest's CPU is in fact a vCPU: a thread in the qvm process instance hosting the guest. This qvm process instance's vCPU thread, which runs in the hypervisor host, maintains the floating point state for the guest's CPU, as well as the state of the PMU.
The hypervisor host lazily switches the contexts of its threads' floating point states, which generally reduces the number of context switches it needs to do and saves time in the system overall. With this design, however, if a hypervisor guest attempts to reference its floating point state (i.e., the floating point state of a vCPU thread in its hosting qvm process instance), the physical FPU registers may contain data belonging to another thread in the hypervisor host.
In this case, the guest must exit so that the hypervisor can context switch the FPU registers, loading the values for that guest's vCPU thread. As with other guest exits, this operation requires a guest exit–guest entrance cycle along with its overhead. Reducing the frequency of a guest's attempts to reference its floating point state therefore reduces overhead and can improve performance.
Depending on the platform, programming of timers by a guest may indirectly trigger guest exits.
For example, on ARM platforms, the hypervisor doesn't need to trap a guest attempt to program a virtual timer; the programming takes place in the software of the VM hosting the guest. However, the hypervisor must trap any guest attempts to program a physical timer, which will cause a hardware interrupt and the guest to exit.
Thus, a guest OS that uses high-precision timers continuously reprogramming timer hardware will incur the overhead of the guest exits required to handle the reprogramming.