QSA with QNX Acoustics Management Platform

Concurrent access on playback and capture paths

QNX AMP extends QSA with signal processing functionality using the io-audio Low-Latency Audio Architecture, into which one or more AMP Functional Modules (AFMs) and Acoustic Processing Extension libraries (APXs) are loaded.

Figure 1. AMP audio flow.

AFMs are audio processing modules that connect to PCM devices in the digital domain to perform specialized audio processing functions. In most cases, an AFM is configured to source data from a PCM input splitter and sink data to a PCM software mixer. This configuration allows the AFM to run in parallel with other audio functions and applications that use the same audio PCM hardware device. AFMs provide solutions for features such as acoustic echo cancellation, chime and safety alert monitoring, engine order reduction, engine sound enhancement, external pedestrian alerts, in-car communication, and noise reduction. An AFM can also act as a simple bridge between capture on one PCM device and playback on another.

In contrast, APXs reside within PCM software mixers and input splitters, where they expand and optimize the perceptual quality of an audio stream, as well as provide tap points for audio streaming and injection for analysis purposes.

The low-latency software framework enables the implementation of demanding acoustic algorithms on application processors rather than dedicated Digital Signal Processors (DSPs). It extends the existing audio infrastructure in QNX Neutrino in a modular, completely backwards-compatible way so that applications such as telephony, speech recognition, navigation, and media playback all run entirely unmodified and interact positively with the acoustic signal processing functions of the system.

QSA allows a single audio input and output hardware controller to service one or more microphones and one or more speakers within your target environment. To be able to share these devices between modules/processes/threads concurrently, a PCM software mixer on the playback path mixes multiple audio streams into a single output audio stream that targets the main speakers, and a PCM input splitter on the capture path replicates and distributes the single microphone (mono or multichannel) audio stream to different modules/processes/threads. An example of concurrent access on the playback path is having the media player, the In Car Communications (ICC) module, and the QNX Acoustics for Voice (QAV) module concurrently writing audio to the playback device for one or more cabin speakers. An example on the capture path is having a voice recognition application, the ICC module, and the QAV module concurrently sourcing data from the capture device for one or more cabin microphones. Without the PCM software mixer for playback and the PCM input splitter for capture, only a single module/process/thread can access a hardware device at any given time.

LiveAMP and Acoustics Control Server (ACS)

QNX AMP includes LiveAMP, a set of tuning, diagnosis, and analysis tools that enables real-time adjustment of system parameters, signal streaming, and injection at multiple tap-points in the system, as well as real-time spectrum and waveform displays. AFMs are controlled remotely using the Acoustics Control Server (ACS), which seamlessly links LiveAMP to the AFM and APX modules in io-audio.

AMP audio devices

In addition to the audio devices supported by QSA, AMP publishes the following devices:

AFM devices (/dev/snd/afmCxDy)

AFM devices are the entry points for control and status of an AFM. The snd_afm_* () functions can be used to open and interact with an AFM. To simplify AFM access, a symbolic link that corresponds to the related acoustic processing library is also published (e.g., for the In-Car Communication module, /dev/snd/icc).

AFM PCM devices (/dev/snd/pcmCxDy)

Some AFMs create a PCM device to provide an application with an audio interface to AFM processing. This device simply appears as a new playback, capture, or playback and capture PCM device with a card and device number that matches the associated AFM. Data written to this class of PCM playback device passes through the AFM processing algorithms before being passed to a PCM hardware or software mixer device. The snd_pcm_*() functions can be used to open and interact with the PCM device in the same manner as non-AFM PCM devices. To simplify PCM access, symbolic links (ending with "p" for playback or "c" for capture) are also published (e.g., for the QNX Acoustics for Voice module, /dev/snd/voicep and dev/snd/voicec are added).

AFM mixer devices (/dev/snd/mixerCxDy)

AFMs that create a PCM device can also create a mixer device if volume control is enabled. The card and device number for the mixer device match the associated AFM. The snd_mixer_* () functions can be used to open and interact with the mixer device in the same manner as non-AFM mixer devices.

Because Acoustic Processing Extension (APX) libraries reside within PCM software mixers or input splitters, they do not exist as separate devices. However, the snd_apx_*() functions can still be used to interact with them through the associated PCM device.

Audio library

AMP is supported by the snd_afm_*() and snd_apx_*() functions. In addition to basic controls and status, they also provide an interface to input Vehicle Input (VIN) data from the CAN bus for use by AFMs. For more information, see Audio Library.

Configuring io-audio

The -c option for io-audio allows you to configure the low-latency software framework using a configuration file that is loaded at startup.

You use the configuration file to load the DLLs that provide QNX AMP functionality.

Audio devices are indexed based on the order in which their respective audio drivers are mounted in the io-audio process. Keep this in mind when you create your audio configuration file. The order of the [ctrl] sections determines the order in which the audio drivers are mounted into the io-audio process and thus determine the index of each audio device. Generally, this means you want the [ctrl] section for your main audio device (main microphone and speakers) to be the first [ctrl] section in the audio configuration file and any secondary audio devices (Bluetooth, radio tuner, and so on) and AFMs connected to those devices to follow it.

Audio device indexing uses card and device numbers. Each [ctrl] or [afm] creates a card, with the card indices starting at 0 and incrementing for each section of [ctrl] or [afm] type. Each card has 0 or many devices under it. (If there is an [audiomgmt] section, it is assigned card 0 unless a different value is specified using the unit key.)

You can use the [acs] section in the audio configuration file to load the deva-acs-link DLL. This DLL creates an afm entry point and instantiates the ACS link.

See io-audio in the Utilities Reference for a detailed description of io-audio configuration file.

AMP and audio concurrency management

The audio ducking feature affects AFM audio streams differently than other audio streams when routed through the AFM mixer path in the PCM software mixer (the default routing).

• QSA audio concurrency management never ducks the audio stream of an AFM.
• ASD, CSA, and ICC AFMs do not duck any other audio streams (in typical configurations, these AFMs are always on and would permanently duck other audio in the cabin, if it were allowed).
• You can configure the QAV and Bridge AFMs to duck specific audio streams that play through the main mixer audio path (for example, media playback). However, routing the Bridge AFM through the AFM mixer path is not a typical use case for that AFM.