The file compression mechanism provided with the QNX flash filesystem is a convenient way to cut flash memory costs. It can also be used to reduce system startup times.
The QNX flash filesystem uses popular deflate/inflate algorithms for lossless file compression and decompression of both data files and executable files:
File compression can produce significant space savings. However, it takes longer to access compressed files, and, in fact, all files accessed by inflator.
When compressed files are present every file access must first go to inflator, which either accepts and decompresses the file, or declines it. This step requires message passing and context switches on every open() call.
You can reduce this overhead by pointing inflator only at directories or mountpoints that contain compressed files, or by restricting where compressed files are used.
For more information, see “Compressing files” and “Decompressing files” below, and the Boot Optimization Guide.
When working with compressed files in a flash filesystem, you need to remember that compressed files have two sizes:
For instance, running the disk usage utility du would be practically meaningless under a flash directory with data that is decompressed on the fly. It wouldn't reflect flash media usage at all. It would calculate disk usage based on the virtual sizes of the files, because by looking at the file it would cause the flash filesystem to decompress them.
As a convenience, inflator supports a naming convention that lets you access a compressed file without decompressing it: simply add .~~~ (a period and three tildes) to the file name. If you use this extension, the file doesn't get decompressed, so read operations yield raw compressed data instead of the decompressed data. For example, to get the virtual size of a compressed file, type:
ls -l my_file
but to get the media size, type:
ls -l my_file.~~~
The flash filesystem never compresses any files, but you can decide to compress files at different times:
The most common procedure is to use the deflate utility as a filter for mkefs to compress the files that get built into the flash filesystem. For example, you could create a 16-megabyte filesystem with compression by using the following attributes in the mkefs buildfile:
[block_size=128K spare_blocks=1 min_size=16m filter=deflate] /bin/
You can also use deflate directly in the command-line to pre-compress the files. When mkefs puts a file into a filesystem it is creating, it detects compression signatures; it knows if a file has been pre-compressed, and doesn't attempt to compress it again.
If you have already created the flash filesystem, you can use deflate to compress files, then add them directly in the flash filesystem. For example, here's how to use deflate at the command line to compress the ls file from the image filesystem into a flash filesystem:
$ deflate /proc/boot/ls -o /fs0p0/ls
For more information, see deflate in the Utilities Reference.
The .~~~ extension provides a simple and efficient way to get information about compressed files when you read them. Unfortunately, there is no simple way to write compressed files. The following rules apply when working with compressed files in a flash filesystem:
If you open a compressed file with O_TRUNC from the regular virtual namespace (i.e., without the .~~~ extensions), then the file status will be just as if it were created from this namespace; this is the equivalent of replacing a compressed file.
In other words, if you use the O_TRUNC flag when you open a compressed file, the file will be presented as an uncompressed file with no data. The file size will be zero (0) bytes and not compressed. You will have full POSIX capabilities to work with the file (read/write, etc.).
You can use the inflator resource manager to decompress files on your filesystem as they're accessed. When you run inflator without arguments, it takes over /, placing it in front of any existing filesystems. It then catches each open() first.
If the file is being opened to be read, inflator attempts to open the file itself on an underlying filesystem, checking for the signature of a compressed (deflated) file. If the file was deflated, inflator places itself between the application and the underlying filesystem, so that all reads return the original file data before it was compressed.
This abstraction layer in inflator achieves efficiency and preserves POSIX compliance. Special compressed data headers on top of the flash files provide fast seek times.
This layering is straightforward. Specific I/O functions include those handling the three basic access calls for compressed files:
Because using inflator increases access time, not just to compressed files, but to to all the files it examines, if you know where the compressed files are located, you may wish to limited the directories where you use inflator. For example:
inflator /sbin /bin /usr/sbin /usr/bin
has inflator take over directories where executables are usually located, and inflate files that have been compressed when applications open them for reading.
For more information, see deflate in the Utilities Reference.