Using Lossless Compression

Applicable products

The lossless compression feature is available on the following Teledyne FLIR machine vision Blackfly S GigE cameras:

Application note description

This application note describes the lossless compression feature available on select FLIR machine vision cameras.

What is lossless compression?

Lossless compression is a feature that uses a lossless compression algorithm to reduce the amount of data transmitted from the camera while maintaining the image quality. Compressed images can be streamed in a Teledyne FLIR proprietary format with Spinnaker and then saved to disk. These saved images can later be loaded into Spinnaker for processing.

The benefits include:

• Increase in frame rate
• Decrease in data over the link to allow for more cameras
• Decrease in space required to store the images on disk

As an example, a configuration with the BFS-PGE-50S5 (Sony IMX264) without compression might look like the following:

Using lossless compression with a compression ratio of 1:1.8 and 8-bit pixel format, the configuration could be:

System requirements

Lossless compression is supported in the Spinnaker SDK version 2.4.0.143 (or later).

You may need to upgrade your firmware to include lossless compression.
To download new software or firmware go to:  BFS-PGE Resources and Support and click the Software & Firmware tab.

The feature is supported in the following firmware (or later):

To determine the firmware version number of your camera:

• Query the GenICam Device Control feature DeviceFirmwareVersion.

Camera firmware can be upgraded using SpinView, part of the Spinnaker SDK.

To upgrade the firmware:

1. Download the firmware from the  BFS-PGE Resources and Support Software & Firmware tab.
2. With your camera connected, start the SpinView application.
3. From the Device list, right click the camera and select Update Device Firmware.
   • If you get a Device is Active warning, close the Display pane or click the Disconnect button and right click the camera again.
4. Browse to select the firmware file and click Open.
5. Click Yes to continue.

Do not disconnect the camera during the firmware update process.

Lossless compression is best used in situations where the scene is fixed. Dynamic scenes do not compress as much.

How to enable lossless compression feature

By default, lossless compression is turned off. All the compression nodes are available to access from the Compression tab in SpinView.

To use lossless compression:

1. Select the Pixel Format to be either Mono8 or BayerRG8.
2. Ensure ISP Enable is not selected (disabled).
3. From the Image Compression Mode drop-down, select Lossless. If this node is locked, ensure that ISP has been disabled and that a supported pixel format (Mono8 or BayerRG8) is selected.
4. Select the Adaptive Compression Enable checkbox to allow dynamic updates to compression tables to achieve a better compression ratio. If unchecked, a fixed encoding table is used. Fixed encoding is optimized for typical scenes and provides good performance for a wide variety of images.
5. Set the Max Datarate Threshold to be the maximum desired datarate in B/s. The camera either reduces the frame rate to stay at (or below) the maximum datarate, or drops frames to maintain the frame rate and not exceed the datarate. The method used depends on whether AcquisitionFrameRate is enabled (Drop Frames) or disabled (Reduce Frame Rate).
6. Set Acquisition Frame Rate to be enabled or disabled depending on the desired saturation priority. Enabled prioritizes maintaining the frame rate over dropping frames. Disabled adjusts the frame rate to remain at or below the maximum datarate.

Additional nodes

There are a number of read-only nodes to provide the following information:

• Compression Ratio—The ratio between size of data to transmit and actual uncompressed image size.
• Compression Saturation Priority—Dependent on AcquisitionFrameRate. If frame rate is enabled, the priority is to maintain that frame rate and drop frames if the maximum datarate is reached. If frame rate is disabled, the priority is to reduce the frame rate to achieve the maximum datarate.
• Current Datarate—The compressed datarate of the camera in B/s.
• Compressed FrameDrop Count—The number of frames dropped to keep the datarate below the maximum.

Lossless compression also adds the following information in Chunk Data:

• Chunk Compression Mode—Specifies whether image data is compressed.
• Chunk Current Datarate—Current datarate as calculated by the bitrate control algorithm in B/s. This is only used if frame is compressed.
• Chunk Compression Ratio—Specifies compression ratio for the current frame. This is only used if frame is compressed.

Compression saturation priority

A challenge when compressing images is that they change in size when the scene changes. This change in size can cause the bitrate of the interface to exceed its maximum. There are two modes for handling this scenario:

• Drop frames
• Reduce frame rate

The modes are controlled by the Acquisition Frame Rate Enable node. The method in use is reported by the Compression Saturation Priority node.

Method 1—Drop Frames

The Drop Frame mode prioritizes keeping the frame rate constant over dropping frame. It is best for situations with a relatively constant compression ratio where a constant frame rate is required. Users must determine an appropriate frame rate to minimize the chance of dropping an image. Current and previous frames are averaged to determine if a frame should be dropped.

To use this mode, enable Acquisition Frame Rate.

Method 2—Reduce Frame Rate

The Reduce Frame Rate mode prioritizes keeping the datarate under the threshold over maintaining a constant frame rate. It is best for situations where a variable frame rate is acceptable and a variable compression ratio is expected. The camera determines an acceptable frame rate to prevent dropped frames.

Note that the frame rate may be slightly lower than can be achieved with Drop Frame method (if near the maximum threshold). The default target bitrate threshold is 6% lower than the maximum.

To use this mode, disable Acquisition Frame Rate.

Compression saturation priority example

In a configuration with the BFS-PGE-23S3 (Sony IMX392) with one camera per one GigE NIC, with lossless compression enabled and a compression ratio of 1:1.8 (8-bit pixel format), the camera can achieve 80 FPS using bandwidth of 102 MB/s.

In this example if the compression ratio were to lower to 1.2 for 5 frames, the bitrate would be exceeded. How the camera behaves is dependent on the Compression Saturation Priority.

If the Compression Saturation Priority is DropFrame then the Acquisition Frame Rate of 80 FPS is maintained and approximately 2 frames are dropped.

Note that the first frame is not dropped as the average has not yet exceeded the bitrate threshold.

If the Compression Saturation Priority is ReduceFrameRate then the frame rate is reduced to ~61 FPS while the compression ratio is 1.2. When the compression ratio returns to 1.8, the Acquisition Frame Rate increases back to 80 FPS.

Note that frame timing shifts right as the frame rate is reduced.

Compression performance

The following factors can affect the compression performance:

• Complexity of the scene—Compression performance is scene dependent. Plain, textureless, static scenes have the best compression ratio.
• Sensor characterization, including noise levels—Difference sensors have different characterization, including noise level. This could lead to different compression ratios with different sensors for the same scene.
• Adaptive algorithm—FLIR's lossless compression is adaptive and takes 1-3 frames to achieve an optimal compression ratio.
• Gain/Noise—Lower gain means lower noise level which leads to a better compression ratio.
• Sharpness—An unfocused image means blurred, low spatial frequency which leads to a worse compression ratio.
• Resolution/Region of Interest (ROI)—A smaller ROI leads to a slightly better compression ratio in most cases.
• Exposure—A higher exposure time (with the same gain) creates a larger variance in pixel intensity which leads to a worse compression ratio.

API example

The Spinnaker SDK comes with a compression example located in:

Program Files-->FLIR Systems-->Spinnaker-->src-->Compression