1. Support Center
  2. Understanding Buffer Handling

Understanding Buffer Handling

Applicable products

All Teledyne FLIR machine vision GigE Vision and USB3 Vision cameras

For Blackfly, Chameleon3, Flea2, Flea3, Grasshopper, Grasshopper2 and Grasshopper3 see Accessing the On-Camera Frame Buffer.

Application note description

This application note describes what buffers and buffer handling modes are and how they work, and provides examples on what to expect when capturing images.

What are buffers?

When an image is captured by a camera it is automatically transferred to the PC and stored in the PC's RAM. The space assigned to store that image in memory is called a buffer. By default, the number of buffers allocated in memory is automatically assigned, based on the maximum frame rate of the camera. The number of buffers associated with a camera can be changed by switching the stream buffer count mode from auto to manual and assigning a value to the manual stream buffer count. The maximum number of buffers is determined by the available memory in the RAM and the size of each buffer is dependent on the image size. The minimum number of buffers you can set is 3.

Auto buffer count mode is a feature of Spinnaker version 1.13 or later. Previous versions would set the number of buffers to 10 by default.

A user can only interact with an image once the image has been retrieved from the buffer. Such interaction includes displaying the image on a screen, saving to disk, or applying image processing.

Buffer handling

Buffer handling determines the number of buffers available for each camera, if they can be overwritten, and the order in which images are retrieved.

Buffer handling modes

In Spinnaker, there are four types of buffer handling:

    • Newest First—The application gets the image from the tail of the output buffer queue (the newest available one). This is useful for applications like motion prediction where it is critical to always get the newest frame but older frames can also add valuable information.

    • Newest Only—The application always gets the latest completed image (the newest one). Older images are discarded. Stream buffer count is ignored. This mode is typically used in a live display GUI where it is important that there is no lag between camera and display.

 

    • Oldest First—The application always gets the image from the head of the output buffer queue (the oldest available one).

    • Oldest First Overwrite—The application always gets the image from the head of the output buffer queue (the oldest available). If a new image arrives it overwrites the existing buffer from the head of the queue. This behaves like a circular buffer.

If the output buffer queue is empty, the application waits for a new image until the specified timeout value passes. By default, the timeout value is set to infinite.

The default buffer handling mode for Spinnaker is Oldest First.

In FlyCapture2 there are two handling modes:

  • Drop Frames—the equivalent to Newest Only.
  • Buffer Frames—the equivalent to Oldest First.

The default buffer handling mode in FlyCapture2 is Drop Frames.

Examples of buffer handling modes

Example 1

Trigger the camera 6 times. After each trigger, we retrieve an image and output the FrameID (the Frame ID always starts at 0).

Example conditions:

  • 1 FPS
  • No image processing
  • No changes to the number of buffers or buffer handling mode

Example results:

  • 0, 1, 2, 3, 4, 5

Regardless of the buffer handling mode used, the results should be the same. However, if the camera is triggering faster than the host is capable of handling, or if there is a delay on the host end, then images may be dropped. If so, the outputted frame IDs may not match what is in the results shown above. Also, in a situation where images are not retrieved at the same pace as they are transferred, the buffer handling mode determines which images are retrieved as well as their sequence.

Example 2

Trigger the camera 6 times. After the sixth trigger, we start retrieving three images and output the FrameIDs.

Example conditions:

  • 1 FPS
  • No image processing
  • Set the number of buffers to 3 (the PC only keeps 3 images in memory)
  • Cycle through each buffer handling mode

Example results for each mode:

  • NewestFirst
    • 2, 1, 0
    • No overwrite, the camera keeps the first three images captured, and drops frames 3, 4, 5
    • For Blackfly S USB3 cameras, the sequence is 2, 3, 4. For other USB3 cameras the sequence is 2, 3, 1. See Transfer Queue section below.
  • NewestOnly
    • 5, ERROR
    • The camera keeps the latest image, but after trying to retrieve an image that doesn’t exist, an error occurs
  • Oldest First
    • 0, 1, 2
    • Same as newest first, except the oldest image is retrieved first
  • OldestFirstOverwrite
    • 5, ERROR
    • Frames 0, 1, and 2 have been overwritten, and 3 and 4 are not available; for overwrite buffer handling modes such as OldestFirstOverwrite, it is expected that two buffers are used to cycle images within Spinnaker acquisition engine.  So when the buffer count is set to three, only one image is expected to return to the user so fetching another image without additional triggers will return an error.  Setting the number of buffers to 4 would give the output 4, 5, ERROR.

Example code

In Spinnaker release 1.15 or later, the example code Buffer Handling comes with the installation of the Spinnaker SDK. This example code demonstrates how the different buffer handling modes work using the examples defined above.

Transfer queue (on-camera image buffer)

Every camera has an on-camera image buffer and each image captured passes through this buffer. It can also be used to store images indefinitely so long as the camera remains powered. Multiple images can be stored on the image buffer.

Blackfly S and Oryx cameras have a 240 MB image buffer. The amount of space taken up by each image is four times the maximum resolution of the camera. For example the BFS-U3-13Y3C camera has a resolution of 1.3 MP so each image would take 5.2 MB of space on the image buffer. The maximum number of images that can be stored can be found on the TransferQueueMaxBlockCount node. The number of images currently buffered on the camera can be found on the TransferQueueCurrentBlockCount node.

Although Blackfly, Chameleon3, Flea3, and Grasshopper3 cameras have image buffers, they cannot use the Spinnaker Transfer Queue functionality. These cameras have to use FlyCapture2 to access on-camera image buffer functionality.

USB3 cameras only

If a USB3 camera captures an image and there are no buffers available on the PC, the image is automatically stored in the on-camera buffer. Once space is available on the PC, images in the on-camera buffer are automatically transfered to the PC. If using the Newest First handling mode, this interferes with the expected sequence of images (see the example 2 results above).

By default, Blackfly, Chameleon3, Flea3, and Grasshopper3 cameras can have up to one image buffered on the camera. Blackfly S cameras can have up to the defined value in the TransferQueueMaxBlockCount node.

Troubleshooting

Why are dropped frames occurring when I'm saving images to disk?

The speed at which RAM operates is usually much faster than the write speeds of hard drives, even those with solid state drives. With image processing and high bandwidth transfers from USB3 or GigE cameras, it's possible that the images are transferred to the PC at a faster rate than the hard disk can handle. One solution to this issue is to increase the number of buffers associated with the camera, or to use a faster hard drive system. For more information see Saving Images at High Bandwidth.