Using Spinnaker on ARM and Embedded Systems

With the release of version 1.5 of the Spinnaker SDK, you can program and operate Teledyne FLIR USB3 machine vision cameras on an ARM device in a Linux environment. This application note explains the components and steps that are necessary to get started with Spinnaker and ARM as well as the limitations of using Spinnaker on an ARM device. It provides a summary and instructions for streaming Teledyne FLIR USB3 machine vision cameras using Spinnaker on ARM-based embedded boards. It includes examples of some of the more commonly used embedded boards. The benchmark results show that embedded boards are able to support high speed machine vision applications.

Note: Testing is ongoing. Wherever possible, limitations have been noted; however, as more testing is completed this information may change. All possible configurations of ARM and cameras may not experience the same results.

Preparing for use

Before you use your camera, we recommend that you are aware of the following resources:

  • Getting Started Manual for the camera—provides information on installing components and software needed to run the camera.
  • Technical Reference for the camera—provides information on the camera’s specifications, features and operations, as well as imaging and acquisition controls.
  • Firmware updates—ensure you are using the most up-to-date firmware for the camera to take advantage of improvements and fixes.
  • Tech InsightsSubscribe to our monthly email updates containing information on new knowledge base articles, new firmware and software releases, and Product Change Notices (PCN).

Supported System Configuration

Before installing Spinnaker, you must have the following prerequisites:

  • Jetson Orin, Xavier, Nano, Raspberry Pi 4, TX2, or other OMAP4 device with an ARMv7 Cortex processor or newer (Cortex-A7, -A8, -A9, -A15… etc).
  • A Teledyne FLIR USB3 camera
  • Other FLIR machine vision cameras (FireWire, USB2, or CameraLink) are NOT supported, although we have tested GigE cameras to stream on select ARM boards.
  • An external power supply such as a powered USB hub or GPIO power cable.
  • Linux Ubuntu 18.04

Configuring the Operating System and Installing the Required Libraries

For Spinnaker to run on a Linux Ubuntu system, the following dependencies must be installed:

  • Ubuntu 18.04

user$: sudo apt-get install libusb-1.0-0

Installing the Spinnaker SDK

To install the Spinnaker SDK:

  1. Download Spinnaker SDK for ARM from our Spinnaker page.
  2. Copy your spinnaker-<version>_arm.tar.gz package on a network or USB drive then copy it to a location on the ARM board.
  3. Untar the installation package:
    tar xvfz spinnaker-<version>_arm.tar.gz
  4. Run the install script:
    cd spinnaker-<version>_arm/
    sudo sh install_spinnaker_arm.sh
  5. Follow the instructions of the script. This installs all the Spinnaker libraries, example code, sample applications, and documentation. Additionally, the script prompts you to add your username to flirimaging group so that you can access your camera in SpinView without being a super user (sudo). If you don’t add users to the flirimaging group, cameras can only be accessed by the super user.
  6. Restart your board for the user permissions to take effect.

Compiling the Examples

The Spinnaker SDK includes a number of example applications to help get you started in programming common API tasks. Example files are installed under /usr/src/spinnaker/src.

We suggest you copy the extracted folder and sub-folders to a location with write access.

To compile the examples, install the GNU C++ (g++) compiler that is included with the build-essential package:

user$ sudo apt-get install build-essential

To compile a specific example, run the makefile located in the example directory. Binaries are copied to the bin directory, and libraries are copied to the lib directory. For example:

user$ cd <copied folder>/spinnaker/src/Acquisition
user$ make

Limitations Using ARM

Linux users do not have access to Microsoft Windows-only technologies such as:

  • DirectShow
  • Cognex AIK
  • Twain
  • Managed .NET API
  • ActiveX

Spinnaker on an ARM device does not support:

  • OpenGL
  • Teledyne FLIR FireWire machine vision cameras
  • Teledyne FLIR GigE machine vision cameras
    Note: We have tested GigE cameras to stream on select ARM boards and noticed the bandwidth allowed for GigE cameras was lower than a desktop system would provide. See Streaming Cameras on Embedded Systems for detailed test result.
  • Teledyne FLIR CameraLink machine vision cameras
  • Recording videos

Note: Spinnaker on an ARM device has limited color processing options.

Additional ARM device limitations:

  • The speed of the processor affects the maximum available frame rate. The maximum frame rates achieved with a Windows configuration may not be available.

Viewing Images and Videos

We suggest the following tools for image and video viewing. 

For image viewing:

For video recording:

Spinnaker SDK does not provide video recording API for ARM devices. However, you can save .raw images and then append these images into a video file later using FFmpeg. Please contact FLIR Machine Vision technical support to get more information about converting images into a video. This is recommended over using function such as cv2.VideoWriter because encoding images into a video file takes a long time on ARM devices and you may run out of RAM space.

For video viewing:

  • VLC media player

For working with Glade files:

Removing Spinnaker

Uninstall by manually removing the Spinnaker files, as in the following example:

user$: sudo sh
remove_spinnaker-arm.sh

Delete any extracted files or newly compiled files on your system.

Streaming on Embedded Systems

Note: The BFS-U3 was powered via the USB3 interface.

System Configuration forNVIDIA Jetson Xavier

  NVIDIA Jetson Xavier Specification
CPU 8-core ARM v8.2 64-bit
GPU 512-core Volta GPU with Tensor Cores
RAM 32 GB
Host Adaptor Driver xhci-hcd
Operating System Ubuntu 18.04
Software Spinnaker 2.0.0.109 Linux ARM64
Camera and Firmware BFS-U3-16S2M (firmware 1912.0.226.0)

Getting Started on the NVIDIA Jetson Xavier

    1. Follow NVIDIA’s Developer Kit User Guide file to install the JetPack software. Or you can watch this short video to get started with Jetson Xavier Development Kit: https://www.youtube.com/watch?v=-nX8eD7FusQ
    2. Install 64-bit Spinnaker ARM by downloading the latest Spinnaker 64-bit ARM and follow the README file for installation instructions. Spinnaker 2.0.0.109 SDK - ARM64 - Ubuntu 18.04 (64-bit) was used for this application note.

    By default, the USB file size limit on Linux is set to only allow streaming up to a total of 2MB over USB3. As of Spinnaker 2.0.0.109, the Ubuntu install script auto-configures the USBFS limit during install if the option is chosen. If you declined this option during install, you can manually increase the USBFS limit by adding the following:

    sudo sh -c 'echo 1000 > /sys/module/usbcore/parameters/usbfs_memory_mb'

    to this file:

    /etc/rc.local

This resets USBFS memory size upon restart. Recompile your kernel with this command included if you need the setting applied upon boot up.

Benchmarks for the NVIDIA Jetson Xavier

These are the results from a console application that continuously captures images using BFS-U3-16S2M-CS. The resulting benchmark using the console application is shown below.

BFS-U3-16S2M-CS
  Requested
Frame
Rate
Processed
Frame
Rate
CPU
Usage
Bandwidth
1440 x 1080 Mono8 226 FPS 226 FPS ~28% 361 MB/s

System Configuration for Raspberry Pi 4

  Raspberry Pi 4 Specification
CPU Quad core Cortex-A72 64 bit
GPU VideoCore VI
RAM 1/2/4 GB RAM
Host Adaptor Driver xhci-hcd
Operating System Ubuntu 18.04
Software Spinnaker 2.0.0.109 Linux ARM64
Camera and Firmware BFS-U3-16S2M (firmware 1912.0.226.0)

Getting Started on the Raspberry Pi 4

  1. Download Ubuntu 18.04.4 LTS for Raspberry Pi from Ubuntu website and follow installation instructions here.
  2. Install 64-bit Spinnaker ARM by downloading the latest Spinnaker 64-bit ARM and follow the readme file for installation instructions. Spinnaker 2.0.0.109 SDK - ARM64 - Ubuntu 18.04 (64-bit) was used for this application note.

By default, the USB file size limit on Linux is set to only allow streaming up to a total of 2MB over USB3. As of Spinnaker 2.0.0.109, the Ubuntu install script auto-configures the USBFS limit during install if the option is chosen. If you declined this option during install, you can manually increase the USBFS limit by adding the following:

sudo sh -c 'echo 1000 > /sys/module/usbcore/parameters/usbfs_memory_mb'

to this file:

/etc/rc.local

Benchmarks for the Raspberry Pi 4

These are the results from a console application that continuously captures images using BFS-U3-16S2M-CS. The resulting benchmark using the console application is shown below.

BFS-U3-16S2M-CS
  Requested
Frame
Rate
Processed
Frame
Rate
CPU
Usage
Bandwidth
1440 x 1080 Mono8 226 FPS 226 FPS ~40% 361 MB/s

System Configuration for Jetson Nano

  Jetson Nano Specification
CPU Quad core ARM A57
GPU 128-core Maxwell
RAM 4 GB RAM
Host Adaptor Driver xhci-hcd
Operating System Ubuntu 18.04
Software Spinnaker 2.0.0.109 Linux ARM64
Camera and Firmware BFS-U3-16S2M (firmware 1912.0.226.0)

Getting Started on the Jetson Nano

  1. Visit Getting Started With Jetson Nano Developer Kit to get started with Jetson Nano. You need a microSD card to use it as a boot device and for main storage.
  2. Install 64-bit Spinnaker ARM by downloading the latest Spinnaker 64-bit ARM and follow the README file for installation instructions. Spinnaker 2.0.0.109 SDK - ARM64 - Ubuntu 18.04 (64-bit) was used for this application note.

By default, the USB file size limit on linux is set to only allow streaming up to a total of 2MB over USB3. As of Spinnaker 2.0.0.109, the Ubuntu install script auto-configures the USBFS limit during install if the option is chosen. If you declined this option during install, you can manually increase the USBFS limit by adding the following:

sudo sh -c 'echo 1000 > /sys/module/usbcore/parameters/usbfs_memory_mb'

to this file:

/etc/rc.local

Benchmarks for the Jetson Nano

These are the results from a console application that continuously captures images using BFS-U3-27S5M. The resulting benchmark using the console application is shown below.

BFS-U3-27S5M
  Requested
Frame
Rate
Processed
Frame
Rate
CPU
Usage
Bandwidth
1936 x 1464 Mono16 67 FPS 67 FPS ~52% 380 MB/s

System Configuration for NVIDIA Jetson TX2

  NVIDIA Jetson TX2 Specification
CPU Quad core ARM A57
GPU Pascal GPU with 256 CUDA cores
RAM 8 GB
Host Adaptor Driver xhci-hcd
Operating System Ubuntu 18.04
Software Spinnaker 2.0.0.109 Linux ARM64
Camera and Firmware BFS-U3-19S4C-C (firmware 1910.0.120.0)

Getting Started on the NVIDIA Jetson TX2

  1. Follow instructions in this document: https://developer.nvidia.com/jetson-tx2-developer-kit-user-guide to download JetPack. We have tested JetPack 4.4 (Ubuntu 18.04).
  2. Install 64-bit Spinnaker ARM by downloading the latest Spinnaker 64-bit ARM and follow the readme file for installation instructions. Spinnaker 2.0.0.109 SDK - ARM64 - Ubuntu 18.04 (64-bit) was used for this application note.

By default, the USB file size limit on Linux is set to only allow streaming up to a total of 2MB over USB3. As of Spinnaker 2.0.0.109, the Ubuntu install script auto-configures the USBFS limit during install if the option is chosen. If you declined this option during install, you can manually increase the USBFS limit by adding the following:

sudo sh -c 'echo 1000 > /sys/module/usbcore/parameters/usbfs_memory_mb'

to this file:

/etc/rc.local

Benchmarks for the NVIDIA Jetson TX2

These are the results from a console application that continuously captures images using BFS-U3-19S4C-C. The resulting benchmark using the console application is shown below.

BFS-U3-19S4C
  Requested
Frame
Rate
Processed
Frame
Rate
CPU
Usage
Bandwidth
1616x1240 BayerRG8 131 FPS 131 FPS ~50% 250 MB/s

Troubleshooting

USB camera does not show up in SpinView

Cause: your username is not added to flirimaging group.

Solution: To have full write/read access to your USB3 camera without being super user (sudo), you need to add your userid to flirimaging group. To do this, you can provide your username when prompted during Spinnaker installation. Restart your board for the user permissions to take effect. If you missed this step during Spinnaker installation, you can run this command in terminal:

usermod -a -G flirimaging exampleuserid

Image transfer fails to start when image size is bigger than 2 MB

Cause: The USBFS buffer size is too small (16 MB by default). We recommend increase the USBFS value to 1000. Use the following command to check usbfs_memory_mb size:

cat /sys/module/usbcore/parameters/usbfs_memory_mb

Solution: Increase the memory by adding the following line:

sudo sh -c 'echo 1000 > /sys/module/usbcore/parameters/usbfs_memory_mb'

to this file:

/etc/rc.local

Note: From Spinnaker version 2.0.0.109, you do not need to set usbfs_memory_mb=1000 manually. You will be prompted to set this automatically during installation

Camera is detected but cannot stream

Cause: The USB3 bus power provided by the ARM board is not sufficient to power the camera.

Solution: Power the USB3 camera externally using a GPIO power supply or externally powered hub.