Using Spinnaker on ARM and Embedded Systems

With the release of version 1.5 of the Spinnaker SDK, you can program and operate Teledyne FLIR USB3 machine vision cameras on an ARM device in a Linux environment. This application note explains the components and steps that are necessary to get started with Spinnaker and ARM as well as the limitations of using Spinnaker on an ARM device. It provides a summary and instructions for streaming Teledyne FLIR USB3 machine vision cameras using Spinnaker on ARM-based embedded boards. It includes examples of some of the more commonly used embedded boards. The benchmark results show that embedded boards are able to support high speed machine vision applications.

Note: Testing is ongoing. Wherever possible, limitations have been noted; however, as more testing is completed this information may change. All possible configurations of ARM and cameras may not experience the same results.

Preparing for use

Before you use your camera, we recommend that you are aware of the following resources:

• Getting Started Manual for the camera—provides information on installing components and software needed to run the camera.
• Technical Reference for the camera—provides information on the camera’s specifications, features and operations, as well as imaging and acquisition controls.
• Firmware updates—ensure you are using the most up-to-date firmware for the camera to take advantage of improvements and fixes.
• Tech Insights—Subscribe to our monthly email updates containing information on new knowledge base articles, new firmware and software releases, and Product Change Notices (PCN).

Supported System Configuration

Before installing Spinnaker, you must have the following prerequisites:

• Jetson Orin, Xavier, Nano, Raspberry Pi 4, TX2, or other OMAP4 device with an ARMv7 Cortex processor or newer (Cortex-A7, -A8, -A9, -A15… etc).
• A Teledyne FLIR USB3 camera
• Other FLIR machine vision cameras (FireWire, USB2, or CameraLink) are NOT supported, although we have tested GigE cameras to stream on select ARM boards.
• An external power supply such as a powered USB hub or GPIO power cable.
• Linux Ubuntu 18.04

Configuring the Operating System and Installing the Required Libraries

For Spinnaker to run on a Linux Ubuntu system, the following dependencies must be installed:

• Ubuntu 18.04

user$: sudo apt-get install libusb-1.0-0

Installing the Spinnaker SDK

To install the Spinnaker SDK:

1. Download Spinnaker SDK for ARM from our Spinnaker page.
2. Copy your spinnaker-<version>_arm.tar.gz package on a network or USB drive then copy it to a location on the ARM board.
3. Untar the installation package:
   tar xvfz spinnaker-<version>_arm.tar.gz
4. Run the install script:
   cd spinnaker-<version>_arm/
sudo sh install_spinnaker_arm.sh
5. Follow the instructions of the script. This installs all the Spinnaker libraries, example code, sample applications, and documentation. Additionally, the script prompts you to add your username to flirimaging group so that you can access your camera in SpinView without being a super user (sudo). If you don’t add users to the flirimaging group, cameras can only be accessed by the super user.
6. Restart your board for the user permissions to take effect.

Compiling the Examples

The Spinnaker SDK includes a number of example applications to help get you started in programming common API tasks. Example files are installed under /usr/src/spinnaker/src.

We suggest you copy the extracted folder and sub-folders to a location with write access.

To compile the examples, install the GNU C++ (g++) compiler that is included with the build-essential package:

user$ sudo apt-get install build-essential

To compile a specific example, run the makefile located in the example directory. Binaries are copied to the bin directory, and libraries are copied to the lib directory. For example:

user$ cd <copied folder>/spinnaker/src/Acquisition
user$ make

Limitations Using ARM

Linux users do not have access to Microsoft Windows-only technologies such as:

• DirectShow
• Cognex AIK
• Twain
• Managed .NET API
• ActiveX

Spinnaker on an ARM device does not support:

• OpenGL
• Teledyne FLIR FireWire machine vision cameras
• Teledyne FLIR GigE machine vision cameras
  Note: We have tested GigE cameras to stream on select ARM boards and noticed the bandwidth allowed for GigE cameras was lower than a desktop system would provide. See Streaming Cameras on Embedded Systems for detailed test result.
• Teledyne FLIR CameraLink machine vision cameras
• Recording videos

Note: Spinnaker on an ARM device has limited color processing options.

Additional ARM device limitations:

• The speed of the processor affects the maximum available frame rate. The maximum frame rates achieved with a Windows configuration may not be available.

Viewing Images and Videos

We suggest the following tools for image and video viewing. 

For image viewing:

• SpinView
• gimp
• ImageJ

For video recording:

Spinnaker SDK does not provide video recording API for ARM devices. However, you can save .raw images and then append these images into a video file later using FFmpeg. Please contact FLIR Machine Vision technical support to get more information about converting images into a video. This is recommended over using function such as cv2.VideoWriter because encoding images into a video file takes a long time on ARM devices and you may run out of RAM space.

For video viewing:

• VLC media player

For working with Glade files:

• Glade

Removing Spinnaker

Uninstall by manually removing the Spinnaker files, as in the following example:

user$: sudo sh
remove_spinnaker-arm.sh

Delete any extracted files or newly compiled files on your system.

Streaming on Embedded Systems

Note: The BFS-U3 was powered via the USB3 interface.

System Configuration forNVIDIA Jetson Xavier

Getting Started on the NVIDIA Jetson Xavier

1. 1. Follow NVIDIA’s Developer Kit User Guide file to install the JetPack software. Or you can watch this short video to get started with Jetson Xavier Development Kit: https://www.youtube.com/watch?v=-nX8eD7FusQ
   2. Install 64-bit Spinnaker ARM by downloading the latest Spinnaker 64-bit ARM and follow the README file for installation instructions. Spinnaker 2.0.0.109 SDK - ARM64 - Ubuntu 18.04 (64-bit) was used for this application note.

   By default, the USB file size limit on Linux is set to only allow streaming up to a total of 2MB over USB3. As of Spinnaker 2.0.0.109, the Ubuntu install script auto-configures the USBFS limit during install if the option is chosen. If you declined this option during install, you can manually increase the USBFS limit by adding the following:

   sudo sh -c 'echo 1000 > /sys/module/usbcore/parameters/usbfs_memory_mb'

   to this file:

   /etc/rc.local

This resets USBFS memory size upon restart. Recompile your kernel with this command included if you need the setting applied upon boot up.

Benchmarks for the NVIDIA Jetson Xavier

These are the results from a console application that continuously captures images using BFS-U3-16S2M-CS. The resulting benchmark using the console application is shown below.

System Configuration for Raspberry Pi 4

Getting Started on the Raspberry Pi 4

1. Download Ubuntu 18.04.4 LTS for Raspberry Pi from Ubuntu website and follow installation instructions here.
2. Install 64-bit Spinnaker ARM by downloading the latest Spinnaker 64-bit ARM and follow the readme file for installation instructions. Spinnaker 2.0.0.109 SDK - ARM64 - Ubuntu 18.04 (64-bit) was used for this application note.

By default, the USB file size limit on Linux is set to only allow streaming up to a total of 2MB over USB3. As of Spinnaker 2.0.0.109, the Ubuntu install script auto-configures the USBFS limit during install if the option is chosen. If you declined this option during install, you can manually increase the USBFS limit by adding the following:

sudo sh -c 'echo 1000 > /sys/module/usbcore/parameters/usbfs_memory_mb'

to this file:

/etc/rc.local

Benchmarks for the Raspberry Pi 4

These are the results from a console application that continuously captures images using BFS-U3-16S2M-CS. The resulting benchmark using the console application is shown below.

System Configuration for Jetson Nano

Getting Started on the Jetson Nano

1. Visit Getting Started With Jetson Nano Developer Kit to get started with Jetson Nano. You need a microSD card to use it as a boot device and for main storage.
2. Install 64-bit Spinnaker ARM by downloading the latest Spinnaker 64-bit ARM and follow the README file for installation instructions. Spinnaker 2.0.0.109 SDK - ARM64 - Ubuntu 18.04 (64-bit) was used for this application note.

By default, the USB file size limit on linux is set to only allow streaming up to a total of 2MB over USB3. As of Spinnaker 2.0.0.109, the Ubuntu install script auto-configures the USBFS limit during install if the option is chosen. If you declined this option during install, you can manually increase the USBFS limit by adding the following:

sudo sh -c 'echo 1000 > /sys/module/usbcore/parameters/usbfs_memory_mb'

to this file:

/etc/rc.local

Benchmarks for the Jetson Nano

These are the results from a console application that continuously captures images using BFS-U3-27S5M. The resulting benchmark using the console application is shown below.

System Configuration for NVIDIA Jetson TX2

Getting Started on the NVIDIA Jetson TX2

1. Follow instructions in this document: https://developer.nvidia.com/jetson-tx2-developer-kit-user-guide to download JetPack. We have tested JetPack 4.4 (Ubuntu 18.04).
2. Install 64-bit Spinnaker ARM by downloading the latest Spinnaker 64-bit ARM and follow the readme file for installation instructions. Spinnaker 2.0.0.109 SDK - ARM64 - Ubuntu 18.04 (64-bit) was used for this application note.

By default, the USB file size limit on Linux is set to only allow streaming up to a total of 2MB over USB3. As of Spinnaker 2.0.0.109, the Ubuntu install script auto-configures the USBFS limit during install if the option is chosen. If you declined this option during install, you can manually increase the USBFS limit by adding the following:

sudo sh -c 'echo 1000 > /sys/module/usbcore/parameters/usbfs_memory_mb'

to this file:

/etc/rc.local

Benchmarks for the NVIDIA Jetson TX2

These are the results from a console application that continuously captures images using BFS-U3-19S4C-C. The resulting benchmark using the console application is shown below.

Troubleshooting

USB camera does not show up in SpinView

Cause: your username is not added to flirimaging group.

Solution: To have full write/read access to your USB3 camera without being super user (sudo), you need to add your userid to flirimaging group. To do this, you can provide your username when prompted during Spinnaker installation. Restart your board for the user permissions to take effect. If you missed this step during Spinnaker installation, you can run this command in terminal:

usermod -a -G flirimaging exampleuserid

Image transfer fails to start when image size is bigger than 2 MB

Cause: The USBFS buffer size is too small (16 MB by default). We recommend increase the USBFS value to 1000. Use the following command to check usbfs_memory_mb size:

cat /sys/module/usbcore/parameters/usbfs_memory_mb

Solution: Increase the memory by adding the following line:

sudo sh -c 'echo 1000 > /sys/module/usbcore/parameters/usbfs_memory_mb'

to this file:

/etc/rc.local

Note: From Spinnaker version 2.0.0.109, you do not need to set usbfs_memory_mb=1000 manually. You will be prompted to set this automatically during installation

Camera is detected but cannot stream

Cause: The USB3 bus power provided by the ARM board is not sufficient to power the camera.

Solution: Power the USB3 camera externally using a GPIO power supply or externally powered hub.