Instrumental’s Novel AI Inspection Platform Leverages Teledyne FLIR Machine Vision Cameras

Manufacturing accounts for more than half of the world’s total gross world product (GWP) at $40 trillion, but 20% of every dollar spent goes to waste, according to manufacturing optimization company Instrumental. This represents a problem worth $8 trillion, or 10% of the GWP. Founded in 2015 by former Apple engineers, Instrumental has developed an optimization and inspection platform combining cloud software, machine vision inspection, artificial intelligence (AI), and electronic test data that aims to reduce waste by enabling engineers to not only stop problems at the end of the inspection line but to fix them upstream as well. 

Get More from Your Data, Manage Factories Remotely    

A 2021 State of NPI survey of 100 electronics brands by Instrumental found that 76% of engineering time is spent on tasks that are easily automatable or could be expedited with better data. The waste in engineering time resources, among other factors, makes it difficult for brands to keep pace with the innovation speed that the market demands. Instrumental acts as an engineering multiplier and answer to remote new product introductions by offering automated defect detection, root cause analysis, reporting, secure product data management, and native anytime, anywhere collaboration tools.

Instrumental’s manufacturing optimization platform aggregates images and functional test data from throughout the supply chain, leveraging artificial intelligence to automatically rank potential root causes. This enables Instrumental customers to accelerate issue resolution, gain faster yield, and improve quality.

During mass production, operators on assembly lines place products into Instrumental imaging stations, press a button, and wait four-to-five seconds for either a green or red light. If the products pass inspection, the light shows green, and the operator sends the product down the line for further assembly. Operators check the product in the imaging system again between key assembly steps, providing customers valuable image data as well as pass-fail results on the product in its various stages.

The system also offers an application programming interface (API) for electronic testing, measurements, and other relevant data collected on the manufacturing line. This information is immediately available on Instrumentals’ cloud software, for various users such as product design engineers, quality engineers, and other operations employees access to testing results along with image data.

“Instrumental focuses on allowing users to get more out of their data, which allows them to be agile and manage factories remotely,” said Tobias Harrison-Noonan, Director of Business Development and Solutions at Instrumental. “This capability has never been more important than now, given the big shift that occurred during the COVID-19 pandemic, which prevented engineers from getting into their factories.”

How it Works

Instrumental offers a drop-in imaging station — its standard, typical implementation — as well customized options for automated production line deployment. Inside the station, a color machine vision camera from Teledyne FLIR, typically one with a 20 MP rolling shutter CMOS image sensor. The system uses lenses from Edmund Optics and features industrial bar lights for illumination. Customers have the choice of using the Instrumental station as an out-of-the-box solution, with easy installation coordinated by expert teams in the US and China. Additionally, customers can use their own hardware if they choose.

In the competitive landscape of machine vision cameras, Instrumental chose Teledyne FLIR cameras because of its Spinnaker software development kit (SDK). Harrison-Noonan notes, “Based on our research, the Spinnaker SDK looked extremely approachable and would be fairly easy to integrate, and we were right.”  Additionally, there is a large family of cameras in the Teledyne FLIR Blackfly S series which provides flexibility for our customers to change the camera with minimal additional software development.”

“For example,” he continued, “while we deploy the 20 MP Blackfly S USB3 camera as a default, the team recently needed a Blackfly S camera with a global shutter to reduce motion blur on a moving conveyor belt application. We had to do very little software modifications because the camera is in the same family.”

In a similar situation, the Instrumental team deployed a monochrome camera with infrared lights for a specific application that required seeing through transparent material. Again, Instrumental was able to easily use a different Blackfly S model.

Intuitive Setup, Fast ROI Within 24 Hours

While the Instrumental platform typically sees use in electronics applications, the station can accommodate many different types of products. Defect training is done in the cloud, and Instrumental mitigates security concerns by walking new users through its series of extensive measures and protocols on its enterprise-grade hosting security for its managed cloud platform. Instrumental’s machine learning training process requires fewer images than a targeted deep learning inspection typically does. Here, the software analyzes terabytes of data and makes correlations.

“The platform is intentionally designed to be generalized — we are not offering AI as a service where the team spends two months designing a very specific algorithm,” said Harrison-Noonan. “Typically, we only require 30 images of a product, and the AI starts automatically identifying new issues with no user input, which really simplifies the entire process for a non-vision specialist.”

Inference takes place on the edge — typically on a Dell computer with a powerful GPU — so that the system can make faster decisions and still operate in the event of an internet outage. Once the system has 30 images in place, it can begin to analyze data and highlight abnormalities for the operator, who can then confirm or remove a highlighted issue.

Additionally, multiple users from anywhere in the world can access a single project and contribute data to the training phase and then use that repository as a central data collaboration platform. 

“Design engineers in the US using the Instrumental platform for failure analysis, for example, often share results and actions directly with quality and factory teams in Asia for accelerated global communication,” said Harrison-Noonan.

Defect resolution with micrometer-precision

Image: Blackfly S USB3 camera model

Using the Blackfly S 20 MP camera allows the Instrumental imaging station to capture high levels of detail required for inspecting small electronic micro assemblies where very small errors can lead to failure. For tasks such as cell phone inspection, for example, Instrumental will size the optics, set the working distance accordingly, and calculate resolution from there.

“Instrumental uses a calculator that lets us make estimates on how many pixels will be required to pick up a defect,” said Harrison-Noonan. “Normally we may say 5 or 6 pixels to pick up a defect, which means about 150 to 200 µm defect sizes for common mobile electronic-sized devices, which is generally what we aim to achieve in such applications.”

Teledyne FLIR offers six different Blackfly S camera models with 20 MP rolling shutter global reset image sensors. These models come in USB3, GigE, and board-level options and boast quantum efficiency ranging from 65.39% to 80% and frame rates up to 18 FPSs, with image sensors ranging from 0.4 MP to 24.5 MP.

What’s Next

While Instrumental’s system may typically use a color, visible spectrum camera, the company has recently been expanding into new data sources such as infrared, and even X-ray and 3D imaging technologies.

“As companies in different fields become either more aware or more open to adopting cloud technologies, the system is able to help them meet and exceed their metrics, even in a pandemic,” said Harrison-Noonan. “Teledyne FLIR offers a large portfolio of cameras used in disparate applications, giving our system the flexibility to handle new tasks.”

He added, “Leading electronics manufacturers around the world use Teledyne FLIR cameras to maintain quality and meet production schedules, even throughout the pandemic. By leveraging the Instrumental system and Teledyne FLIR is key to our ability to adapt to customer needs with new data sources.”

Learn more about Instrumental at https://instrumental.com/about and Teledyne FLIR machine vision cameras at www.teledyneflir.com/machine-vision.