AWM Frictionless™ - Autonomous Store Checkout Made Possible with Machine Vision Cameras

Imagine entering a store, taking an item off the shelf, and then simply walking out of the store and receiving a receipt for the item via email. It sounds like the future, but it’s what shopping currently looks like with AWM Frictionless™ shopping experience—a fully autonomous and contactless retail shopping system developed by AWM Smart Shelf. Using Blackfly S GigE Machine Vision Cameras, AWM Frictionless™ allows customers to pick up items and be automatically charged as soon as they leave the store, without needing to queue up, scan, or physically pay for their items.

The first AWM Frictionless™ store in California was launched in March 2020. A micro-market convenience store in Santa Ana, California, the store is filled with useful convenience items like snacks, sodas, and water, and refills on the basic necessities like soap, hygiene products, and milk. “Toilet paper actually was the first purchase when we opened,” reveals Kaitlyn Kempiak, Director of Marketing at AWM.

How does an autonomous contactless shopping experience work?

First the customer downloads the store app and fills out their payment information. The app provides them with a QR code that they scan to be let through the electronic doors. Once inside, the customer shops as normal. The products they pick up are tracked and added to the shopper’s digital basket, and then they just walk out of the store.

“Customers appreciate the convenience of having the store located inside their apartment building,” Kempiak says, “especially during COVID-19 pandemic, there is no scanning, no speaking with a store associate, complete friction free shopping.” Eventually the store will be open 24/7, allowing residents and the public to enjoy contactless shopping at any time.

Items in an AWM Frictionless™ store are tracked using Blackfly S GigE cameras and a deep learning algorithm trained to correctly identify products and add them to the shopper’s basket. The system also knows how to remove an item from the virtual shopping basket if the customer decides to put it back on the shelf. It even keeps track of a shopping bag if the customer puts it down and wanders into the next aisle.

Why did AWM Choose Blackfly S Cameras for Frictionless checkouts?

When it came to choosing a camera for Frictionless, AWM chose machine vision cameras; citing quality, reliability, helpful documentation, and great customer support. Additionally, key features provided by Blackfly S cameras for AWM’s application included the ability to adjust white balance to represent colors accurately, color correction matrix to reproduce the same colors in different lighting conditions, and the ability to process images from many cameras quickly with features like Chunk Data Timestamp and Packet Delay.

“To perform accurate tracking and 3D reconstruction across many cameras (32+ in some cases) it’s important we have accurate time information, down to the millisecond, for when frames were captured, and not necessarily the time when frames arrive at the computer for processing. This is especially true for GigE cameras operating over a network. The Chunk Data Timestamp feature on the Blackfly S cameras allows us to do this,” says AWM.

Multiple cameras are installed on the ceiling and can see the entire store. In case anyone thinks they can take advantage of the system by sneaking a small item—like a pack of gum—into their hand or sleeve without being spotted by a camera, the visual data is also supported by weighted shelving. The system knows exactly how much each item weights, and this second data input allows it to be more reliable and accurate.

The Color Correction Matrix is another important feature of Blackfly S cameras that helps to reliably reproduce colors in various lighting conditions. Getting consistent color is very important in Deep Learning applications to increase accuracy of the neural network and to reduce the amount of training data needed for the neural network.

Touch-free, automated retail experiences—in the real world

The Frictionless experience is already in micro-markets right now, and AWM has laid the groundwork for it to expand into other spaces, including conventional retail stores, convenience stores, and even supermarkets. AWM Frictionless™ can be adapted to existing stores, offering a fully autonomous and contactless shopping experience or simply another easy option for customers to checkout. The AI continues to be trained on new products—and AWM is already working on a produce recognition tool for deployment in grocery stores.

What do customers think?

Customers report that the frictionless experience is remarkably easy, saves time, and reduces health risks. Kempiak also notes how many apartment residents she’s seen making the micro-market part of their daily routine. The AWM Frictionless™ experience is revolutionary considering the value of contactless checkouts, extended store hours, easier stock management, higher security, and reduced checkout times.

More stores and shopping formats are being added as the machine learning solution continues to develop and makes shopping easier, and even more convenient.

Learn more about AWM Frictionless™ at www.smartshelf.com and Machine Vision and Deep Learning solutions at https://www.flir.com/machine-vision