Grab: Fast and Accurate Sensor Processing for Cashier-Free Shopping
This addresses the practical challenge of implementing cashier-free shopping for retailers and customers, but it is incremental as it builds on existing infrastructure and methods.
The paper tackles the problem of enabling cashier-free shopping without store redesign by proposing Grab, a system that uses sensor fusion and tracking algorithms to accurately associate shoppers with items, achieving over 90% precision and recall in a pilot deployment.
Cashier-free shopping systems like Amazon Go improve shopping experience, but can require significant store redesign. In this paper, we propose Grab, a practical system that leverages existing infrastructure and devices to enable cashier-free shopping. Grab needs to accurately identify and track customers, and associate each shopper with items he or she retrieves from shelves. To do this, it uses a keypoint-based pose tracker as a building block for identification and tracking, develops robust feature-based face trackers, and algorithms for associating and tracking arm movements. It also uses a probabilistic framework to fuse readings from camera, weight and RFID sensors in order to accurately assess which shopper picks up which item. In experiments from a pilot deployment in a retail store, Grab can achieve over 90% precision and recall even when 40% of shopping actions are designed to confuse the system. Moreover, Grab has optimizations that help reduce investment in computing infrastructure four-fold.