PAL: Intelligence Augmentation using Egocentric Visual Context Detection
This addresses the problem of personalized and privacy-preserving intelligence augmentation for users through wearable technology, though it appears incremental in combining existing components.
The researchers developed PAL, a wearable system for egocentric visual context detection that uses on-device deep learning models to recognize objects, faces, and activities with over 80% accuracy on in-the-wild data, and tested it for applications like behavior change.
Egocentric visual context detection can support intelligence augmentation applications. We created a wearable system, called PAL, for wearable, personalized, and privacy-preserving egocentric visual context detection. PAL has a wearable device with a camera, heart-rate sensor, on-device deep learning, and audio input/output. PAL also has a mobile/web application for personalized context labeling. We used on-device deep learning models for generic object and face detection, low-shot custom face and context recognition (e.g., activities like brushing teeth), and custom context clustering (e.g., indoor locations). The models had over 80\% accuracy in in-the-wild contexts (~1000 images) and we tested PAL for intelligence augmentation applications like behavior change. We have made PAL is open-source to further support intelligence augmentation using personalized and privacy-preserving egocentric visual contexts.