EgoSelf: From Memory to Personalized Egocentric Assistant
For developers of egocentric assistants, this work addresses the challenge of personalizing assistance using long-term user data, but the improvement over baselines is not quantified.
EgoSelf introduces a graph-based interaction memory and a personalized learning task to predict future user interactions from egocentric video, enabling personalized assistance. Experiments show effectiveness, though no concrete numbers are provided.
Egocentric assistants often rely on first-person view data to capture user behavior and context for personalized services. Since different users exhibit distinct habits, preferences, and routines, such personalization is essential for truly effective assistance. However, effectively integrating long-term user data for personalization remains a key challenge. To address this, we introduce EgoSelf, a system that includes a graph-based interaction memory constructed from past observations and a dedicated learning task for personalization. The memory captures temporal and semantic relationships among interaction events and entities, from which user-specific profiles are derived. The personalized learning task is formulated as a prediction problem where the model predicts possible future interactions from individual user's historical behavior recorded in the graph. Extensive experiments demonstrate the effectiveness of EgoSelf as a personalized egocentric assistant. Code is available at \href{https://abie-e.github.io/egoself_project/}{https://abie-e.github.io/egoself\_project/}.