MemoryDiorama: Generating Dynamic 3D Diorama from Everyday Photos for Memory Recall
This addresses memory recall enhancement for individuals using personal media, but it is incremental as it builds on existing concepts of augmented cues and 3D generation.
The paper tackles the problem of enhancing autobiographical memory recall by developing MemoryDiorama, a system that transforms everyday photos into dynamic 3D dioramas in mixed reality, resulting in more internal and in-cue details during recall and increased perceptual details and visual vividness ratings in a user study with 18 participants.
We present MemoryDiorama, a prototype system that introduces augmented memory cues, a concept that extends captured personal media with AI-generated contextual information to enhance autobiographical memory recall. MemoryDiorama transforms everyday photos into dynamic 3D dioramas in mixed reality by integrating LLM-based scene analysis with 3D object generation, animation, and spatial composition. The system extracts geographic information, object attributes, lighting conditions, and atmospheric elements from the photos. It then animates these elements with generative components such as object animations, human motion, geographical effects, and particle effects to provide richer cues for memory recall. We evaluated MemoryDiorama in a within-subject user study with 18 participants, comparing three conditions: Photo-Only, Static Diorama, and MemoryDiorama. Compared with both Photo-Only and Static Diorama, MemoryDiorama elicited more internal and in-cue details during recall. It also increased perceptual details and visual vividness ratings, suggesting richer recollective experience.