Memory Encoding Model
This work addresses brain encoding for cognitive tasks, offering a novel approach that improves prediction accuracy, though it is incremental in enhancing existing models with memory inputs.
The paper tackled the problem of predicting brain activity during vision-memory tasks by introducing a Memory Encoding Model that incorporates memory-related information as input, achieving a single model score of 66.8 and winning the Algonauts 2023 competition.
We explore a new class of brain encoding model by adding memory-related information as input. Memory is an essential brain mechanism that works alongside visual stimuli. During a vision-memory cognitive task, we found the non-visual brain is largely predictable using previously seen images. Our Memory Encoding Model (Mem) won the Algonauts 2023 visual brain competition even without model ensemble (single model score 66.8, ensemble score 70.8). Our ensemble model without memory input (61.4) can also stand a 3rd place. Furthermore, we observe periodic delayed brain response correlated to 6th-7th prior image, and hippocampus also showed correlated activity timed with this periodicity. We conjuncture that the periodic replay could be related to memory mechanism to enhance the working memory.