How Many Bytes Can You Take Out Of Brain-To-Text Decoding?
This work addresses the challenge of enhancing brain-computer interfaces for medical and scientific applications, such as aiding speech, but it is incremental as it builds on existing decoders.
The authors tackled the problem of evaluating and improving brain-to-text decoders by proposing an information-based metric and augmenting existing methods, resulting in a performance improvement of over 40% compared to a baseline model. They also analyzed decoder dynamics and estimated idealized performance to identify error sources.
Brain-computer interfaces have promising medical and scientific applications for aiding speech and studying the brain. In this work, we propose an information-based evaluation metric for brain-to-text decoders. Using this metric, we examine two methods to augment existing state-of-the-art continuous text decoders. We show that these methods, in concert, can improve brain decoding performance by upwards of 40% when compared to a baseline model. We further examine the informatic properties of brain-to-text decoders and show empirically that they have Zipfian power law dynamics. Finally, we provide an estimate for the idealized performance of an fMRI-based text decoder. We compare this idealized model to our current model, and use our information-based metric to quantify the main sources of decoding error. We conclude that a practical brain-to-text decoder is likely possible given further algorithmic improvements.