Predict or classify: The deceptive role of time-locking in brain signal classification
This reveals a critical methodological flaw in neuroscience research that could lead to false claims about predicting decisions, making it significant for researchers in brain-computer interfaces and cognitive science, though it is incremental in refining existing analysis techniques.
The study tackled the problem of misinterpreting brain signal classification as prediction in decision-making experiments, showing that time-locking to an event can yield above-chance classification accuracy even when signals contain no predictive information, with accuracy reaching levels like 60-70% before the event.
Several experimental studies claim to be able to predict the outcome of simple decisions from brain signals measured before subjects are aware of their decision. Often, these studies use multivariate pattern recognition methods with the underlying assumption that the ability to classify the brain signal is equivalent to predict the decision itself. Here we show instead that it is possible to correctly classify a signal even if it does not contain any predictive information about the decision. We first define a simple stochastic model that mimics the random decision process between two equivalent alternatives, and generate a large number of independent trials that contain no choice-predictive information. The trials are first time-locked to the time point of the final event and then classified using standard machine-learning techniques. The resulting classification accuracy is above chance level long before the time point of time-locking. We then analyze the same trials using information theory. We demonstrate that the high classification accuracy is a consequence of time-locking and that its time behavior is simply related to the large relaxation time of the process. We conclude that when time-locking is a crucial step in the analysis of neural activity patterns, both the emergence and the timing of the classification accuracy are affected by structural properties of the network that generates the signal.