Beating Transformers using Synthetic Cognition
This work addresses the challenge of improving reasoning in AI for sequence classification tasks, though it appears incremental as it extends an existing Synthetic Cognition approach to sequences.
The authors tackled the problem of developing context-aware reactive behaviors for sequence classification by proposing a mechanism for Synthetic Cognition to handle sequences, and they achieved superior performance over DNA foundation models, obtaining the best scores on more benchmark tasks.
The road to Artificial General Intelligence goes through the generation of context-aware reactive behaviors, where the Transformer architecture has been proven to be the state-of-the-art. However, they still fail to develop reasoning. Recently, a novel approach for developing cognitive architectures, called Synthetic Cognition, has been proposed and implemented to develop instantaneous reactive behavior. In this study, we aim to explore the use of Synthetic Cognition to develop context-aware reactive behaviors. We propose a mechanism to deal with sequences for the recent implementation of Synthetic Cognition, and test it against DNA foundation models in DNA sequence classification tasks. In our experiments, our proposal clearly outperforms the DNA foundation models, obtaining the best score on more benchmark tasks than the alternatives. Thus, we achieve two goals: expanding Synthetic Cognition to deal with sequences, and beating the Transformer architecture for sequence classification.