Incremental Sequence Classification with Temporal Consistency
This work addresses the problem of making accurate and efficient predictions in sequence-based tasks for applications like text classification and AI verification, representing an incremental improvement with specific gains.
The paper tackled incremental sequence classification by developing a novel loss function based on temporal consistency, which improved predictive accuracy on text classification benchmarks and enhanced the ability to verify large language model generations for correctness in math problems after observing only a few tokens.
We address the problem of incremental sequence classification, where predictions are updated as new elements in the sequence are revealed. Drawing on temporal-difference learning from reinforcement learning, we identify a temporal-consistency condition that successive predictions should satisfy. We leverage this condition to develop a novel loss function for training incremental sequence classifiers. Through a concrete example, we demonstrate that optimizing this loss can offer substantial gains in data efficiency. We apply our method to text classification tasks and show that it improves predictive accuracy over competing approaches on several benchmark datasets. We further evaluate our approach on the task of verifying large language model generations for correctness in grade-school math problems. Our results show that models trained with our method are better able to distinguish promising generations from unpromising ones after observing only a few tokens.