Bounded Context Management for Tabular Foundation Models on Stream Learning
For practitioners of tabular stream learning, this work provides a practical and effective context management strategy that outperforms traditional methods, though it is an incremental application of TFMs to a known problem.
The paper addresses tabular stream learning under distribution shift by using tabular foundation models (TFMs) with a novel context management policy (CURE) instead of updating model parameters. CURE achieves up to 27.0% relative improvement over classical stream learners across seven streams.
Tabular stream learning requires predictions on sequentially arriving examples under distribution shift. While standard methods adapt by updating model states, tabular foundation models (TFMs) make predictions conditioned on a labeled context in an in-context manner, making them a natural alternative for stream learning. This shifts the challenge from how to update the model to how to manage the context. We propose a future information view that yields three practical requirements for context management: preserve recent examples, retain uncertain examples, and remove redundant examples. We instantiate these requirements as CURE (Context management via Uncertainty-aware admission and Redundancy aware Eviction), a context-managing policy with entropy-gated admission and redundancy-aware eviction. Across seven streams, CURE shows up to 27.0% relative improvement over classical stream learners, remains robust across multiple TFM backbones, and ranks first among other policy variants. Code and datasets are available at https://github.com/morcellinus/CURE-ICML-FMSD.