Self-supervised Pretraining for Decision Foundation Model: Formulation, Pipeline and Challenges
This work addresses the problem of enhancing decision-making processes for AI systems, but it is incremental as it builds on existing self-supervised pretraining methods from language and vision.
The paper tackles the problem of improving decision-making by integrating large-scale self-supervised pretraining into downstream tasks, proposing a Pretrain-Then-Adapt pipeline and surveying related work to address challenges in sample efficiency and generalization.
Decision-making is a dynamic process requiring perception, memory, and reasoning to make choices and find optimal policies. Traditional approaches to decision-making suffer from sample efficiency and generalization, while large-scale self-supervised pretraining has enabled fast adaptation with fine-tuning or few-shot learning in language and vision. We thus argue to integrate knowledge acquired from generic large-scale self-supervised pretraining into downstream decision-making problems. We propose Pretrain-Then-Adapt pipeline and survey recent work on data collection, pretraining objectives and adaptation strategies for decision-making pretraining and downstream inference. Finally, we identify critical challenges and future directions for developing decision foundation model with the help of generic and flexible self-supervised pretraining.