Reinforcement Learning with Generative Models for Compact Support Sets
This addresses the problem of data efficiency in machine learning for practitioners needing to improve model performance with limited labeled data, representing a novel method for a known bottleneck.
The paper tackles the problem of inefficient information extraction from foundation models for small sample sizes by proposing a reinforcement learning framework that generates compact synthetic support sets to augment neural network classification performance. The framework increased classification accuracy by significant margins without requiring additional labeling or data.
Foundation models contain a wealth of information from their vast number of training samples. However, most prior arts fail to extract this information in a precise and efficient way for small sample sizes. In this work, we propose a framework utilizing reinforcement learning as a control for foundation models, allowing for the granular generation of small, focused synthetic support sets to augment the performance of neural network models on real data classification tasks. We first allow a reinforcement learning agent access to a novel context based dictionary; the agent then uses this dictionary with a novel prompt structure to form and optimize prompts as inputs to generative models, receiving feedback based on a reward function combining the change in validation accuracy and entropy. A support set is formed this way over several exploration steps. Our framework produced excellent results, increasing classification accuracy by significant margins for no additional labelling or data cost.