Neural Thickets: Diverse Task Experts Are Dense Around Pretrained Weights
This work addresses the challenge of efficiently leveraging pretrained models for diverse tasks, offering a parallelizable alternative to iterative optimization methods, though it is incremental in its approach to post-training.
The paper tackles the problem of discovering task-specific experts within pretrained models by viewing pretraining outcomes as distributions over parameter vectors, and finds that in large models, diverse task-improving specialists densely populate the neighborhood around pretrained weights, enabling a simple random sampling and ensembling method to achieve competitive performance with standard post-training techniques.
Pretraining produces a learned parameter vector that is typically treated as a starting point for further iterative adaptation. In this work, we instead view the outcome of pretraining as a distribution over parameter vectors, whose support already contains task-specific experts. We show that in small models such expert solutions occupy a negligible fraction of the volume of this distribution, making their discovery reliant on structured optimization methods such as gradient descent. In contrast, in large, well-pretrained models the density of task-experts increases dramatically, so that diverse, task-improving specialists populate a substantial fraction of the neighborhood around the pretrained weights. Motivated by this perspective, we explore a simple, fully parallel post-training method that samples $N$ parameter perturbations at random, selects the top $K$, and ensembles predictions via majority vote. Despite its simplicity, this approach is competitive with standard post-training methods such as PPO, GRPO, and ES for contemporary large-scale models.