Task adaption by biologically inspired stochastic comodulation
This work addresses the challenge of improving learning efficiency and performance in multi-task learning for AI systems, though it appears incremental as it builds on existing gain modulation methods.
The paper tackled the problem of balancing generalizability and adaptability in neural networks by introducing stochastic gain modulation for multi-task learning, achieving state-of-the-art results on the CelebA dataset.
Brain representations must strike a balance between generalizability and adaptability. Neural codes capture general statistical regularities in the world, while dynamically adjusting to reflect current goals. One aspect of this adaptation is stochastically co-modulating neurons' gains based on their task relevance. These fluctuations then propagate downstream to guide decision-making. Here, we test the computational viability of such a scheme in the context of multi-task learning. We show that fine-tuning convolutional networks by stochastic gain modulation improves on deterministic gain modulation, achieving state-of-the-art results on the CelebA dataset. To better understand the mechanisms supporting this improvement, we explore how fine-tuning performance is affected by architecture using Cifar-100. Overall, our results suggest that stochastic comodulation can enhance learning efficiency and performance in multi-task learning, without additional learnable parameters. This offers a promising new direction for developing more flexible and robust intelligent systems.