Meta-Learning Theory-Informed Inductive Biases using Deep Kernel Gaussian Processes
This work provides a scalable approach for integrating theoretical knowledge into data-driven scientific inquiry, particularly in neuroscience, though it is incremental in automating existing manual processes.
The authors tackled the problem of integrating normative theories into data-driven models for biological systems by introducing a Bayesian meta-learning framework that converts theory predictions into probabilistic models using deep kernel Gaussian processes. They demonstrated improved prediction accuracy in mouse retinal ganglion cell recordings compared to baselines, with well-calibrated uncertainty and interpretable representations.
Normative and task-driven theories offer powerful top-down explanations for biological systems, yet the goals of quantitatively arbitrating between competing theories, and utilizing them as inductive biases to improve data-driven fits of real biological datasets are prohibitively laborious, and often impossible. To this end, we introduce a Bayesian meta-learning framework designed to automatically convert raw functional predictions from normative theories into tractable probabilistic models. We employ adaptive deep kernel Gaussian processes, meta-learning a kernel on synthetic data generated from a normative theory. This Theory-Informed Kernel specifies a probabilistic model representing the theory predictions -- usable for both fitting data and rigorously validating the theory. As a demonstration, we apply our framework to the early visual system, using efficient coding as our normative theory. We show improved response prediction accuracy in ex vivo recordings of mouse retinal ganglion cells stimulated by natural scenes compared to conventional data-driven baselines, while providing well-calibrated uncertainty estimates and interpretable representations. Using exact Bayesian model selection, we also show that our informed kernel can accurately infer the degree of theory-match from data, confirming faithful encapsulation of theory structure. This work provides a more general, scalable, and automated approach for integrating theoretical knowledge into data-driven scientific inquiry in neuroscience and beyond.