A Metalearned Neural Circuit for Nonparametric Bayesian Inference
This addresses the practical barriers of complexity and inefficiency in nonparametric Bayesian inference for machine learning applications dealing with open-set classification.
The paper tackled the problem of applying nonparametric Bayesian models to real-world classification with long-tailed, open-set class distributions by metalearning a neural circuit that distills the model's inductive bias. The result is a method that achieves comparable or better performance than particle filter-based approaches while being faster and simpler to use.
Most applications of machine learning to classification assume a closed set of balanced classes. This is at odds with the real world, where class occurrence statistics often follow a long-tailed power-law distribution and it is unlikely that all classes are seen in a single sample. Nonparametric Bayesian models naturally capture this phenomenon, but have significant practical barriers to widespread adoption, namely implementation complexity and computational inefficiency. To address this, we present a method for extracting the inductive bias from a nonparametric Bayesian model and transferring it to an artificial neural network. By simulating data with a nonparametric Bayesian prior, we can metalearn a sequence model that performs inference over an unlimited set of classes. After training, this "neural circuit" has distilled the corresponding inductive bias and can successfully perform sequential inference over an open set of classes. Our experimental results show that the metalearned neural circuit achieves comparable or better performance than particle filter-based methods for inference in these models while being faster and simpler to use than methods that explicitly incorporate Bayesian nonparametric inference.