Practical Equivariances via Relational Conditional Neural Processes
This work addresses a scalability bottleneck for equivariant CNPs, benefiting applications like spatio-temporal modeling and Bayesian optimization.
The authors tackled the problem of scaling equivariant Conditional Neural Processes (CNPs) beyond two input dimensions by proposing Relational Conditional Neural Processes (RCNPs), which effectively incorporate equivariances and demonstrate competitive performance on various tasks.
Conditional Neural Processes (CNPs) are a class of metalearning models popular for combining the runtime efficiency of amortized inference with reliable uncertainty quantification. Many relevant machine learning tasks, such as in spatio-temporal modeling, Bayesian Optimization and continuous control, inherently contain equivariances -- for example to translation -- which the model can exploit for maximal performance. However, prior attempts to include equivariances in CNPs do not scale effectively beyond two input dimensions. In this work, we propose Relational Conditional Neural Processes (RCNPs), an effective approach to incorporate equivariances into any neural process model. Our proposed method extends the applicability and impact of equivariant neural processes to higher dimensions. We empirically demonstrate the competitive performance of RCNPs on a large array of tasks naturally containing equivariances.