Improving Set Function Approximation with Quasi-Arithmetic Neural Networks
This work addresses a fundamental limitation in handling set-structured data for machine learning applications, offering a novel method with broad applicability.
The paper tackled the problem of limited expressivity and transferability in set function approximation by introducing quasi-arithmetic neural networks (QUANNs) with a novel Neuralized Kolmogorov Mean (NKM) as a learnable aggregation function, resulting in outperforming state-of-the-art baselines across diverse benchmarks and learning embeddings that transfer effectively to non-set tasks.
Sets represent a fundamental abstraction across many types of data. To handle the unordered nature of set-structured data, models such as DeepSets and PointNet rely on fixed, non-learnable pooling operations (e.g., sum or max) -- a design choice that can hinder the transferability of learned embeddings and limits model expressivity. More recently, learnable aggregation functions have been proposed as more expressive alternatives. In this work, we advance this line of research by introducing the Neuralized Kolmogorov Mean (NKM) -- a novel, trainable framework for learning a generalized measure of central tendency through an invertible neural function. We further propose quasi-arithmetic neural networks (QUANNs), which incorporate the NKM as a learnable aggregation function. We provide a theoretical analysis showing that, QUANNs are universal approximators for a broad class of common set-function decompositions and, thanks to their invertible neural components, learn more structured latent representations. Empirically, QUANNs outperform state-of-the-art baselines across diverse benchmarks, while learning embeddings that transfer effectively even to tasks that do not involve sets.