Sum-Product-Transform Networks: Exploiting Symmetries using Invertible Transformations
This work addresses the need for more expressive probabilistic models in machine learning, offering an incremental extension to sum-product networks with practical applications in density estimation and anomaly detection.
The paper tackles the problem of improving sum-product networks by incorporating invertible transformations, resulting in Sum-Product-Transform Networks (SPTNs) that maintain tractable inference and achieve state-of-the-art results in density estimation and competitive performance in anomaly detection.
In this work, we propose Sum-Product-Transform Networks (SPTN), an extension of sum-product networks that uses invertible transformations as additional internal nodes. The type and placement of transformations determine properties of the resulting SPTN with many interesting special cases. Importantly, SPTN with Gaussian leaves and affine transformations pose the same inference task tractable that can be computed efficiently in SPNs. We propose to store affine transformations in their SVD decompositions using an efficient parametrization of unitary matrices by a set of Givens rotations. Last but not least, we demonstrate that G-SPTNs achieve state-of-the-art results on the density estimation task and are competitive with state-of-the-art methods for anomaly detection.