Precision Neural Networks: Joint Graph And Relational Learning
This work addresses performance issues in covariance-based learning for graph and relational data, offering a task-aware method that is incremental over VNNs.
The paper tackles the limitations of CoVariance Neural Networks (VNNs) by introducing Precision Neural Networks (PNNs), which use the precision matrix to encode statistical independence and sparsity, and jointly learn network parameters and the precision matrix via alternating optimization, showing effectiveness on synthetic and real-world data.
CoVariance Neural Networks (VNNs) perform convolutions on the graph determined by the covariance matrix of the data, which enables expressive and stable covariance-based learning. However, covariance matrices are typically dense, fail to encode conditional independence, and are often precomputed in a task-agnostic way, which may hinder performance. To overcome these limitations, we study Precision Neural Networks (PNNs), i.e., VNNs on the precision matrix -- the inverse covariance. The precision matrix naturally encodes statistical independence, often exhibits sparsity, and preserves the covariance spectral structure. To make precision estimation task-aware, we formulate an optimization problem that jointly learns the network parameters and the precision matrix, and solve it via alternating optimization, by sequentially updating the network weights and the precision estimate. We theoretically bound the distance between the estimated and true precision matrices at each iteration, and demonstrate the effectiveness of joint estimation compared to two-step approaches on synthetic and real-world data.