On the Computation and Applications of Large Dense Partial Correlation Networks
This work addresses the problem of enhancing network analysis and related applications for researchers in machine learning and statistics, though it appears incremental as it builds on existing L2-regularized solutions.
The paper tackles the overlooked dense solution in network connectivity modeling by deriving a direct computation method for partial correlations, connecting it to spectral graph theory, dimensionality reduction, and uncertainty quantification, and offering new insights into model selection and data preprocessing.
While sparse inverse covariance matrices are very popular for modeling network connectivity, the value of the dense solution is often overlooked. In fact the L2-regularized solution has deep connections to a number of important applications to spectral graph theory, dimensionality reduction, and uncertainty quantification. We derive an approach to directly compute the partial correlations based on concepts from inverse problem theory. This approach also leads to new insights on open problems such as model selection and data preprocessing, as well as new approaches which relate the above application areas.