Tensorization of neural networks for improved privacy and interpretability
This work addresses privacy and interpretability issues in machine learning models, particularly neural networks, by providing a novel tensorization approach, though it appears incremental as it builds on existing sketching and cross interpolation ideas.
The paper tackles the problem of enhancing privacy and interpretability in neural networks by developing a tensorization algorithm that constructs tensor train representations from black-box function access and sample points. It shows that this method can obfuscate neural networks to protect training data patterns and estimate topological phases of matter, while also serving as an efficient initialization for tensor train optimization and achieving a superior memory-time trade-off for model compression compared to conventional methods.
We present a tensorization algorithm for constructing tensor train representations of functions, drawing on sketching and cross interpolation ideas. The method only requires black-box access to the target function and a small set of sample points defining the domain of interest. Thus, it is particularly well-suited for machine learning models, where the domain of interest is naturally defined by the training dataset. We show that this approach can be used to enhance the privacy and interpretability of neural network models. Specifically, we apply our decomposition to (i) obfuscate neural networks whose parameters encode patterns tied to the training data distribution, and (ii) estimate topological phases of matter that are easily accessible from the tensor train representation. Additionally, we show that this tensorization can serve as an efficient initialization method for optimizing tensor trains in general settings, and that, for model compression, our algorithm achieves a superior trade-off between memory and time complexity compared to conventional tensorization methods of neural networks.