Dynamic Neural Graph Encoding of Inference Processes in Deep Weight Space
For researchers working with neural network weight spaces and implicit neural representations, this work provides a novel encoding method that improves classification accuracy.
The paper introduces a dynamic graph representation for neural network weight spaces to capture the sequential nature of inference, and proposes the DNG-Encoder to process these graphs. The method achieves a 10% improvement in INR classification accuracy on CIFAR-100-INR over state-of-the-art.
The rapid advancements in using neural networks as implicit data representations have attracted significant interest in developing machine learning methods that analyze and process the weight spaces of other neural networks. However, efficiently handling these highdimensional weight spaces remains challenging. Existing methods often overlook the sequential nature of layer-by-layer processing in neural network inference. In this work, we propose a novel approach using dynamic graphs to represent neural network parameters, capturing the temporal dynamics of inference. Our Dynamic Neural Graph Encoder (DNG-Encoder) processes these graphs, preserving the sequential nature of neural processing. Additionally, we also leverage DNG-Encoder to develop INR2JLS (Implicit Neural Representation to Joint Latent Space) for facilitate downstream applications, such as classifying Implicit Neural Representations (INRs). Our approach demonstrates significant improvements across multiple tasks, surpassing the state-of-the-art INR classification accuracy by approximately 10% on the CIFAR-100-INR.