Weight Space Representation Learning with Neural Fields
This work addresses representation learning in weight space for neural fields, offering incremental improvements in generation tasks.
The paper tackles the problem of using weights as effective representations for neural fields by constraining optimization with a pre-trained base model and low-rank adaptation (LoRA), resulting in high representation quality and enabling higher-quality generation with latent diffusion models compared to existing methods.
In this work, we investigate the potential of weights to serve as effective representations, focusing on neural fields. Our key insight is that constraining the optimization space through a pre-trained base model and low-rank adaptation (LoRA) can induce structure in weight space. Across reconstruction, generation, and analysis tasks on 2D and 3D data, we find that multiplicative LoRA weights achieve high representation quality while exhibiting distinctiveness and semantic structure. When used with latent diffusion models, multiplicative LoRA weights enable higher-quality generation than existing weight-space methods.