Shape Generation via Weight Space Learning
This work addresses the problem of fine-tuning 3D shape generation models without catastrophic forgetting for applications in computer graphics or design, presenting an incremental approach to weight space exploration.
The paper tackles the challenge of leveraging geometric priors from foundation models for 3D shape generation in data-scarce or noisy scenarios by exploring the weight space as a data modality, showing that small weight changes can drastically alter topology and low-dimensional reparameterizations enable controlled local geometry changes with limited data.
Foundation models for 3D shape generation have recently shown a remarkable capacity to encode rich geometric priors across both global and local dimensions. However, leveraging these priors for downstream tasks can be challenging as real-world data are often scarce or noisy, and traditional fine-tuning can lead to catastrophic forgetting. In this work, we treat the weight space of a large 3D shape-generative model as a data modality that can be explored directly. We hypothesize that submanifolds within this high-dimensional weight space can modulate topological properties or fine-grained part features separately, demonstrating early-stage evidence via two experiments. First, we observe a sharp phase transition in global connectivity when interpolating in conditioning space, suggesting that small changes in weight space can drastically alter topology. Second, we show that low-dimensional reparameterizations yield controlled local geometry changes even with very limited data. These results highlight the potential of weight space learning to unlock new approaches for 3D shape generation and specialized fine-tuning.