Masked Conditioning for Deep Generative Models
This addresses the problem of limited computational resources and sparse data in engineering domains, though it appears incremental as an adaptation of existing conditioning techniques.
The paper tackles the challenge of applying generative models to small, sparsely labeled engineering datasets with mixed data types by introducing a masked-conditioning approach that simulates sparse conditions during training. Results show the method enables efficient variational autoencoders and latent diffusion models to work effectively on 2D point cloud and image datasets, with potential quality improvements through coupling with large pretrained models.
Datasets in engineering domains are often small, sparsely labeled, and contain numerical as well as categorical conditions. Additionally. computational resources are typically limited in practical applications which hinders the adoption of generative models for engineering tasks. We introduce a novel masked-conditioning approach, that enables generative models to work with sparse, mixed-type data. We mask conditions during training to simulate sparse conditions at inference time. For this purpose, we explore the use of various sparsity schedules that show different strengths and weaknesses. In addition, we introduce a flexible embedding that deals with categorical as well as numerical conditions. We integrate our method into an efficient variational autoencoder as well as a latent diffusion model and demonstrate the applicability of our approach on two engineering-related datasets of 2D point clouds and images. Finally, we show that small models trained on limited data can be coupled with large pretrained foundation models to improve generation quality while retaining the controllability induced by our conditioning scheme.