Neural Field Conditioning Strategies for 2D Semantic Segmentation
This work addresses a gap in conditioning strategies for neural fields, which is important for researchers in computer vision and neural representation fields, though it is incremental as it builds on existing methods.
The paper tackled the problem of conditioning neural fields for 2D semantic segmentation by comparing three conditioning methods, finding that Cross-Attention achieved the best results and was competitive with a CNN-based decoder.
Neural fields are neural networks which map coordinates to a desired signal. When a neural field should jointly model multiple signals, and not memorize only one, it needs to be conditioned on a latent code which describes the signal at hand. Despite being an important aspect, there has been little research on conditioning strategies for neural fields. In this work, we explore the use of neural fields as decoders for 2D semantic segmentation. For this task, we compare three conditioning methods, simple concatenation of the latent code, Feature Wise Linear Modulation (FiLM), and Cross-Attention, in conjunction with latent codes which either describe the full image or only a local region of the image. Our results show a considerable difference in performance between the examined conditioning strategies. Furthermore, we show that conditioning via Cross-Attention achieves the best results and is competitive with a CNN-based decoder for semantic segmentation.