Contextualising Implicit Representations for Semantic Tasks
This work addresses a bottleneck for researchers and practitioners using implicit representations by enabling semantic task adaptation without access to original data or encoders, though it is incremental as it builds on prior reconstruction-focused methods.
The paper tackles the problem that implicit representations trained for reconstruction are not useful for semantic tasks, and proposes a contextualising module that enables their use in downstream tasks like semantic segmentation without compromising reconstruction performance, allowing pre-training on larger datasets to improve reconstruction while maintaining segmentation accuracy.
Prior works have demonstrated that implicit representations trained only for reconstruction tasks typically generate encodings that are not useful for semantic tasks. In this work, we propose a method that contextualises the encodings of implicit representations, enabling their use in downstream tasks (e.g. semantic segmentation), without requiring access to the original training data or encoding network. Using an implicit representation trained for a reconstruction task alone, our contextualising module takes an encoding trained for reconstruction only and reveals meaningful semantic information that is hidden in the encodings, without compromising the reconstruction performance. With our proposed module, it becomes possible to pre-train implicit representations on larger datasets, improving their reconstruction performance compared to training on only a smaller labelled dataset, whilst maintaining their segmentation performance on the labelled dataset. Importantly, our method allows for future foundation implicit representation models to be fine-tuned on unseen tasks, regardless of encoder or dataset availability.