AnyUp: Universal Feature Upsampling
This addresses the limitation of existing learning-based upsamplers that require re-training for each feature extractor, improving generalization and efficiency for downstream vision tasks.
The paper tackles the problem of feature upsampling in vision tasks by introducing AnyUp, a method that can upsample any vision feature at any resolution without encoder-specific training, setting a new state of the art for upsampled features.
We introduce AnyUp, a method for feature upsampling that can be applied to any vision feature at any resolution, without encoder-specific training. Existing learning-based upsamplers for features like DINO or CLIP need to be re-trained for every feature extractor and thus do not generalize to different feature types at inference time. In this work, we propose an inference-time feature-agnostic upsampling architecture to alleviate this limitation and improve upsampling quality. In our experiments, AnyUp sets a new state of the art for upsampled features, generalizes to different feature types, and preserves feature semantics while being efficient and easy to apply to a wide range of downstream tasks.