Efficient Universal Perception Encoder
This work addresses the need for efficient, multi-task AI models on resource-constrained edge devices, representing an incremental improvement over previous agglomerative methods.
The paper tackles the challenge of creating a small yet powerful vision encoder for edge devices by introducing the Efficient Universal Perception Encoder (EUPE), which achieves on-par or better performance than domain-expert encoders of the same size across diverse tasks through a novel distillation approach.
Running AI models on smart edge devices can unlock versatile user experiences, but presents challenges due to limited compute and the need to handle multiple tasks simultaneously. This requires a vision encoder with small size but powerful and versatile representations. We present our method, Efficient Universal Perception Encoder (EUPE), which offers both inference efficiency and universally good representations for diverse downstream tasks. We achieve this by distilling from multiple domain-expert foundation vision encoders. Unlike previous agglomerative methods that directly scale down from multiple teachers to an efficient encoder, we demonstrate the importance of first scaling up to a large proxy teacher and then scaling down from this single teacher. Experiments show that EUPE achieves on-par or better performance than individual domain experts of the same size on diverse task domains and also outperforms previous agglomerative encoders. We will release the full family of EUPE models and the code to foster future research.