Prompt-ICM: A Unified Framework towards Image Coding for Machines with Task-driven Prompts
This work addresses image coding for machines, enabling more efficient compression for AI-driven vision tasks, but it is incremental as it builds on existing prompting techniques.
The paper tackles the problem of compressing images for machine analysis rather than human perception, proposing Prompt-ICM, a unified framework that uses task-driven prompts to adjust compression and adapt features for various vision tasks, achieving higher coding efficiency with a single codec and minimal extra parameters.
Image coding for machines (ICM) aims to compress images to support downstream AI analysis instead of human perception. For ICM, developing a unified codec to reduce information redundancy while empowering the compressed features to support various vision tasks is very important, which inevitably faces two core challenges: 1) How should the compression strategy be adjusted based on the downstream tasks? 2) How to well adapt the compressed features to different downstream tasks? Inspired by recent advances in transferring large-scale pre-trained models to downstream tasks via prompting, in this work, we explore a new ICM framework, termed Prompt-ICM. To address both challenges by carefully learning task-driven prompts to coordinate well the compression process and downstream analysis. Specifically, our method is composed of two core designs: a) compression prompts, which are implemented as importance maps predicted by an information selector, and used to achieve different content-weighted bit allocations during compression according to different downstream tasks; b) task-adaptive prompts, which are instantiated as a few learnable parameters specifically for tuning compressed features for the specific intelligent task. Extensive experiments demonstrate that with a single feature codec and a few extra parameters, our proposed framework could efficiently support different kinds of intelligent tasks with much higher coding efficiency.