One is Plenty: A Polymorphic Feature Interpreter for Immutable Heterogeneous Collaborative Perception
This addresses a scalability bottleneck in multi-agent autonomous driving systems where agents cannot modify their perception networks.
The paper tackles the challenge of immutable heterogeneity in collaborative perception for autonomous driving, where agents with different fixed perception networks struggle to interpret each other's features. The proposed PolyInter interpreter improves collaborative perception precision by up to 11.1% compared to state-of-the-art methods while requiring only 1.4% parameter training for new agents.
Collaborative perception in autonomous driving significantly enhances the perception capabilities of individual agents. Immutable heterogeneity, where agents have different and fixed perception networks, presents a major challenge due to the semantic gap in exchanged intermediate features without modifying the perception networks. Most existing methods bridge the semantic gap through interpreters. However, they either require training a new interpreter for each new agent type, limiting extensibility, or rely on a two-stage interpretation via an intermediate standardized semantic space, causing cumulative semantic loss. To achieve both extensibility in immutable heterogeneous scenarios and low-loss feature interpretation, we propose PolyInter, a polymorphic feature interpreter. It provides an extension point where new agents integrate by overriding only their specific prompts, which are learnable parameters that guide interpretation, while reusing PolyInter's remaining parameters. By leveraging polymorphism, our design enables a single interpreter to accommodate diverse agents and interpret their features into the ego agent's semantic space. Experiments on the OPV2V dataset demonstrate that PolyInter improves collaborative perception precision by up to 11.1% compared to SOTA interpreters, while comparable results can be achieved by training only 1.4% of PolyInter's parameters when adapting to new agents. Code is available at https://github.com/yuchen-xia/PolyInter.