Beyond Weight Adaptation: Feature-Space Domain Injection for Cross-Modal Ship Re-Identification
This work addresses cross-modality ship re-identification for maritime target tracking, offering a more efficient fine-tuning approach that reduces reliance on large paired datasets, though it is incremental in adapting existing foundation models.
The paper tackles the problem of cross-modality ship re-identification by proposing a novel parameter-efficient fine-tuning strategy called Domain Representation Injection, which shifts optimization to the feature space to bridge modality gaps without altering pre-trained weights, achieving state-of-the-art performance with minimal parameters, such as 57.9% and 60.5% mAP on the HOSS-ReID dataset using only 1.54M and 7.05M parameters.
Cross-Modality Ship Re-Identification (CMS Re-ID) is critical for achieving all-day and all-weather maritime target tracking, yet it is fundamentally challenged by significant modality discrepancies. Mainstream solutions typically rely on explicit modality alignment strategies; however, this paradigm heavily depends on constructing large-scale paired datasets for pre-training. To address this, grounded in the Platonic Representation Hypothesis, we explore the potential of Vision Foundation Models (VFMs) in bridging modality gaps. Recognizing the suboptimal performance of existing generic Parameter-Efficient Fine-Tuning (PEFT) methods that operate within the weight space, particularly on limited-capacity models, we shift the optimization perspective to the feature space and propose a novel PEFT strategy termed Domain Representation Injection (DRI). Specifically, while keeping the VFM fully frozen to maximize the preservation of general knowledge, we design a lightweight, learnable Offset Encoder to extract domain-specific representations rich in modality and identity attributes from raw inputs. Guided by the contextual information of intermediate features at different layers, a Modulator adaptively transforms these representations. Subsequently, they are injected into the intermediate layers via additive fusion, dynamically reshaping the feature distribution to adapt to the downstream task without altering the VFM's pre-trained weights. Extensive experimental results demonstrate the superiority of our method, achieving State-of-the-Art (SOTA) performance with minimal trainable parameters. For instance, on the HOSS-ReID dataset, we attain 57.9\% and 60.5\% mAP using only 1.54M and 7.05M parameters, respectively. The code is available at https://github.com/TingfengXian/DRI.