SceneMiner: Identity-Preserving Multi-Task Fine-Tuning for Unified BEV Scene Mining
For autonomous driving researchers, this provides a practical method to mine safety-critical scenes without LiDAR or radar, though the performance is moderate and the approach is incremental.
SceneMiner introduces a unified BEV pipeline for mining hard driving scenes using a frozen vision-language backbone, achieving multi-label tagging with mAP 0.4614 and text-prompted retrieval while training only ~102k parameters. The key finding is cross-task interference, mitigated by identity-preserving multi-task fine-tuning.
Mining hard, safety-critical scenes from driving logs is bottlenecked by the absence of difficulty labels, and no single proxy, collision risk, trajectory ambiguity, or semantic rarity suffices to find such scenes on its own. We present SceneMiner, a unified, camera-only bird's-eye-view pipeline that emits complementary mining signals from a frozen vision-language backbone in a single forward pass, with no LiDAR or radar: a retrieval embedding for text-prompted scenario search, a multi-label scene-tag distribution, and a continuous physics-based risk score (a motion forecast is a byproduct, not a contribution). Building such a multi-head model exposes our central finding, a failure mode we term cross-task interference: adding or upgrading one head shifts a shared activation stream and degrades weight-frozen sibling heads, so freezing parameters alone is insufficient. Our contribution, identity-preserving multi-task fine-tuning, removes this interference by zero-initializing every new sub-module and freezing every parameter that feeds the shared stream. The mining heads are thereby preserved bit-identically while training only ~102k parameters. The tagging head reaches mAP 0.4614 (micro-F1 0.5557) on 20 scene tags by pooling each scene into 32 visual tokens, and the embedding head supports text-prompted retrieval, validated qualitatively. Code is available at: https://anonymous.4open.science/r/sceneminer_anonymous-64E5