Benchmarking Learnt Radio Localisation under Distribution Shift
This work addresses the deployability of RF localisation systems for practitioners, providing benchmarking and best practices, but it is incremental as it focuses on characterising existing methods rather than proposing new ones.
The paper tackles the problem of deploying learning-based radio frequency localisation systems under real-world distribution shifts by introducing RadioBench, a benchmarking suite with 8 state-of-the-art networks and 5 new datasets, analyzing 10k models to uncover behaviors across learning, shift proneness, and localisation performance.
Deploying radio frequency (RF) localisation systems invariably entails non-trivial effort, particularly for the latest learning-based breeds. There has been little prior work on characterising and comparing how learnt localiser networks can be deployed in the field under real-world RF distribution shifts. In this paper, we present RadioBench: a suite of 8 learnt localiser nets from the state-of-the-art to study and benchmark their real-world deployability, utilising five novel industry-grade datasets. We train 10k models to analyse the inner workings of these learnt localiser nets and uncover their differing behaviours across three performance axes: (i) learning, (ii) proneness to distribution shift, and (iii) localisation. We use insights gained from this analysis to recommend best practices for the deployability of learning-based RF localisation under practical constraints.