What you see is what you get: Experience ranking with deep neural dataset-to-dataset similarity for topological localisation
This work addresses the challenge of robust visual navigation for autonomous systems by enabling pre-runtime prediction of localization outcomes, though it is incremental as it builds on existing deep learning methods for place recognition.
The paper tackles the problem of ranking visual memories for efficient topological localization by using deep neural dataset-to-dataset similarity to predict localization performance without ground truth, showing that differences in neuron activation statistics correlate with performance across appearance changes like weather and lighting, with validation on datasets such as Nordland and Oxford University Parks demonstrating excellent ranking ability.
Recalling the most relevant visual memories for localisation or understanding a priori the likely outcome of localisation effort against a particular visual memory is useful for efficient and robust visual navigation. Solutions to this problem should be divorced from performance appraisal against ground truth - as this is not available at run-time - and should ideally be based on generalisable environmental observations. For this, we propose applying the recently developed Visual DNA as a highly scalable tool for comparing datasets of images - in this work, sequences of map and live experiences. In the case of localisation, important dataset differences impacting performance are modes of appearance change, including weather, lighting, and season. Specifically, for any deep architecture which is used for place recognition by matching feature volumes at a particular layer, we use distribution measures to compare neuron-wise activation statistics between live images and multiple previously recorded past experiences, with a potentially large seasonal (winter/summer) or time of day (day/night) shift. We find that differences in these statistics correlate to performance when localising using a past experience with the same appearance gap. We validate our approach over the Nordland cross-season dataset as well as data from Oxford's University Parks with lighting and mild seasonal change, showing excellent ability of our system to rank actual localisation performance across candidate experiences.