In Search for a Generalizable Method for Source Free Domain Adaptation
This work addresses the generalizability of SFDA methods for researchers and practitioners, revealing limitations in existing approaches and offering a more robust solution, though it is incremental as it builds on prior SFDA techniques.
The paper tackled the problem of source-free domain adaptation (SFDA) by testing existing methods on bioacoustics data, finding they performed inconsistently and sometimes worse than no adaptation, and proposed a new simple method that outperformed existing ones on bioacoustics shifts while showing strong results on vision datasets.
Source-free domain adaptation (SFDA) is compelling because it allows adapting an off-the-shelf model to a new domain using only unlabelled data. In this work, we apply existing SFDA techniques to a challenging set of naturally-occurring distribution shifts in bioacoustics, which are very different from the ones commonly studied in computer vision. We find existing methods perform differently relative to each other than observed in vision benchmarks, and sometimes perform worse than no adaptation at all. We propose a new simple method which outperforms the existing methods on our new shifts while exhibiting strong performance on a range of vision datasets. Our findings suggest that existing SFDA methods are not as generalizable as previously thought and that considering diverse modalities can be a useful avenue for designing more robust models.