Tight Rates in Supervised Outlier Transfer Learning
This work addresses the critical issue of data scarcity in outlier detection for practitioners, providing theoretical insights into transfer learning, though it appears incremental as it builds on existing frameworks like Neyman-Pearson classification.
The paper tackles the problem of scarce outlier data in supervised outlier detection by analyzing when and how knowledge can be transferred from imperfect source data to a target task, showing that even dissimilar sources can enable fast transfer and that information-theoretic limits are achievable with adaptive procedures.
A critical barrier to learning an accurate decision rule for outlier detection is the scarcity of outlier data. As such, practitioners often turn to the use of similar but imperfect outlier data from which they might transfer information to the target outlier detection task. Despite the recent empirical success of transfer learning approaches in outlier detection, a fundamental understanding of when and how knowledge can be transferred from a source to a target outlier detection task remains elusive. In this work, we adopt the traditional framework of Neyman-Pearson classification -- which formalizes supervised outlier detection -- with the added assumption that one has access to some related but imperfect outlier data. Our main results are as follows: We first determine the information-theoretic limits of the problem under a measure of discrepancy that extends some existing notions from traditional balanced classification; interestingly, unlike in balanced classification, seemingly very dissimilar sources can provide much information about a target, thus resulting in fast transfer. We then show that, in principle, these information-theoretic limits are achievable by adaptive procedures, i.e., procedures with no a priori information on the discrepancy between source and target outlier distributions.