Asymptotic Analysis of Objectives based on Fisher Information in Active Learning
This work addresses a theoretical gap for researchers in active learning, offering insights into existing methods and potential new approaches, but it is incremental as it builds on prior use of FIR without introducing a new paradigm.
The paper tackles the lack of theoretical understanding behind using Fisher information ratio (FIR) as an objective in active learning, showing that FIR asymptotically bounds the expected variance of the log-likelihood ratio and providing a unifying framework for analysis.
Obtaining labels can be costly and time-consuming. Active learning allows a learning algorithm to intelligently query samples to be labeled for efficient learning. Fisher information ratio (FIR) has been used as an objective for selecting queries in active learning. However, little is known about the theory behind the use of FIR for active learning. There is a gap between the underlying theory and the motivation of its usage in practice. In this paper, we attempt to fill this gap and provide a rigorous framework for analyzing existing FIR-based active learning methods. In particular, we show that FIR can be asymptotically viewed as an upper bound of the expected variance of the log-likelihood ratio. Additionally, our analysis suggests a unifying framework that not only enables us to make theoretical comparisons among the existing querying methods based on FIR, but also allows us to give insight into the development of new active learning approaches based on this objective.