Detecting Scarce and Sparse Anomalous: Solving Dual Imbalance in Multi-Instance Learning
This addresses a needle-in-a-haystack problem for real-world anomaly detection applications, but it is incremental as it builds on existing PU learning methods.
The paper tackled the challenge of detecting anomalous samples that are both scarce and sparse, which creates a dual imbalance in Multi-Instance Learning, by reformulating it as a fine-grained PU learning problem and proposing the BFGPU framework; experiments on synthetic and real-world datasets showed its effectiveness.
In real-world applications, it is highly challenging to detect anomalous samples with extremely sparse anomalies, as they are highly similar to and thus easily confused with normal samples. Moreover, the number of anomalous samples is inherently scarce. This results in a dual imbalance Multi-Instance Learning (MIL) problem, manifesting at both the macro and micro levels. To address this "needle-in-a-haystack problem", we find that MIL problem can be reformulated as a fine-grained PU learning problem. This allows us to address the imbalance issue in an unbiased manner using micro-level balancing mechanisms. To this end, we propose a novel framework, Balanced Fine-Grained Positive-Unlabeled (BFGPU)-based on rigorous theoretical foundations. Extensive experiments on both synthetic and real-world datasets demonstrate the effectiveness of BFGPU.