LORD: Leveraging Open-Set Recognition with Unknown Data
This work addresses the challenge of handling out-of-distribution data for deployed classifiers, offering a systematic approach to open-set recognition with potential applications in safety-critical domains, though it is incremental as it builds on existing classifiers and training strategies.
The paper tackles the problem of open-set recognition by introducing LORD, a framework that explicitly models open space during classifier training and uses background data or mixup to improve recognition of unknown data, achieving consistent improvements across benchmarks.
Handling entirely unknown data is a challenge for any deployed classifier. Classification models are typically trained on a static pre-defined dataset and are kept in the dark for the open unassigned feature space. As a result, they struggle to deal with out-of-distribution data during inference. Addressing this task on the class-level is termed open-set recognition (OSR). However, most OSR methods are inherently limited, as they train closed-set classifiers and only adapt the downstream predictions to OSR. This work presents LORD, a framework to Leverage Open-set Recognition by exploiting unknown Data. LORD explicitly models open space during classifier training and provides a systematic evaluation for such approaches. We identify three model-agnostic training strategies that exploit background data and applied them to well-established classifiers. Due to LORD's extensive evaluation protocol, we consistently demonstrate improved recognition of unknown data. The benchmarks facilitate in-depth analysis across various requirement levels. To mitigate dependency on extensive and costly background datasets, we explore mixup as an off-the-shelf data generation technique. Our experiments highlight mixup's effectiveness as a substitute for background datasets. Lightweight constraints on mixup synthesis further improve OSR performance.