DUT-LFSaliency: Versatile Dataset and Light Field-to-RGB Saliency Detection
This work addresses the problem of versatile and efficient saliency detection for both high-performance and mobile devices, which is an incremental improvement for the computer vision community.
This paper introduces a new large-scale dataset for saliency detection across RGB, RGB-D, and light field data, comprising 102 classes and 4204 samples. They also propose an asymmetrical two-stream model where the Focal stream achieves state-of-the-art performance, and the RGB stream reduces model size by 83% and boosts FPS by 5 times compared to the best performing method.
Light field data exhibit favorable characteristics conducive to saliency detection. The success of learning-based light field saliency detection is heavily dependent on how a comprehensive dataset can be constructed for higher generalizability of models, how high dimensional light field data can be effectively exploited, and how a flexible model can be designed to achieve versatility for desktop computers and mobile devices. To answer these questions, first we introduce a large-scale dataset to enable versatile applications for RGB, RGB-D and light field saliency detection, containing 102 classes and 4204 samples. Second, we present an asymmetrical two-stream model consisting of the Focal stream and RGB stream. The Focal stream is designed to achieve higher performance on desktop computers and transfer focusness knowledge to the RGB stream, relying on two tailor-made modules. The RGB stream guarantees the flexibility and memory/computation efficiency on mobile devices through three distillation schemes. Experiments demonstrate that our Focal stream achieves state-of-the-arts performance. The RGB stream achieves Top-2 F-measure on DUTLF-V2, which tremendously minimizes the model size by 83% and boosts FPS by 5 times, compared with the best performing method. Furthermore, our proposed distillation schemes are applicable to RGB saliency models, achieving impressive performance gains while ensuring flexibility.