Semantic See-Through Rendering on Light Fields
This work addresses a specific challenge in computational photography for applications like virtual reality or imaging, representing an incremental improvement over prior methods.
The paper tackles the problem of achieving high-quality see-through rendering in light fields by introducing a semantic refocusing technique that removes foreground residues and maintains smooth transitions across focal depths, demonstrating effectiveness on synthetic and real datasets.
We present a novel semantic light field (LF) refocusing technique that can achieve unprecedented see-through quality. Different from prior art, our semantic see-through (SST) differentiates rays in their semantic meaning and depth. Specifically, we combine deep learning and stereo matching to provide each ray a semantic label. We then design tailored weighting schemes for blending the rays. Although simple, our solution can effectively remove foreground residues when focusing on the background. At the same time, SST maintains smooth transitions in varying focal depths. Comprehensive experiments on synthetic and new real indoor and outdoor datasets demonstrate the effectiveness and usefulness of our technique.