Spatial Lifting for Dense Prediction
This work addresses efficiency and accuracy challenges in vision tasks for researchers and practitioners, offering a new paradigm that is not incremental but introduces a novel approach.
The paper tackles dense prediction tasks like semantic segmentation and depth estimation by introducing Spatial Lifting (SL), a method that lifts 2D inputs to higher dimensions for processing, resulting in competitive performance while reducing model parameters by over 98% and lowering inference costs across 19 benchmark datasets.
We present Spatial Lifting (SL), a novel methodology for dense prediction tasks. SL operates by lifting standard inputs, such as 2D images, into a higher-dimensional space and subsequently processing them using networks designed for that higher dimension, such as a 3D U-Net. Counterintuitively, this dimensionality lifting allows us to achieve good performance on benchmark tasks compared to conventional approaches, while reducing inference costs and significantly lowering the number of model parameters. The SL framework produces intrinsically structured outputs along the lifted dimension. This emergent structure facilitates dense supervision during training and enables robust, near-zero-additional-cost prediction quality assessment at test time. We validate our approach across 19 benchmark datasets (13 for semantic segmentation and 6 for depth estimation), demonstrating competitive dense prediction performance while reducing the model parameter count by over 98% (in the U-Net case) and lowering inference costs. Spatial Lifting introduces a new vision modeling paradigm that offers a promising path toward more efficient, accurate, and reliable deep networks for dense prediction tasks in vision.