Joint Learning of Frequency and Spatial Domains for Dense Predictions
This addresses the need for more efficient and parameter-light models in dense prediction tasks like depth estimation and semantic segmentation, though it appears incremental in combining existing domains.
The paper tackles the problem of neural networks neglecting frequency domain learning by proposing a joint learning paradigm for frequency and spatial domains, which achieves competitive performance in self-supervised depth estimation and semantic segmentation without pretraining and significantly reduces parameters.
Current artificial neural networks mainly conduct the learning process in the spatial domain but neglect the frequency domain learning. However, the learning course performed in the frequency domain can be more efficient than that in the spatial domain. In this paper, we fully explore frequency domain learning and propose a joint learning paradigm of frequency and spatial domains. This paradigm can take full advantage of the preponderances of frequency learning and spatial learning; specifically, frequency and spatial domain learning can effectively capture global and local information, respectively. Exhaustive experiments on two dense prediction tasks, i.e., self-supervised depth estimation and semantic segmentation, demonstrate that the proposed joint learning paradigm can 1) achieve performance competitive to those of state-of-the-art methods in both depth estimation and semantic segmentation tasks, even without pretraining; and 2) significantly reduce the number of parameters compared to other state-of-the-art methods, which provides more chance to develop real-world applications. We hope that the proposed method can encourage more research in cross-domain learning.