Neural Mixture Distributional Regression
This work addresses the challenge of scalable mixture modeling for researchers and practitioners in machine learning, though it appears incremental as it builds on existing concepts combining structured regression with neural networks.
The authors tackled the problem of estimating complex finite mixtures of distributional regressions in high-dimensional settings by proposing neural mixture distributional regression (NMDR), a framework that uses deep learning optimizers without specific model assumptions, and demonstrated its competitiveness through numerical experiments and a deep learning application.
We present neural mixture distributional regression (NMDR), a holistic framework to estimate complex finite mixtures of distributional regressions defined by flexible additive predictors. Our framework is able to handle a large number of mixtures of potentially different distributions in high-dimensional settings, allows for efficient and scalable optimization and can be applied to recent concepts that combine structured regression models with deep neural networks. While many existing approaches for mixture models address challenges in optimization of such and provide results for convergence under specific model assumptions, our approach is assumption-free and instead makes use of optimizers well-established in deep learning. Through extensive numerical experiments and a high-dimensional deep learning application we provide evidence that the proposed approach is competitive to existing approaches and works well in more complex scenarios.