Kernel Selection for Modal Linear Regression: Optimal Kernel and IRLS Algorithm
This provides guidelines for kernel selection in modal linear regression, improving upon existing methods that often rely on Gaussian kernels, but it is incremental as it refines analysis and algorithms within an established framework.
The paper tackles the problem of kernel selection for modal linear regression, showing that a Biweight kernel minimizes asymptotic mean squared error and an Epanechnikov kernel ensures computational efficiency with finite iterations in IRLS.
Modal linear regression (MLR) is a method for obtaining a conditional mode predictor as a linear model. We study kernel selection for MLR from two perspectives: "which kernel achieves smaller error?" and "which kernel is computationally efficient?". First, we show that a Biweight kernel is optimal in the sense of minimizing an asymptotic mean squared error of a resulting MLR parameter. This result is derived from our refined analysis of an asymptotic statistical behavior of MLR. Secondly, we provide a kernel class for which iteratively reweighted least-squares algorithm (IRLS) is guaranteed to converge, and especially prove that IRLS with an Epanechnikov kernel terminates in a finite number of iterations. Simulation studies empirically verified that using a Biweight kernel provides good estimation accuracy and that using an Epanechnikov kernel is computationally efficient. Our results improve MLR of which existing studies often stick to a Gaussian kernel and modal EM algorithm specialized for it, by providing guidelines of kernel selection.