Model Inconsistent but Correlated Noise: Multi-view Subspace Learning with Regularized Mixture of Gaussians
This work addresses noise modeling in multi-view data for machine learning applications, but it is incremental as it builds on existing subspace learning methods with a more sophisticated noise model.
The paper tackles the problem of modeling complex and correlated noise in multi-view subspace learning by proposing a method that uses a regularized mixture of Gaussians to encode shared and view-specific noise, resulting in improved performance as shown in experimental results.
Multi-view subspace learning (MSL) aims to find a low-dimensional subspace of the data obtained from multiple views. Different from single view case, MSL should take both common and specific knowledge among different views into consideration. To enhance the robustness of model, the complexity, non-consistency and similarity of noise in multi-view data should be fully taken into consideration. Most current MSL methods only assume a simple Gaussian or Laplacian distribution for the noise while neglect the complex noise configurations in each view and noise correlations among different views of practical data. To this issue, this work initiates a MSL method by encoding the multi-view-shared and single-view-specific noise knowledge in data. Specifically, we model data noise in each view as a separated Mixture of Gaussians (MoG), which can fit a wider range of complex noise types than conventional Gaussian/Laplacian. Furthermore, we link all single-view-noise as a whole by regularizing them by a common MoG component, encoding the shared noise knowledge among them. Such regularization component can be formulated as a concise KL-divergence regularization term under a MAP framework, leading to good interpretation of our model and simple EM-based solving strategy to the problem. Experimental results substantiate the superiority of our method.