Learning Nonlinear Mixtures: Identifiability and Algorithm
This addresses a foundational issue in machine learning for applications like topic modeling and source separation, but it is incremental as it builds on existing linear mixture frameworks.
The paper tackles the problem of identifiability in nonlinear mixture models, which are more realistic but less studied than linear ones, by proposing a criterion with guarantees and a neural network-based algorithm, showing effectiveness on synthetic and real data.
Linear mixture models have proven very useful in a plethora of applications, e.g., topic modeling, clustering, and source separation. As a critical aspect of the linear mixture models, identifiability of the model parameters is well-studied, under frameworks such as independent component analysis and constrained matrix factorization. Nevertheless, when the linear mixtures are distorted by an unknown nonlinear functions -- which is well-motivated and more realistic in many cases -- the identifiability issues are much less studied. This work proposes an identification criterion for a nonlinear mixture model that is well grounded in many real-world applications, and offers identifiability guarantees. A practical implementation based on a judiciously designed neural network is proposed to realize the criterion, and an effective learning algorithm is proposed. Numerical results on synthetic and real-data corroborate effectiveness of the proposed method.