A State-Space Approach to Nonstationary Discriminant Analysis
This provides a unified and data-efficient foundation for discriminant analysis under temporal distribution shift, addressing a domain-specific problem in time-series classification.
The paper tackled the problem of unreliable stationary classifiers when class-conditional distributions drift over time by proposing nonstationary linear and quadratic discriminant analysis (NSLDA and NSQDA) using state-space models, resulting in consistent improvements over baselines like LDA, QDA, and SVM with robustness to noise, missing data, and class imbalance.
Classical discriminant analysis assumes identically distributed training data, yet in many applications observations are collected over time and the class-conditional distributions drift. This population drift renders stationary classifiers unreliable. We propose a principled, model-based framework that embeds discriminant analysis within state-space models to obtain nonstationary linear discriminant analysis (NSLDA) and nonstationary quadratic discriminant analysis (NSQDA). For linear-Gaussian dynamics, we adapt Kalman smoothing to handle multiple samples per time step and develop two practical extensions: (i) an expectation-maximization (EM) approach that jointly estimates unknown system parameters, and (ii) a Gaussian mixture model (GMM)-Kalman method that simultaneously recovers unobserved time labels and parameters, a scenario common in practice. To address nonlinear or non-Gaussian drift, we employ particle smoothing to estimate time-varying class centroids, yielding fully nonstationary discriminant rules. Extensive simulations demonstrate consistent improvements over stationary linear discriminant analysis (LDA), quadratic discriminant analysis (QDA), and support vector machine (SVM) baselines, with robustness to noise, missing data, and class imbalance. This paper establishes a unified and data-efficient foundation for discriminant analysis under temporal distribution shift.