AI-Driven Early Warning Systems for Student Success: Discovering Static Feature Dominance in Temporal Prediction Models
This work addresses student success in online education by providing incremental insights into model performance and feature importance for timely interventions.
This study tackled the problem of early identification of at-risk students in online learning by analyzing temporal prediction models up to Week 20, finding that static demographic features dominate predictions with 68% importance, enabling assessment-free early prediction, and the LSTM model achieved 97% recall at Week 2 for early intervention.
Early identification of at-risk students is critical for effective intervention in online learning environments. This study extends temporal prediction analysis to Week 20 (50% of course duration), comparing Decision Tree and Long Short- Term Memory (LSTM) models across six temporal snapshots. Our analysis reveals that different performance metrics matter at different intervention stages: high recall is critical for early intervention (Weeks 2-4), while balanced precision-recall is important for mid-course resource allocation (Weeks 8-16), and high precision becomes paramount in later stages (Week 20). We demonstrate that static demographic features dominate predictions (68% importance), enabling assessment-free early prediction. The LSTM model achieves 97% recall at Week 2, making it ideal for early intervention, while Decision Tree provides stable balanced performance (78% accuracy) during mid-course. By Week 20, both models converge to similar recall (68%), but LSTM achieves higher precision (90% vs 86%). Our findings also suggest that model selection should depend on intervention timing, and that early signals (Weeks 2-4) are sufficient for reliable initial prediction using primarily demographic and pre-enrollment information.