Vaarunay Kaushal

Vaarunay Kaushal, Taranveer Singh

Multi-agent systems where Large Language Models (LLMs) deliberate to form consensus have gained significant attention, yet their practical value over simpler methods remains under-scrutinized. We introduce DELIBERATIONBENCH, a controlled benchmark evaluating three deliberation protocols against a strong baseline of selecting the best response from a pool of model outputs. Across 270 questions and three independent seeds (810 total evaluations), we find a striking negative result: the best-single baseline achieves an 82.5% +- 3.3% win rate, dramatically outperforming the best deliberation protocol(13.8% +- 2.6%). This 6.0x performance gap is statistically significant (p < 0.01) and comes at 1.5-2.5x higher computational cost. Our findings challenge assumptions that complexity enhances quality in multi-LLM systems.

Vaarunay Kaushal, Rajib Mall

Early identification of at-risk students is critical for effective intervention in online learning environments. This study extends temporal prediction analysis to Week 20 (50% of course duration), comparing Decision Tree and Long Short- Term Memory (LSTM) models across six temporal snapshots. Our analysis reveals that different performance metrics matter at different intervention stages: high recall is critical for early intervention (Weeks 2-4), while balanced precision-recall is important for mid-course resource allocation (Weeks 8-16), and high precision becomes paramount in later stages (Week 20). We demonstrate that static demographic features dominate predictions (68% importance), enabling assessment-free early prediction. The LSTM model achieves 97% recall at Week 2, making it ideal for early intervention, while Decision Tree provides stable balanced performance (78% accuracy) during mid-course. By Week 20, both models converge to similar recall (68%), but LSTM achieves higher precision (90% vs 86%). Our findings also suggest that model selection should depend on intervention timing, and that early signals (Weeks 2-4) are sufficient for reliable initial prediction using primarily demographic and pre-enrollment information.