David vs. Goliath in Next Activity Prediction: Argmax vs. LSTM, Transformer, and LLM
For researchers and practitioners in predictive process monitoring, this paper challenges the necessity of complex models by showing that a simple baseline can be competitive.
The paper benchmarks next activity prediction methods, finding that a simple argmax baseline matches or approaches the performance of billion-parameter LLMs across seven real-life event logs, with pretraining and model size showing little consistent benefit.
Next activity prediction (NAP) is a cornerstone of predictive process monitoring (PPM), enabling organizations to move from retrospective analysis to proactive process steering. The PPM field has progressed from classical machine learning through deep learning architectures such as LSTMs and Transformers to large language models (LLMs). Despite growing model complexity, no benchmark jointly compares LLMs, Transformers, LSTMs, and simple baselines in a direct sequence modeling setting for NAP. In this paper, we fill this gap with a systematic benchmark. We compare vocabulary-adapted LLMs, Transformers trained from scratch, LLM-distilled Transformers, and LSTMs against a simple counting-based argmax baseline across seven real-life event logs. Our results tell a David vs. Goliath story: pretraining confers no consistent improvement over training from scratch, model size shows little effect on performance, and on most datasets the argmax baseline matches or approaches the performance of billion-parameter LLMs.