Temperature in SLMs: Impact on Incident Categorization in On-Premises Environments
This work addresses the need for cost-effective and confidential incident categorization for SOCs and CSIRTs, though it appears incremental by testing existing models with standard hyperparameters.
The study investigated whether small language models (SLMs) can effectively automate incident categorization in on-premises environments, finding that temperature has minimal impact while model size and GPU capacity are key determinants of performance.
SOCs and CSIRTs face increasing pressure to automate incident categorization, yet the use of cloud-based LLMs introduces costs, latency, and confidentiality risks. We investigate whether locally executed SLMs can meet this challenge. We evaluated 21 models ranging from 1B to 20B parameters, varying the temperature hyperparameter and measuring execution time and precision across two distinct architectures. The results indicate that temperature has little influence on performance, whereas the number of parameters and GPU capacity are decisive factors.