Light over Heavy: Automated Performance Requirements Quantification with Linguistic Inducement
This addresses the need for efficient and accurate performance requirement quantification in engineering tasks like configuration tuning and testing, offering a specialized alternative to general LLM-driven methods.
The paper tackles the problem of automatically quantifying performance requirements from text, which is typically done manually and is error-prone, by presenting LQPR, a lightweight approach that achieves the best performance in 75% or more cases with two orders less cost compared to nine state-of-the-art methods.
Elicited performance requirements need to be quantified for compliance in different engineering tasks, e.g., configuration tuning and performance testing. Much existing work has relied on manual quantification, which is expensive and error-prone due to the imprecision. In this paper, we present LQPR, a highly efficient automatic approach for performance requirements quantification.LQPR relies on a new theoretical framework that converts quantification as a classification problem. Despite the prevalent applications of Large Language Models (LLMs) for requirement analytics, LQPR takes a different perspective to address the classification: we observed that performance requirements can exhibit strong patterns and are often short/concise, therefore we design a lightweight linguistically induced matching mechanism. We compare LQPR against nine state-of-the-art learning-based approaches over diverse datasets, demonstrating that it is ranked as the sole best for 75% or more cases with two orders less cost. Our work proves that, at least for performance requirement quantification, specialized methods can be more suitable than the general LLM-driven approaches.