Business as Rulesual: A Benchmark and Framework for Business Rule Flow Modeling with LLMs
This work addresses the need for automated and interpretable business process automation by providing a benchmark and framework for modeling business rule flows, which is incremental as it builds on existing process mining but focuses on rule flows rather than procedural actions.
The paper tackles the underexplored problem of extracting rule flows from business documents by introducing a novel annotated Chinese dataset (BPRF) with 326 labeled business rules and proposing the ExIde framework for automatic extraction and dependency identification using LLMs, achieving effective results in benchmarking 12 SOTA LLMs on rule extraction and dependency classification tasks.
Process mining aims to discover, monitor and optimize the actual behaviors of real processes. While prior work has mainly focused on extracting procedural action flows from instructional texts, rule flows embedded in business documents remain underexplored. To this end, we introduce a novel annotated Chinese dataset, BPRF, which contains 50 business process documents with 326 explicitly labeled business rules across multiple domains. Each rule is represented as a <Condition, Action> pair, and we annotate logical dependencies between rules (sequential, conditional, or parallel). We also propose ExIde, a framework for automatic business rule extraction and dependency relationship identification using large language models (LLMs). We evaluate ExIde using 12 state-of-the-art (SOTA) LLMs on the BPRF dataset, benchmarking performance on both rule extraction and dependency classification tasks of current LLMs. Our results demonstrate the effectiveness of ExIde in extracting structured business rules and analyzing their interdependencies for current SOTA LLMs, paving the way for more automated and interpretable business process automation.