On the Evaluation of Intelligent Process Automation
This work addresses the problem of inconsistent evaluation in IPA for researchers and practitioners, though it is incremental as it builds on existing systems and use cases.
The paper tackles the lack of formal definition and assessment in Intelligent Process Automation (IPA) by providing a formalization and proposing specific metrics for empirical evaluation, comparing it to related tasks like end-user programming and program synthesis.
Intelligent Process Automation (IPA) is emerging as a sub-field of AI to support the automation of long-tail processes which requires the coordination of tasks across different systems. So far, the field of IPA has been largely driven by systems and use cases, lacking a more formal definition of the task and its assessment. This paper aims to address this gap by providing a formalisation of IPA and by proposing specific metrics to support the empirical evaluation of IPA systems. This work also compares and contrasts IPA against related tasks such as end-user programming and program synthesis.