Classifier-Augmented Generation for Structured Workflow Prediction
This addresses the time-consuming and expertise-heavy process of configuring ETL tools like IBM DataStage for data engineers, representing a domain-specific advancement.
The paper tackles the problem of configuring complex ETL workflows from natural language descriptions by proposing a Classifier-Augmented Generation system that predicts workflow structure and detailed configurations, showing improved accuracy and efficiency while reducing token usage compared to baselines.
ETL (Extract, Transform, Load) tools such as IBM DataStage allow users to visually assemble complex data workflows, but configuring stages and their properties remains time consuming and requires deep tool knowledge. We propose a system that translates natural language descriptions into executable workflows, automatically predicting both the structure and detailed configuration of the flow. At its core lies a Classifier-Augmented Generation (CAG) approach that combines utterance decomposition with a classifier and stage-specific few-shot prompting to produce accurate stage predictions. These stages are then connected into non-linear workflows using edge prediction, and stage properties are inferred from sub-utterance context. We compare CAG against strong single-prompt and agentic baselines, showing improved accuracy and efficiency, while substantially reducing token usage. Our architecture is modular, interpretable, and capable of end-to-end workflow generation, including robust validation steps. To our knowledge, this is the first system with a detailed evaluation across stage prediction, edge layout, and property generation for natural-language-driven ETL authoring.