QUIS: Question-guided Insights Generation for Automated Exploratory Data Analysis
This addresses the problem of resource-intensive and human-dependent automated data exploration for data analysts, though it appears incremental as it builds on existing goal-oriented methods.
The paper tackles the challenge of fully automating Exploratory Data Analysis (EDA) by introducing QUIS, a system that generates questions iteratively to drive insight extraction without human intervention or retraining for new datasets, achieving full automation with no prior training required.
Discovering meaningful insights from a large dataset, known as Exploratory Data Analysis (EDA), is a challenging task that requires thorough exploration and analysis of the data. Automated Data Exploration (ADE) systems use goal-oriented methods with Large Language Models and Reinforcement Learning towards full automation. However, these methods require human involvement to anticipate goals that may limit insight extraction, while fully automated systems demand significant computational resources and retraining for new datasets. We introduce QUIS, a fully automated EDA system that operates in two stages: insight generation (ISGen) driven by question generation (QUGen). The QUGen module generates questions in iterations, refining them from previous iterations to enhance coverage without human intervention or manually curated examples. The ISGen module analyzes data to produce multiple relevant insights in response to each question, requiring no prior training and enabling QUIS to adapt to new datasets.