Towards Automatically Inferring Constraints to Identify Implicit Assumptions in Data Analysis
For data analysts and reproducibility researchers, this work provides a unified perspective and initial tooling to detect implicit assumptions, though it is incremental and preliminary.
The paper addresses reproducibility issues in data analysis scripts due to implicit assumptions (e.g., package versions, data shapes). It proposes a proof-of-concept using static analysis to infer constraints that make these assumptions explicit, enabling automated transformation into reproducible artifacts.
High-level languages such as R or Python are used frequently to analyze and visualize data in the form of scripts or notebooks. However, these artifacts suffer from reproducibility issues due to what we frame as implicit assumptions made by the authors. Such assumptions range from package versions and shapes of involved data tables, to manual and often undocumented setup steps. Within this work, we provide a unified, example-driven perspective on implicit assumptions in data analysis backed by an explorative proof-of-concept implementation. With this perspective, we propose the use of static analysis techniques to identify such assumptions and to make them explicit in the form of code constraints, focusing on the inclusion of data-analysis-specific issues. Such constraints can then be used to automatically transform these scripts into executable and reproducible artifacts, to check these assumptions at runtime, and to serve as documentation to support code reuse and comprehension.