Expectation Versus Reality: The Failed Evaluation of a Mixed-Initiative Visualization System
This work addresses the challenge of designing effective mixed-initiative systems for data visualization, but it is incremental as it reports negative results from flawed experiments.
The study tackled evaluating a mixed-initiative visualization system for handling complex datasets, but the experiments failed to show any impact on user interactions or system usability due to design flaws and unrealistic user expectations.
Our research aimed to present the design and evaluation of a mixed-initiative system that aids the user in handling complex datasets and dense visualization systems. We attempted to demonstrate this system with two trials of an online between-groups, two-by-two study, measuring the effects of this mixed-initiative system on user interactions and system usability. However, due to flaws in the interface design and the expectations that we put on users, we were unable to show that the adaptive system had an impact on user interactions or system usability. In this paper, we discuss the unexpected findings that we found from our "failed" experiments and examine how we can learn from our failures to improve further research.