A Process-driven View on Summative Evaluation of Visual Analytics Solutions
This work addresses the challenge of heterogeneous evaluation practices for visual analytics practitioners, but it is incremental as it provides a high-level overview rather than a new method.
The paper tackles the problem of evaluating visual analytics solutions by proposing a generic evaluation model (GEM) that generalizes the process of usefulness evaluation, serving to educate practitioners, highlight risks, and guide method selection.
Many evaluation methods have been applied to assess the usefulness of visual analytics solutions. These methods are branching from a variety of origins with different assumptions, and goals. We provide a high-level overview of the process employed in each method using the generic evaluation model "GEM" that generalizes the process of usefulness evaluation. The model treats evaluation methods as processes that generate evidence of usefulness as output. Our model serves three purposes: It educate new VA practitioners about the heterogeneous evaluation practices in the field, it highlights potential risks in the process of evaluation which reduces their validity and It provide a guideline to elect suitable evaluation method.