Interpretable Models for Understanding Immersive Simulations
This addresses the need for interpretable models in education and healthcare training simulations, but it is incremental as it focuses on comparative evaluation methods rather than introducing a new paradigm.
The paper tackled the problem of evaluating model interpretability for unsupervised learning on high-dimensional time series from immersive simulations, showing that models optimizing interpretability differ from those optimizing statistical criteria, with a Bayesian approach performing well on both measures.
This paper describes methods for comparative evaluation of the interpretability of models of high dimensional time series data inferred by unsupervised machine learning algorithms. The time series data used in this investigation were logs from an immersive simulation like those commonly used in education and healthcare training. The structures learnt by the models provide representations of participants' activities in the simulation which are intended to be meaningful to people's interpretation. To choose the model that induces the best representation, we designed two interpretability tests, each of which evaluates the extent to which a model's output aligns with people's expectations or intuitions of what has occurred in the simulation. We compared the performance of the models on these interpretability tests to their performance on statistical information criteria. We show that the models that optimize interpretability quality differ from those that optimize (statistical) information theoretic criteria. Furthermore, we found that a model using a fully Bayesian approach performed well on both the statistical and human-interpretability measures. The Bayesian approach is a good candidate for fully automated model selection, i.e., when direct empirical investigations of interpretability are costly or infeasible.