Human Guided Learning of Transparent Regression Models
This addresses the need for interpretable regression models in domains like rankings, though it is incremental as it builds on existing human-in-the-loop and regression techniques.
The paper tackles the problem of predicting continuous values for item orderings by introducing a human-in-the-loop approach called HuGuR, which incorporates human-understandable constraints into gradient boosted regression models; results from a user study show that user-built models outperform baselines on small datasets and perform competitively on others, while being transparent.
We present a human-in-the-loop (HIL) approach to permutation regression, the novel task of predicting a continuous value for a given ordering of items. The model is a gradient boosted regression model that incorporates simple human-understandable constraints of the form x < y, i.e. item x has to be before item y, as binary features. The approach, HuGuR (Human Guided Regression), lets a human explore the search space of such transparent regression models. Interacting with HuGuR, users can add, remove, and refine order constraints interactively, while the coefficients are calculated on the fly. We evaluate HuGuR in a user study and compare the performance of user-built models with multiple baselines on 9 data sets. The results show that the user-built models outperform the compared methods on small data sets and in general perform on par with the other methods, while being in principle understandable for humans. On larger datasets from the same domain, machine-induced models begin to outperform the user-built models. Further work will study the trust users have in models when constructed by themselves and how the scheme can be transferred to other pattern domains, such as strings, sequences, trees, or graphs.