Model selection for behavioral learning data and applications to contextual bandits
This work addresses model selection for behavioral learning data, which is incremental as it adapts existing methods to non-stationary contexts for applications like contextual bandits.
The authors tackled the problem of selecting models that best explain individual learning from behavioral data, proposing a hold-out procedure and an AIC-type criterion adapted for non-stationary dependent data, with theoretical error bounds close to standard i.i.d. cases. They applied these methods to contextual bandit models and demonstrated their use on synthetic and experimental human categorization data.
Learning for animals or humans is the process that leads to behaviors better adapted to the environment. This process highly depends on the individual that learns and is usually observed only through the individual's actions. This article presents ways to use this individual behavioral data to find the model that best explains how the individual learns. We propose two model selection methods: a general hold-out procedure and an AIC-type criterion, both adapted to non-stationary dependent data. We provide theoretical error bounds for these methods that are close to those of the standard i.i.d. case. To compare these approaches, we apply them to contextual bandit models and illustrate their use on both synthetic and experimental learning data in a human categorization task.