Contextual Bandits with Sparse Data in Web setting
It provides an updated overview for researchers and practitioners dealing with sparse data problems in web-based contextual bandit applications, but it is incremental as it synthesizes existing methods without introducing new techniques.
This paper conducted a scoping study to identify and categorize current methods for handling sparse data in contextual bandits within web settings, reviewing 19 method articles from 2017-2020 and organizing them into five categories to facilitate method selection and modification.
This paper is a scoping study to identify current methods used in handling sparse data with contextual bandits in web settings. The area is highly current and state of the art methods are identified. The years 2017-2020 are investigated, and 19 method articles are identified, and two review articles. Five categories of methods are described, making it easy to choose how to address sparse data using contextual bandits with a method available for modification in the specific setting of concern. In addition, each method has multiple techniques to choose from for future evaluation. The problem areas are also mentioned that each article covers. An overall updated understanding of sparse data problems using contextual bandits in web settings is given. The identified methods are policy evaluation (off-line and on-line) , hybrid-method, model representation (clusters and deep neural networks), dimensionality reduction, and simulation.