Fitting a deeply-nested hierarchical model to a large book review dataset using a moment-based estimator
This addresses a computational bottleneck in recommender systems and other contexts with deeply-nested hierarchical models, though it is incremental as it extends an existing method.
The paper tackled the computational challenge of fitting a deeply-nested hierarchical model to a large book review dataset by extending a moment-based estimator, achieving an order of magnitude faster performance than standard maximum likelihood procedures.
We consider a particular instance of a common problem in recommender systems: using a database of book reviews to inform user-targeted recommendations. In our dataset, books are categorized into genres and sub-genres. To exploit this nested taxonomy, we use a hierarchical model that enables information pooling across across similar items at many levels within the genre hierarchy. The main challenge in deploying this model is computational: the data sizes are large, and fitting the model at scale using off-the-shelf maximum likelihood procedures is prohibitive. To get around this computational bottleneck, we extend a moment-based fitting procedure proposed for fitting single-level hierarchical models to the general case of arbitrarily deep hierarchies. This extension is an order of magnetite faster than standard maximum likelihood procedures. The fitting method can be deployed beyond recommender systems to general contexts with deeply-nested hierarchical generalized linear mixed models.