Mixed Membership Recurrent Neural Networks
This addresses the limitation of standard RNNs in handling varying time intervals and group effects for applications like dynamic topic modeling, though it appears incremental as an extension of existing frameworks.
The authors tackled the problem of modeling grouped sequential data with varying time intervals by proposing a mixed membership recurrent neural network that learns group-level base parameters, demonstrating it on 3.4 million online grocery shopping orders from 206K customers.
Models for sequential data such as the recurrent neural network (RNN) often implicitly model a sequence as having a fixed time interval between observations and do not account for group-level effects when multiple sequences are observed. We propose a model for grouped sequential data based on the RNN that accounts for varying time intervals between observations in a sequence by learning a group-level base parameter to which each sequence can revert. Our approach is motivated by the mixed membership framework, and we show how it can be used for dynamic topic modeling in which the distribution on topics (not the topics themselves) are evolving in time. We demonstrate our approach on a dataset of 3.4 million online grocery shopping orders made by 206K customers.