Particle Filter Recurrent Neural Networks
This addresses the challenge of uncertainty modeling in sequential data for applications like text classification and stock prediction, representing a novel method for a known bottleneck.
The paper tackles the problem of handling highly variable and noisy sequential data by introducing Particle Filter Recurrent Neural Networks (PF-RNNs), which model uncertainty in latent states using particles, and results show they outperform standard gated RNNs on a synthetic dataset and 10 real-world datasets.
Recurrent neural networks (RNNs) have been extraordinarily successful for prediction with sequential data. To tackle highly variable and noisy real-world data, we introduce Particle Filter Recurrent Neural Networks (PF-RNNs), a new RNN family that explicitly models uncertainty in its internal structure: while an RNN relies on a long, deterministic latent state vector, a PF-RNN maintains a latent state distribution, approximated as a set of particles. For effective learning, we provide a fully differentiable particle filter algorithm that updates the PF-RNN latent state distribution according to the Bayes rule. Experiments demonstrate that the proposed PF-RNNs outperform the corresponding standard gated RNNs on a synthetic robot localization dataset and 10 real-world sequence prediction datasets for text classification, stock price prediction, etc.