Hidden State Approximation in Recurrent Neural Networks Using Continuous Particle Filtering
This addresses a limitation in RNNs for sequential data prediction, but it appears incremental as it builds on existing particle filtering and encoder-decoder approaches.
The paper tackled the problem of deterministic hidden states in recurrent neural networks by using particles to approximate latent state distributions, extending it to an encoder-decoder mechanism with a continuous differentiable scheme, and demonstrated effectiveness in prediction tasks.
Using historical data to predict future events has many applications in the real world, such as stock price prediction; the robot localization. In the past decades, the Convolutional long short-term memory (LSTM) networks have achieved extraordinary success with sequential data in the related field. However, traditional recurrent neural networks (RNNs) keep the hidden states in a deterministic way. In this paper, we use the particles to approximate the distribution of the latent state and show how it can extend into a more complex form, i.e., the Encoder-Decoder mechanism. With the proposed continuous differentiable scheme, our model is capable of adaptively extracting valuable information and updating the latent state according to the Bayes rule. Our empirical studies demonstrate the effectiveness of our method in the prediction tasks.