Siamese Neural Networks with Random Forest for detecting duplicate question pairs
This work addresses duplicate question detection for platforms like Quora, but it is incremental as it combines existing methods without introducing a new paradigm.
The paper tackled the problem of detecting semantically similar question pairs by using a Siamese Bidirectional GRU combined with a Random Forest classifier, achieving a result in the top 24% on the Quora dataset with about 400k labeled pairs.
Determining whether two given questions are semantically similar is a fairly challenging task given the different structures and forms that the questions can take. In this paper, we use Gated Recurrent Units(GRU) in combination with other highly used machine learning algorithms like Random Forest, Adaboost and SVM for the similarity prediction task on a dataset released by Quora, consisting of about 400k labeled question pairs. We got the best result by using the Siamese adaptation of a Bidirectional GRU with a Random Forest classifier, which landed us among the top 24% in the competition Quora Question Pairs hosted on Kaggle.