On Generality and Knowledge Transferability in Cross-Domain Duplicate Question Detection for Heterogeneous Community Question Answering
This work addresses duplicate question detection for community question answering platforms, but it is incremental as it builds on existing methods without introducing a new paradigm.
This study tackled duplicate question detection in community question answering by comparing deep neural networks and gradient tree boosting, and exploring domain adaptation with transfer learning across three heterogeneous datasets, finding that the concept of a 'duplicate' is domain-dependent, which reduces transfer learning effectiveness.
Duplicate question detection is an ongoing challenge in community question answering because semantically equivalent questions can have significantly different words and structures. In addition, the identification of duplicate questions can reduce the resources required for retrieval, when the same questions are not repeated. This study compares the performance of deep neural networks and gradient tree boosting, and explores the possibility of domain adaptation with transfer learning to improve the under-performing target domains for the text-pair duplicates classification task, using three heterogeneous datasets: general-purpose Quora, technical Ask Ubuntu, and academic English Stack Exchange. Ultimately, our study exposes the alternative hypothesis that the meaning of a "duplicate" is not inherently general-purpose, but rather is dependent on the domain of learning, hence reducing the chance of transfer learning through adapting to the domain.