A Comparison of Document Similarity Algorithms
This work addresses the problem of identifying effective document similarity algorithms for NLP applications like plagiarism detection and text summarization, but appears incremental as it compares existing methods.
This paper compared various document similarity algorithms by categorizing them into statistical, neural network, and corpus/knowledge-based types, and evaluated the most effective ones in each category using benchmark datasets to determine overall usefulness.
Document similarity is an important part of Natural Language Processing and is most commonly used for plagiarism-detection and text summarization. Thus, finding the overall most effective document similarity algorithm could have a major positive impact on the field of Natural Language Processing. This report sets out to examine the numerous document similarity algorithms, and determine which ones are the most useful. It addresses the most effective document similarity algorithm by categorizing them into 3 types of document similarity algorithms: statistical algorithms, neural networks, and corpus/knowledge-based algorithms. The most effective algorithms in each category are also compared in our work using a series of benchmark datasets and evaluations that test every possible area that each algorithm could be used in.