Exploration of Proximity Heuristics in Length Normalization
This work addresses the need for better ranking functions in search engines and language processing applications, but it appears incremental as it builds on existing methods like BM25 with feature engineering.
The paper tackled the problem of constructing effective ranking functions for information retrieval by proposing guidelines for feature engineering and a specific generalized function for recommendation systems, resulting in a proximity feature-based ranking function that outperformed regular BM25 by 52% on unstructured textual data.
Ranking functions used in information retrieval are primarily used in the search engines and they are often adopted for various language processing applications. However, features used in the construction of ranking functions should be analyzed before applying it on a data set. This paper gives guidelines on construction of generalized ranking functions with application-dependent features. The paper prescribes a specific case of a generalized function for recommendation system using feature engineering guidelines on the given data set. The behavior of both generalized and specific functions are studied and implemented on the unstructured textual data. The proximity feature based ranking function has outperformed by 52% from regular BM25.