Diversifying Multi-aspect Search Results Using Simpson's Diversity Index
This addresses search and recommendation quality for users by improving diversity, though it appears incremental as it builds on existing diversification methods with a new metric.
The paper tackles the problem of diversifying multi-aspect search results to reduce redundancy and promote underrepresented items, by adapting Simpson's Diversity Index from biology to create a more effective and efficient quadratic algorithm. Experimental results on the Kaggle shoes dataset show it outperforms previous state-of-the-art diversification methods while reducing computational complexity.
In search and recommendation, diversifying the multi-aspect search results could help with reducing redundancy, and promoting results that might not be shown otherwise. Many previous methods have been proposed for this task. However, previous methods do not explicitly consider the uniformity of the number of the items' classes, or evenness, which could degrade the search and recommendation quality. To address this problem, we introduce a novel method by adapting the Simpson's Diversity Index from biology, which enables a more effective and efficient quadratic search result diversification algorithm. We also extend the method to balance the diversity between multiple aspects through weighted factors and further improve computational complexity by developing a fast approximation algorithm. We demonstrate the feasibility of the proposed method using the openly available Kaggle shoes competition dataset. Our experimental results show that our approach outperforms previous state of the art diversification methods, while reducing computational complexity.