Quantifying consensus of rankings based on q-support patterns
This work addresses the need for better consensus assessment in ranking systems, such as group decision making and information retrieval, but it appears incremental as it builds on existing studies by introducing a new method for a known bottleneck.
The paper tackles the problem of evaluating consensus in rankings from multiple agents, proposing a novel approach based on q-support patterns that quantifies consensus without using correlation or distance functions, and demonstrates its effectiveness through experimental studies.
Rankings, representing preferences over a set of candidates, are widely used in many information systems, e.g., group decision making and information retrieval. It is of great importance to evaluate the consensus of the obtained rankings from multiple agents. An overall measure of the consensus degree provides an insight into the ranking data. Moreover, it could provide a quantitative indicator for consensus comparison between groups and further improvement of a ranking system. Existing studies are insufficient in assessing the overall consensus of a ranking set. They did not provide an evaluation of the consensus degree of preference patterns in most rankings. In this paper, a novel consensus quantifying approach, without the need for any correlation or distance functions as in existing studies of consensus, is proposed based on a concept of q-support patterns of rankings. The q-support patterns represent the commonality embedded in a set of rankings. A method for detecting outliers in a set of rankings is naturally derived from the proposed consensus quantifying approach. Experimental studies are conducted to demonstrate the effectiveness of the proposed approach.