New results on inconsistency indices and their relationship with the quality of priority vector estimation
This work addresses a foundational issue in Multi-Criteria Decision Making by clarifying the relationship between inconsistency indices and decision quality, which is incremental but important for practitioners.
The paper tackles the problem of inconsistency in pairwise comparison matrices (PCMs) used for priority vector estimation, showing through Monte Carlo experiments that existing inconsistency indices do not reliably measure the quality of priority vector estimation, and proposes a new inconsistency characteristic and acceptance approach based on statistical methodology.
The article is devoted to the problem of inconsistency in the pairwise comparisons based prioritization methodology. The issue of "inconsistency" in this context has gained much attention in recent years. The literature provides us with a number of different "inconsistency" indices suggested for measuring the inconsistency of the pairwise comparison matrix (PCM). The latter is understood as a deviation of the PCM from the "consistent case" - a notion that is formally well-defined in this theory. However the usage of the indices is justified only by some heuristics. It is still unclear what they really "measure". What is even more important and still not known is the relationship between their values and the "consistency" of the decision maker's judgments on one hand, and the prioritization results upon the other. We provide examples showing that it is necessary to distinguish between these three following tasks: the "measuring" of the "PCM inconsistency" and the PCM-based "measuring" of the consistency of decision maker's judgments and, finally, the "measuring" of the usefulness of the PCM as a source of information for estimation of the priority vector (PV). Next we focus on the third task, which seems to be the most important one in Multi-Criteria Decision Making. With the help of Monte Carlo experiments, we study the performance of various inconsistency indices as indicators of the final PV estimation quality. The presented results allow a deeper understanding of the information contained in these indices and help in choosing a proper one in a given situation. They also enable us to develop a new inconsistency characteristic and, based on it, to propose the PCM acceptance approach that is supported by the classical statistical methodology.