The disruption index does not measure scientific innovation
For researchers and policymakers relying on bibliometric indices, this paper warns against using the disruption index as a valid measure of innovation, though the critique is incremental as it builds on existing concerns about such indices.
The paper critiques the disruption index used in a recent Science article, arguing that it does not measure scientific innovation and that its use for policy recommendations is flawed. The authors highlight general issues with bibliometric indices that are treated as black-boxed measures of quality.
A paper recently published in Science under the rubric of Policy Article argued that what the authors call scientific disruption declines with academic age, and that this decline is related to the absence of mandatory retirement for older academics. Since its publication, its conclusions and policy suggestions in relation to mandatory retirement have received considerable media attention. Thus, it is worth taking a closer look at the proposed measure of disruption since all the analysis and conclusions are based on the results obtained from this index, thus taking it as valid. The issues we address are not specific to this article and can be found in many papers using bibliometric data that propose a new index on the basis of common sense intuition and then using it as a black boxed instrument to measure quality, innovation or, now, disruption for creating rankings and formulate policy actions on the basis of the calculated values of the index.