Dropping diversity of products of large US firms: Models and measures
This addresses a key economic problem for researchers and policymakers by providing evidence against the common belief of rising product diversity, though it is incremental as it builds on existing firm-similarity models.
The study tackled the assumption of increasing product diversity by analyzing textual descriptions of large US firms' products from 1997 to 2017, finding a steady decline in diversity using various metrics, including firm-similarity measurements.
It is widely assumed that in our lifetimes the products available in the global economy have become more diverse. This assumption is difficult to investigate directly, however, because it is difficult to collect the necessary data about every product in an economy each year. We solve this problem by mining publicly available textual descriptions of the products of every large US firms each year from 1997 to 2017. Although many aspects of economic productivity have been steadily rising during this period, our text-based measurements show that the diversity of the products of at least large US firms has steadily declined. This downward trend is visible using a variety of product diversity metrics, including some that depend on a measurement of the similarity of the products of every single pair of firms. The current state of the art in comprehensive and detailed firm-similarity measurements is a Boolean word vector model due to Hoberg and Phillips. We measure diversity using firm-similarities from this Boolean model and two more sophisticated variants, and we consistently observe a significant dropping trend in product diversity. These results make it possible to frame and start to test specific hypotheses for explaining the dropping product diversity trend.