Assessing Post-Reform Changes in Risk Disclosure Quality with a Multidimensional Text Analysis Approach
For regulators and investors, this work provides a more nuanced method to assess disclosure quality changes over time, revealing complex dynamics masked by single-indicator approaches.
The study proposes a multidimensional text analysis approach combining NLP metrics with statistical methods to evaluate changes in risk disclosure quality, applied to Japan's 2019 disclosure reforms. Results show that while disclosure volume increased, readability declined, and overall information structure improved but descriptive quality stagnated, with varying adaptation across market segments.
While corporate narrative disclosures provide crucial information to capital markets, comprehensively evaluating their qualitative changes over time remains challenging. Narrative text is inherently multidimensional, meaning that an improvement in one textual dimension often occurs alongside changes in others. To capture these underlying dynamics, we propose a longitudinal text analysis approach combining Japanese-language NLP metric extraction with paired testing, shift function analysis, and inter-metric correlation. Our framework extends prior indicator sets by incorporating a cross-section relevance indicator to measure topical alignment between risk disclosures and management strategies. Applying this approach to evaluate Japan's 2019 disclosure reforms, we analyze 19,770 firm-year observations over a 10-year period (FY2015-FY2024). The joint analysis reveals complex shifts in disclosure patterns that are frequently masked by conventional single-indicator methods. Specifically, we find that while disclosure volume increased substantially, it was accompanied by a decline in readability. Furthermore, although the overall information structure improved, specific descriptive quality stagnated, and the degree of adaptation varied across market segments.