Neuron-based Multifractal Analysis of Neuron Interaction Dynamics in Large Models
This addresses a gap in systematic quantitative analysis for researchers studying emergent abilities in large models, offering a new perspective, though it appears incremental as it builds on neuroscience-inspired methods.
The paper tackled the problem of analyzing internal structures and mechanisms driving emergent abilities in large models by proposing Neuron-based Multifractal Analysis (NeuroMFA), a framework that links structural features to capabilities and provides a quantitative measure of network heterogeneity and organization.
In recent years, there has been increasing attention on the capabilities of large models, particularly in handling complex tasks that small-scale models are unable to perform. Notably, large language models (LLMs) have demonstrated ``intelligent'' abilities such as complex reasoning and abstract language comprehension, reflecting cognitive-like behaviors. However, current research on emergent abilities in large models predominantly focuses on the relationship between model performance and size, leaving a significant gap in the systematic quantitative analysis of the internal structures and mechanisms driving these emergent abilities. Drawing inspiration from neuroscience research on brain network structure and self-organization, we propose (i) a general network representation of large models, (ii) a new analytical framework, called Neuron-based Multifractal Analysis (NeuroMFA), for structural analysis, and (iii) a novel structure-based metric as a proxy for emergent abilities of large models. By linking structural features to the capabilities of large models, NeuroMFA provides a quantitative framework for analyzing emergent phenomena in large models. Our experiments show that the proposed method yields a comprehensive measure of network's evolving heterogeneity and organization, offering theoretical foundations and a new perspective for investigating emergent abilities in large models.