SLIM-LLMs: Modeling of Style-Sensory Language RelationshipsThrough Low-Dimensional Representations
This work addresses the need for efficient and interpretable models in computational linguistics for analyzing sensorial language, though it appears incremental as it builds on existing stylistic features and methods.
The paper tackles the problem of modeling relationships between sensorial language and stylistic features by introducing SLIM-LLMs, which use low-dimensional latent representations to achieve performance matching full-scale language models while reducing parameters by up to 80% across five genres.
Sensorial language -- the language connected to our senses including vision, sound, touch, taste, smell, and interoception, plays a fundamental role in how we communicate experiences and perceptions. We explore the relationship between sensorial language and traditional stylistic features, like those measured by LIWC, using a novel Reduced-Rank Ridge Regression (R4) approach. We demonstrate that low-dimensional latent representations of LIWC features r = 24 effectively capture stylistic information for sensorial language prediction compared to the full feature set (r = 74). We introduce Stylometrically Lean Interpretable Models (SLIM-LLMs), which model non-linear relationships between these style dimensions. Evaluated across five genres, SLIM-LLMs with low-rank LIWC features match the performance of full-scale language models while reducing parameters by up to 80%.