Can Artificial Intelligence Write Like Borges? An Evaluation Protocol for Spanish Microfiction
This addresses the problem of assessing aesthetic qualities in AI-generated stories for researchers and literary communities, though it is incremental as it builds on existing evaluation methods.
The paper tackles the challenge of evaluating AI-generated microfictions for literary merit by introducing GrAImes, an evaluation protocol based on literary theory, and validates it with literature experts and enthusiasts.
Automated story writing has been a subject of study for over 60 years. Large language models can generate narratively consistent and linguistically coherent short fiction texts. Despite these advancements, rigorous assessment of such outputs for literary merit - especially concerning aesthetic qualities - has received scant attention. In this paper, we address the challenge of evaluating AI-generated microfictions and argue that this task requires consideration of literary criteria across various aspects of the text, such as thematic coherence, textual clarity, interpretive depth, and aesthetic quality. To facilitate this, we present GrAImes: an evaluation protocol grounded in literary theory, specifically drawing from a literary perspective, to offer an objective framework for assessing AI-generated microfiction. Furthermore, we report the results of our validation of the evaluation protocol, as answered by both literature experts and literary enthusiasts. This protocol will serve as a foundation for evaluating automatically generated microfictions and assessing their literary value.