AI Empathy Erodes Cognitive Autonomy in Younger Users
This addresses a systemic risk to the developmental autonomy of younger users in human-machine interaction, proposing a novel architectural solution.
The paper tackles the problem of affective alignment in generative AI eroding cognitive autonomy in younger users, finding that emotional mirroring and adult-focused reward models can promote emotional dependency instead of facilitating independent emotional management.
Affective alignment in generative AI represents a systemic risk to the developmental autonomy of younger users. Although emotional mirroring is commonly seen as a hallmark of advanced human-machine interaction, it can also manifest as affective sycophancy, reinforcing a user's immediate emotional state. By providing a sense of objectivity to transient anxieties, these systems diminish the cognitive friction necessary for independent emotional management and critical thought. Reward models driven by RLHF could heighten this dilemma by embedding adult-focused definitions of helpfulness, unintentionally promoting emotional dependency in younger users rather than facilitating cognitive reappraisal. This paper exposes the misalignment between adult-labeled reward signals and the developmental requirements of younger users, proposing stoic architectures that emphasize functional neutrality to preserve user autonomy.