Implicit Humanization in Everyday LLM Moral Judgments
For users and developers of conversational AI, this work highlights a previously overlooked form of anthropomorphic projection that can lead to harmful overtrust.
The paper identifies that LLMs reinforce implicit humanization in moral judgment queries, potentially increasing risks like overreliance and misplaced trust. It provides a dataset and analysis of anthropomorphic cues in LLM responses.
Recent adoption of conversational information systems has expanded the scope of user queries to include complex tasks such as personal advice-seeking. However, we identify a specific type of sought advice-a request for a moral judgment (i.e. "who was wrong?") in a social conflict-as an implicitly humanizing query which carries potentially harmful anthropomorphic projections. In this study, we examine the reinforcement of these assumptions in the responses of four major general-purpose LLMs through the use of linguistic, behavioral, and cognitive anthropomorphic cues. We also contribute a novel dataset of simulated user queries for moral judgments. We find current LLM system responses reinforce implicit humanization in queries, potentially exacerbating risks like overreliance or misplaced trust. We call for future work to expand the understanding of anthropomorphism to include implicit userside humanization and to design solutions that address user needs while correcting misaligned expectations of model capabilities.