Your AI, Not Your View: The Bias of LLMs in Investment Analysis
This addresses the risk of unreliable recommendations in real-world investment services, highlighting an underexplored issue with incremental insights into model-specific biases.
The paper tackled the problem of biases in Large Language Models (LLMs) during investment analysis due to knowledge conflicts between pre-trained data and real-time market information, revealing that models tend to prefer technology stocks, large-cap stocks, and contrarian strategies, with these biases often escalating into confirmation bias.
In finance, Large Language Models (LLMs) face frequent knowledge conflicts arising from discrepancies between their pre-trained parametric knowledge and real-time market data. These conflicts are especially problematic in real-world investment services, where a model's inherent biases can misalign with institutional objectives, leading to unreliable recommendations. Despite this risk, the intrinsic investment biases of LLMs remain underexplored. We propose an experimental framework to investigate emergent behaviors in such conflict scenarios, offering a quantitative analysis of bias in LLM-based investment analysis. Using hypothetical scenarios with balanced and imbalanced arguments, we extract the latent biases of models and measure their persistence. Our analysis, centered on sector, size, and momentum, reveals distinct, model-specific biases. Across most models, a tendency to prefer technology stocks, large-cap stocks, and contrarian strategies is observed. These foundational biases often escalate into confirmation bias, causing models to cling to initial judgments even when faced with increasing counter-evidence. A public leaderboard benchmarking bias across a broader set of models is available at https://linqalpha.com/leaderboard