When Summaries Distort Decisions: Information Fidelity in LLM-Compressed Financial Analysis
For financial decision-makers relying on LLM-compressed information, this work identifies a critical failure mode where compression distorts decisions despite factual accuracy, and proposes a mitigation strategy.
LLM-based compression of financial documents can produce fluent, factually plausible summaries that nonetheless alter investment decisions, with fidelity loss occurring through decontextualization and model dependency. The proposed Agentic Context Compression method audits multiple candidate compressions against the original source to preserve decision-relevant context.
Financial decision-makers face more information than they can directly inspect, making context compression necessary. Yet when large language models (LLMs) compress financial source material, they can alter the investment judgment supported by the original source. We frame this problem as information fidelity: compression loses fidelity when it changes the decision induced by the source. In agentic systems, such losses may recur across intermediate steps and amplify throughout the decision process. Across financial filings and earnings-call transcripts, we find that LLM-based compression can produce fluent and factually plausible compressed contexts that nevertheless alter downstream decisions. We analyze two diagnostic patterns associated with fidelity loss: decontextualization, where salient evidence is retained but separated from the caveats and contextual qualifiers needed for correct interpretation, and model dependency, where different compressors expose different views of the same source. We then propose Agentic Context Compression, which generates multiple candidate compressions and audits their disagreements against the original source. Our results suggest that financial compression should be evaluated not only by efficiency or factuality, but also by its ability to preserve decision-relevant context.