Explaining Fine Tuned LLMs via Counterfactuals A Knowledge Graph Driven Framework
This work addresses the problem of interpretability in fine-tuned LLMs for AI researchers, offering a novel explanation method but is incremental as it builds on existing counterfactual and knowledge graph techniques.
The paper tackles the challenge of understanding how fine-tuning with LoRA alters LLMs' reasoning by introducing a counterfactual framework grounded in knowledge graphs, revealing structural dependencies and aligning with parameter shifts in a fine-tuned LLaMA model.
The widespread adoption of Low-Rank Adaptation (LoRA) has enabled large language models (LLMs) to acquire domain-specific knowledge with remarkable efficiency. However, understanding how such a fine-tuning mechanism alters a model's structural reasoning and semantic behavior remains an open challenge. This work introduces a novel framework that explains fine-tuned LLMs via counterfactuals grounded in knowledge graphs. Specifically, we construct BioToolKG, a domain-specific heterogeneous knowledge graph in bioinformatics tools and design a counterfactual-based fine-tuned LLMs explainer (CFFTLLMExplainer) that learns soft masks over graph nodes and edges to generate minimal structural perturbations that induce maximum semantic divergence. Our method jointly optimizes structural sparsity and semantic divergence while enforcing interpretability preserving constraints such as entropy regularization and edge smoothness. We apply this framework to a fine-tuned LLaMA-based LLM and reveal that counterfactual masking exposes the model's structural dependencies and aligns with LoRA-induced parameter shifts. This work provides new insights into the internal mechanisms of fine-tuned LLMs and highlights counterfactual graphs as a potential tool for interpretable AI.