Tracking the Behavioral Trajectories of Adapting Agents
This work provides a way to quantify behavioral changes in adapting agents, which is important for monitoring and controlling agent behavior in autonomous systems.
The authors propose a method to measure agent traits by defining them as directions in text embedding space, achieving 91.2% accuracy and 0.82 Spearman correlation on 68 labeled skill diff pairs for detecting propensity to seek sensitive data.
Text files such as skill files, memory files, and behavioral configuration files play a central role in defining how modern agents act. Through edits by humans or the agents themselves, these files may evolve over time, directly steering the agent's behavior in future interactions. We present a methodology and framework for measuring agent $traits$ by defining traits as directions in the embedding space of a text embedding model. We train a linear model on labeled "before" versus "after" skill file diffs to learn a trait vector, then score arbitrary skill edits by projecting their embedding diffs onto this vector. Evaluated on 68 labeled skill diff pairs for the trait of propensity to seek sensitive data, our method achieves 91.2% sign classification accuracy and a Spearman rank correlation of $ρ= 0.82$ under leave-one-out cross-validation. We build this trait evaluation into a broader agent-to-agent protocol that enables one agent to evaluate another's skill file updates through a trusted intermediary.