Verifiable Self-Evolution for Open-Ended Dialogue Skills via Future-Feedback Prediction
For practitioners of dialogue agents, this provides a method to evolve skills offline without live traffic, though it is an incremental step that requires further validation.
The paper tackles the problem of evolving dialogue skills for frozen language models in open-ended conversations, where counterfactual responses cannot be directly evaluated. It proposes future-feedback skill evolution, which predicts whether an observed response will lead to positive or negative user feedback, achieving >75% prediction accuracy on a sales-assistant dataset.
Textual skills provide a lightweight way to improve frozen language-model agents, but their self-evolution normally requires a stable validation signal. Such signals are natural in mathematics or code, where an answer can be checked after it changes, yet are problematic in open-ended dialogue: changing the assistant response also changes the user's next reaction, so a logged reaction cannot directly evaluate a counterfactual response. We propose future-feedback skill evolution, which first redirects self-evolution from prescribing the current answer to predicting whether the observed answer will lead to a positive or negative subsequent user signal. This prediction task is verifiable on fixed logged tuples and therefore supports validation-gated textual optimization. The evolved feedback skill captures interpretable criteria for response quality and can subsequently serve as a diagnostic and optimization target for answer skills. On a proprietary, privacy-preserving sales-assistant dataset, careful quality filtering and a balanced resolved/unresolved split yield more than 75% prediction accuracy. Beyond this result, the central contribution is a formulation that converts otherwise moving conversational feedback into a fixed offline learning target, enabling reproducible skill evolution without placing every candidate skill in live traffic. We discuss the boundary between observational verification and counterfactual validity, and position the method as an offline optimization stage rather than a replacement for final human or online evaluation.