Dual-Agent Co-Training for Health Coaching via Implicit Adversarial Preference Optimization
For developers of AI health coaching systems, this work addresses the inefficiency of one-sided training by introducing a co-training approach, but the improvement is incremental over existing methods.
The paper tackles the scarcity of human health coaches by proposing a dual-agent framework that interactively co-trains both the coach agent and client simulator, using implicit adversarial preference optimization. The method improves coaching quality across several dimensions, though no concrete numbers are provided.
Motivational-interviewing-based health coaching is an effective approach for improving mental health and promoting healthy behavior change. However, the scarcity of trained human coaches and the high cost of coaching services make such support inaccessible to many people who could benefit from it. This motivates the development of AI health coaches that can provide scalable and affordable support. Existing methods typically optimize only one side of the interaction: they either train a dialogue agent against a fixed client environment or train a client simulator against a fixed assistant. This one-sided setup can limit exploration of the interaction space and may be inefficient at developing the capabilities required by the target agent and pushing its performance boundaries. In this paper, we propose a dual-agent framework that interactively co-trains both the health coach agent and the client simulator. The coach is optimized with DPO using Pareto-dominant response pairs identified by a multi-dimensional LLM judge. In turn, the client is trained adversarially by reversing these preferences, inducing an implicit adversarial training dynamic. We further show that this co-training process admits a natural stochastic-game interpretation. Extensive experiments demonstrate that our method effectively improves coaching quality across several important dimensions.