Learnable Strategies for Bilateral Agent Negotiation over Multiple Issues
This work addresses negotiation for self-interested agents in multi-issue scenarios, offering a novel approach to improve outcomes in automated negotiation systems.
The paper tackles the problem of bilateral negotiation over multiple issues with user preference uncertainty by developing a model that learns interpretable strategy templates and uses deep reinforcement learning to optimize utilities, resulting in outperforming the winning agents of ANAC'19 in individual and social-welfare utilities.
We present a novel bilateral negotiation model that allows a self-interested agent to learn how to negotiate over multiple issues in the presence of user preference uncertainty. The model relies upon interpretable strategy templates representing the tactics the agent should employ during the negotiation and learns template parameters to maximize the average utility received over multiple negotiations, thus resulting in optimal bid acceptance and generation. Our model also uses deep reinforcement learning to evaluate threshold utility values, for those tactics that require them, thereby deriving optimal utilities for every environment state. To handle user preference uncertainty, the model relies on a stochastic search to find user model that best agrees with a given partial preference profile. Multi-objective optimization and multi-criteria decision-making methods are applied at negotiation time to generate Pareto-optimal outcomes thereby increasing the number of successful (win-win) negotiations. Rigorous experimental evaluations show that the agent employing our model outperforms the winning agents of the 10th Automated Negotiating Agents Competition (ANAC'19) in terms of individual as well as social-welfare utilities.