A Fairness-Driven Method for Learning Human-Compatible Negotiation Strategies
This addresses the challenge of developing AI negotiation agents that are compatible with human behavior, though it appears incremental by combining existing techniques like RL and language models.
The paper tackled the problem of AI agents learning human-compatible negotiation strategies by proposing a fairness-driven framework that incorporates fairness into reward design and search, resulting in more egalitarian outcomes and improved negotiation quality.
Despite recent advancements in AI and NLP, negotiation remains a difficult domain for AI agents. Traditional game theoretic approaches that have worked well for two-player zero-sum games struggle in the context of negotiation due to their inability to learn human-compatible strategies. On the other hand, approaches that only use human data tend to be domain-specific and lack the theoretical guarantees provided by strategies grounded in game theory. Motivated by the notion of fairness as a criterion for optimality in general sum games, we propose a negotiation framework called FDHC which incorporates fairness into both the reward design and search to learn human-compatible negotiation strategies. Our method includes a novel, RL+search technique called LGM-Zero which leverages a pre-trained language model to retrieve human-compatible offers from large action spaces. Our results show that our method is able to achieve more egalitarian negotiation outcomes and improve negotiation quality.