Interactor: Agentic RL oriented Iterative Creation for Ad Description Generation in Sponsored Search
For sponsored search systems, this work addresses the challenge of generating informative ad descriptions that incorporate world knowledge and align with landing pages, with demonstrated online impact.
The paper proposes Interactor, a multi-turn iterative creation framework with agentic RL for ad description generation in sponsored search. It significantly outperforms state-of-the-art approaches on industrial datasets and has been deployed online since May 2026, improving ad revenue and user experience.
This paper focuses on automatically generating informative ad descriptions in sponsored search. Unlike ad titles which are usually optimized to attract user click feedbacks, ad descriptions have a longer text span and possess the potential of incorporating world knowledge to address user search intents while presenting the fine-grained selling points of the ads. We propose Interactor, a multi-turn iterative creation framework optimized with agentic RL for ad description generation. The generation model acts as a policy that interacts with a customized environment consisting of multiple generative reward models. Given initial generations by the policy, the customized GenRMs evaluate multi-dimensional qualities including knowledge capacity and landing page consistency, providing both binary signals and reasoning feedbacks. The policy then iteratively refines the descriptions based on such feedbacks to ensure continuous improvement. Experiments on industrial datasets show that the Interactor framework significantly outperforms state-of-the-art approaches in generating knowledge-rich and faithful ad descriptions. Since May 2026, it has been deployed online in a leading search ads system, contributing to both ad revenue and user experience.