Building AI Agents to Improve Job Referral Requests to Strangers
This work addresses job seekers' challenges in securing referrals through online communities, but it is incremental as it builds on existing LLM and RAG methods.
The paper tackled the problem of job seekers writing effective referral requests by developing AI agents that rewrite and evaluate requests, resulting in a 14% increase in predicted success rates for weaker requests without degrading performance on stronger ones.
This paper develops AI agents that help job seekers write effective requests for job referrals in a professional online community. The basic workflow consists of an improver agent that rewrites the referral request and an evaluator agent that measures the quality of revisions using a model trained to predict the probability of receiving referrals from other users. Revisions suggested by the LLM (large language model) increase predicted success rates for weaker requests while reducing them for stronger requests. Enhancing the LLM with Retrieval-Augmented Generation (RAG) prevents edits that worsen stronger requests while it amplifies improvements for weaker requests. Overall, using LLM revisions with RAG increases the predicted success rate for weaker requests by 14\% without degrading performance on stronger requests. Although improvements in model-predicted success do not guarantee more referrals in the real world, they provide low-cost signals for promising features before running higher-stakes experiments on real users.