Federated Unbiased Learning to Rank
This addresses privacy-preserving ranking for on-device search, but it is incremental as it adapts existing unbiased learning to rank methods to federated learning.
The paper tackles the problem of learning ranking functions from biased user interactions in a privacy-constrained, on-device search setting, proposing the FedIPS algorithm that uses federated learning and click propensities to remove position bias, with evaluation on Yahoo and Istella datasets showing robustness across biases.
Unbiased Learning to Rank (ULTR) studies the problem of learning a ranking function based on biased user interactions. In this framework, ULTR algorithms have to rely on a large amount of user data that are collected, stored, and aggregated by central servers. In this paper, we consider an on-device search setting, where users search against their personal corpora on their local devices, and the goal is to learn a ranking function from biased user interactions. Due to privacy constraints, users' queries, personal documents, results lists, and raw interaction data will not leave their devices, and ULTR has to be carried out via Federated Learning (FL). Directly applying existing ULTR algorithms on users' devices could suffer from insufficient training data due to the limited amount of local interactions. To address this problem, we propose the FedIPS algorithm, which learns from user interactions on-device under the coordination of a central server and uses click propensities to remove the position bias in user interactions. Our evaluation of FedIPS on the Yahoo and Istella datasets shows that FedIPS is robust over a range of position biases.