BACH: A Bayesian Admixture of Contrastive Heads for Multi-Interest Two-Tower Retrieval
For recommender systems needing to capture diverse user interests, BACH provides a principled soft-routing approach that outperforms hard-routing multi-interest and single-vector methods.
BACH introduces a Bayesian admixture model for multi-interest two-tower retrieval, using variational inference to learn per-user interest weights and mitigate routing collapse. It achieves improved top-of-ranking retrieval over baselines on MovieLens-20M, Taobao, and Netflix benchmarks.
Two-tower retrievers compress each user into a single embedding, limiting their ability to serve diverse interests. Multi-interest models give each user several heads scored by a maximum inner product, but their hard-routing training under-utilizes heads (routing collapse) and gives no per-user estimate of how much each interest matters for serving. We present \textbf{BACH} (\emph{Bayesian Admixture of Contrastive Heads}), which casts multi-interest two-tower retrieval as a per-user mixture over the heads, fit by variational inference. The soft mixture trains every head (mitigating collapse), produces a per-user weighting of the interests that is reused at serving, and admits a shared global-codebook variant with precomputable retrieval. On three large-scale benchmarks, MovieLens-20M, Taobao, and Netflix, BACH improves top-of-ranking retrieval over hard-routing multi-interest and single-vector baselines at every head count; we further find that scoring every candidate by its best head, consistent with serving, outperforms the usual target-routed training, and that BACH improves further still.