Controlling Authority Retrieval: A Missing Retrieval Objective for Authority-Governed Knowledge
For practitioners in law, pharmaceuticals, and software security, this work addresses the critical problem of retrieving the currently active authority frontier, which is not captured by standard relevance-based retrieval.
The paper introduces a new retrieval objective, Controlling Authority Retrieval (CAR), for authority-governed knowledge domains where newer authorities can override older ones. The proposed two-stage method achieves TCA@5 of 0.975 on security advisories, 0.926 on SCOTUS overruling pairs, and 0.774 on FDA drug records, significantly outperforming dense retrieval baselines.
In law, regulatory regimes for pharmaceuticals and software security, newer authorities can revoke older established ones even when semantically distant. We call this CAR: retrieving the currently active authority frontier for a semantic anchor q, that is, front(cl(A_k(q))). This differs from finding the most similar document by relevance score: argmax_d s(q, d). Theorem 4 characterizes when a set R truly covers the active authority set for q with TCA(R, q)=1, providing conditions necessary and sufficient for any retrieved set R: frontier inclusion (front(cl(A_k(q))) contained in R) and no-ignored-superseder (no superseding document exists in the corpus outside R). Proposition 2 shows that TCA@k <= phi(q) * R_anchor(q) in the worst case over any scope-indexed algorithm, proved by an adversarial permutation argument. We evaluated on three real-world datasets: security advisories (Dense TCA@5=0.270, two-stage 0.975), SCOTUS overruling pairs (Dense TCA=0.172, two-stage 0.926), and FDA drug records (Dense TCA=0.064, two-stage 0.774). A GPT-4o-mini experiment shows Dense RAG produces explicit "not patched" claims for 39% of queries where a patch exists; two-stage cuts this to 16%. Four benchmark datasets, domain adapters, and a single-command scorer are released at https://github.com/andremir/car-retrieval.