Transductive Optimization of Top k Precision
This addresses a specific problem in information retrieval, digital advertising, and reserve design, but it is incremental as it builds on existing transductive learning approaches.
The paper tackles the transductive precision@k problem, where a model must select exactly k test points as positive predictions, and introduces the Transductive Top K (TTK) method that minimizes hinge loss under this constraint, achieving performance that matches or exceeds state-of-the-art methods on 7 UCI datasets and 3 reserve design instances.
Consider a binary classification problem in which the learner is given a labeled training set, an unlabeled test set, and is restricted to choosing exactly $k$ test points to output as positive predictions. Problems of this kind---{\it transductive precision@$k$}---arise in information retrieval, digital advertising, and reserve design for endangered species. Previous methods separate the training of the model from its use in scoring the test points. This paper introduces a new approach, Transductive Top K (TTK), that seeks to minimize the hinge loss over all training instances under the constraint that exactly $k$ test instances are predicted as positive. The paper presents two optimization methods for this challenging problem. Experiments and analysis confirm the importance of incorporating the knowledge of $k$ into the learning process. Experimental evaluations of the TTK approach show that the performance of TTK matches or exceeds existing state-of-the-art methods on 7 UCI datasets and 3 reserve design problem instances.