Ranking Companion: A Visual Analytics Approach to Item-Based Ranking with Hybrid Item Selection
This work addresses the challenge of flexible item selection for non-expert users in personalized ranking creation, but the contribution is incremental as it primarily combines existing methods.
Ranking Companion introduces a visual analytics approach that combines model-driven active learning with human-driven item-selection methods for item-based ranking, enabling users to externalize preferences without attribute knowledge. A user study with 10 participants showed tradeoffs in ranking quality across accuracy, diversity, novelty, transparency, control, and satisfaction.
Personalizing item ranking creation is a challenging task, especially when users lack knowledge of data attributes or the ability to express and formalize their attribute preferences. Item-based ranking creation is an approach allowing users to directly externalize preferences through known-item judgments rather than attribute-based scoring. However, a core challenge of item-based ranking is identifying and selecting representative candidate items for externalizing preferences. Existing approaches rely on singular item-selection methods, limiting flexibility and user control. To address this challenge, we present Ranking Companion, a visual analytics approach for item-based ranking that combines model-driven active learning with human-driven item-selection methods. By drawing from six complementary item-selection methods, users can externalize listwise preferences based on selected candidate items, while an iterative machine learning process with a ranking model calculates ranking results, presented to users alongside explanations for interpretation. We evaluated Ranking Companion in a formative user study with 10 participants, in which participants used each item-selection method across three iterations, revealing tradeoffs in perceived ranking quality across accuracy, diversity, novelty, transparency, control, and satisfaction. Ranking Companion contributes a unified interactive item selection space and provides preliminary empirical guidance toward the hybrid use of multiple complementary item-selection methods in personalized item-based ranking creation.