Clicks Versus Conversion: Choosing a Recommender's Training Objective in E-Commerce
This addresses the practical challenge for e-commerce platforms in selecting effective recommendation objectives to maximize business value, though it is incremental as it compares existing metrics rather than introducing a new method.
The paper tackled the problem of choosing training objectives for e-commerce recommenders, comparing click-through rate (CTR) versus conversion metrics like add-to-cart rate (ACR) and order-submit rate (OSR), and found that optimizing for OSR produced a Gross Merchandise Value (GMV) uplift more than five times larger than optimizing for CTR.
Ranking product recommendations to optimize for a high click-through rate (CTR) or for high conversion, such as add-to-cart rate (ACR) and Order-Submit-Rate (OSR, view-to-purchase conversion) are standard practices in e-commerce. Optimizing for CTR appears like a straightforward choice: Training data (i.e., click data) are simple to collect and often available in large quantities. Additionally, CTR is used far beyond e-commerce, making it a generalist, easily implemented option. ACR and OSR, on the other hand, are more directly linked to a shop's business goals, such as the Gross Merchandise Value (GMV). In this paper, we compare the effects of using either of these objectives using an online A/B test. Among our key findings, we demonstrate that in our shops, optimizing for OSR produces a GMV uplift more than five times larger than when optimizing for CTR, without sacrificing new product discovery. Our results also provide insights into the different feature importances for each of the objectives.