DHEN: A Deep and Hierarchical Ensemble Network for Large-Scale Click-Through Rate Prediction
This work addresses the challenge of improving model performance and efficiency in online advertising services, representing an incremental advancement in CTR prediction methods.
The paper tackled the problem of varying performance in feature interaction learning for click-through rate prediction by proposing DHEN, a deep and hierarchical ensemble network that leverages heterogeneous modules, resulting in a 0.27% improvement in Normalized Entropy and 1.2x better training throughput compared to state-of-the-art baselines.
Learning feature interactions is important to the model performance of online advertising services. As a result, extensive efforts have been devoted to designing effective architectures to learn feature interactions. However, we observe that the practical performance of those designs can vary from dataset to dataset, even when the order of interactions claimed to be captured is the same. That indicates different designs may have different advantages and the interactions captured by them have non-overlapping information. Motivated by this observation, we propose DHEN - a deep and hierarchical ensemble architecture that can leverage strengths of heterogeneous interaction modules and learn a hierarchy of the interactions under different orders. To overcome the challenge brought by DHEN's deeper and multi-layer structure in training, we propose a novel co-designed training system that can further improve the training efficiency of DHEN. Experiments of DHEN on large-scale dataset from CTR prediction tasks attained 0.27\% improvement on the Normalized Entropy (NE) of prediction and 1.2x better training throughput than state-of-the-art baseline, demonstrating their effectiveness in practice.