DexDeepFM: Ensemble Diversity Enhanced Extreme Deep Factorization Machine Model
This work addresses a specific bottleneck in click-through rate prediction models for web applications, representing an incremental improvement over existing methods.
The paper tackles the problem of overfitting and correlated errors in the extreme deep factorization machine model (xDeepFM) for predicting user responses like purchases and clicks by proposing DexDeepFM, which incorporates ensemble diversity measures and an attention mechanism into the objective function, resulting in improved performance on three public datasets.
Predicting user positive response (e.g., purchases and clicks) probability is a critical task in Web applications. To identify predictive features from raw data, the state-of-the-art extreme deep factorization machine model (xDeepFM) introduces a new interaction network to leverage feature interactions at the vector-wise level explicitly. However, since each hidden layer in the interaction network is a collection of feature maps, it can be viewed essentially as an ensemble of different feature maps. In this case, only using a single objective to minimize the prediction loss may lead to overfitting and generate correlated errors. In this paper, an ensemble diversity enhanced extreme deep factorization machine model (DexDeepFM) is proposed, which designs the ensemble diversity measure in each hidden layer and considers both ensemble diversity and prediction accuracy in the objective function. In addition, the attention mechanism is introduced to discriminate the importance of ensemble diversity measures with different feature interaction orders. Extensive experiments on three public real-world datasets are conducted to show the effectiveness of the proposed model.