AEFE: Automatic Embedded Feature Engineering for Categorical Features
This addresses the problem of feature engineering for data analysts in e-commerce, offering an incremental improvement with automated and interpretable methods.
The paper tackles the challenge of constructing combinatorial features from categorical data in e-commerce applications like recommendation systems and click-through rate prediction, proposing AEFE, an automatic feature engineering framework that outperforms classical and state-of-the-art deep learning models in experiments on typical datasets.
The challenge of solving data mining problems in e-commerce applications such as recommendation system (RS) and click-through rate (CTR) prediction is how to make inferences by constructing combinatorial features from a large number of categorical features while preserving the interpretability of the method. In this paper, we propose Automatic Embedded Feature Engineering(AEFE), an automatic feature engineering framework for representing categorical features, which consists of various components including custom paradigm feature construction and multiple feature selection. By selecting the potential field pairs intelligently and generating a series of interpretable combinatorial features, our framework can provide a set of unseen generated features for enhancing model performance and then assist data analysts in discovering the feature importance for particular data mining tasks. Furthermore, AEFE is distributed implemented by task-parallelism, data sampling, and searching schema based on Matrix Factorization field combination, to optimize the performance and enhance the efficiency and scalability of the framework. Experiments conducted on some typical e-commerce datasets indicate that our method outperforms the classical machine learning models and state-of-the-art deep learning models.