DIFER: Differentiable Automated Feature Engineering
This addresses the problem of high computational costs in AutoFE for machine learning practitioners, offering a more efficient solution, though it appears incremental as it builds on existing AutoFE approaches.
The paper tackles the computational inefficiency of automated feature engineering (AutoFE) by proposing DIFER, a differentiable gradient-based method that operates in a continuous vector space, which significantly improves performance and efficiency over state-of-the-art AutoFE methods in experiments on classification and regression datasets.
Feature engineering, a crucial step of machine learning, aims to extract useful features from raw data to improve data quality. In recent years, great efforts have been devoted to Automated Feature Engineering (AutoFE) to replace expensive human labor. However, existing methods are computationally demanding due to treating AutoFE as a coarse-grained black-box optimization problem over a discrete space. In this work, we propose an efficient gradient-based method called DIFER to perform differentiable automated feature engineering in a continuous vector space. DIFER selects potential features based on evolutionary algorithm and leverages an encoder-predictor-decoder controller to optimize existing features. We map features into the continuous vector space via the encoder, optimize the embedding along the gradient direction induced by the predicted score, and recover better features from the optimized embedding by the decoder. Extensive experiments on classification and regression datasets demonstrate that DIFER can significantly improve the performance of various machine learning algorithms and outperform current state-of-the-art AutoFE methods in terms of both efficiency and performance.