FeatureBox: Feature Engineering on GPUs for Massive-Scale Ads Systems
This work addresses efficiency issues for machine learning practitioners in industrial ads systems, offering an incremental improvement over existing methods.
The paper tackles the bottleneck of feature extraction in large-scale ads CTR model training by proposing FeatureBox, an end-to-end GPU-based framework that pipelines extraction and training, reducing intermediate I/O and achieving significant speedups in real-world applications.
Deep learning has been widely deployed for online ads systems to predict Click-Through Rate (CTR). Machine learning researchers and practitioners frequently retrain CTR models to test their new extracted features. However, the CTR model training often relies on a large number of raw input data logs. Hence, the feature extraction can take a significant proportion of the training time for an industrial-level CTR model. In this paper, we propose FeatureBox, a novel end-to-end training framework that pipelines the feature extraction and the training on GPU servers to save the intermediate I/O of the feature extraction. We rewrite computation-intensive feature extraction operators as GPU operators and leave the memory-intensive operator on CPUs. We introduce a layer-wise operator scheduling algorithm to schedule these heterogeneous operators. We present a light-weight GPU memory management algorithm that supports dynamic GPU memory allocation with minimal overhead. We experimentally evaluate FeatureBox and compare it with the previous in-production feature extraction framework on two real-world ads applications. The results confirm the effectiveness of our proposed method.