Making Classical Machine Learning Pipelines Differentiable: A Neural Translation Approach
This addresses the problem of inefficient optimization in multi-model ML pipelines for practitioners, though it is incremental as it builds on existing translation and fine-tuning ideas.
The paper tackles the sub-optimal isolated training of classical ML pipelines by proposing a framework that translates them into neural networks for joint fine-tuning with backpropagation, resulting in increased final accuracy.
Classical Machine Learning (ML) pipelines often comprise of multiple ML models where models, within a pipeline, are trained in isolation. Conversely, when training neural network models, layers composing the neural models are simultaneously trained using backpropagation. We argue that the isolated training scheme of ML pipelines is sub-optimal, since it cannot jointly optimize multiple components. To this end, we propose a framework that translates a pre-trained ML pipeline into a neural network and fine-tunes the ML models within the pipeline jointly using backpropagation. Our experiments show that fine-tuning of the translated pipelines is a promising technique able to increase the final accuracy.