Estimate and Replace: A Novel Approach to Integrating Deep Neural Networks with Existing Applications
This addresses the challenge of leveraging existing application knowledge in deep learning for developers, though it appears incremental as it builds on hybrid integration techniques.
The paper tackles the problem of integrating deep neural networks with existing applications by proposing an 'Estimate and Replace' method, which uses an estimator to mimic application functionality during training and replaces it with the actual application at inference, resulting in improved performance with less data compared to a baseline DNN.
Existing applications include a huge amount of knowledge that is out of reach for deep neural networks. This paper presents a novel approach for integrating calls to existing applications into deep learning architectures. Using this approach, we estimate each application's functionality with an estimator, which is implemented as a deep neural network (DNN). The estimator is then embedded into a base network that we direct into complying with the application's interface during an end-to-end optimization process. At inference time, we replace each estimator with its existing application counterpart and let the base network solve the task by interacting with the existing application. Using this 'Estimate and Replace' method, we were able to train a DNN end-to-end with less data and outperformed a matching DNN that did not interact with the external application.