CFO: A Framework for Building Production NLP Systems
This work addresses the problem of efficient production deployment for NLP/IR systems, though it appears incremental as it builds on existing BERT-based methods.
The paper tackles the challenge of building and deploying interactive NLP and IR systems in production by introducing the CFO orchestration framework, and demonstrates its effectiveness with a question answering system that achieves high-quality results in both academic and industry settings.
This paper introduces a novel orchestration framework, called CFO (COMPUTATION FLOW ORCHESTRATOR), for building, experimenting with, and deploying interactive NLP (Natural Language Processing) and IR (Information Retrieval) systems to production environments. We then demonstrate a question answering system built using this framework which incorporates state-of-the-art BERT based MRC (Machine Reading Comprehension) with IR components to enable end-to-end answer retrieval. Results from the demo system are shown to be high quality in both academic and industry domain specific settings. Finally, we discuss best practices when (pre-)training BERT based MRC models for production systems.