A Continual Development Methodology for Large-scale Multitask Dynamic ML Systems
This addresses scalability issues for developers of large-scale multitask ML systems, though it appears incremental as it builds on modular and extensible model concepts.
The paper tackles the inefficiency of traditional ML development by proposing a continual methodology that integrates design iterations into a single extensible system, resulting in a model that jointly solves 124 image classification tasks with state-of-the-art quality and improved size and compute cost.
The traditional Machine Learning (ML) methodology requires to fragment the development and experimental process into disconnected iterations whose feedback is used to guide design or tuning choices. This methodology has multiple efficiency and scalability disadvantages, such as leading to spend significant resources into the creation of multiple trial models that do not contribute to the final solution.The presented work is based on the intuition that defining ML models as modular and extensible artefacts allows to introduce a novel ML development methodology enabling the integration of multiple design and evaluation iterations into the continuous enrichment of a single unbounded intelligent system. We define a novel method for the generation of dynamic multitask ML models as a sequence of extensions and generalizations. We first analyze the capabilities of the proposed method by using the standard ML empirical evaluation methodology. Finally, we propose a novel continuous development methodology that allows to dynamically extend a pre-existing multitask large-scale ML system while analyzing the properties of the proposed method extensions. This results in the generation of an ML model capable of jointly solving 124 image classification tasks achieving state of the art quality with improved size and compute cost.