Feature Interactions on Steroids: On the Composition of ML Models
This addresses the problem of building reliable and scalable ML systems for software engineers and ML practitioners, but it is incremental as it reframes known issues rather than introducing new solutions.
The paper tackles the challenge of composing machine learning models due to the lack of formal specifications, arguing that this issue is not fundamentally new but resembles existing software engineering problems like feature interactions, and suggests rethinking composition strategies to improve reuse, testing, and debugging.
The lack of specifications is a key difference between traditional software engineering and machine learning. We discuss how it drastically impacts how we think about divide-and-conquer approaches to system design, and how it impacts reuse, testing and debugging activities. Traditionally, specifications provide a cornerstone for compositional reasoning and for the divide-and-conquer strategy of how we build large and complex systems from components, but those are hard to come by for machine-learned components. While the lack of specification seems like a fundamental new problem at first sight, in fact software engineers routinely deal with iffy specifications in practice: we face weak specifications, wrong specifications, and unanticipated interactions among components and their specifications. Machine learning may push us further, but the problems are not fundamentally new. Rethinking machine-learning model composition from the perspective of the feature interaction problem, we may even teach us a thing or two on how to move forward, including the importance of integration testing, of requirements engineering, and of design.