Capabilities for Better ML Engineering
This work addresses the need for more unified and effective engineering practices in ML to build safer, more generalizable, and trustworthy models, though it appears incremental as it builds on existing efforts.
The paper tackles the problem of scattered and biased engineering support in machine learning by proposing a capability-based framework that uses fine-grained specifications for model behaviors to improve ML engineering. Through preliminary experiments, it shows the framework's potential for reflecting model generalizability, providing guidance for the ML engineering process.
In spite of machine learning's rapid growth, its engineering support is scattered in many forms, and tends to favor certain engineering stages, stakeholders, and evaluation preferences. We envision a capability-based framework, which uses fine-grained specifications for ML model behaviors to unite existing efforts towards better ML engineering. We use concrete scenarios (model design, debugging, and maintenance) to articulate capabilities' broad applications across various different dimensions, and their impact on building safer, more generalizable and more trustworthy models that reflect human needs. Through preliminary experiments, we show capabilities' potential for reflecting model generalizability, which can provide guidance for ML engineering process. We discuss challenges and opportunities for capabilities' integration into ML engineering.