The Machine Learning Canvas: Empirical Findings on Why Strategy Matters More Than AI Code Generation
It addresses the problem of project failure for data scientists and organizations, offering a practical framework, but it is incremental as it builds on existing business and engineering concepts.
This study tackled the high failure rate of machine learning projects by developing and testing a Machine Learning Canvas framework, identifying four interconnected success factors (Strategy, Process, Ecosystem, Support) with statistical significance, such as organizational support improving strategy (β=0.432, p<0.001).
Despite the growing popularity of AI coding assistants, over 80% of machine learning (ML) projects fail to deliver real business value. This study creates and tests a Machine Learning Canvas, a practical framework that combines business strategy, software engineering, and data science in order to determine the factors that lead to the success of ML projects. We surveyed 150 data scientists and analyzed their responses using statistical modeling. We identified four key success factors: Strategy (clear goals and planning), Process (how work gets done), Ecosystem (tools and infrastructure), and Support (organizational backing and resources). Our results show that these factors are interconnected - each one affects the next. For instance, strong organizational support results in a clearer strategy (β= 0.432, p < 0.001), which improves work processes (β= 0.428, p < 0.001) and builds better infrastructure (β= 0.547, p < 0.001). Together, these elements determine whether a project succeeds. The surprising finding? Although AI assistants make coding faster, they don't guarantee project success. AI assists with the "how" of coding but cannot replace the "why" and "what" of strategic thinking.