Scaling ML Products At Startups: A Practitioner's Guide
This provides practical guidance for startup practitioners on cost-effective ML scaling, but it is incremental as it builds on existing cost management concepts.
The paper tackles the problem of scaling machine learning products at startups by analyzing costs into variable and fixed categories, proposing a framework to reduce them, with a focus on addressing the high fixed cost of failure diagnosis and continuous improvement.
How do you scale a machine learning product at a startup? In particular, how do you serve a greater volume, velocity, and variety of queries cost-effectively? We break down costs into variable costs-the cost of serving the model and performant-and fixed costs-the cost of developing and training new models. We propose a framework for conceptualizing these costs, breaking them into finer categories, and limn ways to reduce costs. Lastly, since in our experience, the most expensive fixed cost of a machine learning system is the cost of identifying the root causes of failures and driving continuous improvement, we present a way to conceptualize the issues and share our methodology for the same.