Managing ML Pipelines: Feature Stores and the Coming Wave of Embedding Ecosystems
This work tackles the problem of scaling and standardizing ML pipelines for engineers and organizations dealing with embedding-based models, but it is incremental as it builds on existing feature store concepts.
The paper addresses the challenge of managing machine learning pipelines that increasingly rely on self-supervised pretrained embeddings as features, which are not well-supported by existing feature stores designed for tabular data. It introduces a tutorial to discuss these challenges and current solutions for embedding-centric workflows.
The industrial machine learning pipeline requires iterating on model features, training and deploying models, and monitoring deployed models at scale. Feature stores were developed to manage and standardize the engineer's workflow in this end-to-end pipeline, focusing on traditional tabular feature data. In recent years, however, model development has shifted towards using self-supervised pretrained embeddings as model features. Managing these embeddings and the downstream systems that use them introduces new challenges with respect to managing embedding training data, measuring embedding quality, and monitoring downstream models that use embeddings. These challenges are largely unaddressed in standard feature stores. Our goal in this tutorial is to introduce the feature store system and discuss the challenges and current solutions to managing these new embedding-centric pipelines.