SSE: Multimodal Semantic Data Selection and Enrichment for Industrial-scale Data Assimilation
This addresses data overload in industrial applications like autonomous vehicles, offering an incremental improvement in data efficiency.
The paper tackles the problem of unmanageable data growth in industrial AI by proposing a framework to select semantically diverse and important data portions and enrich them, maintaining model performance with smaller datasets and improving it without exceeding original size.
In recent years, the data collected for artificial intelligence has grown to an unmanageable amount. Particularly within industrial applications, such as autonomous vehicles, model training computation budgets are being exceeded while model performance is saturating -- and yet more data continues to pour in. To navigate the flood of data, we propose a framework to select the most semantically diverse and important dataset portion. Then, we further semantically enrich it by discovering meaningful new data from a massive unlabeled data pool. Importantly, we can provide explainability by leveraging foundation models to generate semantics for every data point. We quantitatively show that our Semantic Selection and Enrichment framework (SSE) can a) successfully maintain model performance with a smaller training dataset and b) improve model performance by enriching the smaller dataset without exceeding the original dataset size. Consequently, we demonstrate that semantic diversity is imperative for optimal data selection and model performance.