Dynamic Adaptation in Data Storage: Real-Time Machine Learning for Enhanced Prefetching
It addresses the challenge of managing exponential data growth for storage systems, though it appears incremental by adapting existing streaming methods to a specific domain.
This study tackled the problem of data prefetching in multi-tiered storage systems by applying streaming machine learning to predict file access patterns, achieving substantial improvements in prediction accuracy, memory efficiency, and adaptability.
The exponential growth of data storage demands has necessitated the evolution of hierarchical storage management strategies [1]. This study explores the application of streaming machine learning [3] to revolutionize data prefetching within multi-tiered storage systems. Unlike traditional batch-trained models, streaming machine learning [5] offers adaptability, real-time insights, and computational efficiency, responding dynamically to workload variations. This work designs and validates an innovative framework that integrates streaming classification models for predicting file access patterns, specifically the next file offset. Leveraging comprehensive feature engineering and real-time evaluation over extensive production traces, the proposed methodology achieves substantial improvements in prediction accuracy, memory efficiency, and system adaptability. The results underscore the potential of streaming models in real-time storage management, setting a precedent for advanced caching and tiering strategies.