Accelerating Divisible Load Processing Through Machine Learning: A Practical Framework for Large-Scale Workloads
This work provides a practical acceleration for distributed computing optimization, enabling real-time scheduling and resource allocation in large-scale workloads.
The paper introduces a machine learning framework using a feedforward neural network to predict optimal processing times in Single-Level Tree Network architectures for Divisible Load Theory, achieving 97-99% accuracy and 10-100x speedup over traditional methods.
In this paper, we introduce the first machine learning framework for predicting optimal processing times in Single-Level Tree Network (SLTN) architectures for the Divisible Load Theory (DLT) paradigm. Using a feedforward neural network(FNN) with 16 engineered features, we train a model on 100,000 synthetically generated configurations to predict optimal processing times without explicit formulation of DLT equations. The model achieves 97-99% accuracy (R-square factor) with mean absolute percentage error of 1-5%, demonstrating that neural networks can effectively learn complex load distribution relationships. Feature importance analysis reveals that the model implicitly captures DLT mathematical structure, including load conservation and simultaneous finishing constraints. With inference times under 1 millisecond, the approach provides 10-100x speedup over traditional DLT computation, enabling applications in real-time scheduling, design space exploration, and cloud resource allocation. The method generalizes well across diverse system configurations (n=3 to 20, load size =1 to 100 GB) with consistent accuracy, though performance degrades slightly for very large or highly heterogeneous systems. This work demonstrates the feasibility of using machine learning to accelerate distributed computing optimization while maintaining near-optimal accuracy.