Energy-Efficient and High-Performance Data Transfers with DRL Agents
This addresses the need for more efficient data transfers in science and industry, offering an incremental improvement over existing methods.
The paper tackles the problem of improving performance and energy efficiency in data transfers on shared networks by developing a dynamic deep reinforcement learning framework that adjusts transfer settings, resulting in up to 25% higher throughput and 40% lower energy usage compared to baselines.
The rapid growth of data across fields of science and industry has increased the need to improve the performance of end-to-end data transfers while using the resources more efficiently. In this paper, we present a dynamic, multiparameter deep reinforcement learning (DRL) framework that adjusts application-layer transfer settings during data transfers on shared networks. Our method strikes a balance between high throughput and low energy utilization by employing reward signals that focus on both energy efficiency and fairness. The DRL agents can pause and resume transfer threads as needed, pausing during heavy network use and resuming when resources are available, to prevent overload and save energy. We evaluate several DRL techniques and compare our solution with state-of-the-art methods by measuring computational overhead, adaptability, throughput, and energy consumption. Our experiments show up to 25% increase in throughput and up to 40% reduction in energy usage at the end systems compared to baseline methods, highlighting a fair and energy-efficient way to optimize data transfers in shared network environments.