Holarchic Structures for Decentralized Deep Learning - A Performance Analysis
This addresses performance degradation in decentralized deep learning for IoT applications like smart grids and cities, presenting an incremental improvement over existing methods.
The paper tackles the challenge of self-adaptation in decentralized deep learning for IoT systems, where network uncertainties degrade performance, and demonstrates through 864,000 experiments that holarchic structures are cost-effective in mitigating these issues.
Structure plays a key role in learning performance. In centralized computational systems, hyperparameter optimization and regularization techniques such as dropout are computational means to enhance learning performance by adjusting the deep hierarchical structure. However, in decentralized deep learning by the Internet of Things, the structure is an actual network of autonomous interconnected devices such as smart phones that interact via complex network protocols. Self-adaptation of the learning structure is a challenge. Uncertainties such as network latency, node and link failures or even bottlenecks by limited processing capacity and energy availability can signif- icantly downgrade learning performance. Network self-organization and self-management is complex, while it requires additional computational and network resources that hinder the feasibility of decentralized deep learning. In contrast, this paper introduces a self-adaptive learning approach based on holarchic learning structures for exploring, mitigating and boosting learning performance in distributed environments with uncertainties. A large-scale performance analysis with 864000 experiments fed with synthetic and real-world data from smart grid and smart city pilot projects confirm the cost-effectiveness of holarchic structures for decentralized deep learning.