Learnability with Time-Sharing Computational Resource Concerns
This work addresses the gap between theoretical assumptions of infinite resources and practical limitations in machine learning, particularly for stream learning and supercomputing, though it appears incremental as it extends existing learning theory with resource considerations.
The authors tackled the problem of limited computational resources in machine learning by introducing a theoretical framework for Computational Resource Efficient Learning (CoRE-Learning), which defines machine learning throughput and accounts for resource constraints in learning theory, applicable to stream learning and intelligent supercomputing systems.
Conventional theoretical machine learning studies generally assume explicitly or implicitly that there are enough or even infinitely supplied computational resources. In real practice, however, computational resources are usually limited, and the performance of machine learning depends not only on how many data have been received, but also on how many data can be handled subject to computational resources available. Note that most current ``intelligent supercomputing'' facilities work like exclusive operating systems, where a fixed amount of resources are allocated to a machine learning task without adaptive scheduling strategies considering important factors such as the learning performance demands and learning process status. In this article, we introduce the notion of machine learning throughput, define Computational Resource Efficient Learning (CoRE-Learning), and present a theoretical framework that takes into account the influence of computational resources in learning theory. This framework can be naturally applied to stream learning where the incoming data streams can be potentially endless with overwhelming size and it is impractical to assume that all received data can be handled in time. It may also provide a theoretical perspective for the design of intelligent supercomputing operating systems.