Towards a Systems Theory of Algorithms
This is an incremental opinion piece that aims to reframe the perspective on algorithms for researchers in computational fields, but it lacks concrete problem-solving or applications.
The paper proposes viewing algorithms as open dynamical systems interacting with their environment, advocating for a systems theory of algorithms to address challenges in control, learning, and optimization. It surveys existing instances and outlines modeling, analysis, and design challenges without presenting specific results or numbers.
Traditionally, numerical algorithms are seen as isolated pieces of code confined to an {\em in silico} existence. However, this perspective is not appropriate for many modern computational approaches in control, learning, or optimization, wherein {\em in vivo} algorithms interact with their environment. Examples of such {\em open algorithms} include various real-time optimization-based control strategies, reinforcement learning, decision-making architectures, online optimization, and many more. Further, even {\em closed} algorithms in learning or optimization are increasingly abstracted in block diagrams with interacting dynamic modules and pipelines. In this opinion paper, we state our vision on a to-be-cultivated {\em systems theory of algorithms} and argue in favor of viewing algorithms as open dynamical systems interacting with other algorithms, physical systems, humans, or databases. Remarkably, the manifold tools developed under the umbrella of systems theory are well suited for addressing a range of challenges in the algorithmic domain. We survey various instances where the principles of algorithmic systems theory are being developed and outline pertinent modeling, analysis, and design challenges.