Heiko Hamann

Pranav Kedia, Aaron Gan, Hannah J. Williams et al.

Collective behavior models, such as aggregation and flocking, usually assume self-propelled robots that can directly execute their desired speed and direction of motion without fundamental constraints. However, autonomous sailing robots violate this assumption. Their motion is shaped by wind-dependent propulsion, restricted headings, and spatially varying wind conditions. In particular, maneuverability is coupled to wind speed: in weak wind, sailboats may turn only slowly or not at all, whereas stronger wind enables faster turns. This introduces transient heterogeneity in speed and maneuverability across the flock. We focus on this fast-slow coordination problem in sailing robot flocks. To study this problem, we introduce SailSwarmSwIM, a reduced-order simulator for autonomous sailing robot swarms that captures wind-dependent speed and maneuverability, no-go zones, tacking behavior, and steady or gusty wind fields. To design our novel flocking technique, we start from the Couzin model and introduce a speed-weighted social interaction rule that accounts for each robot's transient motion constraints. A key result is that increasing the social influence of slower robots improves polarization and reduces close encounters. This effect arises from a balance between attraction to fast neighbors, which helps maintain movement, and cohesion around slow neighbors, which prevents the flock from fragmenting. Together, our simulator, SailSwarmSwIM, and the speed-weighted interaction rule provide a modeling framework for studying adaptive collective behavior in robotic fleets whose motion capabilities are continuously shaped by wind.

Mohsen Raoufi, Pawel Romanczuk, Heiko Hamann

Collective decision-making is an essential capability of large-scale multi-robot systems to establish autonomy on the swarm level. A large portion of literature on collective decision-making in swarm robotics focuses on discrete decisions selecting from a limited number of options. Here we assign a decentralized robot system with the task of exploring an unbounded environment, finding consensus on the mean of a measurable environmental feature, and aggregating at areas where that value is measured (e.g., a contour line). A unique quality of this task is a causal loop between the robots' dynamic network topology and their decision-making. For example, the network's mean node degree influences time to convergence while the currently agreed-on mean value influences the swarm's aggregation location, hence, also the network structure as well as the precision error. We propose a control algorithm and study it in real-world robot swarm experiments in different environments. We show that our approach is effective and achieves higher precision than a control experiment. We anticipate applications, for example, in containing pollution with surface vehicles.

Till Aust, Tianfu Zhang, Andreagiovanni Reina et al.

Pulse-coupled oscillator models inspired by firefly synchronization are widely used to study decentralized time coordination in distributed systems. We analyze a discrete-time, discrete-phase firefly-inspired synchronization model and show that collective synchrony emerges only near a critical balance between the quorum threshold (fraction of pulsing neighbors required to trigger a phase update) and the pulse duration (how long agents remain detectable to others). Within this parameter region, the system exhibits bimodal performance: it either reaches near-perfect synchronization or becomes trapped in stable multi-cluster states, where symmetrically phase-offset subgroups mutually reinforce one another and prevent global synchrony. Our analysis shows that reducing connectivity or introducing noise suppresses these low-performance states by breaking such symmetric interactions, indicating that highly connected or noiseless systems are not necessarily optimal for collective synchronization.

Paolo Leopardi, Heiko Hamann, Jonas Kuckling et al.

Task specialization can lead to simpler robot behaviors and higher efficiency in multi-robot systems. Previous works have shown the emergence of task specialization during evolutionary optimization, focusing on feasibility rather than costs. In this study, we take first steps toward a cost-benefit analysis of task specialization in robot swarms using a foraging scenario. We evolve artificial neural networks as generalist behaviors for the entire task and as task-specialist behaviors for subtasks within a limited evaluation budget. We show that generalist behaviors can be successfully optimized while the evolved task-specialist controllers fail to cooperate efficiently, resulting in worse performance than the generalists. Consequently, task specialization does not necessarily improve efficiency when optimization budget is limited.

Pranav Kedia, Rajini Makam, Heiko Hamann et al.

Accurate and continuous localization of Autonomous Underwater Vehicles (AUVs) in GPS-denied environments is a persistent challenge in marine robotics. In the absence of external position fixes, AUVs rely on inertial dead-reckoning, which accumulates unbounded drift due to sensor bias and noise. This paper presents BIND-USBL, a cooperative localization framework in which a fleet of Autonomous Surface Vessels (ASVs) equipped with Ultra-Short Baseline (USBL) acoustic positioning systems provides intermittent fixes to bound AUV dead-reckoning error. The key insight is that long-duration navigation failure is driven not by the accuracy of individual USBL measurements, but by the temporal sparsity and geometric availability of those fixes. BIND-USBL combines a multi-ASV formation model linking survey scale and anchor placement to acoustic coverage, a conflict-graph-based TDMA uplink scheduler for shared-channel servicing, and delayed fusion of received USBL updates with drift-prone dead reckoning. The framework is evaluated in the HoloOcean simulator using heterogeneous ASV-AUV teams executing lawnmower coverage missions. The results show that localization performance is shaped by the interaction of survey scale, acoustic coverage, team composition, and ASV-formation geometry. Further, the spatial-reuse scheduler improves per-AUV fix delivery rate without violating the no-collision constraint, while maintaining low end-to-end fix latency.

Eduard Buss, Till Aust, Heiko Hamann

Purpose: Fast detection of plant stress is key to plant phenotyping, precision agriculture, and automated crop management. In particular, efficient irrigation management requires early identification of water stress to optimize resource use while maintaining crop performance. Direct physiological sensing offers the potential to detect stress responses before visible symptoms appear. Methods: In this study, we recorded electrophysiological signals from greenhouse-grown tomato plants subjected to water stress and developed a framework based on machine learning for online stress detection. The recorded time-series data were processed using a processing pipeline that includes statistical feature extraction and selection, automated machine learning or alternatively deep learning, and probability calibration. Results: Across multiple input time horizons, we found that a 30-minute look-back window strikes the best balance between rapid decision-making and classification performance. Using automated machine learning, the framework achieved classification accuracies of up to 92%, outperforming deep learning approaches. Sequential backward selection reduced the feature set while maintaining performance. Importantly, the framework detects transitions from healthy to stressed states in recordings that were not included in the training set. Conclusion: Overall, we provide a decision-support tool for farmers and establish a foundation for biofeedback-driven irrigation control to improve resource efficiency in (semi-)autonomous crop production systems.

Karthik Soma, Mohamed S. Talamali, Genki Miyauchi et al.

In multi-agent systems, should limited resources be concentrated into a few capable agents or distributed among many simpler ones? This work formulates the split over $n$ resource sharing problem where a group of $n$ agents equally shares a common resource (e.g., monetary budget, computational resources, physical size). We present a case study in multi-agent coverage where the area of the disk-shaped footprint of agents scales as $1/n$. A formal analysis reveals that the initial coverage rate grows with $n$. However, if the speed of agents decreases proportionally with their radii, groups of all sizes perform equally well, whereas if it decreases proportionally with their footprints, a single agent performs best. We also present computer simulations in which resource splitting increases the failure rates of individual agents. The models and findings help identify optimal distributiveness levels and inform the design of multi-agent systems under resource constraints.

Till Aust, Eduard Buss, Felix Mohr et al.

In our project WatchPlant, we propose to use a decentralized network of living plants as air-quality sensors by measuring their electrophysiology to infer the environmental state, also called phytosensing. We conducted in-lab experiments exposing ivy (Hedera helix) plants to ozone, an important pollutant to monitor, and measured their electrophysiological response. However, there is no well established automated way of detecting ozone exposure in plants. We propose a generic automatic toolchain to select a high-performance subset of features and highly accurate models for plant electrophysiology. Our approach derives plant- and stimulus-generic features from the electrophysiological signal using the tsfresh library. Based on these features, we automatically select and optimize machine learning models using AutoML. We use forward feature selection to increase model performance. We show that our approach successfully classifies plant ozone exposure with accuracies of up to 94.6% on unseen data. We also show that our approach can be used for other plant species and stimuli. Our toolchain automates the development of monitoring algorithms for plants as pollutant monitors. Our results help implement significant advancements for phytosensing devices contributing to the development of cost-effective, high-density urban air monitoring systems in the future.

Till Aust, Eirini Balta, Argiro Vatakis et al.

In high-pressure environments where human individuals must simultaneously monitor multiple entities, communicate effectively, and maintain intense focus, the perception of time becomes a critical factor influencing performance and well-being. One indicator of well-being can be the person's subjective time perception. In our project $ChronoPilot$, we aim to develop a device that modulates human subjective time perception. In this study, we present a method to automatically assess the subjective time perception of air traffic controllers, a group often faced with demanding conditions, using their physiological data and eleven state-of-the-art machine learning classifiers. The physiological data consist of photoplethysmogram, electrodermal activity, and temperature data. We find that the support vector classifier works best with an accuracy of 79 % and electrodermal activity provides the most descriptive biomarker. These findings are an important step towards closing the feedback loop of our $ChronoPilot$-device to automatically modulate the user's subjective time perception. This technological advancement may promise improvements in task management, stress reduction, and overall productivity in high-stakes professions.

Carlos Augusto Pinheiro de Sousa, Heiko Hamann, Oliver Deussen

SLAM is a foundational technique with broad applications in robotics and AR/VR. SLAM simulations evaluate new concepts, but testing on resource-constrained devices, such as VR HMDs, faces challenges: high computational cost and restricted sensor data access. This work proposes a sparse framework using mesh geometry projections as features, which improves efficiency and circumvents direct sensor data access, advancing SLAM research as we demonstrate in VR and through numerical evaluation.

Mohamed Roshdi, Julian Petzold, Mostafa Wahby et al. · ibm-research

In Autonomous Driving (AD) transparency and safety are paramount, as mistakes are costly. However, neural networks used in AD systems are generally considered black boxes. As a countermeasure, we have methods of explainable AI (XAI), such as feature relevance estimation and dimensionality reduction. Coarse graining techniques can also help reduce dimensionality and find interpretable global patterns. A specific coarse graining method is Renormalization Groups from statistical physics. It has previously been applied to Restricted Boltzmann Machines (RBMs) to interpret unsupervised learning. We refine this technique by building a transparent backbone model for convolutional variational autoencoders (VAE) that allows mapping latent values to input features and has performance comparable to trained black box VAEs. Moreover, we propose a custom feature map visualization technique to analyze the internal convolutional layers in the VAE to explain internal causes of poor reconstruction that may lead to dangerous traffic scenarios in AD applications. In a second key contribution, we propose explanation and evaluation techniques for the internal dynamics and feature relevance of prediction networks. We test a long short-term memory (LSTM) network in the computer vision domain to evaluate the predictability and in future applications potentially safety of prediction models. We showcase our methods by analyzing a VAE-LSTM world model that predicts pedestrian perception in an urban traffic situation.

Till Aust, Christoph Karl Heck, Eduard Buss et al.

We present a bio-hybrid environmental sensor system that integrates natural plants and embedded deep learning for real-time, on-device detection of temperature and ozone level changes. Our system, based on the low-power PhytoNode platform, records electric differential potential signals from Hedera helix and processes them onboard using an embedded deep learning model. We demonstrate that our sensing device detects changes in temperature and ozone with good sensitivity of up to 0.98. Daily and inter-plant variability, as well as limited precision, could be mitigated by incorporating additional training data, which is readily integrable in our data-driven framework. Our approach also has potential to scale to new environmental factors and plant species. By integrating embedded deep learning onboard our biological sensing device, we offer a new, low-power solution for continuous environmental monitoring and potentially other fields of application.

Eduard Buss, Till Aust, Heiko Hamann

Living plants, while contributing to ecological balance and climate regulation, also function as natural sensors capable of transmitting information about their internal physiological states and surrounding conditions. This rich source of data provides potential for applications in environmental monitoring and precision agriculture. With integration into biohybrid systems, we establish novel channels of physiological signal flow between living plants and artificial devices. We equipped *Hedera helix* with a plant-wearable device called PhytoNode to continuously record the plant's electrophysiological activity. We deployed plants in an uncontrolled outdoor environment to map electrophysiological patterns to environmental conditions. Over five months, we collected data that we analyzed using state-of-the-art and automated machine learning (AutoML). Our classification models achieve high performance, reaching macro F1 scores of up to 95 percent in binary tasks. AutoML approaches outperformed manual tuning, and selecting subsets of statistical features further improved accuracy. Our biohybrid living system monitors the electrophysiology of plants in harsh, real-world conditions. This work advances scalable, self-sustaining, and plant-integrated living biohybrid systems for sustainable environmental monitoring.

Till Aust, Julian Kaduk, Heiko Hamann

As automation and mobile robotics reshape work environments, rising expectations for productivity increase cognitive demands on human operators, leading to potential stress and cognitive overload. Accurately assessing an operator's mental state is critical for maintaining performance and well-being. We use subjective time perception, which can be altered by stress and cognitive load, as a sensitive, low-latency indicator of well-being and cognitive strain. Distortions in time perception can affect decision-making, reaction times, and overall task effectiveness, making it a valuable metric for adaptive human-swarm interaction systems. We study how human physiological signals can be used to estimate a person's subjective time perception in a human-swarm interaction scenario as example. A human operator needs to guide and control a swarm of small mobile robots. We obtain eye-tracking data that is classified for subjective time perception based on questionnaire data. Our results show that we successfully estimate a person's time perception from eye-tracking data. The approach can profit from individual-based pretraining using only 30 seconds of data. In future work, we aim for robots that respond to human operator needs by automatically classifying physiological data in a closed control loop.

Karthik Soma, Yann Bouteiller, Heiko Hamann et al.

Decision-making is an essential attribute of any intelligent agent or group. Natural systems are known to converge to optimal strategies through at least two distinct mechanisms: collective decision-making via imitation of others, and individual trial-and-error. This paper establishes an equivalence between these two paradigms by drawing from the well-established collective decision-making model of nest-hunting in swarms of honey bees. We show that the emergent distributed cognition (sometimes referred to as the $\textit{hive mind}$) arising from individual bees following simple, local imitation-based rules is that of a single online reinforcement learning (RL) agent interacting with many parallel environments. The update rule through which this macro-agent learns is a bandit algorithm that we coin $\textit{Maynard-Cross Learning}$. Our analysis implies that a group of cognition-limited organisms can be equivalent to a more complex, reinforcement-enabled entity, substantiating the idea that group-level intelligence may explain how seemingly simple and blind individual behaviors are selected in nature. From a biological perspective, this analysis suggests how such imitation strategies evolved: they constitute a scalable form of reinforcement learning at the group level, aligning with theories of kin and group selection. Beyond biology, the framework offers new tools for analyzing economic and social systems where individuals imitate successful strategies, effectively participating in a collective learning process. In swarm intelligence, our findings will inform the design of scalable collective systems in artificial domains, enabling RL-inspired mechanisms for coordination and adaptability at scale.

Julian Petzold, Mostafa Wahby, Franek Stark et al.

Pedestrians are particularly vulnerable road users in urban traffic. With the arrival of autonomous driving, novel technologies can be developed specifically to protect pedestrians. We propose a machine learning toolchain to train artificial neural networks as models of pedestrian behavior. In a preliminary study, we use synthetic data from simulations of a specific pedestrian crossing scenario to train a variational autoencoder and a long short-term memory network to predict a pedestrian's future visual perception. We can accurately predict a pedestrian's future perceptions within relevant time horizons. By iteratively feeding these predicted frames into these networks, they can be used as simulations of pedestrians as indicated by our results. Such trained networks can later be used to predict pedestrian behaviors even from the perspective of the autonomous car. Another future extension will be to re-train these networks with real-world video data.

Babak Salamat, Christopher Johannes Starck, Heiko Hamann

Controlling large-scale particle or robot systems is challenging because of their high dimensionality. We use a centralized stochastic approach that allows for optimal control at the cost of a central element instead of a decentralized approach. Previous works are often restricted to the assumption of fully actuated robots. Here we propose an approach for underactuated robots that allows for energy-efficient control of the robot system. We consider a simple task of gathering the robots (minimizing positional variance) and steering them towards a goal point within a bounded area without obstacles. We make two main contributions. First, we present a generalized coordinate transformation for underactuated robots, whose physical properties should be considered. We choose Euler- Lagrange systems that describe a large class of robot systems. Second, we propose an optimal control mechanism with the prime objective of energy efficiency. We show the feasibility of our approach in numerical simulations and robot simulations.

Heiko Hamann, Andreagiovanni Reina

Efficient engineered systems require scalability. A scalable system has increasing performance with increasing system size. In an ideal case, the increase in performance (e.g., speedup) corresponds to the number of units that are added to the system. However, if multiple units work on the same task, then coordination among these units is required. This coordination can introduce overheads with an impact on system performance. The coordination costs can lead to sublinear improvement or even diminishing performance with increasing system size. However, there are also systems that implement efficient coordination and exploit collaboration of units to attain superlinear improvement. Modeling the scalability dynamics is key to understanding efficient systems. Known laws of scalability, such as Amdahl's law, Gustafson's law, and Gunther's Universal Scalability Law, are minimalistic phenomenological models that explain a rich variety of system behaviors through concise equations. While useful to gain general insights, the phenomenological nature of these models may limit the understanding of the underlying dynamics, as they are detached from first principles that could explain coordination overheads among units. Through a decentralized system approach, we propose a general model based on generic interactions between units that is able to describe, as specific cases, any general pattern of scalability included by previously reported laws. The proposed general model of scalability is built on first principles, or at least on a microscopic description of interaction between units, and therefore has the potential to contribute to a better understanding of system behavior and scalability. We show that this model can be applied to a diverse set of systems, such as parallel supercomputers, robot swarms, or wireless sensor networks, creating a unified view on interdisciplinary design for scalability.

Tanja Katharina Kaiser, Heiko Hamann

In collective robotic systems, the automatic generation of controllers for complex tasks is still a challenging problem. Open-ended evolution of complex robot behaviors can be a possible solution whereby an intrinsic driver for pattern formation and self-organization may prove to be important. We implement such a driver in collective robot systems by evolving prediction networks as world models in pair with action-selection networks. Fitness is given for good predictions which causes a bias towards easily predictable environments and behaviors in the form of emergent patterns, that is, environments of minimal surprise. There is no task-dependent bias or any other explicit predetermination for the different qualities of the emerging patterns. A careful configuration of actions, sensor models, and the environment is required to stimulate the emergence of complex behaviors. We study self-assembly to increase the scenario's complexity for our minimal surprise approach and, at the same time, limit the complexity of our simulations to a grid world to manage the feasibility of this approach. We investigate the impact of different swarm densities and the shape of the environment on the emergent patterns. Furthermore, we study how evolution can be biased towards the emergence of desired patterns. We analyze the resilience of the resulting self-assembly behaviors by causing damages to the assembled pattern and observe the self-organized reassembly of the structure. In summary, we evolved swarm behaviors for resilient self-assembly and successfully engineered self-organization in simulation. In future work, we plan to transfer our approach to a swarm of real robots.

Mostafa Wahby, Mary Katherine Heinrich, Daniel Nicolas Hofstadler et al.

Bio-hybrid systems---close couplings of natural organisms with technology---are high potential and still underexplored. In existing work, robots have mostly influenced group behaviors of animals. We explore the possibilities of mixing robots with natural plants, merging useful attributes. Significant synergies arise by combining the plants' ability to efficiently produce shaped material and the robots' ability to extend sensing and decision-making behaviors. However, programming robots to control plant motion and shape requires good knowledge of complex plant behaviors. Therefore, we use machine learning to create a holistic plant model and evolve robot controllers. As a benchmark task we choose obstacle avoidance. We use computer vision to construct a model of plant stem stiffening and motion dynamics by training an LSTM network. The LSTM network acts as a forward model predicting change in the plant, driving the evolution of neural network robot controllers. The evolved controllers augment the plants' natural light-finding and tissue-stiffening behaviors to avoid obstacles and grow desired shapes. We successfully verify the robot controllers and bio-hybrid behavior in reality, with a physical setup and actual plants.

Heiko Hamann, Mohammad Divband Soorati, Mary Katherine Heinrich et al.

Key to our project flora robotica is the idea of creating a bio-hybrid system of tightly coupled natural plants and distributed robots to grow architectural artifacts and spaces. Our motivation with this ground research project is to lay a principled foundation towards the design and implementation of living architectural systems that provide functionalities beyond those of orthodox building practice, such as self-repair, material accumulation and self-organization. Plants and robots work together to create a living organism that is inhabited by human beings. User-defined design objectives help to steer the directional growth of the plants, but also the system's interactions with its inhabitants determine locations where growth is prohibited or desired (e.g., partitions, windows, occupiable space). We report our plant species selection process and aspects of living architecture. A leitmotif of our project is the rich concept of braiding: braids are produced by robots from continuous material and serve as both scaffolds and initial architectural artifacts before plants take over and grow the desired architecture. We use light and hormones as attraction stimuli and far-red light as repelling stimulus to influence the plants. Applied sensors range from simple proximity sensing to detect the presence of plants to sophisticated sensing technology, such as electrophysiology and measurements of sap flow. We conclude by discussing our anticipated final demonstrator that integrates key features of flora robotica, such as the continuous growth process of architectural artifacts and self-repair of living architecture.

Thomas Schmickl, Payam Zahadat, Heiko Hamann

In evolutionary robotics an encoding of the control software, which maps sensor data (input) to motor control values (output), is shaped by stochastic optimization methods to complete a predefined task. This approach is assumed to be beneficial compared to standard methods of controller design in those cases where no a-priori model is available that could help to optimize performance. Also for robots that have to operate in unpredictable environments, an evolutionary robotics approach is favorable. We demonstrate here that such a model-free approach is not a free lunch, as already simple tasks can represent unsolvable barriers for fully open-ended uninformed evolutionary computation techniques. We propose here the 'Wankelmut' task as an objective for an evolutionary approach that starts from scratch without pre-shaped controller software or any other informed approach that would force the behavior to be evolved in a desired way. Our focal claim is that 'Wankelmut' represents the simplest set of problems that makes plain-vanilla evolutionary computation fail. We demonstrate this by a series of simple standard evolutionary approaches using different fitness functions and standard artificial neural networks as well as continuous-time recurrent neural networks. All our tested approaches failed. We claim that any other evolutionary approach will also fail that does per-se not favor or enforce modularity and does not freeze or protect already evolved functionalities. Thus we propose a hard-to-pass benchmark and make a strong statement for self-complexifying and generative approaches in evolutionary computation. We anticipate that defining such a 'simplest task to fail' is a valuable benchmark for promoting future development in the field of artificial intelligence, evolutionary robotics and artificial life.

Debdipta Goswami, Heiko Hamann

The study of collective decision making system has become the central part of the Swarm- Intelligence Related research in recent years. The most challenging task of modelling a collec- tive decision making system is to develop the macroscopic stochastic equation from its microscopic model. In this report we have investigated the behaviour of a collective decision making system with specified microscopic rules that resemble the chemical reaction and used different group size. Then we ventured to derive a generalized analytical model of a collective-decision system using hyper-geometric distribution. Index Terms-swarm; collective decision making; noise; group size; hyper-geometric distribution

Heiko Hamann

Methods of general applicability are searched for in swarm intelligence with the aim of gaining new insights about natural swarms and to develop design methodologies for artificial swarms. An ideal solution could be a `swarm calculus' that allows to calculate key features of swarms such as expected swarm performance and robustness based on only a few parameters. To work towards this ideal, one needs to find methods and models with high degrees of generality. In this paper, we report two models that might be examples of exceptional generality. First, an abstract model is presented that describes swarm performance depending on swarm density based on the dichotomy between cooperation and interference. Typical swarm experiments are given as examples to show how the model fits to several different results. Second, we give an abstract model of collective decision making that is inspired by urn models. The effects of positive feedback probability, that is increasing over time in a decision making system, are understood by the help of a parameter that controls the feedback based on the swarm's current consensus. Several applicable methods, such as the description as Markov process, calculation of splitting probabilities, mean first passage times, and measurements of positive feedback, are discussed and applications to artificial and natural swarms are reported.