Francesco Guidi

Jingfeng Shan, Francesco Guidi, Mehrdad Saeidi et al.

Autonomous agents operating in uncertain environments must balance fast responses with goal-directed planning. Classical MF RL often converges slowly and may induce unsafe exploration, whereas MB methods are computationally expensive and sensitive to model mismatch. This paper presents a human-inspired hybrid RL architecture integrating Pavlovian, Instrumental MF, and Instrumental MB components. Inspired by Pavlovian and Instrumental learning from neuroscience, the framework considers contextual radio cues, here intended as georeferenced environmental features acting as CS, to shape intrinsic value signals and bias decision-making. Learning is further modulated by internal motivational drives through a dedicated motivational signal. A Bayesian arbitration mechanism adaptively blends MF and MB estimates based on predicted reliability. Simulation results show that the hybrid approach accelerates learning, improves operational safety, and reduces navigation in high-uncertainty regions compared to standard RL baselines. Pavlovian conditioning promotes safer exploration and faster convergence, while arbitration enables a smooth transition from exploration to efficient, plan-driven exploitation. Overall, the results highlight the benefits of biologically inspired modularity for robust and adaptive autonomous systems under uncertainty.

Anna Guerra, Francesco Guidi, Davide Dardari et al.

In this paper, we study a joint detection, mapping and navigation problem for a single unmanned aerial vehicle (UAV) equipped with a low complexity radar and flying in an unknown environment. The goal is to optimize its trajectory with the purpose of maximizing the mapping accuracy and, at the same time, to avoid areas where measurements might not be sufficiently informative from the perspective of a target detection. This problem is formulated as a Markov decision process (MDP) where the UAV is an agent that runs either a state estimator for target detection and for environment mapping, and a reinforcement learning (RL) algorithm to infer its own policy of navigation (i.e., the control law). Numerical results show the feasibility of the proposed idea, highlighting the UAV's capability of autonomously exploring areas with high probability of target detection while reconstructing the surrounding environment.

Anna Guerra, Francesco Guidi, Gianni Pasolini et al.

Digital maps will revolutionize our experience of perceiving and navigating indoor environments. While today we rely only on the representation of the outdoors, the mapping of indoors is mainly a part of the traditional SLAM problem where robots discover the surrounding and perform self-localization. Nonetheless, robot deployment prevents from a large diffusion and fast mapping of indoors and, further, they are usually equipped with laser and vision technology that fail in scarce visibility conditions. To this end, a possible solution is to turn future personal devices into personal radars as a milestone towards the automatic generation of indoor maps using massive array technology at millimeter-waves, already in place for communications. In this application-oriented paper, we will describe the main achievements attained so far to develop the personal radar concept, using ad-hoc collected experimental data, and by discussing possible future directions of investigation.