Muzaffar Qureshi

Tochukwu Elijah Ogri, Muzaffar Qureshi, Zachary I. Bell et al.

In this paper, a novel online, safe output-feedback, critic-only, adaptive optimal control framework is developed for safety-critical control of partially observable systems. The developed framework ensures system stability and safety, regardless of the lack of full-state measurements, while learning and implementing a near-optimal controller. The approach leverages linear matrix inequality-based observer design methods to efficiently search for observer gains for effective state estimation. Then, approximate dynamic programming is used to develop an approximate controller that uses simulated experiences to guarantee the safety and stability of the closed-loop system. Safety is enforced by adding a recentered robust Lyapunov-like barrier function to the cost function that effectively enforces safety constraints, even in the presence of state estimation errors. Lyapunov-based stability analysis is used to guarantee uniform ultimate boundedness of the trajectories of the closed-loop system and ensure safety. Simulation studies are performed to demonstrate the effectiveness of the developed method through two real-world safety-critical scenarios, specifically one ensuring that the state trajectories of a given system remain within a given set, and the other ensuring that the system avoids an obstacle.

M. Nomaan Qureshi, Ben Eisner, David Held

While solving complex manipulation tasks, manipulation policies often need to learn a set of diverse skills to accomplish these tasks. The set of skills is often quite multimodal - each one may have a quite distinct distribution of actions and states. Standard deep policy-learning algorithms often model policies as deep neural networks with a single output head (deterministic or stochastic). This structure requires the network to learn to switch between modes internally, which can lead to lower sample efficiency and poor performance. In this paper we explore a simple structure which is conducive to skill learning required for so many of the manipulation tasks. Specifically, we propose a policy architecture that sequentially executes different action heads for fixed durations, enabling the learning of primitive skills such as reaching and grasping. Our empirical evaluation on the Metaworld tasks reveals that this simple structure outperforms standard policy learning methods, highlighting its potential for improved skill acquisition.