Bolun Dai

Maxime Chevalier-Boisvert, Bolun Dai, Mark Towers et al.

We present the Minigrid and Miniworld libraries which provide a suite of goal-oriented 2D and 3D environments. The libraries were explicitly created with a minimalistic design paradigm to allow users to rapidly develop new environments for a wide range of research-specific needs. As a result, both have received widescale adoption by the RL community, facilitating research in a wide range of areas. In this paper, we outline the design philosophy, environment details, and their world generation API. We also showcase the additional capabilities brought by the unified API between Minigrid and Miniworld through case studies on transfer learning (for both RL agents and humans) between the different observation spaces. The source code of Minigrid and Miniworld can be found at https://github.com/Farama-Foundation/{Minigrid, Miniworld} along with their documentation at https://{minigrid, miniworld}.farama.org/.

Fauna Robotics, Diego Aldarondo, Ana Pervan et al.

Recent advances in learned control, large-scale simulation, and generative models have accelerated progress toward general-purpose robotic controllers, yet the field still lacks platforms suitable for safe, expressive, long-term deployment in human environments. Most existing humanoids are either closed industrial systems or academic prototypes that are difficult to deploy and operate around people, limiting progress in robotics. We introduce Sprout, a developer platform designed to address these limitations through an emphasis on safety, expressivity, and developer accessibility. Sprout adopts a lightweight form factor with compliant control, limited joint torques, and soft exteriors to support safe operation in shared human spaces. The platform integrates whole-body control, manipulation with integrated grippers, and virtual-reality-based teleoperation within a unified hardware-software stack. An expressive head further enables social interaction -- a domain that remains underexplored on most utilitarian humanoids. By lowering physical and technical barriers to deployment, Sprout expands access to capable humanoid platforms and provides a practical basis for developing embodied intelligence in real human environments.

Bolun Dai

Model-based control usually relies on an accurate model, which is often obtained from CAD and actuator models. The more accurate the model the better the control performance. However, in bipedal robots that demonstrate high agility actions, such as running and hopping, the robot hardware will suffer from impacts with the environment and deform in vulnerable parts, which invalidates the predefined model. Thus, it is desired to have an adaptable kinematic structure that takes deformation into consideration. To account for this we propose an approach that models all of the robotic joints as 6-DOF joints and develop an algorithm that can identify the kinematic structure from motion capture data. We evaluate the algorithm's performance both in simulation - a three link pendulum, and on a bipedal robot - ATRIAS. In the simulated case the algorithm produces a result that has a 3.6% error compared to the ground truth, and on the real life bipedal robot the algorithm's result confirms our prior assumption where the joint deforms on out-of-plane degrees of freedom. In addition our algorithm is able to predict torques and forces using the reconstructed joint mode.

Bolun Dai, Virinchi Roy Surabhi, Prashanth Krishnamurthy et al.

In this paper, we propose a deep forward-backward stochastic differential equation (FBSDE) based control algorithm for locomotion tasks. We also include state constraints in the FBSDE formulation to impose stable walking solutions or other constraints that one may want to consider (e.g., energy). Our approach utilizes a deep neural network (i.e., LSTM) to solve, in general, high-dimensional Hamilton-Jacobi-Bellman (HJB) equation resulting from the stated optimal control problem. As compared to traditional methods, our proposed method provides a higher computational efficiency in real-time; thus yielding higher frequency implementation of the closed-loop controllers. The efficacy of our approach is shown on a linear inverted pendulum model (LIPM) for walking. Even though we are deploying a simplified model of walking, the methodology is applicable to generalized and complex models for walking and other control/optimization tasks in robotic systems. Simulation studies have been provided to show the effectiveness of the proposed methodology.

Tianyu Li, Bolun Dai

Learning from complex demonstrations is challenging, especially when the demonstration consists of different strategies. A popular approach is to use a deep neural network to perform imitation learning. However, the structure of that deep neural network has to be ``deep" enough to capture all possible scenarios. Besides the machine learning issue, how humans learn in the sense of physiology has rarely been addressed and relevant works on spinal cord learning are rarer. In this work, we develop a novel modular learning architecture, the Generator and Responsibility Predictor (GRP) model, which automatically learns the sub-task policies from an unsegmented controller demonstration and learns to switch between the policies. We also introduce a more physiological based neural network architecture. We implemented our GRP model and our proposed neural network to form a model the transfers the swing leg control from the brain to the spinal cord. Our result suggests that by using the GRP model the brain can successfully transfer the target swing leg control to the spinal cord and the resulting model can switch between sub-control policies automatically.