James Fleming

AI
3papers
5citations
Novelty57%
AI Score46

3 Papers

99.1AIMay 10Code
SimWorld Studio: Automatic Environment Generation with Evolving Coding Agent for Embodied Agent Learning

Haoqiang Kang, Xiaokang Ye, Yuhan Liu et al.

LLM/VLM-based digital agents have advanced rapidly thanks to scalable sandboxes for coding, web navigation, and computer use, which provide rich interactive training grounds. In contrast, embodied agents still lack abundant, diverse, and automatically generated 3D environments for interactive learning. Existing embodied simulators rely on manually crafted scenes or procedural templates, while recent LLM-based 3D generation systems mainly produce static scenes rather than deployable environments with verifiable tasks and standard learning interfaces. We introduce SimWorld Studio, an open-source platform built on Unreal Engine 5 for generating evolving embodied learning environments. At its core is SimCoder, a tool/skill-augmented coding agent that writes and executes engine-level code to construct physically grounded 3D worlds from language/image instructions. SimCoder self-evolves by using verifier feedback (e.g., compilation errors, physics checks, VLM critiques) to revise environments and autonomously add reusable tools and skills to its library. Generated worlds are exported as Gym-style environments for embodied agent learning. SimWorld Studio further enables co-evolution between environment generation and embodied learning: agent performance feedback guides SimCoder to generate adaptive curricula near the learner's capability frontier, so that environments become increasingly challenging as the embodied agent improves. Three case studies on embodied navigation show that self-evolution improves generation reliability, generated environments substantially improve embodied agent performance that generalizes to unseen benchmarks, and co-evolution yields an 18-point success-rate gain over fixed-environment learning and a 40-point gain over an untrained agent.

CVMar 10, 2023
Estimating friction coefficient using generative modelling

Mohammad Otoofi, William J. B. Midgley, Leo Laine et al.

It is common to utilise dynamic models to measure the tyre-road friction in real-time. Alternatively, predictive approaches estimate the tyre-road friction by identifying the environmental factors affecting it. This work aims to formulate the problem of friction estimation as a visual perceptual learning task. The problem is broken down into detecting surface characteristics by applying semantic segmentation and using the extracted features to predict the frictional force. This work for the first time formulates the friction estimation problem as a regression from the latent space of a semantic segmentation model. The preliminary results indicate that this approach can estimate frictional force.

99.3SYMar 20
Distributed State Estimation for Discrete-time LTI Systems: the Design Trilemma and a Novel Framework

Ruixuan Zhao, Guitao Yang, James Fleming et al.

With the advancement of IoT technologies and the rapid expansion of cyber-physical systems, there is increasing interest in distributed state estimation, where multiple sensors collaboratively monitor large-scale dynamic systems. Compared with its continuous-time counterpart, a discrete-time distributed observer faces greater challenges, as it cannot exploit high-gain mechanisms or instantaneous communication. Existing approaches depend on three tightly coupled factors: (i) system observability, (ii) communication frequency and dimension of the exchanged information, and (iii) network connectivity. However, the interdependence among these factors remains underexplored. This paper identifies a fundamental trilemma among these factors and introduces a general design framework that balances them through an iterative semidefinite programming approach. As such, the proposed method mitigates the restrictive assumptions present in existing works. The effectiveness and generality of the proposed approach are demonstrated through a simulation example.