World Models: The Safety Perspective
This work addresses safety concerns for AI agents using World Models in critical applications, but it is incremental as it focuses on reviewing and analyzing existing technology rather than proposing new methods.
The paper reviews and analyzes the impacts of current state-of-the-art World Models (WM) technology from a trustworthiness and safety perspective, based on a comprehensive survey and envisaged application fields, to call for collaborative research on improving WM safety.
With the proliferation of the Large Language Model (LLM), the concept of World Models (WM) has recently attracted a great deal of attention in the AI research community, especially in the context of AI agents. It is arguably evolving into an essential foundation for building AI agent systems. A WM is intended to help the agent predict the future evolution of environmental states or help the agent fill in missing information so that it can plan its actions and behave safely. The safety property of WM plays a key role in their effective use in critical applications. In this work, we review and analyze the impacts of the current state-of-the-art in WM technology from the point of view of trustworthiness and safety based on a comprehensive survey and the fields of application envisaged. We provide an in-depth analysis of state-of-the-art WMs and derive technical research challenges and their impact in order to call on the research community to collaborate on improving the safety and trustworthiness of WM.