Luis Santos

J. K. Terry, Benjamin Black, Nathaniel Grammel et al.

This paper introduces the PettingZoo library and the accompanying Agent Environment Cycle ("AEC") games model. PettingZoo is a library of diverse sets of multi-agent environments with a universal, elegant Python API. PettingZoo was developed with the goal of accelerating research in Multi-Agent Reinforcement Learning ("MARL"), by making work more interchangeable, accessible and reproducible akin to what OpenAI's Gym library did for single-agent reinforcement learning. PettingZoo's API, while inheriting many features of Gym, is unique amongst MARL APIs in that it's based around the novel AEC games model. We argue, in part through case studies on major problems in popular MARL environments, that the popular game models are poor conceptual models of games commonly used in MARL and accordingly can promote confusing bugs that are hard to detect, and that the AEC games model addresses these problems.

J K Terry, Nathaniel Grammel, Benjamin Black et al.

Partially Observable Stochastic Games (POSGs) are the most general and common model of games used in Multi-Agent Reinforcement Learning (MARL). We argue that the POSG model is conceptually ill suited to software MARL environments, and offer case studies from the literature where this mismatch has led to severely unexpected behavior. In response to this, we introduce the Agent Environment Cycle Games (AEC Games) model, which is more representative of software implementation. We then prove it's as an equivalent model to POSGs. The AEC games model is also uniquely useful in that it can elegantly represent both all forms of MARL environments, whereas for example POSGs cannot elegantly represent strictly turn based games like chess.

J. K. Terry, Benjamin Black, Luis Santos

The Arcade Learning Environment ("ALE") is a widely used library in the reinforcement learning community that allows easy programmatic interfacing with Atari 2600 games, via the Stella emulator. We introduce a publicly available extension to the ALE that extends its support to multiplayer games and game modes. This interface is additionally integrated with PettingZoo to allow for a simple Gym-like interface in Python to interact with these games. We additionally introduce experimental baselines for all environments included.

Mohammad Keshavarzi, Luisa Caldas, Luis Santos

This work introduces RadVR, a virtual reality tool for daylighting analysis that simultaneously combines qualitative assessments through immersive real-time renderings with quantitative physically correct daylighting simulations in a 6DOF virtual environment. By taking a 3D building model with material properties as input, RadVR allows users to (1) perform physically-based daylighting simulations via Radiance, (2) study sunlight in different hours-of-the-year, (3) interact with a 9-point-in-time matrix for the most representative times of the year, and (4) visualize, compare, and analyze daylighting simulation results. With an end-to-end workflow, RadVR integrates with 3D modeling software that is commonly used by building designers. Additionally, by conducting user experiments we compare the proposed system with DIVA for Rhino, a Radiance-based tool that uses conventional 2D-displays. The results show that RadVR can provide promising assistance in spatial understanding tasks, navigation, and sun position analysis in virtual reality.