Joint Level Generation and Translation Using Gameplay Videos
This addresses a bottleneck in PCGML for game developers by reducing reliance on labor-intensive secondary representations, though it is incremental as it builds on existing methods.
The paper tackles the problem of limited annotated data in procedural content generation via machine learning by developing a multi-tail framework that uses gameplay videos to perform simultaneous level translation and generation, showing improved performance in both tasks compared to baselines.
Procedural Content Generation via Machine Learning (PCGML) faces a significant hurdle that sets it apart from other fields, such as image or text generation, which is limited annotated data. Many existing methods for procedural level generation via machine learning require a secondary representation besides level images. However, the current methods for obtaining such representations are laborious and time-consuming, which contributes to this problem. In this work, we aim to address this problem by utilizing gameplay videos of two human-annotated games to develop a novel multi-tail framework that learns to perform simultaneous level translation and generation. The translation tail of our framework can convert gameplay video frames to an equivalent secondary representation, while its generation tail can produce novel level segments. Evaluation results and comparisons between our framework and baselines suggest that combining the level generation and translation tasks can lead to an overall improved performance regarding both tasks. This represents a possible solution to limited annotated level data, and we demonstrate the potential for future versions to generalize to unseen games.