SortingEnv: An Extendable RL-Environment for an Industrial Sorting Process
This work addresses the problem of industrial process optimization for engineers and researchers, but it appears incremental as it builds on existing RL methods by applying them to a new simulation environment.
The researchers tackled the problem of optimizing industrial sorting systems by creating a novel RL environment called SortingEnv that simulates material flow with operational parameters like belt speed and occupancy, and they found that common RL algorithms like PPO, DQN, and A2C were evaluated against a rule-based agent, though no concrete performance numbers were provided in the abstract.
We present a novel reinforcement learning (RL) environment designed to both optimize industrial sorting systems and study agent behavior in evolving spaces. In simulating material flow within a sorting process our environment follows the idea of a digital twin, with operational parameters like belt speed and occupancy level. To reflect real-world challenges, we integrate common upgrades to industrial setups, like new sensors or advanced machinery. It thus includes two variants: a basic version focusing on discrete belt speed adjustments and an advanced version introducing multiple sorting modes and enhanced material composition observations. We detail the observation spaces, state update mechanisms, and reward functions for both environments. We further evaluate the efficiency of common RL algorithms like Proximal Policy Optimization (PPO), Deep-Q-Networks (DQN), and Advantage Actor Critic (A2C) in comparison to a classical rule-based agent (RBA). This framework not only aids in optimizing industrial processes but also provides a foundation for studying agent behavior and transferability in evolving environments, offering insights into model performance and practical implications for real-world RL applications.