Deep Reinforcement Learning for Field Development Optimization
This work addresses the need for robust optimization solutions in oil and gas field development, though it is incremental as it adapts existing DRL methods to a specific domain.
The authors tackled the field development optimization problem, which involves determining well parameters to maximize economic metrics, by applying convolutional neural network-based deep reinforcement learning to generate robust policies, achieving results comparable to a hybrid PSO-MADS algorithm.
The field development optimization (FDO) problem represents a challenging mixed-integer nonlinear programming (MINLP) problem in which we seek to obtain the number of wells, their type, location, and drilling sequence that maximizes an economic metric. Evolutionary optimization algorithms have been effectively applied to solve the FDO problem, however, these methods provide only a deterministic (single) solution which are generally not robust towards small changes in the problem setup. In this work, the goal is to apply convolutional neural network-based (CNN) deep reinforcement learning (DRL) algorithms to the field development optimization problem in order to obtain a policy that maps from different states or representation of the underlying geological model to optimal decisions. The proximal policy optimization (PPO) algorithm is considered with two CNN architectures of varying number of layers and composition. Both networks obtained policies that provide satisfactory results when compared to a hybrid particle swarm optimization - mesh adaptive direct search (PSO-MADS) algorithm that has been shown to be effective at solving the FDO problem.