Do Masked Autoencoders Improve Downhole Prediction? An Empirical Study on Real Well Drilling Data
For drilling analytics practitioners facing scarce labeled downhole data, MAE pretraining offers a viable alternative to fully supervised training, though its benefits are modest and architecture-dependent.
This paper evaluates masked autoencoder (MAE) pretraining for downhole drilling metric prediction, finding that the best MAE configuration reduces test mean absolute error by 19.8% relative to a supervised GRU baseline but trails a supervised LSTM baseline by 6.4%.
Downhole drilling telemetry presents a fundamental labeling asymmetry: surface sensor data are generated continuously at 1~Hz, while labeled downhole measurements are costly, intermittent, and scarce. Current machine learning approaches for downhole metric prediction universally adopt fully supervised training from scratch, which is poorly suited to this data regime. We present the first empirical evaluation of masked autoencoder (MAE) pretraining for downhole drilling metric prediction. Using two publicly available Utah FORGE geothermal wells comprising approximately 3.5 million timesteps of multivariate drilling telemetry, we conduct a systematic full-factorial design space search across 72 MAE configurations and compare them against supervised LSTM and GRU baselines on the task of predicting Total Mud Volume. Results show that the best MAE configuration reduces test mean absolute error by 19.8\% relative to the supervised GRU baseline, while trailing the supervised LSTM baseline by 6.4\%. Analysis of design dimensions reveals that latent space width is the dominant architectural choice (Pearson $r = -0.59$ with test MAE), while masking ratio has negligible effect, an unexpected finding attributed to high temporal redundancy in 1~Hz drilling data. These results establish MAE pretraining as a viable paradigm for drilling analytics and identify the conditions under which it is most beneficial.