Statistical Methods to Improve the Quality of Real-Time Drilling Data

Salem Al-Gharbi, Abdulaziz Al-Majed, Abdulazeez Abdulraheem, Zeeshan Tariq, Mohamed Mahmoud

Research output: Contribution to journalArticlepeer-review

5 Scopus citations


The age of easy oil is ending, and the industry started drilling in remote unconventional conditions. To help produce safer, faster, and most effective operations, the utilization of artificial intelligence and machine learning (AI/ML) has become essential. Unfortunately, due to the harsh environments of drilling and the data-transmission setup, a significant amount of the real-time data could defect. The quality and effectiveness of AI/ML models are directly related to the quality of the input data; only if the input data are good, the AI/ML-generated analytical and prediction models will be good. Improving the real-time data is therefore critical to the drilling industry. The objective of this paper is to propose an automated approach using eight statistical data-quality improvement algorithms on real-time drilling data. These techniques are Kalman filtering, moving average, kernel regression, median filter, exponential smoothing, lowess, wavelet filtering, and polynomial. A dataset of +150, 000 rows is fed into the algorithms, and their customizable parameters are calibrated to achieve the best improvement result. An evaluation methodology is developed based on real-time drilling data characteristics to analyze the strengths and weaknesses of each algorithm which were highlighted. Based on the evaluation criteria, the best results were achieved using the exponential smoothing, median filter, and moving average. Exponential smoothing and median filter techniques improved the quality of data by removing most of the invalid data-points; the moving average removed more invalid data-points but trimmed the data range.
Original languageEnglish (US)
JournalJournal of Energy Resources Technology, Transactions of the ASME
Issue number9
StatePublished - Sep 1 2022
Externally publishedYes

ASJC Scopus subject areas

  • Energy Engineering and Power Technology
  • Mechanical Engineering
  • Geochemistry and Petrology
  • Fuel Technology
  • Renewable Energy, Sustainability and the Environment


Dive into the research topics of 'Statistical Methods to Improve the Quality of Real-Time Drilling Data'. Together they form a unique fingerprint.

Cite this