The Automated Data Quality Assessment Tool for Low Frequency Drilling Data from Rig Site

Abstract In drilling operations, various data, such as daily reports and directional surveys, are entered manually into databases. This manual entry process is error-prone, making data unreliable for analytical purposes and requiring time-consuming human verification to identify errors, especially when number of rigs increase significantly. This paper will demonstrate how data quality can be improved at the source by developing an automated tool that assesses data daily, notifying rig personnel to correct errors on the same day. To evaluate data quality from rig entries, we developed over 500+ logic-based business rules to verify data on four key dimensions: completeness, accuracy, consistency, and uniqueness. The business rules were implemented as SQL queries. We then leveraged python script to trigger these queries, compile data quality results, and notify relevant personnel of any detected errors. Additionally, the tool logged each result in a data quality dashboard. This entire process was automatically scheduled each day after new data was synchronized from rigs to the cloud database. After the implementation of this tool across the company’s operating rigs in the Gulf of Thailand, we observed a marked improvement in data integrity. The tool successfully identified discrepancies in entered data, highlighting specific inaccuracies, and enabling timely corrections. With cooperation of rig-site personnel, data quality scores increased significantly, from below 80% to over 95%. This tool also eliminated the need for manual data checks, particularly for structured data like BHA operations, drilling fluid properties, and directional surveys, allowing drilling engineers to focus on what was reported in natural language, such as operational descriptions and remarks. This process reduced each engineer’s workload by an average of one hour per day per rig. Notably, full cooperation from rig-site personnel to correct the data after being notified was crucial to obtaining the full benefit of this tool. This paper introduces the use of digital tool to assist human in time consuming task and presents the concept of data quality control in drilling industry, which is the necessary foundation in data preparation for further uses in analytics or machine learning.