Inspectioneering Journal

Don’t Throw Out Your IDMS Data: Use Statistics to Rebuild Trust

By Vyacheslav Nadvoretskiy, PhD, Principal Data Scientist at Pinnacle, and Andrew Waters, Director, Data Scientist at Pinnacle. This article appears in the September/October 2023 issue of Inspectioneering Journal.


As an industry, we spend a tremendous amount of money, time, effort, and resources collecting data. Despite this, how often do we mistrust the data we have collected and worry about using our data to make reliability decisions? We recognize that quality data is a critical component of a successful mechanical integrity or reliability program. However, erroneous data coupled with a lack of trust prevents us from extracting its full value, possibly limiting the quality of our reliability programs.

In many cases, the result of not trusting data is, paradoxically, collecting more data through inspections, which can further exacerbate reliability challenges. In addition to adding to the volume of data to sift through, over-inspection costs valuable time and money that could be spent on more valuable activities that mitigate asset risk and increase facility uptime.

Suspect data comes in a variety of forms. A typical example is inspection data for thinning over time. A facility might not trust data from a specific time frame or may question the process that was used to collect that data. Even when the data is mostly trustworthy, it is easy to find anomalous data points that don’t make sense (e.g., readings above nominal, readings below critical, etc.). Another common example of suspect data is when asset operating parameters such as temperature, pressure, and metallurgy are missing, which limits the effectiveness of a subject matter expert (SME) to accurately estimate potential corrosion mechanisms and rates for fixed equipment.

In a recent study across 15 global refineries from six different operators, we identified four common industry data integrity challenges:

  1. Outlier data readings
  2. Growth in thickness readings
  3. Potential repairs/replacements
  4. Missing data

In this article, we discuss these common challenges and dive into how facilities can leverage data science and statistical techniques to quickly identify and potentially correct or quarantine suspicious data to create more stable data analytics that can be utilized to drive more confident decisions and value from the data.

This content is available to registered users and subscribers

Register today to unlock this article for free.

Create your free account and get access to:

  • Unlock one premium article of your choosing per month
  • Exclusive online content, videos, and downloads
  • Insightful and actionable webinars
Interested in unlimited access? VIEW OUR SUBSCRIPTION OPTIONS

Current subscribers and registered users can log in now.

Comments and Discussion

Posted by Andy Gysbers on October 27, 2023
Your article does somewhat address state of... Log in or register to read the rest of this comment.

Posted by A Moshaweh on December 11, 2023
What software or techniques used to plot figures... Log in or register to read the rest of this comment.

Add a Comment

Please log in or register to participate in comments and discussions.

Inspectioneering Journal

Explore over 20 years of articles written by our team of subject matter experts.

Company Directory

Find relevant products, services, and technologies.

Training Solutions

Improve your skills in key mechanical integrity subjects.

Case Studies

Learn from the experience of others in the industry.


Inspectioneering's index of mechanical integrity topics – built by you.

Industry News

Stay up-to-date with the latest inspection and asset integrity management news.


Read short articles and insights authored by industry experts.

Expert Interviews

Inspectioneering's archive of interviews with industry subject matter experts.

Event Calendar

Find upcoming conferences, training sessions, online events, and more.


Downloadable eBooks, Asset Intelligence Reports, checklists, white papers, and more.

Videos & Webinars

Watch educational and informative videos directly related to your profession.


Commonly used asset integrity management and inspection acronyms.