by Roderick Munro, Ph.D.

The state of being registered (or in some cases compliant) to ISO 9001 is far more than just following the ISO 9001 standard itself. ISO has published a number of quality management system-related documents, including a series of guidance documents in the ISO 10000 portfolio of quality management standards. One of these documents—ISO 10017, Guidance on statistical techniques for ISO 9001—explains “statistical techniques which follow from the variability that can be observed in the behavior and outcome of processes, even under conditions of apparent stability. Statistical techniques allow better use of available data to assist in decision making, and thereby help to continually improve the quality of products and processes to achieve customer satisfaction.”

I recently observed an application of this in a maintenance office of an organization where one of the seven mechanics had been trained as a Six Sigma Green Belt. In talking with the manager of the group, I asked him if the reports provided by the one mechanic were any better than the others and if that difference might have anything to do with the Green Belt training? After a long pause to think about the question, the manager told me that he did see a definite difference in the person with the additional statistical training. Of course, my next question of the manager was when was he going to train the other mechanics as Green Belts!

For the purpose of this discussion, let’s ask the same question of the internal audit program. Do you think that training internal auditors to be able to better read and use statistical information would benefit your organization and improve your auditor effectiveness? Would it create better informed personnel, better internal audit sampling, and better overall internal audit reports? In my experience, there is a general knowledge gap between the basic understanding of statistical methods and the numerous possibilities of how to present data beyond the bar chart that is most commonly used.

ISO 10017:2003 specifically mentions the categories of:

• 4, Descriptions of statistical techniques identified
• 1, General
• 2, Descriptive statistics
• 3, Design of experiments (DOE)
• 4, Hypothesis testing
• 5, Measurement analysis
• 6, Process capability analysis
• 7, Regression analysis
• 8, Reliability analysis
• 9, Sampling
• 10, Simulation
• 11, Statistical process control (SPC) charts
• 12, Statistical tolerancing
• 13, Time series analysis

For internal auditors, the use of descriptive statistics and sampling plans are mentioned. Many times, when I review the audit plans/schedules of an organization, I find a static planning process in place (past and present). Managers speak the words of “sampling plans” and “random samples;” however, little evidence is found of a living/modifiable internal audit process that is able to adjust to the situation. How embarrassing it is (or should be) for an external auditor to start finding nonconformities within a relatively short time if the local management team thinks that they have a robust internal audit process. In some cases, this has led to a finding in itself that the internal audit process is not effective (something more than just compliant).

Other practical examples of where internal auditors can use statistical knowledge could include:

• In a recent audit, we took the primary key process indicator (KPI) for the company for several years and converted the data into a simple run chart to look for process patterns. The quality manager claimed to have been through 800 audits during the past 20 years—it was a pharmaceutical organization with many layers of governmental-type audits—and said that she had never seen an auditor do this before. This was very interesting to me as I was able to talk with the management team about their business without a deep understanding of their industry, given the patterns that I was observing. This technique has been very useful to me during the years and I am constantly intrigued that many quality managers don’t know how to use, or perhaps are not allowed to use, statistics to show management more information about their businesses.
• Another common data presentation on management reports is that the data are only shown for the current calendar year (a few will show a rolling 12-month period, which should be the minimum). If you only look at the calendar year (or the last 12 months for some companies), then at the beginning of each new cycle, you will be pretty much blind to any patterns in the process. Internal audits should add the previous data to the charts to show a minimum of one year (ideally two or three if the data are available) to understand the process flow and potential trends better. Remember the rules for control charts when reading the run charts—data that are clustered or showing patterns of seven or more points indicate a trend.
• One recent review of a gage calibration program revealed that the engineers monitoring the process were unaware of what a gage repeatability and reproducibility (GR&R) study (sometimes called a measurement system analysis) consisted of. When explaining the idea that all processes have variation, they could understand the need to study the gages for process variation, but had never heard of the concept prior to the audit. The basic premise is to understand the amount of variation in the overall measurement process that comes from the gages themselves, the people using the gages, and the environment in which the gaging is being conducted. There are a number of good references available from the Automotive Industry Action Group (AIAG) to conduct these studies.

If your organization is using some form of balanced score cards for the operations or at specific process areas, this information should be full of data that could be presented in better ways for the people in that operation to understand. Internal auditors increase effectiveness when they experiment with various data ideas (descriptive statistics) to see if there is a more meaningful statistic that will benefit the organization. My primary preference is the run chart and to teach people about how to read variation in a process. The more advanced technique would then be to use control charts. The idea of tracking the process variation can be used in many parts of the organization such as design reviews done on time, delivery schedules, inventory cycle counts accuracy, safety issues, production counts (hourly, daily, weekly, etc.), rejection rates, and so on. For cases where the process changes frequently, I recommend using short run SPC instead of the traditional control chart (with the parts going through the system) to monitor the process.

There are many process flows within any organization that could be tracked, and internal auditors should look for the key processes of the system to set up some form of process flow charting. One important area should be the results of the internal audit process itself as it relates to customer and external audit results. Are patterns present and do the results make sense? Remember, the internal audit process should be so robust that external customers or external auditors have a very difficult time identifying any findings.

Still other areas could include customer satisfaction data, response time to closing corrective action and preventive action requests, the number of completed continual improvement projects, the average time to completion, or even what percentage of personnel have been training in statistical data applications.

Thus the idea of training internal auditors, or better utilizing the statistical training that they may already have, could yield benefits to the organization along the lines of what W. Edwards Deming called “Profound Knowledge:”

• Appreciation of a system
• Knowledge of variation
• Theory of knowledge
• Knowledge of psychology

Allowing and encouraging your internal auditors to use statistical training for conducting the internal audits will cover two of Deming’s ideas. Add the other two, and you will have a very strong process approach to internal auditing.

About the author

Roderick Munro, Ph.D., has more than thirty-five years of process improvement experience in the service and manufacturing industries working from the shop/store floor up into the managerial ranks. He is an ASQ Fellow and a Fellow of The Chartered Quality Institute (CQI) of England. He has extensive experience with lean, Six Sigma, quality engineering, and has worked with the automotive core tools, ISO 9000, QS-9000, ISO/TS 16949, ISO 14000, BS 18001, and Healthcare Systems Engineering. Additionally, he has been active with the Michigan Quality Leadership Award (MQLA) which is the Michigan-level Baldrige process and is an Accreditation Board for Engineering and Technology (ABET) trainer and program evaluator. Munro is the co-author of the ISO/TS 16949 Answer Book, published by Paton Professional.