by Duke Okes
Financial auditors often apply Benford’s Law to look for potential signals of fraud. This law states that in certain data sets, there will be more small values (based on the leading digit) than large, resulting in a distribution that is more triangular instead of normal or rectangular. For example, approximately 30 percent of house numbers can be expected to begin with a one and fewer than 5 percent will begin with a nine, rather than a 10 percent chance for each as one might expect.
This statistical principle can be used by auditors to look at the individual dollar values for a large number of invoices received from suppliers or of payments made by accounts payable. If there is a large variance from the expected triangular distribution, further digging can then be done to see whether there is actually a problem.
Applying Benford’s Law to quality auditing–Distribution analysis
Quality auditors could apply a similar concept by using distribution analysis during their audits. While Benford’s Law may be less likely to apply for quality applications, auditors could alternatively establish a baseline distribution against which to compare future analyses. The analysis would look for shifts in the shape of the distribution, such as average; range; or standard deviation, skew, or truncation. Significant shift might indicate a change in how a process is being managed, such as resource availability or competency, or a change in focus or priorities, which may be worth investigating further.
Here are some potential examples:
- Look at the number of personnel on staff indicated to be qualified to perform specific roles. A shift might indicate high turnover in a specific area, difficulty getting people trained/qualified for those roles, or a lack of cleanup of the records.
- Computer access control. Looking at the number of people signed into specific computer application at a specific time could indicate inappropriate access to that application.
- Supplier ratings. Look at the number of suppliers achieving each performance-rating level. A shift might indicate a change in general supplier performance or a change (intentional or unintentional) in how the rating process has been applied.
- Calibration/maintenance. Look at the number of units calibrated per week of each quarter. A shift might indicate constraints on calibration and/or maintenance resources, difficulties getting access to equipment needing calibration or maintenance, or fudged records.
- Process capability. Look at reported Cpk values, for which a shift could indicate improvement or degradation of actual process performance or fudged values.
- Corrective actions. Look at the number of days required to close out corrective actions. A shift might indicate when the system has become overloaded with real or too many insignificant problems or when efforts to improve the process have been successful.
Much of the data required for such analyses could be extracted from the organization’s enterprise resource planning or quality management system databases. Chi-square or other statistical analysis could also be applied to detect statistically significant shifts, although a visual analysis may be better for picking out smaller details on one specific portion of the distribution analysis.
Note that any changes identified may be good or bad, as well as intentional or unintentional. However, the idea of distribution analysis is to use another source of data to help auditors make better decisions about performance of the management system and where efforts would be better spent increasing audit depth.
About the author
Duke Okes is a knowledge architect who provides guidance for management system design, assessment of those systems through metrics and audits, and the use of root cause analysis to address performance issues. He’s an ASQ fellow and author of Root Cause Analysis: The Core of Problem Solving and Corrective Action (ASQ Quality Press, 2009). He can be reached at www.aplomet.com.
Tags: distribution analysis.