by Duke Okes
Maturity matrixes are quite popular as they allow an organization to assess its degree of development in a specific area. Identification of strengths, weaknesses, and opportunities can then lead to improvement actions, with the matrix then also used as one way to measure that improvement.
Googling the term “maturity matrix” results in more than 24,000 hits. Topics found include maturity of project management, process control, safety, property asset management, reliability management, retail energy management, governance, and marketing. Although perhaps not widely used, the 2009 edition of ISO 9004 also includes a matrix for assessing sustainability of an organization using a quality management perspective.
Because the subsequent Google search returned very few hits for the term “audit maturity matrix,” I decided to use my many years of training auditors as well as speaking and writing on the topic to develop a proposed first draft (see table, and thanks to Larry Whittington for his input). Although the viability will differ depending on factors such as industry sector (e.g., the degree that regulatory bodies affect the design of quality management processes), management philosophy (e.g., the valuing of audits as a management control), and organizational culture (e.g., involvement of many versus a few), organizations should be able to use it to find ways to make the internal management system audit process a more valuable function.
Other potential criteria that could be included in the matrix include:
- Effect on auditors. What effect, if any, does participation as an auditor play in development of the individual (for more on this, see “Where Do Your Auditors Go?” The Auditor, July–August 2008).
- What initiates the audits? Are they driven primarily by a schedule (e.g., a push-driven audit), based on actual performance of the management systems, or at the request of a process owner (e.g., a pull-driven audit)?
- Audit checklist. Is the checklist a standard one used over and over, is it created each time to randomize the audit sample of the system, or it is randomized but with specific criteria included based on previous/current performance of the system?
- Correlation of audit results and organizational outcomes. Do audit findings correlate well to actual performance of the processes being audited? That is, does the number and significance of audit findings predict actual performance of the processes audited (e.g., as based on performance metrics, root cause analysis findings, etc.)?
Note that there is nothing here about process-approach audits; this is intentional. Although designing and implementing a management system based on the process approach certainly makes sense (e.g., what are the inputs and outputs; the required activities, resources, and controls; and interrelationships with other processes), an audit by clause of the standard (see sidebar) is no better or worse than an audit that follows the flow of the product or service and branches out into those interrelated processes. One could make the case that conducting audits from different perspectives, rather than from a single perspective, actually improves the probability of detecting shifts in the management system.
Regardless of how an organization manages the audit process, it’s important to realize that the process constitutes a significant part of the controls used by an organization to detect process gaps that can turn into performance problems. Therefore, it’s important that the audit process itself be continually evaluated for effectiveness and improved so as to be deemed a valuable contribution to the enterprise risk management process.
|Internal Audit Maturity Matrix|
|Area to Assess||Level 1||Level 2||Level 3|
|Focus of audits||Measure compliance only||Also evaluate the degree to which intended outcomes are achieved||Audits also intended to identify operational effectiveness opportunities|
|Frequency of audits||Infrequent (e.g., annually)||Frequent (e.g., quarterly or monthly)||Continuous (layered audits and/or computerized monitoring of activities)|
|Auditors||Most conducted by a single individual||Several personnel but primarily from department responsible for the system (e.g., QA for a QMS)||Cross-functional, including members of multiple departments and levels of personnel|
|Organizational view of audits||Negative||Accepting||Seen as a positive learning process.|
|Integration of audits||Audits look at one management system at a time||Audits look at multiple systems but separate from governance/risk/control (GRC) audits.||Overall organizational audit process is fully integrated and reports to the chief audit executive.|
|Use of metrics||Track number of nonconformities||Also look at audit efficiency and effectiveness||Trends in organizational metrics are used to drive audits as well as measure their long-term effect.|
|© 2014 Duke Okes|
Audit by clause
Imagine an organization with 6,000 instruments in its calibration system and working to accuracies of 0.00001 inches. It would make absolute sense for the organization to occasionally conduct an audit focusing only on ISO 9001:2008 clause 7.6, Control of monitoring and measuring equipment, and not necessarily looking at training of the calibration personnel, control of the calibration procedures, etc. After all, not only is calibration a process (inputs are measurements needed; outputs are calibrated devices provided to the organization to perform these measurements), but training and document control are longer cycle processes (performed less frequently) while the use and potential damage to and/or changed calibration status of the devices is a much shorter cycle time.
About the author
Duke Okes has worked in the quality profession for more than 35 years, first as a quality engineer with TRW Automotive, and since as an independent consultant, trainer, speaker, and author. He has worked with organizations from many sectors, including education, financial, government, healthcare, and manufacturing. He is an ASQ Fellow, is ASQ CMQ/OE, CQE, and CQA-certified, and holds degrees in technology, business, and education. He is the author of Root Cause Analysis: The Core of Problem Solving and Corrective Action and Performance Metrics: The Levers for Process Management.