by John R. Broomfield
An organization’s systems are the responsibility of its top management, so systems that require auditors to be effective are dysfunctional. Managers need to know how well their organizations’ systems are enabling them to determine and fulfill objectives and other requirements. Their systems should provide them with this information through conformance with the requirements of clauses 4.1, 5.5.3, 5.6, 8.2.3, and 8.4 of ISO 9001.
By demanding that auditors supplement the flow of this information through their audit reports, top managers short-circuit, debilitate, or at least weaken their systems. Instead of relying on their systems to help them identify improvement opportunities, auditees expect their auditors to report them. Independence is eroded as the management systems are changed to reflect the suggestions and dependency deepens as the system waits for suggestions from its auditors.
In the quest for value-added audits, too many .auditors are losing their independence. The systems they audit increasingly depend on them to report opportunities for improvement. This is an understandable reaction to the demand for more value from certification audits; however, did the International Accreditation Forum (IAF), United Kingdom Accreditation Service (UKAS), ANSI-RAB National Accreditation Board (ANAB), certification bodies, or registrars even try to sell the value of well-crafted nonconformity statements? No, most of us were silent on their value. Nonconformity statements remain misunderstood, feared, and even loathed by auditees and some auditors.
A few decades of this and the problem will be much worse. Registrars might be seen no longer as even slightly independent! This has already started with internal auditors, many of whom are asked to change the definition of audit itself from: “Systematic, independent, and documented process for obtaining audit evidence and evaluating it objectively to determine the extent to which audit criteria are fulfilled (per ISO 19011).” To: “Systematic and documented process for obtaining audit evidence and suggesting better ways of fulfilling audit criteria (per management preference).”
As auditors we owe our clients and auditees a duty not to short-circuit the systems we audit. Instead of reporting opportunities for improvement (OFI), we should be investigating why a system failed to help the auditee to identify and prioritize the improvement before the audit. Instead, we too often issue OFIs without concern for the systems’ hidden weaknesses. Still, we auditors understand the logic of saying “the only bad nonconformity is the one that remains unreported.”
Improvement, OFIs, and auditor independence: Removing the fear of nonconformity reports
At first we played the effectiveness card. We started talking about “auditing for effectiveness,” as if it’s better than auditing simply for conformity. In fact, it’s the same. Effectiveness is delivering products that meet customer requirements produced by processes that also meet requirements. Thus, ineffectiveness is the same as system nonconformity.
Instead, we should change the focus of our audits through our planning, demeanor, and questions. Focus instead on how well the system helps its users to meet current and future requirements. Plan your audits, sample, investigate, and report on the system in terms of its future performance. We owe it to our customers to see how well their systems help the users to:
- Convert customer needs into cash or continued funding (clauses 4.1 and 7.0)
- Anticipate the needs and requirements of customers (clause 5.4.2)
- Anticipate the needs and requirements of interested parties (clause 5.4.2)
- Plan the fulfillment of future requirements (clause 5.4.2)
- Prepare the organization to fulfill requirements (clause 7.1)
- Select staff so training will not waste resources for competence (clause 6.2.2 c)
- Design services even if the design of the goods comes from the customer (clause 7.3)
- Translate customer needs into product requirements (clause 7.3)
- Translate product requirements into satisfied customers (clause 7.5)
- Improve the performance of suppliers (clauses 8.4 d, 8.5.1, and 7.4.1)
- Work to meet requirements (clauses 5.3, 6.2.2 d, and 8.2.3)
- Monitor and correct processes (clause 8.2.3)
- Mistake-proof processes and products (clauses 4.1, 7.3, 8.5.2, and 8.5.3)
- Measure process and system performance (clauses 8.1 and 8.4)
- Obtain cash for investment (clauses 4.1, 7.0 and 6.1)
- Identify any improvement needed (clauses 8.1, 8.4, and 5.6)
- Invest in prioritized improvement actions (clauses 5.6 and 8.5.1)
- Sustain the business (clauses 5.4.1 and 5.4.2)
All of these goals are supported by the thoughtful use of ISO 9001, as the clause references in parentheses demonstrate. They are enough to make the audited organization’s leaders think and demand more from their systems, but registrars should consider these goals when selling their certification services (more on this later). The consideration of these goals during a registrar’s sales process may powerfully influence the extent to which the auditee applies clause 4.1 of ISO 9001 and its derivative system standards as it prepares for its stage 1 audit.
Considering the audit’s goals before it begins can also remove the fear of nonconformity, depending on how the auditee’s management reacts to news of system nonconformity. When it books an audit, a registrar should ask the organization’s management if it celebrates the absence of reported nonconformity. The registrar can then determine its likely reaction if there are nonconformities found. If an organization punishes its employees for nonconformities, help its managers realize the damage they do when making their people fearful of reporting problems. Take the sting out of nonconformity reports not by soft-peddling them, but by selling the value of well-crafted nonconformity statements.
Auditors add value by examining evidence of how well a system is helping its users to anticipate, determine, and fulfill objectives and other requirements. Auditors add value by delivering carefully worded nonconformity when the evidence shows that the system has failed to help the users to meet requirements.
Still, the value of a well-crafted nonconformity statement often remains suspect.
Improvement, OFIs, and auditor independence: Why do we nanny the systems we audit?
Auditors often think we’re doing a wonderful job when we report OFIs, even when the OFI is ignored. Keep in mind that the information that caused the OFI should have already been reported by the system we are auditing. Even so, we’re happy to issue an OFI without a thought for the system weakness that failed to report the opportunity before we arrived.
Systems are designed to help employees continually improve their ability to meet ever-escalating requirements, add value, and prevent loss. Value begins with the system itself.
Among their other requirements, systems should help their users continually improve. When an auditor sees a potential OFI, he or she should ask why the system failed to inform its users of this opportunity. Such a system failure is also deserves a well-crafted nonconformity report. Value-added audits expose system weaknesses, invoke corrective action, and strengthen the system.
Lead auditors should have been successful as senior managers. Such auditors won’t hesitate from holding top management accountable for demonstrating their commitment to their system for anticipating and meeting the requirements of customers, regulations, employees, and shareholders.
Through their planning, preparation, and demeanor, effective lead auditors can show auditees how well their system helps users to anticipate, determine, and meet customer requirements. They keep all of the audit criteria (as necessary to fulfill the audit objective) open and transparent to the auditee. This also means the auditor makes no attempt to impose his or her opinion or ideas onto the auditee. Effective auditors let the data do the talking.
Auditors who add value resist the temptation to impart improvement ideas or make recommendations. They see the dangers of supplanting the systems they audit. Instead, they determine why the system didn’t independently reveal the necessary improvement to the people in the system. How many auditors overlook systemic weaknesses by reporting specific OFIs? They ultimately damage auditees by increasing their dependency on their auditors’ improvement ideas.
This will continue to be the case while auditors are told that to add value, they must do what the system failed to: provide information on what needs to be improved. Auditors should stop propping up weak systems with their OFIs. Instead, we should be investigating why the systems we audit are failing to inform system users of needed improvement.
Improvement, OFIs, and auditor independence: The role of registrars
Registrars and certification bodies should confer an enhanced reputation on the organizations that they certify and the certificates of conformity they issue. Registrars improve the reputations of organizations they audit by avoiding all apparent and actual conflicts of interest thereby protecting their independence and the impartiality (and value) of their certificates.
Evidence of this commitment to the enduring value of their certificates should permeate every process of certification bodies. For example, before they’re hired, good registrars should explain and sell the value of their nonconformity statements to the systems and performance of their prospective clients. They should make it clear that their auditors will collect and evaluate evidence of the commitment of the organization’s leaders to the development, use, and improvement of its process-based risk management systems. They should explain that their auditors will collect and evaluate evidence of system effectiveness. In this way, they encourage managers to insist on rigorous audits and to view well-crafted nonconformity reports as opportunities for improvement. This approach helps the organization’s managers to not fear nonconformities. Managers who understand these concepts won’t accept OFIs from auditors because they will be more concerned about why their system failed and helping their employees other opportunities for improvement.
Until registrars (and their accreditors) hold themselves and their clients accountable to these standards (beyond ISO 17021), they offer little value beyond a marketing logo.
Improvement, OFIs, and auditor independence: Auditee reaction
Auditors often introduce themselves by saying, “I’m here to see how well your system helps you to do good work.” As a result, many auditees are initially surprised to hear that their system should serve them well, instead of the other way around. Note that ISO 9001:2000 removed the requirement for employees to conform to their procedures. Still, though, there are many nonconformity statements written as if employees are required to conform by the standard. Instead, auditors should be investigating why the system failed to ensure effectiveness and conformity. The following common responses may reveal the following system weaknesses or indeed strength:
- “My boss told me to do it this way.”
- “That procedure has never worked.”
- “I am part of a process improvement team.”
By focusing on how well the system helps people do good work rather than writing up failures to follow procedures, auditees will come to trust the audit process enough to speak openly about how the system needs to be improved and why it isn’t already being improved. They will understand the value of reporting system nonconformities and their leaders know they shouldn’t react badly them. Indeed, they even stopped celebrating the absence of nonconformity reports.
Most people understand the importance of understanding the requirements of standards, having the resources and controls (processes) to meet them, and then working to meet the requirements. They also understand that not all of the standard’s requirements are documented and that customer needs may remain hidden until they are elicited and revealed as requirements.
The job of product designers (for both goods and services) is to convert customer needs into product requirements (specifications, drawings, or images). Production personnel work to eliminate waste from the production process and add value by using the processes for meeting the product requirements. They succeed when they fulfill the requirements of people who invest in their ability to anticipate and meet customer requirements better that any other organization.
When the registrar, the auditor, and the auditee create this kind of improvement-focused environment, they welcome the reporting of genuine system weaknesses as nonconformity statements that require corrective action. These kinds of auditees also abhor suggestions from their auditors.
Conversely, after dysfunctional audits employees talk about how the auditor failed to report valid system weaknesses while their managers celebrate a lack of reported nonconformities. Who can blame employees for disengaging when they see their leaders gaming audits instead of facing up to system weaknesses?
Improvement, OFIs, and auditor independence: Conclusion
Committed leaders ensure that their management systems help employees and suppliers to add value for their customers (and shareholders) by anticipating, determining, and fulfilling requirements. These requirements include continual improvement.
Auditors add value by invoking corrective actions that strengthen the system when it’s failing to help its users to meet requirements and secure needed improvement. Ignored OFIs based on unknown audit criteria don’t strengthen the system. In fact, ignoring OFIs conceal the system weaknesses that created them and actively inhibit improvement.
Beyond auditors weakening the systems they audit, the drafters of ISO 17021 need to re-examine their legitimizing of system dependence and the damage done to auditor independence and the value of the certificate itself.
Sidebar: Improvement, OFIs, and auditor independence:
Four examples of opportunities for improvement that conceal system weaknesses
-
Area under review: Purchasing
Concern: PR-742 requires the company president to sign all purchase orders prior to their issue to suppliers and there is no delegated authority for their approval when the president is unavailable. This could cause problems for the purchasing process.
Recommendation: Review the authority functions in the purchasing process for ensuring consistent process flow success.
System weakness: Failure of the system to define the criteria necessary (defined authority—see clause 5.5.1) for controlled issue of purchase orders when the president is unavailable.
ISO 9001:2008 clause 4.1 (c)
Further investigation by the corrective action team may identify other system weaknesses in the form of single points of control.
-
Area under review: President’s office
Concern: Actions identified from management reviews not assigned to a person responsible for completing them.
Recommendation: Action log to summarize such actions
System weakness: Failure of the system to define responsibility of person authorized to complete the actions flowing from management reviews for ongoing suitability, adequacy, and effectiveness of the management system.
ISO 9001:2008, clause 5.5.1
Further investigation by the corrective action team may identify other system weaknesses regarding the assignment of responsibility and possibly the under-use of the preventive or corrective action processes to control changes the management system.
-
Area under review: System management
Concern: Management representative claims authority to make editorial, administrative, or clarification changes to system documents without going through full change control requirements, including revision level update. However, this authority is not defined in the document control procedure.
Recommendation: Update the document control procedure to provide management representative with this authority.
System weakness: Failure of the system to specify the criteria for making editorial, administrative, or clarification changes to system documents and changing the revision status without going through the full change control process.
ISO 9001:2008 clause 4.1 c
-
Area under review: System management
Concern: Heavy use of textual procedures
Recommendation: Consider flowcharting procedures instead of using text to explain them.
System weakness: Too many textual procedures could the result of (or result in):
- Failure to determine the processes in the system (see 4.1 a)
- Less awareness of how the system works (see 5.5.3 and 6.2.2 d)
- Failure to show the sequence and interaction of processes in the system if every procedure represents a process (see 4.1 b and 4.2.2)
- Failure to analyze processes (see 4.1 d). Text is not evidence of process analysis.
It could be that the auditor did not have time to investigate and obtain evidence of the possible nonconformity shown above. If so, the auditor needs some means of feeding the concern forward to the next audit with the knowledge of the auditee.
About the author
John R. Broomfield has been training system auditors since 1989. He is a certified lead auditor with IRCA and RABQSA. He designs and runs lively advanced auditing workshops for accredited registrars and develops process-based management systems that add value faster and prevent loss sooner while conforming to any number of system standards using a continual improvement methodology. He is a senior vice president with Quality Management International Inc. and can be reached at jbroomfield@aworldofquality.com.
Tags: improvement, auditor independence, OFIs, value-added audits.