by J. P. Russell
Auditor requirements, job duties, responsibilities, knowledge, and skills need to be updated from time to time. Auditors must stay up with the times to be effective and earn their keep.
About every five years, American Society for Quality (ASQ) Certified Quality Auditors (CQAs) participate in a survey to update the CQA Body of Knowledge (BOK). Late in 2011, experts in the field of auditing were interviewed and 2,500 CQAs worldwide were surveyed. The data generated by the survey capture current trends and changes in the auditing profession that keeps us current and prepared for the future. An obvious example of the need to stay up-to-date is the ongoing changes in communication technology.
The data collected from the survey are analyzed and used to update The ASQ Auditing Handbook and the CQA BOK to ensure they are relevant. The BOK is the basis for developing the ASQ CQA examination test questions. The BOK can also be used as the basis for auditor job requirements.
The updating of the BOK includes general topics that must be integrated into existing sections and new, stand-alone sections that cover new or greatly expanded topics such as risk management and external audit program management.
BOK update: General
Since 9/11, security has become an important concern, along with safety and the environment. Auditors should consider security issues when planning audits and report security issues during the audit. When there is potential for harm, there should be individual (as well as organizational) security procedures or guidelines. Safeguarding electronic documentation may be a security concern. The importance of security will vary from organization to organization.
Another general issue is to consider risk when planning and performing an audit. ISO 19011:2011 lists audit performance risks that should be addressed. The first risk to address is in preparing the audit plan (clause 126.96.36.199), the audit team leader should be aware of the risks to the auditee organization created by the audit. For example, the presence of audit team members could influence health, safety, environmental, or quality controls. Someone could be sick, get injured, cause pollution, or interfere with an inspection. During the audit, auditors may observe potential risks that need be reported to management. Audit program managers are expected to identify and manage risks.
An audit may uncover that a supplier or partner has falsified information or is acting irresponsibly. The auditing organization may advise its management to cancel any existing contracts or agreements with that particular organization and find more reputable and socially responsible sources for the item or service. Social responsibility may be a consideration when selecting and maintaining relationships.
The global economy and corresponding supply chain requires auditors to better understand and deal with cultural diversity. This includes regional verbal and nonverbal customs, as well as different languages and social views. To ensure audit objectives are achieved, auditors may need to be familiar with different cultures.
Remote or eAudits
Advances in communication technology have opened up the opportunity to conduct remote eAudits. The 2012 update of the BOK does not include a special section on remote audits, but instead integrates guidance into existing audit process sections. The following points were taken from eAuditing Fundamentals: Virtual Communication and Remote Auditing, by J. P. Russell and Shauna Wilson.
The opening meeting should include communication equipment, software, and internet load access speeds as well as protocols for microphone, mouse, desktop, and keyboard controls. It is recommended using a webcam or other means so that opening meeting participants can be viewed. There should be agreement on the scheduling methods and the collaborative meeting program.
For remote or eAudit exit meetings, it is recommended to either record the exit meeting using video equipment or give participants keyboard and mouse controls to enter their names. The auditor can use the same form to record the minutes of the meeting.
Remote or eAudits have additional information and document control challenges. An individual auditor’s computer may have limited security and be subject to hacking. The auditee and auditor should agree on how auditee information is stored and how electronic files are handled.
Remote audit techniques provide an opportunity to follow up corrective actions in a timely and economical manner. There is no need to wait until the next scheduled audit to verify the corrective action. Perhaps the criticality of the corrective action implementation requires immediate verification.
More service industries are using auditing as a management tool so the BOK needs to be more service-industry friendly.
New or expanded BOK sections
External audit program BOK Part IV.A.6 (supplier audits)
With the establishment of global supply chains, customer and supplier relationships have never been more important or critical to the success of the organization.
Audit program managers need to interface with procurement to ensure contracts contain physical and virtual access clauses and to understand procurement needs to schedule audits or other oversight service of the global supply chain. Oversight may be needed for first, second, and perhaps third-tier suppliers depending on organization objectives, customer requirements, and risk. Interruption of the supply of critical components provided by second- and third-tier suppliers could have a significant effect on the customer’s operations.
The audit group needs to work closely with the procurement department to select, evaluate, approve, and monitor suppliers through audits, qualification surveys, and surveillance. Auditors need to understand “flow down” requirements to ensure requirements are being met. Are there technical requirements such as chemical composition or performance tests? Are there process requirements such as management system certification or process variation objectives? Are there logistical requirements such as storage conditions or special packing? Are there administrative requirements such as first aid instructions or the purchase order number visibility outside the packaging? Auditors may be needed to monitor the entire logistics network including transportation modes, warehouses, special services to maintain product, and distribution centers.
Auditing may be used as one tool to help determine supplier performance. Is the supplier maintaining the required management system? Are product or service requirements achieved? Are contract requirements being met? When and how many audits are needed? Can eAudits be conducted in place of onsite audits, and so on?
The audit program manager should ensure that there are procedures, policies, and a schedule to support the supplier management program.
Best practices: BOK Part IV.A.7
The implementation of best practices should be just as important as correcting nonconformities. Some organizations may call “best practices” good practices, noteworthy achievements, or best in class.
Audit program managers need to analyze audit results to standardize best practices and lessons learned throughout the organization. Auditors or audit program managers may identify best practices by analysis of area measures or metrics or by observation during the audit. When planning or conducting the audit, they may review performance measures to note outstanding performance in a certain area or process. During the audit, auditors may hear comments regarding a best practice. For example, “This process has been error free for six months compared to weekly errors before the change.” Auditors can verify their best practice observations during the audit and record it.
Organizational risk management: BOK Part IV.A.8
Ten years ago the word “risk” was seldom used in the quality field. Now it is used in many situations where there is an element of uncertainty regarding a desirable outcome. Audit program managers should analyze how the audit program affects an organization’s risk level. Conducting audits of an organization’s processes should lower risk levels. Perhaps there is a lower risk of product or service failure or a lower risk of nonconformities and decertification of the management system. Perhaps organization risks can be lowered by greater oversight of critical processes or monitoring the effectiveness of risk treatments.
Organizational actions to lower risk levels could influence the number and frequency of audits performed. Conducting fewer audits or performing remote audits instead of onsite could reduce costs.
From a job description perspective, individuals may be required to either assess and treat risk and/or monitor and observe risk. Assessing risk involves identification of potential risks and evaluations of their significance. Monitoring and observing risk involves oversight of known risks and reporting potential risks. The two duties, assessing vs. monitoring, require different competencies.
From an individual audit perspective, auditors may need to treat or mitigate risks related to the audit. Auditors need to be aware of their auditing risks associated with taking representative samples, influencing outcomes due to simply being present, violating safety or other work rules due to not being familiar with the audit site or industry regulations, safeguarding information, and so on.
Common causes: BOK Part V.D.1
Under the quality tools and techniques part of the BOK, a section was added about causes of variation. When auditors understand the difference between common causes and special causes they will be better able to promote ongoing improvement. If the reason for all problems and nonconformities are special causes, there will be no improvement because it will be considered a one-time cause that is unlikely to recur. Special causes are not considered systematic. Special causes of problems or nonconformities may only require correction or remedial action.
People tend to classify the cause for a problem or nonconformity as a special cause because it is easier to fix and takes less resources in the short term. My experience is that the causes of most problems or nonconformities are common causes. Common causes are systematic and can only be fixed if the underlying cause is eliminated (change the system). If the underlying cause is eliminated, the problem or nonconformity will not recur due to the same cause and will conserve resources in the long term.
Outliers: BOK Part V.D.3
Another section added to the tools and techniques part of the BOK is a discussion of outliers. When a set of data includes an outlier, the analysis of the data can result in misleading conclusions and wrong decisions. Outliers can be embedded in a set of data making them difficult to identify.
The dictionary defines “outlier” as a statistical observation not homogeneous in value with others of a sample. An outlier is a special case of a special cause. An outlier is a data point that deviates markedly from the other data points collected or in the sample. An outlier is a result of a special cause, such as using the wrong test equipment or pulling the sample from the wrong bin. A data point identified as an outlier is abnormal and if not removed from the database will result in skewed, misleading, or false conclusions. If outliers are not addressed, the integrity of the data may be questioned and considered inconclusive or unreliable.
Deletion of outlier data may be the correct thing to do, but this is a subjective judgment. The practice of deleting outliers is frowned upon by many scientists due to the potential of researchers manipulating statistical data for their own self interest. If the cause of the outlier data point is known, it should be verified before removal from the database. When data points are excluded from data analysis, the rationale should be clearly stated in any subsequent report.
Auditors need to be aware of the existence of outliers and possible effects on conclusions drawn from observations and samples.
Risk management tools: BOK Part V.H
There are various methods for managing risk: risk avoidance, mitigation, and trade-offs. Auditors need to understand the options for managing risk. If the combined probability and cost of an undesirable outcome is unacceptable to management, the risk must be addressed. An organization can decide not to perform the activity or action and therefore avoid the risk. This can include subcontracting the activity or action to another organization better suited to perform it. The risk can be mitigated by implementing additional controls such as testing or periodic audits of the activity. Another treatment would be to approve an alternate activity or action that would achieve the same objective.
There are several tools for identification of risks and their assessment. Some of the tools are failure mode and effects analysis (FMEA), hazard analysis and critical control points (HACCP), critical to quality (CTQ) analysis, and health hazard analysis (HHA). Auditors need to be familiar with these tools so that they can verify if controls are effective for managing risk.
One of the valuable lessons organizations have learned in this modern era is that we cannot keep things the same. Our business and social environment keeps changing, and we must continue to adapt or risk being left behind. Many of the changes to the CQA BOK that may affect an auditor’s job requirements could also apply to many other positions within an organization.
About the author
J.P. Russell is founder and managing director of QualityWBT Center for Education, an ASQ Fellow, ASQ-certified quality auditor, voting member of the American National Standards Institute/ASQ Z1 committee, member of the U.S. technical advisory group for International Organization for Standardization technical committee 176. He is the editor of The ASQ Auditing Handbook, fourth edition, (ASQ Quality Press, 2005) and author of many articles and several ASQ Quality Press books, including eAuditing Fundamentals: Virtual Communication and Remote Auditing.