by Denise Robitaille
Audit reports are the product of audits. Without audit reports, audits are incomplete. The information contained in the audit reports will be used by others to make decisions that affect the entire organization. Audit reports provide insight into what processes and functions are working well, perceptions of risk, and identification of what has gone wrong. From these insights will flow corrective actions, preventive actions, lean initiatives, benchmarking activities, and an array of improvement projects.
Internal audits are essential inputs into the management review process. Some of the decisions that occur during strategic planning may be a direct result of audit findings. Therefore, what goes into the audit reports matters.
As with other chapters, the primary focus is internal auditing. However, it’s appropriate to make a brief comment about supplier audits. Any time an auditor conducts a supplier audit, it’s appropriate to send the supplier (or potential supplier) an audit report. It’s unfair to only send a list of requests for corrective action. The organization has extended you the courtesy of its time and deserves to experience the benefit of a complete assessment report. A supplier audit report contains information that will be used to decide whether the company will be added to the approved supplier list. Additionally, it may contain information about capacity, unique processes, or areas of concern that may need to be addressed either through corrective actions or through a joint improvement project with your organization.
The essential fact to keep in mind is that the audit report needs to be informative and it must provide value. It’s important to write a comprehensive audit report. The report doesn’t have to be lengthy, but it should convey a balanced summary of the status of the organization audited. It should mention good practices that have been observed, risks that have been perceived, and problems that have been identified.
The internal audit report should include the following information. (Again, supplier or third-party audits should contain comparable information.)
Date of the audit. When did the audit take place? This provides evidence that audits are being conducted in accordance with the established audit schedule. Frequent lapses in the schedule may be indicative of an erosion in the organization’s commitment to the internal auditing program. It might also reflect a problem in terms of resources. Are auditors being told by their supervisors that they can’t take time from their regular jobs? This might suggest a devaluation of the process by some middle managers. Or, it may simply be that there aren’t enough auditors in the audit pool and it’s impossible to get the audits done within the allotted time.
Another factor to consider is one of practicality. Auditing is an auditable quality management system (QMS) function. Your registrar or some other third-party assessor needs to verify that audits are being conducted in accordance with the plan and that audit conclusions are based on adequate objective evidence.
It’s also appropriate to record the duration of the audit. This helps top management get a clear picture of the resources that are being expended. How much time and money is the organization spending on internal auditing? This can be weighed against the cost-saving problem solving that resulted from good audit findings.
Areas audited. For an internal audit in a small company, this would be as simple as saying what departments were visited. However, a larger company could have an expansive campus with several buildings or multiple locations. In either of these cases, it’s important to record the location, especially if there are findings that the organization may wish to investigate further to see if they are localized or systemic. Conversely, if good practices are observed, it helps with being able to benchmark them later so that they can be applied in other parts of the organization. There’s also the possibility that an activity is conducted at more than one site so it would be important to ensure that the other sites are audited at a later date in the audit cycle.
Standard used. For a third-party audit, it’s a QMS standard such as ISO 9001, ISO/TS 16949, ISO 13485, etc. For internal audits, it’s usually a list of the internal documents associated with the functions and activities audited. Examples would include procedures and work instructions. In some organizations, the auditors are also asked to verify conformance to the applicable management system standard, so they would include reference to the applicable standard, for example ISO 9001:2008. When referencing the standard, it’s appropriate to include the revision year.
Lead auditor and audit team members. Every team has a lead auditor. If there’s only one auditor, that person is the lead auditor. That individual has the ultimate responsibility for generating the audit report. Other team members must be listed in the report, along with any technical experts who may have accompanied the team. They serve to provide very specific technical information that may exceed the knowledge of any of the auditors. These individuals generally aren’t used for internal audits unless the organization has some highly specialized processes.
Bearing in mind again that the audit process itself gets audited, having the names of the interviewees allows the auditor to confirm that none of the internal auditors audited their own work. When conducting surveillance audits I regularly ask what functions the auditors have in the company and compare them against the scope of the audits they have conducted.
Persons interviewed. This provides evidence that the persons who answered questions were actually the process owners who have responsibility for the activity. It’s not uncommon for people to try to be helpful and answer an auditor’s question even if it’s not part of their regular job. Sometimes the auditor finds out too late that he or she wasn’t speaking to the right person. During a closing meeting you may hear a manager say something like, “Francine doesn’t take care of patient intake, so she wouldn’t know where those forms are kept.” As awkward as it is for the auditor, it’s best to find out even this late in the process, so that corrections can be made and unwarranted findings of nonconformance can be removed.
Recording the names of persons interviewed helps auditors provide objective evidence that they’ve fulfilled the requirements of the auditing process.
Good points. An audit isn’t an attempt to amass a collection of bad events. Therefore, an audit report should also mention good points that were observed. “A newly developed software program is facilitating communication between departments on new projects,” “The records show evidence that operators have had training on the ERP system that was introduced three months ago,” or “The corrective action tracking system is better able to calculate the cost of nonconformities and the money saved when problems are solved.” These are all examples of positive comments an auditor might have observed. They serve as an objective indication that resources allocated are showing return on investment.
It’s also nice to acknowledge accomplishments and success stories. People are so accustomed to only hearing about the dreaded “NCs” that they don’t want to read the audit reports—much less appreciate them as opportunities to make things better. It’s gratifying to hear: “Well done.”
Observations (also called opportunities for improvement and often abbreviated OFI). It’s appropriate for auditors to make statements about perceptions of risk or the identification of a process that may not be controlled as well as it should be to prevent problems. They shouldn’t specifically say something is wrong, but they should intimate what might go wrong. Examples might include: “It was observed that the router for fast-turnaround jobs does not provide adequate instruction for certain steps, which could result in errors and defective products needing to be reworked” or “Nonconforming material waiting to be scrapped is stored in close proximity to customer property. Even though the material is tagged, there is a risk of the customer product being accidentally discarded.” Again, in each example, there is no nonconformance. There is, however, the risk that something could easily happen which could result in a problem or nonconformance.
Nonconformities. ISO 9000:2005—Fundamentals and vocabulary—defines a nonconformity as: “a nonfulfillment of a requirement.” When writing up findings of nonconformity, it’s important to be clear and complete. What is the actual nonconformity? What is the requirement? What evidence did you use to conclude that there was a nonconformity?
Let’s take each one of these in turn.
What is the nonconformity? This should be a clear, unbiased statement of fact. For example, “The inspection records provide evidence that material was accepted that exceeded the allowable tolerance range” or “There are no records to provide evidence that the report was reviewed by an authorized reviewer before being sub-mitted to the client.” Note that in neither case is there an accusatory tone or assignment of blame. In neither finding is it stated that a specific individual did something wrong. It’s important to refrain from using people’s names when writing up a finding of nonconformity. Also, remember that you can’t report what you have not observed. It would not be appropriate to say: “Michael passed material through that was out of spec” or “The report wasn’t reviewed.” You didn’t see the operator accept defective product and you don’t know that the report wasn’t reviewed, only that there’s no record.
It’s important to be starkly factual. It’s possible that there was an engineering deviation issued and so the issue isn’t with Michael, but with the function that was responsible for providing the record of the deviation. And, the report could have been reviewed and approved using a new electronic signature that hasn’t yet been documented in the procedure. In both cases, something isn’t right. The manner in which you write it up will determine if they’ll chock it up to “operator error” and “re-train the operator.” Or, if someone will ask why there was an error and look for the true cause, thereby preventing recurrence—or escalation to a more serious occurrence next time.
This isn’t the time to suggest solutions or to presume to know the cause. The statement should not include language that says, “We should try doing this ______” or “Because the document revisions weren’t distributed…” The audit is an objective factual ac-count. What ensues from the audit report is up to someone else.
What is the requirement? It’s inappropriate—wrong, actually—to write up a finding of nonconformity that can’t be tied to a requirement. Without a requirement, all an auditor has is something he or she doesn’t like—and it’s irrelevant to the audit report. This justifies the finding. For the first example, the requirement is found in the customer specification. When citing a document, the auditor must be specific. The statement of requirement might read: “Drawing 7878993, Rev. F, calls out a tolerance of 9.75″ +/- 0.005″.The records indicate that the parts accepted measured 9.68.” For the second example: “Procedure 8.2.4, Rev. C, specifies in section 7.7 that all reports must be reviewed by a second independent reviewer and that the reviewer must sign and date the last page of the report.” Details like revision levels are also important. They sometimes shine a light on the fact that people don’t have the right information—steering the investigation away from the unfortunate and ubiquitous “operator error” root cause.
Having specific information as to the requirement provides several benefits. It demonstrates why there is a nonconformance. This, in turn, dispels any confrontations as to the legitimacy of the finding. And, finally, it helps to launch the root cause analysis when it comes time to investigate the problem.
What is the evidence? The third factor to include is the evidence that substantiates the finding. For the first it would be Inspection Report #15549 from August 30; for the other it would be the Service Report #546 from September 15. This reinforces the justification for the finding and also facilitates the root cause analysis process.
An additional factor can be added if it’s deemed appropriate. I call it the “So What?” factor. This is a brief statement of the reason this problem needs to be addressed. With both of the examples given it would be appropriate to say that the risk is that customers will get defective product. Other concerns might include timely rework or possible regulatory action.
For most audit reports there should be fewer findings of nonconformity than good things or opportunities for improvement.
The last thing to put into the report would be the results of any verification of open corrective actions. This can relate back to earlier statements about improvements that have been observed. Because the audit function owns responsibility for ensuring action is taken on audit findings, this efficient method serves to close the loop on previous audits.
The auditor’s working papers (checklist, notes, samples, and closed out corrective actions) should be either appended to the report or available for review.
Following these basic audit practices should ensure that the information management gets is accurate, reflects the status of the organization, and is detailed enough so that it results in good decisions. This is what makes audits effective. Anything less is a meaningless paper shuffle.
About the author
Denise Robitaille is a member of the U.S. TAG to ISO/TC 176, the committee responsible for updating the ISO 9000 family of standards. She is committed to making your quality system meaningful. Through training, Robitaille helps you turn audits, corrective actions, management reviews, and processes of implementing ISO 9001 into value-added features of your company. She’s an Exemplar Global-certified lead assessor, ASQ-certified quality auditor, and ASQ Fellow. She’s the author of numerous articles and several books, including The Corrective Action Handbook, The Preventive Action Handbook, and her newest book, 9 Keys to Successful Audits, all published by Paton Professional.