by Andrew Barham
Auditor training courses, auditing books and auditing standards all say that when you conduct an audit or an inspection you should use a checklist. And they’re right of course, because how else would you remember what to ask and what to look for? Checklists assist us to know what to ask, and what to look for as an auditor. Generally, they aim to ensure consistency, but checklists are not without their issues. Some auditors may become too reliant on their checklist; and the ability to be consistent comes at the expense of being agile and responsive.
Below, we list the benefits of checklists and the issues that can arise from their use in more detail, then we explore practical tips to use for your next round of audits.
When we refer to audits and audit checklists, this equally (perhaps even more so) applies to inspections and inspection checklists. Why even more so? Well it’s because inspections are often performed more frequently, and so the issues occur more often.
And it does not matter what type of management system you are auditing. It could be a quality management system, an OHS management system, an environmental management system, a food safety management system, or an integrated management system – the issues are still the same.
But first, let’s remind ourselves of the reasons we use checklists and what is good about them.
- Memory jogger. Checklists remind you, the auditor, of the questions to ask, what you have to cover, and the evidence you need to see. It is difficult to forget to ask to see or review something if it’s stated on your checklist.
- Provides consistency. When different people or organisations conduct audits or inspections, checklists helps ensure that everyone asks the same questions, looks for the same evidence, and covers the same scope.
- Continual improvement. If additional information is required, the criteria changes, or if questions need improvement, checklists can be modified and distributed; then all the auditors will be aware of the changed requirements.
Checklists provide a level of consistency and accuracy during the audit that would be difficult to achieve without some kind of guide. However, checklist may also cause problems. We’re not saying, “Don’t use checklists. We just want to bring these potential issues to your attention and offer solutions too.
Many checklists are so big that the amount of time it takes to work through all the questions is greater than the time the auditor or inspector is given to do the job. We’ve heard of checklists with 500 questions, each requiring a definitive answer – and the auditor is given six hours to complete the job.
- The checklist just grows and grows. Over time things change, and as a result additional requirements (therefore new questions) are identified and added. That’s fine, however the reverse doesn’t seem to happen. It’s often nearly impossible to have a question removed. This means that as time goes by, checklists just grow bigger and bigger, which of course brings us back to the first point.
- Irrelevant questions. In some situations, certain questions are just plain irrelevant. They are not applicable and simply not needed. For example, a checklist for a vehicle inspection requires you to examine all four tires, when the actual vehicle being checked may be a motorcycle or, even worse, a boat.
- It’s just dull. If those doing the audit or the inspection have to use the same checklist with the same questions week in week out, they’re going to be bored. Put simply, they’re pretty sure they already know the answers, so there is a tendency for them to write exactly the same answer as last time.
- Tick and flick. Lots of checklists are full of closed questions which require only a yes/no or tick/cross answer, and this often produces a report with lots of checkmarks and very little evidence. In many cases, even the answer to an open question still produces a tick in the report, resulting in hardly any confidence that anything was actually looked at.
- Focus on completion. Because auditors are audited too they will often focus on ensuring they have answered all the checklist questions rather than looking at things properly. They fear that their audit report will be rejected if they did not cover every question.
- Scoring each question. Many organizations use some form of scoring for their audits where some questions or topics are worth more than others. While this has some benefit, it often results in those being audited only focusing on the highest-scoring questions, rather than all of their processes.
- Happy sheet. If the questions on the checklist are too simple and/or superficial, there is a chance that the receiver of the audit/inspection results will be led to believe that everything is compliant when in fact issues do exist.
- Order of the questions. If the order of questions on the checklist does not match the order in which the activities happen, it can result in activities or processes being modified to suit the checklist and satisfy the auditor.
So how do you stop these things from happening with your audit and your inspections? Here we offer some possible solutions to keep your auditors and inspectors fresh and alert in helping your organization remain compliant, effective, and efficient.
- Use more than one checklist. Rather than have one great big checklist, break it down into a number of smaller checklists and rotate them. This stops the tedium of asking the same questions each time and also results in smaller, more manageable and targeted audits.
- Write specific questions. If you need a specific answer or a specific piece of evidence reviewed, then be specific. For example, don’t ask, “Are the people competent?” Instead, “Choose two forklift drivers and record the numbers of their forklift licences and the expiry dates.”
- Give the auditor some freedom. If there is no specific answer to a question, allow the auditor freedom. However, steer away from the tick and flick. Don’t ask, “Are the machines suitable?” Instead, “How did they determine the suitability of the machines?”
- Keep it flexible. Allow questions to be marked N/A if they are not applicable. For example, there may be questions about confined space working, but at a particular audit site, there are no confined spaces.
In summary, by all means use checklists, but they need to be helpful and interesting. The questions have to be flexible when flexibility is allowed, and they have to be specific when a specific answer is required.
The focus should be on confirming that what is required to happen and what is actually happening are the same.
Also, allow enough time for the auditor to do the job properly. If the checklist is too big, then it can all be too rushed – and the audit report becomes yet another document to file away only to be looked at by yet another auditor trying to complete yet another audit checklist.
About the author
Andrew Barham is PricewaterhouseCooper’s director of auditor training and certification. For more information, visit auditortraining.pwc.com.au/.
Andrew is right in stating that checklists have drawbacks, but checklists are not appropriate for some types of audits and should not be promoted as a universal tool. In particular, in Australia especially, government regulators commonly include in planning approvals and resource licences, permits and leases etc a clause requiring a periodic compliance audit to be conducted against that regulatory document plus in many cases a number of additional permits and the management plans required by them. Lenders such as local and international banks also include such clauses in their loan agreements for major projects. This is especially the case for environmental compliance audits where the 500 question scenario is not uncommon. Auditors do not spend six hours doing the audit, a team of two or three certified auditors may be on site for up to five days, with additional commentary required under the regulatory documents from up to six technical experts. In one case in SE Asia I had five auditors on site for 12 days. The development of formal audit protocols is essential, and these sometimes need approval from the regulator, lender or client prior to their use. These are obviously very expensive projects, and the regulators and lenders assess the audits to make sure they have covered every requirement in the nominated documents. Internationally the audit report may also need to be submitted by the client or the lender to the national government and may then become part of the regulatory documentation with the audit recommendations becoming mandatory commitments. Both the auditors and the auditee organisation can be taken to task and even prosecuted for failing to do the audit as required. Checklists are fine for limited audits or inspections looking at a specific issue, especially for quality or safety audits where there are few if any permits with conditions attached against which to audit. However it is really important for both internal and external auditors to be given sufficient time and resources to do the audit properly. ISO 19011 is a good guide.