by Richard A. Vincins

The internal audit process for an organization should not be a stand-alone process, but one that is integrated into the quality management system (QMS). The integration of auditing into the QMS assures that the QMS is being maintained and that it aids or provides assistance in the improvement of the system. Too often internal audits are completed only because regulations and standards say that organizations must complete them.

Auditing is an essential tool in the quality system to challenge the organization, find the nonconformities before registrars and investigators, and provide a basis for correction and continual improvement. This article will break down the various processes in a QMS to understand how audit results can be incorporated into the overall system assessment. Thus, we can try to use this process to integrate the audit results into the QMS, use its tools for success, and understand that this process isn’t something that we have to do only as part of compliance.

There are some fundamental reasons that internal audits are poorly performed or not performed by an organization:

• No trained or qualified staff; not enough resources to conduct independent audits
• Lack of available time to conduct the audits
• No audit preparation
• Audit observations are poorly constructed or poorly written
• No root cause analysis performed on the audit observation or the observation is misunderstood or incomplete

Companies are doing a disservice to their QMS by not performing internal audits and learning from them. Some of their reasons may be justified—they might not have the proper personnel or availability of personnel to perform the audits. This should be an indication to the management team that the organization needs to either dedicate the resources or have outside services assist in its internal audits. We will discuss how the audit observations can be properly written to ensure that corrective action and root cause analysis is performed.

Reporting audit nonconformities

Audit nonconformities or observations need to be written clearly, concisely, and in enough detail that the auditee can understand the issue. Many audit observations aren’t detailed enough. There should be no feelings or opinions in the audit observations, only fact-based statements to the nonconformity observed. The sidebar at the bottom shows an example of an acceptably written audit nonconformity and then an example of a more-descriptive statement that provides proper detail and what includes exactly was observed. The only difficulty with detailed nonconformity statements is that the auditee may fix only the item that’s described instead of conducting root cause analysis to ferret out the source of the nonconformity. If there is only one isolated incident of a nonconformity, then the detailed observation is correct, but advise the auditee to be aware of any similar types of occurrences. A good approach is to cite one or multiple observations with a supporting finding statement.

The audit report is another component of a QMS that includes the objective evidence of audit observations and audit findings. The audit report shows systematic evidence that internal audits are being performed by the organization. Submit the audit report or a summary of the audit report to the management team for communication of nonconformities in the QMS.

One of the main purposes of the audit report is to initiate the corrective action process. Ensure that there is a location on the audit report for determining if corrective action must be done; if no corrective action is completed, write a rationale explaining why it wasn’t. Write audit reports to communicate audit results to the management team, initiate corrective action for the deficiencies observed, and provide enough detail for follow-up activities to be performed.

Creating corrective actions from audit results

Auditors aren’t responsible for completing corrective actions; they are responsible only to identify the need for them. Instead of making this decision based solely upon opinion or “gut feeling,” the audit team can use risk assessment to determine if corrective action should be initiated. The two components of assessing the risk posed by a nonconformity is its effect (severity of occurrence) and magnitude (probability of occurrence). Incorporating risk assessment into the QMS should be an ongoing project. A risk-level approach for corrective action assures that significant issues are addressed properly and that the organization doesn’t spend its resources trying to fix insignificant problems.

Now that you have an audit observation or finding from the process you just audited, you can determine its effects and magnitude. The effect of an audit nonconformity depends on the severity of the observation or finding. The effect or severity of an audit nonconformity applies to products, processes, people, operators, or end users.

When determining the effect of a nonconformity, use a standardized severity scale or one developed internally by the company. An example of an effects scale is shown in figure 1. Ensure that the determination of the effect of the nonconformity is in proper context for the quality system. Assessing the effect of the audit nonconformity on the QMS assists in determining its severity and demonstrating a clear need for corrective action to be initiated.

Figure 1: Example impact table for audit nonconformity

Effect or severity	Description
Negligible	There is minimal risk of injury or failure to system, process, and/or operator.
Minor	The potential observation could result in a non-serious injury or failure to the system, process, and/or operator that does not require intervention.
Significant	The potential observation could result in a non-serious injury or failure to the system, process, and/or operator that requires intervention.
Critical	The potential observation may result in serious injury or significant failure that requires intervention.

The second aspect of assessing audit nonconformities is their magnitude or level of occurrence. During the audit process, the auditor will determine if an observation is an isolated incident or a chronic systemic problem. The audit’s scope may need to be expanded or extended to assess the magnitude or how often the same or a similar nonconformity is occurring. Magnitude is based on probability of occurrence. Probability of occurrence can be linked to various statistical techniques, including sampling plans or probability tables, depending on the product or system requirements. The basis for determining magnitude is to see how pervasive the audit nonconformity is within the QMS.

When the effect and magnitude of the audit nonconformity is established, this information can be tabulated to initiate corrective action. Similar to a risk priority number (RPN), this calculation can be made for generating a corrective or preventive action (CAPA), as shown in figure 2. If the value is one, two, or three on the CAPA response determination table, no CAPA is required; if it’s a five through 10, a CAPA may be required (include rationale if a CAPA isn’t issued); a score of 15 to 50 requires a CAPA.

Figure 2: CAPA response determination table

			Effect scale
			Negligible	Minor	Significant	Critical
			1	3	5	10
Magnitude scale	Continuously	5	5	15	25	50
	Frequently	3	3	9	15	30
	Significant	2	2	6	10	20
	Critical	1	1	3	5	10

Audit results that need attention are brought into the corrective action process for multiple reasons. These reasons including tracking the correction of the audit nonconformity, determining root cause, conducting the corrective action, and following up to ensure that the corrective action was effective. An immediate re-audit may be necessary depending on the risk assessment from the effect or magnitude calculation. Completing the risk assessment of audit results allows the organization to communicate properly to the management team and provides methods for assessing the severity of the nonconformity.

Review audit results in management review

The audit results and subsequent corrective actions are communicated to the management team so it can make decisions on the QMS. Completing a risk assessment for the audit results provides the management team a level of risk associated with the audited processes. This allows the management team to adjust resources, prioritize activities, and provide team members the time to work on high-impact audit nonconformities. The audit results must be unbiased and independent to allow the management team to make proper decisions. The audit results can be used as a barometer for the organization to identify high-risk areas that must be prioritized. The management team has the responsibility to review audit results, corrective actions, quality objectives, and other processes of the QMS to make decisions or changes.

The data presented in the management review must be summary information, which is best shown in graphical format to see trends or significant items. Measureable audit results allow the management team to properly make decisions on the QMS. There are various methods to quantify audit results, such as an average of the risk assessment for a process audit or to monitor the number and/or types of observations that are made. Summarize the audit results with recommendations to allow the management team to re-prioritize the audits or reschedule audits for problem areas. Make sure that the audit results are fact-based and not “touchy feely.” If there is too much detail in the summary, the real issues will get lost. The information presented from audit results must generate changes to the quality system driven by the management team.

Taking actions based on audit results

Management reviews are conducted to determine if the QMS is being maintained, has effective processes, and creates opportunities for improvement. Careful attention to audit nonconformities in conjunction with an effective corrective action system creates opportunities for positive changes. The basis for audit nonconformities can also be extended to supplier audits and third-party audits. The audit results from supplier audits can be introduced back into the supplier management program to assess the current status of a supplier. Audit results from notified bodies or registrars are excellent opportunities to improve the QMS. The same risk assessment for supplier audit nonconformities and external third-party audits can be applied to understand significant issues within the quality system. All of these processes are reviewed and summarized as part of the management review to make appropriate changes to the QMS.

Conclusion

There are many opportunities to integrate audit results and learn from nonconformities to maintain the effectiveness of the QMS. Audit nonconformities must be communicated to the relevant individuals and summarized as part of the management review process. Just as the overall QMS is viewed as a continual improvement process, the auditing process is another cog in this improvement process to assess, change, and improve QMS processes. Use risk assessment for audit nonconformities to determine the effect and magnitude of audit findings. Audit findings that are categorized as higher risk can initiate the corrective action system to assure that root cause analysis, investigation, corrective action, and follow-up activities are performed. The internal audit process is another tool in the arsenal of quality and regulatory professionals to assure the QMS is meeting the requirements of national and international regulations.

<<Insert sidebar>>

Optimizing audit results: Sidebar

Good—Better: A Closer Look

Good: The quality control department procedure for final testing of finished infusion pumps requires that all final test inspectors be trained in operation of the software validation.  A second-shift tester had no training records for this operation.

Better: Training could not be verified for second-shift quality control tester No. 58, who was performing software validation test No. 5 on series 80 infusion pumps. Procedure P-1465 part 2.2 requires documented training. (ISO 13485, 6.2.2(e))

Findings

• Management review procedure isn’t implemented
• The last six corrective actions show no activity and are at least a year old
• Final inspection is ineffective in preventing defective components from being released

About the author

Richard A. Vincins is a Certified Quality Auditor and regulatory affairs consultant with Emergo Group, a global medical device consultancy with headquarters in Austin, Texas. He is responsible for the implementation of quality systems, conducting internal audits, training on quality system tools, and providing regulatory expertise in national and international regulations. Vincins has more than 19 years of experience in the medical device industry, including worldwide regulatory compliance efforts for in vitro device, medical device, and pharmaceutical companies.

Tags: audit results, nonconformity.