Effectiveness, Conformance, Improvement – Every Audit, In That Order

By Matt Leiphart

Good, better and best conceptThere are internal audits that lead a third-party auditor to check the box and state, “Well done, you’re doing internal audits as required. Let’s move on to the next thing on the schedule.” Then there are internal audits that add value. Value-added internal audits assess process effectiveness, conformance, and improvement, in that order.

Effectiveness means achieving business objectives. It is primary because if our processes are not effective, customers are probably unhappy and our costs are rising. We risk the entire organization disappearing from beneath our feet.

Conformance is necessary because we want to know the processes in operation are being performed as designed. Success should come by design, not by accident, and that’s the real reason why internal audits investigate conformance (not because the standard says we must).

Regarding auditing for improvement, you may exclaim, “But ISO 9001:2008 doesn’t require internal audits address improvement!” Yes it does. Section 8.5.1 states, “The organization shall continually improve the effectiveness of the quality management system through the use of…audit results…” Looked at another way, which is more fun – finding nonconformance or finding clues towards the next big breakthrough?

Are We On the Path to Success? Auditing Effectiveness Through Objectives

Auditors and audit programs that focus on conformance miss the most important aspect of any organization. Unmet objectives are a cancer within management systems. Regardless of whether or not a system complies with a registration standard, the first question has to be, “Is the system meeting the objectives determined by top management?”

While management system administrators may be attracted to conformance, putting effectiveness first ensures the future of the organization, and brings top management actionable information to steer the organization to greater success.

Doing a good job auditing effectiveness requires time with top management, and research. Objectives change over time, they evolve as top management refines its focus, as the business environment changes, and as operational risks and opportunities become visible. To audit effectiveness, first learn about the established objectives. This requires more than just obtaining a copy of this year’s objectives. Arrange time with top management to understand why the objectives have been established. Get a handle on the measurements that are being used to determine if the objectives are being met. Determine which processes link to specific objectives. Identify who is leading the efforts to meet the objectives. Time spent with top management to understand the ins and outs of every objective will give you deep insight into what to look for while performing the audit.

Don’t be tempted to audit effectiveness without doing the research. You may get an audit done, but it will not provide value to the organization. You will simply nibble around the edges of effectiveness, and waste your valuable time (and the auditees’ valuable time). Find the courage to get in front of top management and show them objectives are as important to the audit process as they are to the bottom line.

The process for auditing objectives differs from auditing for conformance. The outcome of auditing effectiveness is not a list of nonconformities. In essence, what you are looking for is evidence that processes are making progress towards achieving measurable objectives within the determined timeframes. If processes are not on track to meet objectives, a good internal audit will reveal the reasons why objectives are at risk. It is not uncommon for people to withhold reasons why they are not achieving objectives. Most often there are factors in play that are restraining the process from achieving the desired level of performance. It will take courage to ask the questions that will discover what’s necessary. Auditors who ask the difficult questions, listen to responses, discover what’s happening, and report the situation to top management clearly and without judgment will give the organization valuable information necessary for success.

Remember, organizational survival is not mandatory. Aiding achievement of objectives is an audit’s most important contribution to survival (and profit).

Conformity – Various Sources, Unbalanced Requirements

Auditing conformity is the typical focus of a management system audit. It’s easy and comfortable for most auditors. The standards are well established, the procedures are clear, and there is nothing like visiting a work area and watching confident people do a job well.

Avoid the trap of complacency while auditing for conformance. Just like everything else in life, change is constant, even in the world of conformance standards. Pay attention to changing requirements, whether they are changes to regulations, statutes, customer specifications, procedures, or international standards. Rather than switching on autopilot and asking last year’s audit questions, look for changes and/or upgrades to processes. Have changes been made that are not reflected in procedures? How does the organization roll out process changes when requirements change? Have people been added, changed, or removed from processes, and how was work adjusted so requirements are looked after?

Conformance requirements look the same on paper, but they should not be treated the same in every organization. Some requirements are more important than others depending upon the organization and its scope of operations. (This is a key concept within the planned changes to ISO 9001 and other standards as a result of a document called Annex SL recently released by ISO. More on that in future articles.) Therefore, audits must emphasize specific requirements more so than others. ISO 9001:2008 section 8.2.2 addresses this directly by stating, “An audit programme shall be planned, taking into consideration the status and importance of the processes and areas to be audited…”

Would it make sense for a four-person research and development organization to address training processes the same way a large package sorting operation would? Would the package sorting operation’s training processes become even more important while hiring a platoon of temporary workers in advance of the holiday rush? Differences in scope, operational risk, even the cadence of operations as a contract or program evolves must be considered when planning audits for conformity.

Don’t let complacency regarding requirements lure you into the same old audit every time. Focus on changes while keeping an eye towards what’s most important.

Improvement – Imagination is Key

How many times have you had a good idea that just got dropped because day to day pressures demanded your attention? It’s likely folks in your organization have also had great ideas fall by the wayside. Internal audits provide a conduit for uncovering improvements, and giving them a voice with the people who hold the reins on resources.

Unlike most of the auditing process, auditing for improvement requires an open-minded, possibilities-focused approach. Instead of searching for specific evidence related to a specific act, auditing for improvement requires an active imagination framed by a search for effectiveness, efficiency, opportunity, and gaps that may exist between today’s performance and what’s possible.

Do unnecessary steps exist in procedures that are not required for conformance, and do not contribute to objectives? Are there simpler and quicker ways to perform procedure steps? Identify those issues in your audit report and recommend they be addressed.

Keep your ears open. Just about everyone working in a process has an idea for how to improve it. Ask the question. Identify who had the great idea. Your audit report is a viable method to get information back to management so formal projects can be initiated to bring those great ideas to life.

Don’t be shy about suggesting improvement. Unlike issues of conformity, which are a clear and present risk to operations, management may decide not to implement every idea for improvement. That does not mean your efforts are not appreciated and are not worthwhile. Even if only one of fifty ideas documented in audit reports gets implemented, that number would be zero if the audit process did not include improvement in the scope.

Value-Added Auditing is Not a Stretch

Many organizations consider the auditing process as a necessary cost of doing business. A well rounded approach to auditing provides value, and is well worth the cost. Auditing for effectiveness gives top management a grass-roots view of how likely we are to meet objectives. Customers, lawyers, regulatory agencies, and process owners will appreciate the confidence you give them by checking on conformance. And bringing back a few great ideas you saw while looking for improvements locks it – you’re audit just added value for many interested parties.