The SMS Auditors Are Coming!

Aviation SMS audits are necessary to ensure aviation service providers comply with established guidelines. These established guidelines may come from:

• Clients;
• FAA;
• Transport Canada;
• EASA;
• IS-BAO;
• IATA; or
• Regional civil aviation authorities.

Apprehension naturally grows in safety managers and department heads as the inevitable audit date approaches. Being audited is similar to taking final exams at college; however, much more is at stake with SMS audits. Aviation service providers risk losing their operating certificates if they do not comply.

This article describes an aviation SMS audit and how to prepare your organization for the inevitable day. There will be some best practices thrown in. We'll also discuss a common SMS design flaw that catches many medium to small operators.

Related Aviation SMS Audit Articles

• How to Create an Aviation SMS Audit Plan
• How to Conduct Internal SMS Audits in Aviation Industry
• 4 Things Safety Managers Do to Perform Well on Aviation Safety Audits

What Is an Aviation SMS Audit

An aviation SMS audit is a thorough examination following a defined checklist. These checklists are typically evaluating:

• Safety Policies;
• Safety Accountabilities & Responsibilities;
• Reactive Risk Management Procedures;
• Safety Goals & Objectives;
• Safety Performance Monitoring Activities;
• Proactive Hazard Identification & Safety Risk Assessments;
• Management of Change Procedures;
• SMS Training;
• SMS Documentation;
• Emergency Response Plans (ERP);
• Safety Communication; and
• Safety Promotion Activities.

Aviation SMS audit checklists may be developed independently by the auditing organization (in the case of your clients), or come from a recognized authority, such as IATA, EASA, or Transport Canada. Although an audit follows a systematic checklist, there is considerable room for subjective opinions in the evaluation process.

Before your company undergoes an SMS audit, you should prepare yourself by first understanding "what" is being audited. The best strategy for understanding the requirements is to obtain the aviation safety audit checklist that the auditor will be using.

Difference between Internal Auditors and External Auditors

External auditors are independent personnel assigned by the regulatory agency or the auditing agency. These auditors are employed to assess and evaluate the SMS of aviation service providers that are either under their jurisdictional control or that are providing services to their companies.

It is probable that your company audits vendors or suppliers that provide goods and services under an ongoing contract. In this case, your QA department are external auditors visiting your vendors and suppliers.

External auditors differ from internal auditors in that they do not work for the organization that is being audited. However, internal auditors and external auditors often assess the same elements in their SMS audits.

One of the best preparation tips for external audits is to have your internal auditors audit your organization using similar evaluation standards. If you cannot get an aviation SMS audit checklist from the auditing organization, download some industry-accepted aviation safety audit checklists. We have a few available in this article.

Related Aviation SMS Audit Articles

• Real Difference between an Aviation Safety Audit vs. Inspection
• Audit Checklist: 10 Things to Prepare for Aviation SMS Audits
• How Often Should You Conduct Aviation SMS Audit

What Role Do Auditors Play for Aviation Safety

Basically, your external auditor will be testing and observing whether your company is complying with items in the respective checklist. Evidence may be gathered by:

• Interviews with employees and managers;
• Reviewing SMS documentation;
• Visual indicators of safety promotion; and
• Discussions with top management.

Some operators hire aviation SMS consultants before the expected audit date to coach line managers and top management. Your company may benefit from an SMS coach when:

• You are new to the SMS auditing process;
• Your "part-time" safety manager is over-worked; or
• When your certificate is in jeopardy.

Don't expect your SMS coach to be a miracle worker. SMS coaches cannot be expected to fabricate:

• Risk management documentation;
• Safety training records; or
• Safety promotion documentation.

Even though your SMS coach cannot provide miracles, an SMS coach can provide a plan to present to the aviation SMS auditors. A plan to address known shortcomings is better than no plan at all. A prepared corrective action plan demonstrates a proactive attitude.

Your SMS corrective action plan should include:

• Known shortcomings;
• Who is assigned to deal with these shortcomings; and
• Targeted due date to implement relevant corrective actions.

Corrective action plans can be managed in an audit management system or spreadsheets. If your managers typically procrastinate until the last minute, an audit management system will keep them on track. The audit management system allows operators to manage internal and external audits by:

• building your own checklists;
• scheduling audits;
• conducting audits;
• managing findings and concerns; and
• generate reports (even trend analytics).

Another benefit of an audit management system is that they allow operators to conduct pre-audit assessments using the same auditing forms as the auditor. When internal auditors discover SMS shortcomings, they can flag them as internal audit findings and assign responsible managers to address the shortcomings. Automated email notifications help keep the team on track.

Related Aviation SMS Audit Articles

• 5 Ways to Prepare for Aviation SMS Audits
• How to Think Like an Aviation SMS Auditor
• How IATA Operational Safety Audits Enhance Aviation Safety Process

Independent Auditors Don't Dictate Corrective Actions

When your independent auditor inspects your organization, there may be a few findings or concerns. Or there may be many findings in extreme cases. An independent auditor is not there to tell you how to correct deficiencies. That is not their task. Independent auditors come to audit to determine whether there is evidence that pre-defined standards are being complied with.

If your auditor is providing too much guidance on how to rectify findings or concerns, then this would no longer be an "independent audit." The auditor would have an investment in the process.

Aviation Safety Audits Should Not Be a Surprise

If you are new to being audited, you will be running around as if your hair is on fire the first couple of times.

• Relax.
• Have your documentation prepared and easily accessible.
• Don't offer more than what is being asked for.
• Tell everyone expected to be present that the audit is coming.
• Coach users as to what they should say based on their role or responsibilities in the aviation SMS.

Different SMS Auditors, Different Results?

SMS Auditors are individuals and each has their "pet peeves." Some may focus on policies and procedures, while others may focus on risk management strategies.

After you passed an audit with a few findings, you will naturally address these findings and properly document your organization's actions. However, don't be surprised that the next time, entirely new findings will arise in a different subject area.

I believe SMS auditors focus on different elements based on aviation service providers' SMS maturity level and perhaps how well they like you. Yes, audits are not entirely objective.

In your early years of the SMS implementation process, you may not be expected to have a fully developed process for proactively identifying hazards and monitoring risk controls.

This is entirely understandable because new SMS implementations are focused on:

• SMS Policies & procedures;
• Safety accountabilities;
• Gap analysis;
• SMS implementation plans;
• Reactive risk management strategies; and
• Initial SMS training.

Related Aviation SMS Implementation Articles

• How to Evaluate Risk Controls and Risk to Aviation SMS Implementations
• How to Implement Effective Control Measures
• Is Your Aviation SMS Implementation a Farce? - With Self-Assessments

More mature SMS implementations are expected to have more mature, demonstrable safety risk management (SRM) and safety assurance (SA) processes in their SMS. Therefore, it makes sense for SMS auditors to focus on:

• Proactive hazard analysis;
• Safety risk analysis;
• Hazard identification training;
• Internal and external auditing processes;
• Recurrent SMS training; and
• Predictive analysis (for more complex operators).

Smaller operators with as few as 30 to 40 employees may be asked to show their hazard registers and how they monitor risk controls. From what I've seen, the oil and gas industries are the most demanding when auditing aviation service providers' SMS. They commonly want to see proof of:

• Hazard identification activities (such as the top 5 or 10 hazards' monitoring activity);
• Safety risk analysis on individual hazards that they may randomly choose;
• Documentation of risk controls;
• Proof of monitoring risk controls; and
• Management of change activities for particularly high-risk projects.

Understanding SRM and SA Risk Management Processes

SMS is a process that works well when you have the tools and processes to work the process. After the third or fourth year of SMS implementation, it becomes clear from an SMS audit whether the operator either:

• Understands and has SRM and SA processes implemented in organizational risk management processes;
• Has no desire nor inclination to practice SRM and SA processes; or
• Lacks the tools or processes required for performant SMS implementation.

Related Articles on SRM in Aviation SMS

• 4 Elements of Safety Risk Management (SRM)
• Relationship between Management of Change, SA Process, and SRM Process
• How to Implement SRM Process in Aviation SMS [With Free Checklist]

The above image is common in FAA literature describing the interaction between safety risk management and safety assurance processes. As the image indicates, the SRM side is the "design" phase and the SA side is the "performance monitoring" phase. Performance monitoring is important to provide "assurance" that the design is sound and working as designed.

Most Common SRM and SA Process Design Flaw That Auditors Catch

The SMS safety reporting system's reported issues and audit findings make up the bulk of "Data Acquisition" in SA processes. Understanding how this works early in the SMS implementation phase is important to prepare the organization for effective SMS data management strategies.

Operators with more than 100 employees often find themselves scrutinized heavily in phase 4 of their implementations. They are expected to have a mature or maturing SMS. They are evaluated on how effectively they

• monitor their "system design; and
• evaluate the effectiveness of risk controls.

SRM & SA Data in Different Data Management Systems

Demonstrating effective performance monitoring becomes difficult, and usually is not sustainable when safety managers have their SRM Design data management tools in a separate system than the SA Performance Monitoring data management tools. Let's take a quick example of a sad, but common scenario:

Joe Dillijunt is the safety manager of XYZ Airways and believes he has made it to phase 4 of their SMS implementation after four years of hard work. Safety reporting metrics are adequate. They are averaging seven safety reports per month for 120 employees and Joe believed the safety culture is improving.

Joe knows that a hazard register is expected for mature SMS implementations, so he found a spreadsheet template from the civil aviation authority's website.

The spreadsheet template had all the necessary columns, such as:

• Hazard name and description;
• Worst Credible Hazard Consequence (Outcome);
• Severity of Hazard Consequence;
• Likelihood of Hazard Manifestation;
• Risk Index (calculation of likelihood and severity);
• Mitigation for unacceptable risk assessment (additional risk controls);
• Residual Severity of Hazard Consequence (after mitigation);
• Residual Likelihood of Hazard Manifestation (after mitigation);
• Residual Risk Index (after mitigation);

So far, so good. Joe fills out the hazard register and may even have managers review the Hazard Register spreadsheet in their annual hazard review process.

A year passes.

Safety reports are coming into the safety reporting system, but the hazard register sits abandoned in a folder on the Shared Drive. As safety reports and audit findings enter the reactive and proactive risk management process, there is no way to easily document which hazards are associated with the reported safety issues and audit findings.

Joe Dillijunt has an SRM-SA data management problem.

Related Aviation Risk Management Articles

• Difference between Reactive, Predictive and Proactive Risk Management in Aviation SMS
• How to Practice Reactive, Proactive, and Predictive Risk Management in Aviation SMS
• From Reactive to Proactive Hazard Identification in Aviation SMS

As you can see, there is a disconnect. Large airlines and airport authorities have quality management systems that can be easily adapted to link these identified hazards with reported safety issues and audit findings. However, smaller operators have been grabbing these spreadsheet templates that do not integrate with the reactive risk management processes. The result is that operators are unable to demonstrate effective SMS performance monitoring processes that directly link the system design processes to the system monitoring processes.

Now you can see why a spreadsheet SMS is definitely the wrong technology for aviation service providers that have more than 100 employees. Furthermore, this explanation demonstrates the futility of using stand-alone point solutions in tandem with the spreadsheet hazard register. Point solutions are single isolated systems designed to perform one task, like

• safety reporting
• SMS training management; or
• auditing.

The proper technology to integrate SRM and SA data is an integrated SMS database designed specifically to address ICAO SMS requirements. The number one tip from this article is to use an integrated SMS database to prepare for the SMS audit.

I've been an SMS database architect since 2007. I understand aviation SMS requirements and auditors love SMS Pro because it hits the right notes. There is seamless integration with SRM and SA, which is what auditors really love to see. We won't mention the other high points here, but SMS Pro addresses all SMS requirements, and not just SRM and SA.

When we hear auditors tell our clients that SMS Pro is the best-designed system they have seen, this is why. You can easily monitor the effectiveness of risk control just as easily as you can identify trends.

An SMS database is a very inexpensive technology. What is more expensive? SMS consultants that leave you with spreadsheets. Beware of this trap.

Final Thoughts on Aviation SMS Audit Preparation

Safety managers take SMS audits very personally. After all, an SMS audit is measuring the safety department's effectiveness in complying with standards. Audit results with many audit findings will indicate to management that the safety team is not doing its job.

But more importantly, the accountable executive may not be prepared for bad audit results. There may be an important contract that is in jeopardy due to poor audit results. Every accountable executive is responsible for making sure the aviation SMS is properly implemented and performing in all areas of the organization. Furthermore, the accountable executive is responsible to review organizational safety performance on a regular basis. As we have seen, poor SMS data management design strategies make monitoring SMS performance impossible to sustain year after year.

Audit results with few audit findings may indicate to management that they can relax and perhaps not devote as much energy to monitoring and improving the SMS. These may be false indicators that may not truly reflect the health of your SMS.

Well-meaning and hard-working safety managers may still have audit findings because an SMS' scope is organization-wide and not simply an audit of the safety department. An SMS audit involves the entire organization.

You may be able to fake a "demonstrable SMS" a few times, but the best recourse is to accept the fact that SMS audits are here to stay and you may as well bite the bullet and implement the SMS as it is designed.

You may not be able to justify spending more money on safety due to the finite production limits that generate revenue for the company to survive. In this case, you will need a paper SMS or spreadsheet SMS to demonstrate compliance as best as you can. You may get by with a paper SMS for three or four years before the operating certificate is revoked. This may be enough time for the company's situation to change. Oddly enough, this may be the accountable executive's business strategy.

It is up to safety managers to align their expectations with the accountable executives. Otherwise, safety managers will become disillusioned and feel their contributions to organizational success are unappreciated. If the accountable executive is only expecting a paper SMS, do your best to pass SMS audits with the tools at hand; however, you should explain the risk to the accountable executive.

For the rest of the safety managers that don't work in toxic or unhealthy safety culture, implement your SMS correctly and always be prepared. Keep your SMS documentation up to date. Continually incorporate best SMS practices into daily operations. This will save everyone considerable frustration when SMS auditors visit.

Related Aviation SMS Performance Monitoring Articles

• How to Monitor Aviation SMS Performance - Safety Chart
• My Safety Score - What's Yours? Aviation SMS Performance Monitoring
• How to Monitor the Effectiveness of Control Measures

Do you need auditing tools?

• Checklist manager?
• Audit scheduler?
• Offline audit management?
• Manage findings? Track corrective and preventive actions?
• Automated notifications for overdue items?
• Easy report generation?

SMS Pro's Audit Management System was developed specifically for the aviation industry.

Last updated October 2023.