Aviation SMS audits are necessary to ensure aviation service providers comply with established guidelines. These established guidelines may come from:
Apprehension naturally grows in safety managers and department heads as the inevitable audit date approaches. Being audited is similar to taking final exams at college; however, much more is at stake with SMS audits. Aviation service providers risk losing their operating certificates if they do not comply.
This article describes an aviation SMS audit and how to prepare your organization for the inevitable day. There will be some best practices thrown in. We'll also discuss a common SMS design flaw that catches many medium to small operators.
An aviation SMS audit is a thorough examination following a defined checklist. These checklists are typically evaluating:
Aviation SMS audit checklists may be developed independently by the auditing organization (in the case of your clients), or come from a recognized authority, such as IATA, EASA, or Transport Canada. Although an audit follows a systematic checklist, there is considerable room for subjective opinions in the evaluation process.
Before your company undergoes an SMS audit, you should prepare yourself by first understanding "what" is being audited. The best strategy for understanding the requirements is to obtain the aviation safety audit checklist that the auditor will be using.
External auditors are independent personnel assigned by the regulatory agency or the auditing agency. These auditors are employed to assess and evaluate the SMS of aviation service providers that are either under their jurisdictional control or that are providing services to their companies.
It is probable that your company audits vendors or suppliers that provide goods and services under an ongoing contract. In this case, your QA department has external auditors visiting your vendors and suppliers.
External auditors differ from internal auditors in that they do not work for the organization that is being audited. However, internal auditors and external auditors often assess the same elements in their SMS audits.
One of the best preparation tips for external audits is to have your internal auditors audit your organization using similar evaluation standards. If you cannot get an aviation SMS audit checklist from the auditing organization, download some industry-accepted aviation safety audit checklists. We have a few available in this article.
Basically, your external auditor will be testing and observing whether your company is complying with items in the respective checklist. Evidence may be gathered by:
Some operators hire aviation SMS consultants before the expected audit date to coach line managers and top management. Your company may benefit from an SMS coach when:
Don't expect your SMS coach to be a miracle worker. SMS coaches cannot be expected to fabricate:
Even though your SMS coach cannot provide miracles, an SMS coach can provide a plan to present to the aviation SMS auditors. A plan to address known shortcomings is better than no plan at all. A prepared corrective action plan demonstrates a proactive attitude.
Your SMS corrective action plan should include:
Corrective action plans can be managed in an audit management system or spreadsheets. If your managers typically procrastinate until the last minute, an audit management system will keep them on track. The audit management system allows operators to manage internal and external audits by:
Another benefit of an audit management system is that it allow operators to conduct pre-audit assessments using the same auditing forms as the auditor. When internal auditors discover SMS shortcomings, they can flag them as internal audit findings and assign responsible managers to address the shortcomings. Automated email notifications help keep the team on track.
When your independent auditor inspects your organization, there may be a few findings or concerns. Or there may be many findings in extreme cases. An independent auditor is not there to tell you how to correct deficiencies. That is not their task. Independent auditors come to audit to determine whether there is evidence that pre-defined standards are being complied with.
If your auditor is providing too much guidance on how to rectify findings or concerns, then this would no longer be an "independent audit." The auditor would have an investment in the process.
If you are new to being audited, you will be running around as if your hair is on fire the first couple of times.
SMS Auditors are individuals and each has their "pet peeves." Some may focus on policies and procedures, while others may focus on risk management strategies.
After you pass an audit with a few findings, you will naturally address these findings and properly document your organization's actions. However, don't be surprised that the next time, entirely new findings will arise in a different subject area.
I believe SMS auditors focus on different elements based on aviation service providers' SMS maturity level and perhaps how well they like you. Yes, audits are not entirely objective.
In your early years of the SMS implementation process, you may not be expected to have a fully developed process for proactively identifying hazards and monitoring risk controls.
This is entirely understandable because new SMS implementations are focused on:
More mature SMS implementations are expected to have more mature, demonstrable safety risk management (SRM) and safety assurance (SA) processes in their SMS. Therefore, it makes sense for SMS auditors to focus on:
Smaller operators with as few as 30 to 40 employees may be asked to show their hazard registers and how they monitor risk controls. From what I've seen, the oil and gas industries are the most demanding when auditing aviation service providers' SMS. They commonly want to see proof of:
SMS is a process that works well when you have the tools and processes to work the process. After the third or fourth year of SMS implementation, it becomes clear from an SMS audit whether the operator either:
The above image is common in FAA literature describing the interaction between safety risk management and safety assurance processes. As the image indicates, the SRM side is the "design" phase and the SA side is the "performance monitoring" phase. Performance monitoring is important to provide "assurance" that the design is sound and working as designed.
The SMS safety reporting system's reported issues and audit findings make up the bulk of "Data Acquisition" in SA processes. Understanding how this works early in the SMS implementation phase is important to prepare the organization for effective SMS data management strategies.
Operators with more than 100 employees often find themselves scrutinized heavily in phase 4 of their implementations. They are expected to have a mature or maturing SMS. They are evaluated on how effectively they
Demonstrating effective performance monitoring becomes difficult, and usually is not sustainable when safety managers have their SRM Design data management tools in a separate system than the SA Performance Monitoring data management tools. Let's take a quick example of a sad, but common scenario:
Joe Dillijunt is the safety manager of XYZ Airways and believes he has made it to phase 4 of their SMS implementation after four years of hard work. Safety reporting metrics are adequate. They are averaging seven safety reports per month for 120 employees and Joe believes the safety culture is improving.
Joe knows that a hazard register is expected for mature SMS implementations, so he found a spreadsheet template from the civil aviation authority's website.
The spreadsheet template had all the necessary columns, such as:
- Hazard name and description;
- Worst Credible Hazard Consequence (Outcome);
- Severity of Hazard Consequence;
- Likelihood of Hazard Manifestation;
- Risk Index (calculation of likelihood and severity);
- Mitigation for unacceptable risk assessment (additional risk controls);
- Residual Severity of Hazard Consequence (after mitigation);
- Residual Likelihood of Hazard Manifestation (after mitigation);
- Residual Risk Index (after mitigation);
So far, so good. Joe fills out the hazard register and may even have managers review the Hazard Register spreadsheet in their annual hazard review process.
A year passes.
Safety reports are coming into the safety reporting system, but the hazard register sits abandoned in a folder on the Shared Drive. As safety reports and audit findings enter the reactive and proactive risk management process, there is no way to easily document which hazards are associated with the reported safety issues and audit findings.
Joe Dillijunt has an SRM-SA data management problem.
As you can see, there is a disconnect. Large airlines and airport authorities have quality management systems that can be easily adapted to link these identified hazards with reported safety issues and audit findings. However, smaller operators have been grabbing these spreadsheet templates that do not integrate with the reactive risk management processes. The result is that operators are unable to demonstrate effective SMS performance monitoring processes that directly link the system design processes to the system monitoring processes.
Now you can see why a spreadsheet SMS is definitely the wrong technology for aviation service providers that have more than 100 employees. Furthermore, this explanation demonstrates the futility of using stand-alone point solutions in tandem with the spreadsheet hazard register. Point solutions are single isolated systems designed to perform one task, like
The proper technology to integrate SRM and SA data is an integrated SMS database designed specifically to address ICAO SMS requirements. The number one tip from this article is to use an integrated SMS database to prepare for the SMS audit.
I've been an SMS database architect since 2007. I understand aviation SMS requirements and auditors love SMS Pro because it hits the right notes. There is seamless integration with SRM and SA, which is what auditors really love to see. We won't mention the other high points here, but SMS Pro addresses all SMS requirements, and not just SRM and SA.
When we hear auditors tell our clients that SMS Pro is the best-designed system they have seen, this is why. You can easily monitor the effectiveness of risk control just as easily as you can identify trends.
An SMS database is a very inexpensive technology. What is more expensive? SMS consultants that leave you with spreadsheets. Beware of this trap.
Safety managers take SMS audits very personally. After all, an SMS audit is measuring the safety department's effectiveness in complying with standards. Audit results with many audit findings will indicate to management that the safety team is not doing its job.
But more importantly, the accountable executive may not be prepared for bad audit results. There may be an important contract that is in jeopardy due to poor audit results. Every accountable executive is responsible for making sure the aviation SMS is properly implemented and performing in all areas of the organization. Furthermore, the accountable executive is responsible for reviewing organizational safety performance on a regular basis. As we have seen, poor SMS data management design strategies make monitoring SMS performance impossible to sustain year after year.
Audit results with few audit findings may indicate to management that they can relax and perhaps not devote as much energy to monitoring and improving the SMS. These may be false indicators that may not truly reflect the health of your SMS.
Well-meaning and hard-working safety managers may still have audit findings because an SMS' scope is organization-wide and not simply an audit of the safety department. An SMS audit involves the entire organization.
You may be able to fake a "demonstrable SMS" a few times, but the best recourse is to accept the fact that SMS audits are here to stay and you may as well bite the bullet and implement the SMS as it is designed.
You may not be able to justify spending more money on safety due to the finite production limits that generate revenue for the company to survive. In this case, you will need a paper SMS or spreadsheet SMS to demonstrate compliance as best as you can. You may get by with a paper SMS for three or four years before the operating certificate is revoked. This may be enough time for the company's situation to change. Oddly enough, this may be the accountable executive's business strategy.
It is up to safety managers to align their expectations with the accountable executives. Otherwise, safety managers will become disillusioned and feel their contributions to organizational success are unappreciated. If the accountable executive is only expecting a paper SMS, do your best to pass SMS audits with the tools at hand; however, you should explain the risk to the accountable executive.
For the rest of the safety managers who don't work in toxic or unhealthy safety culture, implement your SMS correctly and always be prepared. Keep your SMS documentation up to date. Continually incorporate best SMS practices into daily operations. This will save everyone considerable frustration when SMS auditors visit.
Do you need auditing tools?
SMS Pro's Audit Management System was developed specifically for the aviation industry.
Last updated October 2024.