How to Know If Your Aviation Safety Management System Is Working

How to Prove Your Aviation SMS Is Working

Proving to stakeholders that your aviation safety management system is working is simply a matter of showing that the primary goal of SMS is being achieved - namely, that:

• Operational environment is safer; and
• Each employee is safer.

There are several prerequisites for being able to show that safety is being improved as a result of SMS and employee involvement:

• Your SMS has actually demonstrated safety improvement;
• Employees are actually reporting issues;
• Your SMS has a good or improving reputation in the company;
• Risk controls are actually improving risk in the operational environment;
• Implementation plan results each year; and
• Audit performance each year.

If you cannot show consistency or improvement in these areas, it’s a good indication that your safety management system is not working optimally. In all but highly developed safety programs, continuous improvement in all of the above areas should be what you look for. In well-developed programs, consistency will be more the norm for effective safety programs.

Why Analyze the Effectiveness of Your SMS

Analyzing whether or not your SMS is working optimally can be a bit nerve-wracking. What if you find out that it is not performing well? How does that reflect on you?

Identifying to what degree your SMS is working requires setting your ego aside (easier said than done). This practice reflects well on management and shows a willingness to do whatever it takes to improve safety.

If you can show that your SMS is working, then you have an extremely powerful tool for:

• Selling skeptical employees on the benefits of participating;
• Generating inclusion in the SMS through showing the “popularity” of the safety program;
• Getting more budget (if applicable);
• Getting even more participation in safety;
• More efficient SMS induction when onboarding new employees; and
• Improving Norms and other aspects of safety culture.

Fear of finding out your SMS is not very good aside, you have every incentive to analyze the effectiveness of your SMS. Here are 6 charts to see baseline SMS performance.

How to Assess Whether or Not Your SMS Is Working

There are roughly 6 areas of your SMS you want to review for whether or not your SMS is working. Depending on your operations, you may need to tweak this list to best fit what is most relevant to your safety.

1. Show an increase in reported issues over time (or at the very least, a consistent number of reports);
2. Show increase in perception of SMS (via safety surveys);
3. Show a decrease in the average risk of reported issues;
4. Show a decrease in the number of injuries, serious and/or minor, or if injuries are not quite applicable to your operations, show an increase in proactive reporting (i.e., issues reported before hazard occurrence);
5. Increase in SMS implementation each year; and
6. Increase in SMS audit results (i.e., decrease in the number of findings).

Taken together, these metrics help you determine at a high level whether your SMS is accomplishing what it sets out to do: improve safety.

Related Articles on Monitoring Aviation SMS Performance

• 7 Charts to Monitor Employee Safety Performance in SMS
• What Employees Should Be Reporting
• 10 Great Reports to Monitor Safety Performance in SMS - With Examples

Show Number of Reported Issues Over Time

The first thing you want to review is hazard reporting volume over time. Ideally, your hazard reporting practices should either:

• Have a steady increase in the number of reported issues over time, or
• For well-implemented SMS, have a consistent number of reports over time.

You use this metric to show that employees are participating. Otherwise, it simply sends the message that safety is being improved for other reasons besides employee involvement.

Show Data on Safety Surveys Over Time

This metric isn’t entirely necessary, but it is a powerful way to show that the SMS is being adopted by your organization as a whole.

Ideally, you will have a metric, such as a line chart showing an increase over time in survey questions regarding employee positive regard for the SMS. An increase in popularity only further justifies your case that your SMS is:

• Active operations;
• Being adopted by the organization; and
• Clearly has mechanisms (promotion) that make it desirable to be adopted.

Of course, to be able to prove such a metric, the following requirements must be met:

• You have been consistently performing safety surveys;
• Your safety surveys have been using the same questions over time; and
• The survey answers are quantifiable.

Surveys with explanation-based answers (such as “fill in the blank”) will not allow you to create a chart.

Related Aviation Safety Survey Articles

• 30 Good Safety Survey Questions
• How to Use Safety Surveys to Improve Safety Culture in Aviation SMS
• 5 Characteristics of Effective Aviation Safety Cultures - With Free Survey

Analyze Average Severity of Reported Issues

Having a chart showing the percentage of reported issues over time based on risk level is a very effective way of showing that your SMS is working. Essentially, it proves that your risk controls are:

• Improving the operational safety of the environment; and
• Employees are practicing better safety awareness, making their own safety better.

Consider the example:

• May 2017:
  • Green/acceptable issues: 62%
  • Yellow/mitigate issues: 30%
  • Red/Unacceptable issues: 8%
• May 2018:
  • Green/acceptable issues: 74%
  • Yellow/mitigate issues: 24%
  • Red/Unacceptable issues: 2%

If you can show a chart that demonstrates numbers with improving Green issues, and decreasing yellow and/or red issues, then you know that your SMS is reducing exposure.

Review Workplace Injuries or Other Key Safety Data

Ultimately, SMS aims to reduce risk to people and reduce injuries or the potential for injuries. Having data to show that individual employees are safer as a result of your SMS is another effective way of showing that your SMS is working.

If you are an operator that works in inherently “injury” filled environments, such as maintenance or ground ops, you would want to have a chart that shows:

• Decreased lost-time injury;
• Decreased injuries requiring hospitalization; and
• Decreased minor injuries (though if you show decreased severe injuries, this might not be necessary).

You might even supplement this chart with an actual story from one of your employees showing a situation where the SMS helped them operate safely where they would have otherwise been injured.

Obviously, not all service operators have departments that are inherently dangerous in terms of bodily harm. Consider Flight Ops, where injuries are rare but the potential for catastrophic injury is real when you consider things like fatigue and other Human Factors.

In this case you may substitute injury metrics for other key safety charts, such as:

• Reduced safety issues from fatigue;
• Reduced safety issues from miscommunication;
• Fewer work days missed due to sickness/illness;
• And so on.

The point is simply to demonstrate how SMS improves individuals' lives in the workplace.

Review Audit Performance and Implementation Progress

The last two ways to show that your SMS is working need little explanation. It’s as simple as this:

• Can you show an increase in implementation over the past year (assuming you are not fully implemented)?
• Can you demonstrate audit performance with fewer findings year over year?

If you can show progress in both areas, it is a strong sign that your SMS is being properly implemented and is functioning as designed.

Last updated May 2024.