What Is Aviation SMS Performance?

Aviation service providers around the world have been devoting considerable resources to implement formal aviation safety management systems (SMS). Each of these SMS implementations requires safety performance monitoring and measurement of critical data elements.

Clearly, many elements and risk management activities that makeup SMS' performance. Where does a safety manager start?

I’m sure you can rattle off many specific examples of SMS performance. What I’m really getting at is the core themes and principles that drive high-quality risk management performance in aviation SMS.

When it comes right down to it, we're talking about the safety management team’s ability to:

• Implement changes quickly (adaptability);
• Implement changes in the actual working environment vs. simply just documenting it; and
• Creating the working conditions for a mature safety culture.

SMS Performance Revolves Around Three Defined Areas

Adaptability, regulatory compliance, safety culture. Just about any performance metric, you can think of will fall into one or more of these areas. Aviation SMS Leading Indicators should be metrics that indicate what drives SMS performance.

One popular technique to assess SMS performance measures management's performance, in terms of responsiveness, of your SMS' risk management processes. How long, on average, does it take for your risk management process to run its full course?

Do you monitor this metric? Do you measure how long it takes to process a safety concern based on operational risk? For example, do you monitor the length of time to process a high-risk safety report versus less critical safety reports?

How many days does it take for your SMS to process safety reports from the time they have been reported until the time it takes to formally close the safety report? You can exclude the subsequent review process, as safety reports could be monitored for several years after they have been reported. We are more interested in capturing the time-lapse from when the safety report is submitted until the safety report is put into the "monitoring phase" where stakeholders and auditors can observe and report.

Related Articles for Leading Indicators in Aviation SMS

• What Are Leading Indicators in Aviation SMS Programs?
• How to Use Leading Indicators in Aviation SMS
• 8 Aviation Leading Indicators That Successful SMS Measure

Why the Average Days to Close Issues Chart Is Important

Our safety chart in question is a fantastic indicator of;

• how quickly management is able to respond;
• efficacy of your SMS' risk management processes; and
• implicitly how much it values safety culture.

For one, the assumption is that when issues are being closed quickly, management is demonstrating its ability to respond to needs and make appropriate changes without hiccups.

If not, then clearly management is struggling to be able to respond. Lack of responsiveness from management happens for several reasons:

1. Poor abilities on the part of management;
2. Apathy;
3. Crippling risk management processes that are not realistic based on the volume of safety reports; and/or
4. Resistance from employees for effecting changes.

Points 1 and 2 demonstrate how much management values safety culture, as apathetic management clearly does not value cultivating safety behaviors, and poor abilities demonstrate an unwillingness to try and actually improve.

Related Articles on SMS Performance Monitoring Charts

• How to Monitor Aviation SMS Performance - Safety Chart
• How to Monitor Aviation Safety Reporting Culture Using Safety Charts
• Safety Chart: Monitor Aviation SMS Contributing Factors

What This Chart Reveals About Your SMS Implementation

As a great example of a leading indicator, average days for closing issues is a great predictor of how well an SMS will be able to respond to safety events and successfully fix safety issues. It is implicit that when safety issues are closed, each safety issue’s Corrective and Preventative Actions is also completed and implemented.

There may be long-term, outstanding corrective actions that require undergoing a regulatory oversight approval process before the corrective action and related safety issue can be "officially closed." Often, six months or longer may be required to obtain regulatory acceptance or approval of newly implemented processes. These are exceptions that occur in a small percentage of your treated, reported safety issues. Whenever possible, your SMS database should accommodate these exceptions to maintain accuracy. You are more interested in evaluating your organization's SMS performance and you will want to avoid obfuscating the data that is inherently dependent on regulatory approval processes.

For this reason, this chart is such a great aviation SMS leading indicator, as it essentially measures:

• How efficiently management can put things (CPAs, changes, etc.) into action; and
• Managerial behavior in relation to risk.

Data about risk management behavior tends to be great for leading indicators because it is behavior that leads to mature safety cultures and additional proactive risk management activities. Fast closure times most often correlate to other indicative areas of risk management, such as:

• The ability to implement quickly indicates a receptive, responsive safety culture;
• Closing issues quickly means management is most likely meeting its performance goals; and
• Management’s willingness to close safety items quickly is a top-down indicator of the organization’s risk management attitude.

Monitoring an aviation SMS' performance in a similar fashion allows senior management to quickly take the pulse of the organization's safety culture. This becomes an excellent SMS performance monitoring chart that deserves a proper place in your SMS performance monitoring dashboard. A quick glance on a regular basis allows managers to determine:

• How is our safety culture?
• Are we responsive compared to other companies' SMS performance?
• Are our metrics reasonable based on risk assessment level?
• Do we need to improve "troubling metrics?"
• Are our risk management processes limiting our ability to be more agile and responsive?

Related Aviation Safety Culture Articles

• What Is Safety Culture in the Aviation Industry?
• What Does Aviation Safety Culture Look Like?
• Aviation Safety Cultures Improved With Integrated Safety and Quality Management Systems

How Is This Chart Created?

The Average Days to Close Issues chart takes into account several different elements:

• Safety issues grouped by their risk index (composite of probability and severity and summarily evaluated as acceptable, mitigable, and unacceptable);
• Date safety issue was reported;
• Date safety issue was closed; and
• Date range to evaluate safety performance data.

This chart simply tabulates the total number of days issues were open in each risk assessment level category and divides each number by the total number of safety issues in each category. For example, if “acceptable” issues were open for a total of 400 days and there were 100 acceptable issues, then the average closure date is 4 days.

This chart is automatically created with a professional, commercially available aviation SMS database, but can be created manually so long as the number of days each issue is open is being tracked. With an SMS database, there will be less work for safety managers as important, mile-stone dates are managed by the system and require no additional interaction from the safety team.

For safety managers using spreadsheets, the most important dates to capture are the date the safety issue was reported and the date the safety issue's status was changed to "Closed." While spreadsheets can drive this performance monitoring chart, your SMS review processes will be more sustainable using an SMS database designed to specifically address SMS documentation requirements.

SMS Performance Chart Enhancement

If you are reading this article, chances are that you currently use SMS Pro to manage SMS documentation. Most users will access this article from within the SMS database by clicking on the chart in the Executive Dashboard.

In 2019, there is a planned upgrade to this chart by comparing performance for the past twelve months with the previous twelve months. We believe this enhancement will add more value to this chart in that it will guide management's decision-making. With the enhancement, managers will be able to determine whether SMS performance is improving or declining.

Understanding which way SMS performance is trending is important for various reasons:

• Management may not want to implement drastic changes when the trend is improving;
• Management may elect to "do nothing" and continue monitoring when the trend is improving; or
• Management has actionable intelligence for fact-based, decision-making instead of basing actions on hunches or false perceptions.

How Does This Chart Relate to SMS Requirements

Because this chart is a fantastic aviation SMS leading indicator, it definitely falls into the category of identification, collation, analysis, and application of indicators to be monitored, which is part of the basic steps for establishing effective safety monitoring as developed by EUROCTONTROL.

Monitoring this chart is the first and most basic step towards establishing a baseline for management’s efficiency in facilitating risk management and demonstrating continuous improvement of the SMS.

Related Aviation SMS Continuous Improvement Articles

• What Continuous Improvement Compliance Looks Like in Real SMS
• FAA Part 5 Compliance | Safety Assurance Continuous Improvement
• 4 Pillars | The Truth About SMS Continuous Improvement and KPIs - With Free KPI Resources

Final Thought: Other Related Charts

Other charts that deal with risk assessment, while not necessarily leading indicators themselves, naturally complement this chart by giving it more context, such as:

• # Issues By Risk;
• # Unclosed Issues By Risk; and
• Average Day to First Haz. Identification (another great aviation SMS leading indicator).

Managers can easily monitor the safety performance of all employees when they have the proper tools. Take measured steps to improve your safety culture.

Last updated June 2024.