Aviation Safety Software Blog by SMS Pro

How to Monitor Aviation SMS Performance - Safety Chart

Written by Tyler Britton | Apr 5, 2019 10:00:00 AM

Which Safety Charts Help Accountable Executives the Most?

In aviation safety management systems (SMS), the accountable executive is responsible for ensuring their SMS is both:

  • implemented according to regulatory standards; and
  • performing in all areas of the organization.

To ensure the SMS is performing as designed, accountable executives should be regularly reviewing organizational safety performance. When substandard safety performance is detected, it is the accountable executive's responsibility to bring the SMS into compliance.

Performance monitoring charts assist accountable executives in making decisions based on available data. What chart(s) should your accountable executive be reviewing?

Related Aviation SMS Performance Monitoring Articles

Why Overdue Issues and CPAs Chart Is Important

SMS performance can be quickly monitored by reviewing two important risk management elements:

  • Reported safety issues; and
  • Corrective actions and preventive actions (CPAs).

You may call CPAs something different, such as:

  • CAPAs (corrective actions and preventive actions);
  • Risk Mitigators; or simply
  • Corrective actions.

It really doesn't matter what you call CPAs. What is important is that CPAs speak volumes about safety culture and how management views the SMS. When management is not engaged in the SMS, you will commonly see poor performance where managers are not addressing safety issues in a timely manner. Alternatively, the root cause of poor CPA closure statistics may be the result of other factors:

  • that the organization doesn't have sufficient resources to manage all safety concerns;
  • managers are not trained to document CPAs; or
  • managers are not "forced/required" to document CPA performance activity.

Overdue reported safety issues also provide considerable intelligence to the accountable executive. When the accountable executive reviews hazard reporting metrics, not only will the accountable executive (and safety team) be looking at whether reported safety issues are being closed on time per your risk management processes, but they also should be reviewing whether the volume of reported safety concerns is commensurate with the size of the organization.

A good rule of thumb for determining whether you have a good safety reporting culture is that for every 100 employees, you should expect an average of ten safety reports per month. When safety reporting numbers fall for unexpected reasons, or if the number of safety reports has never been very high, the accountable executive (through the safety manager) must search for root causes. This is a management issue, but does it stem from:

  • lack of safety promotion activities?
  • apathy toward the SMS?
  • resistance from particular department heads?

In short, reported safety issues and CPAs can tell a lot about the performance of the SMS. When your SMS has many safety issues and CPAs are being neglected and not closed on time, the accountable executive is responsible for bringing the SMS into compliance.

The accountable executive is responsible, but does your account executive have direct access to SMS performance charts and metrics?

Accountable executives don't have time to comb through spreadsheets and SMS databases to look for clues. Simple risk analysis charts allow managers to focus on problem areas by seeing aggregated data formatted in a manner to facilitate timely and accurate decision-making.

An exceptionally powerful risk management chart is an " Overdue Issue and CPAs Chart" that you can create or you may already have available in your SMS database. We'll discuss this intuitive chart design that every safety team should have. It is especially easy enough for the accountable executive to monitor SMS performance at a very high level.

I'm not saying that this is the only chart that accountable executives should have access to, but it is one of the most valuable for reviewing reactive risk management activity early in an SMS implementation. As the organization's SMS' safety culture matures, this performance monitoring tool will only require a quick glance to determine whether there are any major obstacles with the SMS.

Related Articles on Aviation Risk Management

About Overdue Safety Issues and CPAs Chart

The Overdue Issue and CPAs Chart (like the one to the right) is very useful in your organization to more than the safety team. When accountable executives have unfettered access to such a report, they can take immediate action if it is available. I highly recommend aviation SMS management teams keep it close at hand.

This chart measures two things:

  • Issues that are currently overdue (i.e. are past their target closure date), assorted by their assessed risk level; and
  • Corrective Preventative Actions (CPAs) that are currently overdue (i.e. are past their assigned due date), assorted by the parent issue’s assessed risk level.

Your aviation safety management process must be proactive and timely about closing safety issues and corrective actions. Overdue SMS tasks may mean unmanaged and uncontrolled risk. Another possibility for seeing high numbers of overdue SMS items is that your risk management processes are too unwieldy and don't allow for adequate time to properly treat and document safety concerns.

Interpreting the root cause behind many overdue SMS tasks requires common sense and looking at the big picture. Don't always assume there are shirkers or those apathetic to your SMS. It may be your process. It may be expectations are not communicated fully.

You can really make charts like this very sexy. For example, the chart above is interactive and tied directly to risk management data stored in the SMS database. This chart has some very cool features, including:

  • Not only can you immediately see what/where issues and CPAs are overdue by risk level, but
  • You can also see which SMS tasks are overdue by selecting a cell in the risk matrix where overdue items appear and are taken directly to the selected item(s).

Which means you can use this chart as an:

  1. Informational SMS tool;
  2. Task management SMS tool; and
  3. SMS performance monitoring tool.

Your SMS has a safety performance monitoring and measurement requirement that allows aviation service providers to demonstrate continuous improvement of the SMS. Management may detect that the rates for overdue tasks are becoming a concern. In these cases, corrective action is required to determine the causes. Again, don't think the worst about employees unless there are obvious signs of resistance to the SMS.

Related Articles on Resistance to Aviation SMS

What This Chart Reveals About Aviation SMS Implementations

The primary meaning you can and should derive from this chart is very clear, including how:

  • Effective safety management’s performance in managing issues is;
  • Efficient your aviation SMS risk management’s workflows are;
  • Mature your aviation safety culture is; and
  • Active and responsive your department heads are (with CPAs).

It’s a simple chart, but because overdue items say so much about SMS performance, you can derive many quality assessments about the quality and dedication of your aviation SMS.

Things to Watch Out For

There are two perspectives to approach what to analyze:

  1. Through a single glance of the chart at a particular instant in time, and
  2. Through trends you notice in this chart over time.

From a single glance, your main concerns are:

  • Most importantly, there are no overdue high-risk/unacceptable, red issues or CPAs;
  • That you have very few or no overdue CPAs or safety issues; and
  • If you do have overdue items, most or all of them are in the Acceptable range.

If you are monitoring this chart over time, the primary trends you should keep an eye on are:

  • High-risk issues are constantly popping up as being overdue; or
  • An increasing trend of overdue items.

A supplemental line chart that plots this information over time is useful to:

  • identify performance trends;
  • monitor performance considering a seasonal effect; and
  • determining responsiveness over time and by risk level.

Related SMS Trend Analysis Articles

Where This Chart Comes From

This Overdue Issues and CPAs chart at the right comes from several data sources within a professional aviation SMS database. Let's provide a high-level workflow of a common risk management process:

  1. When safety issues are managed, safety managers will set a target closure date for safety issues based on assessed risk – the greater the risk the sooner the targeted date;
  2. Department heads with risk acceptance authority manage corrective and preventive actions (CPAs) to reduce risk to ALARP.
  3. Once safety issues are completely treated, safety managers review documentation, hopefully, set a "next review date," and close the safety issue; and
  4. If safety issues or CPAs become overdue, this chart will update with the issue based on its assessed risk level.

Of course, this can be done manually or with in-house, non-commercial SMS databases. However, it would require some tricky filtering and more manual work than any manager would be willing to give in a real-world scenario. An SMS database is obviously the best technology to drive this SMS performance monitoring chart. Not only is the data centralized and accessible by many managers in the decision-making process, but the data is real-time to give managers the most accurate results whenever they refresh the chart.

Related Aviation SMS Database Articles

How Does This Relate to SMS Requirements

While not necessarily strictly related to aviation SMS requirements, it will definitely affect continuous improvement in aviation SMS. The quicker hazards and risks are identified, prioritized, and controlled, the quicker the program improves.

Moreover, if an aviation SMS auditor shows up at your organization and finds a pile of overdue issues, chances are they aren’t going to ignore it. I’ll let your imagination complete this scenario.

Who Should Care About This Chart?

This chart is definitely the business of every employee participating in your aviation safety management system for different reasons.

Front-line employees are at the most risk. Their hazard reports account for most of the hazard reports that safety management deals with. So on the one hand, front-line employees should be concerned that,

  1. their hazard reports are valued, and,
  2. that risks are being prevented, corrected, and detected.

If issues are constantly overdue, then it means management is slow to mitigate/correct issues, and that the safety culture at management levels needs serious improvement. This is what line employees may think, but in reality, your responsiveness to treat risk may be due to oppressive or outdated processes that don't fit the reality of extant operations.

Management may not want line-level employees to access this report. There are a few very transparent safety cultures. Most companies are advised to restrict access to this chart by only management. You may not want this chart appearing on the front page of the daily newspaper. You will make this call based on your SMS performance and the level of transparency in the aviation SMS.

Management and executives should be concerned about their performance, especially when this chart is routinely scrutinized by the accountable executive. Their primary safety responsibilities are simply to manage and correct issues as soon as they can. When that is not happening, then the risk management program is not functioning. Executives should likewise be concerned that their safety budgets are being used efficiently.

Related Aviation SMS Performance Monitoring Articles

Final Thought: Similar Charts

You can easily create another risk management chart that relates very closely to this one but adds a different twist to monitoring performance. Instead of monitoring only overdue tasks, you can create another chart with the "Number of Unclosed Issues By Risk." This chart can display similar information (but does not display CPAs), with the same functionality for clicking on a box and being taken to relevant issues.

Obviously, unclosed issues will remain open. If you can compare both charts side-by-side, you are able to ascertain whether issues are overdue because

  • Aviation SMS management is bogged down with open issues (i.e. there are many unclosed safety issues); or
  • For other reasons that will require further investigation, such as in the case where the overdue issues account for most of the unclosed safety issues as well.

If you don't have the ability or budget to build these aviation SMS performance monitoring charts, we can help. We provide a complete SMS database solution to allow you to have the baby but without the labor pains.

Not Ready for Live SMS Demo?

Looking for ideas for your own SMS database? Our competitors watch our videos, so maybe you can be inspired too.

Last updated July 2024.