Safety assurance is the third ICAO pillar or component of aviation safety management systems (SMS). While practicing safety assurance, aviation service providers focus on safety performance monitoring and measurement. Monitoring activities include constantly reviewing the effectiveness of established safety risk controls.
Safety risk controls are established under the safety risk management process and typically performed during proactive hazard analysis when safety teams perform safety risk assessments. The output of the proactive hazard analysis is the Hazard Register, also commonly called Hazard Risk Register.
From identified aviation-related hazards, safety teams will select several which are key performance indicators (KPIs). Some people like to call KPIs safety performance indicators or SPIs to draw a more specific focus to the safety aspect of the organization versus KPIs which are more familiar in business management.
Safety assurance monitoring ensures aviation service providers meet or exceed safety objectives through the collection, analysis, and assessment of data relating to organizational performance. By automating KPI performance monitoring, your company can respond more quickly to changing environmental conditions to reduce risk and potential losses.
Monitoring KPIs allows the accountable executive to meet his responsibility to monitor the effectiveness of the SMS. For example, the FAA's Part 5 regulations state the accountable executive has several responsibilities toward the SMS including:
KPIs allow accountable executives a high-level snapshot to more quickly determine whether organizational SMS goals and objectives are being satisfied. However, accountable managers are not the only user group to benefit from SMS KPIs. Other user groups include:
The SMS' scope is very broad and has the potential to capture beneficial information to help operations managers make business-related decisions. KPIs allow managers to focus on the company's most important goals and objectives.
Monitoring the safety performance of KPIs is nearly impossible without modern safety management systems software. Spreadsheets are too cumbersome and prone to multiple challenges, including:
When we discuss monitoring the SMS, the most sustainable solution is to have an SMS database to manage data across the four pillars. Considerable volumes of disparate data are generated from SMS documentation requirements. All but the very small aviation service providers will need data management capabilities to securely store and retrieve this data for KPI monitoring.
Even the smaller companies should be wary of storing SMS data in spreadsheets when employee turnover is a recurring challenge. More than once have we seen all the data from an SMS leave when the safety manager leaves the company. This is a staggering blow to any SMS especially when you consider that you need historical data to demonstrate continuous improvement of the SMS.
Spreadsheets are the wrong technology to store SMS data. Encourage the accountable executive to invest in an SMS database for his benefit to meet his SMS obligations, if not for the benefit of the company, including:
Monitoring Safety Assurance activities using automated tools allows aviation safety professionals to evaluate the overall effectiveness of risk controls within the implemented aviation safety management system. Every airline and airport should be actively monitoring their aviation SMS to ensure:
Following are the suggested steps to automate performance monitoring of your KPIs:
Classification parameters from modern, full-featured aviation SMS software are derived from:
An SMS database may have more capability to include additional classifications, such as job tasks. Based on over a dozen years of empirical evidence, safety teams' most popular methods for classifying issues are by:
Root causes would logically be the third most common classification schema used by aviation safety teams. Since not every reported safety issue requires an in-depth investigation, there will be fewer opportunities to conduct the necessary root cause analysis. Whether you conduct a root cause analysis on every reported safety issue, or for only those exceeding a particular risk level is dependent on your risk management procedures. Based on my observations, only a small percentage of aviation SMS conduct a root cause analysis on every reported safety issue.
Human factors classification schemes are the fourth most common classification schema that I've seen most commonly used in risk management activities. These classifications are usually applied during the performance of in-depth investigations. There are a few human factors classification schemes available on the Internet, but the one we use in our SMS database originally comes from the U.S. Department of Defense's human factors classifications.
Regardless of which classification types you use, your SMS database should have the ability to easily flag any classification parameter as a KPI. Since KPIs change over time, you will need tools to monitor and easily manage KPI performance.
Performance monitoring thresholds may be:
We are assuming you are using an SMS database to track and monitor aviation SMS KPIs. Monitoring KPIs using a spreadsheet is possible, but not sustainable. Your SMS database will have the capability to sum totals based on a date range. Your report totals can be the number of times a KPI classification has been used to get a raw number, or if you are tracking costs associated with recovering and treating reported safety issues, you may sum the total financial costs.
While being able to monitor financial costs in an aviation SMS is desirable, it is seldom practiced due to:
Tracking financial cost data is desirable to your company's bean counters. Furthermore, financial data allows management to focus risk management strategies on types of safety concerns possessing the highest organizational financial costs. The beauty of tracking financial costs in an aviation SMS is that as the damage occurs, you are already tracking mitigation strategies. Going the next step to track financial data, while taking longer to perform, will add actionable intelligence for upper management to make more informed, data-based decisions.
To summarize, most operators track only the number of occurrences of certain data types, which are the KPIs you are monitoring.
When you are monitoring KPIs, you have a couple of options for notifying the appropriate managers. There is little sense in tracking KPIs unless the resulting information is communicated to the proper management personnel to make actionable decisions.
Who should see the KPI monitoring statistics?
How will they be seeing these statistics? In a dashboard chart? In an email report? These are considerations safety teams must make when they have full-featured SMS database tools that automate the monitoring of KPIs. There are more options available depending on the flexibility of your SMS database.
I believe in a perfect world that certain managers should be notified when "exceptions" are noted. This is similar to FDM or FOQA data monitoring. There is a lot of data collected. In fact, there is too much data to routinely scan. So FDM monitoring analysts focus on the "exceptions." What stands out?
The same approach can be applied to KPI classifications. When you set your monitoring thresholds (number of occurrences of financial thresholds), you want to alert management of the detected exception. This information is communicated either by:
The point is to determine beforehand who should be notified when a possible trend is developing:
So far, we know what we want to track and whom to notify or communicate results to. For KPI dashboards, the data is typically available in real-time, meaning that the reports automatically update each time the report is refreshed or when the user enters the SMS database to view the KPI monitoring dashboards.
Web technologies are more reliable today than ten years ago. This affords many more options to communicate KPI statistics to managers. Options today include:
Let's take an example of a safety culture where managers are not regularly monitoring dashboards generated by your SMS database. This is very common, especially with "hands-off" accountable executives and cultures with low hazard reporting numbers. In these cases, best practices dictate that these managers should be alerted by email or text messages whenever a KPI alert is triggered.
Personally, I prefer email notifications because I can get more details about the triggered alert. Text messages are short and may not hold enough actionable intelligence to lead me to the problem area.
Email notifications in tandem with dashboard monitoring charts are most effective. These dashboard monitoring charts should be intuitive enough for managers to drill down and get more details or at least direct the manager to the area of the SMS that requires scrutiny. In this workflow, an email notifies configured managers of the potential trend affecting operations. Managers would then use preferred tools in the SMS database to analyze and evaluate the developing trend.
Modern aviation SMS software should offer aviation safety professionals an easy way to monitor potential KPI trends. Otherwise, your safety team is missing out on essential functionality and opportunities to manage risk.
How your aviation safety team responds to automated KPI trend alerts depends on your company policies. At a minimum, the safety team should review and continue monitoring.
Below are some resources if you wish to compare your KPIs with others in the aviation industry. Aviation maintenance providers may benefit from reviewing the airline and airport KPIs.
Good luck monitoring your aviation SMS KPIs. Automated tools offer tremendous value in this area.
Safety managers should be helping accountable executives satisfy their SMS obligations to monitor the effectiveness of the SMS. In order to do this effectively, and in a sustainable fashion, an SMS database is the best solution.
If you don't have adequate tools to manage SMS documentation, we can help.
Last updated March 2024.