Why Monitor Aviation Safety Key Performance Indicators?

Safety assurance is the third ICAO pillar or component of aviation safety management systems (SMS). While practicing safety assurance, aviation service providers focus on safety performance monitoring and measurement. Monitoring activities include constantly reviewing the effectiveness of established safety risk controls.

Safety risk controls are established under the safety risk management process and typically performed during proactive hazard analysis when safety teams perform safety risk assessments. The output of the proactive hazard analysis is the Hazard Register, also commonly called Hazard Risk Register.

SPIs Are Subset of KPIs

From identified aviation-related hazards, safety teams will select several which are key performance indicators (KPIs). Some people like to call KPIs safety performance indicators or SPIs to draw a more specific focus to the safety aspect of the organization versus KPIs which are more familiar in business management.

Related Articles on Key Performance Indicators (KPIs) in Aviation SMS

• What Is a Key Performance Indicator (KPI) in Aviation SMS?
• How to Set and Monitor Key Performance Indicators (KPIs) in Existing SMS
• How to Automate Key Performance Indicator KPI Monitoring

Safety assurance monitoring ensures aviation service providers meet or exceed safety objectives through the collection, analysis, and assessment of data relating to organizational performance. By automating KPI performance monitoring, your company can respond more quickly to changing environmental conditions to reduce risk and potential losses.

Monitoring KPIs allows the accountable executive to meet his responsibility to monitor the effectiveness of the SMS. For example, the FAA's Part 5 regulations state the accountable executive has several responsibilities toward the SMS including:

1. Ensure the SMS is properly implemented and performed in all areas of the organization;
2. Regularly review organizational safety performance and direct actions necessary to address substandard safety performance

KPIs allow accountable executives a high-level snapshot to more quickly determine whether organizational SMS goals and objectives are being satisfied. However, accountable managers are not the only user group to benefit from SMS KPIs. Other user groups include:

• Safety teams;
• Quality assurance department;
• Training department; and
• Operations managers.

The SMS' scope is very broad and has the potential to capture beneficial information to help operations managers make business-related decisions. KPIs allow managers to focus on the company's most important goals and objectives.

Related Articles on Aviation SMS Goals and Objectives

• How to Shape Your Corporate Vision With Aviation Safety Goals and Objectives
• 3 Goal-Setting Tips for Aviation Key Performance Indicators
• Safety Chart: Maintain Monthly Aviation SMS Hazard Reporting Goals

How to Monitor Key Performance Indicators

Monitoring the safety performance of KPIs is nearly impossible without modern safety management systems software. Spreadsheets are too cumbersome and prone to multiple challenges, including:

• Data security;
• Difficult to track metadata;
• Data integrity from users overwriting or deleting previous entries;
• Unwieldy to document milestones; and
• Displaying resulting charts and graphs across a wide audience in real time.

When we discuss monitoring the SMS, the most sustainable solution is to have an SMS database to manage data across the four pillars. Considerable volumes of disparate data are generated from SMS documentation requirements. All but the very small aviation service providers will need data management capabilities to securely store and retrieve this data for KPI monitoring.

Even the smaller companies should be wary of storing SMS data in spreadsheets when employee turnover is a recurring challenge. More than once have we seen all the data from an SMS leave when the safety manager leaves the company. This is a staggering blow to any SMS especially when you consider that you need historical data to demonstrate continuous improvement of the SMS.

Spreadsheets are the wrong technology to store SMS data. Encourage the accountable executive to invest in an SMS database for his benefit to meet his SMS obligations, if not for the benefit of the company, including:

• the safety team,
• operational managers,
• auditors (both internal and external), and
• especially employees.

Related Aviation SMS Database Articles

• What Is an Aviation Safety Database
• How to Choose the Best Aviation Safety Database Software
• How to Manage Aviation Safety Programs Without Complex SMS Databases

Evaluate Effectiveness of Risk Controls Using KPIs

Monitoring Safety Assurance activities using automated tools allows aviation safety professionals to evaluate the overall effectiveness of risk controls within the implemented aviation safety management system. Every airline and airport should be actively monitoring their aviation SMS to ensure:

• New hazards are identified;
• Associated risks are mitigated;
• Potential safety hazard trends are identified early;
• Regulatory compliance applicable to SMS;
• Safety Assurance activities follow your SMS manual; and
• Validate the system description described in the Proactive Hazard Analysis Process or Safety Risk Assessments.

Following are the suggested steps to automate performance monitoring of your KPIs:

1. Identify KPIs from Your Classification Parameters

Classification parameters from modern, full-featured aviation SMS software are derived from:

• Types of Issues;
• Hazard Register;
• Policies & Procedures;
• Root Cause;
• Human Factors;
• Event Location; and
• Affected aircraft or vehicle parts.

An SMS database may have more capability to include additional classifications, such as job tasks. Based on over a dozen years of empirical evidence, safety teams' most popular methods for classifying issues are by:

• Type of issue (general, flexible classification schema); and
• Hazards from their Hazard Register.

Root causes would logically be the third most common classification schema used by aviation safety teams. Since not every reported safety issue requires an in-depth investigation, there will be fewer opportunities to conduct the necessary root cause analysis. Whether you conduct a root cause analysis on every reported safety issue, or for only those exceeding a particular risk level is dependent on your risk management procedures. Based on my observations, only a small percentage of aviation SMS conduct a root cause analysis on every reported safety issue.

Human factors classification schemes are the fourth most common classification schema that I've seen most commonly used in risk management activities. These classifications are usually applied during the performance of in-depth investigations. There are a few human factors classification schemes available on the Internet, but the one we use in our SMS database originally comes from the U.S. Department of Defense's human factors classifications.

Regardless of which classification types you use, your SMS database should have the ability to easily flag any classification parameter as a KPI. Since KPIs change over time, you will need tools to monitor and easily manage KPI performance.

Related Aviation Human Factors Articles

• Let’s Talk Human Factors - Origin of Dirty Dozen
• Let’s Talk Human Factors - Lack of Communication
• Let’s Talk Human Factors: Distraction Is #4 of the Dirty Dozen

2. Set a KPI Performance Monitoring Threshold

Performance monitoring thresholds may be:

• Number of occurrences; or
• Financial (cost of damage or potential damage).

We are assuming you are using an SMS database to track and monitor aviation SMS KPIs. Monitoring KPIs using a spreadsheet is possible, but not sustainable. Your SMS database will have the capability to sum totals based on a date range. Your report totals can be the number of times a KPI classification has been used to get a raw number, or if you are tracking costs associated with recovering and treating reported safety issues, you may sum the total financial costs.

While being able to monitor financial costs in an aviation SMS is desirable, it is seldom practiced due to:

• Difficulty in acquiring cost data by safety teams;
• Reliance solely on safety teams to enter cost data, vs. having department heads share the data entry burden;
• Inconsistent documentation of cost data relating to reported safety issues; or
• Upper management not enforcing the tracking of safety cost data.

Tracking financial cost data is desirable to your company's bean counters. Furthermore, financial data allows management to focus risk management strategies on types of safety concerns possessing the highest organizational financial costs. The beauty of tracking financial costs in an aviation SMS is that as the damage occurs, you are already tracking mitigation strategies. Going the next step to track financial data, while taking longer to perform, will add actionable intelligence for upper management to make more informed, data-based decisions.

To summarize, most operators track only the number of occurrences of certain data types, which are the KPIs you are monitoring.

3. Determine Notification or Access Strategy When KPIs Thresholds Are Reached

When you are monitoring KPIs, you have a couple of options for notifying the appropriate managers. There is little sense in tracking KPIs unless the resulting information is communicated to the proper management personnel to make actionable decisions.

Who should see the KPI monitoring statistics?

How will they be seeing these statistics? In a dashboard chart? In an email report? These are considerations safety teams must make when they have full-featured SMS database tools that automate the monitoring of KPIs. There are more options available depending on the flexibility of your SMS database.

I believe in a perfect world that certain managers should be notified when "exceptions" are noted. This is similar to FDM or FOQA data monitoring. There is a lot of data collected. In fact, there is too much data to routinely scan. So FDM monitoring analysts focus on the "exceptions." What stands out?

The same approach can be applied to KPI classifications. When you set your monitoring thresholds (number of occurrences of financial thresholds), you want to alert management of the detected exception. This information is communicated either by:

• Dashboard charts;
• Paper (not so much today);
• PowerPoint presentations; or
• Electronic alerts.

The point is to determine beforehand who should be notified when a possible trend is developing:

• Safety managers;
• Department heads;
• Executives?

4. Choose How KPI Trend Monitor Alerts Are Sent

So far, we know what we want to track and whom to notify or communicate results to. For KPI dashboards, the data is typically available in real-time, meaning that the reports automatically update each time the report is refreshed or when the user enters the SMS database to view the KPI monitoring dashboards.

Web technologies are more reliable today than ten years ago. This affords many more options to communicate KPI statistics to managers. Options today include:

• Email;
• SMS Text messages; or
• Only view in KPI monitoring dashboard; or
• Combination of approaches.

Let's take an example of a safety culture where managers are not regularly monitoring dashboards generated by your SMS database. This is very common, especially with "hands-off" accountable executives and cultures with low hazard reporting numbers. In these cases, best practices dictate that these managers should be alerted by email or text messages whenever a KPI alert is triggered.

Personally, I prefer email notifications because I can get more details about the triggered alert. Text messages are short and may not hold enough actionable intelligence to lead me to the problem area.

Email notifications in tandem with dashboard monitoring charts are most effective. These dashboard monitoring charts should be intuitive enough for managers to drill down and get more details or at least direct the manager to the area of the SMS that requires scrutiny. In this workflow, an email notifies configured managers of the potential trend affecting operations. Managers would then use preferred tools in the SMS database to analyze and evaluate the developing trend.

Related SMS Trend Analysis Articles

• Aviation Safety Managers' Best Friend - Trending Charts
• How to Use Trending Charts in Aviation SMS
• Six Steps How to Perform Trend Analysis in Aviation SMS

Final Thoughts on Monitoring Aviation Safety KPI Performance

Modern aviation SMS software should offer aviation safety professionals an easy way to monitor potential KPI trends. Otherwise, your safety team is missing out on essential functionality and opportunities to manage risk.

How your aviation safety team responds to automated KPI trend alerts depends on your company policies. At a minimum, the safety team should review and continue monitoring.

Below are some resources if you wish to compare your KPIs with others in the aviation industry. Aviation maintenance providers may benefit from reviewing the airline and airport KPIs.

Good luck monitoring your aviation SMS KPIs. Automated tools offer tremendous value in this area.

Safety managers should be helping accountable executives satisfy their SMS obligations to monitor the effectiveness of the SMS. In order to do this effectively, and in a sustainable fashion, an SMS database is the best solution.

If you don't have adequate tools to manage SMS documentation, we can help.

Watch Demo Videos Instead

Lists of KPI Resources to Download

Last updated March 2024.