Difficulty of Selecting KPIs in Aviation Safety Programs

Selecting key performance indicators (KPIs) in aviation safety management systems (SMS) is neither simple nor always intuitive. In addition to being time-consuming and costly, selecting and monitoring KPIs can also be a stressful experience.

Lives depend on choosing good KPIs. Your job may depend on selecting good KPIs.

Fortunately, selecting KPIs is helped by the fact that:

• There are proven steps for choosing KPIs in aviation SMS implementations;
• There are documented “rules” for choosing key performance indicators; and
• Having specific safety goals and objectives will inform which KPIs should be chosen.

Related Articles on KPIs in Aviation SMS

• What Is a Key Performance Indicator (KPI) in Aviation SMS?
• 6 Sources for Aviation Key Performance Indicators
• Signs of Good KPIs in Aviation - with Free KPI Resources

Here are 5 rules for selecting KPIs in aviation safety programs:

1 – Relate Directly to Safety Goals and Objectives

Aviation key performance indicators are built-in references to organizational safety goals and objectives. When KPIs do not directly reference a safety goal or objective, that KPI needs to be removed or replaced with another data element more suitable to represent the progress of the safety goal.

Because KPIs relate directly to goals and objectives, KPIs should:

• Encompass every level of company structure;
• Be short, medium, and long term in scope;
• Be SMART (see next section);
• Outline what types of performance are most currently important to aviation SMS implementation (this includes QMS and financial performance); and
• Be strongly justified.

There should be no question about why a particular KPI metric is included in the list of KPIs. If there is the slightest doubt as to why it’s there, safety teams should consider revising the metric to better align with current goals and objectives.

2 – KPIs Should Be SMART

SMART key performance indicators are worth an article of their own. A SMART KPI list in aviation SMS contains carefully considered KPIs that are:

• Specific: a specific qualitative/quantitative metric should be identified;
• Measurable: Qualitative data can be expressed in quantitative safety data;
• Achievable: The safety management system has the ability to monitor these metrics, such as with an aviation safety database or aviation safety management system software;
• Relevant: Metrics are bound to and tailored for goals and objectives; and
• Time-bound: Each metric is expressed in terms of some time period (short-term, medium-term, and long-term).

If any KPI in your list does not meet these standards, it needs to be replaced. If your list of KPIs is not SMART, then the entire thing needs to be reworked. Non-SMART KPIs are bad for business quality and safety.

Related Articles on KPIs in Aviation SMS

• Top 5 KPIs in Aviation Safety
• KPI Dashboard Monitoring Guidelines for Aviation SMS
• How to Automate Key Performance Indicator KPI Monitoring in Aviation SMS

3 – KPIs Need to Be Updated

Company safety goals will change over time. The best indicators of safety performance will also change over time as the aviation SMS is implemented. As goals and objectives change, KPIs need to change too. But updating KPIs is like cleaning out your garage – suddenly your old junk looks much more valuable than it actually is.

You rely on KPIs. They are your bread-and-butter safety data for long periods of time. Throwing them away is not easy. Some of the hard things about updating KPIs are:

• Looking at the metrics every day can make you “blind” to their real worth;
• Becoming “attached” to KPIs, even when their value is spent;
• Selecting new KPIs can be a time-consuming, expensive, and complex process; and
• It’s not always clear how valuable a KPI is.

A couple of good questions to ask yourself in regards to updating each KPI are:

• With the current abilities of the aviation safety program, it is feasible to improve this metric?
• If a stranger looked at this KPI, would he/she immediately know which goals and objectives this metric satisfies?
• Can I say in one sentence why this metric is necessary for describing the current performance of the system?
• If I showed this metric to a safety manager at a different organization would they be impressed?

A yearly or semi-annual “cut-throat” review of KPIs should be sufficient.

4 – KPIs Should Be Manageable

The word “manageable” is usually misunderstood as “achievable” when discussing KPIs. A KPI that is manageable is not just a KPI that can be monitored. Manageable KPIs are KPIs that can be:

• Controlled; and
• Assigned to other managers to be in charge of driving and monitoring performance.

Key performance indicators that arise beyond the influence of your organization are not suitable KPIs. Management should have methods of directly influencing KPIs.

For example, consider the following KPI: Average turnover rate per employee per 5 years. Turnover rate is a metric that management has direct influence over, such as by cultivating a quality, safe, and desirable place to work. This is also a KPI that can be assigned to a manager, such as an HR director.

Related Articles on KPIs in Aviation SMS

• Essential KPIs for Poor Hazard Reporting Cultures in Aviation SMS
• 3 Goal-Setting Tips Using Aviation Key Performance Indicators
• Safety Chart: Visually Presenting Aviation SMS KPIs - with Free KPI Resources

5 – Aviation Performance Indicators Need Custom Tailoring

Key performance indicators in aviation SMS implementations are and need to be mission critical. Mission critical equates to custom tailoring. When we talk about KPIs being custom tailored, this means:

• KPIs are unique (or mostly unique) to the company;
• Metrics may be based on widely distributed KPI metric resources, but were then altered slightly to fit company needs; and
• Other companies could not “plagiarize” your KPI metrics because the KPIs are specific to your organization.

When an aviation SMS is first implemented, KPIs may not be custom tailored. This is perfectly natural. But as KPIs are updated and as the SMS implementation progresses, KPIs should become more specific to the clarified goals and safety needs of your company.

Cookie cutter KPIs – which are KPIs that could easily exist in another company – should be avoided at all costs.

Final Thoughts about Selecting KPIs in Aviation SMS Implementations

KPIs in aviation SMS implementations have been a very popular topic among safety managers for over a dozen years. Most safety managers are looking to see what other operators are selecting for their KPIs. This is a very good practice, as there is no sense in recreating the wheel. So while it is great that safety professionals review the KPIs used at other organizations, don't fall into the trap you need to have "that special KPI" on your list.

In order to be effective and reasonable, selected KPIs must have real, accessible data behind them. For example, if one of your KPIs is "Missed Approach" and your company never has any "missed approaches," then this is not a good KPI to include.

Alternatively, we see naive safety managers create KPIs that they cannot possibly track. As a quick example, "Customer Complaints" is a KPI for an airline I only want to call "Airline XXX" to avoid calling anyone out. The strange thing about seeing "Customer Complaints" in this airline's list of KPIs is that this particular operator had no way of tracking customer complaints.

In this case, the airline had an incomplete strategy for customers to report complaints, but it was not very user-friendly. In this case, Customer Complaints are a good KPI to have, but this particular customer had no effective way to collect ALL customer complaints. Their complaint-gathering tools were incomplete and not promoted. The problem was that most customers never knew that they could complain.

In short, make sure you have reasonable amounts of accurate data that can be collected, organized, and retrieved to analyze the safety data to determine whether operations are meeting their safety and or quality objectives.

If you are creating or updating KPIs, these lists should prove to be a good starting point for generating ideas or metrics to use for custom tailoring:

Do you need tools to:

• collect safety data?
• organize data into logical groupings or classifications?
• analyze data at any time with pre-defined reports?
• alert management of overdue items?

Since 2007, SMS Pro has been working with operators to manage their SMS data. To learn how you can benefit from a low-cost, commercially available SMS database, see these short demo videos:

Last updated August 2023.