Monitor KPIs in Existing Aviation SMS

Setting up and monitoring key performance indicators (KPIs) in existing aviation safety management systems (SMS) can be a daunting task. This especially applies to first-time safety managers or safety managers new to a company who are taking over an existing aviation SMS.

Such safety managers have many questions like:

• Where do I get started?
• What is a key performance indicator?
• How much work will KPIs require?

These are important and natural questions to ask. Now, “existing SMS programs” can vary greatly in structure, maturity, and safety data management strategies. For this reason, we will show you how to do this from the standpoint of an SMS database program using SMS Pro. We very rarely like to discuss SMS Pro in our blog articles. We prefer to educate and help our clients in the role of an SMS partner.

Disusing how to set up and monitor KPIs using this commercially available database software in this article makes sense to:

• Avoid generalizations; and
• Show you step-by-step how this process works for hundreds of aviation service providers around the world.

Here is how to set up and monitor key performance indicators in SMS Pro. This same process can be adapted to any ICAO-compliant SMS.

1 – Understand What KPIs Are by Reviewing KPI Resources

Having a very solid understanding of key performance indicators is essential for monitoring the safety performance of your aviation SMS. Fortunately, we have many such resources:

• What Is a Key Performance Indicator (KPI) in Aviation SMS
• 5 Rules for Selecting KPIs in Aviation Safety Programs
• Signs of Good KPIs in Aviation – Do They Describe Your KPI List?
• How to Automate Key Performance Indicator KPI Monitoring the Easy Way
• How to Choose KPIs in Aviation SMS Programs [6 Steps]
• 6 Sources for Aviation Key Performance Indicators
• 7 Quick Tips for a Key Performance Indicator (KPI) Strategy
• Tips Using Aviation Key Performance Indicator (KPI) Statistics - With Free Downloads
• 4 Pillars | The Truth About SMS Continuous Improvement and KPIs

These resources should prove very useful in not only grounding you in KPIs but also giving you a detailed understanding of:

• What KPIs are;
• Where they come from;
• Whey they’re important;
• What to do with them; and
• How to get real decision-making value from them.

Once you feel comfortable with KPIs, you can begin establishing your KPIs.

2 – Review Goals and Objectives

Goals and objectives will directly influence which KPIs you choose. You could say that KPIs are in chosen in response to objectives. Goals and objectives are created in response to a problem or an opportunity. In other words, KPIs are useful to monitor the performance of your safety objectives.

Along the same thread, if someone asks you "How do you monitor the performance of your SMS," what will you tell them? I'm not trying to put you on the spot here. This could be the accountable executive who needs assurance that the SMS that is consuming his budget is actually adding value to the organization. You see, the accountable executive has his own goals and objectives that must be met.

Relating to aviation SMS, the accountable executive is responsible to:

• Ensure the SMS is properly implemented and performed in all areas of the organization;
• Regularly review organizational safety performance and direct actions necessary to address substandard safety performance

You can also be expected to be asked how you monitor SMS performance from an external auditor, such as the aviation regulatory authority or client auditing your operations to determine the soundness of your SMS implementation.

So what do you tell your auditors when they ask you how you have been monitoring the performance of the SMS? They may change the words a bit and ask instead: "How do you monitor safety performance?" What do you tell them? I know what you should tell them, but this is a thinking exercise.

Once you are grounded with a solid understanding of KPIs and your accountable executive's goals and objectives, you should:

• Review your organization’s goals and objectives;
• Review regulatory authority’s goals and objectives; and
• Review goals and objectives of other upper managers, clients, and stakeholders.

After considering all of this data, ensure that your list of objectives:

• Account for your most important safety needs;
• Are associated with your goals (i.e., each goal may have several objectives, but no objectives should be without a parent goal); and
• Are specific and measurable.

For more information, see our article on How to Create Safety Goals and Objectives in Aviation SMS. If your objectives need updating, you MUST do that before you choose KPIs.

Related Articles on Aviation SMS Goals and Objectives

• 3 Goal-Setting Tips Using Aviation Key Performance Indicators - With Free KPI Resources
• How to Shape Your Corporate Vision With Aviation Safety Goals and Objectives
• What Are Safety Objectives in Aviation SMS – With Examples

3 – Review System (Risk Profile) and Create New List of Potential KPIs

Once you have reviewed your objectives, you can now begin to create a list of potential key performance indicators. Here is how this works – do the following for each objective

1. Review your system, such as your list of identified hazards and other classifications – this happens in SMS Pro’s Proactive Hazard Analysis Tool module; and
2. If you see a hazard or other classification that is closely associated with your objectives, add it to your list of potential KPIs.

Once your KPI list is complete, compare it to your list of objectives to make sure all objectives are accounted for with classification parameters that can be easily applied to safety reports. You will monitor KPI performance as "KPI classification parameters" that can be easily toggled on or off as organizational goals and objectives change.

Safety reports and audit findings are classified during the reactive and proactive risk management processes. To help facilitate the fidelity and accuracy of safety data, you should designate dedicated subject matter experts responsible for classifying safety issues. For smaller organizations, this will normally be the safety manager, while larger companies may have safety committees review and classify reported safety issues during their routine safety meetings.

When too many people start classifying issues in your SMS, there quickly becomes a "consistency problem" that reduces the value of your KPIs. There is an increased risk that the wrong hazards and associated risk controls are reviewed whenever safety issues are classified incorrectly. This is not a good thing.

Safety data must be properly classified to both

• provide decision-making value to management; and
• increase the effectiveness of monitoring risk controls.

Improper use of KPI classifications results in your SMS becoming the waste of time that half of the upper management believes it to be. Garbage in garbage out.

Our SMS' objective is to stop an accident and demonstrate to upper management that your SMS adds measurable value to the organization's bottom line. Of course, it's nice to tell everybody you are mitigating risk to as low as reasonably practical, but this overused cliche does little to excite upper management about aviation SMS' value. Upper management understands financial value, and this is what we want to show them.

Evaluating a KPI requires a bit of common sense. We cannot give you hard and fast rules because there are too many use cases to consider. On one hand, for example, we can assume our risk controls are not working when we count high numbers of a particular KPI classification within a particular time period. The logic stands that if the risk controls were mitigating risk properly, such safety issues would happen less often.

On the other hand, many instances of a KPI being used can be a good thing. Take safety reporting of proactively identified hazards. In this case, we want to see high numbers as this KPI demonstrates a natural evolution from purely reactive risk management cultures to more proactive cultures.

Related Articles on Proactive Risk Management in Aviation SMS

• What Is Proactive Risk Management in Aviation SMS?
• From Reactive to Proactive Risk Management in Aviation SMS
• Reactive or Proactive Safety Culture? 5 Creative Ways to Tell

4 – Compare Potential KPI List With Existing KPIs

Now it’s time to delve back into your existing SMS and evaluate which KPIs have already been chosen. You will compare your new list with the existing risk and combine them by:

• Adding new KPIs from your list of Potential KPIs; and
• Removing KPIs from your existing list of KPIs that are no longer relevant.

In SMS Pro, this is easily done in either the Proactive Hazard Analysis Tool module or Issue Manager module by:

• Going into edit mode;
• Selecting your KPI classification; and
• Checking it as “Is KPI” (for current division) or “Is Global KPI” (for all divisions).

Now it is selected as a KPI and will appear in your list of KPIs to set targeted goals and thresholds during KPI monitoring.

5 – Monitor KPIs With Target Goals and Thresholds

The last step for setting up and monitoring key performance indicators (KPIs) in existing SMS is to:

• Set each KPI’s target goal for the number of occurrences per year for that KPI; and
• Set each KPI’s threshold to alert you that the KPI has reached or exceeded its target goal.

An important to note that:

• A “target goal” is the MAXIMUM limit of any given KPI classification usage for a time period;
• As long as a KPI stays under its target, then it is considered “performing”;
• If a KPI reaches or goes over target, it is considered “non-performing”;
• Thresholds should be set before the target is reached so that safety managers can take appropriate action before a KPI becomes “un-performing.”

In SMS Pro, targets are set in the Goals and Objectives module. You can see a list of all KPIs and note your target for maximum occurrences for a given KPI. You can also see the number of times each KPI has been used for the current year and previous years.

Besides the KPI Objectives History that monitors annual KPI objectives, SMS Pro's Goals and Objectives module also has a Rolling KPI Monitoring Dashboard. This dashboard allows management to quickly monitor aviation SMS performance by reviewing KPI statistics in real time.

Thresholds are configured in either the Proactive Hazard Analysis Tool module or Issue Manager module, in Edit Mode. You can enter your threshold number in the edit section. See this video for how to set KPIs and thresholds.

Thresholds should be set as a number lower than your target. For example, if a KPI has a target of less than 10, your threshold alert should be set at, say, 7. This will alert you that you are approaching non-performance for particular types of safety occurrences.

Final Thoughts on Setup and Monitoring KPIs

KPIs allow managers to cut through mountains of safety data to make decisions based on the organization's most important goals and objectives. Examples in this article used a commercially available SMS database. There are advantages of using a commercial SMS database to manage all your safety data. These advantages include:

• Tried and proven risk management workflows;
• Extremely robust charts and report generation;
• Enhanced data security (most often more secure than your own organization's data security protocols);
• Aviation industry support for continued enhancements;
• Available anywhere Internet connectivity is available;
• Full-featured covering all four pillars (not a point solution);
• Auditors recognize major SMS database products;
• Saves time on SMS documentation management; and
• Product support that exceeds your expectations.

If you have trouble monitoring your SMS performance, we can help. Safety teams can easily manage KPIs and share them with the management to facilitate better decision-making. Sign up for a live demo or simply watch the demo videos.

We want to be your SMS partner.

Watch SMS Database Videos

We hope you have found this helpful for beginning your KPI journey. Please see the following resources which will provide further help:

Last updated November 2024.