Setting up and monitoring key performance indicators (KPIs) in existing aviation safety management systems (SMS) can be a daunting task. This especially applies to first-time safety managers or safety managers new to a company who are taking over an existing aviation SMS.
Such safety managers have many questions like:
These are important and natural questions to ask. Now, “existing SMS programs” can vary greatly in structure, maturity, and safety data management strategies. For this reason, we will show you how to do this from the standpoint of an SMS database program using SMS Pro. We very rarely like to discuss SMS Pro in our blog articles. We prefer to educate and help our clients in the role of an SMS partner.
Disusing how to set up and monitor KPIs using this commercially available database software in this article makes sense to:
Here is how to set up and monitor key performance indicators in SMS Pro. This same process can be adapted to any ICAO-compliant SMS.
Having a very solid understanding of key performance indicators is essential for monitoring the safety performance of your aviation SMS. Fortunately, we have many such resources:
These resources should prove very useful in not only grounding you in KPIs but also giving you a detailed understanding of:
Once you feel comfortable with KPIs, you can begin establishing your KPIs.
Goals and objectives will directly influence which KPIs you choose. You could say that KPIs are in chosen in response to objectives. Goals and objectives are created in response to a problem or an opportunity. In other words, KPIs are useful to monitor the performance of your safety objectives.
Along the same thread, if someone asks you "How do you monitor the performance of your SMS," what will you tell them? I'm not trying to put you on the spot here. This could be the accountable executive who needs assurance that the SMS that is consuming his budget is actually adding value to the organization. You see, the accountable executive has his own goals and objectives that must be met.
Relating to aviation SMS, the accountable executive is responsible to:
You can also be expected to be asked how you monitor SMS performance from an external auditor, such as the aviation regulatory authority or client auditing your operations to determine the soundness of your SMS implementation.
So what do you tell your auditors when they ask you how you have been monitoring the performance of the SMS? They may change the words a bit and ask instead: "How do you monitor safety performance?" What do you tell them? I know what you should tell them, but this is a thinking exercise.
Once you are grounded with a solid understanding of KPIs and your accountable executive's goals and objectives, you should:
After considering all of this data, ensure that your list of objectives:
For more information, see our article on How to Create Safety Goals and Objectives in Aviation SMS. If your objectives need updating, you MUST do that before you choose KPIs.
Once you have reviewed your objectives, you can now begin to create a list of potential key performance indicators. Here is how this works – do the following for each objective
Once your KPI list is complete, compare it to your list of objectives to make sure all objectives are accounted for with classification parameters that can be easily applied to safety reports. You will monitor KPI performance as "KPI classification parameters" that can be easily toggled on or off as organizational goals and objectives change.
Safety reports and audit findings are classified during the reactive and proactive risk management processes. To help facilitate the fidelity and accuracy of safety data, you should designate dedicated subject matter experts responsible for classifying safety issues. For smaller organizations, this will normally be the safety manager, while larger companies may have safety committees review and classify reported safety issues during their routine safety meetings.
When too many people start classifying issues in your SMS, there quickly becomes a "consistency problem" that reduces the value of your KPIs. There is an increased risk that the wrong hazards and associated risk controls are reviewed whenever safety issues are classified incorrectly. This is not a good thing.
Safety data must be properly classified to both
Improper use of KPI classifications results in your SMS becoming the waste of time that half of the upper management believes it to be. Garbage in garbage out.
Our SMS' objective is to stop an accident and demonstrate to upper management that your SMS adds measurable value to the organization's bottom line. Of course, it's nice to tell everybody you are mitigating risk to as low as reasonably practical, but this overused cliche does little to excite upper management about aviation SMS' value. Upper management understands financial value, and this is what we want to show them.
Evaluating a KPI requires a bit of common sense. We cannot give you hard and fast rules because there are too many use cases to consider. On one hand, for example, we can assume our risk controls are not working when we count high numbers of a particular KPI classification within a particular time period. The logic stands that if the risk controls were mitigating risk properly, such safety issues would happen less often.
On the other hand, many instances of a KPI being used can be a good thing. Take safety reporting of proactively identified hazards. In this case, we want to see high numbers as this KPI demonstrates a natural evolution from purely reactive risk management cultures to more proactive cultures.
Now it’s time to delve back into your existing SMS and evaluate which KPIs have already been chosen. You will compare your new list with the existing risk and combine them by:
In SMS Pro, this is easily done in either the Proactive Hazard Analysis Tool module or Issue Manager module by:
Now it is selected as a KPI and will appear in your list of KPIs to set targeted goals and thresholds during KPI monitoring.
The last step for setting up and monitoring key performance indicators (KPIs) in existing SMS is to:
An important to note that:
In SMS Pro, targets are set in the Goals and Objectives module. You can see a list of all KPIs and note your target for maximum occurrences for a given KPI. You can also see the number of times each KPI has been used for the current year and previous years.
Besides the KPI Objectives History that monitors annual KPI objectives, SMS Pro's Goals and Objectives module also has a Rolling KPI Monitoring Dashboard. This dashboard allows management to quickly monitor aviation SMS performance by reviewing KPI statistics in real time.
Thresholds are configured in either the Proactive Hazard Analysis Tool module or Issue Manager module, in Edit Mode. You can enter your threshold number in the edit section. See this video for how to set KPIs and thresholds.
Thresholds should be set as a number lower than your target. For example, if a KPI has a target of less than 10, your threshold alert should be set at, say, 7. This will alert you that you are approaching non-performance for particular types of safety occurrences.
KPIs allow managers to cut through mountains of safety data to make decisions based on the organization's most important goals and objectives. Examples in this article used a commercially available SMS database. There are advantages of using a commercial SMS database to manage all your safety data. These advantages include:
If you have trouble monitoring your SMS performance, we can help. Safety teams can easily manage KPIs and share them with the management to facilitate better decision-making. Sign up for a live demo or simply watch the demo videos.
We want to be your SMS partner.
We hope you have found this helpful for beginning your KPI journey. Please see the following resources which will provide further help:
Last updated November 2024.