Aviation safety management systems (SMS) revolve around the four ICAO pillars:
KPI statistics can be effectively used in each of these four pillars to monitor SMS performance.
In this post, we will explore some common and perhaps overlooked ways to make the most effective use of KPIs in your organization.
Aviation SMS KPIs are used by safety teams and the accountable executive to determine:
We could probably fill this page with ideas as to how KPIs benefit aviation SMS implementations. Your imagination is the limiting factor here. The point is that KPIs can be applied across all four pillars and not focus only on the "sexy" parts of the SMS, which are namely:
Each of the four ICAO four pillars or components possesses opportunities to effectively use KPI data to monitor and correct SMS performance.
Aviation safety managers spend considerable time collecting data that is required to demonstrate a fully functioning, performant SMS. This data comes from a wide variety of sources, including:
You can see from this quick list above the data points you are probably capturing today. You may have more, or you may have less depending on whether you:
If you do have data, it only makes sense that safety managers can now leverage this carefully collected and classified data to maximize the potential use in their SMS. If you are just getting started, or you don't have the capability to capture and store SMS data, don't be too concerned yet. Here are some resources to help you get started on your SMS implementation. Just be aware that this article applies to you and will give you a taste of what to expect in the future as SMS regulatory auditors come to visit.
Some uses for using key performance indicator data include:
Setting goals and objectives is a requirement in every aviation SMS. KPI statistics are necessary to understand where your company has been for the past year(s). KPIs should be able to track the performance of your safety goals and objectives.
Carefully selected key performance indicators are essential to be able to monitor goals and objectives. You have to be able to easily track these KPIs as safety reports and audit data become available. You may have other safety goals and objectives that rely on data coming from safety surveys or interviews to help measure and monitor safety culture.
A centralized SMS database helps monitor safety goals and objectives more effectively and efficiently than a group of "point solutions." Point solutions are software products that do one thing, such as:
Point solutions require more work to monitor safety goals and objectives, but they are better than spreadsheets.
Data gathered for goals and objectives can originate from a multitude of different systems, even systems outside the control of the safety department. This point is driven home so the safety team thinks outside the box when they are looking for ways to take measurements or gather data to measure a particular objective.
The process of setting safety goals and objectives can be challenging for safety managers that have poor safety cultures or they don't know what sort of goals and objectives are reasonable in a new SMS implementation. More than once I have heard honest feedback from safety managers reporting that they use our default safety goals and objectives for their first year. It is a no-brainer if you have some inspiration. There is no rule against using predefined objectives that come from a template.
For those who are paying attention, I'll post these default goals here. They have helped many other safety managers and I'm sure this quick list will help you. But there is another reason I'm posting this following list. It is for you to be inspired as to what sort of data points will be required when you want to monitor your safety goals and objectives.
To help with the following data analysis, examples of safety goals are listed below. We'll analyze data sources by their respective numbers:
Based on the above goals, you can see you will need multiple data sources. I'll list these data sources with their associated number from the above list:
We can see a few major data stores that we must account for when we set our goals and objectives. This is a very important point I'm making. Here is a scenario:
Imagine you are in your annual or semi-annual safety review meeting. You are directing the safety goals and objectives discussion, and a Department Head asks for a particular safety goal or objective.
The second thing you should say after: "That is a great idea," is "Where will the data come from?"
Each objective that is tied directly to a goal will require at least one data point to monitor. These data points are the KPIs to verify the accomplishment of each safety goal and objective.
A hazard register offers employees and managers a glimpse into the most actively monitored hazards affecting the organization at that given time. KPI statistics should be used to determine which hazards belong at the top of the Hazard Register, and those hazards that pose minimal risk.
Not every hazard should factor into your KPI selection. Only choose those that are significant and meaningful to the success of your airline or airport operations.
A challenge of new and immature SMS implementations is that the safety manager doesn't have a hazard register. Furthermore, there is no sustainable way to monitor hazards effectively unless you are
The first approach is preferred, as copying and pasting reported safety issues from one system to another is a sloppy hack. Yes, if you are using "point solutions," then you may not have much choice. But the truth is that these point solutions are not long-term options. It becomes more difficult to monitor as your company grows and the safety culture matures.
If you are managing your hazard register and safety reporting system in a spreadsheet, you are in even worse shape to track KPIs unless you are very small, such as fewer than 50 employees at the most.
KPIs from the hazard register are the number one data source auditors look for.
Trending aviation SMS data can be surprisingly daunting for safety managers who lack adequate tools. Safety managers of both small and large operations tell us that they cannot manage the data manually.
Manually managing SMS data for tracking and analyzing trends is nearly impossible when you are simply the "part-time" safety manager. Part-time safety managers are the "norm" at most smaller organizations. Carefully selected KPIs allow safety managers to focus their trend analysis efforts on a significantly smaller subset of data.
Data collected for trend analysis comes normally from a database. You can do it manually with spreadsheets, but this system is:
There are very few safety managers that I know that like collating data and generating trend charts. The easiest way is to acquire a low-cost, commercially available SMS database that does it all for you. With the SMS database, any manager with the proper security access can easily generate trending charts and perform their own analysis.
Trending charts are a safety manager's best friend and there is MORE! Trending charts can be the safety manager's best tool to "win friends and influence operational department heads!" The SMS database can capture considerably more than safety reports. The SMS database can easily capture:
There is no sense in spending extra budget on other systems when the SMS can capture and monitor KPI data related to other operational areas. This is how your SMS database and SMS trending charts can help safety managers acquire "safety champions." When safety teams show operational department heads how their SMS data can help them make better business decisions, then you are on your way to reducing resistance to the SMS in that department.
Trending data tells a visual story that is hard to beat.
Aviation managers at all levels need to stay abreast of the major issues affecting the performance of the aviation SMS. These managers typically acquire this information from canned reports, dashboards, and automated trend-monitoring tools.
Dashboards could become easily overwhelmed with too much information. KPI statistics, therefore, are the perfect data points to incorporate into any well-developed dashboard.
Along the same line, automated trend monitoring tools should focus on meaningful and potentially costly elements affecting operations. When we consider the safety goals and objectives, we realize that many data sources are needed to present a full picture to management in order for them to be responsive to substandard safety performance.
What types of reports do we need to present to the accountable executive so that his responsibilities are fulfilled? Is it just safety report metrics? Or audit performance? Before we become too hasty and answer this question, what are the accountable executive's responsibilities to the SMS?
For every aviation SMS, accountable executives must ensure the SMS is properly implemented and performing across all areas of the organization, not just flight ops, or ground handling. Operations that could logically affect:
We can see from both this list and the requirement to ensure the SMS is properly implemented AND performing that we need to be able to monitor all SMS documentation requirements to ensure the SMS is performing as designed. Furthermore, the accountable executive is required to review organizational safety performance on a regular basis. Whenever substandard safety performance is identified, such as through reviews and audits, the accountable executive is expected to lead the team that fixes these problems.
Safety managers are at a distinct advantage when creating SMS performance monitoring dashboards. They don't usually understand the data requirements to create a particular chart, or they may not know how to access the underlying data to populate dashboard charts.
The easiest way is to acquire an SMS database that already has SMS performance monitoring dashboard charts. If you are building your own charts, you may be interested in knowing what sorts of charts to put into your dashboards. If you are an SMS database competitor, you are also interested in knowing what sorts of dashboard charts are being delivered to aviation service providers.
This list below (SMS KPI Performance Monitoring Charts) is meant to inspire you to:
Based on your organizational needs at any given time, these charts allow the accountable executive and safety teams to continuously monitor the health of the SMS. These charts can be tied to KPIs, but since KPIs are always evolving, a chart that is valuable one year may lose its value the succeeding year as organizational safety goals and objectives evolve.
This list above is a sample of charts that safety teams can build for the accountable executive and the safety manager to continuously monitor safety performance. Depending on your role in the aviation SMS, you may require different charts. The best solution is to have a user-configurable performance monitoring dashboard. This will allow each manager to focus on charts that are most relevant to the manager's duties and responsibilities.
KPI statistics hold a significant role in the predictive analysis functions of an aviation SMS. To be valuable in predicting the likelihood of certain types of events occurring, you must have meaningful and well-classified KPI data.
Predictive analysis is based on predicting future types of events based on historical data. You will be collecting and analyzing data to detect trends. Above we discussed trend analysis. We won't repeat it here, as there are additional resources listed above in the "Trend Analysis and Trend Identification" section.
Finally, KPI statistics are meaningless if they are not used to communicate risk to organizational stakeholders.
For safety promotion activities, consider using KPI statistics:
As you have seen, one can use KPI statistics in every component of an ICAO-compliant SMS. We see a tragedy every day when safety managers spend so much time each day collecting and classifying data, but they fail to capitalize on the benefits.
Hopefully, this post will prove useful for you to make more effective use of your KPIs. KPIs help managers focus on important elements of the SMS. As you have seen, most KPIs are best tracked using an SMS database. To comply with the intent of aviation SMS, one needs a database to spot trends unless you are an incredibly small company. A small company does not have enough safety data to easily spot a trend in time.
How big should a company be before it should invest in an SMS database? If you have more than 50 employees, you need an SMS database.
For smaller companies, you may get by with spreadsheets, but why suffer the pain? There are low-cost, commercially available SMS databases that take the pain away for even these smaller companies.
When you consider that the accountable executive has to monitor the entire SMS to ensure it is performing properly, then an SMS database is the best way to mitigate risk to the accountable executive.
For a few hundred dollars per month, these small companies can get a safety reporting system with all the risk management benefits, including SMS performance monitoring dashboards.
Why continue to struggle with point solutions and spreadsheets? The technology for a complete SMS data management solution has been available for over ten years. The technology is mature. Why wait?
We can help.
If you need help to monitor SMS performance, check out these videos. We have been developing SMS databases since 2007 and serve all industry segments.
Ready for a live demo of SMS Pro?
Last updated November 2024.