Why Compare Trends in Aviation SMS KPI Data?
Curiosity and upper management may frequently demand that safety managers create reports from safety data that compares their operations with their competitors or with a national, regional or global "average."
Displaying hazard reporting data over time increases managements' understanding of the real performance of airline and airports' safety processes.
Safety managers at all but the smallest aviation service providers are required to monitor the performance of their aviation SMS using data accumulated from the SMS documentation processes and detect trends.
Tracking trends using KPIs are naturally extended to the airline or airport's established goals and objectives that monitor the continuous improvement of the SMS.
Related Articles on Aviation SMS Goals and Objectives
- How to Shape Your Corporate Vision with Aviation Safety Goals and Objectives
- 3 Goal-Setting Tips for Aviation Key Performance Indicators
- Safety Chart: Maintain Monthly Aviation SMS Hazard Reporting Goals
This post describes an approach to compare your airline or airport's KPIs with those from your civil aviation authority, a national average, or against your competitors. These steps will include:
- Identifying the key performance indicator (KPI) of interest;
- Establishing the rate to compare the KPI against other aviation service providers;
- Determine the rate of occurrence at your airline or airport for this KPI of interest; and
- Run the math.
Identify KPI of Interest
Key performance indicators are commonly discussed in the news, civil aviation regulatory reports or aviation SMS trade journals. The media may not explicitly call them KPIs, but may use language similar to: "From 2010 to 2015, there were 2.5 bird strikes per 10,000 operations."
In this case, "Bird Strike" is the KPI to focus on.
Related Articles on Key Performance Indicators (KPIs) in Aviation SMS
- What Is a Key Performance Indicator (KPI) in Aviation SMS?
- How to Set and Monitor Key Performance Indicators (KPIs) in Existing SMS
- How to Automate Key Performance Indicator KPI Monitoring
Establish Rate to Compare KPI
In the scenario above, there were 2.5 events in 10,000 operations; therefore the rate of interest is 2.5 per 10,000. In most cases, your airline or airport will never see 10,000 operations. Your organization may have several hundred or tens of thousands of operations, but probably never the exact number under consideration.
The safety manager's challenge is to compare 2.5 events in 10,000 operations with the number of events at his organization. For the sake of this discussion, let's call the "10,000" our "normalization rate."
Determine KPI's Rate of Occurrence at Your Operations
In order to compare "apples to apples," safety managers will need to determine how many similar events their airline or airport had within the established time period. Aviation safety management databases should easily provide this KPI data, in this case, the number of bird strikes from 2010 to 2015. Safety managers will also need to know how many operations their organization performed each year from 2010 to 2015.
Trying to accumulate this data from spreadsheets will be too burdensome. You may make this comparison, but it will take considerable time and effort to aggregate the data. An SMS database is the proper technology is you are routinely working with statistical data across multiple disparate data sets.
Related Aviation SMS Database Articles
- What Is an Aviation Safety Database
- How to Choose the Best Aviation Safety Database Software
- How to Manage Aviation Safety Programs without Complex SMS Databases
Perform Math to Trend KPI Data
What remains is for the safety manager to adjust the values from these different scales to come to some comparison. In short, he must normalize his company data with the 2.5 bird strikes in 10,000 operations.
Normalizing rates multiply your KPI classification frequency by your normalization rate divided by an annual number of operations (departure/landings or hours).
# of events per year * normalization rate
Annual # Operations
What remains it to chart your data. This can be performed in MS Excel or your aviation SMS software may already provide a way to monitor KPI trends. The better aviation SMS software solutions have this ability.
Understanding KPI Trend Data
Your KPI data may be skewed. Having a high number of occurrences in a given time period or a lack of data may cause your aviation KPI trends to appear suspicious. In this case, you should understand:
Standard Deviation -- a measure of how spread out numbers is - the square root of the Variance.
Variance -- the average of the squared differences from the Mean.
I'll leave it to you to look up your college statistics professor.
What Is Important in Comparing KPI Trend Data
To accurately compare your KPIs against industry norms, you must:
- Accurately classify all reported safety events in your safety program; and
- Have an easy way to tally specific event type your airline or airport has experienced.
Remember, "garbage in is garbage out". When your manager asks you to compare your airline or airport's performance with others, remember this easy formula described above. You may wish to bookmark this post or even share it with others.
Staying on top of classifying reported safety issues is a BIG DEAL! When safety managers neglect to classify reported safety issues, they are wasting an opportunity to accurately identify trends and effectively monitor the health of the aviation SMS.
Reported safety issues may be classified in multiple ways. In a spreadsheet, there are some severe limitations, but an SMS database will normally allow you to classify reported safety issues in countless ways.
A quick rule of thumb is that if your organization has more than 50 employees, you need an aviation SMS database. If you are smaller and are under EASA jurisdiction, an SMS database is required to stored reported safety data. Spreadsheets are no longer permitted.
Related Articles on Using Spreadsheets in Aviation SMS
- See How Spreadsheet Not EASA Compliant Aviation Hazard Reporting Database
- 5 Things Spreadsheets Can't Do for Your SMS
- Spreadsheets vs Software for Aviation Safety Management
When you are normalizing data to monitor KPI trends, you will need proper tools to have a sustainable SMS performance monitoring process. Otherwise, the process will not be sustainable and management cannot be assured of the SMS requirement of safety performance monitoring and measurement are being met.
If you lack tools to adequately monitor the performance of your SMS, we can help.
Not ready for a live demo? Watch demo videos instead.
See common airline or airport key performance indicators.
Published September 2018. Last updated February 2021.
Members of Patrol Squadron (VP) 16 fuel up... image by PICRYL