The Importance of Data in Aviation SMS Programs
High functioning safety management processes always have high quality data acquisition and management practices.
While the great majority of safety management system guidance is designed around the bureaucratic structure of your safety program, the basic fact is that aviation SMS programs depend as much (if not more) on bureacratic structure as on data management.
Just some of the critical safety functions that data provides for SMS programs are:
- Safety decision making;
- Updates to policies, procedures, risk controls, processes, etc.;
- Safety performance monitoring;
- Continuous improvement;
- Safety risk profile;
- Hazard identification;
- Safety issue management;
- And so on.
The list is quite extensive, but you get the idea. Now, when we are talking about data, we are talking about four different types;
- Reportable occurrence data: required to be submitted by oversight regulations;
- Voluntary occurrence data: incidents that employees can submit to assist with safety improvement;
- Observation data: hazard identification in everyday operations; and
- Surveillance data: safety concerns that arise out of specific risk management practices, such as audits, inspections, etc.
Here are best practices for data management in aviation SMS programs.
1 - Standardizing Data with Common Names/Acronyms
Standardizing data is a critical task that you need to implement as soon as possible in your SMS program. Unfortunately, lack of standardization is a problem many risk management programs run into later in their programs. What is the problem with this:
- Data that is hard to manage;
- Data trend analysis that may be unreliable; and
- Data that is hard to mine.
Consider the following scenario (that we have observed in real organizations). A well-developed program has no standardization for issues revolving flight go-around. Over the course of several years, different managers give such issues different names like, “Go around,” “Fly around,” “Overshoot,” as well as several other variations of these terms.
Trying to establish any historical information and trends concerning overshoots will be very difficult because safety managers will need to track down these issues based on many different search names.
From the get go, organizations need to have standard naming conventions such as:
- Prefixes for common issues to use in issue title;
- A list of names that should be given for commonly occurring issues;
- A number-key associated with commonly reported issues that can be included in issue titles, such as Overshoot = 01, Bird Strike = 02, etc.
Compiling these standardizations in a table will ensure that safety mangers can easily look up the correct “code” and include it in the title of each safety issue.
2 - Acquire Best Data Collection Tools
Getting data acquisition tools is absolutely, critically important to your ability for data management. After all, no data – no data management. Here are the four tenants of best-in-class data acquisitions tools:
- Multiplicity: employees and public should have multiple ways or reporting safety issues, such as on a website, through email, through a tool, on their cell phones, etc.
- Usability: Hazard reporting tools should be very user friendly;
- Availability: the available hazard reporting tools should be at employees’ fingertips wherever they are; and
- Detailed: hazard reporting tools should capture all information – enough to capture all relevant information but not so much information to make reporting issues a hassle.
Get your data acquisition tools early. It will help jump start your hazard reporting culture and save you from having to implement a significant change management for new hazard reporting process/tool later.
3 - Planning for Data Collection
Once data is reported, you need to know what to do with it. This is where oversight agency compliance guidelines touch on data management. Part of having a risk management process and safety assurance process involves having a process for handling data.
This includes processes for:
- Validation; and
Establish these processes and test them out early. The better and more efficient your processes, the better you will be at practicing good data management.
4 - Ensure that Data is Valid
Stored data needs only be valid data. There are two types of validity:
- That hazard reports are relevant and truly safety concerns; and
- That data in the hazard reports is realistic, reasonable and true.
Safety reports that come in that are not valid should be documented and discarded. Why? Because you don’t want your data watered down and complicated by irrelevant information.
Also, safety hazard reports needed to be vetted for invalid information. For example, common marks of invalid information on reports are:
- Incorrect, missing, or poorly formatted dates and times;
- Poor classification choices;
- Wrongly checked boxes on reports;
- Incorrectly listed departments, divisions, locations, etc.; and
- Wrong report filled for given type of issue.
Looking for these kinds of invalid information is referred to as a “reasonability check.” All reports should be heavily scrutinized for these mistakes, and corrected as necessary.
Risk management processes that don’t have a built-in vetting process for invalid issues are exposed to conclusions that are incorrect or misleading.
5 - Ensure that Data is Complete
Concerning data, completeness is a measurement of:
- How much data is needed; and
- How much data is available (for a given analysis).
If you have enough data for a given analysis, then your data is complete. But what does it mean to be “complete”? There is no one answer here, but there are several factors that will help you arrive at what “complete” means for you:
- How detailed is your analysis?
- How complex is your analysis?
- How many resources do you have at your disposal?
The implications of these questions involve an interesting revolving door:
- The more data you need, your more resources you need;
- The more complex your analysis (such as complex data comparison), the more resources you need;
- The more complex your analysis, the more detailed and specific information you need; and
- The more specific your analysis, the more resources you need.
So, do you find yourself having to constantly go back and ask reporters for more information? Are you having trouble or feeling insecure about your classifying safety issues? Such considerations will help you know if you have the information you need.
Here are some great tools for regarding data management in aviation SMS programs: