Safety performance monitoring and measurement is an extremely important part of Safety Assurance. It’s how you verify the safety performance of your organization.
By safety performance, we are talking about:
A large portion of aviation SMS safety performance monitoring and measurement rests on your hazard reporting system. This is because your primary means of collecting safety data is through hazard reporting.
Here is how to be compliant with ICAO’s safety performance measurement and monitoring requirements.
The first requirement is that reported issues from your safety reporting system are used to generate sufficient information (including metadata) that can be used during safety performance monitoring.
You will use this data to:
This requirement is met, so long as you can show that reports being submitted are actually being managed.
An aviation SMS should indicate in the Safety Policy which issues are required to be reported and which issues can optionally be reported. Broadly speaking, mandatory issues are types of issues that closely indicate your SMS performance in mitigating risk and meeting compliance.
Safety issues that should be mandatorily reported are:
You should train employees on which issues are mandatory to report and provide guidance that is easy to access.
A best practice is to have employees report everything that could potentially introduce operational risk. This includes ALL close calls. Close calls are especially important to report because their data helps to stop the next accident. Getting employees to report close calls may be difficult at first due to various human factors, including:
Reporting close calls may be mandatory or voluntary, depending on your safety culture. Upper management may resist the additional burden of managing these close calls. It will be imperative to train employees and management about the importance of reporting all these close calls to reduce organizational risk and losses.
Effective hazard reporting performance relies upon a healthy safety reporting culture and reducing resistance to the aviation SMS.
In addition to having mandatory safety issues to report, you should indicate examples of voluntary issues to report. Voluntary issues are issues that are not necessarily critical for measuring safety performance but can provide very useful data for:
In simple terms, having a strong reporting culture for reporting voluntary issues gives your SMS performance context. Strong reporting cultures employ effective hazard identification training to educate employees as to what a hazard is so they can become more engaged in identifying operational hazards and associated risks.
Employees must also be familiar with risk controls so they know how controls keep the "system" safe to mitigate risk to as low as reasonably practical.
Your reporting system must have a means of reporting confidentially. In addition to being a best practice for developing a high-quality safety culture, this is a good way to encourage employees to submit important, sensitive data that might otherwise go unreported.
When sensitive issues are being reported, you can rest assured that your SMS performance monitoring is covering all aspects of the SMS. Confidential issues are usually handled by one person in your company who is not directly tied to the SMS, such as an HR manager. Other employees and managers should not be aware when such issues are reported, nor who reported them.
Confidential reporting may or may not be the same as "anonymous reporting" in your aviation SMS. This is dependent on your organization's:
Anonymous reporting means exactly what it says, i.e., you should not know who reported the safety concern. I have to say that I'm impressed with the level of professionalism displayed by aviation service providers over the past dozen years. Never have we been asked to disclose who reported an anonymous report. Safety teams understand that once the trust is broken, it will be nearly impossible to regain the trust of the extant safety team.
Confidential reporting is not anonymous reporting. Safety teams can see who reported any safety report, but the report is treated as "confidential," meaning that the report will not be released to managers outside the safety department. This is the most common workflow for confidentially reported safety issues, but we have seen (and use) two other strategies for managing confidential issues within an SMS.
A small percentage of aviation service providers designate a single employee who can access (and subsequently manage) confidentially reported issues. This is obviously not a best practice as there is a single point of failure. What happens when this designated "confidential report manager" takes a vacation or is overwhelmed with daily tasks? This strategy to manage confidentially reported safety concerns should only be used by smaller organizations.
Another strategy to manage confidential reports is to assign a committee or team that can both access and manage confidentially reported issues.
A best practice for confidentially reported issues is to ensure all employees understand the difference between anonymous and confidential issue reporting to avoid losing trust.
In an additional effort to encourage safety reporting and generating data collection, ICAO has mandated that you provide:
When these requirements are satisfied, your hazard reporting culture is bound to be much better than if you hadn’t fulfilled them. Better hazard reporting means better data collection resulting in more thorough performance monitoring.
Hazard identification training is understated in aviation SMS. Even in our office, I think that some employees would not know a potential hazard if it was standing next to them. Hazard identification training does not have to be formalized, week-long training courses. Use your safety promotion activities to educate employees a bit at a time to ensure more complete coverage. Use:
Obviously, there is more to your aviation safety management program than just reporting issues. Issues come up during other operations besides normal, front-line operations. Your other operations should be integrated into your reporting system, such as
During these operations, you will naturally uncover information that is highly relevant to the performance of your SMS. Such information (data) needs to be integrated (reported) into your SMS.
From quickly scanning the above list of potential data sources, you quickly realize that spreadsheets are not the desired solution to sustainably manage all this SMS data. A few years ago, the European Union recognized that spreadsheets have neither the flexibility nor the capability to effectively manage safety report data. In short, an SMS database is required to have a performant SMS that will scale with your operational needs for many years to come.
The final output of safety performance monitoring and measurement is Safety Performance Indicators (SPIs/KPIs). These are metrics or other data that you use to track the core, most important issues in your SMS.
SPIs should reflect
The implication here is that safety issues you identify during safety performance monitoring and measurement should be adopted as safety objectives. This is the SMS feedback loop between objectives and performance monitoring.
Hazard reporting, auditing, tracking safety performance... all these activities require modern SMS data management strategies if you ever wish to fully benefit from your SMS.
An SMS database is not for everyone. Don't bother getting an SMS database if:
Very small aviation service providers can get by with spreadsheets and paper; however, it won't be pretty. If you are small and have significant employee turnover, an SMS database will save you bacon.
We can help with a low-cost, commercially available SMS database that has been used worldwide since 2008.
Not ready for a demo? Check out these videos first. If it looks like you can use our system, then you can request a live demo at a later date.
Last updated November 2024.