What Is Safety Performance Monitoring and Measurement

Safety performance monitoring and measurement is an extremely important part of Safety Assurance. It’s how you verify the safety performance of your organization.

By safety performance, we are talking about:

• How well your safety risk controls mitigate risk;
• How well your SMS is reaching safety performance indicators; and
• How well your SMS is maintaining compliance.

A large portion of aviation SMS safety performance monitoring and measurement rests on your hazard reporting system. This is because your primary means of collecting safety data is through hazard reporting.

Related Aviation SMS Hazard Reporting Articles

• What Is Missing Most in Aviation Hazard Reporting Systems?
• Indicators of Good Hazard Reporting Culture
• How to Develop a Hazard Reporting System in Aviation SMS [With Free Checklist]

Here is how to be compliant with ICAO’s safety performance measurement and monitoring requirements.

5.3.66 Safety Performing Provides Data

The first requirement is that reported issues from your safety reporting system are used to generate sufficient information (including metadata) that can be used during safety performance monitoring.

You will use this data to:

• Assign related organizational elements, such as classifications;
• Identify risk control performance;
• Risk assess safety issues and rank them; and
• Analyze all other relevant safety performance concerns.

This requirement is met, so long as you can show that reports being submitted are actually being managed.

Related Aviation SMS Safety Performance Monitoring Articles

• 4 Pillars | How to Conduct Safety Performance Monitoring and Measurement
• 5 Useful Safety Performance Monitoring Tools in Aviation SMS
• FAA Part 5 Compliance | Safety Assurance Performance Monitoring and Data Acquisition Components

5.3.68 Have a Mandatory Reporting System

An aviation SMS should indicate in the Safety Policy which issues are required to be reported and which issues can optionally be reported. Broadly speaking, mandatory issues are types of issues that closely indicate your SMS performance in mitigating risk and meeting compliance.

Safety issues that should be mandatorily reported are:

• Issues related to your safety performance indicators (SPIs/KPIs);
• Issues required to be reported to aviation compliance authority;
• Any issues that are high risk; and
• Any other issues that you feel are critical to your SMS performance.

You should train employees on which issues are mandatory to report and provide guidance that is easy to access.

A best practice is to have employees report everything that could potentially introduce operational risk. This includes ALL close calls. Close calls are especially important to report because their data helps to stop the next accident. Getting employees to report close calls may be difficult at first due to various human factors, including:

• Fear of looking bad;
• Fear of drawing attention to the self or operational area; and
• Fear of supervisor giving an employee the "stink-eye."

Reporting close calls may be mandatory or voluntary, depending on your safety culture. Upper management may resist the additional burden of managing these close calls. It will be imperative to train employees and management about the importance of reporting all these close calls to reduce organizational risk and losses.

Effective hazard reporting performance relies upon a healthy safety reporting culture and reducing resistance to the aviation SMS.

Have You Read

• How to Reduce Resistance to Aviation SMS Programs With Difficult Employees
• 4 Tools to Find Resistance to Aviation SMS
• Checklist to Quantify Resistance to Your Aviation SMS

5.3.69 Have a Voluntary Reporting System

In addition to having mandatory safety issues to report, you should indicate examples of voluntary issues to report. Voluntary issues are issues that are not necessarily critical for measuring safety performance but can provide very useful data for:

• Establishing trends;
• Identifying weaknesses in your SMS;
• Identifying needs for further risk controls;
• Identifying risk controls that are performing well; and
• Globally helping you understand your SMS performance better.

In simple terms, having a strong reporting culture for reporting voluntary issues gives your SMS performance context. Strong reporting cultures employ effective hazard identification training to educate employees as to what a hazard is so they can become more engaged in identifying operational hazards and associated risks.

Employees must also be familiar with risk controls so they know how controls keep the "system" safe to mitigate risk to as low as reasonably practical.

Related Aviation SMS Articles

• Difference Between Hazards, Risks & Control Measures in Aviation SMS
• How to Control Risk in Aviation SMS
• How to Monitor the Effectiveness of Control Measures

5.3.70 Have Means of Confidential Reporting

Your reporting system must have a means of reporting confidentially. In addition to being a best practice for developing a high-quality safety culture, this is a good way to encourage employees to submit important, sensitive data that might otherwise go unreported.

When sensitive issues are being reported, you can rest assured that your SMS performance monitoring is covering all aspects of the SMS. Confidential issues are usually handled by one person in your company who is not directly tied to the SMS, such as an HR manager. Other employees and managers should not be aware when such issues are reported, nor who reported them.

Confidential reporting may or may not be the same as "anonymous reporting" in your aviation SMS. This is dependent on your organization's:

• safety culture;
• safety policy; and
• technical ability to provide both confidential and anonymous reporting.

Anonymous reporting means exactly what it says, i.e., you should not know who reported the safety concern. I have to say that I'm impressed with the level of professionalism displayed by aviation service providers over the past dozen years. Never have we been asked to disclose who reported an anonymous report. Safety teams understand that once the trust is broken, it will be nearly impossible to regain the trust of the extant safety team.

Confidential reporting is not anonymous reporting. Safety teams can see who reported any safety report, but the report is treated as "confidential," meaning that the report will not be released to managers outside the safety department. This is the most common workflow for confidentially reported safety issues, but we have seen (and use) two other strategies for managing confidential issues within an SMS.

A small percentage of aviation service providers designate a single employee who can access (and subsequently manage) confidentially reported issues. This is obviously not a best practice as there is a single point of failure. What happens when this designated "confidential report manager" takes a vacation or is overwhelmed with daily tasks? This strategy to manage confidentially reported safety concerns should only be used by smaller organizations.

Another strategy to manage confidential reports is to assign a committee or team that can both access and manage confidentially reported issues.

A best practice for confidentially reported issues is to ensure all employees understand the difference between anonymous and confidential issue reporting to avoid losing trust.

Related Confidential Reporting Articles

• Confidentiality in Aviation Safety Management
• Learn How Confidential Aviation Hazard Reporting Systems Offer Assurance to Employees
• Why Employees Shouldn’t See All Reported Safety Issues in Aviation SMS

5.3.71 Provide Education and Availability for Reporting

In an additional effort to encourage safety reporting and generating data collection, ICAO has mandated that you provide:

• Education for how to report issues in your system;
• Education on benefits of reporting;
• Positive feedback to employees who report;
• A hazard reporting system that is easily accessible, such as by having multiple ways to report hazards.

When these requirements are satisfied, your hazard reporting culture is bound to be much better than if you hadn’t fulfilled them. Better hazard reporting means better data collection resulting in more thorough performance monitoring.

Hazard identification training is understated in aviation SMS. Even in our office, I think that some employees would not know a potential hazard if it was standing next to them. Hazard identification training does not have to be formalized, week-long training courses. Use your safety promotion activities to educate employees a bit at a time to ensure more complete coverage. Use:

• Safety surveys to educate;
• Safety newsletters to both inform and educate; and
• Safety messages that require documented sign-off, such as read files.

Related Safety Promotion Articles in Aviation SMS

• From Reactive to Proactive Hazard Identification in Aviation SMS
• 4 Tips to Approach Hazard Identification in Aviation SMS
• Aviation SMS Surveys - an Often Neglected Safety Promotion Tool

5.3.72 Have Other Methods of Gathering Performance Data

Obviously, there is more to your aviation safety management program than just reporting issues. Issues come up during other operations besides normal, front-line operations. Your other operations should be integrated into your reporting system, such as

• Auditing process for all internal audit operations;
• Safety surveys used as checklists for job tasks or to gather data about employee participation;
• Internal investigations and inspections during issue management;
• Safety reviews used during change management; and
• Safety studies, which are high-level, periodic reviews of broad safety issues or trends.

During these operations, you will naturally uncover information that is highly relevant to the performance of your SMS. Such information (data) needs to be integrated (reported) into your SMS.

From quickly scanning the above list of potential data sources, you quickly realize that spreadsheets are not the desired solution to sustainably manage all this SMS data. A few years ago, the European Union recognized that spreadsheets have neither the flexibility nor the capability to effectively manage safety report data. In short, an SMS database is required to have a performant SMS that will scale with your operational needs for many years to come.

Related Articles on Using Spreadsheets in Aviation SMS

• See How Spreadsheet Not EASA Compliant Aviation Hazard Reporting Database
• 5 Things Spreadsheets Can’t Do for Your SMS
• Spreadsheets vs Software for Aviation Safety Management

5.3.73 Create Safety Performance Indicators

The final output of safety performance monitoring and measurement is Safety Performance Indicators (SPIs/KPIs). These are metrics or other data that you use to track the core, most important issues in your SMS.

SPIs should reflect

• Your core company objectives; and
• Your core company safety needs.

The implication here is that safety issues you identify during safety performance monitoring and measurement should be adopted as safety objectives. This is the SMS feedback loop between objectives and performance monitoring.

SMS Databases Increase Safety Performance

Hazard reporting, auditing, tracking safety performance... all these activities require modern SMS data management strategies if you ever wish to fully benefit from your SMS.

An SMS database is not for everyone. Don't bother getting an SMS database if:

• You don't expect to be operating for more than a couple of years; or
• You only want to "check the box" (there is a valid business reason for this); or
• Your company is very small (less than 20-30 employees).

Very small aviation service providers can get by with spreadsheets and paper; however, it won't be pretty. If you are small and have significant employee turnover, an SMS database will save you bacon.

We can help with a low-cost, commercially available SMS database that has been used worldwide since 2008.

Not ready for a demo? Check out these videos first. If it looks like you can use our system, then you can request a live demo at a later date.

Last updated November 2024.