What Is Safety Assurance in Aviation SMS

For an aviation safety management system (SMS) to be effective, the SMS must be monitored to assure managers and regulatory authorities that the SMS is both

• implemented correctly, and
• working as designed.

An ICAO-compliant SMS is structured around four main components, which are also known as the Four Pillars. The third aviation SMS component is Safety Assurance (SA). The primary purpose of safety assurance is to demonstrate the SMS remains functional and alert management of substandard safety performance. Substandard safety performance is corrected under the direction of the Accountable Executive.

Related Safety Assurance Articles

• FAA Part 5 - Safety Assurance - You May Not Be Prepared
• FAA Part 5 Compliance | Safety Assurance Performance Monitoring and Data Acquisition Components
• FAA Part 5 Compliance | Safety Assurance Continuous Improvement

Analysis of Data Requires a Repeatable Process

Analysis of Data is the third element in the FAA’s Part 5 Safety Assurance requirements. It is a natural response to the Data Acquisition process. Data analysis is a process. This means two things to you:

1. This process should have multiple steps; and
2. These steps should be repeatable.

Moreover, organizing data analysis as a process ensures that you can make needed changes and improvements to the process that increase efficiency during risk exposure assessments.

When developing processes for your aviation SMS, it is best to have a template to structure your safety processes intuitively. The most logical structure for your SMS process is the use of "Safety Attributes." Safety Attributes (or process attributes), are inherent characteristics of any type of system and apply to an effective aviation SMS.

Safety process attributes have safety requirements built into their design. If you do not have a template to organize your processes, consider using these safety process attributes below to describe your processes. You can find more guidance about safety attributes from the FAA's Safety Management System (SMS) Framework, Revision 3.

1. Responsibility and Authority to complete required activities;
2. Procedures for staff to follow;
3. Controls ensure correct safety outputs are realized;
4. Process Measures of both the processes and their outputs;
5. Interfaces with other departments, both internal and external.

Description of Safety Assurance's Analysis of Data Requirement

Analysis of Data is found in the FAA’s Part 5 .71(b) requirements, and notes that:

The certificate holder must develop and maintain processes that analyze the data acquired through the processes and systems identified [in Data Acquisition] and any other relevant data with respect to its operations, products, and services.

Such data analysis will often be performed on an issue-by-issue basis. That is, safety reports will be submitted (data acquisition) and then processed (analysis of data). Service providers tend to analyze the data from these reports in isolation – this is the easiest way after all. “In isolation” means not considering how the data from a report fits in with the overall trends.

Analysis of Data needs to always happen from the viewpoint of how the current data fits in with the grander scheme of the SMS program’s data trends. It can significantly impact how data from reports are viewed.

Related Safety Assurance Articles

• 4 Pillars | What Safety Assurance Really Means in Aviation SMS
• 40 Questions for Your Safety Assurance Process in Aviation SMS [With Free Checklists]
• Best Tip for Safety Assurance Monitoring in Aviation SMS

Develop Process to Analyze Data

Part 5 says that certificate holders must “develop” a process that analyzes data. Do not confuse “develop” with “maintain,” as they imply two different activities.

Develop means to create the framework for data analysis. This framework should:

• List data sources for analyses;
• Document how data will be analyzed;
• Specify how data will not be analyzed in isolation; and
• Show clear steps in the data analysis process.

Developing an analysis process is the bureaucratic arm of the data analysis team. This team should include:

• senior management involved in safety performance monitoring;
• the safety team;
• the quality assurance team (if one exists);
• subject matter experts; and
• any IT staff that are familiar with potential data sources, such as the safety database and/or auditing database.

Maintain Process to Analyze Data

Part 5 documentation says that in addition to developing the data analysis process, certificate holders must “maintain” a process for analyzing data. As said, development and maintenance are distinct from each other. Maintaining a process means actually practicing data analysis in your organization by:

• Following the data analysis process that is developed by the data analysis team;
• Using the process to analyze safety reports;
• Letting data analysis results inform further components of Safety Assurance (i.e., Safety Performance Assessment); and
• Refine the process as the SMS matures and data acquisition strategies improve.

Showing that you maintain your data analysis process means demonstrating specific risk management processes in which data analysis was performed. There is no requirement to say which specific risk management phase, such as reactive, proactive, or predictive risk management phases. As an example, let's consider a reported safety issue that has been treated in your aviation SMS' reactive risk management processes. An auditor may ask:

• How did you arrive at your findings?
• How did you make data-based safety decisions?
• How did you choose risk controls?
• How are implemented risk controls monitored?

If you can show answers to these questions for any given safety issue in your SMS database or spreadsheet, then you are clearly maintaining your process so long as what you show aligns with your documented data analysis process. Obviously, an SMS database will be superior over spreadsheets when demonstrating SMS compliance with the data analysis requirements.

Related Articles Using Spreadsheets in Aviation SMS

• 5 Things Spreadsheets Can’t Do for Your Aviation SMS
• Spreadsheets vs Software for Aviation Safety Management
• How Spreadsheets Not EASA Compliant Aviation Safety Reporting Database

Analysis of Data of “Products and Services”

It’s easy to focus Safety Assurance activities as being exclusive to operations. However, the FAA makes it clear in 5.71(b) that data analysis is pertinent to “products, and services” as well. In fact, the FAA goes so far as to bold this portion of the requirement in order to drive home the notion that Safety Assurance requirements include quality management system elements as well. For this reason, combining SMS and QMS makes a lot of sense.

The fact is, faulty product and service operations can have strong safety implications in addition to business-quality implications. Just consider the recent American Airlines problem regarding injuring a passenger when removing him from a flight, and suffering major losses due to negative media attention.

This is a perfect example of service quality operations having significant safety implications. Your data analysis should include products and services where applicable.

Goals and Outcomes of Data Analysis Process

The primary goal of the FAA's Part 5 Safety Assurance Analysis of Data element is to organize and examine data. Your data analysis process should cover the following:

• Establish how the current safety concern fits into the overall context of your SMS implementation;
• Ensure that all needed data is at hand;
• Establish and hone data analysis methodology;
• Compare current safety performance against historical safety performance; and
• Allow you to prepare safety reports.

In smaller SMS implementations, analysis of data can be done by one person, such as the designated safety manager. In larger organizations, analysis of data will probably be completed by a dedicated team of individuals who have access to the SMS database and other data sources.

Related Aviation SMS Database Articles

• What Is an Aviation Safety Database for SMS Implementations?
• How to Choose the Best Aviation Safety Database Software
• 7 Signs You Need an Aviation Safety Management (SMS) Database

Final Thoughts on Analysis of Data Requirement

Analysis of data is not simply for demonstrating safety performance to regulatory agencies. There are actually opportunities to be seized.

During the reactive risk management processes, safety teams should be most interested in identifying the root cause(s) of safety events to prevent a recurrence. However, this is not the only time organizations benefit from sincere data analysis. The data analysis process also facilitates brainstorming sessions and stimulates managers to uncover potential new hazards and determine where operational processes may be improved.

Too often, managers are overly concerned with addressing an auditor's needs while neglecting organizational opportunities. When you conduct your data analysis, keep an open mind. Who knows what may surface during these data analysis activities?

When processes are cumbersome, they are not sustainable. This applies to any process, and not simply data analysis processes.

When safety managers spend too much time sifting through data, they won't have adequate time to accomplish their other SMS duties, such as safety promotion activities to improve safety performance. Spreadsheets are easy to use but are not suitable for collecting, storing, and retrieving years' worth of SMS data for trend analysis. Without a doubt, a low-cost, commercially available SMS database is the safety professional's choice when continually being confronted with demonstrating safety performance.

If you have difficulties in demonstrating safety performance or analyzing your safety data, we can help. Please watch these short demo videos of a complete, robust aviation SMS database solution.

Last updated in September 2023.