Purpose of Safety Performance Assessment in Safety Assurance Pillar
Safety performance assessment in FAA Part 5 compliance requirements plays a critical role in bridging the game between higher-level risk management process activities and day to day aviation SMS activities. More practically speaking, this component of safety assurance helps bridge the gap between the everyday hazard mitigation efforts, and large scale decision making and accountability for the overall direction of the safety management system.
By the time you get to safety performance assessment activities, you have:
- Been monitoring safety performance;
- Measuring safety performance;
- Gathering safety data; and
- Analyzing safety data as it is collected.
Now it’s time to make decisions about what to do with that data, based on:
- Effectiveness of risk controls;
- Changes in operational environment; and
- Newly identified hazards.
All of these are part of the decision-making process, which arise from analysis of safety data.
Safety Performance Assessment Is Where You Make Safety Decisions
Making safety decisions involves asking the following questions:
- Is the safety performance of the SMS program effective?
- Is the performance meeting safety goals and objectives?
- Is the performance consistent with compliance expectations?
- Is the performance consistent with the expectations of existing risk controls and documentation?
Answers to these questions will largely dictate what happens next. There are two general types of outcomes from assessment: overall satisfaction or overall dissatisfaction. If answers to these questions indicate dissatisfaction:
- “…the certificate holder must use the safety risk management process described in [the safety risk management process]” as described in Part 5.73(b); and
- The safety concern will also move on to the Continuous Improvement SA element, where safety controls will be developed to correct unsatisfactory performance
If the answers indicate satisfaction:
- It indicates that all aspects of safety performance for the safety element in question are satisfactory; and
- The element in question will cycle back through the SA process to be monitored, measured, and more data acquired and analyzed to ensure ongoing performance.
It’s very important to be clear about what your outcomes are, as they dictate which decision you make.
Outcomes of Safety Performance Assessment
After reviewing the questions listed in the previous section, there are four distinct outcomes, each with a distinct decision that should be made:
- Performance is acceptable and meets all important expectations, whereby performance will continue to be monitored;
- Performance is unacceptable, and the problem seems to lay in compliance/safety policy and/or risk controls, whereby the SA’s Continuous Improvement element will be triggered to develop corrective actions;
- Compliance and safety policy are acceptable, but performance remains unacceptable, whereby the SRM process should be triggered to review how the system addresses the safety concern; and
- New or uncontrolled hazards are discovered, whereby the SRM process will be triggered to document the hazard and/or needed risk control, and assess how the system addresses it.
These outcomes should be documented, whether it’s on a spreadsheet, paper, or aviation safety software.
How to Conduct Safety Performance Assessment
Part 5 indicates that conducting safety performance assessment involves using acquired data and analysis to assess:
- Compliance with your safety risk controls – 5.73(a)(1);
- Evaluate the performance of the SMS – 5.73(a)(2) – though this is a very vague requirement;
- Evaluate effectiveness of safety risk controls, including the identification of any ineffective controls – 5.73(a)(3);
- Identify changes in the operational environment that may induce new hazards – 5.73(a)(4); and
- Identify any new hazards – 5.73(a)(5);
Point number 2 and 4 above need some clarification, as they are quite vague.
What does it mean to evaluate the performance of your SMS? The important implication here is the performance of your SMS in terms of your goals and objectives. A significantly better way that Part 5.73(1)(2) can be phrased is to indicate that you should assess whether or not performance meets the expectations of your goals and objectives.
Secondly, “changes to the operational environment that may induce hazards” seems to be a long winded way of describing hazard mechanisms. Hazardous mechanisms are one of the following:
- Hazardous sources (i.e., a mountain, machine, chemical, etc.);
- Initiating mechanisms (Human factors, safety culture, environmental changes, etc.); and
- Targets of potentially dangerous condition, such as an aircraft, person(s), etc.
Undocumented changes to any of these areas should be considered potentially unsatisfactory for safety performance.
Final Thought: Indications of Safety Performance Assessment Compliance
Now, to the main point of this article that you probably care about most – how do you know if your safety performance assessment is compliant with Part 5.73 requirements? Here are some good indications that you will performance well on audits that deal with the SA Safety Performance Assessment process:
- When reviewing safety issues, you can clearly identify outcomes;
- There is a strong documentation trail that indicates assessment-outcomes, and how you proceeded further (i.e., a note or built in process indicating next steps taken);
- Specific evaluations for compliance, risk control analysis, identified hazards, and goals and objectives considerations; and
- Which data was used to inform these assessments.
The key word here is documentation. When you performance safety performance activities, SHOW YOUR WORK!
For a full overview of FAA compliance, you will find these resources valuable: