How to Review Safety Issues in SMS Programs

Which Issues Should Be Reviewed?

Different organizations will have different requirements when they review safety issues. Whether or not you are reviewing low-risk issues is up to you. While we recommend reviewing all issues, the reality is that due to time and resource constraints, this is not always feasible. The important point here is that you will decide which issues to review based on their risk assessment.

At the least, you should review:

• High risk issues; and
• Medium risk issues.

If you have the resources and time, you will ideally review low-risk issues as well. Why review low-risk issues? You never know what change will take place. A risk control that becomes obsolete can significantly change the risk assessment level of the issue(s) that it was mitigating.

An even simpler way to view this problem is: all issues that were initially assessed as being outside your definition for Acceptable Level of Safety should always be reviewed after they are closed. Beyond that, it’s up to your organization.

Here are the main tasks you need to perform when reviewing reported safety issues.

Review All Details of the Issue

Chances are, you have forgotten many of the details of this issue. Having good documentation is key for this. A recurring theme we hear about during audits is safety managers not having enough documentation, and being unable to answer pointed questions about why certain decisions were made on safety issues.

When reviewing issue details, are concerned with several things:

• What were the facts – i.e, the narrative – of the issue;
• What decisions were made; and
• The reason each decision was made.

In short, in the review process you should have a narrative of “what happened” during the issue and “what happened” during issue management.

Ensure Root Cause Analysis Findings Are Reasonable

Given the details of the issue, review your root cause analysis. With a fresh set of eyes, answer the following questions:

• Does root cause analysis follow a logical pattern (i.e., 5-whys analysis)?
• Is the reason each root cause exists “just because”?
• Does each root cause make sense as a root cause of the current issue?
• Is the issue missing any root causes?

If root causes need to be removed, added, or changed, you should reassign root causes to accurately reflect your new findings.

Related Aviation Root Cause Analysis Articles

• How to Conduct Root Cause Analysis in Aviation SMS
• Is Root Cause Analysis Proactive or Reactive?
• Understanding "Root Cause Analysis" Charts in Aviation SMS Dashboards

Review Classifications by Comparing to Performance Trends

Next, you need to review other classifications you made, including:

• The primary hazard you assigned to the issue;
• The type of issue classifications; and
• Human Factor classifications.

If you documented a risk statement (if [hazard]… then [most likely risk occurrence]), you should ensure that the hazard identified in your risk assessment matches your hazard classification. Much in the same manner as reviewing root causes, ensure that your classifications make sense given the narrative.

As before, if your classifications need to be removed or added, document your update.

Review Performance of Corrective Actions for ALoS

Now that your analysis and classifications have been reviewed, you should be able to assess whether or not the corrective actions that were created in the issue accurately address the root causes, and other factors involved in the issue occurrence.

If there were needed corrective actions that were not created during the issue’s management, or if the corrective actions did not bring the issue within an Acceptable Level of Safety, then you will need to reprocess the issue through the Safety Assurance process. Fortunately, this is needed only rarely.

More likely:

• You will find that all corrective actions were necessary and managed well; or
• You will find that all corrective actions were necessary, but perhaps some of them were not performed as well as they could have been.

If you have been giving CPAs an effectiveness rating, reviewing CPAs will be extremely straightforward. Otherwise, you will have to review the issue and then make another assessment of the performance.

Review Risk Assessments

Finally, review the risk assessments that were given to the issue to ascertain the level of exposure associated with the issue. This includes:

• Review the initial assessment;
• Review the closing assessment; and
• Review any reassessments performed between initial and closing, including why the issue was reassessed.

Now it’s time to answer some important questions about these assessments.

• Does the initial risk assessment match the conditions at the time?
• Does the closing risk assessment accurately take into account all control measures, classifications, and CPAs of the issue?
• Since the issue was managed, have any other important details or insights been discovered that affect the closing risk assessment?

You might, for example, discover that one of the issue’s primary mitigate risk controls has recently been discovered as non-performant. In this case, the closing risk assessment would not be accurate.

Related Aviation Risk Assessments Articles

• What Types of Risk Assessments You Should Perform in Aviation Safety
• How to Perform Risk Assessments Without Aviation Risk Management Software
• How to Justify Severity of Risk Assessments - Best Practices

Reassess Risk

Now you should reassess the issue, taking into account present circumstances:

• Are the CPAs performant?
• Are the risk controls still actively mitigating the main concerns in this issue?
• There have been NO reoccurring safety incidents between now and the time this issue occurred.

If the answer is yes to all of these questions, you can expect the reassessment to either be the same as the closing assessment or perhaps even lower risk. This is exactly what you are looking for. Ensure that you document your reassessment, with strong reasoning for your reassessment.

Set Next Review Date, Close Issue, or Reprocess Through Safety Assurance

If your reassessment is higher risk than the closing risk assessment, then you should strongly consider either:

• Setting a next review date in the near future; or
• Simply reopening the issue and reprocessing it through the Safety Assurance process.

If your reassessment was the same/lower as the closing risk assessment, you can either:

• Set a next review date that is consistent with your policies (usually ~3-4 months); or
• If no next review is needed, you can remove the issue from your validation process.

We suggest that high-risk issues remain in a continuous loop through your issue validation/review process. Low/medium risk issues may be removed from your validation loop as needed.

For more information about reviewing your safety issues, you might find the following two resources helpful: the first is a helpful quiz to see how equipped you are to do in-depth reviews, and the second is demo videos showing you what the best in class review process looks like.

Last updated April 2024.