How to Audit Previously Managed Safety Issues in Aviation SMS

What Is an Audit of Previously Managed Issues in Aviation Safety Management

When you perform internal aviation SMS audits, it’s essential you include the auditing of previously managed issues.

Why should you audit previously managed issues? Because external auditors will randomly sample your managed issues and look very closely at the decisions you made and why you made them. If you don’t randomly sample and audit your issues, you can probably expect an audit finding on your next external audit.

Not only does this make sense, but pretty much every aviation safety manager we have worked with has been handed an audit finding for mismanaged issues. As a safety manager, department head, or accountable executive who is responsible for issue management, these kinds of audit findings:

• Reflect directly on your performance;
• Were preventable; and
• Don’t shed a good light on your aviation safety management system.

The reason why you would particularly wish to avoid these audit findings is that of all compliance aspects of your SMS, managing issues are the most active and most within your direct control.

Furthermore, when all randomly sampled issues look good, it reflects the fact that you are practicing due diligence.

Process for Auditing Managed Issues in Aviation SMS

The process for performing an internal SMS audit in managed issues is simple:

• Gather all completed issues from the past year;
• Have a small team/committee gathered to audit the issues;
• Randomly choose 2-3 issues with initial risk assessments of the low, medium, and high (for a total of 6-9 issues to audit)
  • You can truly select all issues randomly, or
  • You can select some random and some “memorable” issues
• Audit each issue with the committee.

Having a committee is a good idea because it ensures multiple perspectives on an issue, where you may uncover problems that would not be identified with just one auditor.

Goals of Auditing an Issue vs Reviewing an Issue in Aviation Safety Program

Auditing an Issue and reviewing an issue are not the same things. Reviewing an issue is part of a normal process you will do on all or some of your issues, such as those with initial risk assessments of moderate to severe.

The purpose of an initial risk assessment is to:

• Ensure that the risks of the issue remain within an acceptable level of safety;
• Review the current effectiveness of the corrective actions developed in response to the issue; and
• Identify any new issues that may have arisen as a result of new risk controls associated with the issue.

Contrary, the purpose of an internal SMS auditing of issues is to:

• Review that analysis activities were thorough and accurate, such as ensure that the correct:
  • hazard was identified
  • root causes were identified
  • contributing factors were identified
• Ensure that the risk assessments are accurate and justified;
• Ensure that the corrective actions fixed the root causes of the issue and thoroughly mitigated the problem; and
• Identify any indicators of managers not performing due diligence.

As you can see, reviews and audits have very different purposes. Reviews largely (though not entirely) operate under the assumption that managers practice due diligence. Inspections and audits of issues look for signs of not performing due diligence.

Here are some things internal SMS audits of safety issues should review.

Audit Your Risk Analysis Activities

The first step in mitigating and managing risk is analysis. During risk analysis, your goals are to:

• Gather all relevant information;
• Establish flow of events;
• Organize the information with classifications (see how to classify issues in aviation SMS), such as by identifying:
  • primary hazard
  • root causes
  • risks
  • contributing factors
  • Human Factors
• Understand how likely the hazard/risks are to occur in the future – there’s no universal mandate for whether likelihood should apply to the chance of hazard occurrence or risk occurrence; and
• Understand how severe the most likely negative outcome will be in terms of:
  • Financial damage
  • Obstruction of current and/or future missions
  • Environmental damage
  • Damage to property or persons

In short, you will have a very solid understanding of:

• What happened;
• Why it happened; and
• What the main concerns are.

Faulty analysis will result in:

• Inaccurate risk assessments; and
• Poorly justified decision-making.

Audit Your Performed Risk Assessments

The next thing to look at when auditing an issue is:

• Are your risk assessments accurate; and
• Can they be justified by analysis activities?

Initial risk assessments should assess the issue based on existing controls. Closing assessments should be based on all newly implemented controls and/or corrective preventative actions.

Secondly, each risk assessment should be OBVIOUSLY justified for why you chose a severity and likelihood. We strongly recommend that you write 2-3 sentences justifying every risk assessment when you are managing issues. That way there is no question on why/how you did it 6 months later when you are being audited – you can avoid fumbling around, trying to remember a good reason for why you did something.

Related Aviation Risk Assessment Articles

• 4 Pillars | What Is Risk Assessment in Aviation SMS
• How to Justify Severity of Risk Assessments in Aviation SMS
• 5 Questions to Ask Before Making Risk Assessment

Review Corrective Actions for the Issue

During audits for managed issues, make sure you review the CPAs that were used to correct the issue.

• Do the CPAs address the root cause of the problem?
• Were the CPAs reviewed and signed off by the accountable manager?
• Were the CPAs reviewed at a later date to ensure relevancy and effectiveness?
• Is there adequate documentation to prove how the CPA was carried out?
• Was the CPA completed within a reasonable amount of time and/or on time?

You should, at the very least, be able to have clear answers to these questions. At best, you should be able to answer YES to all of these questions.

Review Amount of Time Taken to Manage Issue

One small thing to touch on is to make sure that for any audited issue you have documented;

• The report date of the issue; and
• The closing date of the issue.

This will show auditors that your issues are being completed in a timely manner. Without these pieces of metadata, your issue loses a lot of validity because it’s unclear whether your management's effort on the issue was real. In other words, consider the validity of:

1. A high-risk issue that was completely managed and closed within 3 days of reporting; VERSUS
2. A high-risk issue that was completely managed and closed 6 months after being reported.

Obviously, the second point shows clear incompetency, whereas the first point shows due diligence.

Last updated in June 2024.