Why You Need to Monitor the Effectiveness of Control Measures

Control measures are the heart of your aviation safety management system’s (SMS) risk mitigation efforts. You may call your "control measures" either:

• Risk controls; or
• Simply "controls."

In this article, we'll use the term "control measures," but I am sure many of you call them "risk controls." Either term is permissible; however, I'm detecting a global preference in the aviation industry for the term "risk control." One can readily assume that these "control measures" are items (tangible or abstract) in your organization that prevent hazards from escalating beyond acceptable parameters.

Related Articles on Control Measures in Aviation SMS

• Difference Between Hazards, Risks & Control Measures in Aviation SMS
• How to Implement Effective Control Measures
• How to Evaluate and Justify a Risk Control in Aviation SMS

How Does Your Aviation SMS Monitor Risk Controls?

Oversight agencies like the FAA understand that effective control measures help operators either:

• detect that an undesirable state is imminent, which leads to "the accident;"
• avoid/prevent "the accident" that always seems inevitable to the industry as a whole; or
• lessens the damage's impact should "the accident" occur.

This is why regulators and SMS auditors are always asking, “How do you monitor the effectiveness of risk controls or control measures?”

Aviation service providers need to have quick answers to this question.

In response to pressure from oversight agencies, different companies will have different ways of monitoring control measures. What matters is that you can demonstrate that your monitoring methods work.

But first, operators need to actually develop, implement, and document control measures within the scope of their aviation SMS' documented systems. This happens after

• Identifying hazards;
• Analyzing risk;
• Assessing risk; and
• Reviewing and documenting existing control measures.

Control measures need to actively exist in the operational environment, and not simply exist “on paper.” This seems like a rather obvious point, but you might be surprised how often this issue arises. Are any of the control measures in your SMS only on paper?

There is a chance that you don't have any documented control measures in your aviation SMS documentation. Many operators don't have the data management capabilities to easily document, track, and regularly review their control measures. If this sounds familiar, don't be alarmed. There are a few low-cost, commercially available SMS databases that perform all of these functions within a single system.

To be perfectly blunt, managing control measures in a spreadsheet is like trying to travel by bike across the United States. Yes, it can be done, but very few have the time and energy to do it well. An SMS database is designed to handle this complex, recurring task.

Related Aviation SMS Database Articles

• What Is an Aviation Safety Database for SMS Implementations?
• How to Choose the Best Aviation Safety Database Software
• 7 Signs You Need an Aviation Safety Management (SMS) Database

Types of Monitoring

Assuming that you have controls active in the operational environment, we often see three different types of monitoring activities. We recommend adopting each as a part of your aviation SMS oversight strategy:

• Periodic performance monitoring: evaluating control measures in a formal review process on an annual or semiannual basis, such as with auditing;
• Scheduled monitoring: monitoring control measures on a regular basis, such as reviewing hazard trends and identifying problem control measures once per month; and
• Ongoing monitoring: monitoring affected control measures on a daily basis, such as when hazard reports are submitted.

Demonstrating to oversight agencies that your SMS implementation provides you with the ability to monitor these three ways is impressive. Demonstrating that you actually follow through with this monitoring will earn you best-in-class SMS status.

Here are these three ways in more detail how to monitor the effectiveness of control measures in aviation SMS implementations.

Periodically Monitoring the Performance of Control Measures

Periodically monitoring the effectiveness of control measures involves an infrequent but thorough assessment of control measures. Periodic monitoring activities usually involve:

• Formal review of control measures;
• Stress testing the SMS to expose inadequate control measures; and
• Auditing control measures.

Now, aviation safety programs can have literally hundreds of control measures. It’s not feasible to evaluate each one. In a periodic review. Safety management needs to efficiently monitor the effectiveness of control measures. To do this, safety management should do the following before they begin their periodic review:

• Review safety data charts for trends;
• Review hazard register to see which hazards are continually arising in safety issues; and
• Review classifications during issue management to evaluate which classifications are not being mitigated.

The above steps should significantly reduce the scope of review to only the most potentially inadequate control measures by allowing you to hone in on specific hazards and classifications. Moreover, the steps above indirectly “sign off” on the control measures that are working properly.

Scheduled Monitoring of the Effectiveness of Control Measures

Scheduled monitoring happens more frequently than periodic monitoring, such as during hazard analysis and review. Not all safety issues require an in-depth hazard analysis. Generally, such operations are reserved for mid to high-risk issues.

When such issues are reported, management needs to undertake hazard analysis activities, such as:

• Fishbone diagram root cause analysis;
• Bowtie analysis; or
• Decision trees.

These operations naturally incorporate risk control review into the analysis process, and should quickly point out inadequacies in the risk control, as well as identify which risk controls are meeting needs.

Situations for scheduled monitoring are:

• Responsible manager's mandatory review of hazards;
• Safety cases;
• Management of change;
• Risk analysis and investigations; and
• Risk scenario analysis.

Taking the above points into account, scheduled monitoring of control measure effectiveness should happen at least once per month.

Ongoing Assessment of Control Measure Effectiveness

Ongoing assessments of control measure effectiveness happen almost on a daily basis. This method is used to monitor the effectiveness of control measures through common interactions with the safety management system:

• When issues are reported;
• When corrective preventative actions (CPAs) are created;
• When CPAs are reviewed;
• When issues are validated (reviewed); and
• Other issue management activities.

Ongoing monitoring should be a natural product of issue management. When issues are reported, safety management is tasked with identifying:

• Why the issue was mitigated;
• (if applicable) How the issue could have been further mitigated;
• What controls worked/did not work; and
• If further controls are needed.

In short, issue management forces safety management to look directly at pertinent safety controls to evaluate whether or not they worked as desired. This is a natural way to monitor control measures whenever safety issues are reported.

Have You Also Read

• How to Review Your Aviation Safety Risk Management Procedures
• How to Review Safety Issues in SMS
• Time to Review Your Aviation SMS Data Management Strategy

Final Thought: Aviation Safety Database Is Critical

Having an aviation safety database is indispensable in monitoring the effectiveness of proactive and reactive control measures. Aviation safety databases:

• Help automate much of the work that can cause errors and oversight mistakes in monitoring activities;
• Allow significantly more complex data trending; and
• Provide the kind of extremely reliable data that safety management needs to make good safety decisions.

Not that control measure monitoring can’t happen without an aviation safety database, but it is significantly harder. Just ask SMS auditors which kinds of programs perform better on audit: those with safety databases, or those without.

Monitoring activities can be intimidating. See how SMS Pro uses SRM and SA processes to monitor control measures in these 3 short videos:

Last updated in January 2024.