Why You Need to Monitor the Effectiveness of Control Measures
Control measures are the heart of your safety management system’s risk mitigation efforts.
- Aviation service providers need to have quick answers to this question.
In response to pressure from oversight agencies, different companies will have different ways of monitoring control measures. What matters is that you can demonstrate that your monitoring methods work.
But first, safety programs need to actually develop and implement control measures. This happens after:
- Identifying hazards;
- Assessing risk; and
- Reviewing and documenting existing control measures.
Control measures need to actively exist in the operational environment, and not simply exist “on paper.” This seems like a rather obvious point, but you might be surprised how often this issue arises. Are any of the control measures in your safety management system only on paper?
Have you also read...
- Difference between Hazards, Risks & Control Measures in Aviation SMS
- How to Implement Effective Control Measures
- How to Evaluate and Justify a Risk Control in Aviation SMS
Types of Monitoring
Assuming that you have controls active in the operational environment, we often see three different types of monitoring activities. We recommend adopting each as a part of your SMS oversight strategy:
- Periodic performance monitoring: evaluating control measures in a formal review process on an annual or semiannual basis, such as with auditing;
- Scheduled monitoring: monitoring control measures on a regular basis, such as reviewing hazard trends and identifying problem control measures once per month; and
- Ongoing monitoring: monitoring control measures on a daily basis, such as when hazard reports are submitted.
Demonstrating to oversight agencies that your SMS program provides you with the ability monitor these three ways is impressive. Demonstrating that you actually follow through with this monitoring will earn you best-in-class SMS status.
Here are these three ways in more detail for how to monitor the effectiveness of control measures in aviation SMS programs.
Periodically Monitoring the Performance of Control Measures
Periodically monitoring the effectiveness of control measures involves infrequent but thorough assessment of control measures. Periodic monitoring activities usually involve:
- Formal review of control measures;
- Stress testing the SMS to expose inadequate control measures; and
- Auditing control measures.
Now, aviation safety programs can have literally hundreds of control measures. It’s not feasible to evaluate each one. In a periodic review. Safety management needs to efficiently monitor the effectiveness of control measures. To do this, safety management should do the following before they begin their periodic review:
- Review safety data charts for trends;
- Review hazard register to see which hazards are continually arising in safety issues; and
- Review classifications during issue management to evaluate which classifications are not being mitigated.
The above steps should significantly reduce the scope of review to only the most potentially-inadequate control measures by allowing you to hone in on specific hazards and classifications. Moreover, the steps above indirectly “sign off” on the control measures that are working properly
Scheduled Monitoring of the Effectiveness of Control Measures
Scheduled monitoring happens more frequently than periodic monitoring, such as during hazard analysis and review. Not all safety issues require an in depth hazard analysis. Generally, such operations are reserved for mid to high risk issues.
When such issues are reported, management needs to undertake hazard analysis activities, such as:
These operations naturally incorporate risk control review into the analysis process, and should quickly point out inadequacies in the risk control, as well as identify which risk controls are meeting needs.
Situations for scheduled monitoring are:
- Responsible manager's mandatory review of hazards;
- Safety cases;
- Management of change;
- Risk analysis and investigations; and
- Risk scenario analysis.
Taking the above points into account, scheduled monitoring of control measure effectiveness should happen at least once per month.
Ongoing Assessment of Control Measure Effectiveness
Ongoing assessments of control measure effectiveness happen almost on a daily basis. This method is used to monitor the effectiveness of control measures through common interactions with the safety management system:
- When issues are reported;
- When corrective preventative actions (CPAs) are created;
- When CPAs are reviewed;
- When issues are validated (reviewed); and
- Other issue management activities.
Have you also read....
Ongoing monitoring should be a natural product of issue management. When issues are reported, safety management is tasked with identifying:
- Why the issue was mitigated;
- (if applicable) How the issue could have been further mitigated;
- What controls worked/did not work; and
- If further controls are needed.
In short, issue management forces safety management to look directly at pertinent safety controls to evaluate whether or not they worked as desired. This is a natural way to monitor control measures whenever safety issues are reported.
Final Thought: Aviation Safety Database Is Critical
Having an aviation safety database is indispensable in monitoring the effectiveness of proactive and reactive control measures. Aviation safety databases:
- Help automate much of the work that can cause errors and oversight mistakes in monitoring activities;
- Allow significantly more complex data trending; and
- Provide the kind of extremely reliable data that safety management needs to make good safety decisions.
Not that control measure monitoring can’t happen without an aviation safety database, but it is significantly harder. Just ask SMS auditors which kinds of programs perform better on audit: those with safety databases, or those without.
Monitoring activities can be intimidating. See how SMS Pro uses SRM and SA processes to monitor control meausures in these 3 videos:
Published February 2017. Last updated May 2019.