Practicing vs Verifying SMS Continuous Improvement

It’s important to make the distinction between the

• practicing of continuous improvement and
• verifying that continuous improvement is happening.

Being ICAO compliant with continuous improvement is a systematic review to continuous improvement verification. Actually, practicing continuous improvement will happen in day-to-day operations.

To summarize this distinction:

• Practice continuous improvement: day-to-day operations with risk assessments, KPIs, risk management, risk control review, corrective actions, etc.
• Verify continuous improvement: internal audit of SMS.

You will use internal SMS audits to confirm that your organization is sufficiently practicing continuous improvement.

Periodic Internal Auditing of SMS

To be able to tell stakeholders and auditors that your aviation SMS is practicing continuous improvement, you need to periodically perform an internal SMS audit.

These SMS audits should be conducted:

• By an individual or audit team, depending on the size of your organization;
• At regular intervals, such as semi-annually;
• With all continuous improvement aspects of your SMS taken into account; and
• With all operational stakeholders in mind, including employees and contractors.

The act of performing an internal SMS audit demonstrates that you are dedicated to continuous improvement. That is, the act of verifying continuous improvement is a continuous improvement activity in and of itself.

Fulfilling this aspect of continuous improvement compliance involves being able to show a strong history of consistent internal SMS audit documentation.

Related Aviation SMS Continuous Improvement Articles

• The Truth about SMS Continuous Improvement and KPIs - with Free KPI Resources
• How to Demonstrate Continuous Improvement in Aviation SMS
• FAA Part 5 Compliance | Safety Assurance Continuous Improvement

Have an Internal Audit Plan?

One aspect of conducting periodic internal audits is having an auditing plan. This plan should include:

• Date scheduled for the audit;
• Who will perform the audit;
• Which auditing forms are used to conduct the audit;
• Which departments/divisions are included in the audit;
• Which parts of your SMS will be audited; and
• The frequency that audits will occur.

Your auditing plan does not have to be complex, but it should outline your audits in a way that an external auditor can review your plan and immediately understand the scope of your internal SMS audits.

Audit plans may be displayed on a calendar view, or simply listed in a table in your aviation SMS database. If you don't use an SMS database, an MS Excel spreadsheet will suffice. We have never seen a defined format for an audit plan in over a dozen years. However, we frequently are requested to provide guidance on audit plan preparation.

Audit Plan Samples Historical Issue Management

It’s absolutely critical that your SMS internal audits include a random sampling of historical issues managed. This sampling should review:

• Work that was done to correct this issue;
• Initial and most recent risk assessments; and
• Whether or not the underlying root cause remains at the last risk assessment.

What you should be looking for is that the current risk assessment matches the most recent one. What you don’t want to see is a current risk assessment that is more than your most recent assessment, as this would indicate either:

• Issue was not properly managed;
• Issue was not reviewed frequently enough;
• Documentation processes were either inadequate or neglected;
• Issue was improperly risk assessed.

Sampling issues that have undergone your organization's defined risk management processes are used to verify that:

• your safety department follows risk management procedures as defined in your safety manual; and
• managed issues maintain an acceptable level of safety.

Based on over a dozen years of experience, the most common audit finding we've seen is that aviation companies are not following their SMS manual's defined risk management procedures. If a process is described in your SMS manual, you had better follow what is written or change the procedure to more correctly align with your business processes.

Related Aviation SMS Manual Articles

• How to Create Your Aviation SMS Manual
• Tips Writing SMS Manuals with Aviation Safety Database Solutions
• 3 Best Practices for Your Aviation SMS Manual

Audit Plan Reviews Key Performance Indicators

Key performance indicators (KPIs) highlight the core performance of your safety objectives. Auditing your KPIs involves verifying that:

• KPIs are being monitored;
• KPI monitoring data is up to date;
• All KPIs are up to date; and
• All KPIs have targeted goals.

When KPIs are not current or are not being monitored thoroughly, it shows a lack of commitment towards continuous improvement and will reflect poorly on external audits. Automated KPI monitoring and notifications are a good way to ensure good audit performance. To automate KPI monitoring, you will need an aviation SMS database that is integrated with your aviation SMS':

• Hazard risk register;
• Risk management system;
• Hazard reporting system; and
• Auditing system.

KPIs are also known as SPIs (Safety Performance Indicators). We prefer the term KPIs because we work in both the quality and safety space. KPIs are a big topic in SMS and require considerable attention from the safety team.

Related Articles on Key Performance Indicators in Aviation SMS

• How to Automate Key Performance Indicator KPI Monitoring the Easy Way
• Choosing Key Performance Indicators in Aviation SMS
• What Are Best Practices for Key Performance Indicators in Aviation SMS

Auditing Includes Vendors/Contractors

Your SMS audits should include your contractors. This may mean having a separate auditing form dedicated to assessing the interactions contractors and suppliers have with your SMS. Things you might look for are:

• Are contractors being held accountable for safety behavior?
• How are contractors involved in the SMS?
• Are contractors contributing to safety in positive ways?
• Do contractors have an SMS (if required)?
• Is the SMS a real SMS? or a paper SMS?

The kinds of things you will audit in terms of vendors depend heavily on how many suppliers and/or contractors you have and how involved in your operations they are. But the point is, you need to audit and ensure that your contractors are not hindering the improvement of your SMS through their interactions in your operations.

Accountable Executive Is Aware of Audit Results

Finally, the last way to demonstrate that you are compliant with continuous improvement is to ensure that your account executive is aware of important audit findings. Your accountable executive is ultimately responsible for the SMS. Continuous improvement activities must remain within the purview of the accountable executive; therefore, he/she needs to be aware of:

• Where the SMS is improving;
• Deficiencies in the SMS (findings/concerns); and
• How those deficiencies will be corrected?
• When will all the shortcomings be fixed?

You need to have a demonstrable process documented, such as having all internal audits and corrective action plans signed off that they were reviewed by the accountable manager. To be safe, you may document the process steps for accountable executive audit review in your Safety Policy.

Final Thoughts on Continuous Improvement in Aviation SMS

Operators who are in the early stages of their SMS implementations do not need to be overly concerned by the "continuous improvement" requirement. Keep in mind, that your SMS progress will be evaluated based on:

• Maturity level of your SMS;
• Effectiveness of the SMS; and
• Complexity of your operations.

If you have been practicing the concepts of SMS for more than three years, you can expect some serious scrutiny through continual verification and follow-up audits by inspectors, whether they are:

• Regulatory agency SMS inspectors;
• Clients auditing your SMS; or
• Standards bodies/agencies auditing your operations, such as IOSA, ISAGO, IS-BAO, etc.

For a few years, you may "sneak by" auditors in demonstrating verifiable continuous improvement in your SMS; however, be prepared for the final reckoning when auditors believe you have a mature SMS. In short, you will need proof, i.e., documentation that your SMS is effective, and your SMS is improving. This documentation will be difficult to produce if all you have are paper forms, MS Word documents, and spreadsheets to show to the auditors.

Proactive safety managers prepare for the eventuality of demonstrating continuous improvement by ensuring the SMS is well documented and that the documentation is easily retrievable. Nothing frustrates an auditor more than being told to wait while the safety team "searches" for phantom documentation.

SMS documentation is best stored in a centralized SMS database. This is a best practice. If you are familiar with the European Union's SMS requirements, you will know that a database is required to store hazard reporting data. This is another indicator that an SMS database should be a requirement in your organization to document all your SMS activities.

Are you prepared to demonstrate continuous improvement of your SMS? If not, we can help.

Last updated January 2024.