SMS Pro Aviation Safety Software Blog 4 Airlines & Airports

The Real Way to Assess the Effectiveness of Your Aviation SMS

Posted by Tyler Britton on Apr 4, 2016 6:00:00 AM

Adaptability in Aviation SMS

The Real Way to Assess the Effectiveness of Your Aviation SMS

The basic fact is that truly judging the effectiveness of aviation safety management systems (SMS) is tough.

To merely judge the effectiveness of a program based on how well it meets compliance sounds prescriptive at best.

Judging a program based on a number of incidents and other such numeric data is terribly inadequate. It doesn’t take into account so many other factors that can reveal an SMS' health, such as:

  • How quickly the incident was responded to;
  • The quality of response, i.e. how well was the incident mitigated;
  • How well unexpected/non-routine incidents are handled;
  • Whether affected stakeholders were communicated with in a timely fashion regarding risk;
  • How many stakeholders actually read messages regarding identified hazards' risk;
  • Whether risk management processes were followed according to documented processes; and
  • Percentage of root cause analyses performed on reported safety issues?

Related Articles on Root Cause Analysis in Aviation SMS

Sure, numbers are important, as is being compliant with overseeing organizations, such as the FAA, Transport Canada, European Aviation Safety Association (EASA). However, there is that third intangible component that is, to myself and the safety managers I have spoken with, the most important.

It has been recently called things like:

  • High reliability
  • Resilience engineered
  • Performing

Each of these concepts explores that third intangible component from a different perspective. But one that they all have in common is that they stress the importance of a program or organization's adaptability. The basic idea behind being adaptable is that aviation SMS needs to completely absorb and change with

  • disturbances,
  • disruptions, and
  • surprises.

To be sure, an aviation SMS cannot focus on these things unless they have the aviation SMS database and software tools to easily manage compliance and safety data.

Let’s look at what adaptable means in greater detail.

Download Free Hazard and Risk Assessment Test

Compliance Markers Are Merely Guidelines

Every SMS has different needs, struggles, and therefore different goals. After all, an aviation SMS' goals and objectives must align with the organization's goals and objectives. Otherwise, the incongruence will merely result in frustration and mismatched expectations between the accountable executive, regulatory authority, and safety manager(s).

This is why compliance needs to be considered as a "guideline that needs to be followed," rather than a "rule that needs to be absolutely adhered to."

Aviation SMS programs must be adaptable to survive environmental changes

Of course, many aviation SMS face

  • Considerable resistance from the workforce;
  • Limited financial resources;
  • Roadblocks from management;
  • Mismatched organizational goals;
  • Inconsistent regulatory oversight; and
  • Unstable workforce.

And so compliance is the most realistic goal.

Overcoming such humps, which are also often symptomatic of younger SMS implementations, is challenging, and takes time and effort. And as said, getting to a point of compliance is an important point of becoming adaptable.

But all too often, SMS implementations with plenty of financial resources and workforce backbone do not move past simple “compliance.” They do not adjust regulations to any particular business needs of their organization. They simply do what they need to pass audits and look good on paper. Such is the goal of prescriptive programs.

The main indicator of adaptable programs are ones whose organization's analysis of performance goes beyond compliance. If anything, they may bump heads with SMS inspectors regarding certain areas of compliance if the needs of their organization are inadequately covered under compliant rules. The point is that adaptable organizations recognize the need for customized measures of SMS performance that are specific to the needs of their organization.

Thus, how an organization measures its safety performance is a reflection of how effective it aims to be.

Related Aviation SMS Implementation Articles

Safety Performance Monitoring Workflow for Aviation SMS

Information Flow

Another insightful way to measure the effectiveness of an aviation SMS is how information flows within a company. This a roundabout way of asking: how transparent is the aviation SMS.

When it comes down to it, safety transparency is above all a method of sharing (or not sharing) relevant safety information. Naturally, organizations that are very liberal with information and make a policy giving employees access to almost all safety information will be more transparent organization.

Transparent organizations are safe for several reasons:

  • Employees’ access to relevant operational knowledge makes them more aware of safety concerns
  • Trust at all levels of an organization is stronger
  • Front-line workers are more involved in their aviation SMS
  • Transparent safety cultures have higher safety reporting metrics

Of course, it’s extremely unclear to try and judge how “transparent” an organization is, which is why looking at information flow is much easier to accumulate data for and to analyze results. Questions like:

  • How much safety information is withheld from the general workforce?
  • Does safety information bottleneck around one/several individuals?
  • Do employees understand how information is processed when safety issues are submitted?
  • Can they track the process of submitting safety issues?
  • Are lessons learned published after safety issues have been treated?
  • Can employees access desensitized safety issues to perform their own investigations or self-learning?
  • Is a "Lessons Learned Library" available to all employees? Do they use it?

The questions go on, but the point is that the closer that management’s and employees' access to the same information is, the more transparent and therefore adaptable an organization is.

Related Articles on Transparency in Aviation SMS

Assertiveness of Front Line Employees

Ground crew at airport

This is a small but extremely important point. Being assertive means doing so both in action and vocally.

  • How vocal are employees during safety meetings?
  • Are they sharing their opinions about the aviation SMS, about specific policies and procedures?
  • Are they afraid to speak up because of punitive actions?
  • Are they afraid to deviate from prescribed policies/procedures if they feel that doing so is the most rational decision because they are afraid of being penalized?

In many ways, being assertive hinges on being able to make informed decisions, and thus having access to information (transparency). The importance of being assertive should be pretty clear – everything from

  • more involvement,
  • better safety culture,
  • generally better hazard awareness/identification activity, and
  • better risk mitigation.

That being said, being assertive has to be within reason of course. There is a fine line between assertive employees and impertinent or resistant employees, the latter of which is extremely harmful to safety cultures. When we talk about assertive employees, we are strictly talking about confidence in relation to actions/opinions in aviation safety.

Final Thought: Open to Risk

Developing an aviation SMS that can quickly and efficiently

  • address risk (resilient),
  • perform with consistently safe operations, and
  • identifies potential hazards early on (doesn’t drift into failure)

are all areas of creating an adaptable SMS.

An extremely important part of adaptable organizations is that they are constantly open-minded about sources of risk. Past successes are regarded skeptically and are not considered future guarantees. Rather, adaptable organizations rely more on the fidelity of front-line employees to:

  • Be aware
  • Be very vocal about possible risks
  • And quickly make informed, quality choices

Policies and procedures, and acceptable levels of compliance are further "catch-nets" to create adaptable aviation SMS.

How adaptable is your SMS? You will see a common thread in these aviation safety workflows...

Download Risk Management Procedural Workflows

Last updated October 2023.

Topics: 3-Safety Assurance

Site content provided by Northwest Data Solutions is meant for informational purposes only. Opinions presented here are not provided by any civil aviation authority or standards body.



Benefits of SMS Pro Database

Affordable, Basic Compliance for Small Aviation Service Providers

Best Practices for Aviation SMS


Watch SMS Pro Demo Videos

These two on-demand videos offer:

  • High-level overview of SMS Pro;
  • Hazard Reporting & Risk Management walk-through.
Watch SMS Pro Demo Videos

Subscribe to Email Updates

Recent Posts