How to Use Shortfall Analysis to Identify Aviation SMS Failures

What Is SMS Shortfall Analysis?

In every aviation safety management system (SMS), the accountable executive is responsible for ensuring the SMS is properly implemented and working across the entire organization. In order to fulfill this responsibility, accountable executives and their safety teams need tools to identify and address substandard safety performance.

Despite the fact that many risk management tools provide great insights into how and why safety incidents occur; there have been no dedicated tools for answering the following question:

• Where did the SMS risk management processes fail?

Related Aviation Risk Management Articles

• Difference between Reactive, Predictive and Proactive Risk Management in Aviation SMS
• What Is the Process of Risk Management in Aviation SMS
• 3 Main Components of Aviation Risk Management

Most Important Question in Aviation Risk Management

"Where the processes failed" is an important question to any safety-related situation – perhaps even the important question. SMS Shortfall Analysis is designed to clarify two different things. First, where did the SMS' risk management processes fail in terms of:

1. Risk control measures;
2. Human action; and
3. Safety policies and procedures?

Second, was the failure the result of:

• SMS bureaucracy; or
• Organizational safety culture?

The outcome of performing this analysis is a chart that visually identifies the extent and makeup of the safety-related failure, as well as a table that specifically identifies the different failures.

This risk management tool is called SMS Shortfall Analysis because it is a discovery tool for understanding the shortcomings (i.e., shortfalls) of your SMS' risk management processes.

Related Aviation Risk Management Articles

• 4 Elements of Safety Risk Management (SRM)
• How to Practice Reactive, Proactive, and Predictive Risk Management in Aviation SMS
• From Reactive to Proactive Risk Management in Aviation SMS

Goals of SMS Shortfall Analysis

When we are talking about the goals of SMS Shortfall Analysis, we are answering the question: what are the desired outcomes of performing this analysis? There are two types of goals for this:

• SMS Shortfall Analysis methodology; and
• Performing SMS Shortfall Analysis on safety incidents.

The goals of this risk management methodology are as follows:

• Easy to understand;
• Extremely quick to perform;
• Provide visual cue to general failures – i.e., an “exposure summary”;
• Identify specific enough failures upon which to make fact-based decisions; and
• Quantify the failure for data analysis.

The goals of performing SMS Shortfall Analysis on safety incidents are:

• Understand how the SMS' risk controls failed in the safety incident;
• Understand the types of failures in the safety incident;
• Establish how the situation deteriorated (flow of events); and
• Identify which corrective actions and preventive actions will be needed to fix exposure to mitigate risk to ALARP (as low as reasonably practical).

How SMS Shortfall Analysis Is Unique

SMS Shortfall Analysis is a totally unique risk management tool for several subtle but extremely important reasons. Other risk management tools are what we might call “niche tools,” as they are dedicated to specific aspects of SMS shortcomings:

• Root causes (fishbone);
• Behavioral analysis (decision trees);
• Human factor analysis (human action);
• Incident flow analysis (bowtie analysis); and
• Exposure summary (risk matrix).

All of these approaches provide valuable visual cues for their respective interests, and they paint a solid picture of “what happened.” However, what the above approaches either only suggest, partially address, or don’t address at all, is:

• Where did the SMS fail?

Related Aviation Risk Management Articles

• 4 Critical Elements of the Aviation SMS Risk Management Process
• Most Common Aviation Risk Management Challenges at Airlines & Airports
• 8 Stages of Safety Events in Aviation Risk Management Process

Aviation SMS Shortfall Analysis Provides Explicit Answers

SMS Shortfall Analysis is designed to address this question specifically and provide explicit answers. Moreover, while some analyses, like bowtie analysis, provide an in-depth look at many aspects of the safety event, they:

• Can be hard to understand;
• Can be very time-consuming to create and interpret; and
• Provide a broad picture of what went wrong, but don’t specifically point out the major SMS failures.

Shortfall analysis is unique in that it:

• Provides solid, valuable answers, that can be used to make fact-based decisions;
• Offers an exposure summary; but
• Is much faster and significantly easier to perform than all the above methods (except the ubiquitous risk matrix).

How to Use SMS Shortfall Analysis

SMS Shortfall Analysis is very easy to understand. The workflow is simple, and the steps are clear. For a full walk-through on:

• how to perform SMS Shortfall Analysis,
• examples of SMS Shortfall Analysis,
• situations to use SMS Shortfall Analysis, and
• blank SMS Shortfall Analysis templates-

-please refer to the following free eBook:

If you have any further questions, we are happy to answer them; we also appreciate praise and critical feedback, so contact us for further discussion.

Last updated November 2023.