It is a common misnomer that trend analysis in risk management is used to forecast future performance. This understanding of trend analysis is both reductive and distorts the purpose of trend analysis in aviation safety management systems (SMS).
According to the FAA, the objective of aviation SMS implementations is to:
There is an explicit expectation that aviation SMS enhances operators' ability to identify safety issues and spot trends before they result in a near-miss, incident, or accident. For this reason, the FAA requires carriers to develop and implement aviation SMS.
Source: Federal Register / Vol. 80, No. 5 / Thursday, January 8, 2015 / Rules and Regulations
In aviation safety management systems, the one, thematic question that we need to ask over and over is:
The real goals of trend analysis are to establish whether past performance is:
The assumption is that the trend will continue in the future (future forecast). The purpose is to know where organizational action needs to be taken in order to maintain ALoS.
Einstein supposedly said if he had an hour to solve a problem, he would spend the first 59 minutes understanding it. In the same way, a majority of trend analysis should be spent on the preparation of aggregated safety and operational data.
With trend analysis in risk management, the potential for establishing trends will only be as high as the quality of data at hand. To engage in predictive risk management activities, which trend analysis fits into, your organization needs sufficient data over a prolonged period of time to analyze. Substandard safety reporting cultures are disadvantaged in participating in predictive risk management due to the lack of consistent safety reporting data.
Besides having access to historical reported safety issues and their resulting investigation data, audit findings, and concerns are a second data source that springs to mind when considering safety data for trend analysis. For operators who manage audit findings in spreadsheets, there is hope, but spreadsheets are not the best solution to store predictive risk management data. Audit data stored in databases is more easily mined for trend analysis.
So where do you begin? Here are the basic elements needed for trend analysis.
Gathering data may seem like a step that is so obvious it’s not worth mentioning. However, data-gathering techniques will make or break your ability to prepare data for trend analysis in risk management operations. Your trend analysis can only ever be as good as the data you gather and have available. Garbage in, garbage out.
The ability to perform complex data analysis is one of the primary reasons organizations move to use aviation safety management software and/or an aviation safety database. Software and databases have vastly more potential than manual systems of data analysis. A database for data is just like a calculator for long division - it helps keep data
Manual data gathering simply takes significantly more time and is at risk for errors. So the questions you need to ask yourself right now are:
If your data gathering is mostly automated (i.e., aviation SMS software), then you already have a head start. If data-gathering processes are manual, how are you ensuring the efficacy of your data-gathering techniques? When you are manually entering reports, how much of the report is being input into a spreadsheet?
The purpose of gathering data for trend analysis is to have as much data and minutia available to analyze to facilitate fact-based decision-making processes. During data analysis, data is your currency. The more, the better.
Metadata is data about data. Why is metadata so important for trend analysis?
The best way to describe metadata is to illustrate using an example. You have all your data, ready for analysis. Where do you start? Start breaking your data up with metadata:
The list can go on indefinitely. You must use your imagination and it is best to create a list of questions you are seeking to understand. But breaking down data by the data’s particular points (metadata), such as locations, reporters, type of report, time frame, etc., is a fantastic way to begin establishing trends. Most likely, you will start seeing trends right away based on the metadata.
Metadata is extremely hard to manage without the assistance of an aviation safety database or software. Possible, but difficult. Manual management of metadata requires heavy data entry during safety reporting processes, such as manually entering in ALL minutia. Frankly, a much more cost-effective strategy for SMS data management (not to mention more reliable) is to simply use a low-cost, commercially available safety database or software for this.
Data taxonomy is the grouping or classifying of data. In aviation SMS implementations, data taxonomy translates to:
Data taxonomy is imported because it helps break your data into logical groups with which to establish trends. For example:
There are countless possibilities for using a taxonomy to prepare data for trend analysis. Much like metadata, taxonomic data is a great jumping-off point for trend analysis.
As you work through trend analysis based on metadata, taxonomy, etc., it will be important that you use similar time frames in order to:
For example, simply be clear as to:
This data will ideally be presented in a visual chart showing your time frame on the X-axis, and your data point(s) on the Y-axis. Using compiled trend analysis reports, safety professionals and upper management can draw inferences as to what the future has in store for them. Trending reports are the primary tool used in predictive risk management activities.
Time frames are very important elements to consider. Obviously, the longer the time frame for which you have data, the more reliable your predictive abilities become. Of course, having ten years' worth of safety data offers little value when the data is incomplete and riddled with inconsistencies.
Once you gather the analytical safety data, metadata, taxonomy, and timelines, you need to put your data together to begin trend analysis. Fortunately, the prep work is the most time-consuming part. Once you have all of your data points, it's simply a matter of plugging the data into your timeline.
As said, displaying trend data visually is definitely recommended. Using a line graph to display multiple potential trends is a useful way of comparing trends.
An important quality control check is to perform the "reasonability test." Do the resulting data (or reports) "make sense" and seem reasonable to you? For example, we have a 750% increase in bird strikes from one year to the next. Does this make sense? Unless you are familiar with both operational data and the underlying safety data, you cannot accurately make this determination. I can think of many scenarios where a 750% increase does make sense, such as a significant increase in operations or even a major safety reporting culture change.
The reasonability test must be answered by subject matter experts. If you are relying on your IT experts to make this determination, you are asking the wrong people. Business analysts and upper management should also take part in the "reasonability test." I highly recommend to all safety professionals that they put this question to management whenever safety reports are submitted for review. There may be operational areas that may skew the data that the safety team is not privy to.
An SMS database makes short work out of gathering safety data and automatically generating trend reports. If your management team has expectations of participating in predictive risk management activities, it is highly recommended that your SMS data management strategy includes adopting an SMS database as early in the SMS implementation process as possible. The longer you wait, the less value you will receive from the SMS database due to the reduced amount of available safety data.
There are a few low-cost, commercially available SMS databases that efficiently collect, store and organize SMS documentation requirements. Small companies with more than 40 employees will benefit from an SMS database. If your company is smaller than 40 employees with high employee turnover, then an SMS database is also recommended.
If you are ready to take your SMS to the next level, consider using SMS Pro. Since 2007, SMS Pro has been providing SMS databases to operators worldwide. Check out these short demo videos to learn how your company can benefit.
Similar to trend analysis, Shortfall Analysis is a new technique specifically designed to analyze SMS failures in safety incidents. Here is our free 16-page eBook covering what Shortfall Analysis is, and how to use it:
Last updated May 2024.