LOSA Observer Training/Calibration: Why It’s Important

Posted by Dr. Bob Baron on Jan 11, 2023 6:00:00 AM

Problems With Invalid LOSA Observations

LOSA Observer Training/Calibration: Why It’s Important

Many airlines that conduct LOSAs internally have been confronted with a problem that has significantly affected the validity of their observations.

The problem has to do with observer calibration.

What Is LOSA Observer Calibration

Observer calibration (which may be referred to more formally as inter-rater reliability) is the process of ensuring that each LOSA observer is coding threats, errors, and UASs (Undesired Aircraft States) in the same way. For instance, if three out of eight observers are recording an inoperative lavatory as a UAS then there would be a serious calibration issue between those observers. That will undoubtedly cause a lot of extra work in the data cleaning and verification process. And in some cases, there may be a lot of data that unfortunately wind up being completely unusable.

Maintaining Consistent LOSA Observation Data

To preclude this from happening, LOSA observers need to be thoroughly calibrated and reliability testing needs to be done before any actual observations are made (this can be done using paper scenarios in the classroom).

Observer calibration will help to ensure that each observer is seeing, processing, and recording the same thing (i.e., on the same sheet of paper)—to the extent possible. Although it would be great to have perfect reliability among all observers, we know for a fact there will always be some variation. This is normal. However, if there is significant variability then there is a good chance that your LOSA results will not be very truthful and those results will be met with much management skepticism.

Learn how LOSA can benefit your company. Contact us today for a free consultation or LOSA training.