Annotated Bibliography on Aviation Maintenance

Seeking immediate assistance with a quick turn around. Must be professional and plagiarism free.

writingaviation maintenance

Objective: To better understand the external fac-
tors that influence the performance and decisions of
aviators involved in Naval aviation mishaps.

Background: Mishaps in complex activities, ranging
from aviation to nuclear power operations, are often
the result of interactions between multiple components
within an organization. The Naval aviation mishap data-
base contains relevant information, both in quantitative
statistics and qualitative reports, that permits analysis
of such interactions to identify how the working atmo-
sphere influences aviator performance and judgment.

Method: Results from 95 severe Naval aviation
mishaps that occurred from 2011 through 2016 were
analyzed using Bayes’ theorem probability formula.
Then a content analysis was performed on a subset of
relevant mishap reports.

Results: Out of the 14 latent factors analyzed,
the Bayes’ application identified 6 that impacted spe-
cific aspects of aviator behavior during mishaps. Tech-
nological environment, misperceptions, and mental
awareness impacted basic aviation skills. The remain-
ing 3 factors were used to inform a content analysis
of the contextual information within mishap reports.
Teamwork failures were the result of plan continuation
aggravated by diffused responsibility. Resource limita-
tions and risk management deficiencies impacted judg-
ments made by squadron commanders.

Conclusion: The application of Bayes’ theorem to
historical mishap data revealed the role of latent fac-
tors within Naval aviation mishaps. Teamwork failures
were seen to be considerably damaging to both aviator
skill and judgment.

Application: Both the methods and findings have
direct application for organizations interested in under-
standing the relationships between external factors and
human error. It presents real-world evidence to pro-
mote effective safety decisions.

Keywords: human error, safety, aviation, HFACS

IntroductIon
Human factors researchers have advocated

for decades that “human error” in accidents and
mishaps is actually the result of difficult work-
ing conditions shaped by external influences
(Dekker, 2014; Fitts & Jones, 1947; Norman,
1988; Rasmussen, 1983; Reason, 1990; Woods
& Cook, 1999). The partial nuclear meltdown at
Three Mile Island in 1979, for instance, is an apt
example that demonstrates operator error on the
day of the accident was the result of interactions
between poorly designed displays, inadequate
training for the operators, and mechanical fail-
ures (Meshkati, 1991). To better monitor these
types of interactions and the external influences
large organizations can exert on workers, the
Department of Defense (DoD) instructed the
safety communities within the services to estab-
lish procedures to help observe and categorize
“human error” data. The DoD Human Factors
Analysis and Classification System (HFACS)
became the standardized taxonomy in 2005.

Developed by Doug Wiegmann and Scott
Shappell (2003), DoD HFACS is inspired by
Reason’s (1990) “Swiss cheese” model of acci-
dent causation and attempts to identify how spe-
cific error tendencies (classified as active fail-
ures, known as unsafe acts) are shaped by
higher-level influences (classified as latent fail-
ures, known as preconditions, unsafe supervi-
sion, and organizational influences) (see Figure
1). As an aviation example, a hard landing may
be the result of an aviator not following proce-
dures (unsafe act). But that occurred because
critical information was not communicated (pre-
condition), and it was influenced by inadequate
risk assessment (unsafe supervision) and organi-
zational culture (organizational influences).
DoD HFACS aids safety professionals, with or
without formal human factors training, to inves-
tigate beyond the person (e.g., the aviator or
maintenance personnel) that happens to be clos-
est in time and space to the scene of the mishap.
Results of previous research have suggested that
DoD HFACS is an effective tool toward reducing

771904HFSXXX10.1177/0018720818771904Human FactorsUnderstanding Human Errorresearch-article2018

Address correspondence to Andrew T. Miranda, Naval
Safety Center, 375 A St., Norfolk, VA 23511-4399, USA;
e-mail: andrew.miranda@navy.mil.

Author(s) Note: The author(s) of this article are U.S.
government employees and created the article within the
scope of their employment. As a work of the U.S. federal
government, the content of the article is in the public domain.

Understanding Human Error in Naval Aviation Mishaps

Andrew T. Miranda, Naval Safety Center, Norfolk, Virginia

HUMAN FACTORS
Vol. 60, No. 6, September 2018, pp. 763 –777
DOI: 10.1177/0018720818771904
Copyright © 2018, Human Factors and Ergonomics Society.

mailto:andrew.miranda@navy.mil

https://doi.org/10.1177/0018720818771904

http://crossmark.crossref.org/dialog/?doi=10.1177%2F0018720818771904&domain=pdf&date_stamp=2018-04-26

764 September 2018 – Human Factors

mishap rates (e.g., Belland, Olsen, & Lawry,
2010).

The primary purpose of HFACS is to be a tool
used by safety professionals to help identify
unsafe practices wherever they may occur within
an organization. At the active failure level, there
are two categories of unsafe acts: errors and vio-
lations. Errors are defined as unintentional devi-
ations from correct action, whereas violations
are defined as deliberate deviations from rules or
instructions. Errors are further delineated into
two subcategories: (1) performance-based errors
(PBE) and (2) judgment and decision-making
errors (JDME). The motivation to distinguish
between the two types of errors is inspired by the
seminal work from human factors scholars Jens
Rasmussen (1983) and James Reason (1990),
who both studied the underlying cognitive
mechanisms that contribute to how errors can
manifest depending on the task and person.
Wiegmann and Shappell (2001, 2003) describe
PBE (originally defined as skill-based errors) as

occurring when there is a breakdown of the basic
skills that are performed without significant
conscious thought. For instance, an aviator may
forget to perform a highly practiced task, such as
lowering the landing gear on approach, because
of a warning light distraction. JDME, on the
other hand, are considered “honest mistakes.”
They are outcomes of intentional behaviors and
choices that turn out to be inadequate for the
situation (Wiegmann & Shappell, 2001, 2003).
For example, an aviator may choose to fly into a
seemingly mild storm, but they underestimated
the storm severity, leading to an unsafe situation.

Within the HFACS framework as well as the
greater human factors community, both error
types are viewed as symptoms of deeper trouble
within an organization (Dekker, 2014; Wieg-
mann & Shappell, 2001). That is, the term
human error is often considered an unhelpful
and reductive label used to identify the solitary
person(s) as the weak component with a complex
system encompassing numerous people, tools,

Figure 1. The Department of Defense Human Factors and Analysis Classification System (DoD
HFACS 7.0).

Understanding HUman error 765

tasks, policies, and procedures (Dekker, 2014).
That is why the latent conditions, the higher-
level factors that ultimately shape the work for
the individuals who are typically directly
involved with the mishap, must be examined
when assessing safe practices. DoD HFACS dis-
tinguishes latent conditions into three tiers: pre-
conditions (subdivided into seven categories),
unsafe supervision (subdivided into three cate-
gories), and organizational influence (subdi-
vided into four categories) (see Figure 1).

HFACS has been adopted within a variety of
domains, including the mining industry (Patter-
son & Shappell, 2010) and aviation mainte-
nance (Krulak, 2004). The framework is typi-
cally modified to better accommodate the type
of work being performed. Theophilus et al.
(2017), for instance, developed a version of
HFACS for the oil and gas industry that fea-
tured additional categories covering industry
regulatory standards. The version used by the
DoD has evolved to the current version of DoD
HFACS 7.0 that includes an additional layer of
specificity, known as nanocodes, within each
category. For example, the teamwork category
within the precondition tier includes the nano-
codes critical information not communicated
and task/mission planning/briefing inadequate.
A more thorough description of DoD HFACS
can be found on the Naval Safety Center Web
site (DoD HFACS, 2017).

Numerous studies have analyzed safety data
obtained from HFACS application of safety per-
formance. The most informative methods of
HFACS analysis are those that go beyond a
descriptive understanding of how often particu-
lar failures or errors appear and instead attempt
to learn the linkages between latent failures and
active failures. For example, Li and Harris
(2006) analyzed HFACS data from Republic of
China Air Force mishaps from 1978 to 2002.
Their findings revealed a variety of relationships
between active and latent failures, including
physical/mental limitations as a precondition
lead to higher likelihoods of both judgment and
skill-based errors. Hsiao, Drury, Wu, and Paquet
(2013a, 2013b) created a modified version of
HFACS tailored to aviation maintenance opera-
tions. The researchers incorporated HFACS data
obtained from historical safety audit reports

(instead of past safety performance), along with
known mishap rates, into an artificial neural net-
work that predicted monthly safety performance
with moderate statistical validity (Hsiao et al.,
2013b). Chen and Huang (2014) also developed
a modified HFACS framework for aviation
maintenance and incorporated their data into a
Bayesian network that revealed how various
latent factors influenced maintenance perfor-
mance (e.g., how the physical posture of the
worker likely influenced their ability to inspect
aircraft parts). These studies have demonstrated
the potential for historical HFACS data to help
understand how latent factors can shape
performance.

Similar work has been conducted on DoD
HFACS data obtained from military aviation
accidents. Tvaryanas and Thompson (2008), for
example, performed an exploratory factor analy-
sis on DoD HFACS data alongside data from
mechanical failures obtained from 95 safety
events, including mishaps and near misses, of
U.S. Air Force unmanned aircraft systems. The
results revealed most of the events in their data
set were the result of deficiencies within organi-
zational process and the technological environ-
ment. The authors therefore suggested a need to
incorporate HFACS into a broader human-
systems integration framework and human error
data be considered early in the organizational
process of acquiring new technologies (Tvarya-
nas & Thompson, 2008). Walker, O’Connor,
Phillips, Hahn, and Dalitsch (2011) performed
similar analysis on DoD HFACS data obtained
from 487 Navy and Marine Corps (henceforth
referred to as Naval) aviation mishaps. Using
what they called “lifted probabilities,” they
observed certain facilitatory/inhibitory relation-
ships between specific DoD HFACS nanocodes
(e.g., mental fatigue contributed to an aviator
ignoring a caution or warning). The findings of
the previous studies have demonstrated value in
analyzing the relationships between active and
latent failures within the DoD HFACS frame-
work.

The previous HFACS research mentioned to
this point has been projects analyzing a collec-
tion of existing HFACS data points. Other
research has examined the more practical use of
HFACS as a tool used by safety professionals.

766 September 2018 – Human Factors

For example, in a study simulating Naval avia-
tion mishap investigations, O’Connor and Walker
(2011) found that different investigation teams
may disagree about what DoD HFACS nano-
codes should be assigned as mishap causal fac-
tors, suggesting DoD HFACS has poor interrater
reliability. The implications raise concerns on the
effective use of DoD HFACS at the specific level
of the nanocodes. Cohen, Wiegmann, and Shap-
pell (2015) attempted to address this concern by
reviewing a collection of studies assessing the
rater reliability of HFACS. They concluded that
though DoD HFACS has questionable reliability
at the specific nanocode level, it does provide
adequate agreement and reliability at the broader
category level. With this conclusion in mind,
there remains a gap in the literature of examining
the relationships between active and latent fail-
ures within DoD HFACS from severe mishaps at
the category level.

We wanted to explore this gap while also pre-
senting a new method to analyze DoD HFACS
data. The method we chose to implement was
Bayes’ theorem. This method of conditional
probability analysis when applied to DoD
HFACS data allows us to introduce prior occur-
rences of both active and latent failures to more
accurately observe how the presence of certain
latent failures are related to certain unsafe acts.
Bayes’ theorem allows us to answer questions of
conditional probabilities while minimizing
potential for over- or underestimating miscalcu-
lations (Ramoni & Sebastiani, 2007). With this
method, we could address the following two
research questions:

Research Question 1: What latent failures
impact PBE and JDME separately?

Research Question 2: What latent failures
impact both PBE and JDME?

Bayes’ theorem method
We sought to use Bayes’ theorem to identify

conditional probabilistic relationships between
active and latent failures within a DoD HFACS
data set. The data set was coded from 95 Class
A Naval aviation mishaps from 2011 through
2016. Naval Class A mishaps are defined as
a mishap in which any fatality or permanent
total disability occurs, the total cost of damage

is $2,000,000 or greater, and/or any aircraft is
destroyed (DoD, 2011). We chose to focus our
analysis on Class A mishaps because unlike less
severe mishaps, Class As garner further investi-
gative scrutiny in which the investigation team
includes individuals from various departments
(i.e., safety, operations, maintenance, and medi-
cal) and likely aid from professional aviation
mishap investigators (Department of the Navy,
2014). Furthermore, previous HFACS research
has shown that HFACS error types can change
depending on severity level (Wiegmann & Shap-
pell, 1997). Specifically, JDME are more likely
to be associated with severe mishaps, whereas
PBE are associated with less severe. By examin-
ing a single severity level, we avoided unwanted
influence from that confounding variable. Lastly,
we chose to focus our analysis on unsafe acts
that were determined to be unintentional (i.e.,
JDME and PBE) rather than including violations
that are considered deliberate misconduct.

Bayes’ theorem and hFacs
Statistically speaking, the strength of Bayes’

theorem is that it allows us to make accurate
inferences about the probability of events based
on the prior knowledge of how often the condi-
tions that surround that event occur (Ramoni &
Sebastiani, 2007). This statistical operation is a
convenient formula that lends itself toward bet-
ter understanding of relationships among vari-
ous conditional probabilities (Winkler, 2003).
Therefore, it allows us to acquire deeper mean-
ing from the categories within DoD HFACS
and move beyond the unhelpful label of human
error. When applied to HFACS, the formula
yields an accurate conditional probability that
considers prior occurrences of both active and
latent failures:

P Active Latent
P Active P Latent Active

P Latent
( |

( |
)

)
.=

( )×
( )

The modified Bayes’ theorem formula pro-
vides the answer to the conditional probability
question that reads as, “Given the presence of a
latent failure, what is the probability of an active
failure?” For example, in our data set, 63 out of
95 mishaps cited a JDME, giving the P(JDME) =

Understanding HUman error 767

0.66. Among the 63 JDME mishaps, 30 cited
mental awareness as a latent failure, giving
P(Mental Awareness | JDME) = 0.48. Mental
awareness as a latent failure was cited in 62 out
of the 95 total mishaps, giving P(Mental Aware-
ness) = 0.65. These three variables plugged into
Bayes’ theorem yields 0.48. Now we can say,
given a mental awareness failure, the probability
of a JDME is 48%. With this product, we can
now observe both the magnitude of how much a
latent failure impacts the probability of certain
error types as well as similarities and differences
between the impact of latent failures on active
failures.

Bayes’ theorem results
Before applying Bayes’ theorem to the DoD

HFACS data set, we first determined the prior
probabilities of all three necessary variables.

Table 1 presents the frequency of all 17 failure
categories across all four DoD HFACS tiers.
PBE were the most common type of unsafe
act cited within the mishap data set. Mental
awareness failures were the most common
precondition, being cited in 62 mishaps. Inade-
quate supervision was the most common unsafe
supervision failure, and policy and process
issues was most common among organizational
influences.

Once we obtained the prior probabilities pre-
sented in Table 1 and the necessary prior condi-
tional probabilities, we applied Bayes’ theorem
probability formula to the DoD HFACS data
set. The results are split into three groups: latent
failures with greater impact on JDME, latent
failures with greater impact on PBE, and latent
failures with equal substantial impact on both
types of errors (see Table 2).

TAblE 1: Frequency of Active (Unsafe Acts) and Latent (Precondition, Unsafe Supervision, and
Organizational Influences) Failures Across 95 Class A Mishaps

Failure Type Frequency Probability (%)

Unsafe act
Performance-based error 74 77.89
Judgment/decision-making error 63 66.32
Violations 19 20.00
Precondition
Mental awareness 62 65.26
Teamwork 59 62.11
State of mind 55 57.89
Sensory misperception 26 27.37
Technological environment 17 17.89
Physical environment 15 15.79
Physical problem 10 10.53
Unsafe supervision
Inadequate supervision 44 46.32
Supervisory violations 20 21.05
Planned inappropriate operations 17 17.89
Organizational influences
Policy and process issues 46 48.42
Climate/culture influences 11 11.58
Resource problems 9 9.47
Personnel selection and staffing 5 5.26

Note. The sum of total frequencies within and between Department of Defense Human Factors and Analysis
Classification System tiers exceeds 95 because some mishaps report multiple failures.

768 September 2018 – Human Factors

Five total latent failures produced different
levels of impact on the two types of errors. The
two latent failures that provided the greater
impact on JDME were planned inappropriate
operations and climate/cultural influences. The
three latent failures that provided greater impact
on PBE were sensory misperception, techno-
logical environment, and mental awareness.
Teamwork was the only latent failure that

provided equal substantial impact on both types
of errors (i.e., greater than 50% probability for
both error types).

To better interpret the level of impact that
latent failures have on certain error types, we
graphed the difference between the two types of
errors across the five latent failures (see
Figure 2). This allowed us to add statistical error
bars that help answer questions of plausibility of

TAblE 2: Bayes’ Theorem Probabilities Across Both Error Types

JDME Probability (%) PBE Probability (%)

Greater JDME impact
Planned inappropriate operations 82 18
Climate/cultural influences 64 27
Greater PBE impact
Sensory misperception 27 73
Technological environment 29 71
Mental awareness 48 73
Equal impact
Teamwork 66 68

Note. The latent failures not included in this table were withheld because there was equally inconsequential impact
on both error types. JDME = judgment and decision-making errors; PBE = performance-based errors.

Figure 2. Differences in Bayes’ probability between JDME and PBE across five
latent failures. Each column in the graph represents the difference between the values
presented in the columns of Table 2. The addition of the 95% confidence intervals error
bars allows us to confirm these latent failures likely impact only one type of error.
JDME = judgment and decision-making errors; PBE = performance-based errors.

Understanding HUman error 769

the analysis (i.e., if the error bars overlapped
with zero, it would not be likely that the latent
failure impacts only one type of error).

Bayes’ theorem dIscussIon
There are two valuable findings from the

results of the present study. First, the application
of Bayes’ theorem to a historical DoD HFACS
data set demonstrated an efficient solution
toward understanding relationships between
latent and active failures. To our knowledge, this
was the first application of the Bayes’ theorem
probability formula to any HFACS data set that
identifies how specific error types are impacted
by latent failures. These results confirm that
these relationships must be examined when
using HFACS with the intent to improve safety.

The second valuable finding is the results of
the Bayes’ theorem application itself. Five latent
failures were observed to have substantial
impact on specific error types, and one latent
failure substantially impacted both. Sensory
misperception, mental awareness, and techno-
logical environment were factors that specifi-
cally impacted PBE. All three of these latent
failures are in the preconditions tier, which
Wiegmann and Shappell (2001, 2003) define as
the conditions within and surrounding the indi-
vidual that lead to unsafe acts. Sensory misper-
ception and mental awareness both consider per-
ception, attention, and cognitive factors of the
aviator (e.g., visual illusions, spatial disorienta-
tion, and fixation). This finding is supported by
previous research showing misperceptions and
miscomprehensions can lead to degradation of
situation awareness (Endsley, 1995). While on
its own this conclusion does not give us a deeper
understanding of “human error,” it is likely
related to the third factor within the precondition
tier also specifically impacting PBE: technologi-
cal environment.

Within DoD HFACS, the technological envi-
ronment identifies the overall design of the
workspace. In aviation, for example, the design
of the displays and controls in the cockpit and
the personal equipment in use are included in
this latent factor. Though the technological envi-
ronment was only cited in 17 out of 95 mishaps,
the current findings demonstrate the profound
impact it plays in disrupting basic aviator skill

and performance during severe mishaps. This
finding is supported by previous research show-
ing aviator performance is improved when using
flight displays that feature basic human factors
design principles (e.g., Andre, Wickens, &
Moorman, 1991). Furthermore, previous research
and discussions of situation awareness, which
theoretically encompasses misperceptions and
miscomprehensions, have emphasized the
necessity of well-designed technology for effec-
tively displaying information to the aviator for
maintaining safe performance (Endsley, 2015).
This suggests the technological environment
latent failure within DoD HFACS may also be
related to sensory misperceptions and mental
awareness failures within the preconditions tier,
further emphasizing the importance of the tech-
nological environment latent factor. These find-
ings advance our perpetual efforts as human fac-
tors researchers to demonstrate the importance
of well-designed tools and tasks to maintain safe
performance.

The two latent factors that were observed to
have impact on JDME specifically are both cat-
egorized outside the preconditions tier within
the DoD HFACS model. Planned inappropriate
operations and climate/cultural influences are
located within the unsafe supervision and orga-
nizational influences tiers, respectively. These
two latent factors reflect how the working envi-
ronment is shaped by leadership (both at the
supervisory or organization levels). As a
reminder, JDME errors within HFACS are con-
sidered “honest mistakes” and are the results of
individuals making incorrect choices (Wieg-
mann & Shappell, 2001, 2003). The latent fail-
ure planned inappropriate operations is defined
as a failure of supervision to adequately plan or
assess the hazards associated with an operation.
The current findings demonstrate that this latent
factor may indicate that aviators are put into
unfamiliar situations and therefore situations
with increased risk. Whereas it does not degrade
basic aviation skills per se, it may create unnec-
essary demands of the aviator’s decision-mak-
ing abilities.

The presence of the latent factors of planned
inappropriate operations, climate/cultural influ-
ences, and teamwork and the influences they
had on “human error” within the current mishaps

770 September 2018 – Human Factors

remain unclear. All three categories still encom-
pass human decision makers within an organiza-
tion, whether it be the cooperation of aviators
working toward a common goal (Teamwork) or
their leadership, the squadron or unit command-
ing officers, approving their operations and
assessing the risks of their missions (planned
inappropriate operations and climate/cultural
influences). Therefore, determining that these
latent failures are the causes of “human error”
committed by the single aviator is simply dis-
placing the label human error to higher levels
within the organization where humans are still
working within the constraints and influences of
the complex system. By acknowledging that
“human error,” wherever it occurs within an
organization, is still a result of the working con-
ditions, we determined additional analysis
beyond the data provided by DoD HFACS were
necessary. This allowed us to more thoroughly
address our research question of what latent fac-
tors were influential within the current Naval
aviation mishaps.

lImItatIons oF error
classIFIcatIon

The primary goal of the Bayes’ analysis was
to determine what latent factors impacted spe-
cific error types within Naval aviation mishaps.
This analysis was able to identify relationships
between latent factors and specific error types.
But the results have demonstrated that learn-
ing of these relationships does not guarantee
we will obtain a deeper and more meaningful
understanding as to the conditions that instigate
“human error.” Therefore, we felt it necessary
to address the limitations of DoD HFACS and
error classification systems in general.

The DoD HFACS framework has demon-
strated its effectiveness as an error classification
system, but the current results reveal limitations
that it does not provide valuable contextual
information imperative for gaining a deeper
understanding of “human error.” Studying the
contextual influences and constraints that pro-
voked the human to err is a hallmark of human
factors applied in real-world settings (e.g., Fitts
& Jones, 1947).

DoD HFACS and error classification systems
in general have been criticized for a variety of

reasons. First, as classification systems, they are
not effective at being able to capture information
relevant to the context and constraints workers
faced when being involved in an accident (e.g.,
Dekker, 2003). Second, they are considered too
reliant on hindsight bias, which hinders a deeper
understanding of why individuals did what they
did and particularly why they considered that
their actions (or inactions) would not have led to
a mishap at the time (Woods & Cook, 1999).
Lastly, others have found error classifications
systems and accident models in general can
unintentionally direct accident investigations in
such a way that provides an arbitrary stop-rule
for the investigation (Lundberg, Rollenhagen, &
Hollnagel, 2009). The safety investigation,
being guided by the accident model or error tax-
onomy, will not be inclined to consider other
possible contributors to the accident. Research-
ers and safety professionals, whether in aviation
or other domains, should recognize and consider
these criticisms and limitations of error classifi-
cation systems when analyzing their data.

For the present study, we chose to address the
limitations by extracting more qualitative, con-
textual data from the mishap reports themselves.
This, in essence, is what Fitts and Jones (1947)
understood when studying “human error.” Per-
formance problems are better understood by
examining the real-world constraints placed on
people’s behavior. Latent error categories within
DoD HFACS have helped narrow down this
examination, but we reached a point where the
data set could not provide sufficient resolution to
answer these questions. We sought to extract
context and meaning behind the teamwork
breakdowns, planning inappropriate operations,
and climate/cultural influences. The next section
presents the methods of the content analysis used
to obtain qualitative data about each mishap.

content analysIs method
Qualitative research emphasizes gathering

data relevant to the meaning of people’s lives
within their real-world settings, accounting for
the real-world contextual conditions, and pro-
viding insights that may help explain social
behavior and thinking (Yin, 2016). By ana-
lyzing individual mishap reports for thematic
patterns, we can illuminate concerns about the

Understanding HUman error 771

constraints or expectations the people involved
in the mishaps faced as the events were unfold-
ing. A methodical review of the mishap reports
encourages us to focus on what the people
involved knew and anticipated, thus minimiz-
ing the hindrance of hindsight bias prevalent in
error classification systems. The purpose of the
content analysis was to search across the mishap
reports to reveal thematic explanations: the pat-
tern of information across the reports that pro-
vides meaningful understanding of the condi-
tions and constraints affecting the performance
of the individuals involved. This would provide
insight for how teamwork broke down during
the mishaps as well as what influenced the com-
manders to both plan inappropriate operations
and create an unsafe working atmosphere with
climate/cultural influences.

Each Naval aviation mishap report is com-
prised of various sections, including an event
narrative, list of lines of evidence, set of rejected
causal factors (i.e., factors the mishap investiga-
tion board falsified as being causal to the mis-
hap), accepted causal factors, and list of recom-
mendations.

The content analysis was a methodically iter-
ative process comprised of three activities: (1)
disassembly, (2) reassembly, and (3) interpreta-
tion. Disassembly consisted of focusing on only
the areas of interest within each mishap report
(i.e., the narrative and relevant causal factor
analysis). If additional information was needed,
it would be referenced in the lines of evidence,
other causal factors, or supplemental evidence
stored outside the report within the data reposi-
tory. Researcher-derived notes were recorded
during each read-through. These included para-
phrases and early interpretations about recur-
ring factors that may have been part of a larger
pattern. The derived notes became the ingredi-
ents for the thematic explanations. During reas-
sembly, the derived notes were organized to
develop possible ideas for explanations. The
purpose of this activity was to find, assess, and
challenge robustness of the themes being
abstracted. For example, one of the initial
themes that began to emerge early during the
content analysis was difficulties inherent in
complex geopolitical coordination. Several
events occurred during joint operations, either

within or alongside foreign support. It seemed
at first that the influences of this massive chal-
lenge may have been a factor in planning inap-
propriate operations. This theme dissolved,
however, as the disassembly-reassembly itera-
tions progressed and was not considered as a
meaningful influence across the mishaps.
Lastly, during interpretation and as the thematic
explanations began to formalize, the content
analysis became dedicated to establishing the
pattern of how or why things happened across
the mishap reports. All three activities were part
of an iterative process to foster a flexible and
regular assessment of thematic explanations.

content analysIs results
In an effort to reduce speculation about what

took place during the event, we established cri-
teria for determining what mishap reports would
be acceptable for a content analysis. First, we
screened reports by examining the amount of
information provided within the narrative and
accepted causal factor analysis to determine if it
was adequate and relevant to the latent factors.
For instance, to build a subset of reports for
teamwork, only reports that provided informa-
tion specific to communication and cooperation
between team members actively involved in the
event were included. Because planned inappro-
priate operations and climate/cultural influences
were both observed to impact JDME from the
Bayes’ analysis, these factors were grouped
together for the content analysis. This also
allowed for a larger subset of mishap reports
that provided enough substantiating informa-
tion relevant to studying the contextual condi-
tions commanders faced when assessing and
approving the risk of their missions. The criteria
selection process resulted in 22 reports unique
to teamwork breakdowns and 23 reports unique
to the planned inappropriate operations and cli-
mate/cultural influence grouping.

The content analysis provided three distinct
thematic explanations: one for teamwork factors
and two for planned inappropriate operations
and climate/cultural influences. All privileged
safety information (e.g., locations, dates/times,
specific squadron or unit information, witness
interviews) as well as specific information of
aircraft model, mission type, or specific aerial

772 September 2018 – Human Factors

maneuvers and tactics has been withheld from
this paper. This was done to both comply with
the Naval Aviation Safety Management System
guidelines for not releasing privileged informa-
tion from a mishap report (Department of the
Navy, 2014) and take care in protecting the ano-
nymity of the groups and individuals involved
with the mishaps. Each thematic explanation
begins with a paraphrased quote from a particu-
lar report that provides exemplar context of the
conditions and constraints faced by the people
involved in the mishap. Within the quotes, cer-
tain words and phrases are bolded because they
are direct reflections of the underlying patterns
observed during the content analysis. Further
discussion is then provided. The following three
thematic explanations were abstracted from the
mishap reports following the content analysis.

Plan continuation aggravated by
diffusion of responsibility

Throughout the approach, [instructor pilot,
IP] had recognized multiple mistakes
[by the student pilot, SP], but attempted to
balance the requirements of both instruct-
ing and evaluating, acting as a competent
copilot, while still keeping the aircraft
safe. The IP allowed the maneuver to pro-
ceed and was looking for the SP to make
the necessary control inputs along with the
[lookout crewman, LCM] providing advi-
sory calls. The [LCM] felt that with the
pilots looking out the right, he should
cover the left side as well. Consequently,
the . . . maneuver progressed beyond a rea-
sonable margin of safety.

Plan continuation is a concept in complex,
dynamic systems, like aviation, where human
operators do not notice that a situation is gradu-
ally deteriorating from safe to unsafe. When
human operators begin to perform challenging
and hazardous tasks, they will first notice clear
and unambiguous cues that they recognize the
situation as familiar by remembering previous,
similar experiences (Lipshitz, Klein, Orasanu, &
Salas, 2001). Individuals are not making deci-
sions by assessing the pros and cons of all
choices available but rather are sensitive to cer-

tain occurrences they have seen and experienced
before; thus, they apply their previous experi-
ence to guide their actions and decisions in the
present. Plan continuation errors occur, how-
ever, when the event slowly progresses toward a
more hazardous and riskier situation and subse-
quent cues are much less clear, more ambiguous,
and overall weaker (Orasanu, Martin, & Davi-
son, 2002). These cues do not pull the people
into a different course of action, mostly because
they are anchored to the original, stronger cues,
thus making them less likely to change their
plans (e.g., Bourgeon, Valot, Vacher, & Navarro,
2011).

The current quote starts with an experienced
instructor pilot observing a less experienced stu-
dent pilot make mistakes. In hindsight, this is an
opportunity for the instructor to pause the train-
ing and take over. But in the moment, the instruc-
tor did not consider it unusual for the student to
be making seemingly harmless mistakes charac-
teristic of a pilot-in-training. This problem was
exacerbated by the social dynamics of diffusion
of responsibility.

Often referred to as the bystander effect, dif-
fusion of responsibility is the social tendency
when onlooker intervention during an emer-
gency situation is suppressed by the mere pres-
ence of other onlookers (Darley & Latane,
1968). As an unsafe situation is unfolding, an
onlooker will observe other onlookers not inter-
vening, thus confirming that this situation must
not be an emergency. Others have observed this
tendency within military settings, when the
onlooker is not a bystander per se and is still per-
sonally invested in the outcome of the event
(Bakx & Nyce, 2012; Snook, 2000). The previ-
ous quote demonstrates diffusion of responsibil-
ity during that mishap. The mission was com-
prised of three crew members: an instructor
pilot, a student pilot, and a lookout crewman.
Already with this many people involved, there is
likelihood for diffused responsibility. As the air-
crew was maneuvering around physical obstruc-
tions, the lookout crewman was executing the
single task he intended to complete: Look out
the left side of the aircraft while the pilots look
out the right. Diffusion of responsibility encour-
aged the two pilots to do the reverse with the
lookout crewman; he would look out for hazards

Understanding HUman error 773

in general, both to the right and left, while they
flew the aircraft. In the end, no one was at fault
for not looking out the right side because no one
was considered responsible for looking out the
right side.

This thematic explanation applied to 18 out
of the 22 mishap reports featuring teamwork
breakdowns. Each report discussed the begin-
ning of a multi-crew event that at first seemed
benign and manageable. As it progressed, how-
ever, it became more unstable and unsafe but not
obvious enough to signal to the aircrew that they
should stop. With two or more aviators and/or
aircrew involved, the diffusion of responsibility
worsened matters by unintentionally fostering a
context encouraging people to miss important
information and thus not be able to share it with
one another. The combination of these factors
evoked conditions that allowed small, subtle
changes and threats to go unnoticed, eventually
making it more difficult to recover from error.
The remaining two thematic explanations relate
to planned inappropriate operations and climate/
cultural influences factor grouping.

risk mitigation Program Incompatible
With unexpected hazards and risks

The [risk assessment] document itself,
as well as the [risk management] pro-
gram supporting it, while utilized in its
current form, was inadequate in identi-
fying the risks apparent after the mishap.
The command culture in the execution
of the [risk management program] failed
to identify unusual risks unique to the
[current situation].

Naval aviation squadrons follow the opera-
tional risk management or ORM (Department of
the Navy, 2018) program when assessing poten-
tial hazards (i.e., any condition with the potential
to negatively impact mission) and risks (i.e.,
chance that a hazard will actually cause harm).
ORM describes hazard assessment as “the foun-
dation of the entire [risk management] process. If
a hazard is not identified, it cannot be controlled”
(Department of the Navy, 2018, enclosure 1, p.
7). Thirteen out of 23 mishaps revealed that
squadron commanders were given unreasonable

expectations to algorithmically identify the
exhaustive collection of hazards and risks. These
expectations are incompatible with human judg-
ment in general, including the ability to assess or
anticipate risk.

The present quote demonstrates that there
was an expectation that the risk management
document and program was expected to identify
unusual risks unique to the current event. Like
plan continuation previously mentioned, only
hindsight would reveal that cues would have
been subtle and gone unnoticed by risk asses-
sors. Meanwhile, previous research has demon-
strated the high level of uncertainty within risk
assessment and identification. Orasanu (2010),
for instance, reported on the mismatch between
aviator’s assessment of risk salience versus risk
frequency. Generally, aviators tend to overem-
phasize salient risks (risks more familiar and
severe) and underemphasize frequent risks (less
familiar and seemingly inconsequential). This
was observed specifically in one of the mishap
reports within the subset. It mentioned a squad-
ron commander approving of unsafe operations
because he overemphasized one risk (crew
workday and rest) and did not accurately antici-
pate another risk (flying in a visually degraded
environment). This result is also supported by
previous research emphasizing the inherent sub-
jectivity of assessing risk (Orasanu, Fischer, &
Davison, 2002).

The ORM program may unintentionally be
placing squadron commanders and planners in
overly demanding situations for making judg-
ments. In these rare circumstances where unfore-
seen hazards create an unsafe situation, it is unrea-
sonable to expect commanders to suddenly become
risk prognosticators that can foresee all potentially
adverse outcomes, particularly in our perpetually
increasingly complex environment. Results sug-
gest that existing conceptual models of risk and
hazard management assessment should be exam-
ined for their effectiveness (e.g., Aven, 2016).

limited opportunities for deliberate
Practice of challenging tasks

It is also worth noting that the profi-
ciency concerned here, with regard to
[this particular aviation tactic], cannot be

774 September 2018 – Human Factors

obtained in a flight simulator. Current
versions of flight simulation do not have
the fidelity to simulate [the specific task
demands]. Therefore, such proficiency
must be gained with actual, dedicated
aircraft training.

Like any organization, Naval aviation has
limited resources. These organizational limita-
tions can exacerbate stress at the squadron level,
particularly when there are expectations for the
aviators to perform a certain number of opera-
tions or hours. With limited time, equipment,
and people, commanders are then considered
accountable because they put the aviators into
unsafe situations where the task demands
exceeded their capabilities.

The content analysis found 11 out of 23 mis-
hap reports supported this thematic explanation.
When an aviator, occasionally an amateur, was
put into a situation where the task demands
exceeded performance ability and it resulted in a
mishap, the commander was considered as plan-
ning inappropriate operations or setting up a cli-
mate/cultural workplace that encourages unsafe
behavior. Each of these events, however,
reported that the demanding tactics or maneu-
vers themselves, for a variety of reasons, were
rarely practiced. Deliberate practice is consid-
ered the intentional and effortful engagement
within a task with the intent to improve perfor-
mance (e.g., Ericsson, Krampe, & Tesch-Römer,
1993). An important aspect of deliberate prac-
tice is learning the subtle yet vital contextual
constraints that can impact performance. As the
quote demonstrates, performance of particular
tactics can best be refined in the actual context.

Across the 11 cases, there was one of three
explanations for why these particular skills were
rarely practiced. First, as the quote suggests,
there were resource limitations. Certain rarely
exercised skills could only be practiced in the
real environment. Second, there were no existing
policy requirements for the tactic to be requali-
fied, thus encouraging the skill to go unpracticed
for prolonged periods of time. Lastly, the skill
itself is unique and simply seldom exercised.

There was not enough information within the
mishap reports to delineate the specific nature of
the challenging tasks or the cognitive mecha-
nisms actively involved within the task. Regard-

less, this finding suggests a need to evaluate the
specific types of skills required for these unique
tactics and their susceptibility to performance
degradation. There are a variety of factors that
play a role in skill degradation, but degradations
of overall flight skills have been documented in
civil aviation pilots (e.g., Childs, Spears, &
Prophet, 1983). Clearly work is needed to better
understand the specific demands of these tasks
to help inform practical solutions for limited
degradation. This thematic explanation provides
a thorough description of a contextual constraint
encountered by squadron commanders.

content analysIs dIscussIon
The results of the content analysis revealed

three thematic explanations for the three remain-
ing latent failures. First, teamwork breakdowns
were found to be influenced by plan continua-
tion aggravated by diffusion of responsibility.
Individuals within a multi-crew arrangement
were observed to not intervene or speak up
when the team members all had a shared expec-
tation that a particular task was being performed
by someone else, occurring as an event was
slowly progressing toward a riskier situation.
Second, commanders were placed in difficult
conditions of judgment when using risk mitiga-
tion programs. They were held to an unreason-
ably high expectation of accurately foreseeing
all potential hazards when hazards have been
seen to go unnoticed or underemphasized due to
their subjective nature. Lastly, resource limita-
tions resulted in rare opportunities for aviators
to actively improve their skills within challeng-
ing tasks. These thematic explanations helped
provide a deeper understanding to “human
error” within Naval aviation mishaps.

The underlying problems within all three the-
matic explanations could potentially be miti-
gated by examining principles of resilience
engineering (Hollnagel, Woods, & Leveson,
2007). In short, resilience engineering empha-
sizes that success and failures are more closely
related than we think. When systems fail, it is
typically not because a single component,
whether human or mechanical, broke. But rather,
failure emerges from the vast interactions across
the web of a complex system of components.
Therefore, resilient systems emphasize gather-
ing evidence and data from normal events, not

Understanding HUman error 775

just mishaps, to assess human performance vari-
ation under differing conditions. This type of
holistic view at human performance within the
complex system will lead to improved concep-
tual risk models. Hazards originally unknown or
considered to be minor threats may turn out to
be more threatening than originally considered.

Weber and Dekker (2017) recently provided a
method for assessing pilot performance during
normal events. Observing and understanding pilot
performance during normal events helps provide
deeper understanding of performance constraints
during mishaps. For example, Weber and Dekker
reported pilots not following strict procedures dur-
ing normal operations, which would normally be
considered as being causal to an accident during a
mishap investigation but are actually intentional
deviations during normal events to maintain safety
during demanding situations. Understanding how
the front-line pilots, aviators, or human operators
within any complex system improvise to get the
job done as safely as possible is essential to gain a
deeper understanding for the conditions that con-
tribute to mishaps. These principles also apply
outside Naval aviation and could be implemented
within health care, oil and gas, transportation,
maritime operations, and most any complex sys-
tems (Hollnagel et al., 2007).

General dIscussIon
The goal of this paper was to gain a deeper

understanding of the factors that contributed
to “human error” within severe Naval avia-
tion mishaps. The first attempt to answer that
question was moderately successful. Applying
Bayes’ theorem to the DoD HFACS data set
helped identify that the technological environ-
ment was strongly associated with performance-
based errors. The remaining results, however,
provided little insight for examining beyond
“human error” as the label was displaced to
elsewhere within the framework where humans
were still involved. To better address the research
questions, a subsequent content analysis was
conducted on a subset of mishap reports to
extract qualitative data revealing the contextual
constraints and conditions faced by the people
involved. Three thematic explanations were
derived from the content analysis, all providing
a deeper understanding of “human error” within
Naval aviation mishaps.

The content analysis performed on the infor-
mation within the mishap reports was, to our
knowledge, the first examination of this kind on
these types of rare, extreme cases. The motiva-
tion to perform the analysis came when the
results of the DoD HFACS analysis revealed a
need to keep pursuing more context and infor-
mation on what influenced the errors to occur.
Other domains or organizations who have imple-
mented error classification systems could bene-
fit from this lesson. Error classification systems
are effective at just that: classifying error. Popu-
lating an error database based on safety investi-
gations does not guarantee that the information
within the database can be leveraged to predict
future “human error” occurrences. The current
project, however, demonstrates the limits of
error classification systems regarding the goal of
investigating beyond “human error.”

This project adds to the growing body of lit-
erature emphasizing the need to look beyond
“human error” as an acceptable explanation for
why mishaps and accidents occur (e.g., Tambo-
rello & Trafton, 2017). The motivation of apply-
ing Bayes’ theorem to DoD HFACS data and the
accompanying content analysis originated from
the understanding that “human error” is not
independent of the operating context, supervi-
sory practices, and organizational influences
surrounding aviators and squadron command-
ers. By expanding our knowledge of how con-
textual conditions influence human performance
in real-world military aviation mishaps, we can
begin to work toward solutions that address the
underlying systematic issues. The current proj-
ect demonstrated complementary analysis meth-
ods to provide a meaningful understanding of
“human error” in Naval aviation mishaps.

PractIcal ImPlIcatIons
Identifying the specific latent factors and

contextual conditions that influence the per-
formance of Naval aviators provides valuable
information about where authority figures can
allocate resources and apply interventions to
improve performance. The commanders of
Naval aviation squadrons will want to know the
specific areas of performance that should take
priority. The Naval Aviation Command Safety
Assessment Survey, for instance, is periodically
administered to all members of a squadron and

776 September 2018 – Human Factors

assesses the attitudes, perceptions, and overall
views of safety practices within a squadron
(for more details, see O’Connor & O’Dea,
2007). If results of the survey reveal concerns
of teamwork performance within the squadron,
the current project provides evidence, both from
the DoD HFACS analysis and content analysis,
specific to Naval aviation that this issue should
take top priority.

Furthermore, the current project provided
practical implications outside an esoteric appli-
cation to Naval aviation. Error taxonomies are
established in a variety of domains where
“human error” is susceptible to being considered
causal to accidents within complex systems
(e.g., Taib, McIntosh, Caponecchia, & Baysari,
2011). The results of the Bayes’ theorem analy-
sis revealed an inherent limitation to error tax-
onomies in that they lack the ability to capture
context, meaning, and constraints faced by the
people involved. These aspects of human work
are essential for accurate assessment of the con-
ditions that antagonize human performance to
drift outside the parameters of safe operation.

acKnoWledGments
The views and opinions expressed in this paper

are those of the author and do not necessarily rep-
resent the views of the U.S. Navy, Department of
Defense, or any other government agency. The author
would also like to acknowledge the valuable contri-
butions from Krystyna Eaker, Paul Younes, and Shari
Wiley in support of this paper.

Key PoInts
• “Human error” is an unhelpful yet common expla-

nation for the cause of accidents and mishaps in
complex activities featuring vast combinations of
people and technology (e.g., aviation).

• To better understand the conditions that influence
human error within Naval aviation mishaps, we
analyzed the DoD Human Factors Analysis and
Classification System (DoD HFACS) data and
found that technological environment impacted
performance-based errors among Naval aviators.

• The DoD HFACS analysis, however, was insuffi-
cient at providing meaningful contextual information
imperative for investigating beyond “human error.”

• A subsequent content analysis of mishap reports
found that teamwork failures were the result of

plan continuation aggravated by diffusion of
responsibility.

• Resource limitations and risk management deficien-
cies were observed to constrain the judgments made
by squadron commanders when planning missions.

reFerences
Andre, A., Wickens, C., & Moorman, L. (1991). Display for-

matting techniques for improving situation awareness in the
aircraft cockpit. The International Journal of Aviation Psychol-
ogy, 1(3), 205–218.

Aven, T. (2016). Risk assessment and risk management: Review
of recent advances on their foundation. European Journal of
Operational Research, 253(1), 1–13.

Bakx, G. C. H., & Nyce, J. N. (2012). Is redundancy enough?:
A preliminary study of Apache crew behaviour. Theoretical
Issues in Ergonomics Science, 14(6), 531–545.

Belland, K., Olsen, C., & Lawry, R. (2010). Carrier air wing mis-
hap reduction using a human factors classification system and
risk management. Aviation, Space, and Environmental Medi-
cine, 81(11), 1028–1032.

Bourgeon, L., Valot, C., Vacher, A., & Navarro, C. (2011). Study of
perseveration behaviors in military aeronautical accidents and
incidents: Analysis of plan continuation errors. Proceedings
of the Human Factors and Ergonomics Society 55th Annual
Meeting (pp. 1730–1734). Santa Monica, CA: Human Factors
and Ergonomics Society.

Chen, W., & Huang, S. P. (2014). Human reliability analysis in
aviation maintenance by a Bayesian network approach. In
D. Frangopol & G. Deodatis (Eds.), Safety, reliability, risk
and life-cycle performance of structures and infrastructures
(pp. 2091–2096). London: CRC Press.

Childs, J. M., Spears, W. D., & Prophet, W. W. (1983). Private pilot
flight skill retention 8, 16, and 24 months following certifica-
tion (Accession Number ADA133400). Daytona Beach, FL:
Embry-Riddle Aeronautical University.

Cohen, T. N., Wiegmann, D. A., & Shappell, S. A. (2015). Evaluat-
ing the reliability of the Human Factors Analysis and Classifi-
cation System. Aerospace Medicine and Human Performance,
86(8),728–735.

Darley, J. M., & Latane, B. (1968). Bystander intervention in emer-
gencies: Diffusion of responsibility. Journal of Personality and
Social Psychology, 8(4), 377–383.

Dekker, S. (2003). Illusions of explanation: A critical essay on
error classification. The International Journal of Aviation Psy-
chology, 13(2), 95–106.

Dekker, S. (2014). The field guide to understanding “human error”
(3rd ed.). Burlington, VT: Ashgate Publishing Company.

Department of Defense. (2011). Mishap notification, investigation,
reporting, and record keeping (DOD Instruction 6055.07).
Washington, DC: Author.

Department of the Navy. (2014). Naval aviation safety manage-
ment system (OPNAV Instruction 3750.6S). Washington, DC:
Author.

Department of the Navy. (2018). Operational risk management
(OPNAV Instruction 3500.39D). Washington, DC: Author.

DoD Human Factors Analysis and Classification System. (2017).
Department of Defense Human Factors Analysis and Clas-
sification System: A mishap investigation and data analysis
tool. Retrieved http://www.public.navy.mil/NAVSAFECEN/
Documents/aviation/aeromedical/DOD_HF_Anlys_Clas_Sys

http://www.public.navy.mil/NAVSAFECEN/Documents/aviation/aeromedical/DOD_HF_Anlys_Clas_Sys

http://www.public.navy.mil/NAVSAFECEN/Documents/aviation/aeromedical/DOD_HF_Anlys_Clas_Sys

http://www.public.navy.mil/NAVSAFECEN/Documents/aviation/aeromedical/DOD_HF_Anlys_Clas_Sys

Understanding HUman error 777

Endsley, M. (1995). Toward a theory of situation awareness in
dynamic systems. Human Factors, 37, 32–64.

Endsley, M. (2015). Situation awareness misconceptions and
misunderstandings. Journal of Cognitive Engineering and
Decision Making, 9(1), 4–32.

Ericsson, K. A., Krampe, R. T., & Tesch-Römer, C. (1993). The
role of deliberate practice in the acquisition of expert perfor-
mance. Psychological Review, 100(3), 363–406.

Fitts, P., & Jones, R. (1947). Analysis of factors contributing to 460
“pilot error” experiences in operating aircraft controls (AMC
Memorandum Report TSEAA−694−12). Dayton, OH: Wright-
Patterson AFB, Air Material Command.

Hollnagel, E., Woods, D. D., & Leveson, N. (2007). Resilience
engineering: Concepts and precepts. Burlington, VT: Ashgate
Publishing Company.

Hsiao, Y. L., Drury, C., Wu, C., & Paquet, V. (2013a). Predictive
models of safety based on audit findings: Part 1: Model devel-
opment and reliability. Applied Ergonomics, 44(2), 261–273.

Hsiao, Y. L., Drury, C., Wu, C., & Paquet, V. (2013b). Predictive
models of safety based on audit findings: Part 2: Measurement
of model validity. Applied Ergonomics, 44(4), 659–666.

Krulak, D. C. (2004). Human factors in maintenance: Impact on
aircraft mishap frequency and severity. Aviation, Space, and
Environmental Medicine, 75(5), 429–432.

Li, W. C., & Harris, D. (2006). Pilot error and its relationship with
higher organizational levels: HFACS analysis of 523 acci-
dents. Aviation, Space, and Environmental Medicine, 77(10),
1056–1061.

Lipshitz, R., Klein, G., Orasanu, J., & Salas, E. (2001). Taking
stock of naturalistic decision making. Journal of Behavioral
Decision Making, 14(5), 331–352.

Lundberg, J., Rollenhagen, C., & Hollnagel, E. (2009). What-You-
Look-For-Is-What-You-Find: The consequences of underlying
accident models in eight accident investigation manuals. Safety
Science, 47(10), 1297–1311.

Meshkati, N. (1991). Human factors in large-scale technological
systems’ accidents: Three Mile Island, Bhopal, Chernobyl.
Industrial Crisis Quarterly, 5(2), 133–154.

Norman, D. A. (1988). The psychology of everyday things. New
York, NY: Harper & Row.

O’Connor, P., & O’Dea, A. (2007). The U.S. Navy’s aviation safety
program: A critical review. International Journal of Applied
Aviation Studies, 7(2), 312–328.

O’Connor, P., & Walker, P. (2011). Evaluation of a human factors
analysis and classification system as used by simulated avia-
tion mishap boards. Aviation, Space, and Environmental Medi-
cine, 82(1), 44–48.

Orasanu, J. (2010). Flight crew decision-making. In B. Kanki,
R. Helmreich, & J. Anca (Eds.), Crew resource management
(2nd ed., pp. 147–179). San Diego, CA: Elsevier.

Orasanu, J., Fischer, U., & Davison, J. (2002). Risk perception: A
critical element of aviation safety. IFAC Proceedings Volumes,
35(1), 49–58.

Orasanu, J., Martin, L., & Davison, J. (2002). Cognitive and con-
textual factors in aviation accidents. In E. Salas & G. Klein
(Eds.), Naturalistic decision making (pp. 343–358). Mahwah,
NJ: Lawrence Erlbaum.

Patterson, J. M., & Shappell, S. A. (2010). Operator error and
system deficiencies: Analysis of 508 mining incidents and
accidents from Queensland, Australia using HFACS. Accident
Analysis & Prevention, 42(4), 1379–1385.

Ramoni, M., & Sebastiani, P. (2007). Bayesian methods. In
M. Berthold & D. Hand (Eds.), Intelligent data analysis: An
introduction (2nd ed., pp. 131–168). New York, NY: Springer.

Rasmussen, J. (1983). Skills, rules, and knowledge: Signals, signs,
and symbols, and other distinctions in human performance
models. IEEE Transactions on Systems, Man, and Cybernetics,
SMC-13, 257–266.

Reason, J. (1990). Human error. New York, NY: Cambridge Uni-
versity Press.

Snook, S. A. (2000). Friendly fire: The accidental shootdown of
U.S. Black Hawks over northern Iraq. Princeton, NJ: Princeton
University Press.

Taib, I. A., McIntosh, A. S., Caponecchia, C., & Baysari, M. T.
(2011). A review of medical error taxonomies: A human factors
perspective. Safety Science, 49(5), 607–615.

Tamborello, F. P., & Trafton, J. G. (2017). Human error as an
emergent property of action selection and task place-holding.
Human Factors, 59(3), 377–392.

Theophilus, S. C., Esenowo, V. N., Arewa, A. O., Ifelebuegu,
A. O., Nnadi, E. O., & Mbanaso, F. U. (2017). Human factors
analysis and classification system for the oil and gas industry
(HFACS-OGI). Reliability Engineering & System Safety, 167,
168–176.

Tvaryanas, A. P., & Thompson, W. T. (2008). Recurrent error path-
ways in HFACS data: Analysis of 95 mishaps with remotely
piloted aircraft. Aviation, Space, and Environmental Medicine,
79(5), 525–532.

Walker, P., O’Connor, P., Phillips, H., Hahn, R., & Dalitsch, W.
(2011). Evaluating the utility of DOD HFACS using lifted
probabilities. Proceedings of the Human Factors and Ergo-
nomics Society 55th Annual Meeting (pp. 1793–1797). Los
Angeles, CA: Human Factors and Ergonomics Society.

Weber, D. E., & Dekker, S. W. (2017). Assessing the sharp end:
Reflections on pilot performance assessment in the light of
Safety Differently. Theoretical Issues in Ergonomics Science,
18(1), 59–78.

Wiegmann, D., & Shappell, S. (1997). Human factors analysis of
postaccident data: Applying theoretical taxonomies of human
error. The International Journal of Aviation Psychology, 7(1),
67–81.

Wiegmann, D., & Shappell, S. (2001). Human error analysis of
commercial aviation accidents: Application of the Human Fac-
tors Analysis and Classification System (HFACS). Aviation,
Space, and Environmental Medicine, 72(11), 1006–1016.

Wiegmann, D., & Shappell, S. (2003). A human error approach
to aviation accident analysis: The Human Factors Analysis
and Classification System. Burlington, VT: Ashgate Publish-
ing Company.

Winkler, R. (2003). An introduction to Bayesian inference and
decision (2nd ed.). Gainesville, FL: Probabilistic Publishing.

Woods, D. D., & Cook, R. I. (1999). Perspectives on human error:
Hindsight biases and local rationality. In F. Durso & R. Nicker-
son (Eds.), Handbook of applied cognition (pp. 141–171). New
York, NY: John Wiley & Sons.

Yin, R. K. (2016). Qualitative research from start to finish (2nd
ed.). New York, NY: Guilford Press.

Andrew T. Miranda is an aerospace experimental
psychologist at the Naval Safety Center. He received
a PhD in human factors psychology in 2015 from
Wichita State University.

Date received: July 19, 2017
Date accepted: March 12, 2018

Operational Mishap and Incident Reports by Phase of Flight

Abstract

:

Voluntary reporting programs such as the Aviation Safety Action Program and Aviation Safety Reporting System exist so that organizations are able to detect problems, trends, and hazards during flight operations and make iterative improvements, constituting proactive safety management. Reporting systems are an integral component of safety management systems for flight schools. Understanding the difference in mishap frequency by phase of flight will help the aviation industry become more aware of when errors are occurring during flight training. This study examined the Operational Mishap and Incident Reports (OMIRs) from a collegiate flight program in the southeastern United States. De-identified data from the OMIRs for 2015-2017 were provided for the study. All reports were classified into seven phases of flight: ground-parked, ground-moving, takeoff and climb, cruise, maneuvers, pattern operations, and descent and landing. There was a significant difference in the frequency of OMIRs by phase of flight. Ground phases, both parked and taxiing, had the highest frequency of reports, followed by descent and landing.

Full text:

Headnote

Abstract
Voluntary reporting programs such as the Aviation Safety Action Program and Aviation Safety Reporting System exist so that organizations are able to detect problems, trends, and hazards during flight operations and make iterative improvements, constituting proactive safety management. Reporting systems are an integral component of safety management systems for flight schools. Understanding the difference in mishap frequency by phase of flight will help the aviation industry become more aware of when errors are occurring during flight training. This study examined the Operational Mishap and Incident Reports (OMIRs) from a collegiate flight program in the southeastern United States. De-identified data from the OMIRs for 2015-2017 were provided for the study. All reports were classified into seven phases of flight: ground-parked, ground-moving, takeoff and climb, cruise, maneuvers, pattern operations, and descent and landing. There was a significant difference in the frequency of OMIRs by phase of flight. Ground phases, both parked and taxiing, had the highest frequency of reports, followed by descent and landing.

1.Introduction

Voluntary incident reporting aims to identify variables that may lead to incidents and accidents in order to make corrective actions for future safety (Runciman, 2001). Given that approximately 34% of all general aviation accidents occur during flight training, and around 95% of those incidents are due to human error (Houston, 2012), the identification of mistakes and patterns of error during flight training through voluntary reporting has potential for a powerful effect on safety in aviation.

The purpose of this study was to determine whether there was a difference in the frequency of OMIRs by phase of flight at a collegiate flight school in the southeastern United States. Phase of flight was classified into seven phases: ground parked, ground moving, takeoff and climb, cruise, maneuvers, pattern operations, and descent and landing. The flight school provided archival data from all OMIRs submitted from January 2015 to December 2017 in the flight school’s safety management system (SMS).

Research and awareness is critical to promoting a safer flight-training environment for student pilots and instructors (Houston, 2012). Safety management systems are uncommon at collegiate flight schools. Thus, access to OMIRs provided a unique opportunity to learn about where errors are occurring in flight training. Understanding the variation in incidents and mishaps by phase of flight during flight training can be used to improve awareness for student pilots and flight instructors by highlighting specific phases of flight that incur more OMIRs. This could be the first step in identifying how to reduce the potential for mistakes and incidents that warrant a report. In addition, developing a plan to help reduce the potential of future mishaps will help lead the industry towards a safer flight environment.

This research was limited by the nature of voluntary reporting: it is possible that not all mishaps and incidents were reported. However, the SMS has built in incentives for reporting to increase participation, including immunity for pilots. It is also possible that some situations that warranted a report did not result in an OMIR because the pilot was not aware that the event should be reported through an OMIR. That said, the reporting system was introduced to all new student pilots, and flight instructors help teach new students about the system. Therefore, pilots at the flight school should be aware of the reporting system, when it is appropriate to submit a report, and the benefits to participating in the voluntary reporting system.

1.1. Aviation voluntary reporting systems

Although most aviation accidents have been attributed to human factors (NTSB, 2010), accidents, incidents, and lower-level errors or mishaps can be viewed from either an individual or system approach. The individual perspective focuses on mistakes, potentially caused by fatigue, task overload, or lack of attention. However, the system perspective emphasizes the role of the organization and environment in protecting against mistakes (Reason, 2000). From the organizational perspective, developing a positive culture of safety management is critical, and a central part of this is reporting mistakes so that learning can occur, rather than blame (Reason, 2000).

The aim of voluntary reporting in aviation is the ability to learn from more common small errors, in order to improve safety and avoid more severe accidents. Effective reporting systems in fields from aviation to medicine can enable learning through four essential functions: 1) gathering data that feeds into learning and safety culture rather than punishment or disciplinary action; 2) collecting complete data with closed and open questions; 3) implementing data analysis to translate reports into safety outcomes; and 4) providing constructive rather than critical feedback (Majahjan, 2010). Reporting systems need buy-in from everyone to function well (Majahjan, 2010; Runciman, Merry, & Smith, 2001). To increase participation, anonymity of reports allows for participation without fear of negative career impacts and increases the ability to collect data about causal reasons for errors (Runciman et al., 2001). Alternatively, reporting systems may incentivize participation by offering immunity or partial immunity in exchange for reporting and submission to any training deemed necessary. Regardless of the method, maximizing participation in a reporting system is critical to its effectiveness as a tool to learn from mistakes and proactively manage risk.

In aviation, several reporting systems exist. The Federal Aviation Administration (FAA) established the Aviation Safety Action Program (ASAP) to create voluntary reporting systems through safety partnerships between the FAA and businesses. For example, airlines and unions work with the FAA to promote confidential reporting of unsafe situations or occurrences. While not anonymous, some level of protection from punitive action and confidentiality serves to increase participation. The National Aeronautics and Space Administration (NASA) has oversight of the Aviation Safety Reporting System (ASRS) for the FAA, in order for the FAA, as the regulating agency, to maintain distance from the program. ASRS aims to support human factors research to identify issues and improve safety, is publicly accessible, and accepts submissions from anyone who witnesses a safety incident (e.g. air traffic controllers, pilots, maintenance, flight attendance). One of the central elements is confidentiality and anonymity provided by removing identifiers from data when it becomes part of the database and protection for pilots who submit reports (Cusick, Cortes, & Rodrigues, 2017).

Within organizations, developing a safety culture that places value on voluntary reporting is a central component to a safety management system. Although safety management systems are a recent development at collegiate flight programs, voluntary reporting enables identification of safety issues and iterative improvement. At the collegiate flight program in this study, a voluntary reporting system of OMIRs is used to collect data on safety-related events. This also serves to ensure that everyone involved in the flight program has an outlet to be heard, increasing participation and engagement (Runciman et al., 2001). The OMIRs may be submitted by anyone and serve as the interface to the outside reporting system, ASAP. Therefore, the reports also offer pilots performing operations for the primary business of the flight school immunity, a further incentive for participation. Collection of voluntary reports through the SMS also enables monitoring and safety research.

1.2. Phases of flight

The FAA (2016) breaks flight operations into nine phases of flight: preflight/taxi, takeoff, climb, cruise, descent, maneuvering, approach, landing, and other. While climb, cruise, and descent phases comprise 83% of flight time, only approximately 21.6% of general aviation accidents occur during these phases. In contrast, takeoff and initial climb represent only two percent of flight time but 23% of accidents, and 24% of accidents occur during landing (FAA, 2016). Thus, the takeoff and landing phases of flight alone account for approximately half of the accidents in general aviation.

Pilots are trained to anticipate workload and manage their workload. However, during a given flight, the approach and landing phase is when the pilot workload is highest. Pilots may reach task saturation, which means the workload may exceed the lowest margin of safety (FAA, 2016). This is one reason that a higher percentage of accidents occur during landing: the high workload may result in more human errors.

While the general aviation accident risk is highest in the takeoff and landing phases of flight (FAA, 2016), research is necessary to determine the incidence of mistakes by phase of flight during flight training. The voluntary reporting system consisting of OMIRs at a collegiate flight school provided the means to quantify and explore the differences in mistakes between phases of flight.

2.Methods

We used an ex post facto methodology in order to compare the frequency of OMIRs by phase of flight. An Institutional Review Board exemption (18-168) was approved. Deidentified data maintained the anonymity of individuals who submitted reports, and the study used archival data, having no direct contact with human subjects. The flight school provided de-identified data files, containing all OMIRs submitted from January 2015 through December 2017, or three full calendar years.

The flight phase column in the reports was used as the initial classification of phase of flight. Full reports and descriptions were used to code the 18 different phases of flight found in the dataset into seven categories: (a) ground parked, (b) ground moving, (c) takeoff and climb, (d) cruise, (e) maneuvers, (f) pattern operations, and (g) descent and landing. These categories align closely with the FAA (2016) phases of flight; however, parked and taxiing ground phases were separated, and takeoff and climb were combined. The number of OMIRs for each phase of flight was tallied by month (36 months). Descriptive statistics were calculated in Microsoft Excel. RStudio version 1.1.383 was used to calculate a one-way Analysis of Variance (ANOVA), Tukey’s pairwise comparison, and effect size.

3.Results

A total of 689 OMIRs were provided in the archival data; Table 1 shows the breakdown of OMIRs by year and by total flight hours at the flight school. This represents an average reporting rate of approximately one OMIR per 70 flight hours. A total of 18 OMIRs were excluded because the phase of flight was unknown or unclear from the report, resulting in 671 OMIRs that were used for the analyses. Figure 1 and Table 2 depict the 671 total OMIRs by phase of flight. Ground parked had the highest frequency of reports (177), followed by ground-moving (163) and descent and landing (115). The two ground phases together accounted for just over half of the total reports. The other five phases had fewer reports, with the fewest reports during maneuvers (38). Ground parked had the highest standard deviation (3.5), and maneuvers and cruise had the lowest standard deviations (1.1). Figure 2 depicts the average monthly OMIRs submitted by phase of flight with the standard deviation overlaid.

The ANOVA found a significant effect for phase of flight: F (6, 245) = 17.65, p< .01. To determine which phases of flight had different OMIR frequencies, a post hoc analysis was necessary. Tukey's pairwise comparison showed that 11 pairs of phases of flight had different OMIR frequencies at p< .02 (Table 3). Ground parked and ground moving were not different from each other, but were different from other phases: takeoff and climb, pattern operations, maneuvers, and cruise. Descent and landing was different from cruise and maneuvers. The eta-squared was 0.302, which is a small effect size.

4.Discussion

The data on OMIR frequency by month supported the hypothesis that there would be a difference in the frequency of reporting by phase of flight. As illustrated in Tables 2 and 3 and Figure 2, there is a difference in voluntary reporting based on phase of flight. The frequencies of OMIRs from the ground parked and the ground moving phases were significantly higher than other phases of flight, except descent and landing. Descent and landing had a higher frequency than cruise and maneuvering. However, the effect size (eta squared= 0.302) was small, suggesting that even though there is a statistical difference, the difference in the real world is small.

The difference in voluntary reporting underscores the higher frequencies during the ground parked and ground moving phases, which approximately half of the OMIRs referenced. There are several reasons that this may be the case. First, while still on the ground, the preflight examination of the aircraft and startup procedures have checklists, creating a multitude of potential mishaps if they are not properly completed. Second, while all operations necessarily include some phases of flight (e.g. ground parked, ground moving, takeoff and landing), not all flights will include every phase. Third, a self-report by a flight student or a report by a flight instructor would be most common for OMIRs during flight. However, on the ground, there are other people at the flight line that may observe an unsafe event and submit an OMIR, introducing the potential for more reports made by observers. Anyone can submit a report, and there are more observers on the ground.

Descent and landing was also significantly higher than cruise and maneuvering. Again, while almost all flights involve ramp or taxi phase and landing phase, not all operations have a cruise or maneuver phase, depending on the aims of a particular flight lesson. While parked and taxing reports were the most common, all flights involve a landing phase, and this is a heavy workload phase of flight for the pilot, introducing potential for mistakes, as compared to cruise and maneuvers, which are more straightforward and require a lower pilot workload.

When interpreting these results, the number or frequency of OMIRs should not be equated with risk. That is, even though the ground parked and ground moving phases had the highest OMIR frequencies, other phases of flight may have the higher risks. One cannot automatically assume that a higher number of reports is the result of a higher risk level in a specific flight phase.

Future research should evaluate trends in voluntary reporting over time. Elements such as number of daily operations at the flight line and airspace category should also be examined for their impact on frequency of reports. Finally, research relating the severity of the incident or potential risk with OMIR frequency would help to further understand these relationships and risks.

Voluntary incident reporting enables the aviation industry to proactively identify issues to improve safety. Analyzing OMIRs at collegiate flight training programs increases understanding of which phases of flight have more incidents and mishaps: ground parked, ground moving, and descent and landing. This, in turn, can be used by flight instructors to improve training. However, a higher frequency of reports in a specific phase does not necessarily mean that there is a higher associated level of risk.

References

5.References

Cusick, S. K., Cortes, A. I., & Rodrigues, C. C. (2017). Commercial aviation safety. New York: McGraw Hill Professional.

Federal Aviation Administration (FAA), 2016. Pilot’s Handbook of Aeronautical Knowledge. Federal Aviation Association. U.S. Department of Transportation. FAA-H-8083-25B. Retrieved from 

https://www.faa.gov/regulations_policies/handbooks_manuals/aviation/phak/media/pilot_handbook

Houston, S. J., Walton, R. O., & Conway, B. A. (2012). Analysis of general aviation instructional loss of control accidents. Journal of Aviation/Aerospace Education & Research, 22(1), 35-49.

Mahajan, R. P. (2010). Critical incident reporting and learning. British Journal of Anaesthesia, 105(1), 69-75.

National Transportation Safety Board (NTSB). (2010). Annual review of general aviation accident data 2006. (Annual Review NTSB/ARG-10/01). Washington DC.

Reason, J. (2000). Human error: models and management. BMJ: British Medical Journal, 320(7237), 768-770.

Runciman, B., Merry, A., & Smith, A. M. (2001). Improving patients’ safety by gathering information : Anonymous reporting has an important role. BMJ: British Medical Journal, 323(7308), 298.

Subject: Landing; Flight operations; Students; Descent; General aviation; Flight training; Pilots; Management systems; Aircraft accidents & safety; Flight safety; Participation; Operational hazards; Taxiing; Confidentiality; Flight hazards; Maneuvers; Aviation; Reporting; Human factors research; Learning; Safety management; Workloads

Business indexing term: Subject: Workloads Safety management

Location: United States–US

Company / organization: Name: Federal Aviation Administration–FAA; NAICS: 926120; Name: National Aeronautics & Space Administration–NASA; NAICS: 927110

Publication title: Journal of Management & Engineering Integration; tURLOCK

Volume: 12

Issue: 2

Pages: 105-111

Publication year: 2019

Publication date: Winter 2019

Publisher: International Conference on Industry, Engineering, & Management Systems

Place of publication: tURLOCK

Country of publication: United States, tURLOCK

Publication subject: Business And Economics–Management, Engineering

ISSN: 19397984

Source type: Scholarly Journals

Language of publication: English

Document type: Journal Article

ProQuest document ID: 2382630008

Document URL: 

https://nuls.idm.oclc.org/login?url=https://www.proquest.com/scholarly-journals/operational-mishap-incident-reports-phase-flight/docview/2382630008/se-2?accountid=25320

Copyright: Copyright International Conference on Industry, Engineering, & Management Systems Winter 2019

Last updated: 2020-10-29