Learning From Incidents in Aircraft Maintenance and Continuing Airworthiness Management A Systematic Review
Learning From Incidents in Aircraft Maintenance and Continuing Airworthiness Management A Systematic Review
Learning From Incidents in Aircraft Maintenance and Continuing Airworthiness Management A Systematic Review
Review Article
Learning from Incidents in Aircraft Maintenance and Continuing
Airworthiness Management: A Systematic Review
1 1,2
James Clare and Kyriakos I. Kourousis
1
School of Engineering, University of Limerick, Limerick V94 T9PX, Ireland
2
School of Engineering, RMIT University, Melbourne, Victoria, Australia
Received 10 September 2020; Revised 12 January 2021; Accepted 16 February 2021; Published 25 February 2021
Copyright © 2021 James Clare and Kyriakos I. Kourousis. This is an open access article distributed under the Creative Commons
Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is
properly cited.
The purpose of this systematic review is to highlight the salient elements of learning from incidents in the aircraft maintenance and
continuing airworthiness management area. This involved the review of more than 1,000 publications reflecting practice in
different domains. The cache was eventually distilled to 18 publications of relevance to learning from incidents. The systematic
review of the literature was not intended to be exhaustive, but it was deliberately bound by the parameters of predefined search
terms. A robust analysis was performed on the 18 distilled publications with the use of the NVivo software. A critical and
systematic examination of this body of literature further supported the development of the five codification themes. The analysis of
the literature revealed the benefits of a just culture as an enabler of reporting and learning from incidents. Moreover, it identified
limitations inherent in the current body of knowledge. The most evident being a paucity of literature relevant to the featured
industry segment. Some impediments to learning from incidents are also highlighted. Central to this is the prevalence of lack of
effective focus and practice on satisfactory causation of events. Currently, the efforts applied across many featured domains appear
to be based upon ineffective legacy linear practices. However, emerging investigative philosophies that look beyond direct cause
and effect contain opportunities for practitioners to consider causation through dawning axioms. This systematic review could be
used in the European aviation regulatory activities associated with improving learning from incident in aircraft maintenance and
continuing airworthiness management.
recognition that, as the aviation industry continued to de- and interventions of these events. Effective learning can be
velop, there were a number of factors outside the human at considered as a successful translation of safety information
play with a potential to affect safety behaviour [6]. This into knowledge that actively improves the operating envi-
paradigm-shift informed today’s systematic approach to ronment and helps prevent recurrence of events we can
safety and, in particular, the approach to learning from potentially learn from. Learning in this context can often be
incidents [7]. experienced as modifying or implementing new knowledge
Most people relate safety to freedom from risk and where cultural, technical, or procedural elements are inte-
danger [8]. Unfortunately, risk and danger are often ubiq- grated. Therefore, when learning in this context is trans-
uitous in the presence of high reliability activities. Managing formed into measures to prevent reoccurrence, an
sources of risk and danger are a tall order for some orga- organisation often has a reasonable means of mitigating
nisations. The ICAO Doc 9859 [4] recognises that “aviation future similar events.
systems cannot be completely free of hazards and associated The objective of this systematic review is to examine how
risks.” However, the guidance does acknowledge that, as learning from incidents occurs in aircraft maintenance and
long as the appropriate measures are in place to control these continuing airworthiness management and other sectors
risks, a satisfactory balance between “production and pro- and what issues impact learning in those areas. It also in-
tection” can be achieved. Perrow [3] acknowledges that “we tends to identify the contributing and constraining factors to
load our complex systems with safety devices in the form of learning from incidents. A qualitative review approach was
buffers, redundancies, circuit breakers, alarms, bells, and selected as it has the advantage of providing a deeper
whistles” because no system is perfect. contextual understanding of the literature and can assist
When one thinks of the word “incident,” it conjures up with better research integration. Applying a degree of rigour
the notion of an action that may have grave consequences. and comprehensiveness can assist with advancing knowl-
Similarly, the word “accident” is often used in the context of edge and identifying research gaps and aspects for further
an unplanned event or a particular circumstance. In many research in this particular area.
industrial sectors and business domains, these descriptors The publication’s systematic literature review covered
are used with a degree of interchangeability when the words primary publications up until 2017. As the subject of
are applied to describe events. In the world of aviation, there learning from incidents is a valid topic with potential to
are clear high-level definitions for both event categories, and augment safety, a brief review of a cross-section of the latest
these are based on potential for harm. Throughout aviation, publications was performed to see if a “delta” in the
learning from incidents is often considered to be one means knowledge exists. Insley and Turkoglu [11] reaffirm aircraft
of augmenting what Perrow [3] terms “safety devices.” maintenance is still a key point of concern within many areas
“Experience is the best teacher” according to Kleiner and of aviation. Their work highlights frequently recorded
Roth [9] as they claim that the causes of the mistakes are maintenance related consequences, naming runway excur-
often not featured and continue to be present in the absence sions and air turn-backs in the highest percentile. The study
of learning. In general terms, Nonaka [10] suggests that identified factors relating directly to these events naming
creating new knowledge extends past a mechanistic ap- inadequate and incorrect procedures, poorly executed in-
proach and is strongly related to employees’ insights. An spection tasks, and incorrect installation as common causal
effective enabler of learning in this area is the collation of factors ascribed to the event categories named. These issues
information on incidents. Details of the related processes, are not unique to Europe. Habib and Turkoglu [12] review a
environment, procedures, competencies, and implementing dataset of maintenance-related incidents originating outside
timely corrective actions all have a positive impact on of Europe (Nigeria). Their analysis revealed causal factors
learning and help prevent recurrence in the future. Learning such as poor aircraft husbandry, deficiencies in inspection
from incidents is therefore mainly associated with post- and testing, and inadequate safety oversight (organisation
incident learning. and regulator). Habib and Turkoglu [12] also consider the
Detecting and identifying hazards highlighted through consequential impact of errors as causal elements in sub-
incident reporting systems is recommended by International sequent events. They also highlight the increase in incidents
Civil Aviation Organisation (ICAO) standards and rec- recorded and attribute this to a recent increase in air
ommended practices as an effective means of achieving movements. Batuwangala et al. [13] present the idea that
practicable levels of safe operations. Therefore, objective data forecasted growth in air traffic requires a strong effort to
mined from a reporting system offers the potential to en- ensure aviation incidents continue to be progressively re-
lighten aviation stakeholders and to illuminate weakness duced. They recognise a novel approach to safety im-
that may be present. Such information can assist with a provements will need to be propagated in support of this.
better understanding of events and augment mitigating Although the authors point out some of the benefits of
measures against the potential effects of these hazards. When implementing a safety management system (SMS), they
incidents occur, this can be an indication of a failure in an reaffirm the notion that not all areas of aviation operations
organisation’s process and/or practice. Due to continuous are mandated to comply with SMS requirements. Some of
challenges faced by the organisations in the aviation industry the implementing constraints recorded by Batuwangala et al.
there is potential to learn from resulting incidents and [13] include protection of safety data/reporters, lack of just
precursors. The learning is based on the potential new culture and reporting, and reporting system deficiencies, to
knowledge available from the associated collection, analysis, name a few.
Journal of Advanced Transportation 3
The review of the sample examining a cross-section of This step concluded with the creation of an initial set of
current research in the area of aircraft maintenance and publications, which would further be filtered in next steps.
continuing airworthiness does not identify any significant
new knowledge in support of this publication. The additional
exercise reaffirms the concept that some organisations are 2.2. Practical Screen of Title and Abstract. In this step, each
continuing to ineffectively embrace a desire to learn from title and each abstract were reviewed (practical screen). This
incidents. part of the process not only had to be broad enough to create
a sufficient number of applicable publications but also had to
be practically manageable. The following criteria were laid
2. Materials and Methods down for the practical screen of the source bibliographic
In order to conduct an efficient and effective review, a details, title, and abstract:
structured approach was deemed necessary. Okoli and (i) Subject: related to learning from incidents and past
Schabram [14] state that “a dedicated methodological ap- experiences
proach is necessary in any kind of literature review.” An
(ii) Setting: any high reliability industry or sector where
initial search of literature highlighted a scarcity of best-
learning from incidents is critical.
practice guidelines for conducting systematic literature re-
views in the subject domain. This situation is also experi- (iii) Publication: journal or peer reviewed conference
enced in other sectors as Levy and Ellis [15] and Webster and proceedings
Watson [16] confirm. Qualitative research involves handling (iv) Date range: published post 1992
considerable volumes of data and a degree of discipline is
The output of the practical screen step produces a list of
required so that search results and decisions regarding
publications denoted as the screened set of publications. An
subject inclusions and exclusions are recorded and refer-
Endnote library was created to store and manage the full text
ences are well managed. Endnote was used in support of the
of the retrieved publications.
literature review during this research. An electronic database
is useful for supporting a search strategy, arranging publi-
cations, and storing references [17]. The qualitative data 2.3. Classification to Primary and Secondary Publications.
analysis software NVivo [18] was used to augment the data This step involved the filtering (classification) of publications
management, storage, and analysis associated with the lit- in the following two categories:
erature review. NVivo possesses many functions that are
capable of facilitating the synthesis of a review [19]. How- (i) Primary publications: any research publication
ever, the software does not have the capability of under- based on original data collected by the publications’
standing text and the analytical skills of a researcher cannot author(s)
be replaced in this respect. (ii) Secondary publications: those publications based on
data generated by somebody other than the au-
thor(s), e.g., a review and use of existing literature/
2.1. Search with Predefined Terms. Bandara et al. [19] suggest data developed by another party
two main criteria to consider before a search to identify
papers for extraction and review begins: the source and Effectively, the screened set of publications was split over
search strategy. The source considers which outlets and to a subset of primary publications and subset of secondary
databases to target, and the search strategy refers to the publications. Of those, in the next step, only the subset of
search terms and discipline to be exercised during the primary publications was used.
manuscript extraction process. A systematic search of the
literature was performed in the following databases:
2.4. Application of Inclusion and Exclusion Criteria.
(i) Web of Science [20] Brunton et al. [25] suggest there needs to be explicit in-
(ii) Scopus [21] clusion and exclusion criteria in order for the reviewer to
screen titles and abstracts for topical, population, temporal,
(iii) IEEE Xplore [22]
and methodological relevance. Having a set of criteria helps
(iv) ProQuest [23] to reduce any researcher bias in the screening system. A set
(v) EBSCO [24] of inclusion and exclusion criteria was developed consid-
ering the below objectives and in accordance with the
The following set of predefined terms associated with the
guidelines included in [26, 27]:
thematic of the systematic review was selected to search in
these sources: (i) To review current literature and to identify factors
related to learning from incidents
(i) “learning from incidents”
(ii) To identify obstacles and to learn from incidents
(ii) “learning from experience”
(iii) To make recommendations how learning from in-
(iii) “aircraft maintenance”
cidents might be improved in the aircraft mainte-
(iv) “aircraft management” nance and continuing airworthiness management
(v) “safety management systems” sector
4 Journal of Advanced Transportation
In this context, the inclusion and exclusion criteria Maykut and Morehouse [32] define a propositional
presented in Table 1 were used for the filtering of the subset statement as “a statement of fact the researcher tentatively
of primary publications. The output of this step leads to the proposes, based on the data.” Memos were used to draft
creation of the final set of publications. these summary statements which form part of Section 3 of
this paper.
Table 1: Inclusion and exclusion criteria used for the filtering of the subset of primary publications (table adopted from Clare and Kourousis
[28]).
Included Excluded
Research studies Literature reviews
Qualitative and mixed methods Quantitative methods
Perceptions and experiences Focused on decision-making and legislative requirements
Reference to just culture Not about “no blame” or a punitive approach
High reliability settings Nonhigh reliability settings
Published post 1992
Peer reviewed publications
Industry based settings
Original studies
Table 2: Codification themes used in the NVivo analysis of the final set of publications.
Codification theme Description Origin
Root cause Reason to establish causation Focus group
Reporting Value of reporting to learning from incidents Focus group
Learning from incidents Outcomes of learning from incidents Literature analysis
Just culture Impact of just culture on learning from incidents Literature analysis
Precursors Contribution of precursors to learning from incidents Literature analysis
Subset of Subset of
Application of inclusion and
secondary primary
exclusion criteria
publications publications
Learning
Root cause Reporting from Just culture Precursors
subset of subset of incidents subset of subset of
publications publications subset of publications publications
publications
Figure 1: Flowchart of the overall document screening process and associated steps utilised in the systematic review.
unable to diagnose faults in an accurate and timely manner. The Hobbs and Williamson [42] research study explored
The results of the study indicated that events are often caused patterns of potentially unsafe acts often perpetuated by
by poorly resourced supports, such as system diagnostics aircraft maintenance staff. Violations (routine and excep-
and test equipment. On a practical level, these contributing tional) and mistakes were found to be closely related to
factors are believed to have a negative influence on the deteriorating maintenance standards. A potential relation-
inability to establish adequate root causes and prevent the ship reinforces a link between violations and less than op-
recurrence of faults. timal safety standards. According to the researchers, root
6 Journal of Advanced Transportation
Search with
All publications in
predefined terms
databases
1,000+
289
Subset of Subset of
Application of inclusion and
secondary primary
exclusion criteria
publications publications
186 53
Final set of
publications
18
Figure 2: Output of the progressing filtering process applied during the systematic review, leading to the 18 publications (final set of
publications).
Table 3: A summary of attributes of the papers arising from the systematic literature search.
Paper Methodology Domain
Atak and Kingma [33] Qualitative Aircraft maintenance
Drupsteen and Hasle [34] Qualitative Chemical, construction, and manufacturing
Drupsteen and Wybo [35] Qualitative Healthcare
Drupsteen et al. [36] Qualitative Chemical, construction, energy, government, metal, and transportation
Furniss et al. [37] Qualitative Technology, transport, energy production, and healthcare
Gartmeier et al. [38] Qualitative Healthcare
Gerede [39] Qualitative Aircraft maintenance/regulatory
Gray and Williams [40] Qualitative Healthcare
Bjerg Hall-Andersen and Broberg [41] Qualitative Engineering consultancy
Hobbs and Williamson [42] Mixed Aircraft maintenance
Jacobsson et al. [43] Mixed Petrochemical, food and drug, and energy
Lukic et al. [2] Qualitative Energy
Pickthall [44] Mixed Aircraft maintenance
Silva et al. [45] Mixed Manufacturing, construction, production, and distribution of energy
Steiner [46] Qualitative Production and distribution
Storseth and Tinmannsvik [47] Qualitative Railway and maritime
Ward et al. [48] Qualitative Aircraft maintenance
Zwetsloot et al. [49] Mixed Manufacturing, construction, and others
cause of such violations can often be traced back to the resulting in recurrence of the event or similar ones. Their
prevailing culture within the organisation itself. study was conducted through questionnaire in health ser-
vices’ domains. They found that real learning from incidents
3.2. Reporting. In their work, Gray and Williams [40] ex- can take place as a result of a transformation effort facilitated
amined whether culture surrounding learning from inci- by a holistic approach. The authors refer to “reframed
dents can be compounded by “strategic defence routines,” learning approach;” however, the publication contains little
Journal of Advanced Transportation 7
Codification themes
(identified from focus group)
• Root cause
• Reporting
Final set of Analysis and codification
(identified from literature analysis)
publications with NVivo using themes
• Learning from incidents
• Just culture
18 • Precursors
Learning
Root cause Reporting from Just culture Precursors
subset of subset of incidents subset of subset of
publications publications subset of publications publications
publications
3 9 10 4 3
Figure 3: Distribution of the final set of 18 publications in the five codification themes following the NVivo analysis and codification step of
the systematic review process.
Table 4: Mapping of 18 publications (final set of publications) against the five codification themes.
Precursors Just culture Root cause Reporting Learning from incidents
3 4 3 9 10
Atak and Kingma [33] X
Drupsteen and Hasle [34] X X
Drupsteen and Wybo [35] X X
Drupsteen et al. [36] X
Furniss et al. [37] X
Gartmeier et al. [38] X
Gerede [39] X X
Gray and Williams [40] X
Bjerg Hall-Andersen and Broberg [41] X
Hobbs and Williamson [42] X X X
Jacobsson et al. [43] X X
Lukic [2] X
Pickthall [44] X X
Silva et al. [45] X X
Steiner [46] X
Storseth and Tinmannsvik [47] X
Ward et al. [48] X X X
Zwetsloot et al. [49] X
practical exemplars which would expand more on the details no evidence of collective interdomain learning across
and the applicability of a similar approach to learning from functions. The lessons learned are not through potential
incidents. negative consequences and respective actions arising from a
Gartmeier et al. [38] examined if reporting can be used as reporting system input but brokered through a moderated
a strategy for workplace learning in a health service setting. database. A single “embedded” case study may not support
They have considered error reporting attitudes and behav- the generalizability of the results in other domains. However,
iours in a two-stage study performed via a longitudinal for those who wish to develop a better understanding of
survey. The results suggest that organisations should high- learning processes across knowledge boundaries, the “im-
light benefits of error reporting, ease of use and accessibility plications for practitioners” contained in the study are
of reporting systems are important, and barriers can be considered applicable.
modified to encourage reporting. Steiner [46] conducted a qualitative study set in a
Bjerg Hall-Andersen and Broberg [41] conducted a workshop environment with data collected through semi-
“natural experiment” in an engineering consultancy firm. structured interviews, participant observations, document
Following implementation of an information transfer da- analysis, and note taking. The theoretical shortcomings
tabase, discreet learning processes found to be inter- defined by the literature that relate to barriers to organ-
connected within some domain elements. However, there is isational learning are discussed in the work. One may note
8 Journal of Advanced Transportation
that a consolidating feature of organisational learning, such dialogue. An extensive survey was performed across 27
as reporting of issues and data capture, are not adequately organisations. The qualitative methods (interviews and
discussed in the study. workshops) were applied although they were not formally
Atak and Kingma [33] conducted an ethnographic-based analysed, and their synopses were used to validate the survey
case study in an aircraft maintenance environment, aug- results. The survey component of the research records high
mented by field notes, document reviews, and interviews. scores relating to learning action; however, there were
Tensions between quality assurance and maintenance differences noted between staff’s perception (and manage-
management were identified and the prevailing safety cul- ment) of learning action in approximately 25% of cases.
ture examined in the context of “integration, differentiation, Moreover, there was less diversity recorded across the
and fragmentation.” The study offers a comprehensive learning condition dimension. The researchers considered
picture of the applied challenges experienced by aviation this analogous to organisational commitment to safety.
safety staff from an “embedded” perspective. However, the Safety commitment, communication, culture, and learning
measures to prevent bias and understanding the issues are were examined as individual aspects of implementing a zero-
not well-defined in the publication. accident environment. However, their cumulative rela-
Pickthall [44] examined the mixed methods approach tionship was not fully examined and the impact is not
using a structured interview devised from an academic discussed sufficiently.
format. This study examined issues that arose when aircraft Hobbs and Williamson [42] conducted a mixed method
maintenance staff interacted with complex aircraft systems study examining the application of a previously developed
for defect rectification. Occasionally a “no fault found” “three-way distinction” of unsafe acts questionnaire in an
determination has been found to be made. However, the aircraft maintenance context. An initial questionnaire was
fault-finding inputs in that case were ineffective, and the developed through the application of a disciplined confi-
fault returned soon afterwards. The research considered the dential critical interview technique with 72 aircraft main-
management-influenced behaviours such as time pressures, tenance mechanics. The results yielded 48 elements
poor communication, failure to adopt and share best (validated by air accident experts) and transposed into a
practice, inadequate training, and reluctance to change. The maintenance behaviour questionnaire distributed to 4,600
work uncovered that indispensable resources, such as air- licensed and 300 unlicensed aircraft maintenance mechanics
craft test equipment, integrated onboard diagnostic systems, (1359 questionnaires were returned). The principle com-
and maintenance manuals, often fail to support maintenance ponent analysis was the method used to reduce the number
staff when undertaking diagnosis tasks. The results suggest of variables in the dataset for analysis by extracting those
that these elements can actively constrain maintenance staff considered important to the study. The authors’ choice of
when they attempt to consistently manage effective and analysis does not appear to consider the competence in the
timely defect rectification. Moreover, the results are well context of skill-based errors and complex situations such as
presented and worthy of consideration when developing automation. However, the focus the publication brings on
training material in support of learning from incidents. the need for aircraft maintenance staff to be aware of the
Storseth and Tinmannsvik [47] performed a qualitative cumulative effect of “seemingly insignificant” incidents
study, using semistructured interviews in marine and rail fortifies the need to be proactive when it comes to learning
industries domains, to examine how individuals retro- from incidents.
spectively look back and consider learning from events.
Learning indicators for the study were developed by the
authors in an earlier related study. The research methods 3.3. Learning from Incidents. The objective of Lukic et al. [2]
were augmented by theoretical studies and document study was to highlight factors considered to be important for
analysis. They have found that learning within organisations effective learning, e.g., participants, process, incident, and
takes place within the parameters of “actor-context con- knowledge. Staff involvement and trust were positive at-
stellations” where there are no defined start and finish tributes capable of supporting learning. Attributing blame
points. This assumption is not sufficiently balanced against and poorly developed root causation were found to detract
the need to formally consider the exigency for structure from learning. The research also examined impact of formal
when developing learning from incident outcomes. and informal learning initiatives. Informal learning was
In their research study, Zwetsloot et al. [49] endorse the found to be more difficult to record and codify, and potential
importance of learning when implementing a “zero-accident for learning could be limited in some cases. In their paper,
vision” in nonaviation-related domains. The work also Lukic et al. [2] highlighted that the “over-simplification” of
highlights safety commitment, communication, and safety incidents and contend id, often the reason of incidents, are
culture as learning enablers. Research design was a mixed misunderstood when attempting to translate incident and
method approach using a quantitative survey supported by accident data into knowledge and learning. It is noted there
interviews and workshops. The qualitative component of the is an absence of information on the structure applied to the
research verified that learning was evident throughout the quantitative analysis and how rigour was applied to the
featured organisations. “Learning by doing” was considered process. However, the authors do clarify the analysis was
a more effective approach in support of learning from in- both data and participant driven.
cidents where employees are motivated to fully engage in the The Gerede [39] study considered some of the challenges
process, and supervisors can moderate theme-based safety associated with the successful implementation of safety
Journal of Advanced Transportation 9
management systems (SMS) in aircraft maintenance orga- presented, the staff continue to experience performance
nisations. The SMS structure is comprised of “safety policy constraints if communication remains poor.
and objectives, safety risk management, safety assurance, Jacobsson et al. [43] acknowledge the degree of interest
and safety promotion.” Safety risk management and safety invested in learning from incidents but question the effi-
assurance were found to be important elements under- ciency of learning from incidents in some organisations.
scoring the effectiveness of day to day activities. Failure to They found that event investigations often stop short and
create a just culture and fear of punishment for reporting only partially deal with some of the elements affecting the
shares a common cultural association. The situation is at- event. Although unwelcome events are less prevalent, less
tributed to a potential combination of lack of trust and severe events provide learning opportunities. Analysis of the
negative perceptions associated with organisational culture. learning cycle is valuable and such an approach can offer an
Moreover, Gerede [39] identified that the absence of insight into inherent precursors to accident conditions. They
communication and trust may present implementation present a model featuring: reporting, analysis, decisions,
challenges within the maintenance organisations. If a just implementation, and follow-up in an incident learning cycle
culture does not exist at national aviation authority state format. Assessing effectiveness of an incident learning cycle
level, then it is questionable if the implementation of an SMS was designed from analysing each individual step against the
would be effective. It is unclear if the four structural elements following dimensions: scope, quality, time, and information
of safety management were fully considered during the of the first cycle loop. A general assessment of the second
training or the data gathering phase of the study. This may learning loop was performed using participant interviews.
account for the absence of any direct reference to learning Subject matter experts applied their judgement in support of
from incidents in the study’s findings. developing weighting factors for each of the model elements.
Drupsteen et al. [36] conducted case studies with se- The paper refers to the analysis of incident learning systems
lected individuals in various domains, including trans- but the purpose of conducting the safety audit is not
portation. Their survey considered the following elements: specified. The relationship (if any) between the outcome of
steps in the process where learning is lost, formal organi- the safety audits and the efficiency of the learning systems
sation of steps, efficiency of steps on a daily basis, difference does not appear to be fully articulated.
between espoused and actual performance of steps, and Silva et al. [45] examine how organisations use accident
differences amongst featured areas. In their work, they also information to reduce the occurrence of unwelcome events.
state that “many incidents occur because organisations fail to They suggest it is necessary to achieve a balance between
learn from past lessons” because the traditional approach adequately resourcing safety initiatives and maintaining
often stops short of preventing future incidents. The research acceptable levels of safety. They suggest that factors such as
paper presented a model that examines the investigating and organisational culture, just culture, and event data, if
analysing incidents, planning and prevention, and inter- managed, can contribute to a reduction in events. Learning
vening and evaluating steps in a learning process. The within organisations should address effective information
evaluation stage was found to be a primary learning bot- processing and interpretation. Combining technical and
tleneck and reporting of incidents being next. Results in- social strategies resulted in uncovering four patterns of
dicated daily practice of learning was good, but follow-up practice that corresponded to different levels of learning.
steps in the process are often neglected in comparison to In their work, Drupsteen and Wybo [35] conclude that
incident analysis. There was a significant difference between organisations use experience gained from past events in
how well the investigation and incident analysis stage and order to improve safety. They introduce the term “pro-
the evaluation stage were performed and organised. pensity to learn” which refers to an organisation’s predis-
In their work, Ward et al. [48] offer a concise overview position to learning and suggest an organisation can apply
of key aspects of aircraft maintenance practice and lessons from past events such as warning signals, mistakes,
present an accurate snapshot of the development and incidents, and accidents. They found that hindsight can
architecture of pertinent regulation. Understanding the determine if an organisation did learn from an event, but
aircraft maintenance system complexities is an essential there are no models to assist with gauging the “propensity”
precursor to implementing improvements. Organisa- of an organisation to learn. The object of the study was to
tional processes cannot be explained in terms of a linear expound two sets of indicators that would contribute to
approach due to the nonlinear characteristics of flexibility gauging an organisation’s inclination to learn. Using a
and variability of comprising elements. It was found that previously validated questionnaire, the participants’ per-
the resulting relationship between the individuals and the ception was assessed on learning indicators. They deduced
systems have a direct impact upon the system and pre- from the review of literature that organisations displaying
vailing environment. Their model comprised of the fol- high learning propensity were also successful with learning
lowing elements: system level, process activity, from experience and sharing lessons amongst staff. Indi-
dependencies, and stakeholders. Four reporting veins cators based on three categories (attitudes and organisa-
were uncovered focusing on unique aspects of product tional conditions and systems) utilizing six indicators were
airworthiness and system performance, i.e., data inac- developed to gauge organisational learning. A second set of
curacy, quality assurance, personal injury, and occur- indicators was developed in support of assessing individual
rence reporting and suggested changes were highlighted. propensity to learn from experience, specifically measuring
The researchers found that regardless of how an issue attitude towards each of the stages of a generic learning
10 Journal of Advanced Transportation
process, i.e., detection, analysis, follow-up, evaluation, and methods to those based on performance metrics. These
sharing information. However, as the study was based solely challenges relate to the successful propagation of a just
on the perception of staff, it is unclear if the presented culture which is considered as a basic principle of successful
indicators alone would be satisfactory to elicit enough po- SMS implementation. The study strongly suggests that a
tentially subjective data to reinforce the results. failure to foster a just culture would be considered to have a
Furniss et al. [37] examined Hollnagel et al.’s [50] negative impact upon effective data collection (reporting),
Functional Resonance Analysis Method (FRAM) which organisational learning, and the subsequent ability to learn
explores how functional variability resonates within systems, from incidents.
i.e., how well elements work together in a system. They also Silva et al. [45] put forward the value of information
discuss how FRAM can be modified to support complex gleaned from incidents in support of learning and future
socio technical system improvements. This is presented in event prevention. They examine how organisations utilise
the context of four principles that encase the main as- information and the strategies that assist with the propa-
sumptions (equivalence of success and failure, approximate gation of lessons. They also highlight the need for organi-
adjustments, emergence, and functional resonance) from a sations to encourage a learning culture and suggest the
FRAM practitioner perspective. Their study considered how positive contribution made by reporting. It was found that a
human factor methods “are functionally coupled to a seminal element of organisational learning is a just culture,
broader system of human factors practice” [37]. The four where errors and mistakes can be reported, and violations
steps of the FRAM analysis were augmented by two addi- are managed fairly. In parallel, it is suggested that pro-
tional steps: the purpose of FRAM analysis and respondent portionate organisational responses are required to balance
validation. safety and accountability.
Drupsteen and Hasle [34] examined if organisations can In their work, Drupsteen and Hasle [34] proffer that
learn more effectively from past incidents, and future in- learning from past incidents can assist with understanding
cidents could be prevented. They suggest that learning can be potential future events and possibly reduce their conse-
improved if limiting factors are addressed. The learning quences. The study examines the causes associated with
process in different companies was analysed and discussed. organisations failure to learn from previous events. Trust
The researchers used a topic list to assess if human, technical, and openness were identified as key elements necessary for
or organisational aspects were being addressed and in which organisational learning. In the absence of these values,
elements were related to specific learning phases. They found under-reporting is often evident. The researchers point out
that some of the main causes of the constraints to learning that the presence of what they term a “blame culture” also
can be related to lack of knowledge, unwillingness to report, inhibits learning as potential reporters fear of being treated
causation not established, and uncertainty regarding follow- unjustly for their actions.
up action. Some conditions that enable these deficiencies are
centred around misplaced cultural issues, over-focus on
direct causation, and poorly defined safety management 3.5. Precursors. Ward et al. [48] suggest improvements can
procedures for example. The benefits of considering all be gained when organisational factors with a potential to
active and latent failures as direct and indirect causes, re- contribute to incidents are understood. They consider these
spectively, are unclear. The study concentrated on the la- elements in the context of the reason [8] taxonomy (im-
tency of causation. The authors state learning from incident mediate, workplace, and organisational) of factors as sys-
initiatives should exercise a more generic effort to support temic precursors. An improved understanding of these
prevention. However, one of the limitations stated was the elements can also shift the focus of unwarranted blame from
lack of homogeneity amongst the participating “the individual” within the system. Aviation maintenance
organisations. management systems are increasingly adopting an approach
where identifying systemic precursors contribute to a just
outcome.
3.4. Just Culture. Ward et al. [48] endorse the perception The main purpose of the Drupsteen and Wybo [35] study
that aircraft maintenance is a “highly regulated, safety was to develop a set of indicators capable of determining an
critical, complex, and competitive industry.” They also state organisation’s “propensity to learn.” The researchers argue
that to positively perpetuate the above attributes, it is that the most effective set of indicators are those that could
necessary to further develop an operational model that can be proactively considered as “leading indicators.” Precursors
account for “what is meant to happen and what actually that represent activity-based inputs can signal early degra-
happens.” A just culture is defined as “where people feel they dation of safety systems.
can report mistakes made without fear of punishment One of the main aims of the Hobbs and Williamson [42]
(deliberate acts of damage or violations are different).” The study was to ascertain if unsafe acts could be predicted as a
researchers proffer that a just culture can be considered as an result of analysing self-reported unsafe acts. Their analysis of
effective enabler of good quality incident reporting. demographic variables suggested that the occurrence of
Gerede [39] examines some of the challenges associated routine and exceptional violations was associated with a
with the implementation of the ICAO SMS standards and participants’ age. Higher levels of associated behaviours were
recommended practices which support the aviation industry linked with younger participants. The researchers were able
and regulators to transition from prescriptive oversight to identify potential precursors to aircraft quality issues by
Journal of Advanced Transportation 11
association with less than optimal performance of aircraft importance of reporting (incidents) and enabling facilitators
maintenance staff. The analysis implied a distinction exists such as the presence of a just culture cannot be overstated.
between what are termed routine and exceptional violations. Encouraging a reporting culture also reflects positively on
The former tends to be more frequent and can be associated the potential to learn from reported incidents.
with shortcuts linked to routine tasks. The latter group is of a The literature review also revealed the prevalence of
high-risk nature but occurs less frequently. similar constraints to learning in other industries. Lukic et al.
[2] highlight the increasing focus on learning from incidents
in the health, safety, and environmental areas of the energy
3.6. Common Limitations Identified in the Reviewed industry. They put forward factors they consider to be
Publications. Although there was a distinctive scarcity of important for effective learning which bring a focus on; the
information across the reviewed literature relating to the participants of learning, types of incident, types of knowl-
domain under primary investigation, enablers, and chal- edge, and learning process. Drupsteen et al.’s [36] industrial
lenges to learning in the featured preserves, which were well research (chemical, construction, energy, governmental
noted, learning from incidents across all domains shares a metal, and transport) states that “many incidents occur
kindred desired outcome of delivering lessons that help because organisations fail to learn from past lessons.” They
prevent recurrence of similar incidents in the future. point out that the traditional approach to learning often
However, throughout the review, a few common limitations features only a careful analysis and formulation of lessons in
were discovered in the literature and summarised as follows: the hope future incidents will be prevented. They suggest
(i) All research papers do not follow the same disci- that, in addition to focusing on prevention of reoccurrence,
pline of section title and content. the learning process should be improved which in turn can
contribute to making an organisation safer. Others such as
(ii) Few of the reviewed publications feature enough Jacobsson et al. [43] question the efficiency of learning from
detail in the methodology sections to aid with the incidents in some organisations (petrochemical, food and
exact replication of the featured study. drug, and energy) but suggest there is value in the analysis of
(iii) Details of piloting and testing data gathering in- the learning cycle. Such an approach can offer an insight into
struments such as semistructured templates were inherent weakness that often enables accidents. Silva et al.
scarce. [45] examine how organisations (manufacturing, con-
(iv) The robustness of some analyses was difficult to struction, production, and distribution of energy) use ac-
determine. cident information to reduce the occurrence of unwelcome
(v) The study featured participant perceptions, gauging events. They acknowledge there is a need to achieve a balance
the efficiency of lessons learned was not well between adequately resourcing safety initiatives and main-
supported in the text. taining acceptable levels of safety. In healthcare, Drupsteen
and Wybo [35] suggest an organisation can apply lessons
(vi) Safety culture and just culture are mentioned as arising from past events such as warning signals, mistakes,
pivotal to learning. However, there is no solid incidents, and accidents. Hindsight can assist with deter-
mechanism featured in support of objectively mining if an organisation did actually learn from an un-
measuring either cultural component in an aircraft welcome event, and their study expounds two sets of
maintenance and continuing airworthiness man- indicators that could contribute to gauging an organisation’s
agement environment. inclination to learn. By considering the outputs of research
(vii) The literature review uncovered many instances of in domains parallel to continuing airworthiness, the benefits
formal learning. It was noted that informal learning of proven approaches in other industries could be leveraged
practices were not well represented. and applied without further delay.
Many aspects of current literature are developed from a
4. Conclusions linear or sequential view of how an accident/incident occurs.
This of course might be an appropriate place to start to
The primary aim of learning from incidents is to support examine the retrospective aspects of learning that an un-
actions that contribute to preventing recurrence of unwel- welcome event can provide. However, more proactive
come events. The literature review revealed the existence of a models such as Hollnagel et al.’s [51] FRAM model, as
solid formal architecture capable of delivering lessons within highlighted by Furniss et al. [37], are very capable of de-
the featured domain activities. However, learning from livering more sustainable lessons. Nevertheless, it is evident
incidents is not specifically articulated as a requirement and from the literature search and review that research in the
therefore presently not all elements required are explicitly aircraft maintenance and continuing airworthiness man-
articulated with the regulatory code. Although some domain agement arena are yet not well represented in respect of
requirements mandate formal training, informal learning learning from incidents.
initiatives are not required to be capitalised upon. Addi- One potential benefit of digressing from the traditional
tionally, inadequate incident causation can deflect from view of causation is that models such as FRAM can be
potential learning opportunities arising from reporting. applied in support of specific analysis frameworks capable of
Poorly resourced efforts to establish appropriate causation deciphering: what went wrong, hazards that may have not
are recorded as a central impediment to learning. The been previously considered, and the feasibility of potential
12 Journal of Advanced Transportation
solutions to prevent recurrence. As human systems and [11] J. Insley and C. Turkoglu, “A contemporary analysis of aircraft
artificial intelligence continue to occupy shared workspaces, maintenance-related accidents and serious incidents,” Aero-
an appreciation of exactly how the system works is essential space, vol. 7, no. 6, p. 81, 2020.
in order to deliver effective lessons when unwelcome events [12] K. Habib and C. Turkoglu, “Analysis of aircraft maintenance
do occur. Further research in the continuing airworthiness related accidents and serious incidents in Nigeria,” Aerospace,
area utilizing forward looking frameworks such as FRAM vol. 7, no. 12, p. 178, 2020.
[13] E. Batuwangala, J. Silva, and G. Wild, “The regulatory
will have a positive impact on better understanding event
framework for safety management systems in airworthiness
causation. It will also present a need to examine and aug-
organisations,” Aerospace, vol. 5, no. 4, p. 117, 2018.
ment legislative requirements to support the needs of reg- [14] C. Okoli and K. Schabram, A Guide to Conducting a Sys-
ulatory and ethical oversight of systems that employ a blend tematic Literature Review of Information Systems Research,
of human and autonomous functionality. Elsevier, Amsterdam, Netherlands, 2010.
It is believed that the systematic review could be used to [15] Y. Levy and T. J. Ellis, “A systems approach to conduct an
refine terms of reference for a European legislative working effective literature review in support of information systems
group tasked with improving the content of the imple- research,” Informing Science, vol. 9, pp. 181–212, 2006.
menting regulations in the area of learning of incidents [16] J. Watson and R. T. Watson, “Analyzing the past to prepare
within the context of SMSs in aircraft maintenance and for the future: writing a literature review,” MIS Quarterly,
continuing airworthiness management organisations. vol. 26, no. 2, 2002.
[17] C. Houghton, K. Murphy, B. Meehan, J. Thomas, D. Brooker,
and D. Casey, “From screening to synthesis: using nvivo to
Data Availability enhance transparency in qualitative evidence synthesis,”
Journal of Clinical Nursing, vol. 26, no. 5-6, pp. 873–881, 2017.
The data supporting this systematic review are from pre- [18] NVivo (2020) https://www.qsrinternational.com/nvivo-
viously reported studies and datasets, which have been cited. qualitative-data-analysis-software/home -.
The processed data are available from the corresponding [19] W. Bandara, E. Furtmueller, E. Gorbacheva, S. Miskon, and
author upon request. J. Beekhuyzen, “Achieving rigor in literature reviews: insights
from qualitative data analysis and tool-support,” Communi-
cations of the Association for Information Systems, vol. 37,
Conflicts of Interest pp. 154–204, 2015.
The authors declare that they have no conflicts of interest. [20] Web of Science (2020) https://clarivate.com/
webofsciencegroup/solutions/web-of-science/-.
[21] Scopus (2020) https://www.scopus.com/-.
References [22] IEEE Xplore (2020) https://ieeexplore.ieee.org/-.
[23] ProQuest (2020) https://www.proquest.com/-.
[1] L. Redding, R. Roy, and A. Shaw, Advances in Through-Life [24] EBSCO (2020) https://www.ebsco.com/-.
Engineering Services, Springer, Berlin, Germany, 2017. [25] S. Brunton and A. Thomas, Information Management in
[2] D. Lukic, A. Littlejohn, and A. Margaryan, “A framework for Reviews, Sage, London UK, 2012.
learning from incidents in the workplace,” Safety Science, [26] T. Meline, “Selecting studies for systematic review: inclusion
vol. 50, no. 4, pp. 950–957, 2012. and exclusion criteria. Contemporary issues in communica-
[3] C. Perrow, Normal Accidents Living with High-Risk Tech- tion science and disorders,” ASHA, vol. 33, no. 21-27, 2006.
nologies with a New Afterword and a Postscript on the Y2k [27] H. C. A. Wienen, F. A. Bukhsh, E. Vriezekolk, and
Problem, Princeton University Press, Princeton, NJ, USA, R. J. Wieringa, “Accident analysis methods and models—a
1999. systematic literature review,” Centre for Telematics and In-
[4] ICAO, DOC 9859 Safety Management Manual, ICAO, formation Technology (CTIT), New York, NY, USA, TR-
Montreal, Canada, Doc 9859, 2013. CTIT-17-04, 2017.
[5] Transport Canada, Human Performance Factors for Elemen- [28] J. Clare and K. I. Kourousis, “Learning from incidents: a
tary Work and Servicing On-Line, Transport Canada, Ottawa, qualitative study in the continuing airworthiness sector,”
Canada, 2003. Aerospace, vol. 8, no. 2, p. 27, 2021.
[6] A. Hobbs, “The links between errors and error-producing [29] J. Kitzinger, “The methodology of focus groups: the impor-
conditions in aircraft maintenance,” in Proceedings of 15th tance of interaction between research participants,” Sociology
FAA/CAA/Transport Canada Symposium on Human Factors of Health and Illness, vol. 16, no. 1, pp. 103–121, 1994.
in Aviation Maintenance and Inspection, London, UK, Sep- [30] E. S. Bogardus, “Social distance in the city,” Proceedings and
tember 2001. Publications of the American Sociological Society, vol. 20,
[7] N. G. Leveson, “Applying systems thinking to analyze and pp. 40–46, 1926.
learn from events,” Safety Science, vol. 49, no. 1, pp. 55–64, [31] J. H. Frey and A. Fontana, “The group interview in social
2011. research,” The Social Science Journal, vol. 28, no. 2,
[8] J. T. Reason, Managing the Risk of Organisational Accidents, pp. 175–187, 1991.
Ashgate Publishing, Farnham, UK, 1997. [32] P. Maykut and R. Morehouse, Beginning Qualitative Research:
[9] A. Kleiner and G. Roth, Learning Histories: A New Tool for A Philosophic and Practical Guide, Falmer Press, London, UK;
Turning Organizational Experience into action, MIT Center Washington, D.C., USA, 1994.
for Coordination Science, Cambridge, MA, USA, 1997. [33] A. Atak and S. Kingma, “Safety culture in an aircraft main-
[10] I. Nonaka, “The knowledge-creating company,” Harvard tenance organisation: a view from the inside,” Safety Science,
Business Review, vol. 69, no. 6, p. 96, 1991. vol. 49, no. 2, pp. 268–278, 2011.
Journal of Advanced Transportation 13