Competency-Based Training and

Assessment (CBTA) Expansion within

the Aviation System

White Paper
Contents
Abbreviations and Acronyms ..................................................................................................................................................... 1
1. Introduction .......................................................................................................................................................................... 2
2. Aviation System and Safety Management System ................................................................................................. 4
2.1 Aviation system components ....................................................................................................................................... 4
2.2 Safety performance enhancements ........................................................................................................................... 5
2.3 Safety Risk Management ................................................................................................................................................ 6
3. Competency-Based Training and Assessment (CBTA) including Evidence-Based Training (EBT) ......... 9
3.1 Definitions ............................................................................................................................................................................ 9
3.2 Context................................................................................................................................................................................10
3.3 Competencies and Threat and Error Management .............................................................................................11
3.4 Training system performance .....................................................................................................................................12
4. CBTA opportunities ......................................................................................................................................................... 14
4.1 Safety enhancement ......................................................................................................................................................14
4.2 Training effectiveness and efficiency ......................................................................................................................16
4.3 Adapted to the individual pilot and instructor needs ..........................................................................................18
4.4 EBT Data Report update ...............................................................................................................................................18
5. CBTA challenges ............................................................................................................................................................... 21
5.1 Alignment of Safety Data and Training Data taxonomies .................................................................................21
5.2 Training data quality .......................................................................................................................................................23
5.3 License recognition ........................................................................................................................................................25
5.4 Training data protection................................................................................................................................................28
6. Conclusion and recommendations ............................................................................................................................. 31
Annex 1. Pilot competencies .................................................................................................................................................... 33
Annex 2. Instructor/Evaluator competencies ..................................................................................................................... 37
Annex 3. IATA ACTF TEM-based Accident Classification Taxonomy ......................................................................... 39
Annex 4. CBTA Instructor/Evaluator initial standardization .......................................................................................... 44
Annex 5. Authority CBTA Inspectors training and qualification example from EASA EBT regulation ............. 46
Annex 6. Data protection example from EASA EBT regulation ..................................................................................... 47
Abbreviations and Acronyms

AOC Operator holding an Air Operator Certificate

ATO Approved Training Organization

ATPL Airline Transport Pilot License

OEM Original Equipment Manufacturer

CAA Civil Aviation Authority

CCQ Cross Crew Qualification (an Airbus term)

CPL Commercial Pilot License

ICAP Interagency Committee for Aviation Policy

MPL Multi-Crew Pilot License

OCC Operational Control Centre

PPL Private Pilot License

Note: “Organization” refers to an ATO, an AOC or an OEM

1
1. Introduction
The civil aviation industry has been the fastest growing means of transportation in the last few decades.
The technical advances and considerable efforts to improve flight safety have led our industry to be
widely recognized as one of the safest means of transportation in terms of number of
passengers/kilometres.

These good results illustrate the safety commitment of our industry, which has been able to continuously
adapt, develop and implement consolidated standards related to personnel training and licensing, to
operational procedures and to the airworthiness of the airplanes.

Additionally, since the early stages of aviation, the accidents and incidents investigations protocols were
formalized to produce safety recommendations, to implement corrective action plans, and more recently,
to establish and maintain accident and incident databases. These databases facilitate the effective
analysis of information on actual or potential safety deficiencies and to determine any preventive actions
required.

Today, the aviation system safety performance has achieved an increased level of maturity as States,
airlines, training organizations, manufacturers and other service providers have begun implementing State
Safety Programs (SSPs) or Safety Management Systems (SMSs) that permit to have an harmonized safety
management approach among stakeholders, which consequently generates safety benefits.

As a matter of fact, this modern global safety management is significantly sustained by safety data as
both SSP and SMS mandate the establishment and maintenance of a formal process to collect, capture
and enable the analysis of hazards based on a combination of reactive and proactive safety data
collection methods.

However, today, the safety data integrates a very limited amount of training data or training records from
Civil Aviation Authority Licensing departments, Approved Training Organizations (ATOs), Air Operators
(AOCs) and other service providers.

In the context of the expansion of new training methodologies such as Competency-Based Training and
Assessment (CBTA) for several categories of personnel, and Evidence-Based Training (EBT) for pilots, the
aviation system is gaining access to a significant volume of training data that relates directly to human
performance.

Therefore, the purpose of this white paper is to inform about CBTA and its associated benefits, while
describing the nature and the value of the upcoming CBTA training data. The paper identifies the
opportunities and challenges related to the CBTA expansion and proposes recommendations for its
implementation by States and the industry.

In particular, this paper proposes solutions to integrate training data into the safety management system
to enhance operational safety, with the goal to cope with the increased complexity of the aviation system
due to the advent of different models of operations, the introduction of advanced technologies, the
design of new procedures and the enforcement of environmental constraints.

The role of training data, in regard to license recognition and training efficiency enhancements, is also
covered as there is, in the long-term, a global need for licensing harmonization under CBTA programs,

2
and, in the short and medium-term, for optimization of the training capacities in the context of the post
COVID restart of operations.

3
2. Aviation System and Safety Management System

2.1 Aviation system components

As this white paper relates to CBTA in the pilot and pilot instructor domain, the aviation system and safety
management are described from the pilot training and licensing, and operational perspective. As any
other system, the aviation system is constituted of three essential components where the pilots (and the
pilot instructors) represent the people, the operational procedures represent the processes, and the
aircraft represents the technology.

From an international perspective, since 1948, the States have agreed to adopt common standards for
pilot qualification, ICAO Annex 1 (Personnel Licensing), for operational procedures, Annex 6 (Operation of
Aircraft), and for aircraft design and certification, Annex 8 (Airworthiness of Aircraft). These standards also
describe the role and the requirements that are applicable to the organizations delivering pilot training
(ATOs), conducting operations (AOCs) and producing the aircraft (Original equipment manufacturer
[OEM]). The standards also define the obligation of the States in terms of certification of personnel and
organizations.

Since 1951, Annex 13 defines the standards that are applicable to the States in terms of accidents and
incidents investigations. The safety recommendations arising from accidents and incidents
investigations, combined with the continuous consolidation of the different annexes’ standards, have
been a key enabler to flight safety enhancements

Summary:
Aviation System components
People Processes Technology
Pilots and Instructors Operational Aircraft
Procedures

ICAO Standards

Service providers
Approved Training Air Operators Original Equipment
Organizations (ATO) (AOC) Manufacturers (OEM)
Deliver training Conduct operations Design and produce
aircraft

4
2.2 Safety performance enhancements
The aviation safety performance system has incrementally progressed in time by focussing on specific
areas of activity. From its origins until the end of the 1960s, safety performance was mainly enhanced by
technical developments that permitted to reduce the rate of aircraft system failures or malfunctions, and
with the integration of safety recommendations arising from accidents and incidents investigations, and
the continuous consolidation of the different annexes’ standards.

By the early 1970s, the frequency of aviation accidents had significantly declined due to major
technological advances and enhancements in safety regulations. Aviation became a safer mode of
transportation. The focus of safety endeavours was extended to include human factors. Those years saw
the promotion of Human Factors through the introduction of Crew Resources Management (CRM) training
programs as well as Line Oriented Flight Training (LOFT), which focused on the application of CRM in in-
flight scenarios.

During the mid-1990s, safety began to encompass organizational factors as well as human and technical
factors. Additionally, routine safety data collection and analysis, using reactive and proactive
methodologies, enabled organizations to monitor known safety risks and to detect emerging safety
trends. These enhancements provided the knowledge and foundation that have led to the current safety
management approach.

By the beginning of the 21st century, many States, airlines, training organizations and manufacturers had
embraced the safety approaches of the past and evolved to a higher level of safety maturity. They are
implementing State Safety Programs (SSP) or Safety Management System (SMS) and are reaping the
safety benefits.

ICAO Annex 19 (Safety Management), and related documents, describe the requirements related to the
SSP and SMS that sustain this global proactive approach to safety. In particular, the Safety Management
Manual (ICAO Doc 9859) provides guidance on interface management between organizations, which can
make a significant contribution to safety.

It is to be noted that States shall establish a process to investigate accidents and incidents in accordance
to Annex 13, in support to the State’s safety management.

Summary:

Safety Performance enhancements

5
2.3 Safety Risk Management
The safety management framework described in Annex 19, which is applicable to both States and
organizations, is composed of the following four elements:
 Safety policy, objectives, and resources,
 Safety risk management,
 Safety assurance, and
 Safety promotion.

Although differences exist in terms of scope and responsibilities, there are a lot of similarities between the
States and the organizations in the application of a safety management framework. In particular, the
Safety Risk Management process follows similar steps. First, the identification of hazards based on a
combination of a reactive and proactive methodology. Second, the assessment and management of the
risks associated with the identified hazards.

Before elaborating on hazard identification, it is important to remember that the States also have specific
obligations in terms of safety risk management. Among others, the States have obligations in regard to
licensing, certification, authorisation, and approval (CE-6) and resolution of safety issues (CE-8), which
correspond, respectively, to the State’s safety oversight critical elements number 6 and 8.

The role of CBTA in the support of CE-6 and CE-8 is described in Chapter 4. CBTA Opportunities, in this
document.

Actual hazard identification methods

The reactive hazard identification methodology involves the analysis of past outcomes or events. Hazards
are identified through the investigation of safety occurrences. Incidents and accidents are an indication of
system deficiencies and should, therefore, be analysed to determine which hazard(s) contributed to the
event.

The proactive hazard identification methodology involves collecting safety data of lower consequence
events, or process performance, and analysing the safety information or frequency of occurrence to
determine if a hazard could lead to an accident or incident. The safety information for proactive hazard
identification comes primarily from flight data analysis (FDA) programs and the safety reporting systems.

A safety reporting system includes a mandatory occurrence reporting that tends to collect more technical
information and operational deviations (e.g., hardware failures, level bust, etc.), than human performance
aspects. A voluntary safety reporting system will permit to address the need for a greater range of safety
reporting to acquire more information on human factors related aspects, and to enhance aviation safety.

6
Summary:

Example of hazard identification methodology

Reactive Reactive/Proactive Proactive
E.g., Analysis E.g., Analysis of event including E.g., Analysis of
Accident-Incidents Undesired Aircraft States Threat and Error
Management

Safety Data
ICAO Annex 19 defines Safety Data as “A defined set of facts or set of safety values collected from
various aviation-related sources, which is used to maintain or improve safety. Note.— Such safety data is
collected from proactive or reactive safety-related activities, including but not limited to: a) accident or
incident investigations; b) safety reporting; c) continuing airworthiness reporting; d) operational
performance monitoring; e) inspections, audits, surveys; or f) safety studies and reviews.

The effective management of safety is highly dependent on the effectiveness of safety data collection,
analysis, and overall management capabilities. Reliable safety data and safety information is needed to
identify trends, make decisions, evaluate safety performance in relation to safety targets and safety
objectives, and to assess risk.

Many ATOs and AOCs have collected a wealth of safety data and safety information, from mandatory and
voluntary safety reporting systems, as well as from automated data capture systems. This safety data and
safety information allows organizations to identify hazards and supports safety performance
management activities at the organization’s level.

ICAO Annex 19 requires States to establish a Safety data collection and processing system (SDCPS) to
capture, store, aggregate and enable the analysis of safety data and safety information to support the
identification of hazards that cut across the aviation system.

The safety data, safety information and their related sources are also subject to protection protocols in
order to ensure their continued availability, with a view to using them to maintain or improve aviation
safety, while encouraging individuals and organizations to report safety data and safety information. The
protection protocols are not intended to relieve sources of their safety related obligations or interfere
with the proper administration of justice.

7
Actual Training Data
Today, the safety data integrates a very limited amount of training data or training records from the CAA’s
Licensing Department (via certification records), ATOs (training data) and AOCs (training records).
Additionally, under traditional task-based training the quality of the training data and records does not
provide sufficient visibility on the pilot’s and instructor’s abilities to contribute efficiently to safe
operations. This limitation exists because traditional training focusses on a few technical skills, while
human performance encompasses a broader set of non-technical skills and attitudes.

Summary:

Example of typical safety data and safety information sources

8
3. Competency-Based Training and Assessment (CBTA) including
Evidence-Based Training (EBT)

3.1 Definitions
CBTA is defined by ICAO as training and assessment that are characterized by a performance
orientation, emphasis on standards of performance and their measurement, and the development of
training to the specified performance standards.

The goal of competency-based training and assessment is to provide a competent workforce for the sake
of a safe and efficient air transportation system.

CBTA is a training methodology sustained by robust course design, instructor qualification and data
collection to continuously enhance training efficiency and effectiveness.

As experience with CBTA has grown, the aviation industry has realized that CBTA is a better way to
develop a competent workforce when compared to the traditional task- or hours-based training and
checking.

CBTA is applicable to all spectrum of pilot training from pilot aptitude testing, pilot initial licensing
training, Instructor/Evaluator training and operator training.

EBT is defined by EASA as assessment and training based on operational data that is characterized by
the development and assessment of the overall capability of a pilot across a range of competencies,
rather than by measuring the performance in individual events or maneuvers.

EBT is a CBTA program that uses specific training topics as vehicles to develop the pilot competencies.
The training topics and their associated frequency were defined during the EBT design phase, through the
analysis of both safety and training data from a worldwide perspective.

EBT emphasizes training versus checking and promotes learning from positive performance.

With EBT, pilots are more competent and confident to perform their job in operations.
Illustration of an EBT module sequence

Today, EBT is a CBTA program applicable to operator recurrent training only.

9
3.2 Context
In 2006, ICAO supported a performance-based approach to training with the publication of standards for
the Multi-crew pilot license (MPL), which is the first license that is CBTA compliant.

In 2013, CBTA principles were extended to operator recurrent training with the publication of the ICAO
Doc 9995, Manual for Evidence-based Training (EBT).

In 2016, ICAO published Amendment 5 to PANS-TRG, General provisions for competency-based training
and assessment. This defined the role of the pilot competencies in the context of Threat and Error
Management (TEM) and provided a basis for the further development of CBTA.

In 2020, ICAO published Amendment 7 to PANS-TRG. This formalized the global expansion and
applicability of CBTA principles to all licensing training (ICAO Annex 1) and operator training (ICAO Annex
6).

These CBTA standards support the IATA Total Systems Approach (TSA), which stands for the application
of CBTA across all aviation disciplines in general, and to all modules and roles in a pilot’s entire career.
Hence, the defined competencies for pilots, instructors and evaluators should consistently be applied
throughout pilot aptitude testing, initial (ab-initio) training, type rating training and testing, command
upgrade, recurrent and evidence-based training and instructor and examiner selection and training.

In the last 15 years, many regulators have implemented CBTA principles and standards. The following
examples illustrate, among others, the global expansion of CBTA across the world:
 MPL was adopted in Europe as a common standard by the Joint Aviation Regulations (JARs) in
2006
 EBT, since the publication of Doc 9995, Manual of Evidence-based Training in 2013, has been
accepted as an alternative means of compliance to recurrent training and checking by several Civil
Aviation Authorities (e.g., the General Civil Aviation Authority (GCAA) of the United Arab Emirates)
 The Australian Civil Aviation Safety Regulations (CASR) introduced competency-based training
standards for all CASA flight crew qualifications in 2014
 EASA introduced EBT principles in 2016 and baseline EBT requirements were officially adopted by
the European Commission in December 2020
 EASA has launched a Rulemaking Task (RMT 0194) to introduce CBTA principles in the Aircrew
regulation (results expected in 2022)

10
Summary:

Global Expansion of CBTA within the Aviation System

3.3 Competencies and Threat and Error Management

Competencies are defined by ICAO as a dimension of human performance that is used to reliably predict
successful performance on the job. A competency is manifested and observed through behaviors that
mobilize the relevant knowledge, skills, and attitudes to carry out activities or tasks under specified
conditions.

The pilot competencies are the following:

Pilot competencies
 Application of Knowledge [KNO]  Communication [COM]
 Application of Procedures and  Situation Awareness and Management
Compliance with Regulations [PRO] of Information [SAW]
 Aeroplane Flight Path Management,  Leadership and Teamwork [LTW]
automation [FPA]  Workload Management [WLM]
 Aeroplane Flight Path Management,  Problem Solving and Decision Making
manual control [FPM] [PSD]

The pilot competencies were officially introduced as a new standard to measure the pilot’s performance
between 2008- 2013 when the design of EBT took place. The detailed pilot competency set is provided in
Annex 1.

The pilot competencies encompass what was previously known as technical and non-technical skills to
include the CRM skills of workload management, situational awareness, decision making, communication
and leadership, which are of upmost importance to ensure flight safety.

IATA also led the definition of a pilot instructor-evaluator competency set that was endorsed by ICAO in
2018. See the detailed instructor/evaluator competency set in Annex 2.

Under CBTA, Threat and Error Management (TEM) is naturally and fully embedded in the training
curriculum. The pilot and Instructor/Evaluator (IE) competencies provide individual and team

11
countermeasures to threats and errors to avoid a reduction of safety margins during training and
operations.

Summary:
From a Human Performance perspective,
Competencies = Countermeasures in TEM

3.4 Training system performance

CBTA is a performance-based training program that integrates, by design (Instructional System Design),
continuous monitoring and evaluation of the course.

Under CBTA, the training system performance is measured and evaluated through a feedback process in
order to validate and refine the curriculum, and to ascertain that the organization’s program develops pilot
competencies and meets the training objectives.

The typical CBTA feedback process should use defined training metrics to collect data in order to:
 identify trends and ensure corrective action where necessary,
 identify collective training needs,
 review, adjust and continuously improve the training program,
 further develop the training system, and
 standardize the instructors.

The typical metrics include but are not limited to:

 differences in success rates between training topics
 grading metrics
 trainee’s and instructor’s feedback, which provides an individual perspective as to the quality and
effectiveness of the training
 differences in success rates between different trainee cohorts
 distribution of errors for various training topics, scenarios and aircraft class or types
 distribution of the level of performance within the range of competencies and outcomes
 instructor inter-rater reliability data

12
Moreover, regulators and industry have agreed that the feedback process should be included in the
AOC and/or ATO Safety Management System and compliance monitoring.

13
4. CBTA opportunities

4.1 Safety enhancement

The shift, in terms of safety benefits, from traditional prescriptive task-based training to CBTA, is mainly
due to the expansion of the scope and nature of the training, and the enhancement of the measurement
of the performance.

Traditional training, which is hours-driven and task-based, focuses on training mainly three technical
elements: handling skills, automation management and application of procedures. The content of the
traditional skill test or proficiency check is based on exercises where the measurement of pilot
performance is mainly based on a set of fixed, predetermined criteria represented by numeric flight path
deviation tolerances.

In contrast, CBTA aims at assessing, developing, and enhancing the pilot competencies (see Annex 1) and
the Instructor/Evaluator competencies (see Annex 2). CBTA also uses more scenario-based training for
more realism and facilitation techniques by the instructor to support the pilot’s development; this
enhances the pilots’ competence and increases their confidence. Under a CBTA program, the pilots are
more resilient when managing unexpected situations in everyday operations.

Moreover, under CBTA the performance of the pilot is determined with more accuracy by using objective,
observable performance criteria that state whether (or not) the desired level of performance has been
achieved.

Additionally, the training metrics sustaining the monitoring and enhancement of the CBTA training
system’s performance constitute the core of the CBTA training data that should be collected and
analysed by the CAAs, ATOs and AOCs.

These training metrics, required under CBTA programs, were originally developed under modern training
programs such as Advanced Qualification Programs (AQP) regulated by the FAA, and Alternative Training
and Qualification Programs (ATQP) regulated by EASA.

To illustrate the specific value of the training data generated by a CBTA-EBT program, it is important
to remember that:
 Competency is a dimension of human performance that is used to reliably predict successful
performance on the job. A competency is manifested and observed through behaviors that
mobilize the relevant knowledge, skills, and attitudes to carry out activities or tasks under specified
conditions, and
 From a human performance perspective, the competencies represent the individual and team
countermeasures to manage the threats and errors and to avoid a reduction of safety margins.

Therefore:
 The more competencies’ Observable Behaviors are timely demonstrated when required, the better
the threat and error management should be. This should lead to the maintenance of the safety
margins.

14
  Per opposition, the competencies’ Observable Behaviors that have not been demonstrated when
they were required could result in the mismanagement of the threats and errors. This could lead to
a reduction of safety margins.

Hence, the training metrics relate directly to threat and error management and the recognition and
recovery of the potential reductions of safety margins that may have happened during training or
evaluation.

As an example, the following grading metrics (comprising four categories of metrics) had been introduced
within the European regulatory framework in the context of baseline EBT implementation.

Example of grading metrics mandated by the Evidence-Based Training European Regulation

Level 0 (competent metrics): The information whether the pilot(s) is (are) competent or not.

Level 1 (competency metrics): Level of performance reflected by numeric grade of the competencies (e.g., 1 to 5).

Level 2 (observable behavior metrics): The instructors record OBs predetermined or required by the organization
(Regulatory or Policy requirements).
Level 3 (TEM metrics): The instructor records Threats, Errors or Reduction of Safety Margin predetermined or required
by the organization.

The collection and analysis of these CBTA-EBT training metrics within the global Safety Management
System should, first, enhance a proactive hazard identification, second, support a more predictive
approach to hazards identification by providing visibility on the individual and the team countermeasures
(the competencies) to efficiently manage the threats encountered and errors committed in both training
and operational contexts.

The obvious value of these training metrics, from a single organization perspective, becomes exponential
when organizations are interacting with each other. This is the case when AOCs rely on ATOs to provide
the pilot workforce. The AOC and the ATO should collaborate to exchange the relevant elements of each
organization’s CBTA training metrics. As a very basic example, the AOC should provide to the ATO (in
charge of the AOC’s pilot training) the most relevant threats encountered in operations for the ATO to
introduce these threats within the flight training sessions of the type rating course.

Summary:
Example of hazard identification methodology expansion
Reactive Reactive/Proactive Proactive Proactive/Predictive
E.g. Analysis E.g. Analysis of event E.g. Analysis of E.g. Analysis of
Accident-Incidents including Threat and Error CBTA -EBT Training metrics
Undesired Aircraft Management
States

15
From a State perspective, CBTA should also support the efforts to enhance safety by providing a more
robust licensing system. In particular, the critical element “CE-6. Licensing, certification, authorization, and
approval obligations”, which is part of the State’s safety oversight system, is reinforced by the integration
of the CBTA training metrics. These metrics should permit enhancement in the accuracy and the reliability
of the pilot’s or instructor’s performance assessment requirements for the license, qualification or
certificate issuance, revalidation, and renewal. At the organizational level, the global collection and
analysis of the CBTA metrics should positively complement the actual safety performance indicators with
the goal to achieve better safety records.

Note: CE-6 mandates that States implement documented processes and procedures to ensure that
individuals and organizations performing an aviation activity meet the established requirements before
they are allowed to exercise the privileges of a license, certificate, authorization or approval to conduct
the relevant aviation activity.

CBTA should also facilitate the resolution of safety issues [another critical element of the State Safety
oversight system (CE-8.)] by providing more detailed and reliable trends from different organizations
about pilot and instructor/evaluator performance, in order to manage threats and errors in both training
and operational context. The States could interact proactively with the organizations under their oversight
by documenting and sharing all interface safety issues, safety reports and lessons learned, as well as
safety risks between interface. Sharing enables transfer of knowledge and working practices that could
improve the safety effectiveness of each organization.

Summary:

Enhanced State safety risk management via CE-6 and CE-8

4.2 Training effectiveness and efficiency

CBTA implicitly provides dynamic, effective, and efficient programs because it respects the instructional
system design concept. In particular, the ADDIE principles (Analyze, Design, Develop, Implement and
Evaluate) ensure that the training program is adapted to the organization and the pilot needs while making
best usage of training media and devices.

16
In practicality, using the defined pilot and Instructor/Evaluator competencies allows course designers to
get a clear idea of the scope of the training required to qualify pilots and instructors. This will enable them
to:
 Create consistent training programs
 Define training objectives effectively
 Allocate instructor resources and training media effectively
 Train pilots and Instructors/Evaluators specifically for their assignments; additionally, when
changing assignment or adding new assignments, the training needs can easily be identified

CBTA’s effectiveness and efficiency is also based on the consistent use of the same set of competencies
during the entire career path of the pilot, from aptitude testing, to PPL, CPL, MPL, ATPL, through operator
training, as well as for pilot Instructor/Evaluator.

This consistent use of pilot and instructor competencies facilitates training data exchange, the
benchmark of training metrics and training data analysis, in order to enhance individual courses, a
company’s training pathway and the performance of the global training system.

Additionally, CBTA drives and enables pilots and instructors to reach their highest level of performance
during all their training, and potentially beyond the training, and during their operational duties.

Practically, adopting a competency-based training approach for both pilots and IEs offers AOCs/ATOs the
opportunity to optimize training.

Efficiency can be improved by:

 Increasing effectiveness of instruction and evaluation
 Reducing the number of failures
 Identifying and avoiding duplications and overlaps in existing courses
 Merging content of different fleets courses
 Cooperating with other AOC/ATOs
 Introducing position/type optimized courses, e.g., for OCC, CCQ, requalification and bridge courses
 Standardizing the formats of the courses
 Optimizing scheduling and training time
 Using consistent data-driven feedback from students, instructors, and evaluators for course
evaluation

Regulators generally recognize the potential benefits of CBTA-EBT by supporting its expansion and
by providing training credits after a successful implementation by an organization.

17
4.3 Adapted to the individual pilot and instructor needs
CBTA follows a training plan with some inbuilt flexibility, and all pilot and instructor competency in all
stages and phases of training. As such pilots and instructors:
 Benefit from a training tailored to their individual needs
 Gain greater confidence in their ability to manage the unexpected and build resilience
 Are more motivated through the individualization of training and use of applied and relevant
scenarios
 Are supported and mentored to continuously improve in all areas, and, where feasible, the training
plan and time allocation is shifted toward the areas of the trainee’s weakness or concerns,
maximizing the effectiveness of the instructional contact time

Summary: CBTA is more trainee centered.

4.4 EBT Data Report update

The Evidence-Based Training project is one of the major achievements of the IATA Training and
Qualification Initiative (ITQI) launched in 2007. EBT was endorsed by ICAO in 2013, with the publication of
Doc 9995, Manual of Evidence-based Training. EBT is a major safety initiative that arose from an industry-
wide consensus that, in order to reduce the airline accident rate, a strategic review of recurrent and type-
rating training for airline pilots was necessary.

The whole concept behind the ITQI EBT project was to enhance flight safety, through data collection and
analysis and the use of the pilot competencies as countermeasures against the threats and errors
encountered in flight operations. The aim of EBT is to develop, maintain and assess the competencies
required to operate safely, effectively and efficiently in a commercial air transport environment, while
addressing the most relevant threats according to evidence collected in accidents, incidents, flight
operations and training.

Consequently, a review of available data sources, their scope, and relative reliability was undertaken. This
was followed by comprehensive analyses of the data sources chosen, with the objective of determining
the relevance of existing pilot training and to identify the most critical areas of training focus according to
aircraft generation. The publication of the EBT Data Report, 1st Edition, in 2014 was the result of the
corroboration of independent evidence from safety and training sources, which included among others,
flight data analysis, reporting programs and a statistical treatment of factors reported from an extensive
database of aircraft accident reports.

The IATA EBT Data Report ,1st Edition, states that EBT will continue to evolve as a result of continuous
feedback and the incorporation of new evidence as it becomes available. Hence, four years after its
publication, in view of the rapid changes in aircraft technology and in the operational environment, a
review of the latest data was necessary to assess the relevance of the EBT curriculum. Moreover, there
was also a need to look at the training data now available from operators that have implemented EBT
since its endorsement by ICAO in 2013. To support IATA in this analysis, an IATA EBT Subgroup,
constituted of representatives from operators and Approved Training Organizations (ATOs) that have
implemented EBT, was created.

18
During the review of the data sources and the methodology, the EBT Accident-Incident Study was
identified as one of the cornerstones of the EBT Data Report to ensure the relevance of the EBT
curriculum.

The EBT Accident-Incident Study is a two-stage analysis. Stage 1 involves the analysis of accident-
incident reports by a team of qualified analysts. This team of experts analyzes the reports and identifies
any threats, errors and pilot competencies (where the pilot competencies have been weak as
countermeasures) that have been identified as contributive factors to the accident or incident. IATA
Training and Licensing developed and provided a specific standardization (a 2-hour computer-based
training) to the analysts and designed an electronic tool to collect the results of the analysts. The
standardization of the analysts ensured the global consistency of the analysis, while the tool supported
the accuracy of the reporting/recording of the analysis results.

The standardization ensures accuracy and correctness of the data collected, while the tool supports the
global consistency of the analysis.

Stage 2 of the study is based on the results of Stage 1 and involves a statistical analysis within the six
generations of aircraft. The process enables the prioritization of training topics by training criticality from
a generational perspective, using the dimensionality of risk, clustering, and effectiveness of training. In
particular, Stage 2 of the study process applies the principles of risk management (risk probability vs risk
severity) by using an algorithm to prioritize training topics and determining training criticality.

Hence, the EBT Accident-Incident Study provides objectivity (qualification of the analysts) and reliability
during the analysis (algorithmic process) that consequently induces a strong relevance to the EBT
curriculum, in terms of training topic definition.

Therefore, one of the recommendations of the IATA EBT Subgroup has been to extend the analysis
methodology of the EBT Accident-Incident Study to lower consequence events such as, for example, the
reduction of safety margins events captured via mandatory occurrence reporting. This methodology
could also be applied to LOSA observations data and to Simulator Operations Quality Assurance (SOQA)
data.

In the context of the expansion of CBTA, there is an additional opportunity for the EBT Accident-Incident
analysis methodology to be applied to the CBTA training data collected in the training context and in
operations. This would permit to continuously evaluate the relevance of the EBT training program in light
of both safety data and training data collected at the organizational and State levels.

However, the opportunity to continuously update the EBT Data Report also represents an important
challenge in regard to storage, access, and protection of this sensitive and intimate training data, which
will need to be addressed by the industry.

19
Summary:
The extension of the EBT Accident-Incident Study methodology to the majority of the safety and training
data streams should permit a continuous and more robust update of the EBT curriculum.

Actual: EBT accident-incident study methodology

Future: Methodology applied to the majority of the data streams

20
5. CBTA challenges

5.1 Alignment of Safety Data and Training Data taxonomies

Safety data has been categorized using taxonomies and supporting definitions so that the data can be
captured and stored using meaningful terms. Common taxonomies and definitions establish a standard
language, improving the quality of information and communication. The aviation community's capacity to
identify and focus on safety issues is greatly enhanced by sharing a common language. Taxonomies
enable analysis and facilitate information sharing and exchange.

There are several common industry aviation taxonomies. Some examples include:
 ISIT (IATA Safety Incident Taxonomy): An occurrence category taxonomy that is part of IATA’s
accident and incident reporting system. ISIT sustains the IATA Global Aviation Data Management
(GADM) program which is the world’s most diverse aviation data exchange program. Data captured
in GADM databases comprise accident and incident reports, ground damage occurrences and
flight data from more than 470 different industry participants.
 ADREP (Accident/Incident Data Reporting) Taxonomy: An occurrence category taxonomy that is
part of ICAO’s accident and incident reporting system. It is a compilation of attributes and the
related values that allow safety trend analysis on these categories.
 Commercial Aviation Safety Team (CAST)/International Civil Aviation Organization (ICAO) Common
Taxonomy Team (CICTT): Task supported by IATA to develop common taxonomies and definitions
for aircraft accident and incident reporting systems.

The safety taxonomies are generally sufficiently detailed but, unfortunately, safety taxonomies are not
always consistent between databases. In which case, a data mapping should be used to standardize
safety data and safety information based on equivalency.

The safety taxonomies are generally organized around generic components that allow the user to capture
the nature of the contributive factors, the undesired aircraft state (UAS), and the end states, with a view to
aid the identification, analysis, and coding. As an example, the generic components of the IATA Accident
Classification Taxonomy are the latent conditions, the threats, the errors, the Undesired Aircraft State, the
end states, and the flight crew countermeasures.

As explained in Chapter 4, the training metrics relate directly to threat and error management and the
recognition and recovery of any reduction in safety margins that may have happened during training or
evaluation. Therefore, the generic components of the training data taxonomy should be similar to the
safety data taxonomy, and these two taxonomies should merge whenever the taxonomy content satisfies
both safety and training interests.

Hence, the safety data taxonomy should be aligned with the training data taxonomy, as it relates to flight
crew countermeasures, by adopting the pilot and instructor competencies. This step should be easy to
achieve and could be supported by the standardization (2-hour computer-based training) provided by
IATA Training and Licensing to the EBT Accident-Incident Study analysts.

Therefore, the training data taxonomy should be aligned with the safety data taxonomy as it relates to the
threats, errors, undesired aircraft states and end states codification, while safety taxonomy should be
aligned with the training data taxonomy as it relates to the flight crew countermeasures codification,

21
represented by the observable behaviors (OBs) of the pilot and instructor competencies. The States
should provide high level guidance about the safety and training data taxonomies alignment.

The template below illustrates, in a practical way, the integration of an extract of safety data taxonomy to
collect the level 3 (TEM metrics) grading metric mandated by the EBT European regulation. As the training
metrics are mainly captured by the instructors/evaluators in the training or operational dynamic
environment, a simple transfer of the safety taxonomy within the training metrics would not be a
reasonable solution. The ATO and AOC should be able to adapt the level of granularity and to select
the relevant taxonomy elements to be collected by each organization during operations and training.

Example of grading metrics mandated by the Evidence-Based Training European Regulation

Level 0 (competent metrics): The information whether the pilot(s) is (are) competent or not.

Level 1 (competency metrics): Level of performance reflected by numeric grade of the competencies (e.g. 1 to 5).

Level 2 (observable behavior metrics): the instructors record OBs predetermined or required by the organization
(Regulatory or Policy requirements).
Level 3 (TEM metrics): the instructor records threats, errors or reduction of safety margin predetermined or required by
the organization.

Example of threats, errors, and reduction of safety margins extracts from safety taxonomy that the ATO/ AOC
could define as relevant to be collected during a training or evaluation event.

1.Phase of Flight: GND, TO, CLB, CRZ, DES, APP, LDG, GA

2.Threats or EBT Training Topics [TT01 Adverse Weather, TT02 Adverse wind, TT03 System malfunctions…TT18
Workload, distraction, pressure]

E - Environmental Threats
E01 Meteorology
E01.01 Thunderstorm
E01.02 Poor Visibility/IMC
E01.03 Gusty wind/ windshear
E01.04 Icing conditions
…
A - Airline Threats
A01 Aircraft Malfunction
A01.01 Uncontained engine failure
A01.02 Contained engine failure (incl overheat and prop fail)
A01.03 Landing gear/ tires
…

3.Errors
H - Aircraft Handling Errors
H01 Manual handling/Flight Controls
H02 Ground Navigation (Surface nav)
H03 Automation (settings/selections)
H04

P – Procedural Errors
P01 SOP adherence/ cross-verification
P01.01 Intentional
P01.02 Unintentional
…

22
 4.Reduction of safety margins
U - Aircraft Handling
U01 Abrupt Aircraft Control
U02 Vertical, Lateral or Speed Deviations
U03 Unnecessary Weather Penetration
U04 Unauthorized Airspace Penetration
U05 Operation Outside Aircraft Limitations U06 Unstable Approach
U07 Continued Landing after Unstable Approach
…

Summary:

Solution for the Alignment of Safety and Training Data Taxonomies

Safety Taxonomy Training Taxonomy

Threats & Training topic

Errors

Undesired Aircraft State & Reductions of Safety Margins

End State

Flight Crew Countermeasures Pilot & Instructor

Competencies
(Annex 1 and Annex 2)

5.2 Training data quality

The alignment of the safety and training data taxonomies significantly facilitates the processing of data to
produce meaningful safety information in useful forms such as diagrams, reports, or tables. However,
there are a number of important considerations related to data processing, including: data quality,
aggregation, fusion, and filtering.

As training data relates to human performance, this chapter elaborates on the data quality aspects that
should be implemented to ensure a proper analysis. For the training data quality to be clean and fit for
purpose, it is important that this data collection happens in a very controlled environment.

ICAO doc 9859 (Safety Management System) indicates that data quality involves the following aspects:
a) cleanliness: data cleansing is the process of detecting and correcting (or removing) corrupt or inaccurate
records from a record set, table, or database and refers to identifying incomplete, incorrect, inaccurate or
irrelevant parts of the data and then replacing, modifying, or deleting the dirty or coarse data.

b) relevance: relevant data is data which meets the organization’s needs and represents their most important
issues. An organization should assess the relevance of data based on its needs and activities.

23
 c) timeliness: Safety data and safety information timeliness is a function of its currency. Data used for decisions
should reflect what is happening as close to real time as possible. Judgement is often required based on the
volatility of the situation. For example, data collected two years ago on an aircraft type still operating the same
route, with no significant changes, may provide a timely reflection of the situation. Whereas data collected one
week ago on an aircraft type no longer in service may not provide a meaningful, timely reflection of the current
reality.

d) accuracy and correctness: data accuracy refers to values that are correct and reflect the given scenario as
described. Data inaccuracy commonly occurs when users enter the wrong value or make a typographical error.
This problem can be overcome by having skilled and trained data entry personnel or by having components in the
application such as spell check. Data values can become inaccurate over time, also known as “data decay”.
Movement is another cause of inaccurate data. As data is extracted, transformed and moved from one database
to another, it may be altered to some extent, especially if the software is not robust.

The cleanliness aspects should be facilitated by allocating indicators to the different training or evaluation
events that the ATO or AOC are conducting. This filtering should permit to attach meaningful information
to each training or evaluation event. This would be the case when a different subset of data could be
identified for the training data collected during a line evaluation in operations (e.g., subset Alpha), during a
line evaluation in a flight simulator (e.g., subset Bravo), during an Upset Prevention and Recovery training
(e.g., subset Charlie), etc. This example illustrates the fact that each subset has its own value but
processing subset Charlie with subsets Alpha and Bravo could corrupt the quality of the results.

The relevance of the training data, the need for alignment for the safety and training data taxonomies and
the timeliness aspects, already elaborated in Section 5.1 above, are also fully applicable to the training
data.

The accuracy and correctness of the training data are fundamental aspects, as today the training data is
mainly collected by an Instructor/Evaluator (IE). This explains the reinforcement of the IEs’ initial and
recurrent standardization content of the CBTA programs. Additionally, training data collection is also
applicable to the IE population to ensure their performance level and the continuous enhancement of the
IE training programs. The regulators moving to CBTA-EBT also mandate to the organizations that are
implementing CBTA to put in place an Instructor Concordance Assurance Program (ICAP), which is a
critical element to obtain and maintain the CBTA approval.

The industry has also developed tools to provide the IEs with the recording of technical parameters
related to the flight crew performance. These tools support the IEs’ competency assessment by giving
access to objective training data and consequently increasing the accuracy and the correctness of the
training data quality.

Summary:

Solution to the training data quality challenge

- CBTA Instructor standardization and related Instructor Concordance

Assurance Program (ICAP) =>Refer to Annex 4
- Introduction of advance technology to support instructor competency
assessment

24
5.3 License recognition
The introduction of CBTA is an important shift in training, which has a significant impact on the issuing, the
revalidation, and the renewal of licenses and certificates.
Under traditional task-based and hour-based training, an applicant (pilot or instructor) seeking the
privilege of a license or certificate must follow a training course composed of different elements that
combine theoretical knowledge, ground and flight training. These elements prescribe minimum training
time and experience that assume that the training objectives can be achieved within that timeframe at a
normal pace. The training objectives focus on the completion of a tasks list that does not permit to reliably
predict successful performance on the job.

Once the training course is completed, the applicant is generally recommended for testing by the
organization or person responsible for the training. The content of the traditional skill test or proficiency
check is based on the restitution of exercises or maneuvers where the measurement of pilot performance
is mainly based on a set of fixed predetermined criteria represented by the flight path deviation numeric
tolerances. The skill test and proficiency check contents are harmonized across the different regulations
and are generally composed of a list of specific maneuvers that must be satisfactorily performed to obtain
a “pass” mark. See example below.

Example of a skill test or proficiency check content under traditional training

Maneuver/Procedure Limits for flight deviation

Heading with all engines operating ± 5°

Speed with simulated engine failure + 10
knots/– 5 knots…

-Take-offs with simulated engine failure between V1 and V2 Pass or Fail

-Windshear at take-off/ landing Pass or Fail

… …

-Landing with simulated jammed horizontal stabilizer Pass or Fail

Skill Test or Proficiency check global result Pass or Fail

Under CBTA, the aim of the training is to develop the nine pilot competencies and the four
Instructor/Evaluator competencies. The training course is also generally composed of theoretical
knowledge, ground, and flight training elements. Nevertheless, the training objectives are considered
satisfactorily completed when there is sufficient evidence to ensure that the trainee has achieved
competency, without any reference to prescribed training time, and that he meets the interim and/or final
competency standards. Under CBTA, the competency standards are the goals to be achieved, while the
tasks and the maneuvers are the vehicles to develop the competencies.

In the CBTA context, the evaluation of the applicant corresponds to a skill test or a proficiency check for
the issuing, revalidation and renewal of licenses and certificates. The evaluation of the applicant is a
summative assessment that is carried out at defined points during the training and/or at the end of the
training. During summative assessments, the decision is either “competent” or “not competent” with
respect to the interim or final competency standard(s).

25
Practically speaking, during the evaluation, the Instructor/Evaluator collects evidence on the presence,
the robustness and the effectiveness of the competencies by observing, recording and classifying the
Observable Behaviors demonstrated or not demonstrated by the applicant during the evaluation session.
This data collection and analysis is necessary for the Instructor/Evaluator to assess the applicant’s
performance in regard to the competency standards. See example below.

Example of a CBTA evaluation

Line Oriented The instructor collects evidence

- Departure Airport
 Observe performance (behaviors) during the evaluation.
- Introduction of relevant threats  Record details of effective and ineffective performance
during the flight profile (behaviors) observed during the evaluation (‘record’ in this
- Destination Airport (or Alternate context refers to instructors taking notes).
Airport)

End of the Evaluation session

 Classify observations against the Observable Behaviors
(OBs) and allocate the OBs to each competency (or
competencies).
 Assess the performance by determining the root cause(s)
according to the competency framework. Low performance
would normally indicate the area of performance to be
remediated in subsequent training.

Evaluation result Competent or not competent

This transition from traditional training to CBTA is a challenge for the personnel conducting the evaluation
as they must adopt the CBTA philosophy and apply a new methodology to assess the applicant’s
performance. This aspect is covered in Annex 4, which provides details for CBTA instructor
standardization and related Instructor Concordance Assurance Program (ICAP).

The transition from traditional training to CBTA also represents several challenges for the States that
integrate CBTA within their regulatory framework as the CAAs must define the competency standards to
be applied for the issuing, revalidation and renewal of the licenses and certificates. This implies that the
national competency standards related to each license or certificate delivered under CBTA are:
1) Acceptable from an international standard perspective, to ensure license recognition
2) Adapted to the different licenses: for private pilots, commercial pilots, multi-crew pilot, airline
transport pilots and related flight instructors
3) Comprehensive for the licensing personnel in charge of the evaluation of the individuals and the
organizations

The Chicago Convention Article 32 a) states that “the pilot of every aircraft and the other members of the
operating crew of every aircraft engaged in international navigation shall be provided with certificates of
competency and licenses issued or rendered valid by the State in which the aircraft is registered”.

26
Therefore, it is of upmost importance that an international competency standard be defined for the
licenses or certificates issued under CBTA. The newly reconstituted ICAO Personnel Training and
Licensing Panel should address this challenge and propose harmonized international solutions for points
1) and 2) mentioned above.

The challenge of the CBTA international license recognition represents an opportunity as well to have a
fresh start for a revisited common international competency standard for all licensed personnel.

It is to be noted that the methodology to assess the pilot and instructor competency (VENN Model) has
been implemented for more than a decade now in the context of EBT implementation.

The VENN model is a methodology to ensure the maximum level of consistency and objectivity to
assessments performed in a CBTA program.

The VENN model that has already been implemented/adopted by many organizations and regulators
should be endorsed by ICAO to provide harmonization on how the competency assessment is performed,
but the definition of the international competency standards should be defined by ICAO.

Competency assessment method, VENN methodology

To assess how well the trainee demonstrated the competency during training or evaluation, the trainer should
assess the associated OBs of each competency against the following dimensions by determining:
 How many OBs the trainee demonstrated when they were required;
 How often the trainee demonstrated the OB(s) when they were required; and
 What was the outcome of the threat management and error management relating specifically to the
competency being assessed?

The competency assessment (HOW WELL) is the combination of the number of OBs demonstrated and their
frequency of demonstration and the consequential outcome of the Threat and Error Management relating
specifically to the competency being assessed.

The “HOW MANY” dimension provides evidence related to the acquisition of the competency.

The “HOW OFTEN” dimension provides evidence related to the robustness of the competency.

The “Outcome of TEM” dimension provides evidence related to the effectiveness of the competency as individual
and team countermeasures against the threats and errors.

Detailed IATA competency assessment guidance:

https://www.iata.org/contentassets/c0f61fc821dc4f62bb6441d7abedb076/competency-assessment-and-evaluation-
for-pilots-instructors-and-evaluators-gm.pdf

Concerning point 3) above, another critical element of the State Safety Program is related to the
qualification of the State personnel (licensing personnel, pilot inspectors, etc.) and the technical guidance,
the tools and the information that should be provided to the personnel to perform their duties.

When introducing CBTA into the licensing system, the States should provide awareness and training to
ensure that licensing and operation personnel are able to evaluate an individual’s competency or an
organization’s ability to deliver CBTA programs, or, more generally, to interpret the role of the training data

27
within the global safety management. The content of this training is generally very similar to the CBTA
instructor initial standardization, while addressing specifically the oversight aspects of the CBTA
programs.

Annex 5 provides an example of pilot inspector training and qualification in the context of the introduction
of EBT into the European regulatory framework.

Summary:

Solution to the license recognition challenge

ICAO Personnel Training and Licensing:
- Defines a minimum competency standard for licenses issues under CBTA
- Endorses the competency assessment methodology (VENN model)
- Adopts a suitable training and qualification for CAAs’ personnel in charge of CBTA

5.4 Training data protection

As training data is part of the safety data within the safety management system, the protection
requirements that apply to safety should logically be applicable to the training data.

The objective of protecting training data is to ensure its continued availability, with a view to maintain or
improve aviation safety by continuously enhancing pilots’ and instructors’ performance and further
developing the training system. In this context, the importance of implementing protections cannot be
overstated.

The protections are not intended to relieve sources of their safety related obligations or interfere with the
proper administration of justice. Certain types of safety data and safety information that are protected
under Annex 19 may, in certain circumstances, be subject to other protection requirements. For example,
Annex 19 specifies that when an investigation under Annex 13 has been instituted, accident and incident
investigation records listed in Annex 13 are subject to the protections accorded in Annex 13, not those in
Annex 19.

Even though there are a lot of similarities between safety and training in regard to the protection
protocols, training data management is specific, as the States, the organizations, the pilots, and the
instructors have a particular interest in using it at the individual level.

To illustrate, in a practical way, the need to have access to training data at the individual level, let us have a
look at the EBT program; that is, an operator’s recurrent training program composed of six EBT modules
across a three-year period (two EBT modules per year). It should be noted that the EBT program permits
compliance with the ICAO standards related to the license revalidation (Annex 1) and the pilot proficiency
checks (Annex 6).

28
Each EBT module is clustered in three phases:
 The evaluation phase comprises a line-orientated flight scenario (or scenarios) to assess all
competencies and identify individual training needs
 The maneuvers training phase, comprising training to proficiency in certain defined maneuvers
 The scenario-based training phase, comprising a line-orientated flight scenario (or scenarios) to
develop competencies and address individual training needs.

To address the individual pilot’s training needs during the scenario-based training phase, in regard to the
evaluation phase, there is an obvious individual pilot training data transmission between the evaluation
and scenario-based training phases that should be managed in a controlled environment.

From a broader EBT perspective, the individual training data also supports the tailored training across the
six EBT modules within the three-year program.

This example related to EBT provides the rationale for the need to access the individual pilot training data:
 From a pilot’s perspective: to get access to a training tailored to his needs
 From an instructor’s perspective: to deliver adapted training to the individual pilots’ needs
 From an operator perspective: to adapt the training sessions to the individuals’ needs when
necessary and to implement the instructor concordance assurance program (ICAP refer to Annex
4)
 From a State perspective: to access individual training records when necessary (license
revalidation aspects) and perform oversight of the EBT training program to include the ICAP

Beyond EBT, which is an operator CBTA recurrent training example, CBTA expansion for all licensing and
operator training implies the use of individual training data from the early stages of the pilot’s career path:
during the selection process (Pilot Aptitude Testing), during the initial and advanced licensing training, and
during the operator training.

The benefits of CBTA are consequential to proper training data collection and analysis from a worldwide
and regional perspective (e.g., EBT Data Report), from an organizational perspective (operator’s pilot fleet
specific population) but also from an individual perspective (tailored training to pilot or instructor needs).

Hence, CBTA training data should always be protected and used in a de-identified format for global safety
management, while some protocols should permit the use of individual pilot data in the interest of
“routine” CBTA program operations. Routine CBTA program operations refers to CBTA program delivery
and monitoring by an ATO/AOC and oversight by the CAA.

Therefore, the newly reconstituted ICAO Personnel Training and Licensing Panel should address this
challenge and propose new standards levels in Annex 1 and Annex 6 for the protection of the CBTA
training data in the context of “routine” CBTA program operations, and their interrelation with Annex 13
and Annex 19 data protection standards should be clarified.

29
The following schematic provides general guidelines regarding the interaction between the protective
frameworks in Annexes 1, 6, 13 and 19, and is meant to be used in consultation with the applicable
provisions.

Summary:

Solution to the CBTA training data protection

ICAO Personnel Training and Licensing Panel to define the training data protection
protocols

30
6. Conclusion and recommendations
CBTA has been supported by IATA and the industry for more than 15 years now and its actual expansion
within the aviation system should continue to be supported by proper awareness and implementation
efforts by States and industry. CBTA is sustained by specific training data that brings additional value for
the global enhancement of safety management.

This paper has identified several challenges related to the introduction of CBTA within the aviation system
and proposes several solutions regarding the essential components of our aviation system, which are the
people, the process, and the technology.

People
The introduction of CBTA implies that the State and the organisations provide suitable awareness to the
personnel. In particular, the pilots need to clearly understand the impact of CBTA in regard to their own
training and evaluation. This paper proposes already existing instructor CBTA standardization to achieve
performance on the job and to ensure the quality of the training data collected. The instructor’s CBTA
standardization should be a guideline for the training and qualification of both the States’ CAA personnel
and the organizations’ SMS staff.

Process
This paper proposes robust procedures for competency assessment and evaluation. These procedures
and methods (VENN model), which have been positively implemented by the industry and adopted by
regulators, should be endorsed by ICAO as well as the associated instructor concordance assurance
program.

The ICAO Personnel training and licensing panel should also formalize an acceptable competency
standard for the issuance, the revalidation and the renewal of a license delivered under a CBTA program
and define the protection protocols applicable to the training data.

The alignment of the safety and training data taxonomies should be conducted as a global safety initiative
and therefore could be part of the ICAO Global Aviation Safety Plan (GASP) to ensure proper
implementation across the states and industry.

Technology
Today, training data collection relies heavily on the instructors/evaluators, the efforts to develop new
tools that take advantage of advanced technology should be maintained to increase the volume of
objective training data collection. This effort should enhance the quality of the data collected and should
also increase the training system efficiency.

The volume, the value, and the sensitivity of the upcoming CBTA training data will necessitate the creation
of a new and safe data repository that should permit access to the data for the benefit of global safety. As
an example, the continuous updating of the EBT Data Report will only be possible if there is a common
repository for several operators to record their safety and training data. Other international cooperation
to collect training data would be beneficial to ensure license recognition and global safety levels.

31
Summary:

Recommendations

People
 Instructors standardized according to industry best practices (e.g., IATA guidance
for instructors)
 CAAs’ licensing and operation personnel trained for CBTA
 ATOs’ and AOCs’ SMS staff trained for CBTA

Process
 ICAO to endorse industry best practices for competency assessment (e.g., VENN
model) and associated ICAP
 ICAO to define a minimum competency standard for licenses issues under CBTA
 ICAO Personnel Training and Licensing Panel to define the training data protection
protocols
 Alignment of the safety and training data taxonomies

Technology
 Develop innovative tools to increase the collection of objective training data
 Consider options for international training data repository setup and access

32
Annex 1. Pilot competencies

Competency
Observable behaviors
Description
Application of knowledge OB 0.1 Demonstrates practical and applicable knowledge of
limitations and systems and their interaction
Demonstrates knowledge and OB 0.2 Demonstrates required knowledge of published
understanding of relevant operating instructions
information, operating instructions,
aircraft systems and the operating OB 0.3 Demonstrates knowledge of the physical environment,
environment the air traffic environment including routings, weather, airports
and the operational infrastructure
OB 0.4 Demonstrates appropriate knowledge of applicable
legislation
OB 0.5 Knows where to source required information
OB 0.6 Demonstrates a positive interest in acquiring knowledge
OB 0.7 Is able to apply knowledge effectively

Application of procedures and OB 1.1 Identifies where to find procedures and regulations
compliance with regulations OB 1.2 Applies relevant operating instructions, procedures and
techniques in a timely manner
Identifies and applies appropriate
procedures in accordance with OB 1.3 Follows SOPs unless a higher degree of safety dictates
published operating instructions an appropriate deviation
and applicable regulations OB 1.4 Operates aeroplane systems and associated equipment
correctly
OB 1.5 Monitors aircraft systems status
OB 1.6 Complies with applicable regulations.
OB 1.7 Applies relevant procedural knowledge

Communication OB 2.1 Determines that the recipient is ready and able to receive
information
Communicates through appropriate OB 2.2 Selects appropriately what, when, how and with whom to
means in the operational communicate
environment, in both normal and
non normal situations OB 2.3 Conveys messages clearly, accurately and concisely
OB 2.4 Confirms that the recipient demonstrates understanding
of important information
OB 2.5 Listens actively and demonstrates understanding when
receiving information
OB 2.6 Asks relevant and effective questions
OB 2.7 Uses appropriate escalation in communication to resolve
identified deviations
OB 2.8 Uses and interprets non-verbal communication
in a manner appropriate to the organizational and social culture

33
 OB 2.9 Adheres to standard radiotelephone phraseology and
procedures
OB 2.10 Accurately reads, interprets, constructs and responds
to datalink messages in English

Aeroplane Flight Path OB 3.1 Uses appropriate flight management, guidance systems
Management, automation and automation, as installed and applicable to the conditions
OB 3.2 Monitors and detects deviations from the intended flight
Controls the flight path through path and takes appropriate action
automation
OB 3.3 Manages the flight path safely to achieve optimum
operational performance
OB 3.4 Maintains the intended flight path during flight using
automation while managing other tasks and distractions
OB 3.5 Selects appropriate level and mode of automation in a
timely manner considering phase of flight and workload
OB 3.6 Effectively monitors automation, including engagement
and automatic mode transitions

Aeroplane Flight Path OB 4.1 Controls the aircraft manually with accuracy and
Management, manual control smoothness as appropriate to the situation
OB 4.2 Monitors and detects deviations from the intended flight
Controls the flight path through path and takes appropriate action
manual control
OB 4.3 Manually controls the aeroplane using the relationship
between aeroplane attitude, speed and thrust, and navigation
signals or visual information
OB 4.4 Manages the flight path safely to achieve optimum
operational performance
OB 4.5 Maintains the intended flight path during manual flight
while managing other tasks and distractions
OB 4.6 Uses appropriate flight management and guidance
systems, as installed and applicable to the conditions
OB 4.7 Effectively monitors flight guidance systems including
engagement and automatic mode transitions

34
 Leadership and Teamwork OB 5.1 Encourages team participation and open communication
OB 5.2 Demonstrates initiative and provides direction when
Influences others to contribute to a required
shared purpose
OB 5.3 Engages others in planning
Collaborates to accomplish the OB 5.4 Considers inputs from others
goals of the team OB 5.5 Gives and receives feedback constructively
OB 5.6 Addresses and resolves conflicts and disagreements in a
constructive manner
OB 5.7 Exercises decisive leadership when required
OB 5.8 Accepts responsibility for decisions and actions
OB 5.9 Carries out instructions when directed
OB 5.10 Applies effective intervention strategies to resolve
identified deviations
OB 5.11 Manages cultural and language challenges, as
applicable

Problem Solving and Decision OB 6.1 Identifies, assesses and manages threats and errors in a
Making timely manner
OB 6.2 Seeks accurate and adequate information from

Identifies precursors, mitigates appropriate sources

problems; and makes decisions OB 6.3 Identifies and verifies what and why things have gone
wrong, if appropriate
OB 6.4 Perseveres in working through problems while prioritizing
safety
OB 6.5 Identifies and considers appropriate options
OB 6.6 Applies appropriate and timely decision-making
techniques
OB 6.7 Monitors, reviews and adapts decisions as required
OB 6.8 Adapts when faced with situations where no guidance or
procedure exists
OB 6.9 Demonstrates resilience when encountering an
unexpected event

35
 Situation awareness and OB 7.1 Monitors and assesses the state of the aeroplane and its
management of information systems
OB 7.2 Monitors and assesses the aeroplane’s energy state, and
Perceives, comprehends and its anticipated flight path.
manages information and
anticipates its effect on the OB 7.3 Monitors and assesses the general environment as it may
operation affect the operation
OB 7.4 Validates the accuracy of information and checks for
gross errors
OB 7.5 Maintains awareness of the people involved in or
affected by the operation and their capacity to perform as
expected
OB 7.6 Develops effective contingency plans based upon
potential risks associated with threats and errors
OB 7.7 Responds to indications of reduced situation awareness

Workload Management OB 8.1 Exercises self-control in all situations

OB 8.2 Plans, prioritizes and schedules appropriate tasks
Maintain available workload capacity effectively
by prioritizing and distributing tasks
using appropriate resources OB 8.3 Manages time efficiently when carrying out tasks
OB 8.4 Offers and gives assistance
OB 8.5 Delegates tasks
OB 8.6 Seeks and accepts assistance, when appropriate
OB 8.7 Monitors, reviews and cross-checks actions
conscientiously
OB 8.8 Verifies that tasks are completed to the expected
outcome
OB 8.9 Manages and recovers from interruptions, distractions,
variations and failures effectively while performing tasks

36
Annex 2. Instructor/Evaluator competencies
Competency
Observable behaviors
Description
Pilot Competencies Refer to observable behaviours in the Pilot Competencies in
Annex 1 above
Refer to the description in the Pilot
Competencies in Annex 1 above
Management of the learning IOB 2.1 Applies TEM in the context of instruction/evaluation
environment IOB 2.2 Briefs on safety procedures for situations that are likely
to develop during instruction/evaluation
Ensures that the instruction, IOB 2.3 Intervenes appropriately, at the correct time and level
assessment and evaluation are (e.g., progresses from verbal assistance to taking over control)
conducted in a suitable and safe IOB 2.4 Resumes instruction/evaluation as practicable after any
environment intervention
IOB 2.5 Plans and prepares training media, equipment and
resources
IOB 2.6 Briefs on training devices or aircraft limitations that may
influence training, when applicable
IOB 2.7 Creates and manages conditions (e.g., airspace, ATC,
weather, time, etc.) to be suitable for the training objectives
IOB 2.8 Adapts to changes in the environment whilst minimizing
training disruptions
IOB 2.9 Manages time, training media and equipment to ensure
that training objectives are met

Instruction IOB 3.1 References approved sources (operations, technical,

and training manuals, standards and regulations)
Conducts training to develop the IOB 3.2 States clearly the objectives and clarifies roles for the
trainee’s competencies
training
IOB 3.3 Follows the approved training program
IOB 3.4 Applies instructional methods as appropriate (e.g.,
explanation, demonstration, facilitation, discover with
assistance, discover without assistance)
IOB 3.5 Sustains operational relevance and realism
IOB 3.6 Adapts the amount of instructor inputs to ensure that
the training objectives are met
IOB 3.7 Adapts to situations that might disrupt a planned
sequence of events
IOB 3.8 Continuously assesses trainee’s competencies
IOB 3.9 Encourages the trainee to self-assess
IOB 3.10 Allows trainee to self-correct in a timely manner
IOB 3.11 Applies trainee-centered feedback techniques (e.g.,
facilitation, etc.)
IOB 3.12 Provides positive reinforcement

37
 Interaction with the trainees IOB 4.1 Shows respect for the trainees (e.g., for culture,
language, experience)
Supports the trainees’ learning and IOB 4.2 Shows patience and empathy (e.g., by actively listening,
development reading non-verbal messages and encouraging dialogue)
IOB 4.3 Manages trainees’ barriers to learning
IOB 4.4 Encourages engagement and mutual support
IOB 4.5 Coaches the trainees

and IOB 4.6 Supports the goal and training policies of the
Demonstrates exemplary behaviour operator/ATO and Authority
(role model) IOB 4.7 Shows integrity (e.g., honesty and professional
principles)
IOB 4.8 Demonstrates acceptable personal conduct, acceptable
social practices, content expertise, a model for professional and
interpersonal behaviour
IOB 4.9 Actively seeks and accepts feedback to improve own
performance

Assessment and Evaluation IOB 5.1 Complies with Operator/ATOs and Authority
requirements
Assesses the competencies of IOB 5.2 Ensures that the trainee understands the assessment
the trainee process
IOB 5.3 Applies the competency standards and conditions
IOB 5.4 Assesses trainee’s competencies
IOB 5.5 Performs grading
IOB 5.6 Provides recommendations based on the outcome of
the assessment
IOB 5.7 Makes decisions based on the outcome of the
summative assessment
IOB 5.8 Provides clear feedback to the trainee
and
Contributes to continuous training IOB 5.9 Reports strengths and weaknesses of the training
system improvement system (e.g., training environment, curriculum,
assessment/evaluation) including feedback from trainees
IOB 5.10 Suggests improvements for the training system
IOB 5.11 Produces reports using appropriate forms and media

38
Annex 3. IATA ACTF TEM-based Accident Classification Taxonomy
END STATES (unrecoverable)
S01 CFIT
S02 Loss of Control in-flight
S03 Runway Collision
S04 Mid-air Collision
S05 Runway/Taxiway Excursion
S05.01 RWY Excursion Overrun
S05.02 RWY Excursion Lateral
S05.03 TXY Excursion
S06 In-flight Damage
S07 Ground Damage
S08 Undershoot
S09 Hard Landing
S10 Gear Up Landing/ Gear Collapse
S11 Tail Strike
S12 Off Airport Landing/Ditching
S98.01 Deliberate Act – Security
S98.02 Deliberate Act – Suicide
S99 OTHER

UNDESIRED AIRCRAFT STATES (flight crew induced, recoverable)

U - Aircraft Handling
U01 Abrupt Aircraft Control
U02 Vertical, Lateral or Speed Deviations
U03 Unnecessary Weather Penetration
U04 Unauthorized Airspace Penetration
U05 Operation Outside Aircraft Limitations
U06 Unstable Approach
U07 Continued Landing after Unstable Approach
U08 Long, Floated, Bounced, Firm, Off centerline, Canted, Porpoised Landing
U09 Rejected Take-off after V1
U10 Controlled Flight Toward Terrain
U99 Other
V – Ground Navigation (Surface Nav)
V01 Proceeding towards wrong taxiway/ runway
V02 Wrong taxiway, ramp, gate or hold spot
V03 Runway/ taxiway incursion
V04 Ramp movements, including when under marshalling
V05 Loss of aircraft control while on the ground
V99 Other
W – Incorrect Aircraft Configurations
W01 Brakes, Thrust Reversers, Ground spoilers

39
W02 Systems (Fuel, Elec, Hydraulic, Pnem, A/C, Press, Inst)
W03 Landing Gear
W04 Flight Controls/ Automation
W05 Engine
W06 Weight & Balance
W99 Other

ERRORS (flight crew deviation)

H - Aircraft Handling Errors
H01 Manual handling/Flight Controls
H02 Ground Navigation (Surface nav)
H03 Automation (settings/selections)
H04 Systems/Radio/Instruments (settings/selections)
H99 Other
P – Procedural Errors
P01 SOP adherence/ cross-verification (see breakdown)
P01.01 Intentional
P01.02 Unintentional
P01.03 Unknown
P02 Checklist (see breakdown)
P02.01 Normal checklist (error)
P02.02 Abnormal checklist (error)
P03 Callouts
P04 Briefings
P05 Documentation (see breakdown)
P05.01 Incorrect weight and balance/ fuel information
P05.02 Incorrect ATIS/ clearance
P05.03 Misinterpreted items on paperwork
P05.04 Incorrect or missing log book entries
P06 Failure to Go-Around
P06.1 Failure to go-around after destabilization on approach
P06.2 Failure to go-around after a bounced landing
P99 Other
C – Communication Errors
C01 Crew to External communication
C01.01 With ATC
C01.02 With cabin crew
C01.03 With ground crew
C01.04 With Dispatch
C01.05 With Maintenance
C02 Pilot to Pilot

40
THREATS (occurs outside the influence of the flight crew)
E - Environmental Threats
E01 Meteorology (see breakdown)
E01.01 Thunderstorm
E01.02 Poor Visibility/IMC
E01.03 Gusty wind/ windshear
E01.04 Icing conditions
E01.05 Hail
E02 Lack of Visual Reference
E03 Air Traffic Services
E04 Birds/foreign objects
E04.01 Birds
E04.02 Wildlife
E04.03 Foreign objects
E05 Airport Facilities (see breakdown)
E05.01 Poor signage/lighting, faint markings, rwy/txy closures
E05.02 Contaminated runways, taxiways, poor braking action
E05.03 Trenches, ditches, intruding structures
E05.04 Airport perimeter control/fencing / Wildlife control
E06 Navaids (see breakdown)
E06.01 Malfunction, lack, or unavailable
E06.02 Uncalibrated
E07 Terrain/Obstacles
E08 Traffic
E08.01 Aircraft
E08.02 Vehicle
E09 RWY Surface Incursion
E09.01 Aircraft
E09.02 Vehicle
E09.03 Wildlife
E09.04 Other
E99 Other

A - Airline Threats
A01 Aircraft Malfunction (see breakdown)
A01.01 Uncontained engine failure
A01.02 Contained engine failure (incl overheat and prop fail)
A01.03 Landing gear/ tires
A01.04 Brakes
A01.05 Flight Controls (see breakdown)
A01.05.01 Primary flight controls
A01.05.02 Secondary flight controls (flaps, spoilers)
A01.06 Structural Failure
A01.07 Fire/Smoke
A01.08 Avionics, flight instruments
A01.09 Autopilot/ FMS

41
A01.10 Hydraulic system failure
A01.11 Electrical power/ generation failure
A01.99 Other
A02 MEL item
A03 Operation pressure
A04 Cabin events
A05 Ground events
A06 Dispatch/paperwork
A07 Maintenance events
A08 Dangerous goods
A09 Manual/charts/checklists
A99 Other

B - Psychological/Physiological Threats
B01 – Fatigue
B02 – Optical illusion/visual mis-perception
B03 – Spatial disorientation & spatial/somatogravic illusion
B04 – Crew Incapacitation

LATENT CONDITIONS (present in system before accident)

O01 Design (design shortcomings and defects)
O02 Regulatory Oversight
O03 Management Decisions (cost cutting)
O04 Safety Management (absent or deficient)
O05 Change Management (deficiencies in monitoring change)
O06 Selection Systems (deficient selection standards)
O07 Ops Planning & Scheduling (deficiencies in crew rostering, flight time limits)
O08 Technology & Equipment (available safety equip not installed)
O09 Flight Operations (see breakdown)
O09.01 SOPs & Checking
O09.02 Training Systems
O10 Cabin Operations (see breakdown)
O10.01 SOPs & Checking
O10.02 Training Systems
O13 Ground Operations (see breakdown)
O13.01 SOPs & Checking
O13.02 Training Systems
O14 Maintenance Operations (see breakdown)
O14.01 SOPs & Checking
O14.02 Training Systems
O15 Dispatch (see breakdown)
O15.01 SOPs & Checking
O15.02 Training Systems
O16 Flight watch/following/support

42
O99 Other

FLIGHT CREW COUNTERMEASURES

L – Team Climate
L01 Communication Environment
L02 Leadership (see breakdown)
L02.01 Captain shows leadership and coordinates flight deck activities
L02.02 FO is assertive and able to take over as leader
L03 Overall crew performance
L99 Other
M – Planning
M01 SOP Planning
M02 Plans stated
M03 In flight decision making/contingency management
M03.01 - Pro-active: Inflight Decision Making
M03.02 – Re-active: Contingency Management
M99 Other
N – Execution
N01 Monitor/ Cross-check
N02 Workload management
N03 Automation Management
N04 Taxiway/ Runway management
N99 Other
R – Review/Modify
R01 Evaluation of Plans
R02 Inquiry
R99 Other

ADDITIONAL CLASSIFICATION
I Insufficient Data
Y Incapacitation

43
Annex 4. CBTA Instructor/Evaluator initial standardization
The CBTA IE initial standardization program comprises
 CBTA IE training, and
 CBTA assessment of competence.

CBTA IE training
The CBTA IE training course should be delivered by a qualified CBTA IE.
The CBTA IE training course should comprise both theoretical and practical training.
At the completion of CBTA IE training, the applicant CBTA IE should:

(1) have knowledge of CBTA, including the following underlying principles:

 threat and error management,
 CBTA,
 learning from positive performance,
 building resilience, and
 data-driven training.

(2) demonstrate knowledge of Instructional System Design, the structure and the method of training
delivery for each phase of the AOC/ATO CBTA program;
(3) demonstrate knowledge of the principles of adult learning and how they relate to CBTA;
(4) conduct objective observations based on a competency framework, and document evidence of
observed performance;
(5) relate specific performance observations of competencies;
(6) analyze trainee performance to determine competency-based training needs and recognize
strengths;
(7) evaluate performance using the competency-based grading system;
(8) apply appropriate teaching styles during training to accommodate trainee learning needs;
(9) facilitate trainee learning, focusing on specific competency-based training needs; and
(10) conduct a debrief using facilitation techniques.

An IE may be given credit for parts of the above if the IE has previously demonstrated competence in
those topics.

44
CBTA assessment of competence
Prior to delivering CBTA, the IE should undergo an assessment of competence, conducted during a
practical CBTA session by a person nominated by the AOC/ATO and acceptable to the Licensing
Authority.

CBTA Instructor/Evaluator – Recurrent standardization

The CBTA IE should complete annual (or at a specific interval approved by the authority) recurrent
standardization comprising
 Refresher CBTA training to develop the IE’s competence to conduct CBTA; and
 Concordance training

Note: “concordance” means inter-rater-reliability . It is the consistency or stability of scores between

different CBTA IE; it gives a score (or scores) of how much homogeneity, or consensus, there is in the
ratings given by IEs (raters).

Recurrent standardization should incorporate de-identified grading data to show where grading is
consistent or where there is inconsistency. Use of example scenarios that demonstrate appropriate
grading have proven to be helpful in calibrating the IE workforce. Providing individual IE grading data in
comparison to the entire population of IE can also be a useful tool to help individual instructors see how
they perform compared to their peers.

The standardization could also incorporate feedback received from pilots that received CBTA and a
review of relevant inter-rater reliability data.

At regular intervals not to exceed three years, the IE should undergo a CBTA assessment of competence,
conducted during the delivery of a practical CBTA session.

Instructor Concordance Assurance Program (ICAP)

STANDARDISATION OF CBTA INSTRUCTORS — ACCEPTABLE INSTRUCTOR CONCORDANCE
The authority may require a minimum acceptable level of concordance. This may be a non-exhaustive list:
 Set a minimum acceptable level of concordance per aircraft fleet or by group of instructors.
 Set a minimum acceptable level of concordance per competency.
 Set a minimum acceptable level of concordance for all operators under its oversight, or a minimum
acceptable level of concordance per operator (or type of operator) based on the risk of the
operator.

45
Annex 5. Authority CBTA Inspectors training and qualification example
from EASA EBT regulation

QUALIFICATION AND TRAINING — INSPECTORS

(a) For the initial approval and oversight of an operator’s EBT programme, the inspector of the
competent authority should undertake EBT training as part of their required technical training.
At the conclusion of the inspector training, the inspector should:
(1) know the principles of EBT, including the following underlying principles:
(i) competency-based training;
(ii) learning from positive performance;
(iii) building resilience; and;
(iv) data-driven training;
(2) know the structure of an EBT module;
(3) know the method of training delivery for each phase of an EBT module;
(4) know the principles of adult learning and how they relate to EBT;
(5) recognise effective observations based on a competency framework, and document evidence
of observed performance;
(6) recognise and relate specific performance observations of competencies;
(7) recognise trainee performance to determine competency-based training needs and recognise
strengths;
(8) understand methods for the evaluation of performance using a competency-based grading
system;
(9) recognise appropriate teaching styles during simulator training to accommodate trainee
learning needs;
(10) recognise facilitated trainee learning, focusing on specific competency-based training needs;
and
(11) understand how to conduct a debrief using facilitation techniques.

(b) The objective of such training is to ensure that the inspector:

(1) attains the adequate level of knowledge in the principles of approval and oversight of the EBT
programmes; and

(2) acquires the ability to recognise the EBT programme suitability.

46
Annex 6. Data protection example from EASA EBT regulation

DATA PROTECTION 1
a) The objective of protecting the EBT data is to avoid inappropriate use of it in order to ensure the
continued availability of such data, to maintain and improve pilot competencies.

b) The data access and security policy should restrict information access to authorised persons.

c) The data access and security policy should include the measures to ensure the security of the data
(e.g. information security standard).

d) The data access and security policy (including the procedure to prevent disclosure of crew identity)
should be agreed by all parties involved (airline management and flight crew member representatives
nominated either by the union or the flight crew themselves).

e) The data access and security policy should be in line with the organisation safety policy in order to
not make available or to not make use of the EBT data to attribute blame or liability.

f) The operator may integrate the security policy within other management systems already in place
(e.g. information security management).

DATA PROTECTION 2
(a) The data access and security policy may, as a minimum, define:

(1) a policy for access to information only to specifically authorised persons identified by their
position in order to perform their duties. The required authorised person(s) does (do) not need to
be the EBT manager; it could be the EBT programme manager or a third party mutually acceptable
to unions or staff and management. The third party may also be in charge of ensuring the correct
application of the data access and security policy (e.g. the third party is the one activating the
system to allow access to the authorised persons);

(2) the identified data retention policy and accountability;

(3) the measures to ensure that the security of the data includes the information security standard
(e.g. information security management systems standard e.g. ISO 2700x-ISO 27001, NIST SP 800-
53, etc.);

(4) the method to obtain de-identified crew feedback on those occasions that require specific follow-
up; and

(b) When there is a need for data protection, it is preferable to de-identify the data rather than
anonymise it.

47