Reducing rate of false FDM events

EOFDM conference – Köln – 23/01/2013

Edouard Garnier de Labareyre Thierry Donadey

FDM System Engineer - Sagem Flight Data Analysis Expert – Air France
Context of this project

 Internal algorithms refactoring at Air France

 Current observation : difficulties to have reliable statistics which are representative
of the reality
 Complete re-engineering of FDM procedures in progress

 What is Svetlana ?
 E.U. – Russian project with Sagem (2010-2012): Improving flight data analysis with
new techniques such as data-mining
 One of the topic of the project was the improvement of existing FDM process

The research leading to these results has received funding from the European
Union Seventh Framework Programme (FP7/2007-2013) under grant agreement n
ACPO-GA-2010-265940 SVETLANA.

1 / EOFDM CONFERENCE / JANUARY 23 - 2013

Context of this project

 Air France – Sagem collaboration within Svetlana

 Sharing of skills:

operational FDM experience + software expertise

 Improve analysis results quality as well as reusability

 Promote documented procedures
 Validate a workflow for high quality procedures development

2 / EOFDM CONFERENCE / JANUARY 23 - 2013

Summary

1) Understanding the causes of false events

2) Detecting and correcting invalid parameters

3) Using parameter invalidity information

4) Results for Air France fleet

5) Conclusion

3/ EOFDM CONFERENCE / JANUARY 23 - 2013

 Understanding the causes
of false events

Examples

4/ EOFDM CONFERENCE / JANUARY 23 - 2013

False “SevereTurbulence” (VRTG involved)

5/ EOFDM CONFERENCE / JANUARY 23 - 2013

False “Speed low” (CAS involved)

6/ EOFDM CONFERENCE / JANUARY 23 - 2013

False “Go around” (ALT STD involved)

7/ EOFDM CONFERENCE / JANUARY 23 - 2013

Identification of responsibilities in false event detection

“A chain is only as strong as its weakest link”

Weak detection algorithm Bad input data

 Ifthe algorithm is too dependant on  Inputdata can be noisy, which often
a single parameter, or not covering cause a simple algorithm to fail
enough from possible situations, it
will fail in any unexpected situation

 Willnot be developed in this  Two approaches

presentation, but is also part of the  filter data in each event algorithm
project
 filter data once and for all

8/ EOFDM CONFERENCE / JANUARY 23 - 2013

 Detecting and correcting
invalid parameters

Theory

9/ EOFDM CONFERENCE / JANUARY 23 - 2013

Looking for patterns

Air speed Altitude Vertical acceleration

Roll

 Peaks, isolated or grouped

 Value gaps

 Parameter out of normal range

10 / EOFDM CONFERENCE / JANUARY 23 - 2013

Correction is not enough

?
?
? or

 It is often not possible or not reasonable to correct the parameter value.

 We cannot honestly tell that a parameter is “corrected” if we are not confident

in the correction that has been applied

11 / EOFDM CONFERENCE / JANUARY 23 - 2013

Parameter invalidation

 When no correction is possible, the parameter should be invalidated

 What an invalidation means:

“This parameter should not be used during this period, because it was not
possible to correct it with a sufficient level of confidence. ”

 Need to attach a boolean “validity” parameter to each parameter we want to

monitor
12 / EOFDM CONFERENCE / JANUARY 23 - 2013
Implementation

Peak
Correction
detection parameter corrected
(9-median
filter)
Gap
parameter
detection

Out of Global
invalidation parameter validity
range
detection VALID

INVALID

13 / EOFDM CONFERENCE / JANUARY 23 - 2013

Median Filter

 Input 9 elements

5 9 3 1000 15 11 7 17 13
 Sort

3 5 7 9 11 13 15 17 1000
 Result is the element in median position

11

14 / EOFDM CONFERENCE / JANUARY 23 - 2013

 Using parameter invalidity
information

In practice

15 / EOFDM CONFERENCE / JANUARY 23 - 2013

Impact on existing algorithms

 Any algorithm can start using parameter invalidation and correction by

introducing a simple test on parameter invalidation

BEFORE AFTER
IF ( CAS > limit ) THEN IF ( CAS_VALID ) THEN
EVENTV(…) IF ( CAS_CORRECTED > limit ) THEN
ENDIF EVENTV(…)
ENDIF
ENDIF

 But this could also be used in other powerful ways

 Behave in degraded mode if one of the parameters needed for the computation is
invalid
 Use it for additional parameter computation
 Anything else you could imagine that uses PARAM_VALID and
PARAM_CORRECTED !

16 / EOFDM CONFERENCE / JANUARY 23 - 2013

The invalidation paradox

 If we stop here, we’re still missing something…

ROLL

PITCH

Chances are strong that this flight doesn’t raise any event about parameter PITCH or
ROLL as it’s quite always invalid ! Yet this flight deserves to be seen.

 Need a maintenance event that monitors the invalidity ratio on the whole
flight, for each parameter which goes through invalidation & correction
algorithms.
cas_invalidity_ratio = TOTAL_INVALID_SAMPLES / TOTAL_SAMPLES
IF ( cas_invalidity_ratio > 0.25) THEN
EVENTV(…)
ENDIF

17 / EOFDM CONFERENCE / JANUARY 23 - 2013

Improving FDM workflow

 Having this new maintenance oriented event on parameter validity reinforces

the maintenance retroaction loop of the FDM process

Classical FDM loop (Safety)

Maintenance loop

FDM
Pilot Aircraft Recorder
tool

18 / EOFDM CONFERENCE / JANUARY 23 - 2013

 Results on Air France fleet
( B747 & A340 )

Earnings

19 / EOFDM CONFERENCE / JANUARY 23 - 2013

Overview of Air France fleet

 AF Fleet
 256 aircrafts
 820 flights per day

Aircraft
Fleet Flight / year
Number

A320 146 240 162

A330 15 10 353
A340 13 7 141
A380 8 3 786
B747 10 4 772
B777 64 32 226

20 / EOFDM CONFERENCE / JANUARY 23 - 2013

« Shift » definition

 Parameter apparently correct: analysis possible with human eye

 Zoom on this parameter show some shifts: analysis not possible for IT system

21 / EOFDM CONFERENCE / JANUARY 23 - 2013

Results for AF B747 fleet

 Replay of B747 flights for year 2011

 Non corrected and corrected parameter shift number measuring.

 Shift threshold :
- CAS : 30kts
- ALTST : 300ft
- PITCH : 10
- ROLL : 10

Shift number on Shift number on

Parameter %
input param corrected param
CAS 38 797 45 99.9
ALTST 297 568 2 110 99.3
PITCH 1 086 57 94.8
ROLL 919 120 86.9

22 / EOFDM CONFERENCE / JANUARY 23 - 2013

Results for AF A340 fleet

 Replay of A340 flights for year 2011

 Non corrected and corrected parameter shift number measuring.

 Shift threshold :
- CAS : 30kts
- ALTST : 300ft
- PITCH : 10
- ROLL : 10

Shift number on Shift number on

Parameter %
input param corrected param
CAS 15 085 2 773 81.6
ALTST 127 198 1 682 98.7
PITCH 35 174 0 100.0
ROLL 36 644 83 99.8

23 / EOFDM CONFERENCE / JANUARY 23 - 2013

Parameter correction with median filter

24 / EOFDM CONFERENCE / JANUARY 23 - 2013

Gap detection not robust enough

25 / EOFDM CONFERENCE / JANUARY 23 - 2013

Median fliter clipping

26 / EOFDM CONFERENCE / JANUARY 23 - 2013

Conclusion

 Successful and very rich cooperation between two types of expertise

A promising start, still some things to improve

 Dealing with the parameters which don’t support well the 9-median filter (e.g.
PITCH & ROLL) with adaptive median filter
 Retrofit of the whole event detection set to use parameter invalidity
 Improvement of some event detection algorithms
 Covering the whole event set

 Paves the way to a common set of procedures in the AGS

 This work also led to the development of a methodology

27 / EOFDM CONFERENCE / JANUARY 23 - 2013