Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Scribd Logo
Upload

Welcome to Scribd!

  • 58 views

    Bombers and Seagulls

    Uploaded by

    hpgerald
    The document discusses binary classification and calculating the area under the ROC curve. It includes graphs of true positive rate vs false positive rate and cost per false negative vs threshold. The optimal threshold is determined by comparing the cost per false negative to the cost per false positive.

    Copyright:

    © All Rights Reserved
    Download as XLSX, PDF, TXT or read online from Scribd

    Bombers and Seagulls

    Uploaded by

    hpgerald
    58 views8 pages
    The document discusses binary classification and calculating the area under the ROC curve. It includes graphs of true positive rate vs false positive rate and cost per false negative vs threshold. The optimal threshold is determined by comparing the cost per false negative to the cost per false positive.

    Original Description:

    Excel data for the binary classification model

    Original Title

    Bombers-And-Seagulls

    Copyright

    © © All Rights Reserved

    Available Formats

    XLSX, PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    The document discusses binary classification and calculating the area under the ROC curve. It includes graphs of true positive rate vs false positive rate and cost per false negative vs threshold. The optimal threshold is determined by comparing the cost per false negative to the cost per false positive.

    Copyright:

    © All Rights Reserved
    Download as XLSX, PDF, TXT or read online from Scribd
    Download as xlsx, pdf, or txt
    58 views8 pages

    Bombers and Seagulls

    Uploaded by

    hpgerald
    The document discusses binary classification and calculating the area under the ROC curve. It includes graphs of true positive rate vs false positive rate and cost per false negative vs threshold. The optimal threshold is determined by comparing the cost per false negative to the cost per false positive.

    Copyright:

    © All Rights Reserved
    Download as XLSX, PDF, TXT or read online from Scribd
    Download as xlsx, pdf, or txt
    of 8

    Example as animated in Binary Classification Video 1

    Bombers in red, Seagulls in black

    Part 1
    Score False Positive Rate
    0.00
    97 0.06
    93 0.06
    90 0.13
    86 0.19
    83 0.19
    80 0.25
    77 0.31
    75 0.31
    74 0.38
    70 0.44
    66 0.44
    62 0.50
    60 0.56
    55 0.63
    50 0.69
    44 0.75
    39 0.81
    33 0.88
    20 0.94
    11 1.00

    Part 3
    Binary Outcomes
    Score "+" = 1, "-" = 0
    97 0
    93 1
    90 0
    86 0
    83 1
    80 0
    77 0
    75 1
    74 0
    70 0
    66 1
    62 0
    60 0
    55 0
    50 0
    44 0
    39 0
    33 0
    20 0
    11 0
    total "+" - Bombers
    4

    Copyright Daniel Egger/ Attribution 4.0 International (CC BY 4.0)


    Part 2
    True Positive Rate Calculating the Area Under the C
    0.00 x-axis distance
    0.00 0.06
    0.25 0.00
    0.25 0.06
    0.25 0.06
    0.50 0.00
    0.50 0.06
    0.50 0.06
    0.75 0.00
    0.75 0.06
    0.75 0.06
    1.00 0.00
    1.00 0.06
    1.00 0.06
    1.00 0.06
    1.00 0.06
    1.00 0.06
    1.00 0.06
    1.00 0.06
    1.00 0.06
    1.00 0.06

    binary outcomes - 0,1 Total at threshold


    reversed for ease of calculation False Positives True Positives
    0 0
    1 1 0
    0 1 1
    1 2 1
    1 3 1
    0 3 2
    1 4 2
    1 5 2
    0 5 3
    1 6 3
    1 7 3
    0 7 4
    1 8 4
    1 9 4
    1 10 4
    1 11 4
    1 12 4
    1 13 4
    1 14 4
    1 15 4
    1 16 4
    total "-" Seagulls total events
    16 20

    Attribution 4.0 International (CC BY 4.0)


    ng the Area Under the Curve as a Series of Rectangles
    average of y values rectangle
    1.00
    0 0.0000
    0.13 0.0000 0.90

    0.25 0.0156
    0.80
    0.25 0.0156
    0.70
    0.38 0.0000
    0.50 0.0313 0.60

    0.50 0.0313
    0.50
    0.63 0.0000
    0.40
    0.75 0.047
    0.75 0.047 0.30

    0.875 0.000 0.20


    1 0.063
    0.10
    1 0.063
    1 0.063 0.00
    0.00 0.10 0.20 0.30 0.40 0.50 0.60 0.70
    1 0.063
    1 0.063
    1 0.063
    1 0.063
    1 0.063
    1 0.063

    0.750
    Part 4 - Cost Function
    Cost per FN £50,000,000
    FP rate = x TP rate = y number FNs product
    0 0 4 £200,000,000
    0.06 0.00 4 £200,000,000
    0.06 0.25 3 £150,000,000
    0.13 0.25 3 £150,000,000
    0.19 0.25 3 £150,000,000
    0.19 0.50 2 £100,000,000
    0.25 0.50 2 £100,000,000
    0.31 0.50 2 £100,000,000
    0.31 0.75 1 £50,000,000
    0.38 0.75 1 £50,000,000
    0.44 0.75 1 £50,000,000
    0.44 1.00 0 £0
    0.50 1.00 0 £0
    0.56 1.00 0 £0
    0.63 1.00 0 £0
    0.69 1.00 0 £0
    0.75 1.00 0 £0
    0.81 1.00 0 £0
    0.88 1.00 0 £0
    0.94 1.00 0 £0
    1.00 1.00 0 £0

    Note: minimum cost threshold is dete


    (cost per FN)/(cost per FP)
    ROC Curve

    Column E

    0.40 0.50 0.60 0.70 0.80 0.90 1.00

    Cost per FP £4,000,000


    number FPs product Total Cost Cost per event
    0 £0 £200,000,000 £10,000,000
    1 £4,000,000 £204,000,000 £10,200,000
    1 £4,000,000 £154,000,000 £7,700,000
    2 £8,000,000 £158,000,000 £7,900,000
    3 £12,000,000 £162,000,000 £8,100,000
    3 £12,000,000 £112,000,000 £5,600,000
    4 £16,000,000 £116,000,000 £5,800,000
    5 £20,000,000 £120,000,000 £6,000,000
    5 £20,000,000 £70,000,000 £3,500,000
    6 £24,000,000 £74,000,000 £3,700,000
    7 £28,000,000 £78,000,000 £3,900,000
    7 £28,000,000 £28,000,000 £1,400,000
    8 £32,000,000 £32,000,000 £1,600,000
    9 £36,000,000 £36,000,000 £1,800,000
    10 £40,000,000 £40,000,000 £2,000,000
    11 £44,000,000 £44,000,000 £2,200,000
    12 £48,000,000 £48,000,000 £2,400,000
    13 £52,000,000 £52,000,000 £2,600,000
    14 £56,000,000 £56,000,000 £2,800,000
    15 £60,000,000 £60,000,000 £3,000,000
    16 £64,000,000 £64,000,000 £3,200,000
    Minimum per event
    £28,000,000 £1,400,000

    inimum cost threshold is determined by "cost function ratio)


    r FN)/(cost per FP) here: 12.5

    You might also like