Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Scribd Logo
Upload

Welcome to Scribd!

  • 12 views

    ABTesting

    Uploaded by

    Vivek Jadhav
    This document discusses A/B testing and the process of experimentation. It covers topics like causation, modeling vs experimentation, examples of A/B tests, and outlines the typical testing process from establishing a hypothesis to gathering and analyzing results.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd

    ABTesting

    Uploaded by

    Vivek Jadhav
    12 views21 pages
    This document discusses A/B testing and the process of experimentation. It covers topics like causation, modeling vs experimentation, examples of A/B tests, and outlines the typical testing process from establishing a hypothesis to gathering and analyzing results.

    Original Title

    ABTesting (1)

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    This document discusses A/B testing and the process of experimentation. It covers topics like causation, modeling vs experimentation, examples of A/B tests, and outlines the typical testing process from establishing a hypothesis to gathering and analyzing results.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    12 views21 pages

    ABTesting

    Uploaded by

    Vivek Jadhav
    This document discusses A/B testing and the process of experimentation. It covers topics like causation, modeling vs experimentation, examples of A/B tests, and outlines the typical testing process from establishing a hypothesis to gathering and analyzing results.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    of 21

    A/B Testing

    Agenda

    • Causation
    • Modeling vs Experimentation
    • Examples
    • A/B Test
    • Testing Process
    • Summary
    Road not
    taken?
    Have you ever wondered where you might
    be today had you done something
    differently in the past?
    Causation
    Did social media marketing make revenue
    grow?
    Modeling vs
    Experimentation
    Modeling

    Advantages
    • Easy & cheap
    Disadvantages
    • Needs a lot of data
    • Uncertainty due to omitted variable
    bias
    Experimentation

    Advantages
    • Accurate
    Disadvantages
    • Comparatively expensive

    Media vs No Media
    How do we know if
    the new medicine Test
    works?

    EffectivenessDrug
    Control

    EffectivenessPlacebo

    Placebo Effect
    Experimentation
    Generic

    Personalized

    Experimentation
    Generic

    Personalized

    A/B Test
    40%

    90%
    Generic

    Personalized

    A/B Test
    Read Rate

    90%

    40%

    Generic Personalized

    Testing Process
    Test the effect of marketing email subject on read rate

    Hypothesis Establish Sample Size Randomize treatment


    Read rate with personalized and control
    subject = Read rate with Baseline, detectable effect, Customers
    generic subject power, significance level
    Read Rate

    90%

    40% Gather Data Track KPI


    Results Customer data, Subject , and Primary: Read rate
    Email response data Secondary: CTR, Revenue
    Generic Personalized

    This initial test will gauge the potential. If results are promising, multivariate tests can further estimate
    impact of subject variations. Once rich data is gathered in the future, predictive models can optimize
    email subjects.

    Testing Process
    Read Rate Results

    90%

    40%

    Generic Personalized

    This initial test will gauge the potential. If results are promising, multivariate tests can further estimate
    impact of subject variations. Once rich data is gathered in the future, predictive models can optimize
    email subjects.

    Testing Process
    Multivariate Testing
    Multivariate Testing
    Multivariate Testing
    Multivariate Testing
    Multivariate Testing

    https://vwo.com/blog/multivariate-testing/
    Takeaway
    • Causal Modeling
    • Experimentation
    • A/B Testing Process

    You might also like