Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
SlideShare a Scribd company logo

Ch 7

•Download as PPTX, PDF•
•
Saumil Shah
Saumil Shah

The document discusses factors that determine how long a test run should be for optimizing a landing page, including: 1) Data rate, or number of conversions per day, which affects how quickly data is collected during the test. 2) Size of improvements found, which could be significant for obvious problems or vary depending on unknowns. 3) Size of the test, measured by the search space of alternative designs, which increases exponentially with additional elements tested. 4) Potential for variable interactions, where settings of one variable positively or negatively influence others.

1 of 12
Ch – 7 The Math Of Tuning
How Long Should My Test Run??? • The data rate (number of conversions per day) • Size of improvements found (Percentage Improvement) • Size of your test (Number of alternative designs) • The confidence in your answer ( How sure you need to be )
Data Rate • The data rate decribes how quickly you collect data during your test. • The volume of traffic for landing page optimization tests is best measured in the number of conversion actions per day ( and not the number of unique visitors). • You can decrease the size of the test. This can be done by decreasing the granularity of your test elements.
Size of Improvements Found • If you have managed to uncover a clearly superior version of your landing page, the performance improvement would quickly become apparent. • often, an initial round of changes will fix some of the obvious problems and improve performance significantly. • Since you do not know the size of the possible improvements ahead of tme, the length of the time required for the tuning test may vary signifantly.
Size of your Test • The size of your test can be measured by the size of the search space that you are considering. • The search space is the whole universe of the alternative designs possible in your test. • A simple head to head test has a search space size of 2 ( the originl, and the alternative landing page version that you are testing.) • If you are testing multiple elements on the page, you need to multiply together the number of alternative versions for each one.
Variable Interactions • When the setting for one variable in your test positively or negatively influences the setting of another variable is variable interaction. • If they have no effect then they are said to be independent. • In a positive interaction, 2 variables create a synergistic effect. • In a negative interaction, 2 variables undercut each other and cancel out some of the individual effects.
Ch 7
Ch 7
Ch 7
Ch 7
Ch 7
Ch 7

More Related Content

Ch 7

  • 1. Ch – 7 The Math Of Tuning
  • 2. How Long Should My Test Run??? • The data rate (number of conversions per day) • Size of improvements found (Percentage Improvement) • Size of your test (Number of alternative designs) • The confidence in your answer ( How sure you need to be )
  • 3. Data Rate • The data rate decribes how quickly you collect data during your test. • The volume of traffic for landing page optimization tests is best measured in the number of conversion actions per day ( and not the number of unique visitors). • You can decrease the size of the test. This can be done by decreasing the granularity of your test elements.
  • 4. Size of Improvements Found • If you have managed to uncover a clearly superior version of your landing page, the performance improvement would quickly become apparent. • often, an initial round of changes will fix some of the obvious problems and improve performance significantly. • Since you do not know the size of the possible improvements ahead of tme, the length of the time required for the tuning test may vary signifantly.
  • 5. Size of your Test • The size of your test can be measured by the size of the search space that you are considering. • The search space is the whole universe of the alternative designs possible in your test. • A simple head to head test has a search space size of 2 ( the originl, and the alternative landing page version that you are testing.) • If you are testing multiple elements on the page, you need to multiply together the number of alternative versions for each one.
  • 6. Variable Interactions • When the setting for one variable in your test positively or negatively influences the setting of another variable is variable interaction. • If they have no effect then they are said to be independent. • In a positive interaction, 2 variables create a synergistic effect. • In a negative interaction, 2 variables undercut each other and cancel out some of the individual effects.