Study Design I:

Experimental Design
The process of science

Observational
or
experimental
study
Generalities of study design
• Having an adequate sample size is critical both
experiment and observational studies
• To few samples and you will be unlikely to detect real
treatment effects
• However, there is no point in including more samples
than needed, as they are expensive
 Experiments vs. Observational
Studies
• In biology, two types of investigations are carried out:
observational and experimental
• Observational studies

__________________________________
• Experimental studies can identify

__________________________________
 Experiments vs. Observational
Studies
• In an observational study, investigators observe
subjects and measure variables of interest without
assigning treatments to the subjects
• The treatment that each subject receives is
determined beyond the control of the investigator
 Experiments vs. Observational
Studies
• Observational Study Example:
• Find 100 women age 30 of which 50 have been smoking a
pack a day for 10 years while the other 50 have been smoke
free for 10 years
• Measure lung capacity for each of the 100 women
• Analyze, interpret, and draw conclusions from data
 Experiments vs. Observational
Studies
• In an experiment, investigators apply treatments to
experimental units (people, animals, plots of land, etc.)
and then proceed to observe the effect of the
treatments on the experimental units
• In an experimental study, there must be at least two
treatments and the experimenters (rather than nature) must
assign them to units or subjects
 Experiments vs. Observational
Studies
• Experimental Example:
• Find 100 women age 20 who do not currently smoke
• Randomly assign 50 of the 100 women to the smoking
treatment and the other 50 to the no smoking treatment
 Experiments vs. Observational
Studies
• Experimental Example:
• Subjects receive regular checkups in order to determine
their smoking status, lifestyle, and health issues
• After 10 years measure lung capacity for each of the 100
women
• Analyze, interpret, and draw conclusions from data
 Experiments vs. Observational
Studies
• So what's the difference?
• In our observational study example, we could not account
for several issues:
• Selection bias
• Confounding variables
• Experimental artifact
Selection bias
• Selection bias
_________________________________________________
_________________________________________________
_________________________________________________
• Many smokers are of lower socioeconomic groups, thus may suffer
from health problems regardless of being smokers
Confounding variables
• Confounding variables – are variables that mask or
distort the relationship between measured
variables in a study
Confounding variables
• e.g. taking supplemental oxygen might increase the
chance of survival during a climb of Mount Everest
• However, what if climbers who carry oxygen are
just better prepared
Confounding variables
• Being prepared is really what increases survival, not
oxygen
• Its just that those who carry oxygen are more likely
to survive
Confounding variables
• Preparedness distorts the causal relationship
between the measured relationship (oxygen and
survival) and thus a confounding variable
Confounding variables
• In a properly designed experiment oxygen would be
assigned at random and thus irrespective of
preparedness
Confounding variables

Oxygen No oxygen
Prepared Oxygen/Prepared No oxygen/prepared
Unprepared Oxygen/Unprepared No oxygen/unprepared
Experimental artifacts
• An experimental artifact is a bias in a
measurement produced by unintended
consequences of experimental procedures
Experimental artifacts
• The diving reflex optimizes respiration by
preferentially distributing oxygen stores to the
heart and brain which allows staying underwater
for extended periods of time
Experimental artifacts
• The most noticeable effects are on the
cardiovascular system, e.g. slowed heart rate
• This is seen in many mammals and was suggested
to be also present in birds
• Specifically, diving birds were hypothesized to reduce
heart rate when they dive
Experimental artifacts
• To test this hypothesis, researchers forcibly dunked
birds underwater while measuring their heartrate
Experimental artifacts
• This was refuted when researchers monitored birds
diving during normal foraging attempts
• Heart rate did not lower during natural diving
events
Experimental artifacts
• The act of forcing the birds underwater created an
artifact which resembled the mammalian diving
reflex
Experimental artifacts
• To prevent experimental artifacts, researchers
should make conditions as natural as possible
• However, more natural conditions might introduce
more sources of variation
• ________________________
• ________________________
• In many cases, observational studies can provide
important insight into which is the best setting for
an experiment
What make a good experiment?
• A good experiment is designed with two objective
in mind:
• ______________________________________________

• ______________________________________________
Reducing bias
• To reduce bias, the experiment should include the
following elements:
• A control group – a group of subject who do not receive
the treatment of interest but who otherwise experience
similar conditions as the treated subjects
Reducing bias
• To reduce bias, the experiment should include the
following elements:
• Randomization – the random assignment of treatments
to units in an experimental study
Reducing bias
• To reduce bias, the experiment should include the
following elements:
• Randomization breaks the association between possible
confounding variables and the explanatory variable,
allowing the causal relationship between the
explanatory and response variables to be assessed
Reducing bias
• To reduce bias, the experiment should include the
following elements:
• Blinding – concealing information from the participants
and the researchers about which of them receive which
treatments
Reducing bias
• To reduce bias, the experiment should include the
following elements:
• Blinding prevents researchers and participants from
changing their behavior, consciously or unconsciously,
based on their knowledge of which treatment they were
receiving or administering
Reducing bias
• To reduce bias, the experiment should include the
following elements:
• In a single-blind experiment, participants are unaware
of the treatment they have been assigned
Reducing bias
• To reduce bias, the experiment should include the
following elements:
• In a double-blind experiment, the researchers
administering the treatments and measuring the
response are also unaware of which subject are
receiving which treatments
Reducing sampling error
• The difference between the sample and population
parameter values is called sampling error
• Sampling error can obscure the detection of real
effects
Reducing sampling error
Small samples can result in sampling error
Sample
Population Green 80%
Red 20%

Green 50%
Red 50%
Reducing sampling error
• Replication is the repetition of an experimental
condition so that the variability associated with the
phenomenon can be estimated
• Increasing replication reduces sampling error
Reducing sampling error
• However, replication can mean several things
• It can mean the number of individuals used
• It may also mean the experimental unit which
treatments are assigned
 Fertilizer 1 =

Fertilizer 2 = Replication
• Consider an
Zero replicates
experiment looking
at the affects of two
Eight pots
Eight replicates
fertilizers on plant
growth
• Four experimental
designs are
Zero replicates
considered

Two chambers
Two replicates
 Fertilizer 1 =

Fertilizer 2 = Replication
• The first design
Zero replicates
obviously has
_______________
Eight pots
Eight replicates
_______________
because it has only
two pots, one for
each fertilizer
Zero replicates treatment

Two chambers
Two replicates
 Fertilizer 1 =

Fertilizer 2 = Replication
• The second design
Zero replicates
involves growing
multiple plants, in
Eight pots each treatment
Eight replicates
together
• This _____________
• They share the same
environment and are
Zero replicates
thus comparable
• Each pot is the
replicate, 4 for each
fertilizer type
Two chambers
Two replicates
 Fertilizer 1 =

Fertilizer 2 = Replication
• The third design
Zero replicates involves using two
growth chambers,
Eight pots
one for each
Eight replicates fertilizer type
• This design
_______________
• This is because
Zero replicates
each of the two
fertilizer types
were confided to
two growth
chamber
Two chambers
Two replicates
 Fertilizer 1 =

Fertilizer 2 = Replication
• The final design
Zero replicates
involves using two
growth chambers,
Eight pots
Eight replicates
but having each
fertilizer
represented in
each chamber
Zero replicates • This design
_______________
• This is called a
blocked design
Two chambers
Two replicates
Reducing sampling error
• Blocking - arranging of experimental units in groups
(blocks) that are similar to one another
• Blocking can account for extraneous variation
Reducing sampling error
• A __________________________–experimental
units into multiple groups
• Within each block, treatments are randomly
assigned to experimental units
Reducing sampling error
• Differences between treatments are evaluated only
within blocks
• Much of the variation arising from differences
between blocks is accounted for
Reducing sampling error
• In other words, if something is atypical about one
block, the addition of the other blocks will reduce
its influence
Reducing sampling error
The Randomized Block Design
Reducing sampling error
• Another approach to reducing sampling error is to
apply a balanced design, where all treatments have
equal sample size
Experiments with more than one
factor
• Up until now, we have considered only experiments
that have a single treatment variable (aka a factor)
• However, many experiments investigate more than
one factor
Experiments with more than one
factor
• In a factorial design, every combination of
treatment from two (or more) treatment variables
is investigated
Experiments with more than one
factor
• Notice in the example below that both levels of the
noise level factor have a corresponding set of
individuals in all levels of the age group factor
Experiments with more than one
factor
• An advantage of factorial designs is that they can
asses interactions very well
• Interaction means that the effect of one variable
depends upon the state of the other variable
Experiments with more than one
factor
• Interactions can be visualized by plotting one factor
on the x axis and the dependent variable on the y
axis
• The 2nd factors levels are shown as lines
Dependent Variable

Factor 2 Level A
Factor 2 Level B

A B
Factor 1 (has 2 levels)
Experiments with more than one
factor
• Recall that interaction means that the effect of one variable
depends upon the state of the other variable
• This example shows no interaction because the lines do not
cross
Dependent Variable

Factor 2 Level A
Factor 2 Level B

A B
Factor 1 (has 2 levels)
Experiments with more than one
factor
• This example does show interaction because the
lines do cross
Dependent Variable

Factor 2 Level B

Factor 2 Level A

A B
Factor 1 (has 2 levels)