Chapter 7

Learning

PowerPoint®
Presentation
by Jim Foley
Overview: Topics in this Chapter

 Definitions What do we mean

 Classical conditioning by “learning”?
 Operant conditioning Learning is the
 Biological and process of
cognitive components acquiring new
of learning and relatively
enduring
 Observational information or
learning behaviors.
How does learning happen other than
through language/words?
We learn from We learn by
experience: association:
1. when we learn to 1. when two stimuli
predict events we (events or sensations)
already like or don’t tend to occur together
like by noticing other or in sequence.
events or sensations 2. when actions become
that happen first. associated with
2. when our actions have pleasant or aversive
consequences. results.
3. when we watch what 3. when two pieces of
other people do. information are linked.
Associative and Cognitive Learning
 Associative Learning 
Classical Operant
conditioning: conditioning:
learning to link two changing
stimuli in a way that behavior choices
helps us anticipate in response to
an event to which consequences
we have a reaction

Cognitive learning: acquiring

new behaviors and
information through
observation and information,
rather than by direct
experience
 Operant and Classical Conditioning are
Different Forms of Associative Learning
Classical conditioning: Operant conditioning:
 involves respondent behavior,  involves operant behavior,
reflexive, automatic reactions chosen behaviors which
such as fear or craving “operate” on the environment
 these reactions to  these behaviors become
unconditioned stimuli (US) associated with consequences
become associated with which punish (decrease) or
neutral (thenconditioned) reinforce (increase) the
stimuli operant behavior
There is a contrast in the
process of conditioning.
The experimental (neutral) The experimental (consequence)
stimulus repeatedly precedes the stimulus repeatedly follows the
respondent behavior, and operant behavior, and eventually
eventually triggers that behavior. punishes or reinforces that
behavior.
Associative Learning
Classical Conditioning Stimulus 1: See
lightning
How it works: after repeated
exposure to two stimuli Stimulus 2: Hear
occurring in sequence, we thunder
associate those stimuli with each Here, our response to
other. thunder becomes
Result: our natural response to associated with
one stimulus now can be lightning.
triggered by the new, predictive
stimulus.
After Repetition
Stimulus: See lightning
Response: Cover ears to avoid sound
Associative Learning:
Operant Conditioning
 Child associates his “response” (behavior) with consequences.
 Child learns to repeat behaviors (saying “please”) which were
followed by desirable results (cookie).
 Child learns to avoid behaviors (yelling “gimme!”) which were
followed by undesirable results (scolding or loss of dessert).
Cognitive Learning
Cognitive learning refers to acquiring new behaviors
and information mentally, rather than by direct
experience.
Cognitive learning occurs:
1. by observing events and the behavior of others.
2. by using language to acquire information about
events experienced by others.
Classical Conditioning
Topics to help you build new
associations about conditioning
 Classical conditioning vs Operant
Conditioning
 Pavlov’s experiments and concepts
 Acquisition, Extinction,
Spontaneous Recovery
 Generalization and Discrimination
 Applications of Classical
Conditioning
Ivan Pavlov’s Discovery
While studying salivation in dogs, Ivan Pavlov
found that salivation from eating food was
eventually triggered by what should have been
neutral stimuli such as:
 just seeing the food.
 seeing the dish.
 seeing the person who brought the food.
 just hearing that person’s footsteps.
Find the US, UR, NS, CS, CR in the following:

Your romantic partner always uses the same

shampoo. Soon, the smell of that shampoo makes
you feel happy.

The door to your house squeaks loudly when you

open it. Soon, your dog begins wagging its tail when
the door squeaks.

The nurse says, “This won’t hurt a bit,” just before

stabbing you with a needle. The next time you hear
“This won’t hurt,” you cringe in fear.

You have a meal at a fast food restaurant that causes

food poisoning. The next time you see a sign for that
restaurant, you feel nauseated.
 Acquisition refers to the initial
Acquisition stage of learning/conditioning.

What gets “acquired”?

 The association between a neutral
stimulus (NS) and an unconditioned
stimulus (US).
How can we tell that acquisition has
occurred?
 The UR now gets triggered by a CS
(drooling now gets triggered by a bell).
Timing
For the association to be acquired,
the neutral stimulus (NS) needs to
repeatedly appear before the
unconditioned stimulus (US)…about a
half-second before, in most cases. The
bell must come right before the food.
12
 Acquisition and Extinction
 The strength of a CR grows with conditioning.
 Extinction refers to the diminishing of a conditioned response. If
the US (food) stops appearing with the CS (bell), the CR decreases.
Spontaneous Recovery [Return of the CR]
After a CR (salivation) has been conditioned and then extinguished:
 following a rest period, presenting the tone alone might lead to a
spontaneous recovery (a return of the conditioned response
despite a lack of further conditioning).
 if the CS (tone) is again presented repeatedly without the US, the
CR becomes extinct again.
Generalization and Discrimination
Please notice the narrow, psychological definition .

Ivan Pavlov conditioned Ivan Pavlov conditioned dogs

dogs to drool when to drool at bells of a certain
rubbed; they then also pitch; slightly different
drooled when scratched. pitches did not trigger
drooling.
Generalization: the Discrimination: the learned
tendency to have ability to only respond to a
conditioned responses specific stimuli, preventing
triggered by related stimuli. generalization.

MORE stuff makes you drool. LESS stuff makes you drool.
 Ivan Pavlov’s Legacy
Insights from
specific
applications
Insights about
science • Substance abuse
involves
Insights about • Learning can be conditioned
conditioning in studied triggers, and
general objectively, by these triggers
quantifying (certain places,
• It occurs in all actions and
creatures. events) can be
isolating avoided or
• It is related to elements of
biological drives associated with
behavior. new responses.
and responses.
John B. Watson and Classical
Conditioning: Playing with Fear
 9-month-old Little Albert was not afraid of rats.
 John B. Watson and Rosalie Rayner then clanged a
steel bar every time a rat was presented to Albert.
 Albert acquired a fear of rats, and generalized this fear
to other soft and furry things.

 Watson prided
himself in his
ability to shape
people’s
emotions. He later
went into
advertising.
 Operant Conditioning
Topics you may find rewarding
Skinner’s Experiments: Shaping Behavior
Types of Reinforcers:
 Positive and Negative
 Primary and Conditioned
 Immediate and Delayed
Reinforcement Schedules
 Fixed- and Variable-Ratio and Interval
Punishment:
 Positive and Negative
 The downsides
Applications of Operant Conditioning,
in School, Work, Sports, and Home
Operant vs. Classical Conditioning,
revisited
 Operant Conditioning
Operant conditioning involves How it works:
adjusting to the consequences An act of chosen behavior (a
of our behaviors. Examples: “response”) is followed by a
reward or punitive feedback
 We may smile more at work from the environment.
after this repeatedly gets us Results:
bigger tips.  Reinforced behavior is more
 We learn how to ride a bike likely to be tried again.
using the strategies that  Punished behavior is less
don’t make us crash. likely to be chosen in the
future.

Response: Consequence: Behavior

balancing a ball receiving food strengthened
 Thorndike’s Law of Effect

Edward Thorndike placed cats in a puzzle box;

they were rewarded with food (and freedom)
when they solved the puzzle.
Thorndike noted that the cats took less time
to escape after repeated trials and rewards.
Thorndike’s law of effect: behaviors followed
by favorable consequences become more
likely, and behaviors followed by unfavorable
consequences become less likely.
B.F. Skinner: Behavioral Control
B. F. Skinner saw potential for
exploring and using Edward
Thorndike’s principles much more
broadly. He wondered:
 how can we more carefully
measure the effect of
consequences on chosen
behavior?
 what else can creatures be taught
to do by controlling B.F. Skinner trained
consequences? pigeons to play ping
pong, and guide a
 what happens when we change video game missile.
the timing of reinforcement?
B.F. Skinner: The Operant Chamber
 B. F. Skinner, like Ivan Pavlov, pioneered more controlled
methods of studying conditioning.
 The operant chamber, often called “the Skinner box,”
allowed detailed tracking of rates of behavior change in
response to different rates of reinforcement.

Bar or lever
that an animal
presses,
randomly at Recording
first, later for device
reward

Food/water dispenser
to provide the reward
Reinforcement
 Reinforcement: This meerkat has just
feedback from the completed a task out
environment that makes in the cold
a behavior more likely to
be done again.
 Positive +
reinforcement: the
reward is adding
something desirable For the meerkat,
 Negative - this warm light is
reinforcement: the desirable.
reward is ending
something unpleasant
Shaping Behavior
Reinforcing Successive Approximations

Shaping: guiding a creature toward the behavior by

reward behavior that comes closer and closer to the
desired behavior.
Shaping the teacher:
Students could smile and
nod more when the
instructor moves left, until
the instructor stays
pinned to the left wall.
A cycle of mutual reinforcement

Children who have a temper tantrum when they are

frustrated may get positively reinforced for this
behavior when parents occasionally respond by giving
in to a child’s demands.
Result: stronger, more frequent tantrums
Parents who occasionally give in to tantrums may get
negatively reinforced when the child responds by
ending the tantrum.
Result: parents giving-in behavior is strengthened
(giving in sooner and more often)

25
Discrimination

 Discrimination: the ability to become

more and more specific in what
situations trigger a response.
 Shaping can increase discrimination, if
reinforcement only comes for certain
discriminative stimuli.
 For examples, dogs, rats, and even
spiders can be trained to search for
very specific smells, from drugs to
explosives.
 Pigeons, seals, and manatees have
been trained to respond to specific
shapes, colors, and categories. Bomb-finding rat
 Why we might work for money
 If we repeatedly introduce a neutral stimulus before
a reinforcer, this stimulus acquires the power to be
used as a reinforcer.

 A primary reinforcer is a stimulus that meets a basic

need or otherwise is intrinsically desirable, such as
food, sex, fun, attention, or power.

 A secondary/conditioned reinforcer is a stimulus,

such as a rectangle of paper with numbers on it
(money) which has become associated with a
primary reinforcer (money buys food, builds power).
 A Human Talent:
Responding to Delayed Reinforcers
 Dogs learn from immediate reinforcement; a treat five
minutes after a trick won’t reinforce the trick.
 Humans have the ability to link a consequence to a
behavior even if they aren’t linked sequentially in time.
 A piece of paper (paycheck) can be a delayed reinforcer,
paid a month later, if we link it to our performance.
 Delaying gratification, a skill related to impulse control,
enables longer-term goal setting.
 How often should we reinforce?

 Do we need to give a reward every single time? Or is

that even best?
 B.F. Skinner experimented with the effects of giving
reinforcements in different patterns or “schedules”
to determine what worked best to establish and
maintain a target behavior.
 In continuous reinforcement (giving a reward after
the target every single time), the subject acquires the
desired behavior quickly.
 In partial/intermittent reinforcement (giving
rewards part of the time), the target behavior takes
longer to be acquired/established but persists longer
without reward.
 Different Schedules of
Partial/Intermittent Reinforcement
We may schedule  Fixed interval schedule:
our reinforcements
based on an  Every so often
interval of time  Variable interval schedule:
that has gone by.  Unpredictably often

We may plan for a  Fixed ratio schedule:

certain ratio of
rewards per  Every so many behaviors
number of  Variable ratio schedule:
instances of the  After an unpredictable
desired behavior. number of behaviors
Which Schedule of Reinforcement is This?
Ratio or Interval? Fixed or Variable?
1. Rat gets food every third time it presses the lever FR
2. Getting paid weekly no matter how much work is done FI
3. Getting paid for every ten boxes you make FR
4. Hitting a jackpot sometimes on the slot machine VR
5. Winning sometimes on the lottery you play once a day VI/VR
6. Checking cell phone all day; sometimes getting a text VI
7. Buy eight pizzas, get the next one free FR
8. Fundraiser averages one donation for every eight houses VR
visited
9. Kid has tantrum, parents sometimes give in VR
10. Repeatedly checking mail until paycheck arrives FI
Results of the different schedules of reinforcement
Which reinforcements produce more
“responding” (more target behavior)?

 Fixed interval: slow,

unsustained responding Rapid Fixed interval
responding Fixed interval
If I’m only paid for my Rapid responding
near time
near time forfor
reinforcement
Saturday work, I’m not reinforcement
going to work as hard on
the other days.
 Variable interval: slow, Variable interval
Steady
consistent responding responding
If I never know which day
my lucky lottery number
will pay off, I better play it
every day.
Effectiveness of the ratio schedules of
Reinforcement
Fixed ratio
 Fixed ratio: high rate of
responding
Buy two drinks, get one Reinforcers
free? I’ll buy a lot of them!
 Variable ratio: high,
consistent responding,
even if reinforcement
stops (resists extinction) Variable ratio

If the slot machine

sometimes pays, I’ll pull
the lever as many times as
possible because it may
pay this time!
 Operant Effect: Punishment
Punishments have the opposite effects of reinforcement.
These consequences make the target behavior less likely
to occur in the future.

- Negative
+ Positive Punishment
Punishment You TAKE AWAY
You ADD something something pleasant/
unpleasant/aversive desired (ex: no TV
(ex: spank the child) time, no attention)--
MINUS is the
“negative” here

Positive does not mean “good” or “desirable” and

negative does not mean “bad” or “undesirable.”
 When is punishment effective?
 Punishment works best in natural settings when we
encounter punishing consequences from actions such as
reaching into a fire.
 In that case, operant conditioning helps us to avoid
dangers.
 Punishment is less effective when we try to artificially
create punishing consequences for other’s choices;
 Severity of punishments is not as helpful as making
the punishments immediate and certain.
Applying operant conditioning to parenting
Problems with Physical Punishment
 Punished behaviors may simply be suppressed, and restart
when the punishment is over.
 Instead of learning behaviors, the child may learn to
discriminate among situations, and avoid those in which
punishment might occur.
 Instead of behaviors, the child might learn an attitude of
fear or hatred, which can interfere with learning. This can
generalize to a fear/hatred of all adults or many settings.
 Physical punishment models aggression and control as a
method of dealing with problems.
More effective forms of operant conditioning
The Power of Rephrasing
 Positive punishment: “You’re playing video games instead
of practicing the piano, so I am justified in YELLING at you.”
 Negative punishment: “You’re avoiding practicing, so I’m
turning off your game.”
 Negative reinforcement: “I will stop staring at you and
bugging you as soon as I see that you are practicing.”
 Positive reinforcement: “After you practice, we’ll play a
game!”
Summary: Types of Consequences
Adding stimuli Subtract stimuli Outcome
Positive + Negative – Strengthens
Reinforcement Reinforcement target behavior
(You get candy) (I stop yelling) (You do chores)
Positive + Negative – Reduces target
Punishment Punishment behavior
(You get (No cell phone) (cursing)
spanked)
uses desirable uses unpleasant
stimuli stimuli
 B.F. Skinner’s Legacy

B.F. Skinner’s View Critique

 The way to modify behavior is  This leaves out the value of
through consequences. instruction and modeling.
 Behavior is influenced only by  Adult humans have the ability
external feedback, not by to use thinking to make
thoughts and feelings. choices and plans
 We should intentionally create  Natural consequences are
consequences to shape the more justifiable than
behavior of others. manipulation of others.
 Humanity improves through  Humanity improves through
conscious reinforcement of free choice guided by wisdom,
positive behavior and the conscience, and responsibility.
punishment of bad behavior.
 Applications of Operant Conditioning

School: long before Sports: athletes Work: some

tablet computers, B.F. improve most in the companies make
Skinner proposed shaping approach in pay a function of
machines that would which they are performance or
reinforce students for reinforced for company profit
correct responses, performance that rather than
allowing students to comes closer and seniority; they
improve at different closer to the target target more
rates and work on skill (e.g., hitting specific behaviors
different learning pitches that are to reinforce.
goals. progressively faster).
More Operant Conditioning Applications
Parenting
1. Rewarding small improvements toward desired behaviors works
better than expecting complete success, and also works better
than punishing problem behaviors.
2. Giving in to temper tantrums stops them in the short run but
increases them in the long run.

Self-Improvement
Reward yourself for steps you take toward your goals. As you
establish good habits, then make your rewards more infrequent
(intermittent).
 Contrasting Types of Conditioning
Classical Conditioning Operant Conditioning
Associating events/stimuli Associating chosen behaviors
Basic Idea with each other
Organism with resulting events
associates events.
Involuntary, automatic Voluntary actions “operating”
Response reactions such as salivating on our environment
NS linked to US by repeatedly Behavior is associated with
Acquisition presenting NS before US punishment or reinforcement
CR decreases when CS is Target behavior decreases
Extinction repeatedly presented alone when reinforcement stops
Spontaneous Extinguished CR starts again Extinguished response starts
Recovery after a rest period (no CS) again after a rest (no reward)
When CR is triggered by Response behavior similar to
Generalization stimuli similar to the CS the reinforced behavior.
Distinguishing between a CS Distinguishing what will get
Discrimination and NS not linked to U.S. reinforced and what will not
 If the organism is Operant vs. Classical
learning associations Conditioning
between its behavior
and the resulting
events, it is...
operant conditioning

If the organism is
learning associations
between events that it
does not control, it is...

classical conditioning
Effects of Biology and Cognition;
Learning by Observation
Topics you can learn with
the help of your body,
thinking, and observation
 Biological constraints on
conditioning
 Cognitive processes in
conditioning
 Learning by Observation,
including Mirroring and
Imitation
 Prosocial and Antisocial
Effects of Observational
Learning
Summary of
factors
affecting
learning
 Biology influences on Conditioning
Classical Conditioning
 John Garcia and others found it was
easier to learn associations that make
sense for survival.
 Food aversions can be acquired even if
the UR (nausea) does NOT immediately
follow the NS. When acquiring food
aversions during pregnancy or illness,
the body associates nausea with
whatever food was eaten.
 Males in one study were more likely to
see a pictured woman as attractive if the
picture had a red border.
 Quail can have a sexual response linked
to a fake quail more readily and strongly
than to a red light.
Role of Biology in Conditioning
Operant Conditioning
 Can a monkey be trained to peck with
its nose? No, but a pigeon can.
 Can a pigeon be trained to dive
underwater? No, but a dolphin can.
 Operant conditioning encounters
biological tendencies and limits that
are difficult to override.
 What can we most easily train a dog to
do based on natural tendencies?
 detecting scents?
 climbing and balancing?
 putting on clothes?
 Cognitive Processes in Conditioning
In classical conditioning In operant conditioning
 When the dog salivates  In fixed-interval
at the bell, it may be reinforcement, animals do
due to cognition more target
(learning to predict, behaviors/responses
even expect, the food). around the time that the
 Knowing that our reward is more likely, as if
reactions are caused by expecting the reward.
conditioning gives us  Expectation as a cognitive
the option of mentally skill is even more evident
breaking the in the ability of humans to
association, e.g. respond to delayed
deciding that nausea reinforcers such as a
associated with a food paycheck.
aversion was actually
caused by an illness.
 Latent Learning
 Rats appear to form cognitive maps.
They can learn a maze just by
wandering, with no cheese to
reinforce their learning.
 Evidence of these maps is revealed
once the cheese is placed somewhere
in the maze. After only a few trials,
these rats quickly catch up in maze-
solving to rats who were rewarded
with cheese all along.
 Latent learning refers to skills or
knowledge gained from experience,
but not apparent in behavior until
rewards are given.
Learning, Rewards, and Motivation
 Intrinsic motivation refers to the desire to perform a
behavior well for its own sake. The reward is internalized as a
feeling of satisfaction.
 Extrinsic motivation refers to doing a behavior to receive
rewards from others.
 Intrinsic motivation can sometimes be reduced by external
rewards, and can be prevented by using continuous
reinforcement.
 One principle for maintaining behavior is to use as few
rewards as possible, and fade the rewards over time.
What might happen if
we begin to reward a
behavior someone
was already doing and
enjoying?
Learning by Observation
 Can we, like the rats exploring the maze with no reward,
learn new behaviors and skills without a direct experience of
conditioning?
 Yes, and one of the ways we do so is by observational
learning: watching what happens when other people do a
behavior and learning from their experience.
 Skills required: mirroring, being able to picture ourselves
doing the same action, and cognition, noticing consequences
and associations.
Observational Learning Processes
The behavior of others serves as a model, an
Modeling example of how to respond to a situation; we may try
this model regardless of reinforcement.
Vicarious  Vicarious: experienced indirectly, through others
Vicarious reinforcement and punishment means
Conditioning our choices are affected as we see others get
consequences for their behaviors.
Albert Bandura’s Bobo Doll Experiment (1961)
 Kids saw adults punching an inflated doll while narrating
their aggressive behaviors such as “kick him.”
 These kids were then put in a toy-deprived situation…
and acted out the same behaviors they had seen.
 Mirroring in the Brain
 When we watch others doing or feeling something,
neurons fire in patterns that would fire if we were
doing the action or having the feeling ourselves.
 These neurons are referred to as mirror neurons,
and they fire only to reflect the actions or feelings of
others.
 From Mirroring to Imitation
 Humans are prone to spontaneous imitation of both
behaviors and emotions (“emotional contagion”).
 This includes even overimitating, that is, copying adult
behaviors that have no function and no reward.
 Children with autism spectrum disorder are less likely to
cognitively “mirror,” and less likely to follow someone
else’s gaze as a neurotypical toddler (left) is doing below.
Prosocial Effects of Observational Learning

 Prosocial behavior
refers to actions
which benefit others,
contribute value to
groups, and follow
moral codes and
social norms.
 Parents try to teach
this behavior through
lectures, but it may
be taught best
through modeling…
especially if kids can
see the benefits of
the behavior to
oneself or others.
Antisocial Effects of Observational Learning

 What happens when we learn

from models who demonstrate
antisocial behavior, actions that
are harmful to individuals and
society?
 Children who witness violence in
their homes, but are not physically
harmed themselves, may hate
violence but still may become
violent more often than the
average child.
 Perhaps this is a result of “the
Bobo doll effect”? Under stress,
we do what has been modeled for
us.
 Media Models of Violence
Do we learn
antisocial
behavior such
as violence
from indirect
observations
of others in
the media?

Research shows that viewing media violence leads to

increased aggression (fights) and reduced prosocial behavior
(such as helping an injured person).
This violence-viewing effect might be explained by imitation,
and also by desensitization toward pain in others.
 Summary
 Classical conditioning: Ivan Pavlov’s salivating dogs
 New triggers for automatic responses
 Operant conditioning: B.F. Skinner’s boxes and his
pecking pigeons
 Consequences influencing chosen behaviors
 Biological components: constraints, neurons
 Observational learning: Albert Bandura’s Bobo
dolls, mirroring, prosocial and antisocial modeling
Epilogue: Learning about Learning
 From Pavlov and Watson to
Skinner and Bandura,
scientists have added to our
understand of how we
develop our associations,
habits, our reactions and our
patterns of behavior.
 Crucial to their success in
learning how learning works:
curiosity, and a willingness to
study systematically ,
experimenting under
controlled conditions,
developing and testing ideas.
 Photo Credits
• Slide 9:Archives of the History of American Psychology, The University of Akron
• Slide 17:
– Archives of the History of American Psychology, The University of Akron
– Brown Brothers
• Slide 18:
– © The New Yorker Collection, 1993, Tom Cheney from cartoonbank.com. All Rights Reserve
– The New Yorker Collection, 2001, Mick Stevens from cartoonbank.com. All Rights Reserved.
• Slide 20: Eric Isselee/Shutterstock
• Slide 21: Bachrach/ Getty Images
• Slide 23: Reuters/Corbis
• Slide 26: Gamma-Rapho/Getty Images
• Slide 44:
– Courtesy of Albert Bandura, Stanford University
– Marina Jay / Shutterstock
• Slide 46: Courtesy of Kathryn Brownson, Hope College
• Slide 47: Marina Jay / Shutterstock
• Slide 49: Will & Deni McIntyre/ Science Source
• Slide 52: Courtesy of Albert Bandura, Stanford University
• Slide 53: Reprinted with permission from The American Assocition for the Advancement of Science.
Subiaul et al., Science 305: 407-410 (2004) c2004 AAAS
• Slide 54: Meltzoff, A. N., Kuhl, P. K., Movellan, J., & Sejnowski, T. J. (2009). Foundations for a new science of
learning. Science 325, 284-288
• Slide 55: Zumapress/ Newscom
• Slide 56: David Strickler/The Image Works
• Slide 57: Stanislav Solntsev/Getty Images
• Slide 59: Will and Deni McIntyre/Science Source