Chapter 6 Operant Conditioning: Introduction Text Book Notes

Behaviors that are influenced by their consequences are called operant behaviors.
The effects of those consequences upon behavior are called operant conditioning.
The response operates on the environment to produce a consequence.
AKA instrumental conditioning because the response is instrumental in
producing the consequence

Edwin L. Thorndike- first experimental studies of operant conditioning in 1890s.
Studied on animals
Blief that intellectual ability of animals could only be properly assessed by
systematic investigation
Developed puzzle box- cat in box, food outside, figure out how to escape from
box to receive food. (press treadle to open gate)
Repeated trials- learned to escape more quickly, but no sudden
improvement. (no sudden insight)
Response that work was gradually strengthened while the responses
that did not were gradually weakened.
Law of effect: behaviors leading to a satisfying state of affairs are
strengthened stamped in while behaviors leading to an unsatisfying or annoying
state of affairs are weakened stamped out
The extent to which the consequences of a behavior are annoying or
satisfying determine whether that behavior will be repeated.

Skinner- 1920s- first believed believed behavior was best analyzed as a reflex but
abandoned theory- scientific analysis of behavior required a procedure that yielded
regular patterns of behavior.
Invented operant conditioning chamber- Skinner Box
Free operant procedure- rat freely responds with a particular behavior for
food and may do so ay any rate (also pigeon pecking at response key)
Behavior can be put in two categories-
Involuntary, reflexive- classically conditioned- respondent behavior
Behavior that seems more voluntary in nature and controlled by their
consequences rather than what precedes them- operant behavior

OPERANT CONDITIONING
Operant conditioning: a type of learning in which the future probability of a
behavior is affected by its consequence. (like Thorndikes law of effect but less
mentalistic annoying/ satisfying)
Emphasize effect of the consequence on the future probability of the behavior
Resembles natural selection- but of behaviors. Mini evolution of an
organisms behavior, behaviors that are adaptive and lead to favorable outcomes
become more frequent while behaviors that are non adaptive and do not lead to
favorable outcomes become less frequent.
Three components:
1. A response that produces a certain consequence
2. The consequence that serve to either increase or decrease the probability
of the response that preceded it.
3. A discriminative stimulus that precedes the response and signals that a
certain consequence is now available.

Operant Behavior: a class of emitted responses that result in certain consequences;
these consequences affect the future probability or strength of those responses.
Operant responses sometimes = operants
i.e. lever press food pellet
the future probability of lever pressing increases
The behavior in question is an operant response (R) because its occurrence results
in a certain consequence and that consequence affects the future probability of that
response.
Classically conditioned behaviors are elicited by a stimuli, operant behaviors are
emitted by the organism.
Operant behavior is usually defined as a class of responses- all responses in that class
are capable of producing the consequence (hit the lever hard, soft, right paw, left..)
Easier to predict the occurrence of a class of responses than the exact
response

Reinforcer: consequences that strengthen a behavior (S^R = reinforcing stimulus)
1. Follows a behavior and
2. The future probability of that behavior increases
Punisher: consequences that weaken a behavior (S^P = punishing stimulus)
1. it follows a behavior
2. the future probability of that behavior decreases
-Do not say the person or animal has been reinforced or punished, but the behavior has
been reinforced or punished.
-Reinforcer and Punisher refer to specific consequences used to strengthen or weaken
a behavior (the food itself)
-Reinforcement and Punishment refer to the process and procedure by which a certain
consequence changes he strength of a behavior (use of food to reinforce a behavior)
-Reinforcers and punishers are formally defined entirely by their effect on behavior.
Define consequences as Reinforcer or Punisher in relation to effect on
behavior, not pleasantness.
Response strengthened via reinforcement can be weakened by the with drawl of
reinforcement- extinction.
Much gentler and slower than punishment

When a behavior is consistently reinforced or punished in the presence of certain
stimuli, those stimuli beigin to influence the occurrence of the behavior
Discriminative stimulus (S^D) is a stimulus in the presence of which responses
are reinforced and in the absence of are not reinforced
A signal that indicates a response will be followed by a reinforcer
Set the occasion for the behavior
Does not elicit the behavior, but increases the probability of the behavior
occurring
It indicates the consequence is now available.
Three term contingency- Discriminative stimulus, the operant behavior, and the
reinforcer/ punishment.
ABC- antecedent event, behavior, and consequence
Stimuli that signals a response will be punished- discriminative stimulus for
punishment
S^D^P
Discriminative stimulus for extinction- stimulus signal for the nonavailability of a
previously available reinforcer
S^delta
Like CC, in OC, a stimulus can be both an SD (OC) and a CS (CC). (tone (sd, cs) for
food/ salivation)

FOUR TYPES OF CONTINGENCIES
If a response is followed by a reinforcer, a contingency of reinforcement exists. (the
delivery of a reinforcer is contingent upon the response)
If a response is followed by a punishment, a contingency of punishment exists.
Positive and negative reinforcers and punishments.
Positive means something is added, and negative something is taken away.
Appetitiveness is irrelevant.
Does the consequence consist of something being presented or withdrawn?
Does the consequence serve to strengthen or weaken the behavior?
S^R+ positive reinforcement S^R- negative reinforcement. Same for S^P
Sometimes it is a judgment call if it is positive or negative. Matter of interpretation
Negative reinforcement behaviors:
Escape behavior results in termination or stopping of an aversive stimulus
Avoidance behavior occurs before the aversive stimulus is presented and prevents
its delivery.
The reduction in one persons behavior as a result of punishment can negatively
reinforce the behavior of the person who implemented the punishment.
Punishment is often successful in immediately getting a person to stop behaving in a
way we dislike
Create problems in the long run

POSITVE REINFORCEMENT: FURTHER DISTINCTIONS
The more immediate the reinforcement is, the stronger its effect on the behavior.
Behaviors that appear to be strengthened by a long delayed reinforcer are often
under the control of an instruction- bridge gap
Primary (unconditioned) reinforcer an event that is innately reinforcing
Born to like, rather than learn to like
Food, water, temp, sex
Many associated with physiological needs and effectiveness is tied to a state
of deprivation.
Secondary (conditioned) reinforcer- an event that is reinforcing because it has
been associated with some other reinforcer
Events that we have leaned to like because they have become associated with
other things that we like.
i.e. seek out certain music associated with a romantic episode because of its
plesant associations
Discriminative stimuli associated with reinforcers function as secondary
reinforcers.
Generalized (secondary)reinforcer is a type of secondary reinforcer that has been
associated with several other reinforcers.
Money and attention
Used in behavior modification- Token economy- earn tokens for desireable
behavior and exchange for back up reinforcers
Behaviors that are associated with reinforcement can become reinforcing on
their own.
Intrinsic reinforcement is reinforcement provided by the mere act of performing
the behavior
Extrinsic reinforcement is reinforcement provided by some consequence that is
external to the behavior (reading to complete an exam)
Give extrinsic reinforcer for an activity that is intrinsically reinforcing-
Cameron and Pierce- extrinsic rewards usually have little or no effect on
intrinsic motivation. Occasionally undermine intrinsic motivation but only when the
reward is expected, the reward is tangible, and the reward is given for simply
performing the task despite performance quality.
Verbal rewards increase intrinsic motivation, and tangible rewards for high
quality performance.
Natural reinforcers are reinforcers that are typically provided for a certain
behavior. They are an expected consequence of a the behavior within that setting
Contrived (artificial) reinforcers are reinforcers that have been deliberately
arranged to modify a behavior. They are not a typical consequence of the behavior
in that setting.
Utilize natural reinforcers whenever possible, when contrived are used,
ultimate intention is to let the natural contingencies take over.
Intrinsic and extrinsic: extent to which the behavior itself is reinforcing
Natural and contrived: the extent to which a reinforcer has been artificially imposed so
as to manipulate a behavior

SHAPING
Shaping- the gradual creation of new operant behavior through reinforcement of
successive approximations to that behavior. Reward for each step closer to desired
behavior.