Chapter 1: Research Methods in Psychology / Research

Methodology

Psychology is the scientific study of behavior and mental processes.

Psychology is the study of human mind and behavior.

To make a theory & study scientific:

 It should be supported by empirical evidence and be based on this

evidence
 It should be falsifiable, that is, it should be possible for the theory or
study to be proven wrong.
 There should be a history of independent attempts to test the theory
or replicate the study.

Quantitative and Qualitative Data

Quantitative data – numerical, like scores on a task or totals collected

from experiments, correlations, questionnaires or observations using
behavioral categories.

Qualitative data – descriptive, e.g. answers to open questions from self-

reports and detailed findings from observation and content analyses.

Experimental Methods

Confounding variables -> impacts DV

Extraneous variables -> distractions

Laboratory experiment Field experiment

Strengths Strengths
 Easy to control extraneous  More representative and
variables (increases internal naturalistic
validity)  Less demand characteristics
 Independent variables are  Researcher manipulates the
controlled by IV
researchers/experimenters  But many extraneous
 Standard procedures allow variables cannot be
replication and reliability controlled
Weakness
 Artificial situations -> low
ecological validity Weakness
 Create demand  Too much extraneous
characteristics (expectancy variables
 effect, screw-you effect, and  Too much to control
social desirability)  Internal validity decreases
 Research bias  Harder to replicate
 Raises ethical issues

Quasi experiment Natural experiment

 Quasi refers to not really  Researchers cannot
scientific manipulate independent
 IV is not contr0lled by the variable
researchers  Can be used to study real-
- IV are set according to world issues
certain trait and  In normal situations
characteristics
 Can be used to study real-
world issues
Ex) Fiji Island (TV introduction)
Ex) London’s taxi driver
(Maguire)

Experimental Designs

1. Independent design – different groups of participants are used in

each level of the IV
o Limitation: Participant variability – Differences in the traits
and past experiences of the participants in each condition may
affect the results of the study.

o Random allocation minimizes participant variability

2. Repeated measure – each participant performs in every level of the

IV
o Limitation: Order effect (Practice effect, Fatigue effect)

To minimize order effect: Counterbalancing.

3. Matched pairs design – an experimental design where pairs of

participants are matched in terms of key variables, such as age. One
member of each pair is then placed into the experimental group and
the other member into the control group.

To minimize Demand Characteristics & Research Bias:

Single blind - only participants do not know to which group they have been
assigned.

Double Blind - both the patient and the researcher are ignorant as to the
condition that the participant is in.

Self-Report Methods: Interviews and Questionaries

Interviews
1. Structured Interview
o Prepared questions
o Easy to compare the responses
o No topic deviation but can be biased
o Time efficient
Limitations:
 Researcher bias (can lead to certain response)
 Not flexible
2. Semi-structured Interview
o First prepared questions -> flexible questions
o Bidirectional conversation
o Sensitive topics
o Formation of Rapport

Limitations:

3. Unstructured Interview
o No prepared questions
o Difficult to manipulate the interviewee’s opinion, memory,
thoughts
o Detailed
o Difficult to compare/analyze
o Time consuming
o Example) Witnesses, Survivor

Limitations:

Social desirability

Interviewer effect – a type of bias that emerges when a characteristic of

an interviewer (race, age, gender etc.) influences the responses given by
the interviewee

Focus group
 o A special of semi-structured interview that is conducted
simultaneously with a group of 6-10 people
o Key factor: participants are encouraged to interact with each-
other and the interviewer serves as a facilitator
o Participants: discuss responses to every question and react to
each other’s statement (agree & disagree & enrich other’s
perspective)
o Facilitator: observe group dynamics and make use of it by
directing group members’ interaction so that they stay focused
on the research topic

Advantages
- Quick way to get information from several participants at the same
time
- Creates a more natural and comfortable environment than a face-
to-face interview
- Easier to respond to sensitive questions when you are in a group
- Multiple perspectives are discussed so a more holistic
understanding of the topic is achieved

Limitations
- If one of the participants is especially dominant, this may distort
the responses of the other participants
- More difficult to preserve anonymity and confidentiality
- Focus groups are especially demanding in terms of sampling and
creating interview transcripts
- Conformity

Questionaries
o Time and cost efficient
(ex. Easy to send email to participants)
o Response bias such as always answering “no” or always ticking the
left-hand box
o Reverse question
o Honest response -> confidentiality
o Relatively easy to analyze quantitative data
o Questionnaire return rates may be low and the sample may be biased
o Population validity

Observations
- Self-reports can only measure what a person thinks they would do,
whereas observations allow researchers to record actual responses
 - Naturalistic observation
o Higher validity
o Inter-observer reliability between multiple observers may
be low
Ex. They make different records from same behavior
o High inter-observer reliability is +

- Controlled or contrive observation

o Confederates (formal way to say: actor)

Observational techniques
 Participant
 Influence other members
- Difficult to record
 Non-participant
 Objective

 Covert (non-disclosed, 몰래) -> more naturalistic & representative
 Overt (disclosed, 대고) -> ethically correct, increases reactivity

Credibility and generalizability in the experiment: types of validity

Construct validity – characterizes the quality of operationalizations

(Defines how well a test or experiment measures up to its claims. It refers to
whether the operational definition of a variable actually reflect the true
theoretical meaning of a concept.)

Generalisation to a real life situation is increased if the construct validity of

the study is high and the way the researcher is measuring or manipulating
the variables matches the way those variables might be seen in real life.

= how well a test measures the concept it was designed to evaluate

Internal validity – clear IV and DV relationship, cause-effect

External validity – to what extent can we apply the result to, characterizes
the generalizability of findings in the experiment
1. Population validity
o Whether the researcher can generalize the findings from
their sample to a larger group of people
2. Ecological validity
o Low ecological validity - > difficult to apply/generalize the
result to real life situation
 3. History validity
o Low history validity -> too old experiment

Threats to internal validity

Based on Campbell (1969):

1. Selection
 Not equivalent at the start of the experiment: apart from
the planned IV-related difference, they differ in some
other variable
 Cannot be sure if the post-experiment differences between
groups reflect the influence of the IV or this other variable
2. History
 Outside events that happen to participants in the course of the
experiment
 Become a problem when they can potentially influence the DV
or are not evenly distributed in the comparison groups

3. Maturation
 Participants go through natural developmental processes such
as fatigue or simple growth
4. Testing effect
 The first measurement of the DV may affect the second
measurements
 Order effect
5. Instrumentation
 Occurs when the instrument measuring the DV changes slightly
between measurements
 An “instrument measuring” is often an observer
6. Regression to the mean
 Source of bias that becomes a concern when the initial score on
the DV is extreme (either low or high)
7. Experimental mortality
 The fact that some participants drop out during an experiment,
which may become a problem if dropouts are not random
 Raises ethical and methodological issues
8. Demand characteristics
 When participants understand the purpose of the experiment
and change their behavior subconsciously to fit that
interpretation
  Socially desirable way
9. Experimenter bias
 The researchers unintentionally exerts an influence on the
results of the study
Rosenthal and Fode (1963)
 Rats were split into two groups at random: but assistants were
told that one group was “maze-bright” and the other one was
“maze-dull
 Results are artifact: it was caused by experimenter bias rather
than any genuine differences between the groups of rats

Reliability – the consistency of a measure, whether the results from the

same participants would be similar each time

Credibility –

Triangulation
 A combination of different approaches to collecting and
interpreting data
 Used to enhance the credibility of a study

1. Method triangulation
 Use of different methods in combination can compensate for
their individual limitations and reinforce their strengths
 More than one research methods (interviews and
observations)

2. Researcher triangulation
 Combining observations/interpretations of different
researchers

3. Theory triangulation
 Using multiple perspectives or theories to interpret the data

4. Data triangulation
 Using data from a variety of accessible sources
 Ex. If participants during an interview refer to certain
documents, these documents may be studied in order to
gain a clearer understanding of the participants’
experiences

Researcher bias & Participant bias

Participation bias

1. Acquiescence bias
 Tendency to give positive answers whatever the question.

2. Social desirability bias

 Participants’ tendency to respond or behave in a way that they
think will make them liked or accepted

3. Dominant respondent bias

 Occurs in a group interview setting when one of the participants
influences the behavior and responses of the others

4. Sensitivity bias
 A tendency of participants to answer regular questions honestly,
but distort their responses to questions on sensitive subjects

5. Sunk cost fallacy

Researcher bias

1. Confirmation bias
 Occurs when the researcher has a prior belief and uses the
research in an unintentional attempt to confirm that belief

2. Leading question bias

 Occurs when respondents in an interview are inclined to answer in
a certain way because the wording of the question encourages
them to do so

3. Question order bias

 Occurs when responses to one question influence the participant’s
responses to the following questions

4. Sampling bias
 Occurs when the sample is not adequate for the aims of the
research

5. Biased reporting
 Occurs when some findings of the study are not equally
represented in the research report

Sampling
 Opportunity/ Convenience: participants are chosen because they are
available
  Random: all members of the population are allocated members & a
fixed amount of these are selected in an unbiased way
 Volunteer: participant volunteer to participate -> motivation (bias
may arise)
 Snowballing: participant collect other available participants
 Stratified: subdivision -> more representative
 Systematic:
 Purposive: participants are chosen because they possess
characteristics relevant to the research study

Types of generalizability

Firestone (1993) distinguished between three types of generalizability

1. Sample-to-population generalization
o Starts by researchers identifying the target population and
then selecting a sample that is representative of this
population.
o Best approach to achieve this to use random sampling
o Population validity
o Difficult

2. Theoretical generalization
o Generalization is made from particular observations to a
broader theory
o Construct validity
o In qualitative research, theoretical generalization is achieved
through rigorous analysis and interpretation of research
findings: we can generalize to a wider theory if data saturation
was achieved, thick descriptions provided, analysis was in-depth
and free of biases, and so on
o Theory plays a much greater role in qualitative research than in
quantitative

3. Case-to-case generalization
o Also known as transferability
o Transferability – the degree to which the results of qualitative
research can be generalized or transferred to other contexts or
settings
 o
o generalizability - experiment, quantitative data
o transferability -descriptive, qualitative data

Correlations
 Correlational analysis investigates two variables measured on ordinal,
interval or ratio scales to look for a relationship between them
 Investigating the relationship between two variables
 Never cause-effect
 Positive correlation and negative correlation
 Bidirectional ambiguity
 Ex) Usually in biological studies such as genetic factor

Positive Correlation Negative Correlation

 Two variables increase in the  Two variables increase in
same direction opposite direction

 Illustrated using a Scattergram – graph where data points represent

scores on two measured variables
 Scattergram shows the strength of the relationship (how closely
linked the two variables are)
Ex. If the correlation is strong the points will all lie close to the line of
best fit; If it is weak they will be more scattered

Correlation efficient

+1 = Perfect positive correlation

0 = no correlation
-1 = Perfect negative correlation

Ex)
0.7-0.9: strong positive correlation
0.4-0.6 significant positive correlation / moderate positive correlation
0.1-0.3 very weak positive correlation
Ethical considerations & guidelines

1. Protection from harm/ Participant Protection

 No physical or psychological harm

2. Informed Consent
 Agreement and give real consent

3. Debriefing
 When participants know they have participated in a study they
should get an explanation
4. Deception
 Should be avoided. If deceived, participants should be told the
real aim of the study as soon as possible.
 Used to prevent demand characteristics

5. Right to Withdraw
 Leave the study at any point, regardless of payment, and be
allowed to withdraw their data

6. Confidentiality
 Individual results and personal information should remain
secret and safe

7. Privacy in observations
 Privacy should not be invaded

Research Methods
1. Experiment – cause and relationship
o Lab experiment
o Quasi-experiment vs. True experiment
Quasi-experiment (groups divided based on traits)
True experiment (groups divided randomly)
o Natural experiment
o Field experiment

2. Self-report
o Interview
1. Structured
2. Semi-structured
3. Unstructured
 Focus group
 o Questionnaire (scale)
o Survey (usually qualitative)
3. Observation
o Participant vs. Non- participant
o Covert vs. Overt
o Naturalistic vs. Controlled
4. Correlational study
o R= 0.457 (correlational coefficient)
5. Case study
o Method triangulation
o Rare case
o Ex) H.M. Experiment
 Scoville & Millner (1957) / Corkin (1990)
 Hippocampus & Medial temporal lobe
 Originally suffer from epilepsy

Random sampling & Random allocation

o Random sampling: before research -> find the groups
o Random allocation: during research

Mean & Median

o Mean -> use when SD small (the group does not have outlier)
o Median -> use when there is outlier
T-test
P 0.05 5%