MODULE 4: QUANTITATIVE METHOD

1
 LESSON 1: OVERVIEW ABOUT QUANTITATIVE RESEARCH

LESSON PREPARATION

1- Watch the video (4:20 mins): https://www.youtube.com/watch?v=dokZ5HSYZIA

and take notes of key characteristics of the quantitative research designs below:

Quantitative research designs Your notes

Descriptive research

Correlational research

Quasi-experimental research

Experimental research

LEARNING OBJECTIVES
By the end of this lesson, students are expected to be able to:

1. Describe the differences among four types of quantitative research

2. Describe the differences between research populations and samples

MATERIALS

- Babbie, E. R. (2010). The Practice of Social Research (12th ed). Cengage.

- Creswell, J. W. (2013). Research Design: Qualitative, Quantitative, and Mixed
Methods Approaches. Sage.
- Muijs, D. (2004). Doing Quantitative Research in Education: With SPSS. Sage.

2
SECTION 1: OVERVIEW ABOUT QUANTITATIVE RESEARCH
Quantitative research is defined as a systematic investigation of phenomena by gathering
quantifiable data and performing statistical, mathematical, or computational techniques.
Main characteristics of quantitative research are:
• The data is usually gathered using structured research instruments.
• The results are based on larger sample sizes that are representative of the population.
• The research study can usually be replicated or repeated, given its high reliability.
• The researcher has a clearly defined research question to which objective answers are
sought.
• All aspects of the study are carefully designed before data is collected.
• Data are in the form of numbers and statistics, often arranged in tables, charts, figures,
or other non-textual forms.
• Project can be used to generalize concepts more widely, predict future results, or
investigate causal relationships.
• The researcher uses tools, such as questionnaires or computer software, to collect
numerical data.

Quantitative research could be classified into: Descriptive, Correlational, Causal-

Comparative/Quasi-Experimental, and Experimental Research:
1. Descriptive research seeks to describe the current status of an identified variable. These
research projects are designed to provide systematic information about a
phenomenon. The researcher does not usually begin with a hypothesis, but is likely to
develop one after collecting data. The analysis and synthesis of the data provide the test of
the hypothesis. Systematic collection of information requires careful selection of the units
studied and careful measurement of each variable.

2. Correlational research attempts to determine the relationship between two or more

variables using statistical data. In this type of design, relationships between and among a
number of facts are sought and interpreted. This type of research will recognize trends and
patterns in data, but it does not go so far in its analysis to prove causes for these observed
patterns. Cause and effect is not the basis of this type of observational research. The data,
relationships, and distributions of variables are studied only. Variables are not manipulated;
they are only identified and are studied as they occur in a natural setting.

3
*Sometimes correlational research is considered a type of descriptive research, and not as
its own type of research, as no variables are manipulated in the study.

3. Causal-comparative/quasi-experimental research attempts to establish cause-effect

relationships among the variables. These types of design are very similar to true
experiments, but with some key differences. An independent variable is identified but not
manipulated by the experimenter, and effects of the independent variable on the
dependent variable are measured. The researcher does not randomly assign groups and
must use ones that are naturally formed or pre-existing groups. Identified control groups
exposed to the treatment variable are studied and compared to groups who are not.

When analyses and conclusions are made, determining causes must be done carefully, as
other variables, both known and unknown, could still affect the outcome.

4. Experimental research, often called true experimentation, uses the scientific method to
establish the cause-effect relationship among a group of variables that make up a
study. The true experiment is often thought of as a laboratory study, but this is not always
the case; a laboratory setting has nothing to do with it. A true experiment is any study
where an effort is made to identify and impose control over all other variables except
one. An independent variable is manipulated to determine the effects on the dependent
variables. Subjects are randomly assigned to experimental treatments rather than
identified in naturally occurring groups.

EXER C ISE

1. Decide if the following research focuses are examples of Descriptive (Des),

Correlational (Cor), Quasi-experimental (Quasi), or Experimental (Exp) research
design. Tick the appropriate column.

Research focuses Des Cor Quasi Exp

1- A description of how second-grade students
spend their time during summer vacation
2- The relationship between diet and anxiety
3- The relationship between ACT scores and the
freshman grades

4
4- A description of the kinds of physical
activities that typically occur in nursing
homes, and how frequently each occurs

5- The effect of preschool attendance on social

maturity at the end of the first grade

6- The effect of part-time employment on the

achievement of high school students

7- The effect of a new treatment plan on breast

cancer

8- The effect of teaching with a cooperative

group strategy or a traditional lecture
approach on students’ achievement

9- A comparison of the effect of personalized

instruction vs. traditional instruction on
computational skill

10- The relationships between the types of

activities used in math classrooms and
student achievement

K EY T AK EAWAY
• Descriptive research seeks to describe the current status of an identified variable.
• Correlational research attempts to determine the extent of a relationship between two
or more variables using statistical data.
• Causal-comparative/quasi-experimental research attempts to establish cause-effect
relationships among the variables. However, the independent variable is not
manipulated by the experimenter.
• A true experimental research aims to establish the cause-effect relationship among a
group of variables that make up a study. It identifies and imposes control over all other
variables except one. The independent variable is manipulated to determine the effects
on the dependent variables.

5
SECTION 2: POPULATIONS VERSUS SAMPLES

In social scientific research, a population is the cluster of people, events, things, or other
phenomena that you are most interested in; it is often the “who” or “what” that you want to
be able to say something about at the end of your study. Populations in research may be
rather large, such as “the American people,” but they are more typically a little less vague
than that. For example, a large study for which the population of interest really is the
American people will likely specify which American people, such as adults over the age of 18
or citizens or legal residents. A sample, on the other hand, is the cluster of people or events,
for example, from or about which you will actually gather data. Some sampling strategies
allow researchers to make claims about populations that are much larger than their actually
sample with a fair amount of confidence. Other sampling strategies are designed to allow
researchers to make theoretical contributions rather than to make sweeping claims about
large populations. We’ll discuss both types of strategies later in this module.

Let’s look at the following example: The research question that we wish to answer is: “How
do men’s and women’s college experiences differ, and how are they similar?” The population
in this case could be all college students across all nations from all historical time periods.
Would you expect to be able to collect data from all college students across all nations from
all historical time periods? Unless you plan to make answering this research question your
entire life’s work (and then some), I’m guessing your answer is a resounding no way. So what
to do? Does not having the time or resources to gather data from every single person of
interest mean having to give up your research interest? Absolutely not. It just means having
to make some hard choices about sampling, and then being honest with yourself and your
readers about the limitations of your study based on the sample from whom you were able
to actually collect data.

Sampling is the process of selecting observations that will be analyzed for research purposes.
Both qualitative and quantitative researchers use sampling techniques to help them identify
the what or whom from which they will collect their observations. Because the goals of
qualitative and quantitative research differ, however, so, too, do the sampling procedures of

6
the researchers employing these methods. We’ll look at how sampling typically works in
quantitative research in the later sections.

EXERCISES

1. Read through the methods section of a couple of scholarly articles describing

empirical research. How do the authors talk about their populations and samples, if
at all? What do the articles’ abstracts suggest in terms of whom conclusions are
being drawn about?
2. Think of a research project you have envisioned conducting as you’ve read this text.
Would your population and sample be one and the same, or would they differ
somehow? Explain.

KEY TAKEAWAYS

• A population is the group that is the main focus of a researcher’s interest; a sample is
the group from whom the researcher actually collects data.
• Populations and samples might be one and the same, but more often they are not.
• Sampling involves selecting the observations that you will analyze.

7
LESSON 2: SAMPLING IN QUANTITATIVE RESEARCH
LESSON PREPARATION

1- Watch the video about probability sampling techniques:

https://www.youtube.com/watch?v=jauhoR7w1YM

Take notes of key contents and any questions that you may have. Share your notes with
classmates.

LEARNING OBJECTIVES
By the end of this lesson, students are expected to be able to:

1. Describe how probability sampling differs from nonprobability sampling.

2. Define generalizability, and describe how it is achieved in probability samples.

MATERIALS

- Delice, A. (2010). The Sampling Issues in Quantitative Research. Educational

Sciences: Theory and Practice, 10(4), 2001-2018. Retrieved from:
https://files.eric.ed.gov/fulltext/EJ919871.pdf
- Durrheim, K. & Painter, D. (2006). Collecting quantitative data: Sampling and
measuring (Chap.7). In M. T. Blanche, M. J. T. Blanche, K. Durrheim, & D. Painter
(Eds.). Research in practice: Applied methods for the social sciences (pp.131-159).
Juta and Company Ltd.
- Fowler, F.J. (2009) Survey Research Methods, 4th edition, Thousand Oaks, CA: Sage.

8
SECTION 1: SAMPLING IN QUANTITATIVE RESEARCH

Quantitative researchers are often interested in being able to make generalizations about
groups larger than their study samples. While there are certainly instances when quantitative
researchers rely on nonprobability samples (e.g., when doing exploratory or evaluation
research), quantitative researchers tend to rely on probability sampling techniques. The goals
and techniques associated with probability samples differ from those of nonprobability
samples. We’ll explore those unique goals and techniques in this section.

2.1 Probability Sampling

Probability sampling refers to sampling techniques for which a person’s (or event’s)
likelihood of being selected for membership in the sample is known. The reason for this
practice is that, in most cases, researchers who use probability sampling techniques are
aiming to identify a representative sample from which to collect data. A representative
sample is one that resembles the population from which it was drawn in all the ways that are
important for the research being conducted. If, for example, you wish to be able to say
something about differences between men and women at the end of your study, you better
make sure that your sample doesn’t contain only women. That’s a bit of an oversimplification,
but the point with representativeness is that if your population varies in some way that is
important to your study, your sample should contain the same sorts of variation.

Obtaining a representative sample is important in probability sampling because a key goal of

studies that rely on probability samples is generalizability. In fact, generalizability is perhaps
the key feature that distinguishes probability samples from nonprobability samples.
Generalizability refers to the idea that a study’s results will tell us something about a group
larger than the sample from which the findings were generated. In order to achieve
generalizability, a core principle of probability sampling is that all elements in the researcher’s
target population have an equal chance of being selected for inclusion in the study. In
research, this is the principle of random selection. The important thing to remember about
random selection here is that, as previously noted, it is a core principal of probability
sampling. If a researcher uses random selection techniques to draw a sample, he or she will
be able to estimate how closely the sample represents the larger population from which it

9
was drawn by estimating the sampling error. Sampling error is a statistical calculation of the
difference between results from a sample and the actual parameters of a population.

EXERCISES

1. Read the text and note down the definitions for each terminology. Share your
definitions with your classmates.

Terminologies
Definitions
1. probability sampling

2. representative sample

3. sample

4. generalizability

5. random selection

6. sampling error

7. parameters

SECTION 2: TYPES OF PROBABILITY SAMPLES

There are a variety of probability samples that researchers may use. These include simple
random samples, systematic samples, stratified samples, and cluster samples.

Simple random samples are the most basic type of probability sample, but their use is not
particularly common. Part of the reason for this may be the work involved in generating a
simple random sample. To draw a simple random sample, a researcher starts with a list of
every single member, or element, of his or her population of interest. This list is sometimes
referred to as a sampling frame. Once that list has been created, the researcher numbers
each element sequentially and then randomly selects the elements from which he or she will
collect data. To randomly select elements, researchers use a table of numbers that have been
generated randomly. There are several possible sources for obtaining a random number
table. Some statistics and research methods textbooks offer such tables as appendices to the

10
text. Perhaps a more accessible source is one of the many free random number generators
available on the Internet. A good online source is the website Stat Trek, which contains a
random number generator that you can use to create a random number table of whatever
size you might need (http://stattrek.com/Tables/Random.aspx).

Systematic sampling techniques. As you might have guessed, drawing a simple random
sample can be quite tedious. Systematic sampling techniques are somewhat less tedious but
offer the benefits of a random sample. As with simple random samples, you must be able to
produce a list of every one of your population elements. Once you’ve done that, to draw a
systematic sample you would simply select every kth element on your list. But what is k, and
where on the list of population elements does one begin the selection process? k is
your selection interval or the distance between the elements you select for inclusion in your
study. To begin the selection process, you’ll need to figure out how many elements you wish
to include in your sample. Let’s say you want to interview 25 fraternity members on your
campus, and there are 100 men on campus who are members of fraternities. In this case,
your selection interval, or k, is 4. To arrive at 4, simply divide the total number of population
elements by your desired sample size.

Stratified sampling is a good technique to use when, as in our example, a subgroup of interest
makes up a relatively small proportion of the overall sample. In our example of a study of use
of public space on campus, we want to be sure to include weekdays and weekends in our
sample, but because weekends make up less than a third of an entire week, there’s a chance
that a simple random or systematic strategy would not yield sufficient weekend observation
days. As you might imagine, stratified sampling is even more useful in cases where a subgroup
makes up an even smaller proportion of the study population, say, for example, if we want to
be sure to include both men’s and women’s perspectives in a study, but men make up only a
small percentage of the population. There’s a chance simple random or systematic sampling
strategy might not yield any male participants, but by using stratified sampling, we could
ensure that our sample contained the proportion of men that is reflective of the larger
population.

11
Cluster sampling. Up to this point in our discussion of probability samples, we have assumed
that researchers will be able to access a list of population elements in order to create a
sampling frame. This, as you might imagine, is not always the case. Let’s say, for example, that
you wish to conduct a study of hairstyle preferences across the United States. Just imagine
trying to create a list of every single person with (and without) hair in the country. Basically,
we’re talking about a list of every person in the country. Even if you could find a way to
generate such a list, attempting to do so might not be the most practical use of your time or
resources. When this is the case, researchers turn to cluster sampling. Cluster
sampling occurs when a researcher begins by sampling groups (or clusters) of population
elements and then selects elements from within those groups.

Let’s take a look at a couple of examples. Perhaps you are interested in the workplace
experiences of public librarians. Chances are good that obtaining a list of all librarians that
work for public libraries would be rather difficult. But I’ll bet you could come up with a list of
all public libraries without too much hassle. Thus you could draw a random sample of libraries
(your cluster) and then draw another random sample of elements (in this case, librarians)
from within the libraries you initially selected. Cluster sampling works in stages. In this
example, we sampled in two stages. As you might have guessed, sampling in multiple stages
does introduce the possibility of greater error (each stage is subject to its own sampling error),
but it is nevertheless a highly efficient method.

Holt and Gillespie* (2008) used cluster sampling in their study of students’ experiences with
violence in intimate relationships. Specifically, the researchers randomly selected 14 classes
on their campus and then drew a random subsample of students from those classes. But you
probably know from your experience with college classes that not all classes are the same
size. So if Holt and Gillespie had simply randomly selected 14 classes and then selected the
same number of students from each class to complete their survey, then students in the
smaller of those classes would have had a greater chance of being selected for the study than
students in the larger classes. Keep in mind with random sampling the goal is to make sure
that each element has the same chance of being selected. When clusters are of different sizes,
as in the example of sampling college classes, researchers often use a method
called probability proportionate to size (PPS). This means that they take into account that

12
their clusters are of different sizes. They do this by giving clusters different chances of being
selected based on their size so that each element within those clusters winds up having an
equal chance of being selected.

*Holt, J. L., & Gillespie, W. (2008). Intergenerational transmission of violence, threatened egoism,
and reciprocity: A test of multiple psychosocial factors affecting intimate partner violence. American
Journal of Criminal Justice, 33, 252–266.

EXERCISES

1. Match the sampling strategies with the descriptions

Probability
sampling Description
strategies
a. Researcher randomly selects clusters then randomly selects
1. Simple random
elements from selected clusters.
b. Researcher creates subgroups then randomly selects
2. Systematic
elements from each subgroup.
c. Researcher randomly selects elements from sampling
3. Stratified
frame.

4. Cluster d. Researcher selects every kth element from sampling frame.

Answers: 1- 2- 3- 4-

2. Name the sampling methods in the following figures:

13
3. Imagine that you are about to conduct a study of people’s use of public parks. Explain
how you could employ each of the probability sampling techniques described earlier to
recruit a sample for your study.

KEY TAKEAWAYS

• In probability sampling, the aim is to identify a sample that resembles the population
from which it was drawn.
• There are several types of probability samples including simple random samples,
systematic samples, stratified samples, and cluster samples.

14
 LESSON 3: DESIGNING EFFECTIVE QUESTIONS AND
QUESTIONNAIRES
LESSON PREPARATION

1- Watch the following video (5:27 mins) and take note of things that should be avoided
when designing a survey questionnaire:
https://www.youtube.com/watch?v=eFzGdQrr2K8&t=192s

LEARNING OBJECTIVES
By the end of this lesson, students are expected to be able to:

1. Recall different types of survey questions, their advantages and disadvantages.

2. Describe some of the ways that survey questions might confuse respondents.
3. Evaluate and improve close-ended survey questions.
4. Recall some practical considerations when designing a survey (i.e., length and
ordering of the questions, collection of personal information, piloting the
questionnaire).

MATERIALS

- Babbie, E. (2010). The practice of social research (12th ed.). Belmont, CA: Wadsworth.
- Colosi, L. (2006). Designing an effective questionnaire. Research brief available
online at:
https://ag.purdue.edu/extension/pdehs/Documents/Designing%20an%20Effective%
20Questionnaire.pdf
- Taylor-Powell, E. (1998). Questionnaire Design: Asking questions with a purpose.
University of Wisconsin Extension.

15
SECTION 1: TYPES OF SURVEY QUESTIONS

If you’re sure that a questionnaire is the most appropriate method for your research, you
need to decide whether you intend to construct a closed-ended, open-ended or combination
questionnaire. In open questions respondents use their own words to answer a question,
whereas in closed questions prewritten response categories are provided.
Open- ended questions
Open-ended questions are those that do not place restrictions on the answers respondents
can provide. One such example is, “What are the biggest challenges you face when disciplining
your child?” This question allows participants to answer in their own words. As a result, open-
ended questions yield more varied responses than closed-ended questions, and may highlight
responses that evaluators could not have anticipated. However, the information yielded from
open ended questions takes much longer to read through and to code in order to identify
common responses, which could slow down the reporting of results. As such, one must weigh
the importance of free expression of responses against the time required to get useful
information from each question.
In addition, because open-ended questions capture only one person’s experience on a given
question, it is difficult to report results for the entire group. This is due to the many possible
interpretations of each respondent’s views, and the inability to state with certainty that each
participant’s interpretation of both the question and their responses are consistent. However,
open-ended data can be useful when exploring the range of possible responses to a question
or topic.
Close-ended questions
Another type of question is a close-ended question in which respondents must choose among
specific response options for each question. For example, when asking “What are the biggest
challenges you face when disciplining your child?” a close-ended question may offer the
following choice of answers: 1) getting my child to listen; 2) losing my temper; or 3) there are
no challenges to disciplining my child. There are many possible ways to structure responses
to close-ended questions, including forced choices, agree/disagree and Likert scales, which
will be discussed later.
Advantages of close-ended questions are that carefully chosen response options allow for the
same frame of reference for all participants when choosing an answer. The answers to a close-

16
ended question are pre-determined, and as a result, they are both more specific than open-
ended questions and are more likely to promote consistency among respondents in terms of
understanding both the question and responses.
For example, Converse and Presser (1986) provide a classic scenario: consider the open-
ended question, “People look for different things in a job. What would you most prefer in a
job?” They found that the most common answer was “pay.” However, upon further analysis
it appeared that some respondents meant “high pay” and some meant “steady pay.” Because
the wording of answer choices was not specific enough, it was not possible to know which
person meant which. Contrast this with a closed-ended question, “People look for different
things in a job. Which one of the five following things would you most prefer in a job?” that
offers the following response choices:
1. work that pays well;
2. work that I am proud of;
3. autonomy;
4. friendly co-workers; or
5. steady pay/job security.
Because the respondents who meant “steady pay” chose “steady pay/job security” and the
others who meant “high pay” on the open- ended question could choose “work that pays
well” the distinction among them was apparent based on the answer choices available for the
question. As a result, areas in which an answer to an open-ended question can be unclear are
easily solved by using a closed-ended question with greater specificity in response choices. It
is the greater specificity and consistency yielded by closed-ended questions that allows one
to generalize results that can be reported across respondents. In addition, the use of closed-
ended choices allows for a timelier and more systematic analysis of data collected. Instead,
one only needs to compute frequency of responses for each question, or means on a Likert
scale to gauge impact of the program itself. Disadvantages are that there is no room for
participants to express responses in their own words, as they must use only pre-determined
choices to answer a question.
When deciding whether to use open- or closed-ended questions, it is important to think about
your goals. A general rule of thumb to follow is – if a researcher knows the specific information
needed to answer a question – and requires a single frame of reference among respondents,
closed-ended responses are preferred. If however, a researcher is not sure what the range of

17
possible responses are to a question, and hopes to conduct a preliminary exploration of a
topic, open-ended questions will work better (Colosi, 2006).

EXERCISES
1. Read the text and complete the following table with NO MORE THAN THREE WORDS from
the text.
Open questions Closed questions
Do not place (1) ___________ on the Respondents must choose among specific (4)
answers respondents can provide. ____________.
Difficult to (2) ___________ for the entire Easy to code as it allows for the same (5)
group. _____________ for all participants
Can be useful when exploring the range of There’s no place for participants to (6)
(3) ______________ to a topic. ____________ in their own words.

SECTION 2: WORDING AND STRUCTURE OF QUESTIONS

When constructing each question think about the words you use. Avoid jargon and technical
terms whenever possible. Try not to use words which may have a double meaning or be
misinterpreted, as some words have different meanings for different groups of people. Don’t
use emotive words. Make sure the question is not ambiguous. And, above all, avoid questions
which will cause annoyance, frustration, offence, embarrassment or sadness. You should
never make someone feel uncomfortable, for whatever reason, as a result of filling in your
questionnaire.
Questions should be kept short and simple. This will avoid many of the problems outlined
above. Check that a question is not double-barrelled, that is, two questions in one. If it is, ask
two questions rather than one. Also, avoid negative questions – the type which have ‘not’ in
them as this can be confusing, especially when a respondent is asked to agree or disagree.
Make sure that your questions don’t contain some type of prestige bias. This phrase refers
to questions which could embarrass or force respondents into giving a false answer. They
might do this if they do not want to look ‘bad’ in front of the researcher, or they might do it
because it is expected behaviour. Questions about income or educational qualifications might

18
illicit this type of response, so you need to be careful about how you try to obtain this
information.
Some issues may be very sensitive and you might be better asking an indirect question
rather than a direct question. Promising confidentiality and anonymity may help, but many
respondents can, understandably, be skeptical about these promises. If you ask an indirect
question in which respondents can relate their answer to other people, they may be more
willing to answer the question.
Using closed-ended questions
If you are constructing a closed-ended question, try to make sure that all possible answers
are covered. This is particularly important for time and frequency questions such as ‘how
often do you...’ You need to make sure that all the frequencies are covered so that
respondents aren’t constrained in their answers and tick a box which isn’t right for them. Also,
you want to make sure that you don’t artificially create opinions by asking someone a
question about which they don’t know, or don’t care. You need to make sure that you include
a ‘don’t know’ category in this case.
Example
Jack is a self-employed IT man, who works on his own in an office based at home. He was
recently asked to fill in a questionnaire that included the following question:

Do your work colleagues swear . . . ?

- More than you?
- About the same as you?
- Less than you?
- They don’t swear.

This question is irrelevant to Jack’s work situation because he does not have work colleagues.
Moreover, it also made him feel that the whole questionnaire was not relevant and;
therefore, he threw the questionnaire into the bin.
If you expect people to take the time to fill in your questionnaire, you must make sure that
they can answer the questions and that they feel they are relevant and worthwhile. Not only
does this question presume that the respondent has work colleagues, but it also presumes
that the respondent swears, which can cause offence to some people.

19
Avoiding leading questions
Don’t asking leading questions. The question ‘How often do you wash your car?’ might seem
innocuous enough. However, it makes two assumptions. Firstly, it assumes that the
respondent has a car and secondly, it assumes the respondent washes his car. It could be
considered a prestige bias question. Would a respondent feel bad if they didn’t have a car
and therefore would tick ‘four times a week’ anyway? Would they feel bad if they don’t ever
wash their car but feel the researcher expects them to? If you need to ask this question, you
should ask a filter question first to find out whether the respondent actually owned a car.
Then you would need to ask: ‘If you wash your car, how many times a year?’ By wording the
question in this way and by being careful about the frequency list, you’re not leading the
respondent into answering in a certain way.

EXERCISES

1. Read the following questions and decide what is wrong with them. Think about how
you might overcome the problems you have identified.

1. Do you go swimming?
- Never
- Rarely
- Frequently
- Sometimes
2. What do you think about the Green Peace attempt to blackmail the Government?
3. What is wrong with the young people of today and what can we do about it?
4. How many books have you read in the last year?
- None
- 1-10
- 10 -20
- 20 -30
- Over 30
5. What is the profit of your company, to the nearest one hundred pounds?
6. What do you think should be done about global warming?

SECTION 3: PRACTICAL CONSIDERATIONS WHEN CONSTRUCTING A QUESTIONNAIRE

3.1 Length and ordering of questions

When you’re constructing a questionnaire, keep it as short as possible. If it has to be longer
because of the nature of your research, think about whether your respondents will actually
take the time to fill it in. Some people will do so if they feel there is some personal benefit to

20
be gained. This is why long consumer behaviour surveys offer entry into large prize draws for
completed questionnaires. If your budget is limited, you might be able to offer a copy of the
final report or other information which may be of use to the respondent as an incentive.
Be realistic about how long a questionnaire will take to fill in. Saying it will take a ‘moment’ is
not helpful. Include filter questions with answers such as ‘If no, go to question 28’.
Psychologically it’s good for respondents to be able to jump sections as it stops people
becoming frustrated by unnecessary or irrelevant questions.
When designing a questionnaire, it is also a good idea to start with easy questions which
respondents will enjoy answering, thus encouraging them to continue filling in the
questionnaire. If you begin with complex questions which need long responses, your
respondents will be less likely to fill in the form. If you’re constructing a combined
questionnaire, keep your open-ended questions for the end as, once someone has spent time
completing the rest of the questionnaire, they are more likely to continue with those
questions which take a little more effort to complete.
When constructing a questionnaire, you need to make it as interesting as possible and easy
to follow. Try to vary the type and length of questions as variety provides interest. Group the
questions into specific topics as this makes it easier to understand and follow. Layout and
spacing are extremely important. If your questionnaire looks cluttered, respondents will be
less likely to fill it in.
3.2 Collecting personal information
Opinion is divided about where personal information should be included on a questionnaire.
Some believe that this may better be put at the end, as people are more likely to fill in this
information when they have already invested time and energy in completing the rest of the
form.
As the researcher, you need to think about collecting only that type of personal information
which is completely relevant to your research. Be sensitive to the type of information people
will be reluctant to give. It is important not to ask for too much personal information as
respondents will become suspicious and want to know why you want the information. You
also need to assure them that their personal information will be treated confidentially.
3.3 Piloting the questionnaire
Once you have constructed your questionnaire, you must pilot it. This means that you must
test it out to see if it is obtaining the results you require. First of all, ask people who have not

21
been involved in its construction to read it through and see if there are any ambiguities which
you have not noticed.
Once this has been done, alter the questions accordingly, then send out a number of
questionnaires to the type of people who will be taking part in the main survey. Make sure
they know it is a pilot test and ask them to forward any comments they may have about the
length, structure and wording of the questionnaire. Go through each response very carefully,
noting comments and looking at the answers to the questions as this will help you to discover
whether there are still ambiguities present.
Alter the questionnaire again. If you have had to undertake major alterations, you may need
to pilot the questionnaire again. This may seem a rather long and laborious process, but it is
incredibly important, especially if you’re intending to send out a large number of
questionnaires.
(*) Refer to Appendix 4.2 for a Questionnaire Design Checklist

EXERCISES
1. Decide if the following statements are True or False. Tick the appropriate column.

True False
1. It is not acceptable to give incentives to people who
completed a questionnaire.
2. Filter questions may help direct respondents to
questions that are more relevant to them.
3. It’s a good idea to vary the type of questions in the
questionnaire.
4. The questions in the questionnaire should be of the
same length.
5. Grouping the questions into specific topics make it
easier for respondents to understand and follow.
6. It’s a good idea to put complex questions at the
beginning of the questionnaire.
7. Questions related to personal information should be
placed at the beginning of a questionnaire.
8. A questionnaire should collect as much personal
information about the respondents as possible.
9. The questionnaire should be piloted among the
designer(s)/ developer(s) before being sent to potential
participants.

22
2. Do a little Internet research to find out what a Likert scale is and when you may use
one.

3. Write a closed-ended question that follows the guidelines for good survey question
construction. Have a peer in the class check your work (you can do the same for him or
her!).

KEY TAKEAWAYS

• Brainstorming and consulting the literature are two important early steps to take
when preparing to write effective survey questions.
• In open-ended questions, respondents use their own words to answer a question,
whereas in closed-ended questions, prewritten response categories are provided.
• Make sure that your survey questions will be relevant to all respondents and that you
use filter questions when necessary.
• Collect only personal information that is completely relevant to your research.
• Getting feedback on your survey questions is a crucial step in the process of
designing a survey.
• Piloting is an important step for improving one’s survey before actually administering
it.

23
 APPENDIX 4.1

QUESTIONNAIRE DESIGN CHECKLIST

▪ Make your questionnaire as short as possible.
▪ Make sure people will be able to answer your questions. Don’t assume knowledge or
make it seem that you expect a certain level of knowledge by the way your questions
are worded.
▪ Start with easy to answer questions. Keep complex questions for the end.
▪ Ask for personal information at the end.
▪ Use a mix of question formats.
▪ Don’t cause offence, frustration, sadness or anger.
▪ Avoid double-barrelled questions.
▪ Avoid words with emotional connotations.
▪ Avoid negative questions.
▪ Avoid jargon and technical words.
▪ Avoid words with multiple meanings.
▪ Avoid leading questions.
▪ Avoid vague words such as ‘often’ and ‘sometimes’. Provide all possible responses in
a closed question. Consider as many alternatives as possible.
▪ Use specific time frames when asking about behaviour. Use specific place frames, e.g.
‘In which country were you born?’

24