RESEARCH METHODOLOGIES

Lecture 065
Population, Sampling & DATA COLLECTION
Focus on Questionnaires
Dr. Mokhtar Ali Amrani
Introduction
• Improving data collection techniques will enhance the
accuracy, validity, and reliability of research findings.

• Regardless of the methods chosen, quality research

includes collecting quality data.

• A researcher should Be neutral for collecting data.

• Participant observation, face-to-face interviews, focus-

group interviews, questionnaires, and member checking
were used as data collection methods.

• The prepared questionnaire or interview should be checked

for:
• 1.Wording 2.Format 3.Flow 4.Duration
 Research Methodology
The research method is a strategy of enquiry, which moves from the
underlying assumptions to research design, and data collection (Myers,
2009).

Although there are other distinctions in the research modes, the most
common classification of research methods is into qualitative and
quantitative.

Quantitative research methods were originally developed in the natural

sciences to study natural phenomena. Qualitative research methods were
developed in the social sciences to enable researchers to study social and
cultural phenomena.

Both quantitative and qualitative research studies are conducted in

education. Neither of these methods is intrinsically better than the other; the
suitability of which needs to be decided by the context, purpose and nature
of the research study in question.
Research Methodology
Some researchers prefer to use mixed methods approach by taking advantage of
the differences between quantitative and qualitative methods, and combine
these two methods for use in a single research project depending on the kind of
study and its methodological foundation (Brysman and Burgess, 1999, p. 45).

An obvious basic distinction between qualitative and quantitative research is the

form of data collection, analysis and presentation. While quantitative research
presents statistical results represented by numerical or statistical data,
qualitative research presents data as descriptive narration with words and
attempts to understand phenomena in “natural settings”. This means that
qualitative researchers study things in their natural settings, attempting to make
sense of, or to interpret, phenomena in terms of the meanings people bring to
them.” (Denzin and Lincoln, 2000, p. 3).

Qualitative research are most appropriate when the researcher wants to

become more familiar with the phenomenon of interest, to achieve a deep
understanding of how people think about a topic and to describe in great detail
the perspectives of the research participants.
 Research Methodology
Quantitative researchers measure variables on a sample of subjects and express
the relationship between variables using effect statistics such as correlations,
relative frequencies, or differences between means; their focus is to a large
extent on the testing of theory.

In qualitative research, a hypothesis is not needed to begin research; It employs

inductive data analysis to provide a better understanding of the interaction of
“mutually shaping influences” and to explicate the interacting realities and
experiences of researcher and participant (Lincoln and Guba, 1985).

All quantitative research requires a hypothesis before research can begin.

Three basic purposes of quantitative research as: to describe, to compare and

to attribute causality.

The most typical application of qualitative research in instructional technology

seems to be that of “case studies”.
 For research, to discover the level of
Sampling techniques service at Yemeni restaurants throughout
a country, the population from which you
would select your sample would be all at
• Sampling techniques enable Yemeni restaurants in that country.
you to reduce the amount
of data you need to collect
by considering only data
from a subgroup rather
than all possible cases or
elements.
• The full set of cases from
which a sample is taken is
called the population . In
sampling, the term
‘population’ is not used in
its normal sense
 The need to sample
Sampling provides a valid alternative to a census
(‫ )تعداد عام للسكان‬when:
1. it would be impracticable for you to survey
the entire population;
2. your budget constraints prevent you from
surveying the entire population;
3. your time constraints prevent you from
surveying the entire population.

testing an entire population of products to destruction, such as to establish the

actual duration of long-life batteries, would be impractical for any manufacturer.
An overview of sampling techniques
Sampling techniques available to you can be
divided into two types:
1. probability or representative sampling;
2. non-probability sampling.
probability samples (representative sampling)
• With probability samples the chance, or probability,
of each case being selected from the population is
known and is usually equal for all cases.
• you need to make inferences (‫ )استدالالت‬from your
sample about a population to answer your research
question(s) and to meet your objectives and to
estimate statistically the characteristics of the
population from the sample.

• Probability sampling is often associated with survey

and experiment research strategies.
probability samples (representative sampling)
The process of probability sampling can be divided
into four stages:
1. Identify a suitable sampling frame based on
your research question(s) and objectives.
2. Decide on a suitable sample size.
3. Select the most appropriate sampling
technique and select the sample.
4. Check that the sample is representative of the
population.
Ahmed selected his classmate as a sample to estimate the life of acid batteries in
Taiz city.
Sampling Frame (Examples)
• If your research question or objective is concerned with
members of a student society, your sampling frame will be
the complete membership list for that society.
• If your research question or objective is concerned with
registered childminders (‫ )روضة أطفال‬in a local area, your
sampling frame will be the directory of all registered
childminders in this area.
• If your research question or objective is concerned with
registered Kidney Failure in Taiz city, your sampling frame
will be the directory of all registered Kidney Failure in Taiz
city only.
Obtaining a sampling frame is therefore essential if you
are going to use probability sampling.
It is important to ensure that your sampling frame is as
complete, accurate and up to date as possible.
Deciding on a suitable sample size
• The larger your sample’s size the lower the
likely error in generalizing to the population.

• Probability sampling is a compromise

between the accuracy of your findings and the
amount of time and money you invest in
collecting, checking and analyzing the data.
Deciding on a suitable sample size
Your choice of sample size within this compromise is governed by:
1. the confidence you need to have in your data – that is, the level
of certainty that the characteristics of the data collected will
represent the characteristics of the total population;
2. the margin of error that you can tolerate – that is, the accuracy
you require for any estimates made from your sample;
3. the types of analyses you are going to undertake – in particular,
the number of categories into which you wish to subdivide your
data, as many statistical techniques have a minimum threshold
of data cases for each cell.
4. the size of the total population from which your sample is being
drawn.
Table 7.1 provides a rough guide to the different minimum sample sizes required
High Response Rate
The American Association for Public Opinion Research
(2008) defines four levels of non-response that can be
reported for questionnaires and structured interviews:
 complete refusal : none of the questions answered;
 break-off : less than 50 per cent of all questions
answered other than by a refusal or no answer (this
therefore includes complete refusal);
 partial response : 50 per cent to 80 per cent of all
questions answered other than by a refusal or no
answer;
 complete response : over 80 per cent of all questions
answered other than by a refusal or no answer.
Estimating response rates and actual
sample size required
 Secondary Data
• Students are increasingly expected to consider
undertaking further analyses of data that have
already been collected for some other purpose.
Such data are known as secondary data and
include both raw data and published summaries.
• These data can be further analyzed to provide
additional or different knowledge, interpretations
or conclusions
Research Design
Research design can be thought of as the logic or master plan of a research
that throws light on how the study is to be conducted.

It shows how all of the major parts of the research study– the samples or
groups, measures, treatments or programs, etc– work together in an attempt
to address the research questions. Research design is similar to an architectural
outline.

The research design can be seen as actualization of logic in a set of procedures

that optimizes the validity of data for a given research problem.

According to Mouton (1996, p. 175) the research design serves to "plan,

structure and execute" the research to maximize the "validity of the findings".
The Case study strategy
Case study is an empirical inquiry that investigates a contemporary
phenomenon within its real-life context, especially when the boundaries
between phenomenon and context are not clearly defined. The case study
approach is especially useful in situations where contextual conditions of the
event being studied are critical and where the researcher has no control over
the events as they unfold.

The case may also be a program, an event, or an activity bounded in time and
place. According to McMillan and Schumacher (2001), a case study examines a
bounded system or a case over time in detail, employing multiple sources of
data found in the setting. All the collected evidences are collated (‫ )تقارن‬to arrive
at the best possible responses to the research question(s).

it allowed for the adoption of both qualitative and quantitative data collection
methods which presents a more coherent picture of a unique situation. The case
study approach makes use of multiple methods of data collection such as
interviews, document reviews, archival records, and direct and participant
observations and subsequently ‘thick descriptions’ of the phenomena under
study (Yin, 2003).
DATA COLLECTION METHODS
• Primary data refer to information obtained firsthand by the researcher
on the variables of interest for the specific purpose of the study.
• Secondary data refer to information gathered from sources already
existing
• Focus groups consist typically of 8 to 10 members with a moderator
leading the discussions for about 2 hours on a particular topic, concept,
or product.
• focus groups are used for (1) exploratory studies, (2) making
generalizations based on the information generated by them, and (3)
conducting sample surveys.
• Panels, like focus groups, are another source of primary information for
research purposes. Whereas focus groups meet for a one-time group
session, panels (of members) meet more than once

Slide 14-27
DATA COLLECTION METHODS
• Panels can be either
• Static panels (i.e., the same members serve on the panel over
extended periods of time) or
• Dynamic panels (i.e., the panel members change from time to time as
various phases of the study are in progress).
• The main advantage of the static panel is that it offers a good and
sensitive measurement of the changes that take place between two
points in time—a much better alternative than using two different
groups at two different times.
• The disadvantage is that the panel members could become so sensitized
to the changes as a result of the endless continuous interviews that their
opinions might no longer be representative of what the others in the
population might hold. Members could also drop out of the panel from
time to time for various reasons, thus raising issues of bias due to
mortality.

Slide 14-28
DATA COLLECTION METHODS
• The advantages and disadvantages of the dynamic panel are
the reverse of the ones discussed for the static panel.
• Trace measures, or unobtrusive (‫ )غير بارز‬measures as they
are also called, originate from a primary source that does
not involve people
• The advantage of seeking secondary data sources is savings in
time and costs of acquiring information.

Data collection methods are an integral part of research design.

Slide 14-29
Types of Interviews
 Interviews
• Data collection methods are an integral part of research
design
• Unstructured interviews are so labeled because the
interviewer does not enter the interview setting with a
planned sequence of questions to be asked of the
respondent.
• The objective of the unstructured interview is to bring some
preliminary issues to the surface, so that the researcher can
determine what variables need further in-depth investigation
• Structured interviews are those conducted when it is known
at the outset (beginning) what information is needed.

Slide 14-31
 Interviews
• The main purpose of the unstructured interview is to explore
and probe into the several factors in the situation that might
be central to the broad problem area.

• Disadvantages. The main disadvantages of face-to-face

interviews are the geo- graphical limitations they may
impose on the surveys and the vast resources needed if
such surveys need to be done nationally or internationally.

Slide 14-32
Example of Interviews
QUESTIONNAIRE
• Definition
A questionnaire is a pre-formulated, written set of
questions to which the respondent records his answers.

• Questionnaire has the advantage of taking it to a wider

audience compared to interviews, but has a disadvantage of
not being possible to customize it to individuals as it is
possible with other methods of data collection.
• Steps
1. Determine the content of the questionnaire
2. Determine the form of response
3. Determine the wording of the questions
4. Determine the question sequence
5. Write cover letter
Questionnaires
Table 9.1 Advantages and Disadvantages of Different Questionnaires
Mode of data Advantages Disadvantages
collection
Personally  Can establish rapport and motivate respondent.  Explanations may introduce a bias ( ‫محاباة او‬
administered  Doubts can be clarified. ‫)نزعة‬.
questionnaires  Less expensive when administered to groups of  Take time and effort.
respondents.
 Almost 100% response rate ensured.
 Anonymity of respondent is high.

Mail  Anonymity is high. (‫)اسم مستعار‬  Response rate is almost always low. A 30%
questionnaires  Wide geographic regions can be reached. rate is quite acceptable.
 Token gifts can be enclosed to seek compliance.  Cannot clarify questions.
 Old-fashioned and. Obsolete. (‫)طريقة عفى عليها الزمن‬  Follow-up procedures for nonresponses are
necessary.
 Respondent can take more time to respond at
convenience. Can be administered electronically, if
desired.

Electronic  Easy to administer.  Computer literacy is a must.

questionnaires  Can reach globally.  Sampling issues.
 Easy to access specific target groups.  High non-response.
 Very inexpensive.  Not always possible to generalize findings.
 Fast delivery.  Respondent must be willing to complete the
 Respondents can answer at their convenience like survey.
the mail questionnaire.  People find invitations via e-mail rude and
 Automatic processing of answers. offensive; mails are deleted or people
complain.
Slide 9-39
1. Questionnaire Content
• Framework
Need information for all constructs in framework

• Measurement: Operationalizing
– Objective construct:
• 1 element/items
=> 1 question
– Subjective construct:
• multiple elements/items
=> multiple questions

Slide 9-40
 2. Response Format

• Closed vs. Open-ended questions

– Closed questions
• Helps respondents to make quick decisions
• Helps researchers to code
– Open-ended question
• First: unbiased point of view
• Final: additional insights
• Complementary to closed question: for
interpretation purpose

Slide 9-41
3. Question wording
• Avoid double-barreled questions

• Avoid ambiguous questions and words

• Use of ordinary words

• Avoid leading or biasing questions

• Social desirability

• Avoid recall depended questions

Slide 9-42
Question Wording
• Use positive and negative statements
– Dresdner delivers high quality banking service
Dresdner has poor customer operational support
– Avoid double negatives

• Limit the length of the questions

Rules of thumb:
– < 20 words
– < one full line in print

Slide 9-43
4. Question Sequence

Personal and sensitive data at the end

Slide 9-44
5. Cover Letter

• The cover letter is the introductory page of the

questionnaire

• It includes:
– Identification of the researcher
– Motivation for respondents to fill it in
– Confidentiality
– Thanking of the respondent

Slide 9-45
Example: Cover letter