Doing Research 43 Page
Doing Research 43 Page
Doing Research 43 Page
CH
how do you design a research project, what steps do you have to take, how do
you collect data and what are the pitfalls?
AR
and evaluation and recommendations. Many up-to-date examples are used to
establish the link to everyday practice.
This 5th edition has been completely revised. The various research steps are
presented in clear procedure guides, so that it is easy to see where you are in the
research process. Each chapter also has checkpoint assignments to test whether
The
and How
SE
you have understood the theory correctly. You can find practical tools, tests and
answers to the checkpoint assignments on www.doingresearch5thedition.com.
of A hyW s
VERHOEVEN
RE
but it can also be used as a reference book for applied research.
ear d
G
workshops and courses. She is the author of several 5TH
successful teaching methods in the field of methodology. EDI
ch
EN
TION
EV
IN
HO
ISBN 978-90-2442-475-7
ER
DO
LV
NE
www.doingresearch5thedition.com
9 789024 424757 www.boomhogeronderwijs.nl
Nel Verhoeven
Fifth edition
Subject to the exceptions set out in or pursuant to the Dutch Copyright Act [Auteurswet], no part of this
book may be reproduced, stored in a database or retrieval system, or published in any form or in any way,
electronically, mechanically, by print, photoprint, microfilm or any other means without prior written
permission from the publisher.
Insofar as the making of copies of this edition is allowed on the basis of Section 16h of the Dutch Copyright
Act, the legally due compensation should be paid to the Dutch Reproduction Rights Association [Stichting
Reprorecht] (P.O. Box 3051, 2130 KB Hoofddorp, The Netherlands, www.reprorecht.nl). For the inclusion
of excerpts from this edition in a collection, reader and other collections of works (Section 16 of the Copy
right Act 1912) please contact the Dutch Publication and Reproduction Rights Organisation [Stichting
Publicatie- en Reproductierechten Organisatie] at P.O. Box 3060, 2130 KB Hoofddorp, www.stichting-pro.
nl).
No part of this book may be reproduced in any form, by print, photoprint, microfilm or any other means
without written permission from the publisher.
We have done our best to find all copyright owners of photographic and other material in this publication.
We ask anyone who believes that his or her material has been used in this work without prior permission
to please contact us.
ISBN 978-90-2442-4757
ISBN 978-90-2442- 4764 (e-book)
NUR 741
www.boomhogeronderwijs.nl
www.doingresearch5thedition.com
The fifth edition of Doing Research has been laid out differently. The reasons
for this are the changing landscape within higher education, the personal and
interactive approach to research education and the changing exit qualifications
for graduates. Whereas fifteen years ago the emphasis was on having as many
research skills as possible, and producing what to all intents and purposes is an
academic graduation thesis, nowadays the emphasis is much more on acquir-
ing skills and producing a professional product. Research skills still come in
handy in this respect. In fact, they are indispensable if the student is to produce
a good professional product. That said, the role and place of research skills are
evolving to become an instrument for addressing questions in practice. This is
a great place to have in tertiary education, if you ask me. And it’s the right place
too. It means having a different range of research skills, and for that the book
needed a new layout, one with many short chapters. Unlike previous editions,
each chapter deals with one subject only. The four phases of applied research
are still the guiding principle.
For the rest, much attention has been paid to case studies in this edition. Apart
from many new examples, the associated website has been changed.
New tools are available in this edition, both on the website and in the book. For
example, checkpoints are included in the chapters; these are short knowledge
questions that students can use to test whether they have understood the text
correctly. The book also contains procedure guides, which help to stick to the
right order and make choices while designing and carrying out research.
The new edition of Doing Research is even easier to navigate, and is hands-on,
with lots of infographics and visuals, and short, inspiring texts.
Many thanks
Many people were involved in the compiling of all editions of this book. When
I wrote the first edition, Bob Bouhuijs of Windesheim Christian University,
Annete Bogstra of University College Utrecht and Jan van Leeuwen of Fontys
University Eindhoven all gave me advice. Jan Willem Zeijseink advised me on
Chapter 6 and Rika Verhoef did the same for parts of Chapter 7. For the more
recent editions, Peter Swanborn, Siep van der Werf of the University of Applied
Sciences Amsterdam, and Anya Luscombe all went through the text meticu-
lously and put forward their suggestions for changes. For the fourth edition,
Mirca Groenen gave detailed editorial comments and went through the words
for the index register with a fine-toothed comb. For the fifth edition, Desiree
Joosten-ten Brinke was helpful with advice about peer feedback. I am very
grateful to Lineke Oppentocht and Mieke van Dalen for the critical opinions
they gave me while I was writing the fifth edition. As a result, I have looked at
my texts with different eyes.
Many teachers took the trouble to complete a questionnaire about the book
and to provide me with their useful suggestions. Finally, I would like to thank
Esther den Hollander, Astrid van der Schee, Elsbeth Bouman and Gerdi Smit
for their quick and useful advice and of course for their unflagging support
during the writing process.
Happy reading!
Nel Verhoeven
Ovezande, January 2019
Part 1 Design15
6 Research proposal 95
6.1 Looking back on the design process 95
6.2 Starting points for the research proposal 97
6.3 From central question to data collection 99
6.4 Compiling a research proposal 104
6.5 Time schedule 106
10 Sampling 179
10.1 Population and sample 179
10.2 Random samples 183
10.3 Random samples 186
11 Operationalization 191
11.1 Operationalization, what is it? 191
11.2 Designing questionnaires 194
11.3 Determining interview topics 203
Part 3 Analysis233
Index 391
Sources 403
3 Look at the answers to the questions on the website under in the ‘check-
points’;
5 Check whether you have mastered the material properly by going through
the learning objectives again.
If you have finished all chapters of a part, you go to that part on the website.
You can then do the following:
Method structure
The research procedure can be divided into research d
t 4 n antions On PDae
o da Dtew r
phases Doing Research is based on four phases:
reco valua ar
n
elt 1pnen
mm ti
P
1
e
esrig
1. Design
E
2. Data collection
3. Analysis
t2
r
Pa a
t Pa a
An
n
ly 3
e c ta
r
tio
s is D
4. Evaluation and recommendations co
ll
Procedure guide
To help you find your way through the various
7 Quantitative data
collection methods
Select and put forward arguments to
substantiate quantitative data collection method:
in the book:
Research quality
9 Check reliability of the research design
Check validity of the research design
Check usability of the research design
Operationalization
11 Quantitative: Qualitative:
The chapters of that part are mentioned in the procedure guide. For each chap-
ter, you will find an overview of the most important steps that are discussed
in that chapter. At the end of each chapter, we show you a piece from the pro-
cedure guide, namely the chapter that you’ve just been through. This way you
always know exactly where you are in the process of doing research.
Recurring components
In each chapter of the book you will come across several set elements:
What experiences do volunteers and customers have of the handing out of food at the
food bank? One hundred and thirty volunteers and customers at the Utrecht Food bank
were given a written questionnaire. Of these, 39 were completed and returned. That is a
response rate of 30%. Are the results usable? Yes! Statistical generalization is not an abso-
lute must. For the food bank, the results are very valuable because with Utrecht Food
Bank can use them to improve the system they have in place for handing out food.
Checkpoint 5.1
Key concepts
Central question The central question that you aim to answer with your
research.
Website
You need the book’s website so that you understand and process the material
properly. You can access the website doingresearch5thedition.com using the
unique activation code on page 4 of this book.
The website is structured in the same way as the book: in four parts. Each part
covers the following aspects:
Extra assignments – questions with feedback so that you can check your
progress
Concepts trainer – to test your knowledge per section on all key concepts and
many other concepts discussed in the book
Research tools – overviews, templates and checklists that you can use in the
various research phases
Research proposal
6
Write and present the research proposal
Draw up a schedule
Process feedback
All important topics for this first research phase are discussed in six chapters.
Chapter 1 is an introduction. You learn what research is, what you need it for,
what the basic principles of research (methodology) are, and what characteris-
tics a good researcher and good research have. The chapter also discusses the
four research phases.
Chapter 2 deals extensively with the contacts with the client, and selecting and
determining a subject.
Chapter 3 is about the background to your research and what you have to do to
prepare this background: a bit of preliminary research. Here you will be given
a handy tool to help you write the background: the 6W method. In that pre-
liminary research, you will look for information about the subject you have
chosen. In this chapter you will get information about the options you have to
make your search easier, and you will learn how to use a logbook as an aid in
your research.
Chapter 5 is the chapter that discusses demarcation, the theory and model
building. You will be taught how to demarcate the subject of your project, how
to define the most important concepts and – again – how you can use prelimi-
nary research for this. You do this preliminary research to gather information
about previous research results, and to look for theoretical principles. You will
be taught how best to go about designing a research model. You are introduced
to the search procedure to find theoretical or model-based principles that you
can use. You then put together the theoretical chapter.
Chapter 6 is about your research proposal. The conditions that proposals must
meet vary according to the type of study. This chapter gives you a general over-
view of the various components that constitute a research proposal for most
studies.
If you go back a page, you will see the procedure guide for Part 1 of your
research: the design. This way you can see at which points in time during your
research certain parts are dealt with, and where you have to make important
decisions. At the end of each chapter, we show you a piece from the procedure
guide, namely the chapter that you’ve just been through.
Doing research
Doing research is the analyzing of a problem or a situation according to a spe-
cific phased plan. You do it using tools that have been developed for this. By
approaching the situation or problem in this way, you can answer questions
and solve problems.
Working according to a phased plan is a systematic approach. The tools you use
here can be compared with tools in a toolbox. Everything that you need is in
that toolbox. All you have to do is unpack it. Needless to say, you have to have
the right tools.
You start with an idea for a project, or you have a question from your profes-
sional field. You elaborate this idea or question in greater detail, you look for
sources, you formulate the key question for your research and you make a plan.
You develop certain expectations about the results. You then start collecting
and processing information. You supervise the project throughout, and keep
an eye on the quality. Then you evaluate it, and finally you report on the results
and present your recommendations.
When you’re on a journey, you make similar choices. First you choose your
destination. Then you find out about the best way to travel, and you make a
plan. You plan your whole trip based on your expectations, with an expected
arrival time and a few stopovers along the way. You gather your information
about departure times, hotels, travel times and so on. During the journey you
keep checking whether you’re still on the right route, whether you’re on time
and whether all the various parts of your travel plan are still on track. At the
end of your journey, you will know whether your plan was a good one. You look
back (evaluation) and you look forward (recommendations). You also assess the
quality of your journey. What went well, and what would you do differently or
better next time?
A research journey is not one-way traffic. You don’t only assess your own
research process. You also share your experiences with peers (fellow travellers);
by studying one another’s methods, you learn from one another.
Research in steps
Maybe you don’t realize it, but if you have a problem or a question, you often use
research techniques to find the answer. Have a look at Example 1.1:
A famous pop group is coming to town. They’re playing at Madison Square Garden. You
and your friends go down to the box office to buy tickets. The queue is around the block!
What if the tickets are sold out! You look at the queue and the number of people waiting
at the box office. You take the shortest queue.
• Your question is: how can you get to the ticket office in the shortest possible
time?
• To answer your question, you observe the queues. Then you count the num-
ber of people waiting.
• You conclude that the waiting time will be shortest in the shortest queue.
• You go to the queue with the fewest people.
In the Doing Research method, you gain experience by applying the logical
sequence of each research project. You learn which steps you need to take for
each research project. The steps are the same everywhere: whether you are try-
ing to predict the waiting time at a box office, or you are trying to design a
complicated research project about the causes of a certain disease.
Simple or complex
The example about the queue at the box office is simple. But if you’re doing
research for a client, there’s much more to it than that. Your research design is
generally much broader. Many people may be involved in the research project,
you will be conducting extensive and complicated analyses, you will be writ-
ing a detailed research report, and you will be presenting it to the client. Also,
quite often you have to make recommendations, or present a design, a plan or
measures. That said, whether research projects are complicated or simple, they
follow the same pattern.
If you want to learn how to do research, you have to do more than merely acquire
knowledge from a book. A book can teach you how to set up a plan, design a
research project or define a central question. You can also learn how to carry
out an analysis and how you compile a report and account for the findings. But
when you conduct research, you have to be able to combine the knowledge and
skills that you’ve acquired; you have to develop a kind of helicopter view of your
research. Doing research has much more to do with skill than with knowledge.
In brief, you have to get experience. You learn research by doing it.
Informal or systematic
Is anyone who makes a casual observation automatically a proper researcher?
No, not quite. When it comes to informal observations, i.e., in daily life, if you
are not really planning to conduct research, you may use your own frame of ref-
erence to draw your conclusions. That means that you subconsciously assume
that other people do the same as you, as Example 1.2 shows. Real researchers
don’t do that. They use systematic observation. In other words, they base their
observation on a phased plan, without having preconceived ideas about the
outcome.
The weather is great, it’s the end of August and you’re on vacation. You’re visiting London
and it seems like an excellent day for a boat trip on the Thames. Once on the boat you
see a group of young Oriental people. Complete with cameras and strange shoes, they’re
jabbering in what sounds like Japanese, but then it would, wouldn’t it? Must be Japanese
tourists, you think to yourself.
Are they really?
What you’re looking at is a group of first year students who have come to study at the
London School of Economics. It’s their introduction week.
Because you yourself are on vacation, you automatically assume that the young
Japanese people are too. You use your own frame of reference to draw your
conclusions.
Practical considerations
You don’t do research in a vacuum. First you draft a research plan; you define
the central question; you check whether anyone else has researched the prob-
lem and what their conclusion was; you set a deadline and you draw up a budget
to see what is necessary (and available) for carrying out the study; you consult
your supervisor, your client, and your co-researchers. These are the practical
aspects of research.
Markers
Besides these practical aspects, there are other, underlying approaches that
can be used to classify research. This is known as methodology. Based on these
approaches, you can describe your research in various ways; they are the mark
ers of the research. For instance, there’s a difference between fundamental and
applied research, between qualitative and quantitative research and between
induction and deduction. These are described in the following sections.
The Doing Research method is mainly about designing and conducting applied
research, but it is also important to know something about the theory under
lying it. Knowing about these basic research norms is crucial for a good under-
standing of the function of research.
In principle, and according to the science of methodology, there are two main
types of research: fundamental (empirical) research and applied research. Uni-
versity students are more often than not confronted with fundamental research,
like the research in Example 1.3. College students, on the other hand, mainly
carry out.
The main difference between these two approaches is the type of problem to
be solved. The questions raised in fundamental research generally answer
questions that aim at developing knowledge. So they are theoretical questions.
Applied research, however, is all about solving problems that have a practical
application, i.e., practical questions. Sometimes the theoretical question that is
answered turns out to be a good solution for a practical problem. In that case
fundamental research is also applied research.
Nicotine is one of the most addictive substances around. Neurobiologist, Taco de Vries,
does research into this kind of addiction, in particular into how the brain reacts to nico-
tine. He does this by first establishing what nerve cells in the front of the brain (the pre-
frontal cortex) are involved in nicotine addiction. With that information, he can ascertain
in two ways whether nicotine addiction can be tackled: with medicine to combat a
relapse after stopping and with behavioral intervention, i.e., intervening to solve a prob-
lem. Taco wants to investigate the effect of these two methods on smoking addiction.
Source: www.zonmw.nl
The following section explores the difference between theoretical and practical
questions.
Of course, it may well be that fundamental research tests a theory that can
also help to solve a social problem. For instance, fundamental research into the
shifting of sandbanks in coastal areas may well have social relevance if it leads
to better protection against floods. So the difference is not very clear: knowl-
edge questions can also be answered in applied research, and practical ques-
tions are also investigated in fundamental research.
Example 1.3 outlines a fundamental research project. The question in this
research is a fundamental one. The researcher wants to test the functioning
of the brain in those who are addicted to smoking. The question also has an
applied aspect to it, because the knowledge gained from this research may
improve ways to tackle addiction.
Example 1.4 shows two reasons for doing applied research. This is a topic that
originates in society (Highlands Sports Club). The aim is to improve the organ-
izing of a running event.
For many years, Highlands Sports Club has held an international cross-country event.
Cross country involves running, not only on the roads, tracks and paths, but also across
fields and streams. The distances and the age categories vary. Every year, lots of volunteers
are involved in the event, and the race is broadcast on television. At the annual evaluation
meeting that follows, the question is always raised about what the participants, staff and
spectators thought of the race. The board wants to see the answer in figures. The board
also wants to know exactly what the
target group is for the competition in
terms of age groups. This will allow
them to keep the sponsors happy and
perhaps even attract new sponsors.
Not only that, the results of this eval-
uation can be used to improve the
organizational aspects of the event.
Research into the levels of satis
faction among the three groups of
people is carried out.
Key concepts
A research plan is drawn up to assess the level of satisfaction among the cross-country
participants (i.e., the athletes). To gather the information, they decide to carry out a sur-
vey in which about 20% of the 2,500 athletes will be interviewed. They will assess levels
of satisfaction for a number of aspects (in other words ‘parts’) by ‘rating’ them. The back-
ground characteristics of the athletes themselves will also be recorded (gender, age etc.).
Their responses will be analyzed numerically (quantitatively), by comparing the various
ratings. Differences between the ratings will be compared according the various groups
as well, i.e., young versus old, men versus women and so on.
When qualitative methods are used, the researcher carries out research in the
field, i.e., in reality. He or she is mainly interested in the meaning that a person
attaches to a situation or experience. The research subjects, i.e., the people being
researched, are studied in their environment as a whole. This is also known as
holism. This means that the experience is seen as a part of the person’s percep-
tion of their lives, and not as a separate, independent entity. This means that it
is interpretative in nature.
When you gather information for qualitative research, the methods you use
are open and flexible, and you can intervene when the unexpected happens.
The information is not recorded in numbers, but in every day language (Maso
& Smaling, 1998). In this approach, language is the tool of qualitative research
methods (see Example 1.6).
Recreational attractions and parks are often evaluated qualitatively. During a short
interview, guests can be asked what they found to be the most striking, fun, irritating or
boring part of their experience. These days, researchers often use tablets for this. Guests
are asked to write down a few sentences about what they thought of the park or attrac-
tion. The information is then analyzed qualitatively.
Some researchers view qualitative research results as less reliable than those
based on quantitative research. Other researchers believe that figures do not
offer enough insight into the issues at hand because the figures cannot reveal
the information underlying them. Qualitative researchers make a case for
methods that not only focus on the figures, but also listen to what the people
have to say (Wester, 1991). The parliamentary committee of inquiry method is
an example of this.
Imagine that you want to investigate whether there is any difference between non-verbal
aggression shown by children aged 6 and those aged 10 when they’re playing. You may
decide to observe the behavior of children in Grade 2 and those in Grade 5 during break
time, on two consecutive days. You make a note of what you see on an observation form
and then you compare what you have noted. This is qualitative research.
‘Committee of Inquiry’ is a misnomer, from the perspective of methodology at least,
because investigations of this type are actually qualitative. They are open interviews
involving a set number of people and experts, whereas if you conduct a normal survey
you present the respondents with a fixed set of questions.
Key concepts
In Doing Research you learn about qualitative and quantitative methods because
they are both used. Obviously, the difference between these methods is in the
way in which you conduct the research, but also in the way you look at research.
The emphasis in qualitative research is on the meaning that respondents give to
a situation based on their own background, say the respondent’s context (and
the researcher, more on this in Chapter 16). In quantitative research the empha-
sis is on making phenomena measurable and on generalization, i.e., objectively
measuring characteristics and drawing a valid, general conclusion. The corre-
sponding context is not measured. Qualitative research often involves collect-
ing a lot of information about a few people, while quantitative research is not
so much (or at least less) about information, but about a large group of people
instead.
Qualitative Quantitative
Emphasis on meaning/context Yes No
Number of respondents Few Many
Data per respondent A lot Not much
Type of information In-depth Superficial
Objectively measurable (numerical) No Yes
Statistically generalizable No Possibly
Yet these two methods are not in opposition; instead they can supplement each
other very well so that a comprehensive answer to the problem is given (see
Doorewaard, Kil & Van de Ven, 2015). It’s not that clear cut.
Key concepts
Mixed method approach Research that uses qualitative and quantitative methods.
When you do inductive research, the theory is not known in advance. The
researcher’s objective, therefore, is to develop the theory as he or she goes along.
In formal terms, the researcher is looking for ‘empirical regularities’ (Tijmstra
& Boeije, 2011, p. 32). This means that he or she makes statements based on the
observations, i.e., the information.
Researchers who use inductive methods often (but not always) use qualitative
research in which they work from the ‘particular’ (the information collected) to
the ‘general’ (the theory to be developed). So inductive research is about devel-
oping a theory (see Example 1.9). Iteration (repetition) is a guiding principle in
this because it leads to a higher standard of results. This is the process that the
researcher follows: he or she gathers and analyzes the information and draws
the first conclusions. This establishes the kind of additional information that is
required. The researcher then goes on to gather and analyze new information.
In other words, he or she is using an iterative process. Each time the researcher
links the results to the results obtained previously and so a theory is formed.
More about this in Chapter 6.
Suppose you are researching social media usage among students at a college in Michi-
gan. You collect information using interviews, you analyze the information and you dis-
cover that a degree of self-motivation arises. You go on to collect additional information
about self-motivation, you analyze the information once again and you draw conclusions
about the students’ attitudes to work. With these conclusions, you present a model for
the digitizing students’ attitude to work.
Bloom’s revised taxonomy (Anderson, Krahwohl & Meyer, 2001), for instance, describes
various cognitive competency levels. Based on this theory, you develop assumptions
about the cognitive competency levels when deploying social media in tertiary educa-
tion. What you are actually doing is applying a theory to a new subject. You gather infor-
mation and analyze it to see whether the theory holds for your subject.
Now, you may be under the impression that researchers who use inductive
research don’t have any expectations. That is not entirely true. Based on their
knowledge and expertise, researchers have certain expectations about their
research findings, but in inductive research these expectations are not based on
a model or theory, whereas they are in deductive research. Thyme and Boeije
(2011) call this an exploratory hypothesis. You test an exploratory hypothesis
using a qualitative method.
Key concepts
Checkpoint 1.1
– Which methods of data collection were used in the examples for the research:
a. into nicotine addiction (Example 1.3)?
b. at the Highlands Sports Club (Example 1.4 and 1.5)?
c. into guest experience (Example 1.6)?
d. in the parliamentary enquiry (Example 1.7)?
e. into the customer buying patterns at the gardening center (below, Example 1.11)?
– Suppose you do research into student satisfaction at your college. You conduct inter-
views with students and lecturers, you design a survey and the analyze findings from
previous research into student satisfaction. Which approach have you chosen to
answer your central question?
There are rules (criteria) for the quality of research that all researchers stick to
– regardless of whether they are doing applied or fundamental research. That is
what this section is about. Chapter 9 will discuss these rules in detail.
First and foremost, there are rules about the researcher him- or herself. This sec-
tion starts with that. Secondly, there are quality criteria for conducting research
Researchers differ from those who are not researchers in three respects: their
attitude, knowledge and skill.
Attitude
A (critical) researcher can be distinguished by his or her attitude. That attitude
must be objective. That is to say that the personal preferences of the researcher
must not influence the research. Your own personal opinion of the situation
is not what it’s about. This kind of objectivity is not always possible. After all,
researchers are people too, with their own views and opinions. This may be
a problem in qualitative research in particular, because these researchers are
often involved with the people or groups they are researching.
That is why it is important that you as a researcher strive for openness in your
research, that you are open to comments from your peers, and that you are
accountable for your findings. This attitude is important in fundamental and
scientific research. If your research findings are contradicted by other research,
it’s not because your research was inferior, it’s because the findings were refuted.
This means that your research may be the first in a series of thorough analyses.
The research is ongoing, evolving. This scientific attitude is important because
it will reinforce your research findings.
In Section 1.2.2 on ‘falsifiability of statements’ you can find out more about the
openness of research.
Are you doing applied research? Then having a research-oriented and critical
attitude is also important. You learn to look at your own research critically,
from a ‘distance’. You also learn to not just accept other people’s research as
true, but to examine it critically. This critical, research-oriented attitude may
stand you in good stead later on in your profession. It could be, for instance,
that certain measures may benefit the management of an organization, so that
the organization is distracted from the true problems. This may not necessarily
be the result of conscious decisions; often organizations have no idea where
the root of the real problem lies. Independent research can shed light on the
real issues, ensure that good solutions are developed to address them and that
these solutions are effective. And that they are independent of the opinions and
objectives of those concerned.
Knowledge
Obviously you can’t apply research methods if you don’t know what they
are. Knowledge of methods is and always will be an important part of doing
research. And you can’t avoid this in Doing Research either. Knowledge about
research methods, on the other hand, is consistent: you always need to know
which research methods are out there, what criteria they use and what the pros
and cons are of applying a particular one.
Alongside knowledge about methods, you will also need to know about the
subject you intend to research. This is the kind of knowledge that you have to
acquire or freshen up each time. You can look for information on the subject;
you can read about it.
Skill
You polish the skills you need for research by actively going out and doing it. You
become familiar with all the aspects of research, step by step, through examples
and cases. Then you learn how to apply this knowledge. That knowledge is not
only about the theory of doing research. It also contains a whole lot of ‘recipes’
that you can use when it comes to stats, software and so on. And about ‘tricks’
that you have to learn. For instance, how to select a research group, how to enter
data into a software package, how to conduct a test, interpret analysis results,
or create a diagram.
Example 1.11 shows you how you can use a diagram to show buying behavior.
A gardening center wants to gear its purchasing policy to suit customers’ buying pat-
terns. The buying behavior of certain products is analyzed for a whole year. This is what a
diagram showing buying patterns looks like in Excel:
Nov March
bulbs
seeds
Oct Apr
flowers
trees & bushes
Sep May
Aug Jun
Jul
Below we discuss quality criteria for carrying out and interpreting research.
Reliability
Research is generally judged by the reliability of its results. The reliability of
research has to do with the extent to which it is free of random errors. These
are deviations in your research that are not verifiable, ones that are caused by
unknown factors. Clients often use research results to make important deci-
sions, or to determine policy. It must therefore be of good quality. Random
errors in research can affect the reliability.
Research that another researcher carries out in the same way should lead to the
similar results. So the research can be replicated. There should also be agree
ment among the researchers about the findings of the research. That is why it is
important for the researcher to have an independent attitude (Section 1.2.1); he
or she must view the research objectively, regardless of the results.
Falsifiability of statements
One objective of research is to get results about things that are observable in
‘reality’. A subject, question or assertion must be falsifiable. This means that you
can’t make assertions like ‘angels exist’ or ‘Manchester United is the best’. These
are assertions that cannot be proven, that cannot be verified through testing.
They are speculative (not based on facts) and subjective (it’s your opinion). The
point of research, quantitative research in particular, is to make assertions that
can be tested.
Making assertions in public means that they can be tested, that research
based on the same design can be repeated in another situation. This means
that research can be replicated. This replicability of research is therefore crucial
when it comes to assessing the reliability of research.
Informativeness
The information content of your statements must be optimal. This is another
aspect that is linked to the falsifiability of your assertions. To be able to verify a
statement (i.e., for quantitative research: the verifiability criterion), it needs to
be accurately formulated, you need to know what it is you are about to research,
when and with whom. This is why it is important that you describe your sub-
ject accurately (Scheepers, Tobi & Boeije, 2016; Swanborn, 2010, pp. 243-244;
Swanborn, 1987, pp. 35 et seq.). You have to be sure you indicate:
• the situation that you are referring to;
• the boundaries within which your research applies;
• the groups involved;
• the period that is relevant to your research;
• the ‘domain’ of your research, i.e., the whole ‘area’ that is relevant to your
research, and all the elements that your research involves. The larger the
domain, the more informative your statements will be.
External validity
What would happen if we went about making statements about situations
without researching them? What if we made rules without evaluating them?
Imagine if the shops stocked things without finding out beforehand if they
sell. What if people were given medicine without investigating what the effects
might be and whether they are the right drugs for the sickness that ails them?
Imagine if we went by what they have to say in the papers about certain groups
of people, and dealt with these people harshly without investigating the facts of
the matter first.
Researchers don’t go about things in this way: they work differently. Based on
their results, researchers want to be able to make assertions about as many
eople or situations as they can. They analyze a specific part of ‘reality’ and
p
make their statements based on that. If these statements have been verified
accurately, and according to all the criteria for this, then they can be considered
valid for a larger group or other situations: they are generalizable. This gener-
alizability forms the external validity of the research, a special kind of validity.
There are roughly two kinds of external validity (see Example 1.12):
• By statistical generalization, we mean that statistical tests (quantitative) are
used to assess whether results are generalizable or not. So it’s technical.
• In qualitative research, researchers will try to get results that are comparable
to results from similar situations, i.e., theoretical generalization (Scheepers
et al., 2016, p. 268).
Applied research may be carried out within one department of an organization, because
that department has an issue that needs to be researched.
It is not necessary to apply the results to the whole organization, but they may be relevant
to similar departments, even though the study was not conducted in those departments.
The results are generalizable in terms of their content.
It may also involve applied research because the organization as a whole has issues that
need to be researched. The sample for the survey is a randomly chosen and representa-
tive selection of staff, and the results are valid for the entire organization. They are statis-
tically generalizable.
Key concepts
There are also several practical criteria that all types of research must meet.
Efficiency
Research must be efficient. This means that the costs should be in proportion
with the results, and the schedule should be feasible.
Feasibility
In line with this, research must be feasible. You must have a big enough team
of researchers, be able to approach your sample, collect and analyze data and
report on time.
Usability
A general consideration that is particularly relevant to applied research is that
it should be usable and relevant to practice (Andriessen & Butter, 2016). Many
of the criteria mentioned are open to discussion, usability is not one of them.
There is no point at all in doing research that has no practical application. No
one needs research that goes straight into the dustbin. Example 1.13 describes
an investigation at the Food Bank in Philadelphia. Despite the fact that only a
third of the volunteers completed the questionnaire, the comments that volun-
teers gave are useful for the Food Bank.
What experiences do volunteers and customers have of the handing out of food at the
food bank? One hundred and thirty volunteers and customers at the Philadelphia food
bank were given a written questionnaire. Of these, 39 were completed and returned. That
is a response rate of 30%. Are the results usable? Yes! Statistical generalization is not an
absolute must. For the food bank, the results are very valuable because the Philadelphia
Food Bank can use them to improve the system they have in place for handing out food.
Key concepts
Checkpoint 1.2
Which quality aspects were not taken into account sufficiently in the following studies?
– A company that produces baby milk pays for research into the effect on toddlers of
vitamin D in this milk.
– The voting behavior of a panel is followed during election polls. The results will be
validated on all voters who are entitled to vote.
– Research among students of the Fiducia Student Fraternity shows that students
don’t rate organized activities highly. But only five students took part in the survey.
When you’re designing and conducting research you are constantly asking
questions, for example:
• What am I going to research?
• Why am I going to research it?
• Who am I going to research?
• How am I going to do my research?
• Where am I going to do my research?
• When am I going to do my research?
You don’t only ask yourself these questions when you start researching, when
you’re busy designing your research. During the research, too, you continually
ask yourself questions. You stop along the way and check your progress. You
look back, and then forwards. Are things going according to plan? Are you on
the right track when it comes to the content? What was the issue again? Are you
on schedule? Are you within the budget? Sometimes you have to stop in your
tracks, reconsider your whole research project, talk to your client or go back to
the drawing board. Your findings are paramount because the organization you
are doing the research for has to be able to use them. That is why you go over
and over again this process of ‘marking time’, looking back and forwards and
moving on again, like a cycle (see Figure 1.1).
This section shows what such a fixed structure (also referred to as a research
cycle) looks like and how it produces a research report as well as a professional
product.
Research cycle
1 Design
The most important objective in this phase is that you achieve a proper objec-
tive and central question. If you don’t do this properly, you run the risk that
your research will become unwieldy and you won’t be able to draw clear and
usable conclusions.
The next step is to design your research, showing how you intend to address the
question, which data collection methods you will use to do so, how much time,
what you’ll need, and who will be involved in your research. You also specify
which research instruments you’ll be using. Finally you specify how you are
going to analyze the information, and which method you will be using for this.
2 Data collection
After you have finalized the design, you go on to conduct the research. This
is when you go about collecting the information that you need to answer the
research question or questions. There are many strategies for this, depend-
ing on the number of subjects (people, objects, organizations) that you will be
researching, the nature of the question (does it lend itself to qualitative or quan-
titative research?), and the time and budget available.
3 Analysis
You then go on to process and analyze the data that you have gathered. As is
the case in Phases 2 and 3, there are several methods to choose from when it
comes to data analysis, both qualitative and quantitative (for numerical infor-
mation), depending on the kind of data collected. This will be discussed later
in this method.
In Figure 1.1, there is a diagram showing the four phases of research. A part of
this book is dedicated to each phase.
d
t 4 n antions On PDae
o da Dtew r
reco valua ar
elt 1 en
mm ti
P
1
e
esrigpn
E
t2
r
Pa a
t Pa a
An
ly 3 n
e c ta
r
tio
s is D
ll
co
Once you have completed all the phases of the research, you round the project
off with a report that gives an account of the research methods used, and in
which you present the results (Phase 4). But that is not the whole story. You
can also summarize these results in an instrument that will be useful for the
client. In other words, a professional product. These professional products vary
depending on the discipline, subject and central question. It may comprise rec-
ommendations, a plan of action, a treatment plan, a proposal for measures or
recommendations. If you are study engineering, your research may provide
advice when using certain kinds of sustainable insulation materials. If you’re
an accountant, your professional product may consist of a new system for pro-
cessing financial data. If you are a nurse, then your recommendations may be
about implementing a treatment plan.
So it is important to ask yourself how the research will be used. What is the
role of your research? Its purpose? Will you be writing a report about it, or
will you use it to give advice, or to present a design or plan of action? Nowa-
days clients and internship companies are more likely to ask for a professional
product rather than a research report. This does not detract from the fact that
research is an essential part of your project. But in the final phase of your pro-
ject, when you draw your conclusions, there will be another element: the pro-
fessional product.
The choice between a research report and a professional product is actually
not that important. What is important is to realize that the journey you take is
roughly parallel and that you have to use the same toolkit, the same research
skills in the process.
In short, the results are different depending on your professional practice,
regardless of whether you produce a professional product or only a research
report. But the method used, the controls, the technique, analysis and justifica-
tion are the same. These are the phases of your research project. So the cycle that
you go through to arrive at a professional product is the same as for a research
report (Figure 1.1), the difference being that a product follows the evaluation.
Checkpoint 1.3
– Example 1.14 shows you how a study among volunteers is planned. Which research
phases do you recognize in this example?
– What could the end product be here: a research report and/or professional product?
Why?
The Greendale Volunteers Society wants answers to the following question: how can the
Volunteers Society attract volunteers from the surrounding municipalities? A researcher
compiles a questionnaire with several practical questions for the volunteers they already
have. The questions are about background characteristics, like age, gender, level of edu-
cation, and about how they spend their time and what motivates them. The researcher
uses a scale that has already been used to measure motivations for volunteering
(Lindeman, 1996). This motivation scale is the result of the preliminary research that the
researcher conducted. The scale consists of statements that relate to social contacts, car-
ing for o
thers, the satisfaction it gives and the contribution to society. The volunteers
interviewed indicate to what extent they agree with the statements. This gives insight
into their motivation to help as volunteers. The results help the Volunteer Society to
organize a campaign for attracting new volunteers.
Website
Handy tools:
– research tools: checklists, questionnaires and templates for setting up a logbook
and a research proposal
CH
how do you design a research project, what steps do you have to take, how do
you collect data and what are the pitfalls?
AR
and evaluation and recommendations. Many up-to-date examples are used to
establish the link to everyday practice.
This 5th edition has been completely revised. The various research steps are
presented in clear procedure guides, so that it is easy to see where you are in the
research process. Each chapter also has checkpoint assignments to test whether
The
and How
SE
you have understood the theory correctly. You can find practical tools, tests and
answers to the checkpoint assignments on www.doingresearch5thedition.com.
of A hyW s
VERHOEVEN
RE
but it can also be used as a reference book for applied research.
ear d
G
workshops and courses. She is the author of several 5TH
successful teaching methods in the field of methodology. EDI
ch
EN
TION
EV
IN
HO
ISBN 978-90-2442-475-7
ER
DO
LV
NE
www.doingresearch5thedition.com
9 789024 424757 www.boomhogeronderwijs.nl