Topic TheResearch

Design

LEARNING OUTCOMES
By the end of this topic, you should be able to:

1.

Define what is research design;

2.

Elaborate the functions of research design;

3.

Differentiate the good research designs from the weak research

designs; and

4.

Analyse the differences between a true experimental design and a

quasi-experimental design;

INTRODUCTION

This topic focuses on how we carry out analysis on operating activities, by

analysing accrual measures of both revenues and expenses. We will also learn
about the net income analysis. Finally, we are going to learn to distinguish the
components of income and impact for financial analysis.

TOPIC 5 THE RESEARCH DESIGN

5.1

59

WHAT IS A RESEARCH DESIGN?

Say that you have already decided what you want to study. The next question
you should ask yourself is how are you going to conduct your study? What are
the dos and donts in the process of undertaking the research? The answer to this
question comprises the core of a research design.

A research design is a procedural plan that is adopted by a researcher to answer

questions in a valid way. It is very objective and accurate. Normally, a research
design will determine the type of analysis you should carry out to get the desired
results. To what extent your design is good or bad will depend on whether you
are able to get the answers to your research questions. If your design is poor, the
results of the research also will not be promising.
How do you go about getting a good research design that will provide sufficient
answers to your research work? It is not easy and there is no fixed method to tell
others how to do it. The best approach will be to examine different types of
research designs, list down the strengths and weaknesses and then make your
decisions.
As a research student, you should have a good understanding of your
research\problem such as the method that you would like to use in your work,
be clear about your research questions and what is it that you intend to establish.
In ICT, it is always a good practice to understand the technological context
behind every research problem before going in depth. Never select a design and
then try to fit the research questions to the design! It should be the other way
around. It is very important for you to see if the design could answer your
research questions. It is necessary for you to choose a design that will give you
the optimised result over factors that explain the research results obtained.

60

5.2

TOPIC 5 THE RESEARCH DESIGN

THE FUNCTIONS OF RESEARCH DESIGN

A research design relates to the identification of procedures and logistical

arrangements to start a study and also at the same time emphasises the
importance of quality in producing optimised research results. It glues all the
components and subcomponents in a research project together. In typical
experimental research design, there are a few symbols that a researcher should
know which have been widely used to show the design of a study. These
symbols are:
R = Random assignment:subjects are assigned in random and into
various groupings
X = Means treatment, which maybe design and implementation,
performance evaluation, simulation study and so forth
O = Observation or Measurement
( e.g. data rate, baud rate, respond time, bufferring period,etc.)

Generally, we can divide the functions of research design into two:

(a)

Constructing an operational procedure to execute the tasks required in

completing your research work.

(b)

Ensuring these procedures are sufficient to get valid, objective and answers
which are accurate to the questions posed in the research work.

One of the most fundamental aspects of a research design is to specify everything

in depth and clear. This is to ensure that a reader will understand what method
to follow and how to follow it. A research design should have the following:
(i)

Naming the study design (for example as comparative, cross-sectional or

random control)

(ii)

How will the study population be identified?

(iii) What are the sampling methods used ( we will go further about sampling in
the coming topics)?
(iv) What method of data collection will be used in the research work?
(v)

How will ethical issues be considered?

TOPIC 5 THE RESEARCH DESIGN

61

SELF-CHECK 5.1
Identify the two functions of research design.

5.3

WEAK DESIGNS

5.3.1

One-shot Design

Lets say you want to examine whether consumer knowledge makes better
service provision in e-commerce. You measure e-commerce service provision
with the number of transactions. To test this idea, you choose a group of
consumers who use e-commerce for purchasing and these consumers with ecommerce knowledge yield higher transactions rate. You conclude that
consumers with knowledge of e-commerce increases the number of e-commerce
overall transactions. The design is weak for the following reasons:
(a)

Selection Bias: The group you selected as subjects might already have some
fundamental knowledge about e-commerce.

(b)

Background: The organisation of the groups you select might use ecommerce in their daily business transactions, so it might influence the
results.

5.3.2

One-Group Pretest-Posttest Design

Using the same example as above, to ensure that there are no pre-existing
characteristics among the consumers, a pretest may be administered. If the
respondents perform better in e-commerce service provision after being briefed
on e-commerce prior to the pretest, then you can attribute it to the frequency of
briefing. Table 5.1 summarises the reasons why this design is considered weak.

62

TOPIC 5 THE RESEARCH DESIGN

Table 5.1: Reasons why One-Group Pretest-Posttest Design is weak

Reason

Explaination

(a)

Maturation

If the time gap between the pretest and posttest is

long, it is possible that the subjects may have
matured due to developmental changes and
modification.

(b)

Testing

Sometimes the time gap between pretest and

posttest is too short and there is possibility that
subjects cannot recall on e-commerce usage
methodology.

SELF-CHECK 5.2
1.

Identify the differences between one-shot design and one-group

pretest-posttest design.

2.

Why are these designs considered weak?

5.4

TRUE DESIGNS

It is important for us to look into true experimental designs or widely known as

true designs in research methodology. In true designs, a researcher should
have a research design that enables him/her to have control over the situation in
terms of assignment of subjects into groups, to decide on who gets the treatment
condition and to decide the amount of treatment condition that subjects receive
(Christensen 1988). In this topic, we will discuss two major types of true design
after-only and before-after. The major difference between these two designs is
that:
(a)

the after-only design depends on posttest

(b)

before-and-after design depends on a pretest and a posttest.

TOPIC 5 THE RESEARCH DESIGN

5.4.1

63

The After-only Design

In this design, let us say that a researcher is aware that a population is exposed to an
intervention. Therefore, he/she wishes to study its impact on the population. In this
design, pretest is usually constructed based on situation before an intervention; or
from information available from existing record. In the ICT, after-only-design is
often used in system verification based research and at times in design and
implementation procedures. Figure 5.1 shows the after-only design.

Figure 5.2 : The After-Only Design

Source: Adapted from Figure 8.7 Research Methodology: Ranjit Kumar, page 102

This design is mainly used in IT impact assessment studies. In real-life, many

programmes operate without the benefit of a planned evaluation at the
programme planning stage. This is due to the fast changing nature of the ICT
field and it is not possible to follow strictly the sequence such as collection of
baseline information, implementation of programme and then programme
evaluation. In normal practice, the adequacy of this design relies on the accuracy
of data obtained on the prevalence of incident or impact before an intervention is
introduced. Let us take a look at a few examples of situations.
(a)

the impact of web services in e-commerce technology;

(b)

the impact of decline in network performance of a campus WAN; or

(c)

the impact of IT on undergraduate education.

In these types of situations, it is expected that accurate details are kept regarding
the scope of the study. Therefore, any change is due to the introduction of
intervention or policy changes of the research issue.

64

TOPIC 5 THE RESEARCH DESIGN

5.4.2

The Before-and-after Design

One of the good examples of true design is the before-and-after research design.
This includes experimental and control groups. This design overcomes the
problem of retrospectively constructing the before observation by introducing it
before the intervention takes place in the research subject. This design solves the
problem with after-only design that is comparative of the before-and-after
observations (Figure 5.2).

Figure 5.2: Measurement of change through a before-and-after design

SELF-CHECK 5.3
1.

What is the core strength of true designs in research?

2.

What is the major difference between the two types of true design
discussed in this section?

5.5

THE QUASI-EXPERIMENTAL DESIGN

What is meant by Quasi-Experimental Design?

Quasi-Experimental Design is a design that does not provide full
scalable control towards confounding variables.
~ Chirstensen and Johnson 2000

TOPIC 5 THE RESEARCH DESIGN

65

This could be due to random assignment of samples in a particular study. In

some cases, it is not possible to assign samples to groups which require a strong
experimental research.
For example, in ICT, there are times when researchers faced with situation where
all the requirements of a true design experiment cannot be fulfilled. This scenario
is particularly true when dealing with IT impact based research and studies.

5.5.1

Non-Equivalent Control Group Design

This design group contains both experimental and control groups. However,
subjects are not randomly assigned to the groups. This means that the subjects in
the groups may not be equivalent on all variables defined. In this design, both
groups are given first a pretest and then a posttest (once the treatment is given to
the experimental group). Both scores of posttest and pretest are compared to each
other to check if there is any significant differences. Figure 5.3 below shows the
non-equivalent control group design.

Figure 5.3: Non-Equivalent Control Group Design

Source: Adapted from www.socialresearchmethods.net

We must know here that we cannot randomly assign subjects, it means we can be
sure that extraneous variables or factors will creep into our experiment and
threaten its internal validity (we will discuss in depth about validity in the
coming topics). A good researcher will take some measures in ensuring the
subjects in the experimental and control groups are as similar as possible. In ICT
research, it might be important variables such as infrastructure, technology
feasibility, ICT testbed and so forth.

5.5.2

Interrupted Time Series Design

This design requires a researcher to take a series of measurements both before

and after the treatment. A good example of deployment of this research design is
ICT communication research and performance modelling of network. Subjects
from a single group will be pretested a number of times during the foundation

66

TOPIC 5 THE RESEARCH DESIGN

phase, exposed to the treatment, and then posted a number of times after the
treatment (Figure 5.4). Foundation phase refers to the testing conducted before
the treatment to alter behaviour.

Figure 5.4: Interrupted Time-Series Design

Source: Adapted from www.socialresearchmethods.net

SELF-CHECK 5.4
1.

What is the meaning of quasi-experimental design?

2.

What is defined as non-equivalent control group design?

ACTIVITY 5.1
To learn more about Quasi-Experimental design, visit the following
website: http://www.socialresearchmethods.net/kb/quasiexp.php
http://allpsych.com/researchmethods/quasiexperimentaldesign.html
What did you learn from your readings? Share your thoughts on lessons
learned from this topic with your coursemates. Post your comments in the
myLMS forum. Remember, sharing is caring!

A research design consists of strategy and specific procedures in seeking

answers to a specific research question.

A weak research design does not allow controlling extraneous variables into
experiment.

Some examples of weak designs are one-shot design and one-group pretestposttest design

A true research design allows us to maintain control over a situation in terms

of variables assignment.

TOPIC 5 THE RESEARCH DESIGN

67

A quasi-experimental design is a design that does not provide full control

over variables.

Examples of quasi-esperimental design are non-equivalent control group

design and interrupted time-series.

Experimental design
Quasi experimental design
Non-equivalent design
Interrupted time series design

Weak Research design

True design
After only design
Before-and-after design

1.

Construct a research case for a true experimental design in ICT.

2.

What are Quasi-experimental research designs? How do they

differ from true experiments?

Books
Christensen, L. (1988). Experimental methodology. Boston: Allyn and Bacon Inc.
Mitchell, M., & Jolley, J. (1988). Research design explained. New York: Holt,
Rinehart & Winston.
Internet Resources
Trochim, W. K. (2007). Research method tutorials [Electronic version]
http://www.socialresearchmethods.net/tutorial/tutorial.htm
Vagner , K. W. (2007). Introduction to research methods [Electronic version]
http://psychology.about.com/od/researchmethods/ss/expdesintro.htm