TOPIC 9

RESEARCH DESIGN

The research design is intended to provide an appropriate framework for a study. A very
significant decision in research design process is the choice to be made regarding research
approach since it determines how relevant information for a study will be obtained; however, the
research design process involves many interrelated decisions.

The quantitative research design that you set in your research study should reflect
the type of research questions/hypotheses that you have set. When we talk about quantitative
research designs, we are typically referring to research following either
a descriptive, experimental, quasi-experimental and relationship-based research design.

However, there are also specific goals that you may want to achieve within these research
designs. You may want to: (Goal A) explore whether there is a relationship between different
variables; (Goal B) predict a score or a membership of a group; or (Goal C) find out the differences
between groups you are interested in or treatment conditions that you want to investigate:

Thus, the research design refers to the overall strategy that you choose to integrate the
different components of the study in a coherent and logical way, thereby, ensuring you will
effectively address the research problem; it constitutes the blueprint for the collection,
measurement, and analysis of data. Note that the research problem determines the type of design
you should use, not the other way around!

The length and complexity of describing research designs in your paper can vary
considerably, but any well-developed description will achieve the following:

1. Identify the research problem clearly and justify its selection, particularly in relation to
any valid alternative designs that could have been used,
2. Review and synthesize previously published literature associated with the research
problem,
3. Clearly and explicitly specify hypotheses [i.e., research questions] central to the
problem,
4. Effectively describe the information and/or data which will be necessary for an
adequate testing of the hypotheses and explain how such information and/or data will
be obtained, and
5. Describe the methods of analysis to be applied to the data in determining whether or
not the hypotheses are true or false.
 The methods section of a research proposal must contain all the necessary information that
will facilitate another researcher to replicate your research. The purpose of writing this section is
to convince the readers that the methods you plan to use are sound and this is the most suitable
approach to address the problem you have chosen.

You must also demonstrate your understanding of alternative methods to show that you
have made a sound and well-calculated judgment. The methods section of your research proposal
should answer the following questions:
What is your study design and why did you choose it?
Is the study qualitative or quantitative?
What are the methods you will use to collect data?
Who will be the participants of your study?
What procedures/activities will the study involve?
How long will the experiments/study take to get completed?

You can further break down the types of research design into five categories:

1. Descriptive research design: In a descriptive design, a researcher is solely interested in

describing the situation or case under their research study. It is a theory-based design method which
is created by gathering, analyzing, and presenting collected data. This allows a researcher to
provide insights into the why and how of research. Descriptive design helps others better
understand the need for the research. If the problem statement is not clear, you can conduct
exploratory research.

2. Experimental research design: Experimental research design establishes a relationship

between the cause and effect of a situation. It is a causal design where one observes the impact
caused by the independent variable on the dependent variable. For example, one monitors the
influence of an independent variable such as a price on a dependent variable such as customer
satisfaction or brand loyalty. It is a highly practical research design method as it contributes to
solving a problem at hand. The independent variables are manipulated to monitor the change it has
on the dependent variable. It is often used in social sciences to observe human behavior by
analyzing two groups. Researchers can have participants change their actions and study how the
people around them react to gain a better understanding of social psychology.

3. Correlational research design: Correlational research is a non-experimental research

design technique that helps researchers establish a relationship between two closely connected
variables. This type of research requires two different groups. There is no assumption while
evaluating a relationship between two different variables, and statistical analysis techniques
calculate the relationship between them.
 A correlation coefficient determines the correlation between two variables, whose value
ranges between -1 and +1. If the correlation coefficient is towards +1, it indicates a positive
relationship between the variables and -1 means a negative relationship between the two variables.
4. Diagnostic research design: In diagnostic design, the researcher is looking to evaluate
the underlying cause of a specific topic or phenomenon. This method helps one learn more about
the factors that create troublesome situations.

5. Explanatory research design: Explanatory design uses a researcher’s ideas and thoughts
on a subject to further explore their theories. The research explains unexplored aspects of a subject
and details about what, how, and why of research questions.

TOPIC 10
RESPONDENTS AND SAMPLING PROCEDURE

What Is Survey Sampling?

Surveys would be meaningless and incomplete without accounting for the respondents that
they’re aimed at. The best survey design practices keep the target population at the core of their
thought process.

‘All the residents of the Zone 5 slums in Metro Manila’, ‘every NGO in Region 3’ and ‘all
students below the age of 16 in Bulacan’ are examples of a population; they are countable, finite
and well-defined.

When the population is small enough, researchers have the resources to reach out to all of
them. This would be the best case scenario, making sure that everybody who matters to the survey
is represented accurately. A survey that covers the entire target population is called a census.

However, most surveys cannot survey the entire population. This is when sampling
techniques become crucial to your survey.

Why Is It Important?

Resource Constraints
If the target population is not small enough, or if the resources at your disposal don’t give
you the bandwidth to cover the entire population, it is important to identify a subset of the
population to work with – a carefully identified group that is representative of the population.
 This process is called survey sampling, and it is one of the most important aspects of survey
design.

Whatever the sample size, there are fixed costs associated with any survey. Once the survey
has begun, the marginal costs associated with gathering more information, from more people, are
proportional to the size of the sample.

Drawing Inferences About the Population

Researchers are not interested in the sample itself, but in the understanding that they can
potentially infer from the sample and then apply across the entire population.

A sample survey usually offers greater scope than a census. Working within a given
resource constraint, sampling may make it possible to study the population of a larger geographical
area or to find out more about the same population by examining an area in greater depth through
a smaller sample.

3 Features to Keep in Mind While Constructing a Sample

Consistency
It is important that researchers understand the population on a case-by-case basis and test
the sample for consistency before going ahead with the survey. This is especially critical for
surveys that track changes across time and space where we need to be confident that any change
we see in our data reflects real change – across consistent and comparable samples.

Diversity
Ensuring diversity of the sample is a tall order, as reaching some portions of the population
and convincing them to participate in the survey could be difficult. But to be truly representative
of the population, a sample must be as diverse as the population itself and sensitive to the local
differences that are unavoidable as we move across the population.

Transparency
There are several constraints that dictate the size and structure of the population. It is
imperative that researchers discuss these limitations and maintain transparency about the
procedures followed while selecting the sample so that the results of the survey are seen with the
right perspective.
 Now that we understand the necessity of choosing the right sample and have a vision of
what an effective sample for your survey should be like, let’s explore the various methods of
constructing a sample and understand the relative pros and cons of each of these approaches.

Sampling methods can broadly be classified as probability and non-probability.

3 Probability Sampling Techniques

When each entity of the population has a definite, non-zero probability of being
incorporated into the sample, the sample is known as a probability sample.

Probability samples are selected in such a way as to be representative of the population.

They provide the most valid or credible results because they reflect the characteristics of the
population from which they are selected.

Probability sampling techniques include random sampling, systematic sampling, and

stratified sampling.

1. Random Sampling

When: There is a very large population and it is difficult to identify every member of the
population.

How: The entire process of sampling is done in a single step with each subject selected
independently of the other members of the population. The term random has a very precise
meaning and you can’t just collect responses on the street and have a random sample.

Pros: In this technique, each member of the population has an equal chance of being
selected as subject.

Cons: When there are very large populations, it is often difficult to identify every member
of the population and the pool of subjects becomes biased. Dialing numbers from a phone book
for instance, may not be entirely random as the numbers, though random, would correspond to a
localized region. A sample created by doing so might leave out many sections of the population
that are significant to the study.

Use case: Want to study and understand the rice consumption pattern across rural
Philippines? While it might not be possible to cover every household, you could draw meaningful
insights by building your sample from different districts or villages (depending on the scope).
2. Systematic Sampling

When: Your given population is logically homogenous.

How: In a systematic sample, after you decide the sample size, arrange the elements of the
population in some order and select terms at regular intervals from the list.

Pros: The main advantage of using systematic sampling over simple random sampling is
its simplicity. Another advantage of systematic random sampling over simple random sampling is
the assurance that the population will be evenly sampled. There exists a chance in simple random
sampling that allows a clustered selection of subjects. This can be avoided through systematic
sampling.

Cons: The possible weakness of the method that may compromise the randomness of the
sample is an inherent periodicity of the list. This can be avoided by randomizing the list of your
population entities, as you would randomize a deck of cards for instance, before you proceed with
systematic sampling.

Use Case: Suppose a supermarket wants to study buying habits of their customers. Using
systematic sampling, they can choose every 10th or 15th customer entering the supermarket and
conduct the study on this sample.

3. Stratified Sampling

When: You can divide your population into characteristics of importance for the research.

How: A stratified sample, in essence, tries to recreate the statistical features of the
population on a smaller scale. Before sampling, the population is divided into characteristics of
importance for the research — for example, by gender, social class, education level, religion, etc.
Then the population is randomly sampled within each category or stratum. If 38% of the population
is college-educated, then 38% of the sample is randomly selected from the college-educated subset
of the population.

Pros: This method attempts to overcome the shortcomings of random sampling by splitting
the population into various distinct segments and selecting entities from each of them. This ensures
that every category of the population is represented in the sample. Stratified sampling is often used
when one or more of the sections in the population have a low incidence relative to the other
sections.
 Cons: Stratified sampling is the most complex method of sampling. It lays down criteria
that may be difficult to fulfill and place a heavy strain on your available resources.
Use Case: If 38% of the population is college-educated and 62% of the population have not been
to college, then 38% of the sample is randomly selected from the college-educated subset of the
population and 62% of the sample is randomly selected from the non-college-going population.
Maintaining the ratios while selecting a randomized sample is key to stratified sampling.

3 Non-Probability Sampling Techniques

Non-probability sampling techniques include convenience sampling, snowball sampling

and quota sampling.

In these techniques, the units that make up the sample are collected with no specific
probability structure in mind. The selection is not completely randomized, and hence the resultant
sample isn’t truly representative of the population.

1. Convenience Sampling

When: During preliminary research efforts.

How: As the name suggests, the elements of such a sample are picked only on the basis of
convenience in terms of availability, reach and accessibility.

Pros: The sample is created quickly without adding any additional burden on the available
resources.

Cons: The likelihood of this approach leading to a sample that is truly representative of the
population is very poor.

Use Case: This method is often used during preliminary research efforts to get a gross
estimate of the results, without incurring the cost or time required to select a random sample.

2. Snowball Sampling

When: When you can rely on your initial respondents to refer you to the next respondents.

How: Just as the snowball rolls and gathers mass, the sample constructed in this way will
grow in size as you move through the process of conducting a survey. In this technique, you rely
on your initial respondents to refer you to the next respondents whom you may connect with for
the purpose of your survey.
 Pros: The costs associated with this method are significantly lower, and you will end up
with a sample that is very relevant to your study.

Cons: The clear downside of this approach is that you may restrict yourself to only a small,
largely homogenous section of the population.

Use Case: Snowball sampling can be useful when you need the sample to reflect certain
features that are difficult to find. To conduct a survey of people who go jogging in a certain park
every morning, for example, snowball sampling would be a quick, accurate way to create the
sample.

3. Quota Sampling

When: When you can characterize the population based on certain desired features.

How: Quota sampling is the non-probability equivalent of stratified sampling that we

discussed earlier. It starts with characterizing the population based on certain desired features and
assigns a quota to each subset of the population.

Pros: This process can be extended to cover several characteristics and varying degrees of
complexity.

Cons: Though the method is superior to convenience and snowball sampling, it does not
offer the statistical insights of any of the probability methods.

Use Case: If a survey requires a sample of fifty men and fifty women, a quota sample will
survey respondents until the right number of each type has been surveyed. Unlike stratified
sampling, the sample isn’t necessarily randomized.

Related reading: When to do quota sampling and how to do it correctly

Probability sampling techniques are superior, but the costs can be prohibitive. For the initial
stages of a study, non-probability sampling techniques might be sufficient to give you a sense of
what you’re dealing with. For detailed insights and results that you can bank upon, move on to the
more sophisticated techniques as the study gathers pace and takes a more concrete structure.

Once you have created your sample, go ahead and start creating an effective survey
by choosing the right survey question types.
Determining sample size: how to make sure you get the correct sample size

How many is enough? Over the years, researchers have grappled with the problem of
finding the perfect sample size for statistically sound results. Here we shed light on some methods
and tools for sample size determination.

What is sample size?

Sample size is a frequently-used term in statistics and market research, and one that
inevitably comes up whenever you’re surveying a large population of respondents. It relates to the
way research is conducted on large populations.

So what is sampling, and why does sample size matter?

When you survey a large population of respondents, you’re interested in the entire group,
but it’s not realistically possible to get answers or results from absolutely everyone. So you take a
random sample of individuals which represents the population as a whole.

The size of the sample is very important for getting accurate, statistically significant results
and running your study successfully.

If your sample is too small, you may include a disproportionate number of individuals
which are outliers and anomalies. These skew the results and you don’t get a fair picture of the
whole population.

If the sample is too big, the whole study becomes complex, expensive and time-consuming
to run, and although the results are more accurate, the benefits don’t outweigh the costs.

If you’ve already worked out your variables you can get to the right sample size quickly
with the online sample size calculator below:

Confidence Level:
90%

Population Size:
1000

Margin of Error:
5%
Ideal Sample Size:
Answ er

If you want to start from scratch in determining the right sample size for your market
research, let us walk you through the steps.

Learn how to determine sample size

To choose the correct sample size, you need to consider a few different factors that affect
your research, and gain a basic understanding of the statistics involved. You’ll then be able to use
a sample size formula to bring everything together and sample confidently, knowing that there is
a high probability that your survey is statistically accurate.

The steps that follow are suitable for finding a sample size for continuous data – i.e. data
that is counted numerically. It doesn’t apply to categorical data – i.e. put into categories like green,
blue, male, female etc.

Stage 1: Consider your sample size variables

Before you can calculate a sample size, you need to determine a few things about the target
population and the level of accuracy you need:

1. Population size

How many people are you talking about in total? To find this out, you need to be clear
about who does and doesn’t fit into your group. For example, if you want to know about dog
owners, you’ll include everyone who has at some point owned at least one dog. (You may include
or exclude those who owned a dog in the past, depending on your research goals.) Don’t worry if
you’re unable to calculate the exact number. It’s common to have an unknown number or an
estimated range.

2. Margin of error (confidence interval)

Errors are inevitable – the question is how much error you’ll allow. The margin of error,
AKA confidence interval, is expressed in terms of mean numbers. You can set how much
difference you’ll allow between the mean number of your sample and the mean number of your
population. If you’ve ever seen a political poll on the news, you’ve seen a confidence interval and
how it’s expressed. It will look something like this: “68% of voters said yes to Proposition Z, with
a margin of error of +/- 5%.”
 3. Confidence level

This is a separate step to the similarly-named confidence interval in step 2. It deals with
how confident you want to be that the actual mean falls within your margin of error. The most
common confidence intervals are 90% confident, 95% confident, and 99% confident.

4. Standard deviation

This step asks you to estimate how much the responses you receive will vary from each
other and from the mean number. A low standard deviation means that all the values will be
clustered around the mean number, whereas a high standard deviation means they are spread out
across a much wider range with very small and very large outlying figures. Since you haven’t yet
run your survey, a safe choice is a standard deviation of .5 which will help make sure your sample
size is large enough.

Stage 2: Calculate sample size

Now that you’ve got answers for steps 1 – 4, you’re ready to calculate the sample size you
need. This can be done using an online sample size calculator or with paper and pencil.

5. Find your Z-score

Next, you need to turn your confidence level into a Z-score. Here are the Z-scores for the
most common confidence levels:
90% – Z Score = 1.645
95% – Z Score = 1.96
99% – Z Score = 2.576

If you chose a different confidence level, use this Z-score table (a resource owned and
hosted by SJSU.edu) to find your score.

6. Use the sample size formula

Plug in your Z-score, standard of deviation, and confidence interval into the sample size
calculator or use this sample size formula to work it out yourself:
 This equation is for an unknown population size or a very large population size. If your
population is smaller and known, just use the sample size calculator.

What does that look like in practice?

Here’s a worked example, assuming you chose a 95% confidence level, .5 standard
deviation, and a margin of error (confidence interval) of +/- 5%.

((1.96)2 x .5(.5)) / (.05)2

(3.8416 x .25) / .0025
.9604 / .0025
384.16
385 respondents are needed

Troubleshooting your sample size results

If the sample size is too big to manage, you can adjust the results by either decreasing your
confidence level increasing your margin of error

This will increase the chance for error in your sampling, but it can greatly decrease the
number of responses you need.

Source:
https://www.qualtrics.com/experience-management/research/determine-sample-size/

Sample Size Calculator

This Sample Size Calculator is presented as a public service of Creative Research

Systems survey software. You can use it to determine how many people you need to interview in
order to get results that reflect the target population as precisely as needed. You can also find the
level of precision you have in an existing sample.

Before using the sample size calculator, there are two terms that you need to know. These
are: confidence interval and confidence level. If you are not familiar with these terms, click here.
To learn more about the factors that affect the size of confidence intervals, click here.
Enter your choices in a calculator below to find the sample size you need or the confidence interval
you have. Leave the Population box blank, if the population is very large or unknown.

Determine Sample Size

Confidence Level: 95% 99%

Confidence Interval:

Population:

Sample size needed:

Find Confidence Interval

Confidence Level: 95% 99%

Sample Size:

Population:

Percentage: 50

Confidence Interval:

Sample Size Calculator Terms: Confidence Interval & Confidence Level

The confidence interval (also called margin of error) is the plus-or-minus figure usually
reported in newspaper or television opinion poll results. For example, if you use a confidence
interval of 4 and 47% percent of your sample picks an answer you can be "sure" that if you had
asked the question of the entire relevant population between 43% (47-4) and 51% (47+4) would
have picked that answer.

The confidence level tells you how sure you can be. It is expressed as a percentage and represents
how often the true percentage of the population who would pick an answer lies within the
confidence interval. The 95% confidence level means you can be 95% certain; the 99% confidence
level means you can be 99% certain. Most researchers use the 95% confidence level.
When you put the confidence level and the confidence interval together, you can say that
you are 95% sure that the true percentage of the population is between 43% and 51%. The wider
the confidence interval you are willing to accept, the more certain you can be that the whole
population answers would be within that range.
For example, if you asked a sample of 1000 people in a city which brand of cola they
preferred, and 60% said Brand A, you can be very certain that between 40 and 80% of all the
people in the city actually do prefer that brand, but you cannot be so sure that between 59 and 61%
of the people in the city prefer the brand.

Factors that Affect Confidence Intervals

There are three factors that determine the size of the confidence interval for a given
confidence level:
Sample size
Percentage
Population size

Sample Size
The larger your sample size, the more sure you can be that their answers truly reflect the
population. This indicates that for a given confidence level, the larger your sample size, the smaller
your confidence interval. However, the relationship is not linear (i.e., doubling the sample size
does not halve the confidence interval).

Percentage
Your accuracy also depends on the percentage of your sample that picks a particular
answer. If 99% of your sample said "Yes" and 1% said "No," the chances of error are remote,
irrespective of sample size. However, if the percentages are 51% and 49% the chances of error are
much greater. It is easier to be sure of extreme answers than of middle-of-the-road ones.
When determining the sample size needed for a given level of accuracy you must use the worst
case percentage (50%). You should also use this percentage if you want to determine a general
level of accuracy for a sample you already have. To determine the confidence interval for a specific
answer your sample has given, you can use the percentage picking that answer and get a smaller
interval.

Population Size
How many people are there in the group your sample represents? This may be the number
of people in a city you are studying, the number of people who buy new cars, etc. Often you may
not know the exact population size. This is not a problem. The mathematics of probability prove
that the size of the population is irrelevant unless the size of the sample exceeds a few percent of
the total population you are examining. This means that a sample of 500 people is equally useful
in examining the opinions of a state of 15,000,000 as it would a city of 100,000. For this reason,
The Survey System ignores the population size when it is "large" or unknown. Population size is
only likely to be a factor when you work with a relatively small and known group of people (e.g.,
the members of an association).
The confidence interval calculations assume you have a genuine random sample of the relevant
population. If your sample is not truly random, you cannot rely on the intervals. Non-random
samples usually result from some flaw or limitation in the sampling procedure. An example of
such a flaw is to only call people during the day and miss almost everyone who works. For most
purposes, the non-working population cannot be assumed to accurately represent the entire
(working and non-working) population. An example of a limitation is using an opt-in online poll,
such as one promoted on a website. There is no way to be sure an opt-in poll truly represents the
population of interest.

Source:
https://www.surveysystem.com/sscalc.htm

TOPIC 10
RESEARCH INSTRUMENTS

Instrument, Validity, Reliability

The Instrument
 Instrument is the general term that researchers use for a measurement device (survey, test,
questionnaire, etc.). To help distinguish between instrument and instrumentation, consider that
the instrument is the device and instrumentation is the course of action (the process of developing,
testing, and using the device).

Instruments fall into two broad categories, researcher-completed and subject-completed,

distinguished by those instruments that researchers administer versus those that are completed by
participants. Researchers chose which type of instrument, or instruments, to use based on the
research question. Examples are listed below:

Researcher-completed Instruments Subject-completed Instruments

Rating scales Questionnaires
Interview schedules/guides Self-checklists
Tally sheets Attitude scales
Flowcharts Personality inventories
Performance checklists Achievement/aptitude tests
Time-and-motion logs Projective devices
Observation forms Sociometric devices

Usability

Usability refers to the ease with which an instrument can be administered, interpreted by
the participant, and scored/interpreted by the researcher. Example usability problems include:
Students are asked to rate a lesson immediately after class, but there are only a few minutes
before the next class begins (problem with administration).

Students are asked to keep self-checklists of their after school activities, but the directions
are complicated and the item descriptions confusing (problem with interpretation).

Teachers are asked about their attitudes regarding school policy, but some questions are
worded poorly which results in low completion rates (problem with scoring/interpretation).

Validity and reliability concerns (discussed below) will help alleviate usability issues. For
now, we can identify five usability considerations:

How long will it take to administer?

Are the directions clear?
How easy is it to score?
Do equivalent forms exist?
 Have any problems been reported by others who used it?

It is best to use an existing instrument, one that has been developed and tested numerous
times.

Validity

Validity is the extent to which an instrument measures what it is supposed to measure and
performs as it is designed to perform. It is rare, if nearly impossible, that an instrument be 100%
valid, so validity is generally measured in degrees. As a process, validation involves collecting and
analyzing data to assess the accuracy of an instrument. There are numerous statistical tests and
measures to assess the validity of quantitative instruments, which generally involves pilot testing.
The remainder of this discussion focuses on external validity and content validity.

External validity is the extent to which the results of a study can be generalized from a
sample to a population. Establishing eternal validity for an instrument, then, follows directly from
sampling. Recall that a sample should be an accurate representation of a population, because the
total population may not be available. An instrument that is externally valid helps obtain
population generalizability, or the degree to which a sample represents the population.

Content validity refers to the appropriateness of the content of an instrument. In other

words, do the measures (questions, observation logs, etc.) accurately assess what you want to
know? This is particularly important with achievement tests. Consider that a test developer wants
to maximize the validity of a unit test for 7th grade mathematics. This would involve taking
representative questions from each of the sections of the unit and evaluating them against the
desired outcomes.

Reliability

Reliability can be thought of as consistency. Does the instrument consistently measure

what it is intended to measure? It is not possible to calculate reliability; however, there are four
general estimators that you may encounter in reading research:

Inter-Rater/Observer Reliability: The degree to which different raters/observers give

consistent answers or estimates.

Test-Retest Reliability: The consistency of a measure evaluated over time.

Parallel-Forms Reliability: The reliability of two tests constructed the same way, from the same
content.
 Internal Consistency Reliability: The consistency of results across items, often measured
with Cronbach’s Alpha.

Relating Reliability and Validity

Reliability is directly related to the validity of the measure. There are several important
principles. First, a test can be considered reliable, but not valid. Consider the SAT, used as a
predictor of success in college. It is a reliable test (high scores relate to high GPA), though only a
moderately valid indicator of success (due to the lack of structured environment – class attendance,
parent-regulated study, and sleeping habits – each holistically related to success).

Second, validity is more important than reliability. Using the above example, college
admissions may consider the SAT a reliable test, but not necessarily a valid measure of other
quantities colleges seek, such as leadership capability, altruism, and civic involvement. The
combination of these aspects, alongside the SAT, is a more valid measure of the applicant’s
potential for graduation, later social involvement, and generosity (alumni giving) toward the alma
mater.

Finally, the most useful instrument is both valid and reliable. Proponents of the SAT argue
that it is both. It is a moderately reliable predictor of future success and a moderately valid measure
of a student’s knowledge in Mathematics, Critical Reading, and Writing.

Validity and Reliability in Qualitative Research

Thus far, we have discussed Instrumentation as related to mostly quantitative measurement.

Establishing validity and reliability in qualitative research can be less precise, though
participant/member checks, peer evaluation (another researcher checks the researcher’s inferences
based on the instrument (Denzin & Lincoln, 2005), and multiple methods (keyword: triangulation),
are convincingly used. Some qualitative researchers reject the concept of validity due to the
constructivist viewpoint that reality is unique to the individual, and cannot be generalized. These
researchers argue for a different standard for judging research quality.

Source:

https://researchrundowns.com/quantitative-methods/instrument-validity-reliability/

Examples of Questionnaire

The following are the different reasons that might have inspired you why you enrolled in the
Bachelor of Science in Accountancy (BSA) program. Please check (✓) the rating which you think
influenced you in taking up BSA program using the following scale:
 (5) Strongly Agree (4) Agree (3) Neutral/Undecided (2) Disagree (1) Strongly
Disagree

PERSONAL FACTORS 5 4 3 2 1
I enrolled in the BSA program because:
1. This is my dream.
2. I like to answer questions in figures.
3. I would enjoy being an Accountant.
4. I like accounting and it is my personal interest.
5. I would find accounting as very interesting field.
6. I can do better in calculation/working with numbers.
7. My foundation studies in senior high school are related to my course.
8. I always performed well in my academics during senior high school.
9. I always got good grades in math or accounting during my senior high school.
10. others, pls specify: ______________________________________________
REFERENCE FACTORS 5 4 3 2 1
I enrolled in the BSA program because:
1. I want to be with my friends.
2. I have a relative who is an accountant.
3. My parents wanted me to choose accounting course.
4. My friends encouraged me to take accounting course.
5. My brother/sister encouraged me to take accounting course.
6. My former teacher suggested that I should take accounting course.
7. My parent’s occupation motivated me to choose accounting course.
8. Promotional information/mass media made me aware of the course.
9. My parent’s educational background influenced me to choose accounting course.
10. others, pls specify: ______________________________________________
JOB-RELATED FACTORS 5 4 3 2 1
I enrolled in the BSA program because:
1. The accounting profession is well respected.
2. There will be more jobs available for accounting graduates.
3. Accounting is a field with a very broad exposure to business.
4. Being an accountant brings a lot of prestige and social status.
5. I can get a high paying job if I graduate with accounting degree.
6. I believe accounting is a profession in par with medicine and law.
7. I believe it can provide me with a better career life and job security.
8. There will always be a great job market demand for accounting graduates.
9. I want to work independently and put up my own company related to accounting.
10. others, pls specify: ________________________________________
MEDIA AND PUBLICITY FACTORS 5 4 3 2 1
I enrolled in the BSA program because:
1. School fairs and programs influenced me to choose accounting course.
2. The information from the school’s website ignited my interest in accounting.
3. I received letters from various institutions which convinced me to take BSA program.
4. The advertisement on various mass media influenced me to choose accounting as my
course.
5. The banners along the roads about the accounting course offered by the school
motivated me in choosing accounting.
Please rate the contribution of the MPA Program of XYZ University to your personal and
professional growth. Please put a check (✔) on appropriate space:

Contribution Contribution is Contributon Contribution No

is VERY HIGH is FAIR is POOR Contribution
Program Contributions HIGH (4) (3) (2) At All (1)
(5)
Meeting the needs of the profession
Exposure to international community
Exposure to local community
Information technology skill
Personality development
Decision-making skill
Problem solving skill
Critical thinking skill
Human relation skill
Opportunities abroad
Research capability
Communication skill
Salary improvement
Learning efficiency
Organizational skill
Values formation
Leadership skill
Job promotion
Moved to another company that considered my MBA
Degree
Other: pls. specify___________

How would you rate the MBA Program of XY University ? Please put a check (✔) on
appropriate space:
EXCELLENT VERY GOOD FAIR POOR
Program Aspects (5) GOOD (3) (2) (1)
(4)
Professor's practice and method of teaching
Professor's knowledge of subject matter
Teaching and learning environment
Commitment given to research
Time to complete the program
Teacher-student relationships
Range of courses/subjects
Relevance to profession
Extracurricular activities
 Interdisciplinary learning
Quality of instruction
Library resources
Class size
Modular Approach (6 consecutive meetings per subject)
Other: pls. specify_________________

The following are the different teaching practices of the faculty members, please check (
✓) the rating which you think best describes their performance using the following scale:
3 – Always 2 – Sometimes 1 – Never

Delivery of Instruction Always Sometimes Never

Encouraging contact between students and faculty
Share past experiences, values, and attitudes.
Try to get to know their students by name by the end of the first
three weeks of the term.
Encourage students to participate in consultation
Seek out students who seem to be having problems with the course
Seek out students who are missing class frequently
Provide personalized feedback on student assignments
Ask the students how they are doing.
Help students network with other faculty in their area of interest
Attend and support events led by student groups.
Use email regularly to encourage and inform.
Developing reciprocity and cooperation among students
Engage in "ice breaker" as well as learning activities during the first
week of classes that encourage students to get to know one another.
Ask students to share information about each other’s backgrounds
and academic interests.
Create "learning communities," study groups, and team projects
with individual accountability.
Arrange the physical classroom environment, if possible, to better
foster interaction.
Encourage students to prepare together for classes or exams.
Ask students to give constructive feedback on each other’s work.
Ask students to explain difficult ideas to each other.
Use small group discussions, collaborative projects in and out of
class, group presentations, and case study analysis.
 Ask students to discuss key concepts with other students whose
backgrounds and viewpoints are different from their own.
Encouraging active learning
Provide concrete, real-life scenarios to help students apply
theoretical concepts
Provide application activities that go beyond topics or activities
provided in the textbook
Ask questions frequently that require participation through
discussion groups, response cards, polling or use of classroom
response systems, learning partners, panels, or games
Encourage students to suggest new readings, projects or course
activities
After providing test results, ask students what they will do differently
next time
Provide a variety of options for completing projects and major
assignments from which students can choose
Ask students to present their work to the class.
Ask students to summarize similarities and differences among
research findings, artistic works or laboratory results.
Encourage use of professional journals.
Use technology to encourage active learning.

TOPIC 11
DATA GATHERING PROCEDURES

Academic work is not easy. The quality of the content is determined by the credibility of
the content. Therefore, you need many data to justify your viewpoint. Although the internet has
made things easy for those who need information, it is important to verify the reliability of the data
you collect before using it.
The data gathering procedure you employ in your paper determines if you receive a piece
that is trustworthy or not. Therefore, it is crucial to employ the best procedure to get the perfect
results. It improves the quality of the paper and makes you sound scholarly.

Most people struggle when need to gather data. While some do not know the data collection
methodologies to follow, the majority do not have the experience in data handling. Eventually,
they prepare papers that only earn them low grades. What is the remedy in such cases? Have a
look at a perfect data gathering procedure example to be well-versed with the procedure that can
work for your situation. In the process, you can make your work easier and improve the general
quality of the papers you can prepare.

What Is the Definition of Data Gathering Procedure?

Research writing involves the handling of statistical data. Therefore, you need to know the
best data to use in your paper. The definition of data gathering procedure is that it is the technique
used to obtain the information used in a research to substantiate the claims made by a writer. To
get the perfect outcome, you should use the best procedure. If you are unsure of how to obtain your
data, it is advisable to hire experts in this field to offer assistance. We have data experts who can
help with these tasks.

What are the data collection methods that you can use? They are explained below:

Use of surveys
The method is mainly effective for those who need qualitative data to use in their academic
documents. In surveys, open-ended questions are used. What kind of information can be collected
using this method? They include the perception people have on a product, attitudes towards
government policy, the beliefs people hold, or the knowledge people have on a given issue, among
other information types. For the exact information needed, the questions should not be leading and
should cover the exact areas needed by the researchers. The data is later analyzed to obtain the
conclusions needed.

Conducting interviews
This quantitative research data gathering procedure is used to obtain from people on a one-
on-one basis. In this case, the researcher should have several predetermined questions. The
interview questions can be close-ended, like in the case where the interviewees are expected to
provide the ‘YES’ or ‘NO’ type of responses. It can also have open-ended questions in which the
respondent has the freedom to provide a response they are comfortable with. To ensure the data
collected is rich in the content required, the interviewer should ensure there are follow-up questions
for areas where the respondent may provide ambiguous information.

There are different ways the interviews can be conducted. The first way is to do it face-to-
face. As the respondent provides the answers, the interviewee can record them by writing or tape-
recording. The data collected is later sorted and written in the paper. The other method is through
phone conversations. Your respondent should provide the answers required as you keep a clean
record that you can use later to write the paper needed.

Taking a focus group

 In this case, the interviewee can take a group and get the information from them. There is
a set of predetermined questions that are inquired from the respondents in turns. The method is
effective when different people hold varied opinions on the same issue. Focus groups differ
depending on the type of responses required in the probe. To get the most reliable results from this
method, the number of people in the group should be between 5 and 10 people.

Direct observation
The data gathering procedure for qualitative research applies the sensory organs such as
the eyes to see what is going on, ears to hear the things going on, and the ears to smell. The method
helps the researcher to avoid bias in what people say.

Content Analysis
The researcher uses data that is already available and supports their point of view. Different
documents can be used in this case, including newspapers with reputation, research articles from
known experts, approved government reports, and other online data sources that can be of help in
this case. For the reliability of the data, different sources should be used for research.

It is you to determine the methodology that can work for your case when it comes to data
collection. Choosing a wrong procedure may mean that you obtain unreliable or irrelevant data.
You do not want to face the frustrations of presenting data that is unrelated to your topic. Therefore,
it is advisable to hire an expert who understands how things work as far as data is concerned. We
come in handy in such situations. Do not use faulty data gathering procedures when we can assist
you in collecting the best data using our proven collection techniques.

What Determines the Sample of Data Gathering Procedure

Not all the procedures are effective for your paper. What applies to one paper may not be
recommended for another. What are the factors in assessing to settle on the best procedure? Get
answers:

The Course and Topic of Study Handled

Different courses require varying procedures when it comes to the collection and handling
of data. While there are those courses where secondary information sources can work, others need
data that one obtains first first-hand. For example, the type of data that is acceptable for those
handling engineering courses is not the same as what works for those pursuing psychology. The
same applies to the topic. The data needs for different subjects vary. Therefore, you must analyze
the needs of your course and topic before selecting a procedure for data gathering.

Personal Preferences in Data Gathering

The convenience encountered in data gathering varies from one person to the next. What
one person considers to be hard may be easy for another. On a personal level, you should opt for
a procedure that you are comfortable with. It is you who decide on the topic, settle on the data,
analyze and come up with the conclusion. Therefore, selecting a procedure you are sure can work
for you is fundamental. A convenient information gathering procedure saves you from stress.

What Should You Do Before Data Gathering?

You should not embark on the data gathering if you are unsure of what is required. The
first step is to analyze and understand the topic you have. The keywords encountered determine
whether you need a quantitative or qualitative type of data. Where you are expected to settle on
your own topic, take something you are sure you can easily obtain data to defend.
The next procedure is to study the guidelines that are provided for doing the paper and
collection of the data. For example, some professors insist that a student should use a given method
of data collection. Your grade depends on whether you adhere to that specification or not.

Source:
https://www.masterpapers.com/blog/data-gathering-procedure

Survey Data Collection: Definition, Methods with Examples and Analysis

Survey Data: Definition

 Survey data is defined as the resultant data that is collected from a sample of respondents
that took a survey. This data is comprehensive information gathered from a target audience about
a particular topic of interest to conduct research on the basis of this collected data.

There are many methods used to gather survey data for statistical analysis in research.
Various mediums are used to collect feedback and opinions from the desired sample of individuals.
While conducting survey research, researchers prefer multiple sources to gather data such as online
surveys, telephonic surveys, face-to-face surveys, etc. The medium of gathering survey data
decides the sample of people that are to be reached out to, to reach the requisite number of survey
responses.

Factors of collecting survey data such as how the interviewer will contact the respondent
(online or offline), how the information is communicated to the respondents etc. decide the
effectiveness of gathered data.

Survey Data Collection Methods with Examples

The methods used to collect survey data have evolved with the change in technology. From
face-to-face surveys, telephonic surveys to now online and email surveys, the world of survey data
collection has changed with time. Each survey data collection method has its pros and cons and
every researcher has a preference for gathering accurate information from the target sample.

The survey response rates for each of these data collection methods will be different as
their reach and impact is always different. Different methods are chosen according to specific
target population characteristics as well as intent to examine human nature under various
situations.

Types of Survey Data based on Deployment Methods:

There are four main survey data collection methods – Telephonic Surveys, Face-to-face
Surveys, and Online Surveys.

Online Surveys

Online surveys are the most cost-effective and can reach the maximum number of people
in comparison to the other mediums. The performance of these surveys is much more widespread
than the other data collection methods. In situations where there is more than one question to be
asked to the target sample, certain researchers prefer conducting online surveys over the traditional
face-to-face or telephone surveys.
 Online surveys are effective and therefore require computational logic and branching
technologies for exponentially more accurate survey data collection, vs any other traditional means
of surveying. They are straightforward in their implementation and take a minimum time of the
respondents. The investment required for survey data collection using online surveys is also
negligible in comparison to the other methods. The results are collected in real-time for researchers
to analyze and decide corrective measures.
A very good example of an online survey is a hotel chain using an online survey to
collect guest satisfaction metrics after a stay or an event at the property.
Learn more: Quality Of Life Survey Questions + Sample Questionnaire Template

Face-to-face Surveys

Gaining information from respondents via face-to-face mediums is much more effective
than the other mediums because respondents usually tend to trust the surveyors and provide honest
and clear feedback about the subject in-hand.

Researchers can easily identify whether their respondents are uncomfortable with the asked
questions and can be extremely productive in case there are sensitive topics involved in the
discussion. This online data collection method demands more cost-investment than in comparison
to the other methods. According to the geographic or psychographic segmentation, researchers
must be trained to gain accurate information.

For example, a job evaluation survey is conducted in person between an HR or a manager

with the employee. This method works best face-to-face as the data collection can collect as
accurate information as possible.

Telephone Surveys

Telephone surveys require much lesser investment than face-to-face surveys. Depending
on the required reach, telephone surveys cost as much or a little more than online surveys.
Contacting respondents via the telephonic medium requires less effort and manpower than the
face-to-face survey medium.

Interviewers, if located at the same location, can cross-check their questions to ensure
error-free questions are asked to the target audience. The main drawback of conducting telephone
surveys is that establishing a friendly equation with the respondent becomes challenging due to
the bridge of the medium. Respondents are also highly likely to choose to remain anonymous in
their feedback over the phone as the reliability associated with the researcher can be questioned.
 For example, if a retail giant would like to understand purchasing decisions, they can
conduct a telephonic, motivation and buying experience survey to collect data about the entire
purchasing experience.

Paper Surveys

The other commonly used survey method is paper surveys. These surveys can be used
where laptops, computers, and tablets cannot go and hence they use the age-old method of data
collection; pen and paper. This method helps collect survey data in field research and helps
strengthen the number of responses collected and the validity of these responses.

A popular example or use case of a paper survey is a fast food restaurant survey where the
fast food chain would like to collect feedback on the dining experience of its patrons.
Types of Survey Data based on the Frequency at which they are Administered:

Surveys can be divided into 3 distinctive types on the basis of the frequency of their distribution.
They are:

Cross-Sectional Surveys
Cross-sectional surveys are an observational research method that analyzes data of
variables collected at one given point of time across a sample population. population or a pre-
defined subset. The survey data from this method helps the researcher understand what the
respondent is feeling at a certain point in time. It helps measure opinions in a particular situation.
For example, if the researcher would like to understand movie rental habits, a survey can be
conducted across demographics and geographical locations. The cross-sectional survey, for
example, can help understand that males between 21-28 rent action movies and females between
35-45 rent romantic comedies. This survey data helps for the basis of a longitudinal study.

Longitudinal Surveys

Longitudinal surveys are those surveys that help researchers to make an observation and
collect data over an extended period of time. This survey data can be qualitative or quantitative in
nature and the survey creator does not interfere with the survey respondents.
For example, a longitudinal study can be carried out for years to help understand if mine workers
are more prone to lung diseases. This study takes year and discounts any pre-existing conditions.
Retrospective Surveys

In retrospective surveys, researchers ask respondents to report events from the past. This
survey method offers in-depth survey data but doesn’t take as long to complete. By deploying this
kind of survey researchers can gather data based on past experiences and beliefs of people.
For example, if hikers are asked about a certain hike – the conditions of the hiking trail, ease of
hike, weather conditions, trekking conditions, etc. after they have completed the trek, it is a
retrospective study.

Survey Data Analysis

After the survey data has been collected, this data has to be analyzed to ensure it aids
towards the end research objective. There are different ways of conducting this research and some
steps to follow. They are as below:

Survey Data Analysis: Steps and Tips

There are four main steps of survey data analysis:

Understand the most popular survey research questions: The survey questions should align
with the overall purpose of the survey. That is when the collected data will be effective in helping
researchers. For example, if a seminar has been conducted, the researchers will send out a post-
seminar feedback survey. The primary goal of this survey will be to understand whether the
attendees are interested in attending future seminars. The question will be: “How likely are you to
attend future seminars?” – Data collected for this question will decide the likelihood of success of
future seminars.

Filter obtained results using the cross-tabulation technique: Understand the various
categories in the target audience and their thoughts using cross-tabulation format. For example, if
there are business owners, administrators, students, etc. who attend the seminar, the data about
whether they would prefer attending future seminars or not can be represented using cross-
tabulation.

Evaluate the derived numbers: Analyzing the gathered information is critical. How many
of the attendees are of the opinion that they will be attending future seminars and how many will
not – these facts need to be evaluated according to the results obtained from the sample.
Draw conclusions: Weave a story with the collected and analyzed data. What was the intention of
the survey research and how does the survey data suffice that objective? – Understand that and
develop accurate conclusive results.

Survey Data Analysis Methods

Conducting a survey without having access to the resultant data and the inability to drawing
conclusions from the survey data is pointless. When you conduct a survey, it is imperative to have
access to its analytics. It is tough to analyze using traditional survey methods like pen and paper
and also requires additional manpower. Survey data analysis becomes much easier when using
advanced online data collection methods with an online survey platform such as market research
survey software or customer survey software.
Statistical analysis can be conducted on the survey data to make sense of the data that has been
collected. There are multiple data analysis methods of quantitative data. Some of the commonly
used types are:

Cross-tabulation: Cross-tabulation is the most widely used data analysis methods. It uses a
basic tabulation framework to make sense of data. This statistical analysis method helps tabulate
data into easily understandable rows and columns and this helps draw parallels between
different research parameters. It contains data that is mutually exclusive or have some connection
with each other.

Trend analysis: Trend analysis is a statistical analysis method that provides the ability to
look at survey-data over a long period of time. This method helps plot aggregated response data
over time allows drawing a trend line of the change, if any, of perceptions over time about a
common variable.

MaxDiff analysis: The MaxDiff analysis method is used to gauge what a customer prefers
in a product or a service across multiple parameters. For example, a product’s feature list,
difference with the competition, ease of use and scale, pricing, etc. form the basis for maxdiff
analysis. In a simplistic form, this method is also called the “best-worst” method. This method is
very similar to conjoint analysis but is much easier to implement and can be interchangeably used.

Conjoint analysis: As mentioned above, conjoint analysis is similar to maxdiff analysis

only differing in its complexity and the ability to collect and analyze advance survey data. This
method analyzes each parameter behind a person’s purchasing behavior. By using conjoint
analysis, it is possible to understand what exactly is important to a customer and the aspects that
are evaluated before purchase.

TURF analysis: TURF analysis or Total Unduplicated Reach and Frequency Analysis, is a
statistical research methodology that assesses the total market reach of a product or service or a
mix of both. This method is used by organizations to understand the frequency and the avenues at
which their messaging reaches customers and prospective customers. This helps them tweak their
go-to-market strategies.

Gap analysis: Gap analysis uses a side-by-side matrix question type that helps measure the
difference between expected performance and actual performance. This statistical method for
survey data helps understand the things that have to be done to move performance from actual to
planned performance.
 SWOT analysis: SWOT analysis, another widely used statistical method organizes survey
data into data that represents strength, weaknesses, opportunities, and threats of an organization or
product or service that provides a holistic picture of competition. This method helps to create
effective business strategies.

Text analysis: Text analysis is an advanced statistical method where intelligent tools make
sense of and quantify or fashion qualitative and open-ended data into easily understandable data.
This method is used when the survey data is unstructured.

Source:

https://www.questionpro.com/blog/survey-data-collection/

TOPIC 12
DATA ANALYSIS AND STATISTICAL ANALYSIS

Difference between Data Analysis and Statistical Analysis

Fifty years, ago, the lines between "data analysis" and "statistical analysis" were pretty
clear. But as data analysis evolved, those lines became blurred. The differences between the two
terms are now very much a grey area, but there are still a few notable differences.

What is "Data Analysis"?

Data scientists and statisticians typically define "data analysis" in different ways.
For a data scientist, data analysis is sifting through vast amounts of data: inspecting, cleansing,
modeling, and presenting it in a non-technical way to non-data scientists. The vast majority of this
data analysis is performed on a computer.

If you're a statistician, instead of "vast amounts of data" you'll usually have a limited
amount of information in the form of a sample (i.e. a portion of the population); Data analysis is
performed on this sample, using rigorous statistical techniques.

Both data scientists and statisticians use data to make inferences about consumer cohorts,
a general population, or target market. However, they will approach the issue of data analysis quite
differently.
 A data analyst will have a data science toolbox (e.g. programming languages like Python
and R, or experience with frameworks like Hadoop and Apache Spark) with which they can
investigate the data and make inferences.

On the other hand, a statistical analyst will generally use mathematical-based techniques
like hypothesis testing, probability and various statistical theorems to make inferences. Although
much of a statistician's data analysis can be performed with the help of statistical programs like R,
the analysis is more methodical and targeted to understanding one particular aspect of the sample
at a time (for example, the mean, standard deviation or confidence interval).

The lifecycle of data is key to data workflow in data science:

You can perform many data analysis steps in data science with very little statistical basis:
data prep, transforming data.

What is Statistical Analysis?

Generally speaking, statistical analysis is the science of uncovering patterns and trends in
data, using statistics. Note the key word here is "statistics". In order to perform any statistical
analysis at all you have to use statistics. Historically, only statisticians used statistical techniques
on data. And data science wasn't even a thing in the mainframe days of tape mounting and Cobol
programming. But as data science has evolved, it's blended with many areas once thought to be
the exclusive realm of the statistician: data visualization, optimization, high-dimensional analysis
to name but a few.

Data Analysis vs. Statistical Analysis

There is a large grey area: data analysis is a part of statistical analysis, and statistical
analysis is part of data analysis. Any competent data analyst will have a good grasp of statistical
tools and some statisticians will have some experience with programming languages like R.

If you're confused about where the line is, or where that separation occurs, the key question
really is, Are the two fields of data science and statistics really separate entities?

Statistical analysis is used in order to gain an understanding of a larger population by

analyzing the information of a sample. Statistical analysis allows inferences to be drawn about
target markets, consumer cohorts and the general population by expanding findings appropriately
to predict the behaviour and characteristics of the many based on the few.

Data analysis is the process of inspecting, presenting and reporting data in a way that is
useful to non-technical people. Because data is next to useless if it can’t be understood by the
decision-makers who need to use it, data analysts act as translators between the numbers and
figures and the people who need to know about them.
Source:
Stephanie Glen
Difference between Data Analysis and Statistical Analysis
https://www.datasciencecentral.com/profiles/blogs/difference-between-data-analysis-and-
statistical-analysis