CHAPTER III

RESEARCH METHODOLOGY
This chapter describes the methodology of the research. It covers the
research approach, research design, research subject, data collecting techniques,
research technique, and data analysis technique.
A. Research Approach
This research employed a quantitative approach with pre-test and post-
test. A quantitative inquiry approach helps comprehend a phenomenon
(Creswell, 2012). The major characteristics of quantitative research among
other things are investigating a research problem by explaining a relationship
between variables, collecting numeric data from a large number of people using
instruments with fixed questions and responses, and analyzing data, comparing
groups, or relating variables through the use of statistical methods (Creswell,
2012). Moreover, according to Setiyadi (2006), the quantitative design seeks
to determine the validity of a hypothesis by analyzing the available data. This
indicates that the quantitative approach is a research design that statistically
investigates theories and data collection.
B. Research Design
This research employs a quasi-experimental design to determine whether
there is a significant difference between pupils who use VOA videos and those
who do not. According to Creswell (2012), quasi-experimental designs (also
known as intervention studies or group comparison studies) are procedures that
determine whether a particular activity or material affects participant results.
According to Creswell (2012), quasi-experimental research is an
experimental situation in which the researcher assigns subjects, but not
randomly. The researcher took classes based on the teacher's suggestion, and
the participants formed groups because the experimenter was unable to
construct groups for the experiments artificially. The researcher also separated
the class into experimental and control groups. In addition, Lapp & Fisher
(2011) state that participants in a quasi-experimental study are assigned to the
experimental and control groups by a method other than random assignment.
 This research required five sessions to complete. At the first meeting, this
research conducted the pre-tests for the first session for experimental and
control groups. The VOA video implementation was utilized for the next three
encounters with the experimental group. In the control group, the research
employed more conventional methods of teaching by only using the student
textbooks that have been provided. And for their final class session, this
research conducted post-tests for both groups.
Table 3. 1 Quasi-Experimental Design

Group Pre-test Treatment Post-test

Experimental Group O1 X O2

Control Group O1 - O2

Note : O1: Pre-test of experimental group and control group

O2: Post-test of experimental group and control group
X: Treatment by using VOA video
Therefore, the research design can be concluded as the following figure:

Using Authentic VOA Video

Material to Improve
Students’ Listening Skill

Experimental Class Control Class

Pre-test Pre-test

Using VOA video Without using VOA video

 Post-test Post-test

Result

Figure 3. 1 Research Procedure

This research employs an experimental design, especially a quasi-

experimental design with two groups, the experimental class and the control
class. Creswell (2012) states that experimental design is a procedure in
quantitative research in which the researcher determines whether an activity or
material affects the outcomes of participants. The characteristic of this research
is to compare two groups of comparable subjects, one of which will receive
treatment and the other will not. The treatment given to the experimental class
asked them to listen to VOA videos in the learning materials.
C. Research Subject
The research subject in this research contains the research site,
population, sample, and sampling technique.
1. Research Site
This research was conducted in the 10th grade of SMK Tunas Sinar
Mandiri Tanjungsari Bogor which is located on Street Transyogi, Tanjungsari,
Bogor, West Java. This school was selected as the research site due to its
proximity to the researcher's home and its English Teacher's easy accessibility.
2. Population
According to Creswell (2012), a population is a group of individuals who
all have the same qualities. The population in this research are all of the 10th-
grade students at SMK Tunas Sinar Mandiri in the academic year 2022–2023.
There are three classes in 10th grade.
 Table 3. 2 Population of the Students

Class X (Total Students)

XA 30
XB 30
XC 30

3. Sample
A sample is a subset of a population that a researcher examines to
conclude that population (Creswell, 2012). There are two significant sampling
method categories: probability and non-probability (Elfil & Negida, 2017). A
non-random sample is a type of sample in which the sampling elements are
chosen using a method other than a mathematically random process (Neuman
et al., 2014). The non-probability (non-random) sampling technique is used in
this research. The research sample included two classes X B and X C from the
population. Then put the students into two groups.
Table 3. 3 The Research Sample

Group Class
Control Group XB
Experiment Group XC

4. The Sampling Technique

According to Arikunto (2013), sampling is the process of choosing a
research group from a population. The two strategies for choosing samples are
probability sampling and non-probability sampling (Blaxter et al., 2010).
Simple random sampling, systematic random sampling, stratified random
sampling, cluster sampling, and multistage sampling are all types of probability
sampling. A non-probability sampling, on the other hand, includes quota
sampling, accidental sampling, and purposeful sampling.
D. Data Collection Technique
The data collection technique provides data information results. In this
section, the validity test, reliability test, and level of difficulty test are
described.
1. Validity Testing
Validity measures the degree to which the targeted content area and the
test must conform to the score. Creswell (2012) states that validity is the
process of gathering information that shows the intended test interpretation (of
the idea or construct the test is expected to evaluate) corresponds to the test's
stated purpose.
The validity of this research uses content validity. Content validity can
be conducted by comparing the instrument with the material taught (Sugiyono,
2017). This indicates that the purpose of the validity test is to determine
whether the instrument that is administered to a sample group is suitable. The
appropriateness of the test scoring for the pre-test and post-test is also
determined by this validation. Thirty 10th grade who are not a part of the
experiment and control are used in this validation test. The test score must be
valid. This test was analyzed by Anates v.4 application (can be seen in
appendix). Based on the validity test, those 30 questions are valid and can be
used for pre-test and post-test.
The result can be taken as proving the validity of the test or question. The
table below provides the interpretation as follows:

Table 3. 4 Coefficient Correlation

Interval Coefficient Degree of Relationship

0,00 – 0,199 Very low
0,20 – 0,399 Low
0,40 – 0,599 Average
0,60 – 0,799 High
0,80 – 1,000 Very high
2. Reliability
Reliability is a necessity for every good test and a test must be reliable as
a measuring instrument. Reliability is related to measuring consistency (Heale
& Twycross, 2015). Reliability is the ability of individual scores from an
instrument to be virtually identical to the table across administration, both from
causes of measurement error and consistent (Creswell, 2012). Besides that,
according to Bachman (2004), reliability also shows whether the instrument is
reliable and can be used as a device to collect the data or not. This research
used Anates v.4 to measure the reliability and the test was given to thirty
students in tenth grade. The result shows that the reliability trial test is 0,95
(can be seen in the appendix).
Guilford's index can be used to determine the test's high and low
coefficients of reliability; the results are shown in the table below:

Table 3. 5 The Interpretation Reliability on the Matter

Index Reliability Interpretation

0,00 < ri 0,20 = 0,20 Very low
0,20 < ri 0,20 = 0,40 Low
0,40 < ri 0,20 = 0,60 Medium
0,60 < ri 0,20 = 0,80 High
0,80 < ri 0,20 = 1,00 Very high

3. Level of Difficulty Test

A good test should meet the criteria and be standardized. An appropriate
test is neither too easy nor too difficult. Difficulty index is the number that
shows the level of difficulty of a test (Suharsimi, 2006). In this index, the lower
index of the test means the more difficult the text is and the higher index of the
test means the easier the text is. This test was analyzed by Anates v.4 (can be
seen in the appendix).
 The following table provides an interpretation of the difficulty level
value:

Table 3. 6 Interpretation of Difficulty Index

Difficulty of Index Interpretation

P < 0,30 Difficult

0,30 ≤ P ≤ 0,70 Medium
0,70 ≤ P ≤ 1,00 Easy

E. Research Technique
The research technique in this research used a pre-test, treatment, and
post-test.
1. Pre-test
According to Creswell (2012), a pre-test is given before students receive
treatment which aims to measure some attribute or characteristic in assessing.
It means that students give a pre-test before giving a treatment to improve
students listening skills. In other words, the pre-test is used to know the
students’ basic competence and knowledge about the subject that would be
taught. The form of the pre-test given about “House Fires” was fill-in-the-blank
questions. This pre-test was administered to students in control group. While
in the experimental group the pre-test with video titles “Chooses Native Plants
for Your Garden” was fill-in-the-blank which consist of 30 questions.
2. Treatment
Treatment is intended to influence an outcome in the experimental
design. In this research, treatment was conducted in the experimental and
control group (Creswell, 2012). The experimental group was treated with VOA
video, while the control group was treated without VOA video. This research
did the treatment three times in both groups.
a. Experimental Group Treatment
In the first meeting, students learn about report text, both the definition and
types of report text, likewise illustrations in their daily life. After that,
 students are invited to listen and pay attention to an example of report text
taken from VOA entitled "Help with Buying Young Garden Plants in the
Spring" which lasts for 4 minutes. Then students are given a sheet of work
and are invited to discuss with their groups to answer the fill-in-the-blank
that has been provided. This video was played three times because it was
feared that there were students who were left behind. When finished, student
representatives present the results of the group assignment in front of the
class. For the second and third meetings, the action learning technique was
the same as the first meeting, only the video used was different. At the
second meeting, the video with the title "Plants, Flowers to Enjoy by
Moonlight" 4 minutes. Furthermore, at the last treatment, students were
given a video entitled "When Planning Spring Garden, Think About Next
Winter's Too" The duration of this video was 5 minutes.
b. Control Group Treatment
For the control group, the treatment was the same as for the experimental
group, including learning about the report text, both its definition and type.
Besides that, this group also gave 3 videos from the first treatment meeting
to the last treatment meeting. It is just that the video provided is not from
VOA but from the tenth-grade textbook, they are about the conversation
between Ira and friska, then matching the news with the appropriate
response, and the last is news broadcast on radio.
3. Post-test
Post-test provides a measure of some attributes or features which is
measured for the research participations after they get treatment (Creswell,
2012). This post-test is important because it measures the effect of treatments
on students. The test will be given similar to the pre-test; students are given a
test that consists of 30 fill-in-the-blank questions.
F. Data Analysis
This research used the comparative technique to analyze the data, which
is analyzed and compared to the score of the experimental and control groups.
This technique is used to prove statistically whether there is any significance
 or not. The data analysis process has four steps. These include the normalcy
test, homogeneity test, hypothesis test, and calculation of the index N-gain.
1. Normality Test
According to Sugiyono (2017), the normality test is a statistical
procedure used to determine how data are distributed. To test the normality
test, this research used SPSS (Statical Package for Social Science). There are
two normality tests in SPSS: The Kolmogorov-Smimov test and the Shapiro-
Wilk test. The criterion of SPSS:
a. If respondents ≥ 50, the normality test uses Kolmogorov Smimov
b. If respondents ≤ 50, the normality test uses Shapiro-Wilk
In this research, the respondents are 30 students in the tenth grade in
SMK Tunas Sinar Mandiri. It means that there were fewer than 50 responses
in the sample. Shapiro Wilk is therefore used in this research. If the significant
score of the Shapiro-Wilk test is higher than 0.05, the data is normally
distributed, and if the score is lower than 0.05, the data deviates from a normal
distribution. According to Flowers (2009), these are the rules for a hypothesis:
a. H0 = significant score > 0.05 means the data is normality distributed
b. H1 = significant score < 0.05 means the data is not normality distributed
2. Homogeneity Test
This computation aims to see the similarity between the condition of the
two populations. This test's result must be higher than 0,05 to have
homogeneous data.
Throughout the research, SPSS is used as the method of data analysis.
Also the homogeneity test is calculated using SPSS. The criteria for the test:
a. Ha is accepted if the Sig value >0.05, the data is homogeneous
b. H0 is accepted if the Sig value <0.05, the data is not homogeneous
3. Hypothesis Test
A hypothesis is a statement in quantitative research that make a
prediction or conjecture about the outcome of a relationship between attributes
or characteristics (Creswell, 2012). In other words, hypotheses predict whether
 or not the research will impact the final result. The following is the hypothesis
of this research, which is described below.:
a. Null hypothesis (H0) accepted if accounted >0.05t-test. It means there is no
significant difference in students' listening skills using VOA video material.
b. Working Hypothesis (Ha) accepted if accounted <0.05 t-test. It means that
there is a significant difference in students' listening skills using VOA video
material.
Sugiyono (2017) states that an independent sample t-test will be applied
if the data are normally distributed and the sample is homogeneous. The
independent sample t-test will be calculated by SPSS. The criteria of the test
are:
a. If the probability value or sig. (2-tailed) < 0.05. it means there is a difference
significant between the results of the pre-test and post-test.
b. If the probability value or sig. (2-tailed) > 0.05. it means there is no
difference significant between the result of the pre-test and post-test.
4. Calculating N-Gain
Normalized Gain (N-Gain) is to find out the improvement of students'
learning outcomes by analyzing the results of the test before and after learning
(Hake, 1999) cited in Sundayana (2018). The N-gain formula appears as
follows:
posttestscore−pretestscore
N-Gain = , with category:
𝑚𝑎𝑥𝑖𝑚𝑢𝑚 𝑠𝑐𝑜𝑟𝑒−𝑝𝑟𝑒𝑡𝑒𝑠𝑡 𝑠𝑐𝑜𝑟𝑒

Table 3. 7 N-gain effectiveness interpretation category by Hake (1999) as

cited in Sundayana (2018)

Persentage Interpretation
< 40% Not effective
40 – 55% Less effective
56 – 75% Moderately effective
>76% Effective