Development and validation of science,

technology, engineering and mathematics

(STEM) based instructional material
Cite as: AIP Conference Proceedings 1848, 060001 (2017); https://doi.org/10.1063/1.4983969
Published Online: 30 May 2017

Ineu Gustiani, Ari Widodo and Irma Rahma Suwarma

ARTICLES YOU MAY BE INTERESTED IN

Science, technology, engineering, mathematics (STEM) as mathematics learning approach in

st
21 century
AIP Conference Proceedings 1868, 050024 (2017); https://doi.org/10.1063/1.4995151

Beating the numbers through strategic intervention materials (SIMs): Innovative science
teaching for large classes
AIP Conference Proceedings 1848, 060014 (2017); https://doi.org/10.1063/1.4983982

STEM-based workbook: Enhancing students’ STEM competencies on lever system

AIP Conference Proceedings 1848, 060005 (2017); https://doi.org/10.1063/1.4983973

AIP Conference Proceedings 1848, 060001 (2017); https://doi.org/10.1063/1.4983969 1848, 060001

© 2017 Author(s).
 Development and Validation of Science, Technology,
Engineering and Mathematics (STEM) based Instructional
Material
Ineu Gustiani1,a), Ari Widodo2) and Irma Rahma Suwarma3)
2)
International Program Science Education,, FPMIPA Universitas Pendidikan Indonesia, Jl. Dr. Setiabudi No. 229,
Bandung, 40154, Indonesia
2)
Departmen of Biology Education, FPMIPA Universitas Pendidikan Indonesia, Jl. Dr. Setiabudi No. 229, Bandung,
40154, Indonesia
3)
Departmen of Biology Education, FPMIPA Universitas Pendidikan Indonesia, Jl. Dr. Setiabudi No. 229, Bandung,
40154, Indonesia
a)
Corresponding author: ineugustiani@student.upi.edu

Abstract. This study is intended to examine the development and validation of simple machines instructional material
that developed based on Science, Technology, Engineering and Mathematics (STEM) framework that provides guidance
to help students learn and practice for real life and enable individuals to use knowledge and skills they need to be an
informed citizen. Sample of this study consist of one class of 8th grader at a junior secondary school in Bandung,
Indonesia. To measure student learning, a pre-test and post-test were given before and after implementation of the STEM
based instructional material. In addition, a questionnaire of readability was given to examine the clarity and difficulty
level of each page of instructional material. A questionnaire of students’ response towards instructional material given to
students and teachers at the end of instructional material reading session to measure layout aspects, content aspects and
utility aspects of instructional material for being used in the junior secondary school classroom setting. The results show
that readability aspect and students’ response towards STEM based instructional material of STEM based instructional
material is categorized as very high. Pretest and posttest responses revealed that students retained significant amounts
information upon completion of the STEM instructional material. Student overall learning gain is 0.67 which is
categorized as moderate. In summary, STEM based instructional material that was developed is valid enough to be used
as educational materials necessary for conducting effective STEM education.

INTRODUCTION
The scenario of the world in the 21st century found rapid progress in science and technology. In today's
information age, the main goal of the educational system should be provide students with information acquisition
skills instead of directly providing them with the information they needed, to prepare individuals for real life and
enable individuals to use knowledge and the needed skills to be an informed citizen.
In the 1990’s the National Science Foundation united science, technology, engineering, and mathematics and
created the STEM acronym, a strategic decision made by scientists, technologists, engineers, and mathematicians to
combine forces and create a stronger political voice. In educational practices, one of the most significant challenges
centers on introducing STEM-related issues and developing the competencies to address the issues students will
confront as citizens, because addressing this challenge requires an educational approach that first places life
situations and global issues in a central position and uses the four disciplines of STEM to understand and address the
problem [5]. STEM education is an interdisciplinary and applied approach that is coupled with real world, it
integrates the four disciplines and these subjects cannot and should not be taught in isolation, just as they do not
exist in isolation in the real world or the workforce. The purpose of STEM education is for all students to learn to
apply basic content and practices of the STEM disciplines to situations they encounter in life [5]. In recent years,

Mathematics, Science, and Computer Science Education (MSCEIS 2016)

AIP Conf. Proc. 1848, 060001-1–060001-7; doi: 10.1063/1.4983969
Published by AIP Publishing. 978-0-7354-1520-1/$30.00

060001-1
STEM education has received growing attention nationwide as STEM plays an increasingly critical role in the
nation’s economy, competitiveness and security [9].
Indonesia as a big country with a large number of natural and human resources should take a role within that
development. To be successful, Indonesia’s efforts to improve schools and raise student achievement must include
advancing students’ understanding of STEM. Through STEM education, students learn to become problem solvers,
innovators, creators, and collaborators and go on to fill the critical pipeline of engineers, scientists, and innovators so
essential to the future of Indonesia and the nation. The statistic data from BPS (Central Bureau of Statistic, 2010)
Indonesia has shown that Indonesia’s labors have dominated by 88 millions unskilled labors. The skilled labors
reached about 22.1 million skilled labors, and only 6.5 million mastered in their sector [1]. It indicates that there is
talent gap between the availability of skilled labors which have still not in exact accord to the needs of qualified
labors. Indonesia curriculum emphasizes on interdisciplinary aspects which encourage instructional process through
conceptual, contextual and comprehensive approach. Due to suitability between STEM educations to Indonesia
curriculum, it will be a good step to conduct STEM education in Indonesia education system. It is remarkable to
note that the need for a skilled STEM workforce is crucial that all young people, regardless of their future career
pathway, have the STEM knowledge and skills they need to be an informed citizen in an increasingly scientific and
technological society.
Instructional materials play a vital role in teaching and learning at various levels of education, it provide
opportunities for children to broaden and deepen their knowledge by providing a variety of firsthand,
developmentally appropriate experiences and by helping children acquire symbolic knowledge through representing
their experiences [3]. Association of American Publishers School Division stated that instructional materials’ means
all materials that are designed for use by pupils and their teachers as a learning resource and help pupils to acquire
facts, skills, or opinions or to develop cognitive processes [6]. Instructional materials may be printed or non-printed,
and may include textbooks, technology-based materials, other educational materials, and tests. Review of middle
school curriculum materials that determine how currently available learning materials align with national learning
goals and pedagogical criteria rooted in the literature found that the materials covered many topics at a superficial
level, focused on technical vocabulary, failed to consider students’ prior knowledge, lacked coherent scientific
explanations of real-world phenomena, and provide students with few opportunities to develop explanations of
phenomena [11]. The implementation of STEM education met some challenges such as: 1) in integrating technology
(T) and engineering (E) into science (S) and mathematics (M) activities that arrived from the lack of STEM area
recourses and professionals; and 2) in using STEM contexts because lack of coherency school curriculum to STEM
education program characteristic that arrived from unsupported local school policy [15]. Clearly, there is a need for
instructional materials that align with national science education standards and appropriate to students interests and
requirements for progression.
Internationally, there is a growing concern for developing STEM instructional material to prepare students for a
scientifically and technologically advanced society [8]. With global recognition of the importance of STEM
education, concerns have arisen from both research and curriculum perspectives about the lack of a unified focus
and the need for greater integration of the four disciplines [2]. The challenge then is how to achieve a more balanced
content representation in STEM education. Indonesia confronts several barriers to providing access to the essential,
high-quality educational materials necessary for effective STEM education. One is the need to revise existing
materials to ensure that they align with and support the new curriculum.
This paper is purposely made to answer the call for the need of STEM based instructional materials which help
students to prepare for real life and enable them to use knowledge and skills they need to be an informed citizen. To
this end, the central research questions that are the focus of this paper are How does the readability of STEM based
instructional material?, How does students response towards STEM based instructional material?, and How does
STEM based instructional material affects on students learning?.
STEM is an acronym for science, technology, engineering, and mathematics. STEM education removes the
traditional barriers erected between the four disciplines, by integrating them into one cohesive teaching and learning
paradigm. STEM education is an interdisciplinary approach to learning where rigorous academic concepts are
coupled with real-world lessons as students apply science, technology, engineering, and mathematics in contexts that
make connections between school, community, work, and the global enterprise enabling the development of STEM
literacy and with it the ability to compete in the new economy [4].
The four STEM disciplines described in order to better understand STEM, based on National Academy of
Engineering and National Research Council [13]. Science is the study of the natural world, including the laws of
nature associated with physics, chemistry, and biology and the treatment or application of facts, principles, concepts,
and conventions associated with these disciplines. Knowledge from science informs the engineering design process.

060001-2
Technology, while not a discipline in the strictest sense, comprises the entire system of people and organizations,
knowledge, processes, and devices that go into creating and operating technological artifacts, as well as the artifacts
themselves. Much of modern technology is a product of science and engineering, and technological tools are used in
both fields. Engineering is both a body of knowledge about the design and creation of human-made products and a
process for solving problems. Engineering utilizes concepts from science and mathematics as well as technological
tools. Mathematics is the study of patterns and relationships among quantities, numbers, and space. Unlike in
science, where empirical evidence is sought to warrant or overthrow claims, claims in mathematics are warranted
through logical arguments based on foundational assumptions. Mathematics is used in science, engineering and
technology.
Association of American Publishers School Division stated that instructional materials’ means all materials that
are designed for use by pupils and their teachers as a learning resource and help pupils to acquire facts, skills, or
opinions or to develop cognitive processes. Instructional materials may be printed or non-printed, and may include
textbooks, technology-based materials, other educational materials, and tests [6]. Review of middle school
curriculum materials that determine how currently available learning materials align with national learning goals and
pedagogical criteria rooted in the literature found that the materials covered many topics at a superficial level,
focused on technical vocabulary, failed to consider students’ prior knowledge, lacked coherent scientific
explanations of real-world phenomena, and provide students with few opportunities to develop explanations of
phenomena [11].

RESEARCH METHODOLOGY
This study was conducted at junior secondary school level. Samples of this study consist of one class of 8th
grader at a junior secondary school in Bandung that consists of 40 students. A pre-test and post-test were given
before and after implementation of the STEM based instructional material within the junior secondary school
classroom to measure students’ learning. In addition, a questionnaire of readability was given to examine the clarity
and difficulty level of each page of instructional material. A questionnaire of students’ response towards
instructional material given to students and teachers at the end of instructional material reading session to measure
the clarity, appropriateness, and suitability for use in the junior secondary school classroom setting. During
instructional material implementation, a pretest was given to students to assess their knowledge on the subject before
reading the instructional material. There is no lecture on the instructional material activities was given to the junior
secondary school students. Students were given a posttest at the end of the instructional material reading session to
measure knowledge gains.
The STEM based Instructional material was developed based on the current Indonesia curriculum. Instructional
material contains STEM concepts in simple machines topic for 8th grader of junior secondary school students. There
are science section that talks about the relationship of human body movement with simple machines, technology
section that talks about product of science which is related with simple machines, engineering section that challenge
students to conduct STEM activity based on engineering design process, and mathematics section that talks about
mechanical advantages of simple machines. For assessment purposes, instructional material contains a series of
quizzes, students’ activity worksheet, pretests and posttests that measure students learning growth before and after
reading STEM based instructional material and their response towards STEM based instructional material,
respectively. Instructional material was evaluated by junior secondary school science teachers for clarity,
appropriateness, and suitability for use in the junior secondary school classroom setting

RESULTS AND DISCUSSION

In reporting our findings, we revisit our research questions in turn.

Readability of STEM Based Instructional Material

The results were obtained from readability questionnaire that were administered to students before they read
STEM based instructional material. Students answered 18 question items which is represent each page of
instructional materials.

060001-3
 The questions were formulated based on three aspects/indicators, there are text comprehension, formulating main
idea and quizzes comprehension. The percentage of STEM based instructional material readability aspects can be
seen below.

FIGURE 1. Percentage of Readability Aspects .

The results show that each readability aspect of STEM based instructional material got quite high percentage.
Text comprehension aspect achieve 97.50% indicates that most of students are able to comprehend the reading text
on the instructional material, it was found that 2.5% of students met difficulties in comprehending some science
terminologies about human sceletal system such as radius and ulna. Previous research stated that familiarity and
conceptual difficulty of new words reflect students ’existing knowledge [12]. The word radius and ulna has been
found to be difficult for students to understand in the context of independent reading, whereas another word such as
bone, which can be defined with a single synonym, is more likely to be understood.
Formulating main idea aspect achieve 100% indicates that all of students had ability to formulate the main idea
of the reading text excellently. A study revealed that being able to determine the main idea helps readers to recall
important information and locating the main idea and significant details helps the reader understand the points the
writer is attempting to express, thus identifying the relationship between these will improve comprehension [7].
Quiz comprehension achieve 86.67% indicates that most of students are able to understand the instruction on each
question that was written in the quiz and they also could answer the quiz correctly. The average percentage of
readability aspect is 94.72%, it means that the content of instructional material can be understood by students easily.
Students are able to read and understand the content which is provide in instructional material, formulating main
idea of each page easily and understand the quiz easily.

Student’s Response towards STEM Based Instructional Material

The results were obtained from students’ response towards STEM based instructional material questionnaire that
were administered to students and teachers after they read STEM based instructional material. They answered 25
question items which is represent three aspects of instructional material, there are layout aspect, content aspect and
utility aspect. Each aspect contains four indicators and the questions were formulated based on those indicators. The
percentage of students’ response towards STEM based instructional material can be seen as follow.

FIGURE 2. Percentage of Layout Aspect

060001-4
 FIGURE 3. Percentage of Content Aspect

FIGURE 4.Percentage of Utility Aspect

The percentages of each indicator of those three aspects were interpreted based on the table below.
TABLE I. Questionnaire Percentage Interpretation
Categorization Percentage (%)
Very High 80.1-100%
High 60.1% - 80%
Moderate 40.1% - 60%
Low 20.1% - 40%
Very Low 0% - 20%

There are four indicators that were measured in layout aspect, text clarity that achieve 100% indicates that text
reading on instructional material is clear and easy to read. Picture clarity that achieve 95.67 % indicates that the
picture which was provided in instructional material is clear, have sufficient amount, and contained information or
explanation. Attractiveness achieve 100% indicates that the picture which was provided in instructional material is
very attractive. Picture suitability with content achieve 100% indicates that the picture which was provided in
instructional material is in accordance with science concept. Illustrations are literature in their own right and,
whether used by themselves or integrated with written texts, they sharpen the perception of children, stimulate their
imagination and increase their sense of observation [16].
Content aspect consists of four indicators, there are content representation that achieve 96.71% indicates
that STEM based instructional material could explain science concept and pose question which is related with
human daily life problems, encourage students to pose question or problem to the teacher bravely and encourage
students to conduct peer interaction. Content appropriateness that achieves 96.50% indicates that science concept
which is presented has well-structured, the explanation of concept can be understood by students step-by-step, and
example of the problem is appropriated with the concept. Symbol and sentence clarity that achieve 91.50% indicates
that sentences that were written in instructional material can be read easily, there is no sentence that was ambiguous
and students could understand symbols that were used in this instructional material. Technical term appropriateness

060001-5
that achieved 100% indicates that all of students could comprehend technical term that was written in this
instructional material.
The last aspect is utility aspect that consist of learning benefit or facilitate learning that achieve 94.50%
indicates that this instructional material could either facilitate students to learn levers concept or facilitate teacher to
teach levers concept easily. Learning interest and learning motivation that both achieve 100% indicates that both
students and teachers were interested to use STEM based instructional material and through the use of this learning
material, both students and teachers were motivated to learn science. Introduction of STEM aspect that achieve 85%
indicates that through the use of this learning material, both of teachers and students could recognized science,
technology, engineering and mathematics aspect in levers concept. Based on the table of questionnaire percentage
interpretation, all of indicators in layout aspect, content aspect and utility aspect were categorized as very high. It
indicates that the instructional material provided a good layout, content and utility.

Students Learning
The results were obtained from students’ conceptual understanding instrument that was administered as pretest
and posttest. The instrument consists of 15 multiple choice questions which are formulated based on indicator of
standard competence and basic competence. The average score for both pretest and posttest and N-gain is provided
on the table 2.
TABLE 2. Average Score And N-Gain Score
Test Average Score <g>
Pretest 56.50
0.67
Posttest 75.66

Table 2 show results for the pretest and posttest that were used to measure student learning gains on the STEM
topic covered. At mere glance, it can be observed that after students read the instructional material, students average
score were significantly improved when comparing the before and after results. The average score for the pretest is
56.50. After implementation of the instructional material, the posttest scores increased to 75.66. A positive gain
with score 0.67 was observed for students’ conceptual understanding test.
Table 3. Classification Of N-Gain Interpretation
Score Interpretation
g > 0.7 High
0.3 < g < 0.7 Moderate
g < 0.3 Low

Based on the table of N-gain interpretation classification above, the N-gain score of students were categorized as
moderate [10]. It was found that instructional materials facilitate learning of abstract concepts by helping students to
concretize ideas and also stimulate their imagination. Many educators agree that instructional materials bring about
improvement in the teaching/learning process as well as permit teachers and students to interact as human beings in
a climate where people control their environment for their own best purposes [14]. There is strong evidence that the
choice of instructional materials has large effects on student learning that rival in size those that are associated with
differences in teacher effectiveness [6].

CONCLUSION
In summary, STEM based instructional material that were developed and implemented in junior secondary
school classroom setting can be used as educational materials necessary for conducting effective STEM education.
The instructional material was developed based on current National curriculum and contains STEM concepts in
lever topic and equipped engineering design activity. The result shows that readability aspect of STEM based
instructional material was categorized as very high. Students’ response towards STEM based instructional material
also shows that layout, content and utility of the instructional material achieved very high percentage, pretest and
posttest responses revealed that students retained significant amounts information upon completion of the STEM
instructional material. Students’ overall learning gain was 0.67 which was categorized as moderate through a

060001-6
measured by the pretest and posttest results. In essence, the instructional material is valid enough to be used for
conducting STEM education in classroom setting. It is recommended for academic/research community for use, and
or for further refinement.

ACKNOWLEDGMENT
I would like to thank students and teachers from SMPN 1 Rancaekek who were involved in the validation survey
for this research project.

REFERENCES
1. Badan Pusat Statistik, “Analisis Kemiskinan, Ketenagakerjaan dan Distribusi Pendapatan” (Badan Pusat
Statistik, Jakarta, 2010).
2. B. S. Barret, A.L. Moran, and J.E.Woods, International.Journal.of.STEM.Education, 1(6), pp 5-6, (2014).
3. C.A. Brown, and K. McIlroy, Assessment.and.Evaluation.in.Higher.Education, 36(6), pp 687-699, (2011).
4. J. Brown, “Journal.of.STEM.Education”, 13(5), pp 7-11, (2012).
5. R. B. Bybee, “The case for STEM education: Challenges and Opportunities. National Science Teachers
Association” (NSTA Press, Virginia, 2013).
6. M. Chingos and G. Whitehurst, “Choosing Blindly:Iinstructional Materials, Teacher Effectiveness, and The
Common Core”, (Brookings Institution Washington, 2012).
7. C. Denton, D. Bryan, J.Wexler, D. Reed, S. Vaughn, “Effective instruction for middle school students with
reading difficulties: The reading teacher’s sourcebook” (University of Texas Systems, Texas Education
Agency, 2007).
8. L. D. English and D.T. King, International.Journal.of.STEM.Education, 2(14), pp18-23, (2015).
9. N. Fang, “Work in Progress: Brainstorming with Yo-Yos in High School Outreach: Inspiring Students’ Interest
in Learning Physics” in ASEE Annual Conference, Paper presented at 2013 ASEE Annual Conference, Atlanta
(2013)
10. R. R. Hake, Analyzing Change/Gain Scores. [online]Retrieved January 13, (2016).
11. S. Kesidou and J.E. Roseman, Journal.of.Research.in.Science.Teaching, 39(6), pp 522–549, (2002).
12. W. E. Nagy and E. H Hiebert, “Toward a theory of word selection” (Longman, New York, 2011).
13. National Research Council, “Engineering In K-12 Education: Understanding The Status And Improving The
Prospects” (National Academies Press:Washington DC, 2009).
14. M.C. Pocovi, Journal.Research.of.Science.Teaching, 44(1), pp 107-132 (2007)
15. I. R. Suwarma, “Research on Theory and Practice STEM Education Implementation in Japan and Indonesia
using Multiple Intelligences Approach”, Doctoral Thesis, Shizuoka University (2015).
16. C. D. Wolfenbarger and L. R. Sipe, Language Arts, 84(3), pp 273-280. (2007).

060001-7