sustainability

Article
Development of STEM-Based AI Education Program for
Sustainable Improvement of Elementary Learners
Junhyeok Jang † , Jaecheon Jeon † and Soon Ki Jung *

School of Computer Science and Engineering, Kyungpook National University, Daegu 41566, Republic of Korea
* Correspondence: skjung@knu.ac.kr
† These authors contributed equally to this work.

Abstract: Recent years have seen tremendous technology and research development in the field of
artificial intelligence (AI). Initiatives are being made to build or employ AI in various domains of
society. The field of education is experiencing the same trend. Being able to solve various issues by
utilizing AI and varied learning materials is a crucial fundamental skill for students of all ages. The
purpose of this study is to (1) develop STEM-based artificial intelligence education for K–6 grade
students and (2) measure the effectiveness of the developed program in terms of creative problem-
solving ability, AI literacy, and attitude toward AI. Experimental studies show an improvement in the
average score and that all three elements are statistically significant. Additionally, it is established that
the view of the study’s program and AI is good as a result of the emotional analysis of the comments
freely expressed by the students. This study reflects on the need for establishing effective and various
AI educational programs.

Keywords: elementary school; STEM; artificial intelligence education

Citation: Jang, J.; Jeon, J.; Jung, S.K.

Development of STEM-Based AI
1. Introduction
Education Program for Sustainable
Improvement of Elementary In the field of computer science, artificial intelligence (AI) has experienced several
Learners. Sustainability 2022, 14, golden-age periods after the concept was first agreed at the Dartmouth Conference in 1956,
15178. https://doi.org/10.3390/ but faced a dark age due to the limitations in research because of the underlying hardware
su142215178 technology and difficulties in commercialization [1]. However, since mid-2010, we have
been facing a new revival due to the development of neural networks and parallelization
Academic Editor: Michail
Kalogiannakis
processing technology, and a significant amount of research based on this is being conducted
in academia as understanding and using AI is a very important factor in future human life
Received: 30 September 2022 and national prosperity [2,3]. It is causing many changes and innovations in all areas, such
Accepted: 12 November 2022 as politics, economy, culture, and education in modern society [4]. In line with this trend,
Published: 16 November 2022 countries around the world consider AI as their major future project for their country and
Publisher’s Note: MDPI stays neutral are sparing investment and support, and this is also the case in education [5]. Education
with regard to jurisdictional claims in for adolescent students is a process to prepare for 10–20 years into the future, when these
published maps and institutional affil- students will participate in economic activities. In other words, it is necessary to consider
iations. not only the immediate changes in the present, but also in the future, 10 to 50 years after
the learner becomes an adult and a member of society [6]. Students’ current learning
experiences can have a profound impact on their lives and help guide their overall life.
Therefore, there is no shortage of artificial intelligence education (AIED), even if it plays
Copyright: © 2022 by the authors. a pivotal role in forming the future of human society and contributing to the national
Licensee MDPI, Basel, Switzerland. economic development. Therefore, providing an appropriate approach to AIED and an
This article is an open access article
effective learning experience is an important task for education officials.
distributed under the terms and
What is the goal of education? It is to achieve the learning goals according to the
conditions of the Creative Commons
specific subject and learning elements, to gather these achievements to logically solve the
Attribution (CC BY) license (https://
problems encountered in one’s life, to feel the beauty of life, and to nurture a holistic human
creativecommons.org/licenses/by/
being who enjoys life [7]. Therefore, educators and administrators must provide specific
4.0/).

Sustainability 2022, 14, 15178. https://doi.org/10.3390/su142215178 https://www.mdpi.com/journal/sustainability

Sustainability 2022, 14, 15178 2 of 16

and meaningful education so that the knowledge, skills, and attitudes that students have
experienced and achieved in school education can be used meaningfully in the lives of
learners beyond simply experiencing them. Various teaching and learning methodologies
have been proposed to ensure the significance of education and help students to be truly
useful in their lives, and STEM education is also one of the effective methods. STEM
education provides an experience that allows students to acquire knowledge and skills in
science, technology, engineering, and mathematics in the course of solving real-life problem
situations and use them meaningfully in their lives [8]. Numerous studies have confirmed
that STEM-based learning has a positive effect on learners’ knowledge utilization, proving
the importance and effectiveness of STEM education in the modern curriculum [9].
As a content element of learning, AI education is essential from the perspective of
sustainable education to prepare for the future, and in terms of learning methods, STEM
education is an effective way to help learners use the knowledge acquired in education
meaningfully in their lives. This study aims to develop a STEM-based AIED plan by linking
the importance of the content of AIED with the method efficiency of STEM education and
apply it to analyze the educational effect. Despite the effectiveness and importance of
STEM-based AIED, very few related studies have been conducted, and research results on
elementary school students targeted in this study are harder to find.
The research objectives that guide this study are as follows:
1. Develop a STEM-based AIED program using backward design for K-12 students.
2. Measure the effectiveness of the developed program on students’ creative problem-
solving ability, literacy of AI, and attitude toward AI.

2. Theoretical Background
2.1. The Reseach Trend of STEM and AI Education
VOS viewer, a research bibliographic visualization program, was used to identify research
trends in STEM and AIED. The keywords “artificial intelligence education,” “STEM,” and
“STEAM” were searched in the WoS database by limiting the publication year from 2019
to 2022 and the publication type to article, and using this, the main keywords of related
papers and their relationships were visualized Figure 1. Irrelevant terms were removed
manually from the searched database, and the minimum frequency of occurrence was set to
5 or more to structurally indicate the trend in the research field. Research from the database,
the VOSviewer generated 10 clusters with 185-word items, 1601 links, and 3065 total link
strength. Of the 185-word items, the following words have high values of the total link
strength: “artificial intelligence” (total link strength 1101), “education” (total link strength
420), “machine learning” (total link strength 314), “deep learning” (total link strength 145),
“higher education” (total link strength 120), “big data” (total link strength 110), “STEM”
(total link strength 90), “technology” (total link strength 90), “STEAM” (total link strength
81), “internet of things” (total link strength 75), and “e-learning” (total link strength 62) as
can be seen in Table 1.

Table 1. Co-occurrence word items from 2019–2022 publications on AI and STEM research.

Keyword Occurrences Total Link Strength

Artificial intelligence 617 1011
Education 172 420
Machine learning 150 314
Deep learning 71 145
Higher education 55 120
Big data 42 110
STEM 54 90
Technology 37 90
STEAM 50 81
Internet of things 32 75
e-learning 32 62
 STEM 54 90
Technology 37 90
STEAM 50 81
Sustainability 2022, 14, 15178 Internet of things 32 75 3 of 16
e-learning 32 62

Figure 1. The analysis of articles about AI education and STEM from 2019 to 2022.
Figure 1. The analysis of articles about AI education and STEM from 2019 to 2022.

2.2. AI Education
2.2. AI Education
Since the mid-2000s, AI once again entered a period of revival after 2010, and the
Since the mid-2000s, AI once again entered a period of revival after 2010, and the
demand for AIED has also begun to emerge in society. Although the academic perspective
demand for AIED has also begun to emerge in society. Although the academic perspective
on AIED is being studied in various ways [10], this study intends to systematize AIED using
on AIED is being studied in various ways [10], this study intends to systematize AIED
the South Korean classification system, which is as follows: AI-based education, AI-using
using the South Korean classification system, which is as follows: AI-based education, AI-
education, and AI-understanding education [11]. First, AI-based education refers to the
using education, and AI-understanding education [11]. First, AI-based education refers to
use of AI as a learning management tool, analyzes learner achievement to find strengths
the use of AI as a learning management tool, analyzes learner achievement to find
and weaknesses, and provides appropriate feedback through this. In other words, from the
strengths and weaknesses, and provides appropriate feedback through this. In other
perspective of realizing individualized education, AI is used as a starting point analysis tool
words, from the perspective of realizing individualized education, AI is used as a starting
for education. Second, AI-using education refers to the perspective of using AI services as
point
a toolanalysis tool
to achieve thefor education.
objectives Second,
of the class AI-using education
in the teaching refers toscene.
and learning the perspective of
This is a way
using AI services
for students as more
to pay a toolattention
to achieve tothe
howobjectives
they use of
AIthe class in
to solve theproblems.
their teaching and
Forlearning
example,
scene. This is a way for students to pay more attention to how
when students whose native language is not English learn English, AI will bethey use AI to solve
usedtheir
as a
problems. For when
learning tool example, theywhen students
practice whose native
the accuracy language
of English is not English
pronunciation learn
using Eng-
artificial
lish, AI will be
intelligence used as Third,
speakers. a learning tool when theyeducation
AI understanding practice the accuracy
refers of English
to education that pronun-
develops
ciation using
students’ artificial intelligence
problem-solving ability speakers. Third, AIthe
by understanding understanding
concept of AI, education refers to
implementing its
principles in computer programs, or including AI in other programs. By understanding the
basic concepts of AI and the principles of modern AI led by machine learning and deep
learning, students can improve their problem-solving ability in the process of creating their
own models through public open neural networks or developing programs using them.
Major foreign countries are also paying attention to education by increasing their
investment in AI, which is a major foundational element of future industries. The United
Sustainability 2022, 14, 15178 4 of 16

Kingdom, through the AI Roadbook announced by the AI Council, speaks of the need
to develop a national AI strategy and will work to improve AI and data literacy for all,
which will enable people to understand the risks and benefits of AI with confidence and
information. It was argued that the user should be able to obtain this understanding [12].
In the United Kingdom, there is no direct mention of AI, but in the computer science
area, the basic learning of algorithms and programming provides the foundation for the
understanding and knowledge of AI [13]. The European Union announced a plan to
distribute free online lectures on Elements of AI, an artificial intelligence learning program
developed by the University of Helsinki, to improve the basic AI competency of the EU
citizens. The program consists of six themes, including “What is AI, AI problem solving,
real-world AI, machine learning, neural networks, and social impact,” and is available in
29 official EU languages [14]. In the United States, the Association for the Advancement
of Artificial Intelligence (AAAI) and Computer Science Teachers Association, and CSTA
announced AI4K12 (AI for K-12 students), an AI education guideline for elementary
school students [15]. In AI4K12, the five big ideas for artificial intelligence are perception,
expression and reasoning, learning, interaction, and social impact [16]. South Korea has
established a basic plan for AI education context led by government-related institutions
and advocates linking with computer science and software education. The three areas,
including the understanding of artificial intelligence, the principles and application of
artificial intelligence, and the social impact of artificial intelligence, each include sub-
areas [17]. Table 2 shows the main elements of AI education in the curriculum of the
European Union, the United States, and South Korea.

Table 2. Overview of AIED Curriculum in Each Country [14,15,17].

Country European Union United States South Korea

What is AI Perception
Understanding of AI
AI problem solving Representation and reasoning
Real world AI
Elements of AIED Principles and application of AI
Machine learning Learning
Neural networks Natural interaction
Social impact of AI
Implications Societal impact

AIED, which is intended to be developed in this study, is an education for elementary

school students about AI itself, understanding the principles of AI at a rudimentary level,
and using it to self-directly experience the process of solving real problems. It focuses on
improving overall understanding about AI and improving problem-solving ability.

2.3. STEM Education

STEM education is a convergence educational method that allows students to learn the
learning elements of science, technology, engineering, and mathematics in the process of
solving real-life problem situations, thereby cultivating students’ practical problem-solving
capabilities and composing meaningful knowledge that connects knowledge and life [8].
Discussion on the educational effect and methodology of STEM education has been carried
out in numerous studies, and it is a method of innovative and meaningful teaching and
learning activities for educational members—administrators, teachers, and students—in
the school education field.
Research on STEM education is being conducted in various ways to synthesize its
effects from the perspective of students. It has previous studies that STEM education
improves students’ cognitive, emotional, and psychodynamic areas and has a positive
effect on students’ career interest and maturity [18–21]. STEM education also contributes to
linking ethnic and gender differences, and students can easily solve big challenges that have
yet to be solved by developing 21st century skills such as adaptability, problem-solving
skills, complex communication, and system thinking skills [22–24].
Sustainability 2022, 14, 15178 5 of 16

In particular, Batdi et al. (2019) emphasized that as a result of qualitatively synthe-

sizing related papers, STEM classes induce active participation of students and provide
meaningful learning to students, contributing to cognitive improvement. As a result of
analyzing the impact of STEM education by school level, it was found that the cognitive im-
provement in elementary schools was higher than that of other schools, and the educational
effect in elementary schools was much higher than that of middle and high schools [25].
Research on the improvement of the affective domain through STEM education has
been verified and mentioned internationally through various countries. Kanadli (2019)
synthesizes changes in students’ affective domains based on 22 studies in Turkey and
emphasizes that STEM classes are a good way to improve students’ learning aspirations
and motivation, especially attention and interest [26]. In addition, according to a study
by Kang (2019), the impact of integrated STEAM education in Korea is effective in both
the cognitive and definitive learning of students, but it is especially more effective in the
affective domain [27]. Finally, Murphy et al. (2019) analyzed Australia’s STEM-related
documents and emphasized that STEM education should have an educational strategy in
the direction of improving curiosity and confidence [28].
Even in the psychodynamic area, STEM education consists of active participation
and activities of students, so it can be expected to improve students’ psychodynamic
functions [25]. This can develop students’ ability to plan and practice through project
classes or problem-solving classes, and can also develop engineering skills. These activities
can also improve research skills, communication skills, cooperation, and problem-solving
skills [25,28].
Looking at the empirical research case, it was found that the STEM-based engineering
education targeting non-engineering students in a study targeting university students
increased the students’ AI literacy and AI ethics [29]. In addition, STEM education in
science subjects was effective in enhancing students’ problem-solving ability and critical
thinking [30,31], and had a positive effect on spatial sense and geographic knowledge
development [32].
STEM education positively affects not only the intellectual development of the subjects—
science, technology, engineering, mathematics—constituting it, but also the student’s learning
attitude toward each subject and the learning process. Rather than being fragmented
and isolated knowledge, each learning content is internalized in the learner’s cognitive
process and expanded and applied in various situations. In other words, what is learned at
school goes beyond memorizing and becomes living knowledge that can be used in the
lives of students, which has a developmental effect on students’ learning attitudes, which
becomes an especially important learning experience in their lives. In some countries,
STEM education has also been expanded under the name STEAM education, which has
added art that means literary, artistic, and moral elements.
Based on the above discussion, STEM education and AI are complementary; hence,
students can use STEM education to develop AI literacy, and AI must play a role to learn
STEM knowledge in an interesting way. It should be interactive and connect students’ real
life with what they learn [33].

3. Methods
3.1. Backward Design
Tyler’s curriculum design method, which has been used as a method for setting up
an education plan for a long time, consists of goal defining, learning experience selection,
learning experience organization, and evaluation, which is called forward design [34]. In
the past century, this forward instructional design has been practiced by many schools and
teachers and has produced excellent educational cases and research results, but the learning
goals and the contents of the evaluation plan are not connected to each other, and there is a
gap or the results of evaluation are not satisfactory. It has also caused problems that could
not be utilized for intellectual growth. In the teaching and learning scene, the purpose of
evaluation is not to classify and rank who is good at what, but to help each learner achieve
 an education plan for a long time, consists of goal defining, learning experience selection,
learning experience organization, and evaluation, which is called forward design [34]. In
the past century, this forward instructional design has been practiced by many schools
and teachers and has produced excellent educational cases and research results, but the
learning goals and the contents of the evaluation plan are not connected to each other, and
Sustainability 2022, 14, 15178 there is a gap or the results of evaluation are not satisfactory. It has also caused problems 6 of 16
that could not be utilized for intellectual growth. In the teaching and learning scene, the
purpose of evaluation is not to classify and rank who is good at what, but to help each
learner
their achieve
learning goalstheir
by learning
examining goals by examining
student achievement student achievement and
and understanding understand-
the level of each
ing the level
student. If theofviewpoint
each student. If the viewpoint
of evaluation focusesofonevaluation
the growth focuses on the in
of students growth of stu-
the process
dents
of in therather
learning process of learning
than the learning rather than thethe
outcome, learning
organic outcome, the organic
relationship between relationship
learning
content
between and evaluation
learning contentbecomes important.becomes important.
and evaluation
The
Thebackward
backwardmethod methodof ofinstructional
instructionaldesign
designcan
canfocus
focusmore
moreon onthe
theeducational
educationaluse use
of evaluation. The backward instructional design [35] direction proceeds
of evaluation. The backward instructional design [35] direction proceeds by identifying by identifying
desired
desiredresults,
results,determining
determiningacceptable
acceptableevidence,
evidence,and
andplanning
planninglearning
learningexperiences
experiencesand and
instruction.
instruction. ItIt is
is different
different from
from forward
forwardteaching
teaching and
and learning
learning design
design inin the
the order
orderof ofthe
the
evaluation
evaluation plan,
plan, andand ititisisalso
alsocalled
calledreverse
reverseteaching
teachingandandlearning
learningdesign.
design. This
Thishas
hasthethe
advantage
advantageof offocusing
focusingthe thecontent
contentof ofthe
theevaluation
evaluationthoroughly
thoroughlyonly onlyononthe
thelearning
learninggoal
goal
and
andpreventing
preventingthe theprovision
provision of of
a learning
a learningexperience
experiencethatthat
is not related
is not to the
related toachievement
the achieve-
of
ment of the goal [36]. As shown in Figure 2, the STEM-based AIED programdeveloped
the goal [36]. As shown in Figure 2, the STEM-based AIED program developedin in
this
thisstudy
studywaswasalsoalsodeveloped
developedthroughthroughbackward
backwarddesign
designto tofocus
focusononthe
theachievement
achievementof of
students’
students’achievement
achievementof ofAIED
AIEDlearning
learninggoals.
goals.

Figure2.2.STEM-based
Figure STEM-basedAIED
AIEDdevelopment
developmentstructure
structurediagram.
diagram.

3.2. Research Participants

To investigate the effect of the STEM-based AI education program developed in this
study on learners’ creative problem-solving ability, literacy of AI, and attitude toward
AI, 120 students in five classes in grades K–6 of elementary school were targeted. It was
applied for one semester. Curriculum during this period consisted of the application of
development materials as well as classes of general other subjects individually. All the
participating students (62 males, 58 females) were subjected to verification and statistical
analysis of the application results. Instead of conducting classes in a separate computer
room, the smart devices provided in the classroom were actively used, and hence, an
environment for using one device per person was prepared. Teachers with more than
10 years of experience held a pre-training session and a monthly meeting to discuss the
Sustainability 2022, 14, 15178 7 of 16

progress of the class and effective learning process to improve their competency to operate
the developed program.
Prior to the experiment, participating students responded to two types of preliminary
questionnaires and the results are shown in Table 3. The first survey was conducted to de-
termine the use of information devices during class activities as a question asking students’
ICT using ability, and the average result was 3.89 (5-point Likert scale). The second survey
is a question that asks students about their experiences with various teaching and learning
methods, such as cooperative learning, convergence class, and inquiry learning, to choose
effective learning ways during the STEM class. They answered that they had experience
in various teaching and learning methods, and that more learning occurs in the process
of exploring and discussing with peer learners rather than transferring knowledge from
the teacher. In addition, the students subject to the experiment had sufficient experience in
computer programming education using Scratch and using ICT devices and were familiar
with teaching methods, such as cooperative learning and convergence education, as well as
traditional classes. The actual class was conducted as a method of collaborative learning
with peer learners and inquiry learning that guides them to solve problems on their own.

Table 3. Information on participants.

Class A Class B Class C Class D

N 30 30 30 30
ICT knowledge and experiences 3.92/5 3.88/5 4.01/5 3.78/5
Experience in various teaching and
4.12/5 4.33/5 4.08/5 4.02/5
learning methods

Students participating in this study were informed that they would participate in the
pre-post-developmental test in the form of a questionnaire, and that the questionnaire had
complete anonymity and non-identification. In addition, it was announced that the results
of the test are not analyzed for each individual, but values such as the average score and
variance are calculated for each group. As such, the contents of the test were thoroughly
guaranteed anonymity and there was no room for infringement of individual rights, but
the experiment was conducted on non-adult students, so written consent was obtained
from the students’ parents or guardians regarding the experiment and participation guide.

3.3. Data Collection and Analysis

In this study, students were asked to respond to two types of questionnaires. The
first was a questionnaire for quantitative analysis. It consists of a creative problem-solving
ability test tool (2004, MI Research Team, Psychology Lab, Seoul National University)
and two types of AI-related questionnaires (AI literacy, AI attitude). The reliability of
Cronbach’s alpha for the creative problem-solving ability test tool is 0.93 and consists
of four sub-domains: self-confidence and independence of learners, divergent thinking,
critical logical thinking, and motivational thinking. Creative problem-solving ability is an
important competency in a learner’s life, and the sub-elements of this test tool are good
indicators to check how they affect the development of problem-solving ability, which is
very important to students. Therefore, it can be confirmed whether the effect of STEM-based
AIED is a stepping stone for sustainable growth and meaningful learning in a student’s
lifetime. The AI-related questionnaire used the test tool of Chang-mo Yang (2022), and this
test has a total of 17 questions, with a total of 5 questions on AI literacy, consisting of a
1-point to 4-point Likert scale and its Cronbach’s alpha is 0.72. Scores of 1 were “I do not
know at all”, 2 points were “I have a memory but I don’t know”, 3 points were “I know but
I can’t explain it”, and 4 points was “I know well and can explain it well”. The AI literacy
area asks how well students understand the meaning of AI and how it affects our lives,
and how students can use and apply AI in the process of solving their own problems. The
remaining 12 items are about AI-related attitudes and consisted of a Likert scale ranging
from 1 to 5, with 1 being “not at all”, 2 being “not at all”, 3 being “normal”. A score of
Sustainability 2022, 14, 15178 8 of 16

4 was measured as “agree” and a score of 5 as “strongly agree”. Attitude towards AI is

an ethical area of AI, which is about the positive and negative effects of using AI, and the
tendency to use AI in life. Details of the questionnaire for quantitative research can be
found in Appendix A. The results of these surveys are statistically processed in SPSS to
prove their significance.
The second type of survey is a satisfaction survey for qualitative surveys. We received
participants’ opinions about the STEM-based AIED program in a free form and qualitatively
analyzed participants’ tendencies, interest, and immersion in the development program
using ORAGNE3’s text mining technique. All questionnaires were presented in Korean and
responses were received in Korean in consideration of the age of the test subjects, and for
qualitative analysis, they were translated into English that can contain the same meaning
as much as possible.

4. Results
4.1. Development of STEM Program for AI Education
In this study, we developed a STEM-based Aid program for an elementary K–6 school.
The program is organized around three topics. The developed educational program was
written based on the backward design for effective education and practical development
of students. This allows more goal-oriented activities for students, and teachers can focus
on giving students appropriate feedback. The specific details of the contents are shown in
Figure 3.
In class, students focused on what problems they could find in each situation. Some
students had difficulties in finding problems, but they were observed to identify a problem-
solving plan and participate in the solution process with the help of the teachers or dis-
cussions with fellow students. Students experienced immersion in the learning process
by not only accepting the teacher’s explanation, but also leading their own learning, and
actively used various media to acquire new information and knowledge. This will be
the process in which the knowledge learned at school is integrated into the learner and
becomes meaningful in the student’s life. Figure 4 shows some scenes from the class of the
STEM-based AI program.
Teaching K–6 students about the operation principle and computational process
of artificial intelligence models, such as classification, regression, and natural language
recognition, is not appropriate for the developmental stage. For example, even if students
searched for the operating principle of the CNN network through internet searches, they
did not learn in detail, and the teacher guided the students with easy content that AI finds
feature maps of images through simple but numerous calculations.

4.2. Change in Creative Problem-Solving Ability, Literacy of AI, and Attitude toward AI
Students were asked to respond to a total of three types of questionnaires related
to creative problem-solving ability, AI literacy, and AI attitude. There were significant
differences between the pre-result and the post-result in all sub-domains of the question-
naire on creative problem-solving ability. In all areas of self-confidence and independence,
divergent thinking, critical and logical thinking, and motivation, the average score of the
post-results improved compared to the pre-results, and it was confirmed that they were
statistically significant. This means that in teaching AI to elementary school students
(K–6), the STEM-based educational method is one of the effective methods to help students
develop creative problem-solving ability using AI. In learning about AI, it goes beyond
simple experience or imitative education, that is, by presenting real-life situations, students
discover problems on their own, explore content elements of various disciplines, and solve
problems according to their own plans. It can be analyzed that the STEM education process
is meaningful in the development of learners’ creative thinking skills. The specific survey
results for the experiment are shown in Table 4.
Sustainability 14, x15178
2022, 14,
Sustainability 2022, FOR PEER REVIEW 9 9ofof1716

Figure 3. Cont.
Sustainability2022,
Sustainability 2022, 14,
14, 15178
x FOR PEER REVIEW 10
10ofof17
16

Figure 3. Details of developed STEM-based AIED program.

In class, students focused on what problems they could find in each situation. Some
students had difficulties in finding problems, but they were observed to identify a prob-
lem-solving plan and participate in the solution process with the help of the teachers or
discussions with fellow students. Students experienced immersion in the learning process
by not only accepting the teacher’s explanation, but also leading their own learning, and
actively used various media to acquire new information and knowledge. This will be the
process in which the knowledge learned at school is integrated into the learner and be-
comes meaningful in the student’s life. Figure 4 shows some scenes from the class of the
STEM-based
Figure AIofprogram.
3. Details developed STEM-based
developed STEM-based AIED
AIED program.
program.

In class, students focused on what problems they could find in each situation. Some
students had difficulties in finding problems, but they were observed to identify a prob-
lem-solving plan and participate in the solution process with the help of the teachers or
discussions with fellow students. Students experienced immersion in the learning process
by not only accepting the teacher’s explanation, but also leading their own learning, and
actively used various media to acquire new information and knowledge. This will be the
process in which the knowledge learned at school is integrated into the learner and be-
comes meaningful in the student’s life. Figure 4 shows some scenes from the class of the
STEM-based AI program.
Figure4.
Figure 4. STEM-based
STEM-based AI
AI program
program class
class scene.
scene.

4. Results of
Table Teaching K–6pre/post-survey on creative
students about problem-solving
the operation principleability.
and computational process of
artificial intelligence models, such as classification, regression, and natural language
Sub-Domains Cronbach’s α Mean S.D. t
recognition, is not appropriate for the developmental stage. For example, even if students
(pre) Self-confidence
searched and independence
for the operating 0.638
principle of the CNN network 2.81 internet
through 0.74searches, they
−7.153 **
(post) Self-confidence and independence 0.786 3.43 0.90
(pre) Divergent thinking 0.788 3.02 0.82
−7.354 **
(post) Divergent thinking 0.895 3.68 0.94
(pre) Critical and logical thinking 0.811 3.26 0.81
−7.376 **
(post) Critical and logical thinking 0.879 3.91 0.86
(pre) Motivation
Figure 4. STEM-based AI program class scene. 0.781 3.57 0.79
−6.062 **
(post) Motivation 0.827 4.09 0.78
** p < 0.01.
Teaching K–6 students about the operation principle and computational process of
artificial intelligence
In addition, models,toward
in attitude such asAI,classification,
as shown in regression,
Table 5, theand natural
score of thelanguage
post-test
recognition, is not appropriate for the developmental stage. For example,
improved compared to the pre-test. The mean score improved in all sub-domains, even if students
and
searched
the for the operating
significance principle
probability of less
was also the CNN network
than 0.05, through that
confirming internet searches,
it was they
statistically
Sustainability 2022, 14, 15178 11 of 16

significant. Great progress has been made in the field of social impact of AI characteristics
of AI, which analyzed that the students recognized that AI is better than humans and it can
be applied to many parts of society by using it.

Table 5. Results of pre/post-survey on attitude toward AI.

Sub-Domains Cronbach’s α Mean S.D. t

(pre) Social impact of AI 0.786 3.27 1.02
−8.388 **
(post) Social impact of AI 0.907 4.15 0.97
(pre) AI and communication 0.742 2.84 0.88
−12.123 **
(post) AI and communication 0.860 3.94 0.92
(pre) Interaction with AI 0.635 3.19 0.97
−9.081 **
(post) Interaction with AI 0.807 4.08 0.95
(pre) Emotional exchange with AI 0.632 3.08 0.74
−6.936 **
(post) Emotional exchange with AI 0.613 3.60 0.79
(pre) Characteristics of AI 0.799 2.84 1.06
−11.027 **
(post) Characteristics of AI 0.795 4.06 0.95
** p < 0.01.

Lastly, the AI literacy domain, as shown in Table 6, it was also confirmed that the
results of the post-test improved compared to the pre-test and that it was statistically
significant. The questionnaire on AI literacy is about “Can you explain to others?” the
concepts, functions, principles, ethics, and perceptions of AI. Students made progress in all
areas, which means that students’ level of understanding of AI has been further improved
through STEM-based AIED.

Table 6. Results of pre/post-survey on AI literacy.

Sub-Domains Mean S.D. t

(pre) Concept 2.67 0.87
−10.789 **
(post) Concept 3.58 0.62
(pre) Function 2.62 0.91
−10.629 **
(post) Function 3.54 0.68
(pre) Principle 2.32 0.93
−11.511 **
(post) Principle 3.37 0.94
(pre) Ethics 2.46 0.91
−10.743 **
(post) Ethics 3.44 0.75
(pre) Recognition 2.59 0.84
−11.973 **
(post) Recognition 3.54 0.64
** p < 0.01.

4.3. Sentiment Analysis of Learning Exprience

Students participating in the experiment wrote free-form opinions for qualitative
analysis. They were allowed to freely write their feelings about the educational program
and their opinions on AI, and the responses of students were analyzed using various
sentiment analysis methods, such as Liu-hu, Vader, Multilingual sentiment, and SentiArt,
and the results are shown in Table 7.
First, Liu-hu is a lexicon-based sentiment analysis tool that supports English and Slove-
nian [37] and returns the standardized score difference for positive expressions (maximum
value 100) and negative expressions (maximum value −100). Students’ opinions showed
that the average Liu-hu score was 31.10, which means that the STEM-based AIED program
developed and applied in this study provided students with a positive learning experience.
Second, in the analysis through VADER, a lexicon and rule-based sentiment analysis
tool [38], the positive score (M = 0.33, SD = 0.23) greatly outweighed the negative score
(M = 0.01, SD = 0.05), and between −1 and 1, the average of the compound score with the
range was also 0.43, indicating that the STEM-based AIED program had a positive effect on
the students. Figure 5 is a heat map of the sentiment analysis result through VADER. The
Sustainability 2022, 14, 15178 12 of 16

blue area representing negative emotions is very small, while the green and yellow areas
representing neutral and positive emotions are very wide.

Table 7. Sentiment analysis results.

Method Classification Mean SD Min Max

Liu-Hu Single 31.10 36.11 −100.00 100.00
Sustainability 2022, 14, x FOR PEER REVIEW Positive 0.33 0.23 0.00 1.0013 of 17
Negative 0.01 0.05 0.00 0.50
VADER
Neutral 0.66 0.23 0.00 1.00
Compound 0.43 0.31 −0.06 0.99
Second, in the analysis
Multilingual Single through VADER, a lexicon
22.93 31.21 and rule-based
−33.33 sentiment analysis
100.00
Sentiment
tool [38], the positive 0.78
score (M = 0.33, SD = 0.23) greatly −0.54
0.62 outweighed the negative3.31
score (M
= 0.01, SD = 0.05), andAnger
between −1 and−0.09
1, the average0.59 −1.53score with the
of the compound 1.58range
Fear 0.59 0.70 − 1.62
was also 0.43, indicating that the STEM-based AIED program had a positive effect 2.27
on the
SentiArt Disgust 0.00 0.63 −1.32 1.39
students. Figure 5Happiness
is a heat map of the sentiment analysis
1.32 0.81 result through
−0.46 VADER.3.42 The blue
area representing negative
Sadness emotions is very small,0.50
0.32 while the green and yellow areas
−1.16 3.03 rep-
resenting neutral and positive emotions
Surprise 0.58 are very wide.
0.84 −1.18 2.52

Figure 5. Heat map of sentiment analysis result through VADER.

VADER.

Third, an average score of 22.93 was also output in in multilingual

multilingual sentiment,
sentiment, aa lexicon-
lexicon-
based sentiment analysis tool [39]. Finally,
Finally, in SentiArt, based
based onon the
the vector
vector space
space models’
models’
returning text valence
returning text valence[40],
[40],the
thehappiness
happinessscore
score(M
(M= = 1.32,
1.32, SDSD = 0.81)
= 0.81) andand surprise
surprise score
score (M
(M = 0.58, SD = 0.84) were higher than the anger score (M = − 0.09, SD =
= 0.58, SD = 0.84) were higher than the anger score (M = −0.09, SD = 0.59), meaning nega-0.59), meaning
negative, andoverall
tive, and the the overall
scorescore was 0.78,
was also also 0.78, indicating
indicating a positive
a positive result.result. Through
Through these these
emo-
emotional analysis
tional analysis results,
results, the STEM-based
the STEM-based AIED AIED program
program was was recognized
recognized as a as a positive
positive and
and enjoyable
enjoyable learning
learning experience
experience for students,
for students, whichwhich
can becanexpected
be expected to have
to have a lasting
a lasting and
and positive
positive impactimpact on students’
on students’ lifelong
lifelong education.
education.
5. Discussion and Conclusions
5. Discussion and Conclusions
With the innovative increase in computing power, large-scale parallel processing can
With the innovative increase in computing power, large-scale parallel processing can
be efficiently processed; therefore, artificial intelligence technologies, such as classification,
be efficiently processed; therefore, artificial intelligence technologies, such as classifica-
regression, and natural language recognition, are being developed explosively. In the future,
tion, regression, and natural language recognition, are being developed explosively. In
AI computational ability will be used more actively in various fields of human society, and
the future, AI computational ability will be used more actively in various fields of human
creative problem-solving ability using AI is an essential competency for students who will
society, and creative problem-solving ability using AI is an essential competency for stu-
live in future society.
dents who will live in future society.
STEM education is an effective educational methodology that helps students not only
knowSTEM
whateducation
they haveislearned,
an effective educational
but also methodology
applies them to real life.thatThere
helpsare
students
variousnotviews
only
know what they have learned, but also applies them to real life. There
on AI education, but in this study, it was viewed from the perspective of providing anare various views
on AI education,
experience but insolving
of problem this study,
using it was viewed from
AI. Therefore, the perspective
in this of providing
study, a STEM-based AIEDan
experienceprogram
education of problem solving using
for elementary AI. students
school Therefore, wasindeveloped
this study,and a STEM-based AIED
applied to present
education program for elementary school students was developed and applied
problem situations that students can encounter in real life and to solve problems by linking to present
problemacademic
various situationscontents,
that students can encounter
including in realoflife
AI. The theme theand to solveprogram
developed problemstookby link-
into
consideration the interests of the students and their social issues. The selected topicstook
ing various academic contents, including AI. The theme of the developed program are
into consideration
about the interests
creating a program for theofsocially
the students and their social
disadvantaged, issues. The
an infectious selected
disease topics
prevention
are about
system creating
based a program
on mask for the
recognition, socially
and disadvantaged,
conservation an infectious
of the ecosystem, and disease preven-
the details were
tion system based on
explained in Section 4. mask recognition, and conservation of the ecosystem, and the details
were explained in Section 4.
This study aimed to investigate the change in creative problem-solving ability, atti-
tude toward AI, and AI literacy. In addition, by receiving students’ free opinions, senti-
ment analysis was conducted on the student’s thoughts on the development program and
AI. The experiment was conducted for K–6 students, and the results of this study are as
Sustainability 2022, 14, 15178 13 of 16

This study aimed to investigate the change in creative problem-solving ability, attitude
toward AI, and AI literacy. In addition, by receiving students’ free opinions, sentiment
analysis was conducted on the student’s thoughts on the development program and AI.
The experiment was conducted for K–6 students, and the results of this study are as follows.
First, an AI education program based on STEM education was developed for students
to use AI. The level of difficulty was set for the program to suit the developmental level
of the students, and it was structured to proceed through discussion and cooperative
activities from setting up a plan to solve problems on their own, to implementing them,
and organizing the results.
Second, to review the effectiveness of the development program, pre- and post-tests
were conducted on creative problem-solving ability, attitude toward AI, and AI literacy. In
all three items, the post-test results were improved compared to the pre-test, and statistical
significance was confirmed. In an age based on AI, creative solving ability that thinks about
solutions to problems in various ways is very important, and education should also be set
in the direction of using AI and enhancing creativity. The results of our study could be the
basis for these claims.
Third, as a result of the emotional analysis of free-form responses to questions about
the development program and one’s own thoughts on AI, positive results were found in all
analysis methods, and through this, it was confirmed that the STEM-based AIED education
program provided students with a positive perception of AI and a learning experience. The
fact that students have a positive perception of AI can also be said to have increased their
tendency to accept AI technology in the future. In other words, by using AI in problems
where AI can be used in students’ lives, attitudes to make their lives more enriching and
convenient for themselves are formed.
Looking at the above results, the STEM-based AIED education program developed
by us had a positive effect on students’ creative problem-solving ability and basic literacy
and attitude toward AI. Therefore, education officials should consider providing real-life-
oriented problem-solving experiences using AI among various educational contents when
composing the curriculum and teachers should also recognize the importance and value of
convergence education that includes the use of AI.
This study is a STEM-based AIED program for K–6 students, and it has specificity
and differentiation compared to other studies. However, it cannot be denied that the
classification model consisting of text and images is mainly used among various AI fields
and that the main content is simulation programming to solve problems. Research to
develop various AI-related educational programs for students who are the future leaders
and to confirm their effectiveness should be continuously conducted. This study—the de-
velopment and effectiveness analysis of STEM-based AI education materials for elementary
school students conducted—serves as a beacon of a new field of AI education and STEM
education, and we hope this article can inspire you to research practice.

Author Contributions: Conceptualization, J.J. (Junhyeok Jang) and J.J. (Jaecheon Jeon); methodology,
J.J. (Junhyeok Jang) and J.J. (Jaecheon Jeon); formal analysis, J.J. (Junhyeok Jang) and J.J. (Jaecheon
Jeon); investigation, J.J. (Junhyeok Jang) and J.J. (Jaecheon Jeon); data curation, J.J. (Junhyeok Jang)
and J.J. (Jaecheon Jeon); writing—original draft preparation, J.J. (Junhyeok Jang) and J.J. (Jaecheon
Jeon); writing—review and editing, J.J. (Junhyeok Jang) and J.J. (Jaecheon Jeon); visualization, J.J.
(Junhyeok Jang) and J.J. (Jaecheon Jeon); supervision, S.K.J.; project administration, S.K.J. All authors
have read and agreed to the published version of the manuscript.
Funding: This study was supported by the BK21 FOUR project (AI-driven Convergence Software
Education Research Program) funded by the Ministry of Education, School of Computer Science and
Engineering, Kyungpook National University, Korea (4199990214394).
Institutional Review Board Statement: Not applicable.
Informed Consent Statement: Informed consent was obtained from all subjects involved in the study.
Data Availability Statement: Not applicable.
Sustainability 2022, 14, 15178 14 of 16

Conflicts of Interest: The authors declare no conflict of interest.

Appendix A
Table A1. Preliminary questionnaires.

Questions Result
I have studied using ICT devices. 1, 2, 3, 4, 5
ICT Using Ability You can study more effectively by studying using ICT devices. 1, 2, 3, 4, 5
ICT devices can be used for the purpose of my desired purpose. 1, 2, 3, 4, 5
It is effective to discuss with fellow learners. 1, 2, 3, 4, 5
Experience in various It is fun to think about solving problems and find relevant
teaching and learning 1, 2, 3, 4, 5
information.
methods I have worked with my fellow learners to solve the problem. 1, 2, 3, 4, 5
I know that various knowledge and information are needed to
1, 2, 3, 4, 5
solve the problems in real life.

Table A2. Questionnaires to measure AI literacy.

Subfactors Questions
Q01. What is AI? Concept
Q02. What can AI do? Function
Q03. How does AI work? Principle
Q04. How we should us AI? Ethics
Q05. Whether is it Ai or not? Recognition

Table A3. Questionnaires to measure AI attitude.

Subfactors Questions
Q06. It seems that if technology advances and AI acts like a human
Social impact of AI being, it can be helpful to humans.
Q07. If AI has emotions, it will be useful for humans.
Q08. I can continue the conversation with AI.
AI and communication Q09. I can communicate fluently with AI.
Q10. I can predict what action AI will do.
Q11. It would be useful for AI to decide.
Interaction with AI Q12. It would be convenient to study where AI is used.
Q13. If AI has emotions, I think I can be friends with AI.
Emotional exchange with AI Q14. I am not comfortable living with an AI that has emotions.
Q15. I am comfortable talking to AI.
Q16. I know how fast AI can work.
Characteristics of AI Q17. I understand how useful AI is.

Table A4. Questionnaires to measure creative problem-solving ability.

Subfactors Questions
1. Keep asking questions with a lot of work in class.
2. Find a variety of answers to a given problem and sometimes give a unique answer.
Self-confidence and
3. I freely express my doctor in class, and sometimes I do not have any opinions.
independence
4. I usually have abundant humor and make others funny even when others are not ridiculous.
5. I like to use my head in my study time.
6. I can say novel and extraordinary thoughts.
7. I can solve the problem in a new way that is already known.
Divergent thinking 8. What I made is new and is very different from what other friends made.
9. I create a variety of ideas to solve the problem.
10. I connect what I don’t care about.
11. I know how to distinguish between the facts and imagination.
12. I can trim the ideas and conclusions in class in class.
Critical and logical thinking 13. I know whether it is right for study time or wrong.
14. I can draw conclusions by myself based on various information with my friends.
15. I can find information related to a given problem.
16. I try to do it to the end without giving up hard and difficult.
17. I want to know more about other topics of this subject.
Motivation 18. I have a lot of fun studying at this time.
19. I try to achieve my goals if I think I have not achieved my goal.
20. If I think I have achieved my goal, I set the next step goal.
Sustainability 2022, 14, 15178 15 of 16

References
1. Hendler, J. Avoiding another AI winter. IEEE Intell. Syst. 2008, 23, 2–4. [CrossRef]
2. Manyika, J.; Chui, M.; Lund, S.; Ramaswamy, S. What’s now and next in analytics, AI, and automation. McKinsey Glob. Inst. 2017,
1, 1–12.
3. Leaders, Y.G. World Economic Forum Annual Meeting 2016 Mastering the Fourth Industrial Revolution. 2016. Available online:
https://www3.weforum.org/docs/Media/AM16/AM16MediaFactSheet.pdf (accessed on 27 June 2022).
4. Chen, L.; Chen, P.; Lin, Z. Artificial intelligence in education: A review. IEEE Access 2020, 8, 75264–75278. [CrossRef]
5. Pedro, F.; Subosa, M.; Rivas, A.; Valverde, P. Artificial Intelligence in Education: Challenges and Opportunities for Sustainable
Development; UNESCO: Paris, France, 2019.
6. Organisation for Economic Co-operation Development. The Future of Education and Skills: Education 2030; OECD Education
Working Paper; OECD: Paris, France, 2018. Available online: https://www.voced.edu.au/content/ngv:79286 (accessed on 27
June 2022).
7. Garber, S. The Fabric of Faithfulness: Weaving Together Belief and Behavior; InterVarsity Press: Westmont, IL, USA, 2007.
8. Gonzalez, H.B.; Kuenzi, J.J. Science, Technology, Engineering, and Mathematics (STEM) Education: A Primer; Congressional Research
Service: Washington, DC, USA, 2012.
9. Martín-Páez, T.; Aguilera, D.; Perales-Palacios, F.J.; Vílchez-González, J.M. What are we talking about when we talk about STEM
education? A review of literature. Sci. Educ. 2019, 103, 799–822. [CrossRef]
10. Holmes, W.; Bialik, M.; Fadel, C. Artificial Intelligence in Education; Center for Curriculum Redesign: Boston, MA, USA, 2019; pp.
1–35.
11. Korea Institute for Curriculum and Evaluation. The Concept and Use of Artificial Intelligence (AI) in School Education; Korea Institute
for Curriculum and Evaluation: Jincheon, Republic of Korea, 2020; Volume 12.
12. AI Council. AI Roadmap. Available online: https://www.gov.uk/government/publications/ai-roadmap (accessed on 27 June
2022).
13. Nam, M.; Kim, J.; Hwang, E.; Jin, C. Analysis of AI Curricula and Textbooks for Elementary Education in China. Korean J. Elem.
Educ. 2021, 32, 133–150.
14. University of Helsinki. Elements of AI. Available online: https://www.elementsofai.com/ (accessed on 27 June 2022).
15. ISTE. Artificial Retrieved Intelligence from in Education. Available online: https://www.iste.org/areas-of-focus/AI-in-education
(accessed on 27 June 2022).
16. AI4K12. 5 Big Ideas in AI. Available online: https://ai4k12.org (accessed on 27 June 2022).
17. Ministry of Education. Artificial Intelligence Class at School. Available online: https://www.software.kr/home/kor/eduInfo/
swedu/textbook/index.do?menuPos=136 (accessed on 27 June 2022).
18. Becker, K.H.; Park, K. Integrative approaches among science, technology, engineering, and mathematics (STEM) subjects on
students’ learning: A meta-analysis. J. STEM Educ. 2011, 12, 23–38.
19. Saraç, H. The Effect of Science, Technology, Engineering and Mathematics-STEM Educational Practices on Students’ Learning
Outcomes: A Meta-Analysis Study. Turk. Online J. Educ. Technol.-TOJET 2018, 17, 125–142.
20. Sarica, R. Analysis of Postgraduate Theses Related to STEM Education in Turkey: A Meta-Synthesis Study. Acta Didact. Napoc.
2020, 13, 1–29. [CrossRef]
21. Siregar, N.C.; Rosli, R.; Maat, S.M.; Capraro, M.M. The effect of science, technology, engineering and mathematics (STEM)
program on students’ achievement in mathematics: A meta-analysis. Int. Electron. J. Math. Educ. 2019, 15, em0549. [CrossRef]
22. Yildirim, B. An Analyses and Meta-Synthesis of Research on STEM Education. J. Educ. Pract. 2016, 7, 23–33.
23. Holmes, K.; Gore, J.; Smith, M.; Lloyd, A. An integrated analysis of school students’ aspirations for STEM careers: Which student
and school factors are most predictive? Int. J. Sci. Math. Educ. 2018, 16, 655–675. [CrossRef]
24. Vlasopoulou, M.; Kalogiannakis, M.; Sifaki, E. Investigating Teachers’ Attitudes and Behavioral Intentions for the Impending
Integration of STEM Education in Primary Schools. In Handbook of Research on Using Educational Robotics to Facilitate Student
Learning; IGI Global: Hershey, PA, USA, 2020; pp. 235–256.
25. Batdi, V.; Talan, T.; Semerci, C. Meta-analytic and meta-thematic analysis of STEM education. Int. J. Educ. Math. Sci. Technol. 2019,
7, 382–399.
26. Kanadli, S. A meta-summary of qualitative findings about STEM education. Int. J. Instr. 2019, 12, 959–976. [CrossRef]
27. Kang, N.-H. A review of the effect of integrated STEM or STEAM (science, technology, engineering, arts, and mathematics)
education in South Korea. Asia-Pac. Sci. Educ. 2019, 5, 1–22. [CrossRef]
28. Murphy, S.; MacDonald, A.; Danaia, L.; Wang, C. An analysis of Australian STEM education strategies. Policy Futures Educ. 2019,
17, 122–139. [CrossRef]
29. Lin, C.-H.; Yu, C.-C.; Shih, P.-K.; Wu, L.Y. STEM based Artificial Intelligence Learning in General Education for Non-Engineering
Undergraduate Students. Educ. Technol. Soc. 2021, 24, 224–237.
30. Astuti, N.H.; Rusilowati, A.; Subali, B. STEM-based learning analysis to improve students’ problem solving abilities in science
subject: A literature review. J. Innov. Sci. Educ. 2021, 10, 79–86. [CrossRef]
31. Hacioğlu, Y.; Gülhan, F. The effects of STEM education on the students’ critical thinking skills and STEM perceptions. J. Educ. Sci.
Environ. Health 2021, 7, 139–155. [CrossRef]
Sustainability 2022, 14, 15178 16 of 16

32. Putra, A.K.; Deffinika, I.; Islam, M.N. The Effect of Blended Project-Based Learning with STEM Approach to Spatial Thinking
Ability and Geographic Skill. Int. J. Instr. 2021, 14, 685–704. [CrossRef]
33. KONG, S.C.; OGATA, H.; SHIH, J.-L.; BISWAS, G. The role of Artificial Intelligence in STEM education. In Proceedings of the
Proceedings of 29th International Conference on Computers in Education Conference, Bangkok, Thailand, 22–26 November 2021;
pp. 774–776.
34. Tyler, R.W. Basic Principles of Curriculum and Instruction; University of Chicago Press: Chicago, IL, USA, 2013.
35. McTighe, J.; Thomas, R.S. Backward design for forward action. Educ. Leadersh. 2003, 60, 52–55.
36. Cho, J. Thinking About Backward Curriculum Design. J. Curric. Stud. 2005, 23, 63–94.
37. Hu, M.; Liu, B. Mining opinion features in customer reviews. In Proceedings of the AAAI, San Jose, CA, USA, 25–29 July 2004; pp.
755–760.
38. Hutto, C.; Gilbert, E. Vader: A parsimonious rule-based model for sentiment analysis of social media text. In Proceedings of the
International AAAI Conference on Web and Social Media, Ann Arbor, MI, USA, 1–4 June 2014; pp. 216–225.
39. Denecke, K. Using sentiwordnet for multilingual sentiment analysis. In Proceedings of the 2008 IEEE 24th International
Conference on Data Engineering Workshop, Cancun, Mexico, 7–12 April 2008; pp. 507–512.
40. Jacobs, A.M. Sentiment analysis for words and fiction characters from the perspective of computational (neuro-) poetics. Front.
Robot. AI 2019, 6, 53. [CrossRef] [PubMed]