Αρθρο
Αρθρο
Αρθρο
RESEARCH ARTICLE
ABSTRACT
Recent developments in Artificial Intelligence (AI) have introduced machine Submitted: December 04, 2023
learning and its applications into everyday life. As technology becomes Published: December 29, 2023
increasingly integrated into the educational system, researchers are focused
on developing tools that allow students to interact with machine learning in 10.24018/ejeng.2023.1.CIE.3131
a way that sparks their curiosity and teaches them essential concepts. Our
instructional proposal, titled “Electric Signals in Machine Learning Using
App Inventor,” focuses on applying learning, transfer, and classification University of Thessaly & School of
Pedagogical and Technological Education
models of audio spectrograms to teach students in the first year of high (ASPAITE), Greece.
Secondary school (A’ Lyceum) fundamental concepts of machine learning.
This is accomplished using MIT App Inventor and Arduino’s visual *Corresponding Author:
programming environments. Students will use the website “Personal Audio e-mail: chryssa.xixi@gmail.com
Classifier” to train an audio model and App Inventor to connect computer
science and machine learning. In addition, with the aid of the Arduino
microcontroller, students will engage in visualising Morse code signals and
investigating Physical Computing, allowing them to create digital solutions
that connect to the real world.
Copyright: © 2023 Xixi et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and
reproduction in any medium, provided the original source is cited.
the activity in the lab than in the regular classroom. This 5. Mit App Inventor, Personal Audio Classifier, and
shows that students can understand very complex concepts Arduino
if the learning environment is well designed and organised.
App Inventor is an event-driven, block-based program-
The majority of curricula for all levels of education in most
ming language. It enables individuals with little or no
countries suggest that machine learning and AI should be
programming experience to create applications. The MIT
implemented in schools [15].
App Inventor project is developed with a simple drag-and-
drop interface (designer–blocks). The Designer interface
allows users to drag and place and customise Android
4. Stem in Education
screen elements, such as buttons. The “blocks” interface
Interdisciplinarity, in the most general sense, refers to provides code blocks that can be arranged to create pro-
the participation of multiple scientific disciplines in the grammes [9]. App Inventor extensions enable users to
resolution of a problem. STEM is interdisciplinary in develop applications with robust machine learning capa-
nature and occurs when two or more disciplines of science, bilities.
technology, engineering, and mathematics collaborate to Audio Classification is the process of analysing audio
solve a problem. STEM education accentuates the connec- recordings by “listening” to them. First, the audio stream
tions between the sciences [17]–[20]. In STEM education, serves as input. The time-lapsed audio more closely resem-
interdisciplinarity occurs when students make connections bles a video than an image. To train the model, the audio
between the various disciplines inherent in a problem file stream is partitioned into tiny or relatively small anal-
they are asked to solve. Based on the findings of their ysis windows. The sound is converted into a spectrogram
study, Sakulkueakulsuk et al. [16] determined that AI using a particular variety known as a Mel spectrogram,
and machine learning activities can be utilised as effective which has been demonstrated to be effective because it
instruments for interdisciplinary education in secondary provides a representation that is more similar to human
schools. In addition, they highlight in their findings that hearing. A spectrogram is a highly accurate and compre-
machine learning and AI assist students in adopting futur- hensive representation of acoustic information. It is an
istic and inter-disciplinary perspectives, as well as being image-graph in which the x-axis represents time, the y-axis
cognizant of the incorporation of new technological solu- represents frequency, and the colour of each point repre-
tions into the social context. sents its amplitude. Consequently, a spectrogram displays
Whether referring to natural phenomena or socioeco- amplitude variations for each frequency component of a
nomic and political issues, the study of a real problem signal [22].
provides a solid foundation for STEM. STEM episte- As shown in Fig. 3, the audio representation depicted
mology addresses real-world, complex problems requiring in the spectrogram resembles an image. Convolutional
interdisciplinary approaches [21]. When students are Neural Networks (CNNs) are utilised because of their
engaged in authentic STEM contexts and asked to solve resemblance to biological vision systems, are the best for
problems they encounter on a daily basis, they are more image classification, and have been proven to work very
motivated because the knowledge they are taught is rele- well in spectrograms too. The term “Neural Network”
vant to their lives. In addition, solving authentic problems refers to an interconnected network of simple computing
through STEM provides emotional outcomes, such as elements (neurons) inspired by the Central Nervous Sys-
increased student engagement, perseverance, as well as tem (CNS) of living organisms, particularly humans. It
crucial educational outcomes, such as critical and logical attempts to imitate the behaviour of human brain neurons
thinking and problem-solving skills. using various mathematical models inspired by their bio-
Gao et al. [17] argue that one of the primary educational logical counterparts [23]. Its function combines the way the
goals of STEM is to help students acquire content knowl- human brain thinks with abstract mathematical thought.
edge about one or more STEM disciplines. According A trained artificial neural network’s primary purpose is to
to this strategy, priority is given to one or more specific be able to conduct certain processes independently, such as
cognitive STEM areas, while the other disciplines serve as recognising sounds or images [24].
a means of acquisition knowledge for these cognitive areas. Many online tools for image classification are more
Additionally, they note that another common learning accessible to students and machine learning novices,
goal of STEM education in the cognitive domain is to help whereas audio classification tools are scarcer. Personal
students develop skills that transcend a single discipline. Audio Classifier (PAC) [3] is a public machine learning tool
In other words, students should be able to use knowl- that enables users to construct, analyse, and export cus-
edge from various disciplines. The degree of knowledge tom audio classification models on the web, in browsers,
for each discipline varies depending on the nature of the without requiring specific knowledge of the underlying
problem or situation, despite the fact that the significance technology and programming. It was designed to provide
of each discipline is treated equally (helping to understand a simple and accessible instrument to enable/encourage
the situation). Concerning the affective domain, students individuals with limited machine learning expertise to
should focus their interest on, engagement, attitude, and comprehend and use audio classification by training their
motivation they acquire from STEM content and practises. classification models.
Lastly, STEM education is anticipated to direct students It is an extension of MIT App Inventor that enables
to STEM professions and, by extension, to meet labour users to create custom mobile applications using audio
market demands. classification and export models from PAC. App Inventor
Fig. 3. Audio files are converted into spectrograms and then fed into a CNN plus linear classifier model, which produces predictions about
the class the sound belongs to.
transforms audio data into image representations via the is based on social constructionism [29] and contempo-
computer’s microphone, allowing students to test their rary theories of “information processing.” To enhance the
audio classification models and implement customised inquiry approach, self-activity, and cooperative learning,
models. PAC is hosted at https://c1.appinventor.mit.edu/ it is recommended to incorporate active learning strategies
and enables users to train and classify various types of and real-world examples into the curriculum. Combining
sounds [9], [25]. model training, programming, and Physical Computing
In order to make electronics circuits education more constitutes the intervention’s innovation.
accessible to students, Massimo Banzi and David Cua- Within the context of this module, examples of machine
trielles created the Arduino platform in 2005 in Italy learning applications may be utilised, where students will
[26]. It is comprised of a motherboard with an inte- train their models and view the results on their mobile
grated microcontroller, as well as digital and analogue devices. They can also “upload” a free application for
inputs and outputs. The Arduino platform has evolved sound recognition to the Google Play Store. Through this
into an extremely flexible programming method, based on method, students will be able to create simple applications
low-cost, standardised hardware, suitable for educational and realise that AI is present in numerous applications in
applications. It uses the Wiring programming language their daily lives.
which is based on C++. Today, visual programming lan- Using the App Inventor visual programming environ-
guages such as Ardublock have been devised to encourage ment and the Personal Audio Classifier, the activity titled
student participation. “Electrical signals in Machine Learning with App Inventor
An Arduino board can be connected to standard periph- and Arduino” seeks to integrate the cognitive field of com-
erals, such as temperature sensors, Bluetooth, LEDs, puter science with machine learning. Students will engage
displays, and flame sensors. The large community of in the visualisation of special Morse signals and Physical
members who provide information, examples, and advice, Computing, which connects the real and artificial worlds
resulting in a substantial online knowledge base, is one of computers, using the Arduino microcontroller.
of its advantages. It features an open-source program- Students have basic knowledge of coding from Primary
ming environment, numerous peripheral driver libraries, and lower Secondary School, primarily through Logo-
and online assistance. Additionally, one of its significant like environments (MicroWorlds Pro, Scratch, and Pencil
advantages is its simplicity of use, even for novices, while Code), and have been familiar with the fundamentals of
retaining flexibility for advanced users [26]. It functions Arduino wiring connections from the skills workshops and
equally well on Mac, Windows, and Linux operating sys- the Informatics course [30]. In addition, students in the
tems and facilitates not only programming but also the first and second grades of lower Secondary school learn
interfacing of the electronic components required by an to recognise and convert audio file formats. Furthermore,
application by providing the necessary units (ready-made) in the first year of upper Secondary school, students learn
for hardware connections, either in card form or using the fundamental programming techniques and commands
wires on a Breadboard [26]. of App Inventor and create mobile applications. Regard-
ing the teaching methodology, the fundamental ideas of
Vygotsky and Papert are adopted, according to which the
6. The Teaching Proposal student builds his knowledge by assuming the active role
The incorporation of machine learning into the cur- of a researcher who conducts experiments and exchanges
riculum, and particularly the interdisciplinary approach opinions with his group in order to acquire a meaningful
through Computational Pedagogy [20], is anticipated to understanding of the problem under investigation. As an
pave the way for a future in which engineering learning assistant, the teacher monitors the class and intervenes
will be understood by all. According to Sanusi et al. [27] as necessary. Concerning the cognitive objectives, by the
and Tang et al. [5], students can comprehend fundamental end of the instructional intervention, the students will
concepts through straightforward activities. be able to: familiarise themselves with machine learn-
Machine learning can be incorporated into the curricu- ing applications, create trained audio recognition models
lum of the first year of high school in the laboratory course with machine learning, and understand when there is
“Computer Applications” under the section “Program- a satisfactory level of trust and reliability in the model
ming Environments–Creating Applications.” According to through experimentation. Also, to be able to select and
the Greek curriculum [28], our educational intervention write the appropriate commands for various components,
defining their hierarchy, especially the Personal Audio In the third phase, students build their knowledge
Classifier extension. Furthermore, to engage in coding through activities. It was emphasised that they must give
for Arduino operation, establish a connection between basis to the result returned by “model.mdl” which was a
the mobile device application and Bluetooth, familiarize list of classifications of the model. The list consists of sub-
students with the microcontroller, and wire LEDs and lists, each sub-list contained two elements: the label of the
resistors. Students must be able to encode the long delay classifier that matched the most instances and a confidence
(dash) and brief delay (dot) and then decode, test the level, such as [[sos, 0.88527], [end, 0.34812]], respectively.
result, and do the debugging. Students explore modular Students created the application in App Inventor by writ-
programming and structured programming while writing ing the necessary coding.
code in Arduino and event-driven visual and object- In the fourth phase, students collaborated, communi-
oriented programming in App Inventor with the use of cated, designed, and implemented the artifact using the
multiple selections, lists, and procedure calls. Students Arduino platform. This activity was devised after stu-
are expected to engage in programming complexity and dents attended an online Arduino fundamentals seminar
artefact creation (Arduino), as well as develop critical (https://seminars.etwinning.gr/). In Physical Computing,
thinking, creativity, communication, collaboration, and students exercised fine motor skills by constructing cir-
fine motor skills during construction. Regarding attitudes cuits on a breadboard and developed critical thinking
and behaviours, through participation in the activity, stu- by programming the Arduino platform. Specifically, the
dents should be able to solve problems collaboratively data obtained from the App Inventor application was
(with the division of labour, discussing opinions and expe- “transferred” via Bluetooth to the Arduino development
riences with their group, and improving solutions through platform, which converted it into a visual signal (Led)
mutual feedback) and be oriented towards STEM careers. of a special Morse code. The word was simultaneously
displayed on the computer screen via a serial port.
In the concluding stage, students presented the final
7. Class Organization–Teaching Proposal Phases results of the activities. Then, there was a discussion
The instructional intervention occurs in the school’s regarding the findings, the data, the conclusions, the
computer laboratory and takes place with twenty-one stu- aspects that impressed them the most, and potential
dents from the first grade of the upper secondary school. improvements and suggestions for their work.
Personal Audio Classifier was used to train models on the After completing the instructional intervention, each
following special Morse codes: sos, end, error, transmit, student was given a rubric evaluation of the intervention
and wait. The mobile device reproduced the sound effect to fill out.
for the corresponding Morse signals while simultaneously
visualising it with a LED that turns on and off accord-
8. Conclusion
ing to the Morse coding functions for visualisation. The
teacher intervenes and guides the students if it is necessary. Many researchers, such as Evangelista et al. [2], believe
During the inquiry process, the training of the model, the that introducing a simplified form of machine learning in
construction phase of the application in the App Inventor education without the complexity of programming helps
environment, and the construction of the Arduino, stu- students understand the significance of developing reli-
dents utilised one computer and one Arduino per two to able machine learning models for real-world problems and
three students and carried out the activities collaboratively. applications. They believe that machine learning is a topic
In the initial phase of conceptualization, students struc- that should be taught from a young age because it is so
ture their solutions based on their research queries or widespread. Our proposed intervention aimed to famil-
hypotheses. The video “How Does Morse Code Work?” iarise students with AI through the use of machine learning
served as the discussion’s beginning point [31]. Then, fun- applications and audio recognition model training, ensur-
damental knowledge of Morse code is gathered through ing an adequate level of confidence and reliability. For
brainstorming, research, and discussion. model training, we suggested using the Personal Audio
In the second phase, students were introduced to model Classifier (PAC), a public and free machine learning
training and machine learning through exploration and application that enables the online creation, analysis, and
experimentation using the first worksheet. After the model extraction of custom audio classification models. Using
training with the Personal Audio Classifier, the “.mdl” file the microphone on their computers, students generated
(export option) was generated, which was required for the sound data, which was converted into spectrograms and
subsequent activity. As this was the first time the students then trained with PAC while developing the programme in
had interacted with the Personal Audio Classifier, the App Inventor to generate visual and audio Morse signals.
teacher provided assistance where necessary. Using DAW Moreover, the students actively engaged in the com-
(Digital Audio Workstation), for example, Audacity, stu- plexity of programming and the creation of an artifact
dents then searched for and converted audio files (Morse (Arduino), developed critical thinking, creativity, commu-
tones) to appropriate formats (mp3). The next step was nication, and collaboration skills, and improved their fine
connecting the trained model by importing the “Person- motor skills throughout the time of construction activities.
alAudioClassifier.aix” extension to App Inventor. Finally, Analysing the results of the evaluation rubric submit-
the application interface was designed in the Designer ted by the students revealed that the participants had a
environment, and the “model.mdl” file was imported. greater comprehension of what machine learning entails
and derived multiple benefits from its incorporation into [10] El Naqa I, Murphy MJ. What is machine learning?. In Machine
the educational process. Learning in Radiation Oncology. El Naqa I, Li R, Murphy
M, Eds. Cham, Switzerland: Springer, 2015. pp. 3–11. doi:
The results of the rubric shown that the students’ famil- 10.1007/978-3-319-18305-3_1.
iarisation with AI concepts helped them understand the [11] Deloitte. Thriving in the era of pervasive AI: deloitte’s state of AI
functioning of many modern devices and applications, and in the enterprise (3rd ed.). [Internet]. 2021. Available from: https://
shorturl.at/FG037.
they believe they will acquire fundamental skills essential [12] Georgouli A. Artificial Intelligence. Athens, Greek: Kallipos Open
for their future lives and careers. Most students supported Academic Publications; 2016. Available from: http://hdl.handle.
the idea that applying machine learning sparked their inter- net/11419/3381.
[13] Marques LS, Gresse von Wangenheim C, Hauck JC. Teaching
est, and they are considering pursuing careers in the field of machine learning in school: a systematic mapping of the state of the
AI as the next generation of researchers. There were no sig- art. Inform Educ. 2020;19(2):283–321.
nificant differences between genders in how they worked, [14] Kucak D, Juricic V, Djambic G. Machine learning in education—A
survey of current research trends. Proceedings of the 29th DAAAM
their experiences, or their beliefs about the possibility of International Symposium, Katalinic B, Ed. vol. 29, pp. 406–10,
pursuing AI-related careers. All students found the lesson Vienna, Austria, 2018.
interesting and would like to repeat it in the future. [15] de Oliveira FP, von Wangenheim CG, Hauck JC. TMIC: app inven-
We advocate for the incorporation of machine learn- tor extension for the deployment of image classification models
exported from teachable machine. To be published in computer
ing into the curriculum because we anticipate that it will science. ArXiv. [Preprint]. 2022. Available from: https://arxiv.org/
help students grasp the fundamental concepts of machine abs/2208.12637.
learning, pave the way for a future in which machine [16] Sakulkueakulsuk B, Witoon S, Ngarmkajornwiwat P, Pataranuta-
porn P, Surareungchai W, Pataranutaporn P, et al. Kids making
learning is accessible to all, and pique students’ interest AI: integrating machine learning, gamification, and social con-
in this area of computer science. To enhance exploratory text in STEM education. 2018 IEEE International Conference on
approaches, self-motivation, and collaborative learning, Teaching, Assessment, and Learning for Engineering (TALE), pp.
1005–10, Wollongong, NSW, Australia: IEEE; 2018 Dec 4–7. doi:
we suggest incorporating active educational techniques 10.1109/TALE.2018.8615249.
and real-world examples into the curriculum. [17] Gao X, Li P, Shen J, Sun H. Reviewing assessment of student
App Inventor supports additional extensions, such as learning in interdisciplinary STEM education. Int J STEM Educ.
2020;7(1):1–14.
the Personal Image Classifier (PIC) for image recognition [18] Ntourou V, Kalogiannakis M, Psycharis S. A study of the impact
training. Our team is expected to present future publica- of arduino and visual programming in self-efficacy, motivation,
tions using one of the aforementioned web-based machine computational thinking and 5th grade students’ perceptions on
electricity. EURASIA J Math Sci Tech Ed. 2021;17(5):em1960. doi:
learning tools, which provide a quick, simple, and readily 10.29333/ejmste/1084218.
available method for training a model. [19] Psycharis S. STEAM in education: a literature review on the role of
computational thinking, engineering epistemology and computa-
tional science. Computational STEAM pedagogy (CSP). Scientific
Culture. 2018;4(2):51–72. doi: 10.5281/zenodo.1214565.
Conflict of Interest [20] Psycharis S, Kalovrektis K, Xenakis A. A conceptual framework
for computational pedagogy in STEAM education: determi-
Authors declare that they do not have any conflict of nants and perspectives. Hell J STEM Educ. 2020;1(1):17–32.
interest. doi: 10.51724/hjstemed.v1i1.4, http://www.hellenicstem.com/index.
php/journal.
[21] Mayes R. Quantitative reasoning and its role in interdisciplinarity.
In Interdisciplinary Mathematics Education. Doig B, Williams J,
References Swanson D, Borromeo Ferri R, Drake P, Eds. Cham, Switzerland:
[1] Jordan B, Devasia N, Hong J, Williams R, Breazeal C. PoseBlocks: Springer Nature, 2019. ch. 8, pp. 113–33
a toolkit for creating (and dancing) with AI. Proc AAAI Conf Artif [22] Zeng Y, Mao H, Peng D, Yi Z. Spectrogram based multi-task audio
Intell (17). 2021;35:15551–9. classification. Multimed Tools Appl. 2019;78(3):3705–22.
[2] Evangelista I, Blesio G, Benatti E. Why are we not teach- [23] Qi X, Chen G, Li Y, Cheng X, Li C. Applying neural-
ing machine learning at high school? A proposal. 2018 World network-based machine learning to additive manufacturing:
Engineering Education Forum-Global Engineering Deans Council current applications, challenges, and future perspectives. Eng.
(WEEF-GEDC). Albuquerque, NM, USA: IEEE; 2018 Nov, 12– 2019;5(4):721–9.
16. doi: 10.1109/WEEF%2DGEDC.2018.8629750. [24] Doshi K. Towards Data Science Medium. Audio deep learning
[3] Bhatia N. Using transfer learning, spectrogram audio classification, made simple (part 1): State-of-the art techniques [Internet]. 2021.
and MIT app inventor to facilitate machine learning understanding. [cited 2021 Feb 11] Available from: https://towardsdatascience
Ph.D. Thesis, Massachusetts Institute of Technology; 2020. .com/audio-deep-learning-made-simple-part-1-state-of-the-arttech
[4] Zhu K. An educational approach to machine learning with mobile niques-da1d3dff2504.
applications. Ph.D. Thesis, Massachusetts Institute of Technology; [25] Finzer W, Chao J, Rose C, Jiang S. Story Q-an online environment
2019. Available from: https://hdl.handle.net/1721.1/122989. for machine learning of text classification. Proc AAAI Conf Artif
[5] Tang F, Kawamoto Y, Kato N, Liu J. Future intelligent and secure Intell. 2022;36:12860.
vehicular network towards 6G: machine-learning approaches. Proc [26] Kushner D. The making of arduino. IEEE Spectr. 2011;26:1–7.
IEEE. 2019;108:292–307. [27] Sanusi IT, Oyelere SS, Agbo FJ, Suhonen J. Survey of resources for
[6] Samuel AL. Some studies in machine learning using the introducing machine learning in K-12 context. 2021 IEEE Frontiers
game of checkers. IBM J Res Dev. 1959;3(3):210–29. doi: in Education Conference (FIE), pp. 1–9, Lincoln, NE, USA: IEEE;
10.1147/rd.33.0210. 2021 Oct 13–16. doi: 10.1109/FIE49875.2021.9637393.
[7] Mahesh B. Machine Learning Algorithms-A Review. International [28] Institute of Educational Policy. Curriculum for the course of
Journal of Science and Research (IJSR). 2022;9(1):381–6. computer science A, B, and C grade of high secondary school.
[8] Hu J, Niu H, Carrasco J, Lennox B, Arvin F. Voronoi-based multi- 2023. Available from: https://nickpapag.sites.sch.gr/2023/05/04/
robot autonomous exploration in unknown environments via deep neo-programma-spoudon-gia-pliroforiki-lykeiou-2023/.
reinforcement learning. IEEE Trans Veh Technol. 2020;69:14413– [29] Vygotsky L. Mind in Society: The Development of Higher Psycho-
23. logical Processes. Cambridge, MA: Harvard University Press; 1978.
[9] Lao N. Reorienting machine learning education towards tin- [30] Institute of Educational Policy. Curriculum for the course of
kerers and ML-engaged citizens. Ph.D. Thesis, Massachusetts computer science A, B, and C grade of low secondary school.
Institute of Technology; 2018. Available from: https://hdl.handle. 2023. Available from: https://nickpapag.sites.sch.gr/2023/05/04/
net/1721.1/129264. neo-programma-spoudon-gia-pliroforiki-gymnasiou-2023/.
[31] Concerning Reality. How does morse code work? [Video]. 2019.
Available from: https://www.youtube.com/watch?v=iy8BaMs_JuI
[Accessed May 02, 2022].