Analyzing the outcomes of a robotics workshop on the self-efficacy, familiarity, and content knowledge of participants and examining their designs for end-of-year robotics contests

Abstract

Rapid advances in science and engineering, and pervasive adoption of resulting technological products, are influencing every aspect of human living and fueling a growing demand for a workforce that is adequately prepared for the emerging occupations in STEM fields. Educating students for success in the modern technology-rich workplace requires teachers who have the knowledge, comfort, capability, and training to adopt and integrate new technologies for classroom teaching and learning. Thus, to prepare high school teachers for incorporating robotics in their students’ education and promoting their understanding of engineering concepts and technology applications, a four-week long robotics workshop was designed and conducted annually for three summers. Examination of changes in the workshop participants’ levels of robotics self-efficacy, familiarity, and content knowledge, as well as analysis of outcomes of robotics capstone projects and end-of-year contests, is suggestive of study findings being promising for education researchers and professional development providers interested in leveraging the potential of robotics in STEM education.

1 Introduction

The advancements in scientific discovery, engineering innovation, and technology adoption are shaping all human endeavors including education, work, and leisure. Sustaining this ongoing societal transformation requires an adequately educated STEM workforce. According to the World Economic Forum (2018), one consequence of the ongoing technological advances will be the emergence of new job roles, e.g., Robotics Specialists and Engineers. Preparing students for success in a technology-rich workplace (Varier et al., 2017) requires teachers with knowledge, comfort, and capacity to adopt new technologies effectively for classroom practice (Buabeng-Andoh, 2012; Office of Educational Technology, 2017). A careful incorporation of advanced technologies in developing and implementing curricula and methodologies for STEM education can fulfill the goal of producing a technologically trained and globally competitive U.S. workforce for the $21^{\text {st}}$ century innovation economy (Office of Educational Technology, 2017; Varier et al., 2017). With the U.S. being eliminated from the list of the top 10 countries for scientific innovation (Jamrisko and Lu, 2018), educating and training a strong STEM-prepared U.S. workforce has become urgent. Thus, government, education, and corporate sectors are all focused on creating innovative STEM teaching and learning opportunities for students at all levels. In fact, the Next Generation Science Standards (NGSS) (NGSS Lead States, 2013) emphasize the need for integrating engineering design and practices within the K-12 curriculum.

To impart engaging and interactive STEM learning experiences to students—with personally meaningful and motivating contexts—teachers are increasingly incorporating educational robots (Barker et al., 2012) in classrooms. Recent studies show improvement in student learning with the use of robots (Chin et al., 2014). Students are fascinated with (Vollmer et al., 2011) and appreciate the opportunity to learn with robots, and allowing students to solve real-life problems can enhance their STEM interest (Riojas et al., 2011). The use of robots in teaching embeds opportunities for students to experience kinesthetic learning, i.e., learning by interacting with the physical world (Syofyan and Siwi, 2018), which can engage and retain their attention and improve their understanding of the concepts of a lesson.

Realizing the full promise of robotics in STEM learning necessitates consideration of teacher preparation. Teachers often report low self-efficacy (Bandura, 1982) in their ability to teach science, which in turn affects students’ science learning (Palmer et al., 2015). Schina et al. (2021b) point to low self-efficacy of teachers in digital technologies and their classroom adoption. Prior research has emphasized the importance of developing familiarity with robotics (Schina et al., 2021a) and the need for robotics training for teachers (Vollmer et al., 2011). While teachers indicate that embedding design, engineering, and technology in the classroom is important, they exhibit low levels of familiarity (VandenBos, 2007) in these domains (Hsu et al., 2010). Lack of teacher training is a key challenge preventing the adoption of robotics in K-12 STEM education (Mataric et al., 2007). Teachers’ low level of familiarity with technology can cause a suboptimal integration of robotics in classrooms that can negatively affect student learning.

Many prior studies on educational robotics in K-12 STEM education solely focused either on students or on teachers. Moreover, prior studies have not considered a synergistic combination of robotics and entrepreneurship under the project-based learning (PBL) framework (Barron et al., 1998; Larmer and Mergendoller, 2010). Finally, often teachers may underestimate the potential of students’ ability to succeed in robotics-based learning. This paper is focused on addressing these gaps in the literature. Specifically, collaborative workshops involving teachers and students can reveal to teachers the promise of robotics and entrepreneurship in promoting student learning as well as the ability of students to understand and practice advanced robotics-related concepts. In this spirit, as seen below, we performed comparisons of learning outcomes of students vs. teachers.

To bridge the aforementioned gaps in prior research, we designed a four-week long robotics workshop and conducted it annually for three years, each summer. The workshop simultaneously provided professional development (PD) to high school teachers and educational enrichment to their students within a PBL framework focused on robotics design and entrepreneurship. The workshop curriculum, summarized in subsection 3.1.1, included essential elements required to initiate a novice in building and exploring robotics devices. The workshop was expected to provide several benefits to the teachers and students. First, the workshop participants were expected to gain familiarity with and understanding of the robotics fundamentals. Second, the teachers were to be supported during the academic year (AY) to integrate robotics curriculum and projects in classrooms and after-school programs for their students. Third, the students were encouraged to participate in an end-of-year robotics contest. Since the teachers from the workshop are expected to continue providing robotics education and the PD community may seek to upgrade and conduct similar workshops, it is critical to analyze and share the workshop results and help improve future offerings. While being familiar with a disciplinary domain (e.g., robotics) can catalyze one’s participation in further learning about that discipline, cultivating self-efficacy in the discipline can influence one’s learning ability through their persistence in learning tasks. Accordingly, self-efficacy in and familiarity with robotics are relevant factors to gauge the ability and preparation of teachers to effectively deliver and promote robotics learning to students. Thus, in addition to measuring changes in the content knowledge of participants, it is essential to assess changes in their self-efficacy and familiarity as a result of participating in the robotics workshop. Accordingly, using two survey instruments and a technical quiz, we investigate whether the workshop contributes to any changes in participants’ robotics self-efficacy, familiarity, and content knowledge. To examine the impact of AY activities, we consider design artifacts submitted for the end-of-year robotics contests.

As evidenced through the examination of the pre- and post-workshop survey and technical quiz responses, for both teacher and student participants, numerous statistically significant changes with medium to very large effect sizes were seen in their levels of robotics self-efficacy, familiarity, and content knowledge. Following the summer workshops, through teachers’ incorporation of robotics learning in formal classrooms, clubs, or after-school programs, many students were introduced to the fundamentals and applications of robotics via hands-on experiential learning. Moreover, each year, student teams participated in robotics contests that integrated entrepreneurial explorations and technology creation while intentionally being guided to make meaningful contributions to their communities and society. The results of observations, discussions, reflections, and reviews of students’ robotics designs, under the summer capstone projects and end-of-year contests, indicate that the students gained an appreciation for the real-world relevance of fundamental science, math, and robotics content knowledge and they had ample opportunities to learn, experience, and hone 21^st century skills. The results of this study suggest that robotics education programs can produce significant learning gains among participants, a finding that is promising for education researchers and professional development providers interested in leveraging the potential of robotics in STEM education.

The rest of the paper is organized as follows. Section 2 reviews relevant prior literature to give details of various topics related to this work. Section 3 discusses the structure of the program, including the summer workshops, AY activities, and annual robotics contests. Section 4 enumerates the research questions that are used to examine the efficacy of the workshop and AY follow-up. Section 5 details the research methods, including the instruments and data analysis, used to address the research questions of Section 4. Section 6 provides the pre- and post-workshop survey and quiz results for the participants as well as the highlights of annual robotics contests. Section 7 provides a discussion of the study results, including its limitations and implications. Finally, concluding remarks are drawn in Section 8.

2 Literature review

The theoretical framework of this work is based on PBL, robotics, entrepreneurship, STEM, social cognitive career theory (SCCT), and teacher PD, and various interactions among them. A brief review of these topics follows.

2.1 Project-based learning

PBL is a framework that is widely used in engineering education as it provides a pedagogical approach to facilitate the process of learning new knowledge and gaining a deep understanding through experiential explorations while solving practical problems. Within the PBL framework, to learn about a topic, participants wrestle with challenging problems by posing questions, discussing concepts, anticipating outcomes, formulating investigations, gathering and analyzing data, synthesizing findings, communicating with peers, refining questions, and building artifacts (Blumenfeld et al., 1991). As observed by Barron et al. (1998), PBL embeds learning opportunities through projects that offer ample scope to readily apply concepts and topics learned in classrooms to real-world situations. For example, Sadler et al. (2000) employed cooperative design projects to engage middle school students in constructing physical prototypes for six varied engineering challenges while helping illustrate links between the design challenges and science concepts. PBL has been deemed as a highly effective approach to learn the process of engineering design through the experience of design as active participants (Dym et al., 2005). Moreover, engineering design projects are known to embed myriad opportunities for students to practice and enhance their science and mathematics knowledge (Akins and Burghardt, 2006). Teachers play a vital role in the implementation of PBL in a classroom environment (Kokotsaki et al., 2016). These roles include assistant, emotional support, task manager, reviewer, facilitator, and information collector (Sabouri et al., 2020). Finally, according to Larmer and Mergendoller (2010), PBL consists of seven major steps (see Fig. 1).

Fig. 1

Seven essentials for project-based learning according to Larmer and Mergendoller (2010)

2.1.1 Robotics and PBL

As an interdisciplinary field, robotics requires foundational knowledge from several disciplines. Formulating robotics-based learning in the context of a real-world project constitutes student-centered learning that can spark students’ intellectual curiosity (Endo et al., 2013), e.g., about the underlying content knowledge, while being engaged in hands-on learning. With robotics, students learn both content and thinking strategies through problem-solving (Barron et al., 1998; Blumenfeld et al., 1991) contextualized in a design project entailing more than one solution (Hmelo-Silver, 2004). By promoting knowledge synthesis from multiple domains (Dym et al., 2005), PBL can foster students’ higher-order cognitive skills (Muldoon et al., 2013). Robotics PBL interventions can increase STEM knowledge in secondary school students (Petre and Price, 2004). Educational robot kits (e.g., LEGO (Perdue, 2007), BoeBot (Lindsay, 2010), and VEX (Innovation First Inc, 2007)) offer myriad components that support students’ creativity and allow the use of PBL. For example, robotics sensors can allow students to connect a textbook formula to a tangible measurement they have generated. Robotics PBL activities provide compelling learning experiences and impart critical academic, life, and professional skills to students (Center for Youth and Communities, 2010).

2.1.2 Entrepreneurship and PBL

Today’s students effortlessly interact with technological artifacts (Orsak et al., 2004), yet they often lack an understanding of the engineering and business methods underlying these products (Richards et al., 2002). Combining product and business development experience with robotics can impact students’ STEM abilities and career awareness. Exposing students to the world of technology creation, combined with communication and business skills, can allow them to envision concrete applications of STEM knowledge and skills. PBL integrated with entrepreneurial training addresses a combination of fear of failure, lack of confidence, creativity, flexibility, innovation, and communication skills (Beary, 2013; Nair et al., 2017). Entrepreneurship education programs (EEP) increase multiple markers of education and career success, e.g., self-efficacy, need for achievement, risk-taking propensity, persistence, and being proactive (Beary, 2013; Huber et al., 2014). Exposing students to EEP engenders positive intentions and attitudes that influence their perceptions of desirability and feasibility towards entrepreneurial careers (Cioffi et al., 2014; Krueger, 1993; Pruett, 2012). The Network for Teaching Entrepreneurship (NFTE) has conducted EEP for K-12 students nationally. A study by Nakkula et al. (2003) documents that students showed increased interest in attending college and increased time spent independently reading after NFTE program participation. In a longitudinal study of $\approx 1,300$ former NFTE students, compared to national averages, NFTE program participation was found to be associated with higher rates of high school graduation, STEM degree attainment, self-employment, and higher average income (Beary, 2013). These participants, as a whole, also created hundreds of jobs for others. PBL alone is not sufficient for preparing today’s students to create technology companies of future that produce thousands of well-paying jobs. While PBL may allow students to envision and build innovative products, lacking entrepreneurial strategies their products may not reach customers. Thus, the integration of entrepreneurship into the curriculum is paramount.

2.2 Robotics and entrepreneurship

Robotics, while still a nascent technology, constitutes a platform for innovation to address niche problems with potential to develop into successful businesses. As labor costs rise abroad (Martin, 2012) and use of robots in advanced manufacturing becomes feasible (Javaid et al., 2021), robots have the potential to transform the U.S. manufacturing industry. The decreasing cost of electronics and hardware components, along with advancements in additive manufacturing, promises to lower the cost of robot hardware, making robots economical for diverse industries (Martin, 2012). Students’ fascination with robots can be used to stimulate their interest not only in STEM disciplines but also as a safe context to experience the entrepreneurial process. Robot design activities can embed a comprehensive experience in system integration and product development that can be applied, through the EEP module, to diverse real-world projects. Through PBL experiences, students can practice and hone their creativity, inventiveness, and entrepreneurial skills as well as envision and gain an appreciation of the pathway from STEM education to careers.

2.3 Robotics and STEM learning

Since many STEM principles are inherently incorporated into performing simple tasks with a robot, robotics can illustrate connections between STEM fields and practical applications of classroom learning. For example, students can connect classroom science and math topics to the robotics concepts of travel distance, drive mechanism, and sensor operation. With robotics, students apply tools used by STEM practitioners to visualize and practice STEM concepts that they otherwise find difficult to comprehend (Faisal et al., 2012). Participation in after-school robotics contests is often the only venue where students can explore advanced tools used by engineers and experience engineering design as a model for problem-solving (Benitti, 2012). Such learning opportunities can be made available to students by preparing teachers to adapt a yearlong course in their schools. The NGSS integrate engineering design in science standards, promote science learning in an engineering context, and connect applications of math to science learning. Transition of robotics activities from an after-school setting into the classroom can support this vision of NGSS.

2.4 Social cognitive career theory

SCCT (Lent et al., 2002) provides an empirically based framework wherein cognitive, affective, and contextual factors (e.g., self-efficacy beliefs, outcome expectations, interests, choice goals, and social supports/barriers) mediate students’ selection of academic majors and careers. SCCT offers interest-, choice-, and performance-models, with complex interplay between cognitive and social factors, to reveal the career development pathways from initial interest formation, to activity choice, and finally performing and persisting in a chosen discipline. SCCT studies have revealed cognitive and contextual factors, and their interconnections, that bear on disciplinary selection, action, and persistence across racial and gender groups (Inda et al., 2013; Lent et al., 2011; Wang, 2013). Thus, it is critical to i) promote early STEM interest formation among underrepresented minority (URM) and female students by using robotics to stimulate situational interest (Subramaniam, 2009); ii) build self-efficacy beliefs by offering social support via interactions with well-prepared teachers, graduate students, and researchers; and iii) enable students to envision diverse academic and career options in STEM fields. In Chemers et al. (2011), science self-efficacy and science identity were deemed strong predictors of commitment to undergraduate and graduate STEM studies among 665 URM students. Hands-on experiences with robotics and entrepreneurship can ignite students’ STEM interests, yield gains in their STEM self-efficacy, and promote their STEM self-identity, increasing STEM career likelihood.

2.4.1 Self-efficacy

Bandura (1982) characterized self-efficacy as one’s belief in their ability to “organize and execute courses of action required to deal with prospective situations [$\ldots$]”. Self-efficacy considers one’s assessment of their ability to execute planned actions to succeed in completing the task they have undertaken (Palmer et al., 2015). It is determined from ratings individuals assign to themselves to complete tasks along the four dimensions of confidence, motivation, outcome expectancy, and anxiousness (Carberry et al., 2010). In robotics education, robotics design is integral to both learning and development of robots. Similar to engineering design (Carberry et al., 2010), to meet robotics design requirements, one must consider individual components, subassemblies, and integration of subsystems. One’s self-efficacy in engineering design can impact their learning of engineering itself (Carberry et al., 2010).

2.4.2 Familiarity

Familiarity refers to a form of remembering an event, individual, or object that renders a sense of awareness without a specific recall from memory (VandenBos, 2007). That is, familiarity suggests knowledge about something. Cognitive fluency, prototypicality, and habit all contribute to being familiar with something (Mullin, 2015).

2.5 Robotics-based teacher professional development

Educators are increasingly seeking to adopt robotics to teach STEM concepts across all education levels (Brophy et al., 2008; Cejka and Rogers, 2005; Erwin et al., 2000; Verner et al., 2007). Yet, the potential of robotics remains largely untapped in the school curriculum (Norton et al., 2007), for example as a stand-alone course in high schools. One reason preventing the adoption of robotics in K-12 education is that teachers often find it difficult to connect robotics activities to curriculum outcomes (Norton et al., 2007). The sustainability of robotics activities in K-12 education also depends on the quality of teacher PD. The limited knowledge of teachers in engineering and robotics affects student learning and creates a workforce that is unprepared for STEM occupations. The need for and challenge of robotics PD programs are well documented in the literature (Mataric et al., 2007; Stolkin et al., 2007; Vollmer et al., 2011). Only through teacher PD and comfort with the material and technology can there be curriculum implementation, a multiplier effect, and sustainability (Mataric et al., 2007; Vollmer et al., 2011). Teacher PD programs require significant upgrading to embed engineering content in the teaching of science and math (Kelley and Wicklein, 2009; National Research Council, 2001). To enhance teacher effectiveness, PD programs should support participants in acquiring a conceptual understanding of disciplinary fundamentals and deep knowledge of curricular content, gaining fluency in pedagogical content knowledge, and mastering rapidly evolving tools to promote hands-on learning using contemporary technology (Shulman, 1986; Valdez et al., 1999). Ideally, PD programs need to develop teachers’ capacity to connect STEM learning with real-world problems that are personally meaningful to students. Moreover, PD should support transfer of training by immersing participants in content knowledge, allow modeling and practice of desired skills (e.g., problem-solving and active-learning), promote collective participation, and last for sufficient duration to handle the cognitive demands of new learning (Guskey and Yoon, 2009; Loucks-Horsley et al., 2009).

3 Program structure

Informed by the literature review, the main objectives of our program were to allow the participants to learn fundamentals of robotics and entrepreneurship, and augment this learning through hands-on PBL for increasing their robotics self-efficacy, familiarity, and content knowledge. We conducted a robotics workshop in summer for three consecutive years. Each year, 10 to 18 teachers and 22 to 36 students from 8 to 12 urban high schools attended the workshop. Each year’s workshop included two weeks of guided training and two weeks of collaborative robotics capstone projects. Mentored by an engineering and a science education faculty, each year, four to six undergraduate and graduate engineering students served as workshop facilitators, delivering lectures and supervising hands-on learning. Teachers were expected to deliver the lessons from the workshop to their students in classrooms or after-school programs during the AY. Thus, each teacher was asked to bring two students to the workshop who were to serve as classroom supports during the AY, especially during the hands-on learning sessions. A typical high school class may have 20 to 30 students, requiring three individuals (one teacher and two students) to lead, support, and manage hands-on robotics sessions. For example, a teacher’s student partners from the summer workshop could support them in: distributing and tracking robot components to the class; examining and approving electrical connections of robotics creations of students; and ensuring adherence to various safety precautions for the robot. Throughout the summer workshops, there was ample evidence that often the students absorbed robotics fundamentals and interacted with sensors, motors, electronics, and microcontroller components with a greater ease than the teachers. Thus, these students became a valuable classroom resource for the teachers who were teaching robotics for the first time. At the end of each AY, the schools participated in a robotics-focused entrepreneurship contest. While the workshop was conducted for three years, the AY follow-up was completed for only two years. Due to COVID-19 pandemic, the AY follow-up and end-of-year contest were canceled in March 2020. Thus, the paper presents results from three years of workshop and two years of end-of-year robotics contest.

3.1 Summer workshop

3.1.1 Guided training

The guided training included twenty half-day sessions, each having two parts. The first hour-long part provided short introductions to the scientific, mathematical, and engineering concepts underpinning the session’s focus area. The second three-hour-long part augmented learning through hands-on explorations. The guided training curriculum included the following aspects, all with relevance to robotics: i) mechanical (a refresher on physics of motion and introduction to drive mechanics); ii) electro-mechanical (electrical and electronic components and circuitry, sensors, and motors); iii) computing (Arduino microcontroller and programming (Margolis, 2011)); and iv) applications (introduction, challenge, and capstone). See Fig. 2 and Mallik et al. (2018) for further details of the robotics curriculum. Corresponding to each concept lesson, for experiential learning, the participants performed hands-on activities in teams of two teachers and four to five students. They were provided worksheets containing the principles of session’s concepts, some elementary exploration activities, and instructions for hands-on learning. Experimental demonstrations and activities that followed the introductory lectures were supposed to strengthen participants’ foundations in robotics and stimulate their interest in the workshop. Figure 3a and b show some participants during a lecture session and performing hands-on learning, respectively. Beginning with introductory lessons on robotics fundamentals (Martin, 2000), the participants gradually learned and practiced the major concepts of mobile robots (Siegwart et al., 2011). By day eight, they made their robot follow a black line on the ground and their robot gripper open/close under user or program control. On the last two days, a technology management faculty delivered an entrepreneurship module wherein the participants learned about business planning, business model canvas (Osterwalder and Pigneur, 2010), market analysis, product-market matrix (Ansoff, 1957), Porter’s 5 forces (Porter, 1989), technology S-curve (Schilling and Esmundo, 2009), product development process, and raising capital. They also learned about business incubators, managing intellectual property, and social entrepreneurship. Next, the participants engaged in experiential learning to create and present pitches for their proposed new ventures to the facilitators and peers.

Fig. 2

Robotics content addressed in the workshop

Fig. 3

Participants attending guided training sessions

Incorporation of entrepreneurship can make robotics designs personally relevant and meaningful for participants, specially during capstone projects and robotics contests. Familiarity with and knowledge of entrepreneurship can help students conceive and prototype products with potential market demands. Performing robotics learning within an entrepreneurship context can also support various steps of PBL, e.g., robotics designs that are purpose-driven and have personal relevance to students; activities that hone 21st century skills; and engagement of students in public demonstrations of their creations; among others. In this manner, the adoption of entrepreneurship in the curriculum offers a unique mechanism to engage students in robotics experiences through the PBL framework.

3.1.2 Robotics capstone projects

During the capstone project, teacher-student teams performed engineering design, prototyping, and testing to develop the mechanical structure, drive mechanism, sensor and drive electronics, computer interface, and microcontroller program, which were integrated to produce robots for competing on a mock-up game field inspired from real-world scenarios. Each year, the project staff created and offered different socially relevant robotics challenges. For example, in year three (Y3), the participants developed robots to move along a layout of streets, pick up trash bins from houses along the streets, sort the trash, and deliver it to a sorting facility based on the type of recyclables. Figure 4 shows the mock street view resembling the grid-like streets. Trash bins containing recyclable garbage were modeled using cups placed randomly at the end of the branch in front of houses. The types of trash in the bins were different reflecting real-world scenarios. If a trash bin was present near the house, the robot needed to pick it up and deliver it to the sorting facility classifying the trash bins. The robot needed to identify three different types of recyclable garbage and the method for identifying the garbage type was left open-ended. The robot could traverse the arena by line following using various sensor combinations to make a turn, move forward, or stop.

Fig. 4

Capstone project for year three workshop

3.2 Alignment of workshop activities with PBL

Each year’s capstone project statement was prepared with the seven-step PBL process of Fig. 1 in mind. For example, automating the familiar activity of garbage pickup trucks working in their neighborhoods sought to build on the prior knowledge of participants for engaging their interests. Open-ended aspects of the project encouraged participants to reflect on the scenario and produce creative solutions. For example, participants created varied algorithms for robots to follow lines on the street grid and made use of bin color or bin material to distinguish the trash type. The project engaged students to use advanced robotics technology while learning, experiencing, and honing $21^{\text {st}}$ century skills by collaborating with peers, communicating design concepts to teachers, and critically thinking to defend their designs in front of the facilitators. The guided learning and capstone projects aroused participants’ intellectual curiosity and allowed them to answer their own questions through experiential learning. They explored applications of theoretical learning, revised their understanding of learned concepts, refined their hands-on skills for robot operation, prototyped design and programming ideas, assessed their experimental implementations, and drew conclusions. As participants learned that their initial designs usually did not produce the best products, they were encouraged to follow a systematic process to iteratively refine their design with feedback from peers and facilitators. Finally, they presented their product to the facilitators and others, received feedback on its quality and performance, and reflected on their experiences. The participating teachers engaged in self- and student-assessments as well. Thus, under the PBL umbrella, the participants gained knowledge, skills, experiences, and pride as they successfully worked on various challenges.

3.3 Academic year activities

Some teachers introduced a robotics elective course, which met for two to five days a week, while others conducted after-school robotics programs. All teachers used robotics kits from the workshop and some procured additional robots. They adapted the workshop curriculum for their students’ needs. The students who attended the workshop supported their teachers during the hands-on learning sessions. Each year, four to five graduate engineering facilitators weekly visited schools to observe and support classroom robotics activities. The facilitators and teachers also collaborated to create worksheets, conduct hands-on activities, debug circuits and codes, and assess student work. Each cohort of teachers and facilitators met four times during the AY, along with the project faculty, to share their progress, challenges, and suggested solutions. Most teachers sought to complete the classroom component of their robotics elective by February to allow students to devote the remaining academic calendar to prepare for the end-of-year robotics contest. Table 1 summarizes the demographic information of the students engaged by the teachers at their schools.

Table 1 Demographic information of the students in school

3.3.1 Robotics contest

As evidenced above, PBL offers a compelling framework to learn about and experience robotics, while the adoption of entrepreneurship engenders personal relevance. Integrating entrepreneurship experiences with technology education can expose learners to the world of technology creation while promoting the development of STEM knowledge and skills with a view toward identifying and solving societal challenges. Thus, under the entrepreneurship umbrella, as students identify problems of societal relevance that can be addressed through robotics, they find personal meaning in their robotics explorations. This approach engenders a need to know robotics and exposes various STEM concepts that students need to understand. The contest creates an opportunity for students to take ownership of their learning for being successful in the contest. Thus, through PBL, students envision and build innovative products using robotics knowledge and employ entrepreneurial strategies to help their products reach customers. In this vein, the participants were challenged to identify a home-related problem in Y1 and for a school-related problem in Y2. Since developing participants’ entrepreneurship skills was an integral element of this effort, they had to characterize their solution for its market potential in the form of a presentation to a team of judges.

Description of contest requirements

Student teams from each school had the opportunity to develop several prototypes, however, each school could showcase only one prototype in the contest. The budget to purchase additional robot prototype material was limited to $20. Each team was allowed three to five students to represent their school. The contest prize consisted of paid internships at local start-ups. Winners were selected based on: best overall pitch–first prize, best robotics engineering design pitch–second prize, and best entrepreneurship pitch–third prize.

Identifying and selecting a problem

Teams were advised to identify problems through surveys and analyze opportunities for robotics solutions to make a positive impact. Using suggestions of peers, family, and neighbors students selected appealing options. Next, they developed a problem statement, suggested solution strategies, and selected one deemed the best. The user surveys formed an important factor for the contest, thus the students needed to conduct them diligently. They also needed to explain the iterations involved in coming up with the solution. They focused on selecting a problem that they deemed feasible and interesting to address with an innovative robotics solution.

Designing and building

Each team had to design, build, program, and test a robotics device to address the identified problem. Their design had to be a mechanically stable solution with a practical application and people having an interest to use it. Teams had to submit and present a video demonstration of their design.

Marketing

After identifying a problem and building a prototype, each team had to explore concepts of marketing, customer base, and pricing for creating a marketing plan for their product, a key aspect of entrepreneurship. Moreover, they had to research requirements of budget, materials, resources, and advertising strategy. As a final step, they needed to tie together all their learning to develop a pitch that would persuade people to buy their product.

4 Research questions

The purpose of this study is to examine the efficacy of the four-week long workshops and AY follow-up on the abilities of teachers and students to participate in and perform robotics-based learning activities. Thus, the study analyzes the workshop outcomes on participants’ robotics self-efficacy, familiarity, and content knowledge. Moreover, it examines their design solutions for the robotics contests. The following two research questions are considered.

• 1. What is the extent of change in participants’ robotics self-efficacy, familiarity, and content knowledge after they participate in the summer workshop?

• 2. How successful were the participants in designing, communicating, demonstrating, and pitching their products after participating in the workshop and AY follow-up?

5 Research methods

5.1 Participants

Over three years, the workshop was attended by 44 teachers of whom 43 teachers responded fully to various data collection instruments. The demographic information for the 44 teacher participants is summarized in Table 2. A total of 96 students attended the workshop over three years of whom 81 students responded fully to the data collection instruments. The demographic information reported by 91 students is summarized in Table 3.

Table 2 Demographic and discipline information of the teacherss

Table 3 Demographic information of the students

5.2 Instruments

Since the teachers were expected to deliver robotics education to students during the AY, the workshop integrated instruction and hands-on exploration in robotics and entrepreneurship to increase their self-efficacy, familiarity, and content knowledge. It was paramount to assess participants’ self-efficacy perceptions to establish any possible benefits engendered through the workshop. Moreover, the results of such a study can be used to enhance the content, structure, and organization of the workshop’s future offerings. Even as teachers’ attitudes and beliefs about the significance of robotics and technology are important factors in PD programs (Vollmer et al., 2011), self-efficacy is not the only factor that impacts student learning. As indicated previously, familiarity suggests knowledge about something. Moreover, teachers’ knowledge of the disciplinary content is correlated to their students’ learning gains (Chaney, 1995; Hill et al., 2005). Thus, we also included a familiarity survey and a technical quiz. In summary, the participant responses were obtained for the robotics self-efficacy, familiarity, and content knowledge instruments. While the self-efficacy survey of this study was adapted from Carberry et al. (2010), the familiarity survey and robotics content quiz were self-designed. Sample items from the self-efficacy survey, familiarity survey, and quiz are shown in Fig. 5. Finally, a self-designed rubric, detailed below, was utilized to assess the entries for robotics contests. See below and Mallik et al. (2018) for further details.

Fig. 5

Sample items from the self-efficacy survey, familiarity survey, and quiz

5.2.1 Self-efficacy

The self-efficacy survey included four dimensions to obtain respondent ratings, on a scale of 0 to 100, for their perceived confidence, motivation, success expectation, and anxiety for several items of project-based robotics design (e.g., conducting robotics design, identifying a robotics need, researching a robotics design need, developing robotics design solutions, selecting the best possible robotics design, constructing a robotics prototype, evaluating and testing a robotics design, communicating a robotics design, and redesigning a robot). While the confidence, motivation, and success expectation scores are positively directed, the anxiety-related questions are negatively directed. The anxiety responses are reverse coded by subtracting them from 100, making all four dimensions positively directed.

5.2.2 Familiarity with robotics components and concepts

Since teachers’ disciplinary familiarity impacts student learning, we administered a survey to determine pre/post-workshop changes in respondents’ level of familiarity with the common robotics concepts. Based on the workshop curriculum, familiarity survey had four sections: mechanical, electrical/electronic, sensor/actuator/microcontroller, and programming. It included 21 items on a five-point Likert scale and respondents were asked to rate their level of familiarity with each item (0 = not familiar, 5 = very familiar).

5.2.3 Robotics content knowledge quiz

As teacher content knowledge is an important factor for student learning, we designed a quiz to assess respondents’ knowledge of robotics. The quiz had 30 questions, organized in four sections, to probe respondents’ understanding of the concepts delivered during the workshop. The drive mechanism section had five items on drive train components, differential drive, types of drive mechanisms, and types of motion. The electronics section had 10 items on electrical and electronic components. The actuation/manipulation/localization section had seven items on the terminology, components, and operations related to robots. Finally, the gears/motors section had eight items on scalar, vector, motion, energy, torque, and speed concepts. Performance of individual respondents in the pre/post-workshop quiz gives an effective measurement of their ability to sustain learned knowledge. From the average performance of all the respondents in the quiz, we can gauge the effectiveness of the PD program.

5.2.4 Robotics contest rubric

The robotics contest served as an important data source that included students’: user survey results, robotics artifacts highlighting engineering designs and creativity, a marketing pitch, as well as demonstrations and presentations. Each year, during the contest, teams presented their work to three judges with experiences in entrepreneurship, start-ups, and robotics. Each team analyzed its solution for market potential and presented a pitch by combining all their learning. They explained how they came up with user surveys and analyzed their results. They described design iterations, challenges, and successes in developing prototypes. The judges sought to understand the iterations involved in coming up with the solution. The teams had to explain why they chose the solution, literature research, and the innovation in the product. The judges used the rubric of Appendix A to select the winners.

5.3 Data collection and analysis methods

All survey and quiz data were collected from the participants anonymously using Qualtrics. To examine whether the workshop was effective in increasing participants’ robotics self-efficacy, familiarity, and content knowledge, their responses were obtained at the start and end of the workshop. To examine any statistically significant differences between the means of the pre/post-tests, paired t-tests were performed. The choice of t-tests is guided by our interest in finding the difference between two measurements for the same subject (Elliott and Woodward, 2007), specifically, statistically significant mean differences between the pre/post-test responses of the surveys and quiz. Next, we examined any variations in results for each year. Moreover, we compared the teacher vs. student outcomes and analyzed their correlation. Finally, to assess the quality of student entries in the contest, the judges used the rubrics of Appendix A.

6 Results

6.1 Teacher results

The results summarized in Table 4 show that the teachers had significant pre/post-test gains in robotics self-efficacy, familiarity, and content knowledge. The t-test values in Table 4 show statistical significance for all cases except for self-efficacy and technical quiz scores of Y3 and we reject the null hypothesis for all cases except these two. The Cohen’s d values (Cohen, 1988) in Table 4 indicate a medium effect size of the treatment on teachers’ self-efficacy and a large to very large effect size for all other cases.

Table 4 Survey and quiz results for teachers

6.1.1 Teacher self-efficacy survey results

Figure 6 shows that the confidence dimension had the largest improvement each year, stemming from a low confidence level at the start of teachers’ foray into robotics. However, tasting success in building and programming robots boosted their confidence level. Figure 6 shows that the motivation dimension was higher than other dimensions in pre/post-tests for all three years. This is expected as the teachers self-selected to apply to this program. However, in Y1, the motivation dimension dropped slightly in the post-test (Fig. 6). This may have resulted from our failure to integrate motivational talks providing verbal affirmation of teachers’ progress—a strategy that was included in the early part of the workshop. Teachers’ motivation levels were high when such motivational talks were enacted. Yet, as they handled challenging capstone projects, perhaps their motivation declined and including motivational talks may have boosted their morale. Thus, in Y2, we embedded verbal affirmation throughout the workshop to ensure that the participants did not feel demotivated. Figure 6 shows a slight improvement in teacher motivation in pre/post-tests in Y2. Comparing pre/post-test scores for success in Fig. 6, considerable improvement is seen. The Y2 pre-test shows a drop in the success dimension compared to the Y1 pre-test. However, this drop was overcome in the Y2 post-test. Keeping the lower pre-test success score of Y2 in mind, in Y3 we emphasized to participants that they need not worry excessively about their success in robotics activities since we had planned scaffolds to support them. At each stage of learning, facilitators were available to help the participants overcome any obstacles and participants simply needed to stay focused on each day’s instruction and guidance. As a result, success scores in pre/post-tests improved in Y3. For each year, Fig. 6 shows a small improvement for the anxiety dimension of pre/post-tests. We posit that as most teachers were working with robotics for the first time, they were anxious about their ability to employ robotics in classroom.

Fig. 6

Teacher self-efficacy section wise average score for each year

6.1.2 Teacher familiarity survey results

Table 4 shows that the pre-test familiarity score in Y1 was lower in comparison to Y2 and Y3. Moreover, even as each year’s post-test familiarity score was higher than the corresponding pre-test familiarity score, Y2 and Y3 post-test scores were also higher than Y1 post-test score. Nonetheless, the increase in pre/post-tests showed the largest percentage improvement in Y1 as seen in Table 6. With the pre-test score being highest for Y3, the resulting pre/post-tests improvement was only 72.86% (Table 6). Figure 7 reveals that in the pre-test, the teachers had particularly low scores on two sections: sensor/actuator/microcontroller and programming and they exhibited a significant improvement in these sections in the post-test.

Fig. 7

Teacher familiarity section wise average score for each year

6.1.3 Teacher technical quiz results

As seen from Fig. 8, teachers demonstrated the best performance and the most improvement in the drive mechanism section of pre/post-test. There was a small drop in pre/post-tests average in gears/motors section in Y3. The results in Table 6 show an improvement of 30.09% for the teachers for all the three years combined in the technical quiz.

Fig. 8

Teacher quiz section wise average score for each year

Table 6 shows that among the three areas considered (two surveys and one quiz), the familiarity aspect showed maximum percentage improvement for each year and for all three years combined with 86.89% improvement.

6.2 Student results

Similar to the analysis of results for teacher gains, we analyzed the results for student gains. We had matching pre/post-test data for 81 students and these are summarized in Table 5 and Figs 9, 10 and 11. The t-test values in Table 5 show statistical significance for all cases except for Y3 self-efficacy scores and we reject the null hypothesis for all cases except this one. The Cohen’s d values in Table 5 indicate medium to very large effect sizes for all treatments.

Table 5 Survey and quiz results for students

Fig. 9

Student self-efficacy section wise average score for each year

Fig. 10

Student familiarity section wise average score for each year

Fig. 11

Student quiz section wise average score for each year

6.2.1 Student self-efficacy survey results

Figure 9 shows an improvement in scores for all dimensions of self-efficacy for all three years. Similar to the case of teachers, the largest improvement in pre/post-tests is seen in the confidence dimension for students in all three years. However, unlike teachers, student pre/post-test scores do not show any decline in the motivation scores. The pre-test success score of students in Y2 is seen to be lower than in Y1, which is similar to what we found for teachers. This issue was addressed by ensuring that the participants had access to facilitators whenever they faced problems, and this improved their Y2 post-test success scores. Finally, the students had the lowest score in pre/post-tests for the anxiety dimension. Next, note that the students generally scored better than the teachers on all dimensions of self-efficacy in pre/post-tests for all three years. Even though in Table 5 the post-test self-efficacy scores for the students are generally higher than those of the teachers (see Table 4), since the students also had generally higher pre-test self-efficacy scores, the percentage improvement results for self-efficacy are seen to be better for teachers in Table 6.

Table 6 Percentage improvement in the results

6.2.2 Student familiarity survey results

In Table 5 we see that the familiarity score of students in Y1 pre-test was high (2.46) compared to other years and also compared to teachers’ pre-test familiarity score. Thus, even with the largest post-test student familiarity score in Y1 (3.61), the first year data produced the smallest percentage improvement (46.75%) (Table 6). From Fig. 10, we see that student responses produced a trend similar to that of teacher responses. Specifically, in the pre-test familiarity survey, they indicated lower levels of familiarity with the topics of sensor/actuator/microcontroller and programming and then demonstrated a significant improvement on these topics in the post-test responses. Since the students already had some ideas about the mechanical, electrical, and electronics topics, possibly from high school physics courses, they responded with higher levels of familiarity for these topics. The remaining topics were new to them and they seemed to have gained good levels of familiarity with them. Since the student responses to the familiarity pre-test for Y2 were quite low, while their Y2 post-test responses were similar to Y3, we find that in Table 6 the Y2 familiarity showed the largest percentage improvement of all cases 110.43%.

6.2.3 Student technical quiz results

The technical quiz results in Fig. 11 show the largest improvement in the actuation/manipulation/localization section and relatively minor improvement in the gear/motor section. Overall, for all three years combined, the technical quiz results show an improvement of 38.74% (Table 6).

Finally, using the data in Tables 4 and 5, correlation analyses (Liero and Zwanzig, 2012) for the scores of teachers vs. students are performed. Their average scores for self-efficacy, familiarity, and robotics quiz are found to be highly correlated with the correlation coefficients of 0.977, 0.944, and 0.957, respectively.

6.3 Robotics contest results

Seven teams participated in the contest in spring 2018 and eight teams in spring 2019. A brief description of the first-ranked project for each year is provided below as an illustration of the quality of student work.

• Y1. Pupbot: Pet owners often avoid overnight travel because they cannot leave their pets home alone. According to user survey results, pet owners want a solution that can take care of and give company to pets in the absence of owners. This team created a complete set of robot solutions called Pupbot (see Fig. 12a) to feed, watch, and play with the pet. They mounted a camera on a mobile robot for remote surveillance of the pet, used an ultrasonic sensor to control a food dispenser, and installed a peristaltic pump to control a water dispenser. Moreover, they endowed their mobile robot with an arm mechanism to throw a ball to engage the pet in play. During the contest presentation, the team demonstrated a thorough mastery of robotics concepts, especially, the knowledge of various sensors, and successfully put their learning into practice to address a customer need. They did an extensive literature search and clearly articulated the survey analysis in the presentation. They exhibited excellent collaboration during their prototype demonstration. Overall, the team provided a compelling and informative presentation with careful considerations of entrepreneurial and business strategies.

  Fig. 12

  

  Winning entries of the robotics contests

• Y2. Detec Tech: Based on the surveys conducted at their high school, this team recognized a need to report students’ unauthorized presence in school hallways. In response, Detec Tech robot (see Fig. 12b) was developed as a hallway monitor to determine if a student wandering in the hallways left a classroom without a pass. The robot is instrumented with a timer that sets off upon spotting a student outside classrooms. The student can turn off the robot using their class pass within a one-minute time window. If the one-minute timer finishes before being turned off, the robot reports to the main office to alert the staff. This team showed novelty in combining machine vision technology with the knowledge gained through the robotics curriculum. In addition to gaining a deep understanding of STEM concepts such as mechanical design, sensor technology, and programming, they successfully employed impressive entrepreneurship concepts by proposing marketing strategies, such as various pricing models for different customers, to increase their customer base. Their pitch video successfully emphasized the need for their proposed solution and how it can provide students and teachers with a good study environment. Finally, their presentation showcased their excellent communications abilities.

The entries to and presentations for the contest were assessed using the rubrics of Appendix A. The five categories of the rubrics, viz., robotics engineering, innovation/creativity, entrepreneurship, video/presentation, and written assignments can also be utilized to assess participants’ robotics self-efficacy, familiarity, and content knowledge. Specifically, an indirect measure of participants’ self-efficacy is available from the observations of judges concerning: the confidence that contest participants show during their presentation; the motivation that they demonstrate in persisting to solve a challenging real-world problem; and their expressions of anxiety during the presentation and prototype demonstration. The judges also had opportunities to gauge participants’ robotics familiarity and content knowledge through detailed technical probing in the robotics engineering section of the rubrics, e.g., concerning the programming sophistication, sensors technology, and actuation knowledge, among others. Based on the overall quality of team submissions, confidential scores assigned by the judges, and the project team’s observations, on average, the student participants were deemed to be highly successful across all aspects of the robotics contest.

7 Discussion

Robotics is increasingly proving to be a powerful tool (Chambers et al., 2007; Vollmer et al., 2011) for transforming STEM teaching and learning. Educational robotics can be used for hands-on activities that effectively impart engaging STEM learning experiences (Chambers et al., 2007). The potential of a robotics based curriculum to increase the achievement scores of students in an after-school program has been examined in Barker and Ansorge (2007), however this prior research did not consider formal in-school educational programming. An educational robot based learning system for in-school teaching of students has shown promise to influence motivational factors among students (Chin et al., 2014). To meet the demands of tomorrow’s workforce needs, it is imperative that teachers adopt robotics technologies to impart STEM knowledge to their students (Mataric et al., 2007; Schina et al., 2021a; Vollmer et al., 2011). Effective incorporation of robotics in curriculum, teaching, and learning requires that teachers develop comfort and familiarity with and knowledge of robotics. Efficient PD can enhance technology (e.g., robotics) knowledge of teachers and promote successful integration of technology in STEM curriculum (Dana et al., 2001). However, there is limited prior research on the development and examination of PD that permits teachers to create curricula materials and hone classroom practices for robotics-based teaching and learning. While You et al. (2021) recently developed and examined a robotics-based PD program, it included only teachers as summer program participants and sought to embed robotics in middle school science and math classrooms. Regarding the role of entrepreneurship in K-12 education, the development of entrepreneurship knowledge among primary school students has been shown to yield a robust positive effect on their non-cognitive skills (Huber et al., 2014). Moreover, the impact of entrepreneurship workshops, conducted with support from college students, on the entrepreneurship self-efficacy of middle and high school students has been examined (Cioffi et al., 2014). Yet, prior research has not considered professional development of teachers and educational enrichment of students in programs that synergistically combine robotics and entrepreneurship under the PBL framework. Thus, we developed a program to enhance teachers’ capacity to engage students in robotics through PBL (Larmer and Mergendoller, 2010). We brainstormed to design the PD curriculum and structure and iteratively revised it using participant feedback. We had clearly defined goals of formulating robotics activities under PBL that would help participants gain theoretical knowledge, thinking strategies, and entrepreneurship skills. Participants received numerous opportunities for professionalization and partnership by interacting with engineering students and faculty. Since effective PD can support the transfer of training (Loucks-Horsley et al., 2009), we purposely assessed and supported teachers to enhance their self-efficacy, familiarity, and content knowledge.

7.1 Self-efficacy

Individuals with low self-efficacy in a discipline do not: believe that they can learn the content of that discipline, trust their abilities to teach the discipline, and have confidence in their students’ ability to learn the discipline (Palmer et al., 2015). Self-efficacy encompasses nuanced issues, thus we considered participants’ self-efficacy in learning robotics concepts by embedding throughout the project the following four components of self-efficacy (Bandura, 1977).

• i) Performance accomplishment is related to an individual experiencing success in assigned tasks (Bandura, 1977). During guided training, capstone experience, and end-of-year contest, the participants conducted experiential learning activities assembling, programming, and operating robotics devices for real-world situations to credibly perform assigned tasks. Throughout the project, they were prompted to answer questions and explain their hands-on learning activities. Moreover, they took a technical quiz in robotics before and after the workshop. Project activities gradually and purposefully increased in difficulty from low, to moderate, and finally to high. Many participants were undertaking robotics activities for the first time but with practice, and by following instructions, they became adept in performing them. As they experienced success, their interests and confidence increased. Moreover, scaffolds such as assembly diagrams, circuit schematics, programs, and one-on-one tutorials were slowly faded as participants progressed in their abilities. Facilitators’ instrumental assistance played a major role in participants’ performance accomplishment. In this manner, the participants were offered multiple opportunities to taste success, which contributed to their sense of performance accomplishment (Bandura, 1977).

• ii) Vicarious learning, also known as modeling, occurs when one learns by observing successful peers perform tasks (Bandura, 1977). Participants with varied educational backgrounds and levels of expertise worked in teams for hands-on learning. As team members experienced difficulty in construction, circuitry, or programming, they received support from others who were strong in the corresponding areas. The teams who successfully completed their robotics design tasks were encouraged to help the teams that were lagging. The facilitators encouraged the participants to observe other teams and learn from one another. Observing peers successfully completing challenging robotics tasks allowed the participants to envision being persistent and successful themselves (Bandura, 1977). Finally, facilitators’ career-related role modeling contributed to participants’ vicarious learning.

• iii) Verbal persuasion is related to the verbal affirmation that an individual can master or complete the assigned task (Bandura, 1977). In the workshop, a common and effective way to address this was by incorporating motivational talks. Whenever some participants felt demotivated, the facilitators delivered such talks or played compelling videos. The participants received answers to their queries by interacting with the facilitators. It was observed that participants’ motivation improved when videos of motivational talks were played. Moreover, verbal encouragement from the facilitators positively impacted participants’ engagement in the workshop activities.

• iv) Emotional arousal is related to the state of increased physiological activity (Bandura, 1977). It can be caused by the anxiety of not being able to complete a given task, lowering confidence and negatively affecting one’s performance. Most workshop participants were working with robots for the first time, so it was natural for them to feel nervous, especially during the initial days. Some participants were afraid of damaging components and others were concerned about not understanding concepts or not being able to complete assigned tasks. With each day, as their familiarity increased they experienced a cascade of successes and the negative influence of emotional arousal was lowered. The facilitators observed that when a participant accidentally damaged any components, they became demotivated and hesitant to continue the work. The facilitators encouraged them to follow safety measures and to learn from their mistakes. If a participant continued to experience difficulty after several days of the workshop, the facilitators engaged in one-on-one interactions with them to identify and address the source of their challenges. During their project demonstrations, the participants were able to work on anxiety related to public speaking. Over-excitement of working with novel tools and techniques can also contribute to the emotional arousal. Emotional support of the facilitators was critical in the management of participant emotions.

7.2 Familiarity

Each year, the familiarity survey score showed pre/post-test improvement in averages for teachers and students in all its four sections: mechanical, electrical/electronic, sensor/actuator/microcontroller, and programming. To be an effective robotics engineer, it is important to be knowledgeable in all the four sections. For the robotics design projects, in each team, group members who were good at working with the mechanical, electrical, or programming aspects, initially focused on working on the corresponding design elements. However, as each member of a team needed to know all basic aspects of their robotics design, they engaged in learning from their peers. While students had been previously exposed to the basic concepts of mechanical, electrical, and electronics disciplines in their physics courses, for most students the principles of sensing and programming were challenging and it took them time to grasp these concepts. This was true for the teachers also who had math or science backgrounds and were working with sensors, microcontrollers, and coding for the first time. Having observed this pattern and to prevent overwhelming the participants, we followed a slow but steady pace in introducing relevant topics. Specifically, analyzing the pre-test responses and identifying the low scores permitted us to adapt the PD schedule, not only in the first year but also in the subsequent years. Thus, in the post-test responses, participants scored on average above 3 out of 5 for all sections of the familiarity survey.

7.3 Content knowledge

The technical quiz had four sections: drive mechanism, electronics, actuation/manipulation/localization, and gears/motors. A robotics engineer must have sufficient fundamental knowledge and practical experience in all four sections. It is seen from the pre/post-tests that the teacher respondents performed the best and showed the largest improvement in the drive mechanism section. The student respondents also scored the most on the drive mechanism section in the pre/post-tests but they showed the largest improvement in the actuation/manipulation/localization section. There was a drop in pre/post-test average in the gears/motors section in the third year, both for teachers and students. Several questions in this section entailed mathematical calculations and ongoing interactions with the participants revealed that some of them made computational mistakes, despite having a conceptual understanding, causing the drop in the post-test scores. Since in the pre-test students had low scores in the electronic section, the workshop facilitators were vigilant in checking students’ circuit connections to prevent damage to circuit components. In this manner, during the experiential learning sessions, the facilitators were able to address student misunderstandings about electrical connections. The quiz results showed significant improvement in pre/post-test for most sections and an overall improvement.

7.4 Robotics contest

The increased self-efficacy, familiarity, and content knowledge of the teachers in the post-test illustrates that they have the potential to influence student learning in school. In fact, through the end-of-year contest, the facilitators found that the students had learned about robotics, electronics, coding, and entrepreneurship concepts. They demonstrated their learning and skills through real-world projects in the end-of-year contest. They were provided the opportunity to represent their class and winning teams won bragging rights for their school. Moreover, some students from the winning teams won paid internship awards at local start-up businesses. This created awareness about the importance and prospects of robotics education. It also provided encouragement to other students of the school to take part in robotics activities. Students who get exposed to engineering design, entrepreneurship, and internship at an early stage in life have the potential to develop a keen entrepreneurial mindset and make a lasting impact on human life.

7.5 Strategies for NGSS alignment

As stated earlier, NGSS highlight the need for integrating engineering design and practices within the K-12 curriculum. This workshop introduced robotics design principles to the participants. The NGSS also draw effective connections with the Common Core State Standards for Math. Many concepts introduced in the workshop involved mathematical understanding and calculations, e.g., binary to decimal conversion and vice versa, calculations involving gear ratios, sensor calibration, and motor turning angles. Use of PBL and a design canvas (Kline et al., 2017) enabled consideration of the following eight-step engineering design process (Massachusetts Department of Education, 2006).

• i) Identifying a problem: e.g., delivering coffee in a coffee shop, garbage disposal, etc., using robotics.

• ii) Researching a problem: e.g., through online search seeking current solutions, understanding their workings, and identifying their limitations.

• iii) Developing solutions: brainstorming with team members to identify various ways of solving the problem using the tools available at the team’s disposal.

• iv) Selecting solutions: selecting the best possible solution after analyzing the pros and cons of all solutions.

• v) Prototyping: developing a physical prototype representing the selected solution by integrating the mechanical, electrical, electronic, and computing subsystems.

• vi) Testing/evaluation: validating the solution prototype by running different tests under varying conditions.

• vii) Communicating solutions: giving demonstration, making presentation, and preparing a report.

• viii) Redesigning: revising and reiterating the prototype solution by analyzing the feedback from peers and facilitators.

These steps of the engineering design process are consistent with the science and engineering practices (SEPs) of NGSS that entail “asking questions and defining problems”, “analyzing and interpreting data”, and “engaging in argument from evidence”, among others. Starting in Y2, the workshop participants were introduced to the design canvas of Kline et al. (2017), which assists in data collection and analysis while giving prominence to the business aspects of the problem under consideration. The design canvas (Kline et al., 2017) consists of nine elements of an organization’s building blocks, viz., stakeholders, actors, features, interactions, modes, inputs/outputs, functions, components, and designs, and the interactions between them. The design canvas helped the participants systematically advance through the various stages of product design and development. It revealed to them areas requiring greater attention and allowed them to consider feedback from diverse stakeholders, thus improving both the design process and design itself. It allowed them to explore possible solutions through literature search and brainstorming. They recognized and considered tradeoffs among the different solutions. In Y2, the design canvas was used in a treatment vs. control group setting, revealing its efficacy as a tool through which participants can learn better, have better project results, and gain valuable knowledge (Mallik et al., 2019). Design canvas also served as an assessment tool for the facilitators to gauge participant performance in projects and to refine the pedagogical and support strategies.

7.6 Limitations

This study has several limitations. First, the sample of teachers and students who participated in the study and responded to the surveys is small. Second, the responses to the surveys are self-reported by the participants and thus may be limited in their reliability and validity (Fan et al., 2006; West, 2014). Third, the study provides at best the examination of a project implementation in a specific setting, limiting its potential concerning representativeness or generalizability. Some of these limitations may be addressed by conducting summer workshops at scale with additional time, resources, and participants and utilizing multiple ways of assessing participants’ robotics self-efficacy, familiarity, and content knowledge with validated instruments. Finally, using gender-balanced sampling, with a suitable sample number, can improve the representativeness and generalizability of the results (Schina et al., 2021a).

7.7 Implications

Being one of the few studies to outline the challenges and successes of implementing a robotics curriculum with high school teachers and students, this study has implications for STEM educators and PD providers. The study employed and examined varied strategies that can enhance participants’ robotics self-efficacy, familiarity, and content knowledge and student outcomes, as seen through the end-of-year robotics contest. The results suggest that a well-designed and executed robotics education program has the potential to produce significant gains in robotics self-efficacy, familiarity, and content knowledge among participants despite their low performance on these measures when they begin pursuing robotics explorations. Systematic attention to the four components of self-efficacy, careful examination of pre-test responses, and proactive solicitation of participant feedback all contributed to the iterative refinement of workshop curriculum, structure, and activities. The results of the study illustrate that teachers can successfully embed robotics learning, under the PBL framework, at their schools. Finally, combining robotics with entrepreneurial explorations allowed the students to partake in personally meaningful experiential learning with potential for societal benefits.

7.8 Future directions

While this work measured gains in robotics self-efficacy, familiarity, and content knowledge through participant responses to pre/post-tests, future work may include observational methods and assessment rubrics as part of capstone projects to obtain richer and robust measures and analyses. In this work, the student and teacher participants had diverse characteristics (backgrounds, interests, and commitments). A future study may recruit participants with some common characteristics or examine assessment results by duly considering participant characteristics. Future studies can examine how robotics electives and after-school programs can be designed and implemented to explicitly incorporate the disciplinary core ideas and crosscutting concepts of NGSS. By organizing robotics education workshops in an ongoing manner, there will be numerous opportunities to analyze, learn, and improve their varied aspects and mechanics. Research can also examine how changes in the content and organization of curriculum impact participant learning. Further studies can analyze the quality of classroom implementation of teachers’ learning and student performance.

8 Conclusion

There is a growing interest in developing a workforce that is well prepared in STEM disciplines. Equipping students to be successful in the technology-rich modern workplace calls for educators who have pedagogical skills, content knowledge, and technical capacity to integrate new technologies in their curriculum and instruction. Thus, teachers are increasingly seeking to adopt robotics as a technological tool for teaching STEM subjects. The workshop discussed in this paper provided PD to teachers within a PBL framework with a focus on robotics design and entrepreneurship. It sought to impart to teachers and students the ability to effectively work with robotics technology and entrepreneurship in a classroom learning context. By enhancing the robotics self-efficacy, familiarity, and content knowledge, the workshop demonstrated its potential to support teachers overcome barriers that make it difficult to incorporate new technology-rich content in the classroom. The workshop of this paper integrated strategies to serve as gateways for students’ entrepreneurship explorations. Various assessment tools were used to measure participants’ learning gains and the assessment results can be used to characterize the efficacy of the PD itself. We also examined the outcomes of participants’ AY activities through the end-of-year contests. The descriptions of workshop content, hands-on activities, assessments, and AY follow-up may be useful to the PD community that may seek to upgrade and conduct similar workshops. The robotics content learned during the workshop was tailored and adapted by the teachers for their classroom use and the teachers shared various classroom resources with one another. Each school was visited by one facilitator for observing the class and assisting the teacher in lesson delivery. The students learned a significant amount of content about robotics while gaining practical and theoretical engineering knowledge. This observation was validated through the end-of-year contest where student teams presented high quality, competitive projects.

A: Rubics: For each aspect, place a check mark in the column most appropriate

Team Name: __________________________  

Categories	Multipliers
	$X_{1}=1$	$X_{2}=2$	$X_{3}=3$	$X_{4}=4$
	Did no meet requirement	Needs improvement	Satisfactory	Excelled
Robotics engineering
Design (aesthetics, robustness)
Programming sophistication (logic behind the code, complexity, and approach taken)
Sensors and human interface (use of sensors and LCD, type, applications)
Actuation (type of motors, mechanisms used to move parts around)
Functionality: Does the robot work?
Total
Innovation/Creativity
Solution: Was the solution based on a household problem?
Creative problem-solving approaches taken
Total
Entrepreneurship
Marketing (based on survey questions, social media, research, advertising, robot name)
Scalability (feasibility in real world)
Feasibility of robot price
Total
Video/Presentation
Presentation (pitch, video, clear, articulation of product)
Were the result of the survey mentioned?
Teamwork (were all team members involved in some aspect?
Total
Written Assignments
One page typed (12pt font) document answering the questions: (1) What knowledge in robotics did you have before starting this course in robotics? (2) What did you learn by making your robot and participating in this competition?
Survey: Type your survey questions and answers from your interviews
Total (count and report checkmarks in each column)
Overall Total