Introduction

The integration of Artificial Intelligence (AI) solutions in formal education, and more specifically in the classroom, has been lately in the spotlight as the potential solution for – nearly – every “problem”. From supporting the automatic or semi-automatic assessment of students’ performance and tracking of students’ progress (Heffernan and Heffernan 2014; Luckin 2017) to providing students with personalized scaffolding and recommendations (Albacete et al. 2019; Tarus et al. 2018). Consequently, the wide range of AI applications raises the question of whether, how and to what extent we can use AI technologies to support the teachers overcome the challenges they face as part of their work practice. Related research (Holstein et al. 2017; Holstein et al. 2018) focused on eliciting these challenges, needs and desires by introducing the notion of “teacher superpowers.” In this work, we borrowed this notion and used it to elicit the challenges teachers face, similarly to Holstein et al. (2017). We envision that the term “superpower” as a suggestive metaphor for AI, will work as a catalyst for teachers to think out of the box and express their needs and latent wishes without the burden of making them fully explicit or tying them to existing tools.

The focus shift towards AI in education is reinforced by the overall trend towards quantitative, evidence-based policies that take advantage of technological advances to support informed, data-driven decision-making. AI may bring considerable enhancements to human capabilities. At the same time, it sets new challenges we are called to address regarding its appropriate and ethical use Holmes et al. (2019) and Holmes et al. (2019), the fairness of AI algorithms and its proneness to bias (Holstein et al. 2019; Porayska-Pomsta and Rajendran 2019; Gardner et al. 2019), interpretability (Rosé et al. 2019), and its impact (Luckin 2018). Shum and Luckin (2019) pointed out that in order to successfully address concerns and criticism, we need to communicate in “accessible terms” with stakeholders – namely, teachers, students, parents, and potentially unions and policy-makers – the benefits of AI for education but also the potential dangers and pitfalls.

Motivation for Research

In order to establish this communication in accessible terms, we first have to establish a common ground and understanding about the knowledge, perceptions, and expectations that stakeholders have from AI and the challenges that the stakeholders face in context. To that end, our aim was to answer the following research questions in the context of Estonian K-12 formal education:

RQ1. How do Estonian K-12 teachers perceive Artificial Intelligence as a means to support teaching and what are their expectations?

RQ2. What are the perceived challenges that Estonian K-12 teachers face regarding their work practices?

In this paper, we present a study we carried out as part of our efforts to gain insight with respect to our research questions. Additionally, we aimed to establish common ground with Estonian K-12 teachers regarding the benefits and pitfalls that the use of AI in education may entail. Estonia is promoting digitalization and innovation for the society as a whole (Runnel et al. 2009) and in formal education specifically, by supporting the integration of new technologies, acquisition of digital literacy, and the adoption and adaptation of digital pedagogies.Footnote 1 This is reflected by the Index of Readiness for Digital Lifelong Learning (IRDLL), according to which Estonia was ranked first among 27 European countries.Footnote 2 However, earlier studies showed that Estonian K-12 teachers are not aware of the benefits that the integration of AI in the classroom could bring in their work practice, they do not know how to integrate AI-enhanced tools in their classroom and they are not aware whether their schools have policies in place for promoting AI in education.Footnote 3

To contextualize our findings with respect to fairness, accountability, transparency, and ethics (FATE), we revisit the discussion of Aiken and Epstein regarding principles for designing AI systems (Aiken and Epstein 2000) to validate these principles from the perspective of teachers. The contribution of this work is twofold:

  1. 1.

    To provide insight regarding the factors to take into account when designing AI-enhanced technologies and integrate them in the classroom in the context of FATE;

  2. 2.

    To provide insight into the challenges teachers face in technological-intensive work contexts and push forward the discussion about teachers’ professional development in the digital era.

In the following section, we will provide an overview of the background and related work on the topic. Next, we will present the methodological approach, and we will discuss the results of the survey. We will provide a contextualized discussion of the findings, and we will elaborate on the theoretical and practical implications addressing FATE. Finally, we will conclude with the limitations of this research and future work.

Background

In this section, we provide a short overview of the role of AI in education, we theorize about the relationship between education and AI within the scope of FATE and we contextualize our research in Estonian K-12 formal education.

AI in Education

Artificial Intelligence (AI) refers to computational technologies that allow machines (that is, computers) to make decisions by imitating human intelligence (McCarthy 1998). Research on the use of AI in education has been active ever since the late ’70s, for example, in computer-assisted instruction (CAI) and intelligent tutoring systems (ITSs) (Nwana 1990). AI methods were employed either in the design of interactive learning environments that would support learning by doing (Papert 1980) or for the design of computational tutoring systems that would “imitate” human tutors in the way they adapted instruction with respect to the student’s knowledge state (Corbett et al. 1997). This was also reflected in Estonia, with CAI and ITSs shaping common practice since the late 80’s, especially in the field of Mathematics for Higher Education (Prank 1991; Prank and Õunapuu 1997). Since then, AI has been used either as a learning tool – students would learn through experimentation with AI algorithms – or as a technological tool to support the personalization of learning environments and adapt instruction to learner’s needs and personal goals. Papert’s constructionist approach has strongly influenced modern learning environments, especially for STEM (for example, Scratch).Footnote 4 On the other hand, ITSs were widely adopted in K-12 and Higher Education, demonstrating promising results and outcomes (VanLehn 2011). ITSs’ functionality is based on the use of AI-enhanced student models. These models are capable of tracking student performance and providing appropriate content for practicing skills, and fostering knowledge tailored to the individual student’s needs (Albacete et al. 2019; Chounta et al. 2017).

In the context of K-12 education, AI can be used – among others – as a teacher’s tool, either to facilitate the design of learning activities and scaffolding strategies or to support teachers’ awareness by providing information about student’s activity and performance (Pardo et al. 2018; Chounta and Avouris 2016; Roll and Wylie 2016). In the first case, AI can facilitate the adaptation of content to students’ knowledge level and to provide personalized recommendations for learning materials that address students’ specific needs. Learning platforms, such as KnewtonFootnote 5 and Edmodo in collaboration with IBM Watson education,Footnote 6 work towards using AI to improve learning outcomes and students’ achievement specifically for K-12. In the second case, AI is used to facilitate and support the analysis and visualization of students’ data and to provide indicators related to various aspects of learning, such as performance, knowledge state, affective state, cognition and metacognition (Verbert et al. 2013). Such indicators are presented to teachers through teacher-facing dashboards to help them orchestrate their practice and support awareness (van Leeuwen et al. 2018; Holstein et al. 2018; van Leeuwen and Rummel 2020).

Retrospectively, one may argue that early AIED research did not deliver what was promised (Schiff 2020). Reflecting on previous work, we pinpoint three factors that may have contributed to this and can serve as lessons learned for the future:

  • Early AIED research focused on problem representation without considering background knowledge (Chi et al. 1981). This led to the design of learning environments and intelligent tutors that offered instruction and support based on how an expert of the domain would act to carry out a learning activity or task (expert systems). However, research has shown that novice learners operate differently from experts, exactly because they lack prior knowledge that the expert learners already have (Council et al. 2000). Similarly, early AIED research focused on modeling cognition primarily using sets of production rules. One limitation of such approaches was that the author of the production rules should understand the knowledge involved in carrying out a task and the exceptions that may occur. Another limitation was that production rules were hard-coded and could not be changed or adapted later on. In other words, these intelligent systems were not able to learn and adapt (Luckin 2018).

  • Implementation barriers were not carefully considered (Schiff 2020; Nye 2015). Expectations from technology were higher than what technology could deliver. The processing power of the early tutoring systems, the cost of data collection and storage, and the kind of information that was possible to record and analyze were limiting factors concerning what these systems could offer. At the same time, cultural aspects, work habits and so on, hindered the adoption and integration of AI technologies in context.

  • Early AI was not explainable. In other words, the computational approaches that were used to develop these “clever” systems could not provide explanations or justifications for the decisions these systems made (Luckin 2018). The inability to provide explanations along with decisions limits the power of intervention. Consequently, it is hard for the end-user to trust a system if they cannot understand the reasoning behind the system’s actions.

A detailed report on educational platforms used in K-12 (Ferguson et al. 2016) - shows that the “intelligent” educational tools are rarely used in the classroom consistently. According to the New Media Consortium (NMC) Horizon Report for 2017 (Freeman et al. 2017), potential explanations could be that either the schools do not invest resources into integrating cutting-edge technologies into their curricula or that teachers are not adequately trained to integrate AI-enhanced technologies in the classroom.Footnote 7 Another potential explanation could be that the available, AI-enhanced technologies do not sufficiently address K-12 teachers’ needs.

Indeed, studies indicate that there is a need for participatory approaches when it comes to designing AI-enhanced tools for teachers. For example, Holstein et al. (2018) followed a participatory approach where they interviewed middle school teachers using the notion of “superpowers” as a probe to elicit challenges that these teachers typically face. Then, the authors highlighted key themes that appear in teachers’ practice - such as the wish to see students’ thought processes, the potential to know which students are stuck and the ability to clone themselves - and that can be potentially addressed by appropriate design of real-time, teacher-facing supports. This makes evident the need to involve teachers throughout the whole design process to understand their specific needs for real-time analytics, their desires for data visualization, and the challenges they face with respect to interpreting information (Chalvatza et al. 2019).

Situating the Relationship Between Education and AI in the Context of FATE

Research has raised critical concerns about the integration of AI technologies in the classroom and voiced the need for having the ethical and regulatory mechanisms in place to understand and encompass the implications of AI systems (Luckin 2018). Such implications entail the threats AI systems may impose on human interactions and relationships, the quality of learning, and stakeholders’ (namely, teachers and students) privacy. AI in education is usually portrayed as the means for the personalization of learning. However, there is little evidence for the benefits of personalization and especially in combination with reinforcing bias (Bulger 2016). AI has been criticized for putting aside social interaction and social aspects of learning (Hawkins et al. 1982), for focusing on training individual skills at the expense of students well being and relatedness, and for questionable prioritizing in terms of goals and aims (Bulger 2016). At the same time, AI algorithms may amplify negative stereotypes, social inequities and unfairness (Holstein et al. 2019). One could argue that removing data instances that could reinforce bias would be the obvious solution to fair systems. However, this is not a solution in educational contexts where our goal is to support learners by understanding their needs, especially when this results from systemic or racial disparity (Ogbonnaya-Ogburu et al. 2020).Footnote 8

To address ethical concerns for the development of AI systems for education, Aiken and Epstein (2000) proposed a set of design principles based on six fundamental dimensions of human being: ethical, aesthetics, social, intellectual, physical, and psychological. The resulting ten principles placed emphasis on basic human needs, such as the importance of social interaction and well being, and pointed out the need to empower positive attitudes, such as creativity and curiosity. At the same time, some principles aimed to encompass uniqueness, difference, and diversity and reinforce the teacher’s role as the orchestrator and facilitator of learning with the AI system being there to support and not substitute the human.

In our work, we start by assuming that the relationship we establish with AI is more complex than the one that describes it as separate from the context in which it is supposed to operate. Having this in mind, we attempt to revisit the conversation that Aiken and Epstein began about “the ethical principles that can and should guide the development of AI systems for education” (Aiken and Epstein 2000) and to validate these principles from the perspective of teachers towards the integration of AI systems in the classroom as a teacher-facing tool.

The Case of Estonia: Context About K-12 Teachers in Estonia and their Perception About Technology

Estonia is a small country with 516 schools on the K-12 level, with a bit less than 16 thousand teachers (according to statistics provided by the Estonian Ministry of Education and Research, November 10, 2019). In international comparison Estonian teachers are mostly women (84%) of mature age and experience in teaching profession, 49 and 21 years respectively. However, they are indeed very actively participating in continuous professional development activities although most of them feel that they need much more Information and Communications Technology (ICT) skills for teaching (OECD 2019). It is also characteristic that 40% of Estonian teachers are not willing to work in the school for more than five years even if they have very high autonomy in determining their course content (OECD 2020). All K-12 teachers in Estonia need to have a master’s degree or professional licence awarded based on teachers professional standards. The standards provide a competency framework (Pedaste et al. 2019) that also highlights the competences in applying educational technologies in planning, teaching and assessment and supporting teacher continuous professional development. All teachers need to be capable of evaluating their digital competence in order to make plans for professional development. The use of technology in teaching and learning is usually highly appreciated among Estonian teachers; however, due to different reasons, many of the teachers do not use educational technologies systematically. For example, a study surveying the use of mobile devices in science learning in 2016 pointed out that about half of the students belonged to group “non-users” who used mobile devices for learning less than once a month. Only 5% of students used mobile devices frequently for learning and tasks given to them by teachers (Pedaste et al. 2017). This finding shows that students do not consider their mobile devices as a learning tool, and it may be an indication that teachers do not involve technology - in this case, mobile devices - when they design learning tasks.

Methodology

Study Setup

For our research purposes, we conducted a survey among Estonian K-12 teachers. The survey was hosted online using the infrastructure provided by the University of Tartu and it was available for 45 calendar days. Anyone with the link could access and fill in the survey. To recruit participants, we sent out email invitations to teachers’ professional networks. Additionally, the research was supported by the Estonian Information Technology Foundation for Education (HITSA), and the survey was communicated through their social media. HITSAFootnote 9 is an organization that actively promotes the use of information and communication technology in education and supports the preparation of highly competent IT-specialists. To avoid multiple responses from the same individuals, we restricted participation in the survey based on the participant’s IP address. The participants were introduced briefly to the purpose of the research. Additionally, they were asked to provide their consent. The survey was carried out in Estonian. Free text responses were translated into English for further analysis by a professional translator who is also a K-12 teacher and a co-author of this article.

Structure of the Survey

Our overarching goal was to use the results of this survey as input for designing AI solutions for online learning environments. Thus, with this survey, we aimed to gain insight into three aspects: a) teachers’ perceptions, attitudes, and familiarity regarding AI; b) teachers’ perceived challenges regarding their work practices; c) teachers’ work profiles and contexts. To that end, we structured the survey into three parts, each addressing the aforementioned three aspects. For the survey, we maintained a fixed structure as follows:

  • In the first part, we asked about participants’ perceptions, attitudes and familiarity with AI (5 questions). On the one hand, we wanted to grab their attention and engage them in discussing a prominent and potentially controversial topic, and on the other hand, our goal was to set the stage contextually for the second part of the survey;

  • In the second part, we followed up asking participants about perceived challenges (1 question) in order to take advantage of participants engagement (built during the first part of the survey). Furthermore, we wished to capitalize on the implicit connection between AI and superpowers aiming to focused participants’ responses;

  • In the third part, we asked about participants’ work practices and contexts (4 questions) in order to minimize the impact that participants’ fatigue could have on their responses. In other words, we saved for last the questions that did not require much cognitive effort or critical thinking and could be answered using predefined options.

Next, we will elaborate on the three parts of the survey in detail (the instrument is provided in Appendix A.3).

Teachers’ Perceptions, Attitudes and Familiarity Regarding AI

To explore teachers’ perceptions, attitude and familiarity regarding AI and its use in educational contexts (RQ1), we adapted the Artificial Intelligence: Public Perception, Attitude and Trust survey to our context (Holder et al. 2018). The first part of the survey consisted of 5 items. The first two items aimed to explore teachers’ personal knowledge about AI. In particular, the first item asked the participants to rate their knowledge about AI on a 6-point Likert scale (perceived knowledge about AI). The second item provided the participants with five statements about AI and asked them to mark the true statements. For example, one of the statements was: “AI doesn’t necessarily have a physical form. It can be just software”. Our hypothesis was that the combination of responses on these two items would provide insight with respect to the teachers’ knowledge about AI. The third item aimed to record participants’ familiarity with using AI (“Have you ever used an AI application?”). The last two items aimed to explore teachers’ perceptions of the use of AI in education. For the last two items, we provided the participants with sets of positive and negative aspects of AI as recovered from the literature. Additionally, we gave participants the possibility to enter their input using a free text component.

Teachers’ Perceived Challenges in the Classroom

The potential of using AI in order to equip teachers with “superpowers” has been explored in many directions: to promote personalization, to support the identification of struggling students, to facilitate recommendation of learning resources. Therefore, the second part of the survey aimed to uncover teachers’ perceived challenges in the classroom (RQ2). To that end, we adopted the approach of Holstein et al. (2017), who asked teachers: “If you could have any superpowers you wanted, to help you do your job, what would they be?”. In this way, the authors wanted to record teachers’ needs, challenges, and issues without considering potential technological limitations. In this survey, we asked the teachers to name up to three superpowers that would help them do their job. Our aim was to gain a broad understanding of the challenges teachers face and determine potential relations (interconnections or hierarchies) among the challenges teachers face. The teachers were able to provide their input using free text.

Teachers’ Professional Profiles and Work Contexts

The third part of the survey aimed to gather information about the professional profile and the participants’ work context (RQ1). In particular, we asked the participants what kind of learning technologies they use to support their practice, what areas of their work could be potentially supported by AI and whether they would like to know what kind of technologies are behind the tools they use. Additionally, we asked participants how long they have been working as teachers in K-12.

Participants

Overall, one-hundred and forty (140) teachers participated in the survey. After removing incomplete responses, one-hundred and thirty-one (131) individual responses remained. The majority of the participants (129 teachers, 98% of the survey population) stated that they use learning software or applications to support their teaching practice. Most of the participants (37%) were experienced K-12 teachers with more than 20 years of experience, followed by participants working as K-12 teachers between 10 and 20 years (28%), and 17.5% of the teachers had work experience between 5 and 10 years. The remaining 17.5% of the teachers had less than five years of professional experience. The majority of the participants (129 teachers, 98% of the survey population) stated that they use learning software or applications to support their teaching practice. Only 2 out of 131 teachers stated that they do not use any learning applications in their practice. The most popular applications were mostly school management applications or educational repositories that are either designed or adapted to the context of Estonian K12 education (Appendix A.2, Table 7).

Method of the Study

The analytical process of the survey responses consisted of two parts. In the first part, we analyzed the participants’ responses to the first and third sections of the survey to answer the first research question (RQ1). We used descriptive and correlation analysis to determine potential relations between teachers’ perceptions, attitudes, and familiarity regarding AI and their teaching experience or their experience with learning technologies.

In the second part, we analyzed participants’ responses to the second section of the survey to answer the second research question (RQ2). In particular, we created a coding scheme on superpowers, and two human raters applied the coding scheme to participants’ input. We validated coders’ reliability using Krippendorff’s alpha, Cohen’s kappa, and the Intraclass Correlation Coefficient (ICC) and refined the coding scheme and results. Then, we analyzed the codes with respect to their occurrences in participants’ input – that is, how often a code (or else, superpower) is mentioned in participants’ input – using Hierarchical Agglomerative Clustering (HC) to identify frequently referenced groups of codes. Additionally, we analyzed the codes using Latent Class Analysis (LCA) in order to identify underlying relationships and structures among codes and between participants’ responses - that is, to establish patterns of code co-occurrence. Then, we asked an educational technologist expert to qualitatively cluster the codes resulting from the coding scheme and we compared the results of the human clustering with the LCA outcome. To answer the second research question (RQ2), we triangulated the outcomes of the HC, LCA and the human clustering process.

The purpose of the triangulation was twofold:

  • to explore potential semantic or structural relationships between popular codes. For example, whether the codes that were frequently referenced by participants (as obtained by HC) were also occurring together(as obtained by LCA);

  • to contextualize and to enrich the interpretation of the results by synthesizing the findings of the computational and the human clustering aiming at studying the outcome from different standpoints (Cohen et al. 2013).

Results

In this section we present the results from the survey on: a) teachers’ familiarity with AI (RQ1), and b) teachers’ perceived challenges (RQ2).

Teachers’ Perceptions, Attitudes and Familiarity Regarding AI

To assess the teachers’ familiarity with the concept of AI, we asked them to rate on a 6-point scale their overall knowledge about AI from “I have never heard of it” to “I am an expert in AI”. The majority of the teachers who participated in the survey stated that they have either a limited (47%) or fair knowledge (35%) regarding AI. About 4% of the teachers responded that they never heard of AI before, and 8% that they are not sure what it is. On the contrary, 6% of the teachers stated that they know a lot about artificial intelligence (Table 1). No participant perceived themselves as AI experts. In order to validate their perceived knowledge, we provided to the teachers five statements related to AI and asked them to indicate which of these statements were true. Based on their responses’ correctness, we calculated a correctness ratio for each participant - that is, how many statements the participant marked correctly as “true” vs. the number of all statements. The majority of participants (57%) provided 60% correct answers. At the same time, we did not find a statistically significant correlation between the correctness ratio and the participant’s familiarity with AI. However, as seen in Table 1, the participants who rated their knowledge as having heard of the term before but not sure what it is, provided more correct answers than the ones who considered themselves as quite knowledgeable.

Table 1 Participants’ perceived knowledge with respect to AI and the average correctness rate (per group) on the mini-knowledge test
Full size table

Therefore, we cannot safely assume that participants’ perceptions regarding their AI-related knowledge can be based on evidence. We argue that this can be potentially a hindering factor when it comes to integrating AI systems in the classroom: teachers who feel confident about their AI-related knowledge end up struggling due to false perceptions, and teachers who do not perceive themselves as knowledgeable restrain from using AI systems although they do have the necessary skills.

Finally, we asked teachers to indicate whether they have ever used an AI-enhanced educational application. Most participants (44%) indicated that they do not know whether they have used AI-enhanced educational applications. On the contrary, 17% of the participants responded that they have never used an AI-enhanced educational application, and 40% responded that they have used AI-enhanced educational applications before. This is not surprising since the technologies behind educational applications - especially learning analytics and open educational repositories - are usually not communicated to the public (Ferguson et al. 2016; Chounta 2019). The rationale behind this question was two-fold. First, this could serve as an indication (or proxy) of teachers’ familiarity with AI. Second, we argue that transparency on this level may support the integration of a system or a tool in practice.

Our next aim was to gain insight into teachers’ perceptions regarding the use of AI in education. To do so, we asked the participants to list the positive and negative aspects of using AI in education using multiple-choice inputs. The results are depicted in Table 2. Additionally, we allowed them to provide their input.

Table 2 Participants’ perceptions regarding positive and negative aspects of AI use in education
Full size table

Concerning the positive aspects of AI use in education, participants stated - using the open-text answers - that AI could help them to be creative in their practice, to group the student population in terms of their knowledge state, and in organizing their learning materials in terms of difficulty levels.

Concerning the negative aspects of AI use in education, participants raised concerns with respect to the potential of hindering human communication, placing the human factor in the background, and the effectiveness of AI for tasks that require human intelligence, creativity, and empathy. For example, participants mentioned that “real” (that is, human to human) communication cannot be substituted by human-machine communication and that the students want to communicate with human tutors. Participants also stated that their work requires human resources and skills like flexibility, creativity, and responsiveness while they questioned the effectiveness of AI in these terms. In other words, the participants mainly questioned the ethical use of AI in education, and they were mostly concerned of AI as a replacement of the human.

Regarding the nature of work tasks that AI could support, most of the teachers responded that AI could help them with tasks such as grading homework (67%) and common administrative tasks, such as reporting (61%). Also, AI could provide support with planning the lessons either in terms of time (56%) or content (53%) and for monitoring students (53%). Teachers also stated that AI could offer support in terms of personalization of feedback and instruction as well as with finding and adapting appropriate learning material. Some teachers also mentioned that AI could help in the aggregation and analysis of students’ data coming from various sources.

Next, we explored whether the professional experience of the participants - in terms of years - had any effect on their survey responses.

Related research has indicated that teachers with deep knowledge of the subject matter - which potentially may point towards seniority and professional experience - maximized use of educational technologies (Hennessy et al. 2007). However, correlation analysis did not reveal any strong or statistically significant relationship between professional experience and teachers’ familiarity or perceptions regarding the use of AI in education.

Teachers’ Perceived Challenges in the Classroom

In order to uncover the challenges that K-12 Estonian teachers face in their practice and gain insight concerning their needs, we asked the participants to name, at most, three superpowers they wished to acquire in order to help them in their workplace. Overall, 111 participants provided appropriate input to this question (for example, input such as “I do not know” would qualify as inappropriate input). Two researchers – with an expertise in educational technologies – analyzed the participants’ input to identify prominent topics that were referenced as superpowers in teachers’ responses. Overall, 39 topics were identified. Then, they coded the participants’ input using the 39 topics as codes. In particular, the researchers were marking which codes from the 39 previously identified could be traced back in teachers’ responses. Each response could point to more than one – but at least one – code. To ensure inter-rater reliability, we calculated Krippendorff’s alpha and Cohen’s kappa for each one of the 39 codes that the two rates coded. Then, we eliminated the codes for which Krippendorff’s alpha was lower than 0.7. Inter-rater reliability was validated using the Intra-class Correlation Coefficient (ICC). From the 39 codes, 7 were eliminated due to Krippendorff’s alpha criterion (α < 0.7). Cohen’s Kappa and ICC confirmed Krippendorff’s alpha criterion. The aforementioned analysis is presented in Table 6 (Appendix A). The most referenced code in participants’ input was speed (referenced 35 times). The least referenced code in participants’ input was assertiveness and reflection (each referenced one time). The frequency of referencing for all the codes is presented in Fig. 1. Then, we performed Hierarchical Agglomerative clustering (HC) to identify clusters of superpowers based on their frequency of reference (Fig. 2).

Fig. 1
figure 1

The identified codes (superpowers) ordered from the least referenced to the most referenced in participants’ input

Full size image
Fig. 2
figure 2

The groups of codes resulting from the hierarchical clustering

Full size image

The results of the clustering suggest that codes relating to work efficiency and effectiveness – such as speed, cloning, personalization, and assessment – and to human aspects – such as empathy – are the ones most frequently referenced by the teachers. On the contrary, codes that refer to low-level, procedural skills – such as programming skills or error prevention or high-order concepts that are valued in learning contexts but could potentially interfere with systematic forms of teaching (Henriksen et al. 2016) and negatively impact student systematic assessments – such as creativity in teaching – are sparsely mentioned in teachers’ input.

To reveal potential semantic structures or to highlight underlying themes in participants’ needs, we further analyzed the codes for the remaining 32 codes using correlation analysis and latent class analysis, and we triangulated the findings. Correlation analysis revealed some correlations of medium strength between the codes that were identified in participants’ responses (Table 3). In this context, correlations may reveal common themes in participants’ needs. Our results suggest that one theme appearing in participants’ input is the need for support with the preparation of learning materials in terms of content (ρ = 0.42). Another theme is the need of personalization of learning that relates to providing feedback ρ = 0.37) and to monitoring students’ progress for finding student’s misconceptions (ρ = 0.41). The analysis also revealed potential relationships between the need for speeding up assessment processes (ρ = 0.33) and the relationship between teachers’ well being and their need for additional (or even, infinite) energy in the workplace (ρ = 0.32).

Table 3 Correlation analysis for the superpowers’ co-occurance in participants’ input
Full size table

Next, we wanted to identify underlying structures in participants’ feedback. In particular, we were interested in exploring whether the 32 codes identified from participants’ input could be grouped in clusters that would reflect Estonian K-12 teachers’ fundamental needs. To that end, we analyzed the human raters codes using Latent Class Analysis (LCA). In principle, LCA is used for identifying similar groups of response patterns, and it can be perceived as the equivalent of cluster analysis for categorical data (Linzer et al. 2011). We used the Elbow method to determine the appropriate number of clusters, which indicated six (6) clusters for this case. Then, we used latent class analysis to split the 32 codes into six groups based on the human ratings. The groups consisted of 2 to 10 codes. The results of the LCA are presented in Table 4.

Table 4 The groups of superpowers as obtained from the Latent Class Analysis
Full size table

To validate the results of the LCA clustering and to qualitative reflect on the findings, we asked a researcher with expertise in Educational Technologies to group the codes into six thematic categories. Before the task, we discussed each code and their interpretation from a practical perspective - that is, what potential issues they might indicate. We did not provide any information about the thematic groupings. According to the human rater, codes can be organized on a high level of abstraction in three categories. The first is related to the teacher taken in isolation. As such, it comprises what the teacher wishes to be or to have (for example, infinite energy), abilities (for example, mnemonic), and specific skills (for example, analytic skills). The second category identifies superpowers teachers would like to have concerning tasks and activities involving students. The third category is related to the workplace, and it chiefly concerns the management of time and the core business of teachers, that is, all the tasks and activities that regard the “lesson” as the central unit. The grouping of the codes as done by the human rater is presented in Table 5.

Table 5 The groups of superpowers as obtained from the human expert’s qualitative clustering
Full size table

To compare the LCA results and the human-made clustering, we used the Adjusted Mutual Information index (AMI). AMI measures the agreement between two data clusterings correcting the probability of the grouping being made by chance and taking into account imbalanced clusters of different sizes (Romano et al. 2016). The AMI values range between 0 and 1, where 1 signifies that the two clusterings are identical and 0 signifies that the clusterings could be expected due to chance. In our case, AMI was 0.6, indicating that the clustering as resulted from LCA is similar to the expert’s clustering. On the other hand, the AMI index when comparing the LCA and the HC results was 0.35. This suggests low similarity between the LCA and the HC clusters, and therefore not providing evidence of structural relationships between frequently occurring codes.

After completing the clustering process, we attempted to identify high-order themes that appear in the clusters. To do so, we juxtaposed the groups resulting from LCA and from the human-made clustering and qualitatively analyzed the codes that occurred within each group. When analyzing the codes, we took into account what was the target of each intervention. For example, the code “multitasking” signifies the ability to manage several tasks simultaneously, thus pointing towards efficiency when carrying out a task as the target. The human rater participated in this phase of the analysis in order to provide qualitative insights. From this qualitative analysis, we established six themes that span across the two groupings.

  • Effectiveness: The first theme related to the efficacy or effectiveness of teaching in terms of instruction. This theme is evident by codes such as personalization and finding misconceptions, and it points towards teachers’ need to address specific student’s needs. Teachers would like to acquire superpowers that would help them personalize instruction and content with respect to students’ individual needs, that would support them in assessing student’s knowledge state accurately, and that would allow them to do so fast. They emphasized that personalization and assessment are fundamental tasks for their work, but they also pointed out that these tasks are extremely time-consuming;

  • Efficiency: The second theme related to temporal aspects of teaching and time management, and it is evident by codes as time control and multitasking. This theme points towards teachers’ need for efficiency and suggests that time is a critical resource that affects teachers’ practice. Teachers pointed out that they usually struggle to manage their workload, and they expressed their wish to control time, clone, and multitask. Being able to monitor students - so as to be aware of their actions around the clock - and to communicate with students effectively - so that they maintain a common ground and a good work climate - are skills that can contribute to teachers being more efficient in their workplace;

  • Rapport: The third theme that was prevalent in the groupings related to personal relationships, understanding and rapport and it was evident by codes such as empathy and reading thoughts. This theme points towards teachers’ need to support their students with their learning goals and with their mental needs and well-being.

  • Course Planning: The fourth theme related to efficacy of teaching in terms of content preparation or course planning. This theme is evident by codes such as content planning and finding materials and it suggests the need of teachers to retrieve or prepare appropriate material to achieve the learning objectives. It became clear from teachers’ input that course preparation itself- in terms of the lesson or course planning - is a major challenge they face in the workplace. In this work’s context, perceived challenges involve the planning of a course, lesson or seminar in terms of both time and content. Additionally, with respect to content, teachers find it demanding to retrieve and prepare appropriate materials for their lessons;

  • Personal Attributes: The fifth theme related to the abilities - or qualities - a teacher should acquire in order to be successful professionally. Teachers pointed out several attributes that they perceived as helpful for their job. For example, they mentioned that having a good memory (mnemonic) is important in the classroom so as to retrieve information efficiently and to be able to address their students on a personal level. Teachers also mentioned that they have to adapt and have “infinite” energy to perform in different settings and contexts. They also mentioned that preventing errors was very important for them since it is easier and more efficient than having to go back and fix errors.

  • Personal Skills: Contrary to our expectations, many participants pointed out procedural skills that they perceived as necessary for their profession, such as programming, analytical, or foreign language skills. Additionally, teachers mentioned of skills, such as creativity and reflection, that are widely discussed as essential, must-have competencies for the 21st century. This finding may suggest that the skills set required for teaching is changing to accommodate the needs of an IT-enhanced, data-literate society, and it might serve as a suggestion for developing new standards for teaching qualifications.

    The last two themes, that is Personal Attributes and Personal Skills, stand out with respect to the rest of the themes because they are teacher-facing. This means that they specifically point towards the teachers themselves rather than the teacher in relation to the learning objective or the teacher in relation to the classroom.

Discussion

Perceptions and Expectations of Estonian K-12 Teachers Regarding the Use of Artificial Intelligence as a Means to Support Teaching (RQ1)

Most of the study participants were experienced teachers who use contemporary learning technologies to support their classroom practice, such as learning management systems to plan their courses and to communicate with their students and the students’ families and online educational repositories to retrieve learning materials.

Overall, Estonian K-12 teachers are familiar with digital learning technologies and with their integration in the classroom. The results of this survey suggested that the majority of the teachers perceived themselves as having limited (45% of participants) or basic knowledge regarding AI (35% of participants). On the other hand, when asked about AI’s fundamental concepts, the majority of teachers provided on average 60% correctly.

This finding partly confirms earlier reports on Estonian teachers’ familiarity with AI.Footnote 10 Nonetheless, the participants demonstrated positive attitudes toward the use of AI in education. They perceived it as a means to support them in retrieving learning materials, organizing their lessons in terms of content and scheduling, and in reviewing homework assignments. Even though a common argument against AI integration in the workplace is that it will take over from humans and lead to job losses (Luckin et al. 2016), this was a minor concern of Estonian K-12 teachers. On the contrary, participants pointed out their concerns about the effort it would require on their behalf to learn how to appropriately use AI technologies, and potential trust issues that could arise from the AI use. For example, some participants stated that they would not trust AI to carry out tasks without error. Most importantly, participants were critical about how AI could undermine human to human communication and hinder social aspects of learning.

Perceived Challenges in the Workplace for Estonian K-12 Teachers (RQ2)

We found that teachers emphasize being able to read the thoughts of their students, being able to clone themselves, and being able to successfully identify misconceptions, which align well with the findings of Holstein’s et al. (Holstein et al. 2017). Teachers would also like to monitor their students all around and assess the students’ knowledge state but also their emotional and motivational state. However, we also noted differences or additional challenges that the Estonian K-12 teachers face compared to what Holstein et al. found for teachers in North America. For example, one striking difference was that teachers pointed out the importance of their well-being in relation to their professional effectiveness. Another difference was that teachers stated they would value a superpower that allows them to be fluent in all foreign languages. Estonia is a small country with its own language. This limits the number of available learning resources that are distributed through online educational repositories. Estonian teachers either do not have the time or the language skills needed to adapt learning resources written in foreign languages. This was also reflected by the finding that it was hard for teachers to find learning resources, and to prepare learning materials. Finally, Estonian teachers prioritized empathy concerning the relationships they have with their students.

Based on these findings, we identified six high-order themes in terms of challenges teachers face: effectiveness, efficiency, rapport, course planning, personal attributes and personal skills. When designing AI-enhanced solutions to support teachers in addressing challenges, it is crucial to understand and consider the risks that AI could pose for education. A potential approach for supporting teachers’ effectiveness and efficiency would be to provide teachers with AI models that predict or automatically assess students’ performance, such as predictive models of dropouts or models of automatic formative assessment. Such models’ output depends on the existing data these models have been trained upon: a predictive model trained with an imbalanced dataset in terms - for example - gender, will be more effective for the majority gender and less effective for the rest. This algorithmic bias may support or promote unfair and discriminatory policies for certain groups based on gender, socio-economic class, ethnicity, and underrepresented student populations in general (Holstein et al. 2019). It is also important that automatic assessments are computed and delivered in a transparent and explainable way to both teachers and learners as a measure to safeguard fairness and build trust between AI and human stakeholders.

To support teachers in course planning, AI could be used to provide recommendations on learning materials and learning designs. One potential risk of this practice relates to the transparency of the recommender systems (Sinha and Swearingen 2002) - for example, why certain materials are recommended over others - and to the quality of the recommendations - for example, how accurate and appropriate these recommendations are. Another potential risk we see here relates to ethical and accountability aspects around teachers’ roles. One concern is that by over-prescribing automated solutions to teachers, we may run into the danger of undermining teachers’ role. One of the teachers’ responsibilities is to design and orchestrate a course that aims to deliver specified learning objectives. By relying on AI for course planning, teachers’ autonomy may be decreased, and their responsibility could be in question.

In terms of supporting rapport, teachers asked to know more about their students’ thoughts and attitudes, for example, to read students’ minds. An AI system that provides teachers with real-time assessments or indications of students’ cognitive and affective state could be a step towards this direction. However, this may entail privacy and ethical risks concerning students’ data that would need to be collected to deliver these assessments, how data are stored, for what purpose they are used, and to what extent.

Finally, teachers stated that certain challenges they face are due to lack of certain skills (such as foreign language skills or programming) or abilities (such as good memory). We envision that AI can support teachers by offering, for example, language-translation services for learning materials. Specific-purpose AI applications offered as plugins to learning management environments could support teachers with enhancing or scaffolding professional abilities. As with course planning, we see here a risk in terms of diminishing the role of teachers in the classroom and undermining social aspects of learning. Additionally, promoting teachers’ dependency on AI can entail risks in terms of accountability in cases the outcomes of AI are incorrect, inappropriate, or harmful.

Theoretical and Practical Implications Towards FATE

The results of this work aligned with previous research (Holstein et al. 2017) to a great extent regarding the teachers’ perceived challenges in the workplace. This coincidence of findings is important because the two studied cases – that is, the case of Estonian K-12 teachers and the case of North American K-12 teachers – are different in terms of the school environment, organization of work, teaching objectives, technological support, and socio-cultural context. For example, in Holstein’s work, the educational technology was an Intelligent Tutoring System that applies mastery learning (that is, the student practices certain skills until mastering them). In Estonian K-12 education, the main learning technologies are learning management systems and online learning repositories that facilitate courses designed with socio-constructivist approaches as a basis (that is students learn by interacting with learning materials within a social arena, while working with peers and being guided by the teachers). Thus, one implication of this work is that we identified teachers’ perceived challenges common for both settings (such as reading thoughts, and finding misconceptions). This may suggest that those challenges are the ones that relate to the core of the teaching activity - in the sense of transferring knowledge and mentoring - and are not heavily dependent on the context. Additionally, we identified challenges that were different (such as planning of materials, mnemonic, and assessment), which could be attributed either to the pedagogical approach or the learning technology. Furthermore, this work revealed additional challenges the teachers in Estonia face (for example, foreign language skills) - that can potentially be attributed to social and cultural factors - such as the potential limitations the native language may impose on the adaptation of popular learning technologies.

Another implication of this work is that we need to take into account teachers’ professional development. When asked for “superpowers” that could help them in their job, teachers responded by listing either procedural skills (such as programming) or high-order skills (such as creativity and communication). According to the Teaching and Learning International Survey (TALIS), most teachers are willing to have better skills in using ICT for teaching (OECD 2019). In addition, most of them often participate in professional development activities. This indicates the need to re-think and shape up the skillset teachers should acquire in the technologically-rich classrooms we strive to build.

Reflecting on the ongoing discussion about fairness, accountability, transparency, and ethics, we argue that our findings align with the ethical principles for designing AI systems for education proposed by Aiken and Epstein (2000).

Aiken and Epstein derived the ten principles for AIED systems from two philosophical meta-principles (as published in Aiken and Epstein (2000)):

  • “AIED technology should not diminish the student along any of the fundamental dimensions of human being”;

  • “AIED technology should augment the student along at least one of the fundamental dimensions of human being”.

Both meta-principles focus on the student and the fundamental dimensions of human being without explicitly addressing the role of the teacher in the system student - AIED technology. Thus, we envision that our work can contribute towards validating these principles from the teachers’ perspectives for supporting the ethical and successful integration of AI systems in the classroom.

Teachers pointed out the importance of the social aspect of learning reflecting the guideline that an AI system should “encourage collaborative learning and the building of healthy human interactions” (guideline 2). Teachers emphasized that empathy is an important ingredient of a well-functioning classroom, which demonstrates the need to design systems that “respect differences in cultural values” (guideline 8) and “accommodate diversity” (guideline 9).

Teachers’ responses suggest that Aiken’s and Epstein’s principles can be applied to accommodate teachers’ needs. For example, teachers expressed their need to carry out assessment faster and fairer, to closely monitor students, to design and provide appropriate, personalized, timely feedback. This relates to teachers’ efficiency and well being, pointing towards the need to “avoid information overload” (guideline 4) and to “consider ergonomic features” when designing (and integrating) AI systems for the classroom”(guideline 6). Teachers’ input states the need for fairness and appropriateness of instruction, feedback and assessment thus referencing the guideline for systems that “support development of positive character traits” (guideline 3) and that “encourage and do not demoralize the users” (guideline 1). Teachers also voiced their need for support in their own professional development referencing skills that they would like to have in order to perform in technology-rich, “clever” environments. This indicates that by integrating an AI system in the classroom, we also “give teachers new and creative roles that might not have been possible before the use of technology” (guideline 7).For these new creative roles teachers should be equipped with new skill sets. At the same time, we should consider to “avoid glorifying the use of computer systems thereby diminishing the human role and the human potential for learning and growth” (guideline 10).

Finally, it was evident that even though AI is a well-discussed topic, there is the need for further contextualized and conceptualized discussions with stakeholders (Verbert et al. 2020) if we aim to develop AI-enhanced tools that will find their way in the classroom (Ferguson et al. 2016; Chounta 2019). As the findings suggest, it is crucial to communicate to the stakeholder the purpose, the expected benefit and the potential pitfalls of the new technology in order to support its integration. Most importantly, it is important to communicate how the technology addresses the stakeholders’ needs and does not obstruct or hinder what the stakeholder perceives as important.

Conclusion

In this work, we presented the results of a survey that was carried out among Estonian K-12 teachers. The survey aimed to explore teachers’ knowledge, perceptions and attitudes towards AI and the challenges they face in their practice in the context of FATE. We carried out this research keeping in mind that Estonia is a country that actively promotes digitalization in all aspects of public life, and supports explicitly technological innovation in education. Nonetheless, earlier reports had pointed out that Estonian K-12 teachers do not fully realize the potential or usefulness of AI in education even though they have some limited knowledge about AI. Furthermore, they do not use it in their school, or they have no information about their potential use. These earlier findings were partially confirmed in our work. However, it was also shown that Estonian K-12 teachers were positive in using AI in education, especially when it comes to supporting administrative tasks or retrieving and adapting learning materials. Finally, we contextualized our findings towards validating Aiken’s and Epstein’s principles for designing AI systems (Aiken and Epstein 2000) from teachers’ perspective and we provided insights about the theoretical and practical implications of using AI to address teachers’ challenges in terms of FATE.

One limitation of our work is the small number of participants (140 teachers instead of almost 16000 teachers overall employed in Estonia). Another limitation is the choice of instrument (survey). We limited the number of topics addressed in this survey to keep it short and attractive to respondents. Thus, we did not collect information that could be valuable (for example, we did not consider teachers’ subject specialization). Finally, even though Estonia is a relatively small country in terms of both land area and population, it is diverse in terms of population characteristics concerning geography. In this survey, we did not take geographical location into account. Of course, one would expect that teachers in rural areas might not be so welcoming to cutting-edge technologies and potentially would have more fundamental challenges to address rather than educational technologies. To that end, we plan to extend this work in two directions: first, we aim to collaborate with governmental organizations and other education stakeholders (such as numerous schools) to expand our outreach. Second, we plan to carry out a series of interviews and design workshops with K-12 teachers. During these workshops, we will present AI-enhanced technologies to teachers and walk them through the process of integrating them in their classrooms while we emphasize aspects of FATE as an ingredient of meaningful integration. In this context, we have established a collaboration with a general education school where we work along with teachers towards developing multi-purpose, AI-enhanced ITSs.

As Bridle (2018) warned, there is a danger in the “mindless implementation of new technologies that uncritically ingest yesterday’s mistake”. Envisioning AI as the solution to all educational problems might have detrimental effects, especially in terms of fair and ethical AI. This way of thinking sets us apart from AI itself – which is still a formidable resource available in the teacher’s toolbox – and the responsibility its use entails. As Bridle remarked, establishing cooperation between humans and technology - including AI - may turn out to be a way more powerful strategy than the blind reliance on computation alone. The articulation of the different possibilities and their uncertainties that such cooperation entails is the main task of the present educational technology.