1 Introduction

Beyond the phases of the “digital divide”, research has shown that there is still a digital skills gap among students that requires attention (e.g. Goldhammer et al. 2016; Kaarakainen et al. 2017). While students are expected to possess digital skills (Leahy and Dolan 2010), research has indicated that on reaching university, many students do not have the digital skills needed (Verhoeven et al. 2016). Unsurprisingly, one predictor for stronger digital skills at university was having attended an information technology (IT) course in secondary school (De Wit et al. 2012; Verhoeven et al. 2016).

Research has also indicated that apart from their level of digital skills, adolescents and adults were not homogeneous groups with regard to how they engage with digital technologies. Based on a comparison of low and high performers in secondary school, researchers have proposed that low performers spend more time using digital technologies than high performers, but for non-school-related purposes (Hietajärvi et al. 2019), and average and low-performing students tended to use digital technologies to escape a class when they found it to be boring (Bergdahl et al. 2019). After adolescence, individuals with both low and high levels of education were identified as users of digital technologies and the Internet, although only highly educated people used them in ways that further benefitted their future by expanding their knowledge and skills (e.g. Dijk 2005). Hence, inequality in students’ digital skills and ways to engage when using technologies may reinforce both educational and future socioeconomic differences, and while this is only one of the challenges schools are obliged to tackle, it is perceived as rather urgent.

As education uses increasing numbers of digital technologies, students are directed to engage in learning mediated in this way. However, Luu and Freeman (2011) have noted that little is known about students’ digital skills and their relation to engagement and learning outcomes. Research has begun exploring the multi-dimensionality of engagement when learning with technologies (e.g. Halverson 2016; Henrie et al. 2018), and has concluded that engagement in TEL differs from engagement in the traditional, analogue classroom, but has also noted that engagement in TEL is still “ill-conceived” (Halverson 2016). To our knowledge, this study is unique in its aim of further exploring the differences between general engagement and engagement in TEL, and of exploring how the digital skills of upper secondary school students relate to their engagement and disengagement in TEL. More specifically, this study explores the relationship between students’ digital skills and the multidimensional facets of engagement and disengagement when learning with technologies. To do this, we used the Learner-Engagement-Technology instrument (LET) that was developed to capture engagement and disengagement in TEL (Bergdahl et al. 2019).

2 Background

At the beginning of the century, Prensky (e.g. Prensky 2001) coined the expression “digital natives”, with which he launched the idea that individuals born into a technology-rich environment would be more “tech savvy” than earlier generations. Currently, the idea of digital natives is receiving declining interest (Judd 2018), partly because research has not been able to support the generalisation of students’ digital skills (Correa 2016; Verhoeven et al. 2016). Instead, researchers have pointed out that while students may appear to be digitally literate, they might not have the skills required in an educational context (Aesaert et al. 2017; Johnson et al. 2016; Ng 2012).

Indeed, digitalisation mediates active participation in society at large, in private life and in education. Access to technology and digital skills is critical in gaining access to key areas in school and social life (UNESCO 2017). While many terms are used to reflect the level of digital skills (i.e. IT- or ICT-skills, computer- or digital-, −competence or -literacy, etc.). This study uses the term digital skills along with the definition proposed by UNESCO: “The term ‘digital skills’ refers to a range of different abilities, many of which are not only ‘skills’ per se, but a combination of behaviours, expertise, know-how, work habits, character traits, dispositions and critical understandings” (ibid. p. 4).

Recent studies have reported that today’s students do not view digital technologies or the Internet as something related to gender; instead, they expect to be able to use these technologies for all purposes, including learning (Henderson et al. 2017). However, the skills that student use informally in terms of digital technologies differ from those required for learning (Aesaert et al. 2017; Johnson et al. 2016; Ng 2012). While some research has suggested that not all students have the digital skills needed to manage their education effectively (Samuelsson 2014) others have pointed out that there is either no significant relationship (Claro et al. 2012) or a negative correlation (Hatlevik et al. 2015) between digital skills and computer use in school. This indicates that most students learn and develop their digital skills elsewhere, which in turn further supports the notion of digital skills being developed for other purposes than learning. After conducting a study of secondary school students’ digital skills, Verhoeven et al. (2016) concluded that students believe their digital skills are mainly the result of self-study, and when needing help, they prioritise turning to peers, then to parents, and only as a last resort do they ask their teachers (ibid.). Students have reported that teachers do not offer adequate support in using digital technologies (Heerwegh et al. 2016). While informal self-study can reflect an extensive use of various digital technologies for learning (Nouri 2018), several recent studies have concluded that if teachers do not support students’ use of digital technologies for learning, the students may use these tools guided by their own initiative, which has been shown to be less beneficial, or even detrimental, to learning (Aesaert et al. 2017; Bergdahl et al. 2018a; Goldhammer et al. 2016; Hatlevik et al. 2015; Hietajärvi et al. 2019).

In the 2016 Horizon report, Johnson et al. (2016) proposed that however defined, digital skills remain a challenge for higher education. There are several projects that aim to develop students’ digital skills. Research suggests that strong learning networks may support students’ learning, either as students become publishers of their learning (e.g. via learning platforms, Wikispaces, online workspaces etc.) and use these technologies to mediate communication with their peers and teachers to reconstruct the topic and reconceptualise their understanding of a given subject matter (Bergdahl et al. 2018b; Laurillard 2013; Sancho Gil and Padilla Petry 2016), or by extending these digital technologies to form more advanced combinations of virtual environments and digital technologies (e.g. video, audio, broadcast, survey and screencasts) (Uzunboylu et al. 2011; Nouri 2018). Moreover, researchers have pointed out that combining digital technologies (as described above) not only promotes learning of the subject matter, but also students’ digital skills (Sancho Gil and Padilla Petry, 2016; Nouri 2018).

2.1 Engagement and disengagement in technology-enhanced learning

Initially, it was thought that only at-risk students would benefit from interventions aiming to increase their engagement. Today, engagement is a key concept in education, as it is critical for all students (e.g. Fredricks et al. 2004; Skinner and Pitzer 2012); without engagement, there will be no learning. Highly engaged students receive better grades, and students with low engagement run the risk of lower grades and increased absenteeism, and might eventually drop out of school (Finn and Zimmer 2012). Student engagement is traditionally presented as a multidimensional construct with three related and mutually supportive yet distinct dimensions consisting of a behavioural, an emotional and a cognitive aspect (Christenson and Reschly 2012; Fredricks et al. 2004; Skinner et al. 2009). The behavioural engagement refers to students’ active participation, involvement and persistence in a learning activity; the emotional aspect reflects positive emotions and attitudes in a learning activity, and the cognitive engagement refers to concentration and efforts to master a subject.

It has been suggested that engagement is the observable manifestation of motivation (Reeve 2013), although some researchers have pointed out that the behavioural and cognitive dimensions are more easily observed than other dimensions (Wang et al. 2016). Moreover, engagement and disengagement can be conceptualised as parallel but distinctly separate multifaceted constructs with behavioural, cognitive, emotional and social dimensions (Bergdahl et al. 2019; Salmela-Aro et al. 2017; Wang et al. 2017).

Research has shown that people in their everyday lives are closely intertwined with the technical tools they use, meaning that humans and tools work in seamless integration (Säljö 2014). However, when the context changes, human interaction changes, and context affects engagement (Wang and Hofkens 2019). Researchers have pointed out that student engagement in the traditional classroom may differ from student engagement in a TEL setting (Bergdahl et al. 2019; Halverson 2016; Ma et al. 2018). Merely logging in, or being present, does not equal engagement in learning (Finn and Cox 2008). Hence, disengagement can be conceptualised in other ways than as the mere absence of engagement (Salmela-Aro et al. 2017; Wang et al. 2017), and may for example include absenteeism (Balfanz et al. 2007; Tafelski et al. 2017). Research has proposed that disengagement may be related to different kinds of triggering situations, such as notifications causing a distraction, poor design of learning activities, or too few technologies allocated to too many students (Bergdahl et al. 2018a). Salmela-Aro et al. (2016) reported that students could experience social pressure to be online, which may cause stress, exhaustion and feelings of inadequacy. Thus, exploring disengagement in TEL requires more than merely accepting disengagement as being at the lower end of engagement, since it brings a complexity to the learning situation.

Following Fredricks et al. (2004) and Wang et al. (2017), Bergdahl et al. (2019) developed an instrument with the aim of reflecting engagement and disengagement in TEL. These authors extended the operationalisation of student engagement and disengagement to a TEL setting. Echoing the views of Wang et al. (2017), Bergdahl et al. (2019) proposed that student engagement and disengagement are approached as two separate but related constructs, each with a behavioural, emotional, cognitive and social dimension, which may, for example, be identified as follows:

  • Behavioural engagement: This can be observed in the use of digital technologies to support and manage learning, or unauthorised use of technologies, and/or passivity due to limited access to (functional) technology.

  • Cognitive engagement: This may be noted in a student who concentrates easily when working with technology and takes the initiative to use such technology, or who becomes distracted by notifications and/or overwhelmed by information overflow.

  • Emotional engagement: This is reflected by a student who relies on technologies for school work, and has an emotional desire, satisfaction and eagerness to use digital technologies or who uses technologies to escape feelings of boredom, is emotionally drawn to the tool, feels frustration when using digital technologies for learning and/or resists the use of digital technologies.

  • Social engagement: This includes students’ preferences for technology-mediated participation and communication with teachers and peers or student’s tendencies to experience group work with technologies as upsetting or dispiriting, individual work with digital technologies as irrelevant, and does not wish to be left to manage tools without support.

Graham (2006) defines blended learning as a combination of traditional (face-to-face) learning and learning that make use of digital technologies as enablers, enhancer or transformers of the teaching and learning experience. According to Graham, blended learning can be viewed in a continuum reflecting how digital technologies are implemented. In the low-end the digital technologies “enhance” learning without changing the (traditional) pedagogical approach (ibid.). In this kind of setting a learning management system (LMS) is commonly used as a repository for lecture notes or collections of online resources. In the high-end, blended learning is transformational, and disrupts the spatio-temporal limitations of a traditional classroom. In such setting, students might employ a combination of digital technologies which allow all students to simultaneously engage in multiple synchronous and asynchronous multi-modal communications with content, teacher, peers, as well as use responsive or interactive technologies, from the classroom, or elsewhere (e.g. Punie and Redecker 2017). Although there are differences in the complexity of blended learning orchestrations (Bergdahl et al. 2018a), research has indicated that students in blended learning classrooms perform better (Lovett et al. 2008) and perceive learning as more effective than students in traditional (face-to- face) learning settings. Thus, blended learning orchestrations may expand the ways students engage and interact to learn, for example by facilitating active learning through increased opportunities for interaction which in turn excel student engagement (McGee and Reis 2012).

Due to the growing use of online and blended modalities, and the ever-increasing use of digital tools by students, it has been suggested that engagement is particularly relevant to the field of TEL (Henrie 2016). Some research, (e.g. Nouri 2018) have showed that university students engage in learning in a mobile, flexible and multimodal way by using the affordances of different technologies and by employing digital skills related to multimodal literacy. Thus, the possession of certain digital skills creates suitable conditions for productive engagement with learning technologies. However, research focusing on the relation between upper-secondary school students’ digital skills and their engagement in TEL has so far been overlooked.

2.2 The research problem and hypotheses

While there is a consensus today that digital skills are of immense importance in gaining access to different parts of our society (Hatlevik and Christophersen 2013; Van Dijk 2017; Zylka et al. 2015), research exploring the extent to which digital skills affect our learning abilities is inconsistent. Katz and Macklin (2007) claim there is no relationship between digital skills and academic performance, and others (e.g. Tadesse et al. 2018) have pointed out that digital skills are related to school outcomes, and proposed that approaches to digital skills must survey cognitive factors related to students, since “measuring one without the other may ignore significant interactions between the two, potentially distorting the results” (ibid. p. 98). Against this background, we have formulated three hypotheses. We first explore how general student engagement translates into engagement in TEL for upper secondary school students, as follows:

  • H1 We hypothesise that students reporting high levels of general engagement in learning are more likely than students reporting low levels of engagement in learning, to experience higher levels of engagement when learning with technologies.

Moos and Azevedo (2009) put forward that approaching students' reported digital skills is key when conducting research in TEL. Other researchers have found that students' reported digital skills were moderately to highly associated with their actual digital skills (Aesaert et al. 2017; Gebhardt et al. 2014) and related to their engagement (Verhoeven et al. 2016). For example, Verhoeven et al. (2016) drew a parallel between students’ academic self-perception and digital skills, and found that a high level of academic self-perception was related to digital skills in terms of word processing, calculations, spreadsheets, and gathering online information. However, according to Verhoeven (ibid.), vocational students may possess more advanced digital skills. To explore the relationship between digital skills and student engagement and disengagement in TEL in secondary school, we formulate the following hypothesis:

  • H2 We hypothesise that students' reported level of digital skills, (high or low), is related to how they engage and disengage in learning.

While the academic engagement of students is related to their grades and retention rates (Eccles and Wang 2012; Fredricks et al. 2004), studies have shown that disengaged students fall behind, and that there is a need to identify the early signs of disengagement in order to address and prevent a downward spiral (Balfanz et al. 2007). Although researchers have proposed that engagement in learning with technologies is manifested differently than engagement in an analogue setting (Bergdahl, et al. 2019; Halverson 2016; Ma et al. 2018), some have suggested that earlier positive digital experiences, together with a disposition to learn, are related to the students’ level of digital skills and by extension, engagement when learning with technologies (Kim et al. 2018). We build further on these theories of engagement and disengagement, and hypothesise that similar to general engagement in learning, engagement in TEL is also related to the students’ school outcomes. We explore the relationship between engagement and disengagement in TEL and school outcomes, as follows:

  • H3 We hypothesise that student engagement and disengagement in TEL can predict student grades.

3 Method

We distributed the LET-instrument to measure engagement and disengagement in TEL (Bergdahl et al. 2019) and to assess the extent to which students perceived their digital skills to be adequate for learning in school. The responses were analysed using quantitative methods, including descriptive statistics, bivariate correlations, an independent t-test, and stepwise linear regression analysis (Field 2018).

3.1 Context and participants

3.1.1 Selection

There are seven national programmes that qualify students for higher education in Stockholm. With the aim of determining the relation between digital skills and engagement in TEL for upper secondary school students, and which they bring with them into higher education, we employed a purposive sampling technique (Bryman 2016). We approached the largest of the national programmes that qualify students for higher education (Skolverket, The Swedish National Agency for Education 2018). We then approached all of the schools that use TEL by selecting those schools that participated in a digitalisation project. We later deselected the technology programmes, as these students may not be representative in terms of digital skills, and hence might introduce a bias (Verhoeven et al. (2016), made a similar observation). We accepted classes from all programmes with the aim of preparing the students for higher education; (the humanities, the natural sciences, the social sciences and the economics programme. All students were enrolled in an upper secondary school governed by Stockholm City. From the questionnaires (n = 410), 38% of the respondents were male and 62% female (see Table 1).

Table 1 Demographic data of respondents
Full size table

3.2 Ethics

After obtaining the principal’s approval, a research connection was established at each school (via administrative personnel or a teacher). The person administrating the questionnaire was emailed a short introduction to the research study to ensure that all students were given the same initial information and verbal information about informed consent (Appendix 1). To protect respondents from third-party data access, the university’s Survey and Report application was used. When students entered the first page of the questionnaire, a detailed description of the study was offered, and informed consent was requested (Appendix 2). Students were informed that all participation should be of their own free will, and that they could opt out or withdraw their data at any time with no questions asked. Students could not proceed with the questionnaire unless informed consent was given. When questionnaires had been collected, students’ names were pseudonymised, and names and matching pseudonyms (codes) were stored in a separate codebook. The schools were asked to provide previous grades for respondents who had completed the questionnaire. All data were managed in pseudonymised form at all times.

3.3 Measures

3.3.1 Engagement

In addition to examining engagement in TEL, we also asked students to report on their general engagement in learning, using a six-point Likert scale ranging from “never engaged” to “always very engaged”. The grouping variables for the independent samples t-test were coded as 1 (“never engaged” to “somewhat engaged”) and 2 (“most often engaged” to “always highly engaged”).

3.3.2 Grades

Sweden uses an A–F grading system in which A is the highest grade and F is the lowest grade (A–E are different grades of pass and F = fail). The collected grades were converted into a six-point scale (A = 5, B = 4, C = 3, D = 2, E = 1, F or no grade = 0). Using the students’ most recent semester grades and a six-point scale, we calculated the mean and separated students into two groups accordingly: below the mean, at the mean or above the mean (M = 3.4).

3.3.3 Students’ digital skills

We asked the students to estimate their digital skills (the competence required to handle digital technologies at school, e.g., computer/mobile phone /tablet). Using a five-point Likert scale ranging from “none at all” to “very high”, we asked the students to rate their digital skills in relation to the digital tools used in their learning. The grouping variables for the independet samples t-test were coded as 1 (“no digital skills at all”, to “average digital skills”) and 2 (“good digital skills” to “high digital skills”).

3.4 Data collection

We distributed the questionnaire to 872 students in 11 upper secondary schools in Stockholm, Sweden. A total of 552 students filled in the questionnaire. The schools then provided 410 grades, which were matched with the corresponding questionnaire and thus constitute the final sample of this study.

3.5 Data analysis

Data screening was carried out to check for the standard normal distribution (SND) of the participants across the studied variables, using SPSS version 25. The rate of missing data was below 1%. Outliers were identified using mean substitution, reflecting the report within SND (Field 2018). Variables were examined for skewness and kurtosis. We used descriptive statistics to explore the data, and an independent sample t-test to test H1–H3. We also conducted Spearman’s correlation test to explore the associations between the variables and a stepwise regression analysis to identify the engagement and disengagement predictors of performance, measured using students’ final grades.

4 Results

To explore engagement, disengagement, the relation between them and the extent to which upper secondary school students perceive their digital skills to meet their educational requirements, we proposed three hypotheses:

4.1 Hypothesis 1 (H1)

We hypothesised that students reporting high levels of general engagement in learning are more likely than students reporting low levels of engagement in learning, to experience higher levels of engagement when learning with technologies. To test this hypothesis, we performed an independent samples t-test comparing students’ general engagement in learning, using high and low levels of engagement as a grouping variable and a combined variable of engagement in TEL.

Table 2 reveals that the students reporting high (M = 4.23, SD = 0.64) and low (M = 4.02, SD = 0.64) levels of general engagement in learning and TEL differ significantly (t(410) = 0.88, p < 0.05). This means that students who generally display high levels of engagement also display higher levels of engagement in TEL. Thus, we found this hypothesis to be supported.

Table 2 Comparison of high and low levels of general engagement in learning and engagement in TEL using a t-test
Full size table

4.2 Hypothesis 2 (H2)

We hypothesised that students' reported level of digital skills is related to how they engage or disengage when learning with technologies; that students with higher levels of digital skills engage more, and that students with low levels of digital skills disengage more than students with high and low levels of digital skills respectively. To test this hypothesis, we first used engagement indicators to explore if students with high levels of digital skills engage more in TEL than students with low levels. We then used disengagement indicators to explore if students with low levels of digital skills disengage more.

We had asked the students to report the extent to which they were competent in managing the learning technologies at school to support their learning. We conducted two independent sample t-tests to test our hypothesis, one using a dummy coded variable of digital skills, and variables reflecting engagement in TEL. We also conducted a bivariate correlation test to further identify the relationship between students’ digital skills and their engagement in TEL.

Using an independent sample t-test (see Table 3), we found a significant difference between students that reported low levels of digital skills (M = 4.00, SD = 0.66) and students that reported high levels (M = 4.26, SD = 0.61) in terms of their engagement in TEL (t(410) = 0.42, p < 0.01). Thus, the t-test supports our hypothesis that students with strong digital skills engage more in TEL. We conducted an additional t-test to study the hypothesis with separate variables (see Table 4).

Table 3 Comparison between digital skills and overall engagement in TEL using a t-test
Full size table
Table 4 Comparison between digital skills and separate indicators of engagement in TEL using a t-test
Full size table

Table 4 reveals that all but one variable displays a relation between digital skills and engagement in TEL. Students who reported higher digital skills than students with lower digital skills to greater extent:

  1. 1)

    took the initiative to use digital technologies,

  2. 2)

    concentrated easily when working with technologies;

  3. 3)

    experienced that switching between digital technologies increased their engagement,

  4. 4)

    experienced that technologies were used as cognitive enhancements,

  5. 5)

    created using technologies,

  6. 6)

    found asynchronous technology-mediated communication between themselves and the teacher sufficient and satisfying,

  7. 7)

    preferred digital feedback, and

  8. 8)

    used learning materials their teacher had published.

While most of the indicators were significantly related to high levels of digital skills, students with low levels of reported digital skills (M = 3.96, SD = 1.85) reported to a higher extent than students with higher digital skills (M = 3.72, SD = 1.81) that their absenteeism would decrease if they could attend class from elsewhere (t(410) = 0.96, p < 0.05). To further investigate the relation between students’ digital skills and engagement we performed bivariate correlation tests. Using a combined variable of student engagement in TEL, we found a significant correlation with reported digital skills (r = 0.22, p < 0.01).

Table 5 Correlation between digital skills and separate indicators of engagement in TEL
Full size table

Table 5 presents the correlation between digital skills and specific variables. Several significant (albeit weak) correlations between different engagement variables in TEL, that include the use of IT and digital skills, were identified. For example, higher digital skills were associated with engaging in digital creation (r = 0.26, p < 0.01), taking initiative with IT use (r = 0.23, p < 0.01), and satisfaction with teachers use of IT (r = 0.19, p < 0.01). It is noteworthy that students who expressed a general increase in engagement when using digital technologies also reported using digital technologies to support learning (r = 0.40, p < 0.01), that they took more initiative with technology (r = 0.38, p < 0.01), that they could concentrate more easily (r = 0.37, p < 0.01), and that they wanted the school to use more digital technologies (r = 0.49, p < 0.01).

Since digitalisation is the norm in many schools today, it is essential that schools not only focus on motivating their students but also ensure that students have the digital skills to succeed. Having hypothesised that students with high levels of digital skills engage more in TEL than students with low levels, our findings reveal that engagement in TEL is indeed related to students’ level of digital skills, thus supporting the second hypothesis.

Using the disengagement indicators, we then explored the second relationship in H2 (that students with low levels of digital skills disengaged more from TEL than students with high levels of digital skills) (Table 6).

Table 6 Correlation between digital skills and disengagement in TEL
Full size table

The results of the t-test reveal that there is no significant difference between students reporting low levels (M = 3.51, SD = 0.59) and high levels (M = 3.44, SD = 0.59) of digital skills in terms of disengagement in TEL (t(410) = 0.83, p > 0.05). To complement this analysis, we conducted a bivariate correlation test.

Table 7 reveals that variables reflecting disengagement in TEL correlated weakly with low and high levels of digital skills. Two items of disengagement were related to high levels of digital skills: (i) “We are more knowledgeable about IT than the teacher, and are therefore left to decide how technologies should be used for learning” (r = 0.21, p < 0.01), indicating that students with high levels of digital skills ask for the teacher’s knowledge and support when learning with technologies, but might not always receive it; (ii) “It upsets me that one student often does more than the rest in a group” (r = 0.13, p < 0.01), which indicates that when working with technologies, some students do the work on behalf of the others in a group, which affects engagement negatively for both groups. Thus, considering the results of the t-test and the fact that the correlations were weak, we H2 was only partly verified, as we did not find that low levels of digital skills are associated with disengagement in TEL.

Table 7 Correlation between digital skills and disengagement in TEL
Full size table

4.3 Hypothesis 3 (H3)

Finally, we hypothesised that student engagement and disengagement in TEL could predict student grades. We conducted a stepwise regression analysis with combined engagement and disengagement variables as independent factors. Before commencing the regression analysis, the homoscedasticity and normal distribution were checked to confirm that the regression analysis was appropriate.

The final model could explain 17% of the variance in the students’ final grades (F(4, 409) = 7.99, p < 0.001, R2 = 0.17, R2 Adjusted = 0.15). The following predictors were identified:

  1. 1.)

    My absenteeism would decrease if I could attend class from elsewhere (β = −0.19, p < 0.01),

  2. 2.)

    Working alone decreases my engagement (β = −0.15, p < 0.01),

  3. 3.)

    It upsets me that one student often does more than the rest in the group (β = 0.10, p < 0.01),

  4. 4.)

    The most capable student does the work on behalf of the group (β = 0.11, p < 0.05),

  5. 5.)

    I routinely search the Internet (β = 0.10, p < 0.05)

  6. 6.)

    I don’t want to read everything from a laptop (β = 0.14, p < 0.01)

  7. 7.)

    My teacher takes part within applications that I use (β = 0.12, p < 0.05),

  8. 8.)

    I use IT to support my learning (β = 0.19, p < 0.01),

  9. 9.)

    I often switch between playing games/browsing/social media/streaming YouTube (or similar) and learning in class (β = −0.10, p < 0.05),

  10. 10.)

    Autonomously switching between different technologies when doing schoolwork increases my engagement (β = −0.11, p < 0.05).

In light of the second hypothesis, we examined the structural relationships between student grades and student engagement and disengagement in TEL, and hypothesised that this engagement and disengagement could predict student grades. The best regression model could explain a total of 17% of the variance in the student’s final grades, and identified ten significant predictors (five engagement variables and five disengagement variables). Thus, H3 is supported, since variables reflecting both engagement and disengagement in TEL can predict students’ final grades.

5 Discussion

To expand the existing knowledge of student engagement and disengagement in TEL in relation to students’ digital skills and school performance, we tested three hypotheses:

5.1 Results discussion

5.1.1 Hypothesis 1

We hypothesised that students who report high levels of engagement in learning and students who report low levels of engagement in learning differ in terms of their engagement in TEL. We found that there are statistically significant differences between these students’ levels of engagement in TEL. This is in line with previous research, which proposes that student engagement in traditional (analogue) classrooms differs from their engagement in TEL (Bergdahl et al. 2019; Halverson 2016; Ma et al. 2018). The implications of this are that the conceptualisations of engagement need to be expanded to include engagement with technology and the dimension of disengagement, in order to gain a fuller insight into the complexity of engagement in TEL environments.

5.1.2 Hypothesis 2

We hypothesised that students with high levels of digital skills will engage more in TEL than students with low levels of digital skills, and that students with low levels disengage more in TEL than students with high levels (H2). While our analysis partly verified H2, we did not find it supported that low levels of digital skills and student disengagement were related. Previous research has shown inconsistent results regarding whether digital skills are related to academic achievement or not (e.g. Tadesse et al. 2018). Tadesse et al. (2018) argued that digital skills should not be approached in a vacuum, but be considered in conjunction with students’ cognition and interactions, since ignoring these aspects might distort the results (ibid.). We agree with Tadesse et al. (2018), and develop this further. Our findings show that the context in which digital skills are approached (in this case, we included behavioural, emotional, cognitive and social aspects of both student engagement and disengagement) is vital for a nuanced understanding, as our results demonstrate.

We found significant differences between engagement in TEL for students with high and low levels of digital skills. For example, our analysis shows that students with high levels of digital skills can and do take the initiative to use digital technologies; can and do concentrate easily when working with technologies; and can and do switch between digital technologies to increase their engagement. Students with higher digital skills expressed being satisfied when then teacher communication was entirely technology-mediated. However, when testing the hypothesis, we were surprised to find that there was no relation between low levels of digital skills and disengagement. In sum, H2 suggests that any student, regardless of their level of digital skills, will disengage from learning, but that only students with high levels of digital skills display a variety of pro-learning ways to engage in TEL.

5.1.3 Hypothesis 3

Finally, we hypothesised that student engagement and disengagement in TEL could predict student grades. The best regression model could explain a total of 17% of the variance in the student’s final grades, and identified ten significant predictors (five engagement variables and five disengagement variables). There was a relatively equal distribution of predictors of grades reflecting engagement and disengagement in TEL related to the social, behavioural, cognitive and emotional dimensions of student engagement. Analysing these results, we identified some previously unknown factors across the range of variables of engagement and disengagement that predict grades, and we base the following two models on these.

5.1.4 Factors of engagement in blended learning that predict grades

In exploring engagement, we identified five factors that predict grades: learning-oriented technology switching, technologies to support learning and social presence.

Interestingly, students who experience increased engagement via learning-oriented technology switching (i.e. switching between different technologies for a direct learning purpose) get higher grades (see Fig. 1). They seem to work toward a goal, and use technologies to support them on their way. Engaging in TEL was related to having high levels of digital skills. We found that students reporting higher levels of digital skills also used digital technologies to support their learning and found it easier to concentrate when using technologies than students with poorer digital skills. Students also reported that the possibility of using technologies to attend class would reduce their absenteeism. Possible explanations to why students expressed this desire may be a wish that their school starts to make use of digital technologies in new ways, or it may conceal a reluctance to attend school. However, it is important that these students expressed a willingness to participate in class using technologies to reduce their absence.

Fig. 1
figure 1

Model of engagement in TEL that predict grades

Full size image

Another challenge that arises when analogue learning environments become blended learning environments is that the teacher’s social presence is still required; this might be a challenge for schools to realise, since a presence in online environments may be new to some teachers (Bergdahl et al. 2018b). More research could be done to explore how a more flexible presence could become a reality without compromising learning standards, and to explore what is required in online learning environments for students to perceive that teachers are socially present.

Figure 1 displays variables of engagement that predict grades grouped into three categories: learning-oriented technology switching, technologies to support learning and social presence.

5.1.5 Factors of disengagement in blended learning that predict grades

When exploring disengagement, in particular, we identified five factors that predict grades: non-learning-oriented technology switching, lack of a digital balance and unsatisfactory learning interactions.

By comparing outcomes (as displayed in Figs. 1 and 2), we find that it is not the actual “switching between digital technologies” that is problematic, but rather how and with what intentions students switch between technologies. Students displaying learning-oriented technology switching had higher grades, while students who used technology switching to escape from learning (see Fig. 2) had lower grades. Hence, it is not the technologies themselves, and not the technology switching per se, that is good or bad for student learning outcomes, but the underlying motivations that guide and drive the behaviour. It is important to separate this non-learning-oriented technology switching from learning-oriented technology switching, in which students may have one cognitive focus and use several digital technologies to explore, develop, reconceptualise and communicate their ideas.

Fig. 2
figure 2

Model of disengagement in TEL that predict grades

Full size image

Interestingly, the regression analysis highlights that the experience of being directed to “read everything off the screen” is associated with reduced grades. This raises questions with regard to the directives for digitalisation in education. Has digital transformation been pushed into education without considering how students react to constant screen time? While more research is needed, this might be a first indicator that a lack of digital balance may have a negative effect on student grades. We propose that a digital balance is necessary to ensure that learning includes a certain proportion of thought-through variation with regards to the designs of learning other than repetitive student-to-screen interactions, in order to circumvent monotonous learning routines.

Figure 2 shows the variables of disengagement that predict grades, which can be grouped into three categories: non-learning-oriented technology switching, lack of a digital balance and unsatisfactory peer interactions.

It is noteworthy that the importance of functioning relationships (as displayed in Figs. 1 and 2) affect both engagement and disengagement and can predict students’ grades. When viewing the factors of disengagement, we find that these often relate to group work, and more specifically to how work is divided within collaborative groups. Our results indicate that while some students find they repeatedly do more than the others and feel upset, other students report that their engagement decreases if they are left to work alone.

Adjusting this may not necessarily require strong action, although awareness is needed for teachers to design their group work activities in a thought-through manner.

In summary, our results suggest that although all students face disengagement, it is mainly students with low levels of digital skills who do not display the kind of engagement that is needed to be successful when learning with digital technologies, and therefore may risk remaining longer in disengaged modes. While it may be easy to overlook the value of social interactions when students are learning with digital technologies, about half of the factors that predicted grades reflected social engagement or disengagement in TEL. The teacher’s social presence within applications was seen to be important for achieving higher grades, but when a student does the school work on behalf of the group, feels upset at having done more than the others or is directed to learn alone, these are all factors that predict lower grades.

5.2 Limitations

We are aware that our approach relies on students’ self-reported digital skills, and that there might be a possible discrepancy between actual and perceived digital skills. Nevertheless, as we approached student’s digital skills by specifically asking to what extent they feel they have the competence needed for their educational purposes, we were able to explore whether they perceive their competence to be sufficient for their learning. Scales measuring digital skills tend to be technically oriented (e.g. Heerwegh et al. 2016), meaning that they do not necessarily capture the needs of technology use in school and might not reflect whether students perceive their skills to be sufficient at their specific school. We also used self-reports on engagement, which are post-reports rather than reflecting engagement in the moment. Previous research (e.g. Bergdahl et al. 2018a) has reported that student efforts towards their learning activities are related to students’ self-reported levels of engagement. An element of caution and care when describing the statistically significant results is important. Considering the large sample and the small t-values, the statistical differences might be a result of the large sample size. Finally, to gain a more nuanced understanding of engagement and disengagement in TEL, future research may for example combine self-reports with indicators from system log-data (which reflect in-the-moment engagement) or qualitative approaches, and use longitudinal studies to measure changes and consistencies over time.

5.3 Future research

Student engagement and disengagement in TEL is an emerging research field. A further qualitative exploration of the results would be highly informative, and might raise questions such as: why do students with higher digital skills find it easier to concentrate in TEL? Would increased digital skills alleviate this concentration problem? In what situations do different student groups disengage, and why?

However, many aspects remain unexplored, and researchers with an interest in exploring engagement could approach both engagement and disengagement in a blended learning environment, for example by considering how the variables in an analogue setting are related to variables of engagement in TEL. Self-reports and interviews can be combined with in-the-moment data (e.g. system logs, sensors, eye trackers).

6 Conclusions

Many factors may influence engagement, and this study specifically considers student engagement and disengagement in TEL. Two theoretical assumptions are supported by our findings. Firstly, we extend previous research suggesting that engagement in TEL differs from engagement in analogue settings (Bergdahl et al. 2019; Halverson 2016; Ma et al. 2018). Secondly, we agree with earlier researchers (e.g. Wang et al. 2017) who have proposed that disengagement and engagement are separate constructs comprising behavioural, cognitive, emotional and social aspects. We develop this premise by arguing that research that does not complement their approach to engagement with a consideration of disengagement may fail to uncover disruptions, challenges and other negative factors influencing a learning situation when students learn with technologies.

Against this background, we found that high levels of digital skills were related to high levels of engagement in TEL. However, digital skills were not related to disengagement in TEL. This implies that schools wanting to improve the conditions for student engagement in TEL should invest in strengthening students’ digital skills. However, the results also suggest that the possession of digital skills does not inoculate against student disengagement. Instead, disengagement was seen across all levels of digital skills. Secondly, functioning relationships within group work with peers, the social presence of teachers in online environments and the ability to strike a digital balance are beneficial factors for student engagement and outcomes, and may therefore benefit students both within and outside of school, now and in the future.