Keywords

1 Introduction

There is increasing interest in the conceptualization of Self-Regulated learning (SRL) as a dynamic process which unfolds over the course of a learning activity. Mapping out SRL as a dynamic process is of interest, as it may provide opportunities for real time monitoring, evaluation and support of SRL in online learning environments. For example, there may be opportunities for intelligent tools which support tutors in real time as they scaffold learner self-regulation. However, there is limited research on frameworks, which are both theoretically grounded (Matcha et al. 2019), and sufficiently granular to investigate self-regulation in online learning environments.

To address this need, this paper proposes a framework to investigate the impact of tutor practices on learner self-regulation in online environments. Specifically, we focus on an online tutoring environment in which human tutors teach primary school learners (aged 10 years) on a one to one basis, using an interactive whiteboard and tools. We identify signifiers from natural language dialogue between tutor and learner, and explore the applicability of the framework in a pilot study. Our initial findings are not intended to be generalized to a population, but aim to build on research exploring how to operationalize SRL theoretical models in online environments (Hadwin et al. 2007).

2 Framework Development

The framework has been developed through a two stage, mixed methods approach. Firstly, an established theoretical model is adapted to apply to the online tutoring environment. In the second stage, we use a data driven approach to refine the framework.

The framework developed in this research required a granular, fluid model of SRL which could be applied to real world settings, and use online data. After the review of available theoretical models (Winne and Hadwin 1998; Pintrich 2000; Zimmerman 2000), the Winne and Hadwin (1998) model was identified as a suitable model (Fig. 1).

Fig. 1.
figure 1

Winne and Hadwin model (Winne and Hadwin 1998)

Full size image
Fig. 2.
figure 2

Average relative frequency of high-ranking tutor practices (purple) vs mid-ranking tutor practices (red). (Color figure online)

Full size image

The Winne and Hadwin model was selected as it is highly granular and suited to the analysis of fine-grained data that is generated from online environments. Further, the model synthesizes all the various components of SRL from the literature into a heuristic framework (Azevedo et al. 2010; Bannert et al. 2014).

The Winne and Hadwin model was adapted to fit our research purposes, namely to investigate the impact of tutor practices on SRL. The model recognizes tutor practices as an external condition impacting learner SRL. Our framework builds on this, and interprets each sub-component to identify tutor practices which may scaffold learner SRL. Following the adaptation of the model, online tutoring sessions were observed to identify fine grained actions. For our empirical work, we partnered with an industrial supplier named Third Space Learning (TSL), which delivers maths tutoring for primary school children aged 10 years old. Learners and tutors log into a shared online environment, and the learner works through a pre-designed online set of questions, with the guidance of a human tutor on an interactive whiteboard. The data available for analysis includes the online resources, natural language dialogue audio between tutor and student, logfile and whiteboard data. TSL sessions were filtered by topic, and the recordings of 50 randomly selected sessions were observed. Fine grained actions that could be observed from the data were mapped to the theoretical framework. This exercise illustrated that there were a number of tutor actions aimed at promoting certain types of engagement by learners which were not yet captured. For example, the theoretical framework did not distinguish between tutors who lectured versus prompting learners to construct meaning. The ‘Operations’ component was thus broadly defined to refer to the nature of tutor-learner engagement, characterized using the Chi & Wyle ICAP framework (Chi and Wylie 2014). The final framework is presented in Table 1.

Table 1. Framework for exploring tutor practices that influence SRL in online environments
Full size table

3 Pilot Study

We tested the applicability of the framework by applying it to data gathered from tutors manually classified as ‘high ranking’ and ‘mid ranking’. 180 tutors were ranked using student learning outcomes (30%), tutor evaluation test scores (30%), human evaluator scores (30%), and student qualitative ratings for tutors (10%). Tutors in the top ten percentile i.e. with a rank between 1–18, and tutors in the 45th to 55th percentile i.e. with a rank between 81 and 99 were randomly selected. 121 min of audio and whiteboard data from 21 sessions for the selected tutors was extracted and manually tagged using the framework. The data collected was allocated into time bins, as per Table 2:

Table 2. Allocation of session data into time bins
Full size table

We plotted the tutor practices against the time bins, with the size of the bubble being the average relative frequency of each behavior. Figure 2 shows the results for high ranking and mid ranking tutors. We found that high ranking tutors were more likely to demonstrate practices scaffolding open-ended constructive engagement, such as prompting self-explanation. High ranking tutors embedded monitoring throughout the session, while mid ranking tutors used this practice less regularly. We also found that tutor practices boosting learner mindset (e.g. specific praise) were more prominent amongst high ranking tutors, with a relative frequency of 0.42 for all high-ranking tutors, versus 0.28 for all mid ranking tutors.

The next stage of work will refine the framework to include non-audio traces of SRL behaviour (e.g. use of pointer). We will build the data sample to include low ranking tutors, prior to applying modelling and analytics such as process mining and decision trees. We will examine whether we can effectively use audio and non-audio traces to identify tutor practices scaffolding SRL, and the impact of contextual and macro factors on these practices. We will also analyse the implications of our work for operationalizing SRL in online tutoring environments, and the potential for building intelligent tools, which support human tutors in fostering learner SRL.