The relational, co-temporal, contemporaneous, and longitudinal dynamics of self-regulation for academic writing

Abstract

Writing in an academic context often requires students in higher education to acquire a new set of skills while familiarising themselves with the goals, objectives and requirements of the new learning environment. Students’ ability to continuously self-regulate their writing process, therefore, is seen as a determining factor in their learning success. In order to study students’ self-regulated learning (SRL) behaviour, research has increasingly been tapping into learning analytics (LA) methods in recent years, making use of multimodal trace data that can be obtained from students writing and working online. Nevertheless, little is still known about the ways students apply and govern SRL processes for academic writing online, and about how their SRL behaviour might change over time. To provide new perspectives on the use of LA approaches to examine SRL, this study applied a range of methods to investigate what they could tell us about the evolution of SRL tactics and strategies on a relational, co-temporal, contemporaneous and longitudinal level. The data originates from a case study in which a private Facebook group served as an online collaboration space in a first-year academic writing course for foreign language majors of English. The findings show that learners use a range of SRL tactics to manage their writing tasks and that different tactic can take up key positions in this process over time. Several shifts could be observed in students’ behaviour, from mainly addressing content-specific topics to more form-specific and social ones. Our results have also demonstrated that different methods can be used to study the relational, co-temporal, contemporaneous, and longitudinal dynamics of self-regulation in this regard, demonstrating the wealth of insights LA methods can bring to the table.

Introduction

There has been a growing interest in the application of data-driven methods to study self-regulated learning (SRL) in online learning spaces, including in online spaces dedicated to language learning. In the field of computer assisted language learning (CALL), for example, recent studies have been paying attention to the effect of self-regulation on the peer collaboration process in online writing activities (Wang, 2019), on how it influences the development of language learner autonomy (Peeters & Ludwig, 2017), and on the ways it can enhance productive skills like writing and speaking in a foreign language (Viberg et al., 2020a, b, c). Studies on these processes and dynamics are increasingly applying learning analytics (LA) methods to better understand student behaviour online in order to design new ways of assisting and supporting them in their learning trajectory (cf. Viberg et al., 2020a, b, c; Viberg et al., 2020a, b, c). The application of LA methods allows scholars to study multifaceted aspects of students’ SRL and can, therefore, offer new ways to optimise the learning contexts in which SRL develops.

This study focuses on analysing the relational and temporal dynamics of SRL tactics and strategies that can be observed when students develop one of their essential skills in higher education, i.e. their academic writing. Data for this research originated from a case study in which a group of first-year students (n = 124), majoring in English as a foreign language, were asked to use the social networking site (SNS) Facebook as a collaborative writing platform in an academic writing course. SNSs have been a prominent topic of interest in education since the dawn of Web 2.0 because they have offered both researchers and teachers new opportunities to design formal and informal learning opportunities while giving them access to a range of new learner data (Zourou, 2019). In the field of CALL, however, little research has been conducted on students’ collaborative writing processes in online spaces such as SNSs (Peeters, 2019; Peeters et al., 2020; Wang, 2019), let alone on how their behaviour in these spaces relates to SRL.

In order to better understand the continuous, dynamic nature of the ways in which students apply self-regulation (cf. Oxford, 2017), this study opted to apply a range of LA methods and investigate what they can tell us about the evolution of this process. In particular, the present study focuses on the multi-dimensional relational and temporal aspects of SRL. To be able to accurately describe these different aspects, this study first aims to redefine some of the concepts on temporality before analysing them through a range of LA measures. In doing so, we start to address the absence of a clear framework for the definition of ‘temporality’. We then examined which insights into the SRL process could be obtained from using different relational and temporal methods. In order to study how and why to use these different LA methods to reveal the relational and temporal dynamics of SRL, the following research questions have been posed:

• 1. Which learning analytics methods can help reveal the relational, co-temporal, contemporaneous, and longitudinal dynamics of students’ self-regulation tactics and strategies in an academic writing setting?

• 2. How can selected learning analytics methods be used to unveil the evolution of foreign language learners’ use of self-regulation to manage their academic writing process?

Background

Academic Writing and SRL

In the academic writing process, self-regulation is one of the key factors for students to be successful (Golombek et al., 2019; Zimmerman & Bandura, 1994). As stressed by various scholars, writing in this context presents specific challenges in terms of self-regulation (Bereiter & Scadamelia, 1987; Wason, 1980) because writing activities are frequently self-scheduled and conducted alone while they “require creative effort sustained over long periods with all too frequent stretches of barren results'' (Zimmerman & Bandura, 1994, p. 846). Self-regulation in the writing process refers to the “self-initiated thoughts, feelings, and actions that writers use to attain various literary goals, including their writing skills as well as enhancing the quality of the text they create'' (Zimmerman & Risemberg, 1997, p.76). Harris et al. (2002) further stress that, in order to successfully convey your message in an academic writing context, learners have to negotiate different aspects of the academic field, including its rules and mechanisms while, at the same time, not lose track of the overall organisation, form-factors and audiences they will encounter. To achieve this, and hence, to be able to effectively regulate the complex writing process, learners can make use of SRL tactics, strategies and metastrategies (Oxford, 2017; Zimmerman & Risemberg, 1997). In a recent study, Peeters et al. (2020) have demonstrated that foreign language learners can use a number of SRL tactics—i.e., specific activities students undertake to govern their learning, such as making a plan of study or reflecting on their performance—to guide their academic writing process in computer-supported collaborative learning (CSCL) settings, and found that there is a significant positive correlation between students’ application of those tactics and their learning outcomes.

Redefining the Temporal Dimensions of Self-regulation

Research has shown that students’ performance is largely contingent on their ability to efficiently implement, monitor and adjust to the different phases of self-regulation (i.e., plan, monitor and evaluate). SRL theory emphasises the central position of time in the learning process in this regard (Burnette et al., 2013; Zimmerman, 2002, 2008; Zimmerman & Schunk, 2011a, 2011b). As a process, self-regulation is an ordered sequence of events that unfolds and evolves over time (Reimann, 2009; Zimmerman, 2008). There is a wide agreement in the literature on the multifaceted nature of the SRL process, illustrated by several taxonomies for the different cycles, stages and components where students implement a sequence of tactics (activities) or strategies (approaches) to manage their learning (Panadero, 2017). Likewise, a growing corpus of LA literature has capitalised on the abundance of learners’ multimodal trace data to map the SRL process (Gašević et al., 2017; Malmberg et al., 2015; Saint et al., 2020). This article builds on such efforts and further attempts to map the SRL process using, among others, methods that describe temporal events. Temporality, however, has been proven difficult to properly delineate (cf. Saint et al., 2020). This paper, therefore, has aimed to redefine and describe some of the frequently used terminology from different fields. In redefining temporality, we do not attempt to unify or disentangle the overlap between different existing terms, but rather have a clear description of them when assessing the possibilities to map the SRL process using a range of LA methods.

Previous attempts to define and redefine temporality have stumbled on some difficulties (Reimann, 2009), among others the fact that: (1) high resolution trace data is not always compatible with existing theoretical models, (2) there are multiple levels of temporality which are not always addressed, and (3) there is an absence of standard methods that accurately describe temporal events. Addressing the first issue, collected data can be very rich and contain more information that SRL models traditionally account for. For instance, SRL phases are commonly studied over a longer period of time while some data sets can have information on what happens every second of the learning process. In other words, LA data collection methods can offer an abundance of details that are not always easy to align with existing theoretical frameworks. Therefore, attempts have been made to group such activities into meaningful groups of homogenous actions, i.e., sequences of related events (sessions), to make them fit the common time frames of SRL (e.g., Gašević et al., 2017; Peeters et al., 2020). Second, there are multiple levels of temporality that can be captured using LA methods, but which are not commonly described in SRL research. And third, temporal dimensions are often poorly defined in the LA literature and often overlap. For example, the term ‘temporal’ is regularly used to refer to different constructs, e.g., the description of different events in the learning process that all occur at the same time, events that are unfolding periodically and at different time intervals (e.g., tasks), or events that stretch out over the timespan of a full course. The term has even been used to describe the transition between such events. This kind of ambiguity calls for a more unified methodology for the description, investigation and rapport of the temporality of SRL. We, therefore, have built on previous work in the literature, and derived inspiration from other research fields that have studied temporal processes, namely psychology and temporal network studies. We propose a novel grouping for temporal events in this paper:

• 1. Individual traces: actions or activities: these are small events that occur by themselves, and can form the basic units for more complex, grouped events or sessions. Examples include clicking, viewing, reading, submitting or browsing. Individual events can also be referred to as the raw recording of learners’ actions.

• 2. Co-temporal events: different events that occur almost simultaneously or in a very rapid sequence. The time gap between events is considered absent or negligible. Examples include the process of instant messaging or multi-tasking (e.g. listening to instructions while writing down notes for the task). The term is usually used within the ENA literature to describe events that take place within a short window (mostly 3–4 actions) (Csanadi et al., 2018; Shaffer, 2019). Sometimes, the window is bigger (20 events or more). However, the bigger the window, the more difficult it gets to claim that these events co-occur because of their relatively remote temporal proximity. We rather prefer to call those events contemporaneous.

• 3. Contemporaneous events: different events that occur more than a few seconds apart (i.e., more than with co-temporal) but within a single session of one to a few minutes. Contemporaneous events may occur within a longer time frame if the events are strongly linked, like working on solving an exercise or discussing a task. The term is commonly used in psychological networks literature to study the temporal evolution of events and describe the events that occur in close proximity to one another (Epskamp et al., 2018).

• 4. Longitudinal events or a timeline of events: events that occur over a longer period of time, i.e., the duration of an entire assignment, training session or even a course. These can typically take days, weeks or months (Saqr et al., 2018, 2020a, b).

Different elements of a learning task are typically represented on different temporal levels. Individual events (occurring instantly) include elements of the task such as clicking on a link to open an assignment. A group of events that contains clicking, uploading and submitting, for instance, can be considered co-temporal as they occur in very close proximity to one another. However, opening the assignment, reading instructions, watching instruction videos and going through the learning materials in one single session can be classified as contemporaneous. A group of sessions or events that makes up a learning task, from start to finish, could be considered a timeline of events (longitudinal).

Bringing together LA methods for the analysis of SRL dimensions

In order to study the relational and temporal nature of SRL, a number of LA methods were selected. First, we used process mining (PM), a data mining method that is often used to visualise and evaluate learning processes. The ease with which to analyse the temporal dimension of the learning process has made PM popular among researchers. Among others, PM has been used to visualise the SRL process, compare strategies of subgroups of learners (high vs low achievers), find gaps and bottlenecks in the application of SRL strategies, and analyse the association between SRL strategies and personality types (Gašević et al., 2017; Malmberg et al., 2015).

Second, Sequence Mining (SM) is another approach that researchers commonly employ to study SRL. It has, for example, been used to investigate the order of student actions to find meaningful sequence clusters of distinct learning approaches and strategies (López-Pernas et al., 2021). Differential sequence mining has also been used to identify and compare subgroups of learners and pinpoint important differences in behaviour (Kinnebrew, 2013; Kinnebrew & Biswas, 2012). Sequence Pattern Mining (SPM), likewise, is a related method that is used to identify patterns of sequences (Kinnebrew, 2013; Kinnebrew & Biswas, 2012). Closely connected to this approach is social network analysis (SNA), which has also been used before to study the interrelationships between different SRL tactics used by students (Matcha et al., 2019; Peeters et al., 2020).

Third, temporal networks have been used in recent years to study the temporal and relational structure of students’ interactions (Saqr & Nouri, 2020; Saqr et al., 2019). A fundamental difference between temporal networks and all the aforementioned methods is their ability to quantitatively and visually aid in evaluating a relational process on a longitudinal level, i.e., over the period of a course, a term or an entire academic career. Such insights are not offered by the co-temporal methods such as PM that focus exclusively on transitions between events or activities (Saqr & Nouri, 2020).

Lastly, epistemic Network Analysis (ENA) is a method that uses quantitative ethnographic techniques for modelling the structure of interactions or discourse. ENA assumes that (1) modelling can identify meaningful structures and motifs within coded discourse, that (2) discourse is always structured in a meaningful way, and that (3) relationships between coded interactions are more important than the frequency of events (Csanadi et al., 2018; Shaffer, 2019).

In our case, students interact about their academic writing tasks by sharing and discussing their written text and the arguments they make. They reflect, discuss their plans, or become accustomed to socio-cultural or contextual norms. In this study, their interactions have been coded, co-temporally linked and plotted using the different methods mentioned above. The interplay of these methods illustrates to what degree tactics and activities co-occur within discourse, revealing how they are connected and how conversations are prototypically structured, including an overview of the subnetworks of categories of learners or units of discourse.

Studying the temporal patterns of SRL

As mentioned before, grouping and analysing different SRL events requires the use and combination of different methods from different domains (Reimann, 2009; Saint et al., 2020). The temporality of individual events, for example, can be studied using trends and time-series while the transition between events can be examined with PM (Gašević et al., 2017; Peeters et al., 2020; Reimann, 2009). The relationships between co-temporal events can be studied using ENA (Csanadi et al., 2018; Shaffer, 2019), while contemporaneous events may be better described using SM (Matcha et al., 2019, 2020). The timeline of events or sessions can be studied with temporal networks. To better understand the SRL process, and the different temporal patterns that constitute it, various methods are needed to highlight the complex interrelationships between different temporal aspects (Saint et al., 2020). These aspects can be studied as: (1) an aggregate (counts, frequencies or distribution) of different actions or groups of actions, part of the whole course or task, (2) relationships between actions or groups using social networks and epistemic networks, (3) the transition from each action/group of actions using PM and SM, and (4) the longitudinal timeline of events using temporal network plots or longitudinal SM. Please see Table 1 for suggested methods for analysis based on the level of analysis and trace data temporal resolution.

Table 1 Suggested methods for analysis based on trace data temporal resolution

Previous research on academic writing and LA

While earlier LA research on writing analytics has shown promising results when it comes to monitoring students and supporting them in their writing process to improve their writing skills (cf. Gibson et al., 2017), LA research focusing on the relational, co-temporal, contemporaneous and longitudinal dynamics of students’ self-regulation in academic writing remains scarce, with few exceptions (e.g., Peeters et al., 2020). Even recently, Knight et al. (2020) introduced LA tools to provide students with personalised feedback on their writing but did not capitalise on the timeframe in which these events took place. This study strives to bring these temporal features to the foreground and integrate them into its application of LA methods to study self-regulation for academic writing.

Theoretical lens of self-regulated learning

To analyse how foreign language learners employ SRL tactics to manage their academic writing in a CSCL setting, and to frame and discuss them accurately (i.e., in line with established SRL phases and processes in the literature), the theoretical lens of the Strategic Self-Regulation (S2R) Model of language learning has been adopted (cf. Oxford, 2017). The model has been chosen for two main reasons. First, any writing activity is a language learning activity, and the S2R model is specifically aimed at describing the actions of language learners, the target group in this study. Second, this model is grounded in Zimmerman’s (1990) task-phase model of self-regulation that consists of three interdependent phases (forethought, performance and self-evaluation). Oxford (2017) describes task-phase one, strategic forethought, as learners paying attention to the requirements of the task, planning how to address them, and taking steps to act accordingly. In the second phase, strategic performance, the learner implements the plan, monitors how it is going, and decides whether to continue performing the task, stop, or make changes in how to approach the task. In the third phase, strategic reflection and evaluation, the learner makes a judgement about learning outcomes, the effectiveness of selected strategies and tactics, and about the self (e.g., self-efficacy).

The application of this model provides us with a sound ground for the categorisation of the used SRL tactics and is believed to help us understand the SRL process’ dynamic nature. Furthermore, the S2R model addresses meta-strategies, strategies and tactics that language learners can use to regulate different aspects of their learning, including their beliefs, observable behaviours, their internal mental states, and various aspects of the learning context (Oxford, 2017). In the present study, we examined students’ SRL tactics in particular, which refers to more specific manifestations of a strategy and meta-strategy. We did so in order to present an ample number of data points to investigate the SRL process.

Case study settings

For this study, data was used from a case study at [University name]. In this case study, a private Facebook group was integrated in a first-year academic writing course for foreign language majors of English (n = 124). The group served as an online collaboration space for peer review in which students could share their written work, discuss their progress and ask questions about their writing and learning process. Students met with their teachers and peers during the 12 contact hours of the course. The course was blended, including online self-access modules with exercises on academic literacy, and the peer review forum on Facebook. There were no teachers included in the Facebook group since learners were expected to rely on each other for support (Philp, 2016).

Learners had to write three 300-word essays for the course, roughly one each month. After an initial brainstorm and writing session in class, learners were instructed to finish their essays at home and were informed that they could consult with their peers on Facebook about their writing process and the challenges they faced at all times. The essay-writing task added up to 30% of the overall grade for the course, next to in-class tests and assignments, as well as a number of extra marks that could be earned by completing self-access exercises online.

Methods

Data extraction and coding

Data extraction focused on scraping necessary information from the online platform such as posts, comments, participant IDs, post and comment IDs, time stamps, captions and embedded links. Informed consent was obtained from all participants involved prior to data extraction and analysis. The data set was anonymised before any analysis was conducted.

The data set was manually coded following the principles of digital conversation analysis to distinguish recurring themes and motifs in students’ posts and comments (cf. Farina, 2018). In a number of coding phases (cf. DeCuir-Gunby et al., 2011), a team of two coders compiled an exhaustive list of SRL tactics (Peeters, 2018, 2019), after which a team of four coders checked the coded transcripts for coding errors and inter-rater reliability. The team discussed disputed codes until a consensus was reached. The overview presented below (Table 2) forms the basis for this study into students’ SRL activities in an academic writing course. The table below was supplemented with Oxford’s (2017) classification of the different SRL task phases in the S2R model. During the coding process, several posts and comments received multiple labels, indicating multiple tactics were used in one single message, resulting in a final SRL tactic count of 3123 entries.

Table 2 Overview of tactics and S2R task phases in the data set

Data analysis

To create an overview on an aggregate level (where temporal aspects have been discarded), we used frequency analysis to shed more light on the distribution of tactics as well as their relative dominance. We further broke down the frequency distribution, in accordance with the three separate assignments, to compare the distribution of tactics over time (Reimann, 2009). We employed SNA to represent the relationships between aggregate tactics on the course level as well as on the assignment level. To analyse the events at the contemporaneous level, we used SM to describe the sequence of events, as well as PM to illustrate the transitions between them. We further showed the relationships between co-temporal events using ENA. We concluded our analysis by creating a proximity timeline to show the longitudinal unfolding of events over the whole duration of the course, bringing together both temporal and relational dynamics.

The frequency of the coded tactics was plotted to show the distribution of the tactics and strategic SRL phases. Another line plot was created to compare the distribution of tactics among the three assignments, and how each tactic was used.

To reveal the overall relationships and links between the coded interactions, a network was constructed by considering the reply of a post as a source, and the replied-to post as the target. Individual networks were also constructed for each assignment to compare the relationships between tactics over time.

To reveal the relational aspect of co-temporal events, we used the ENA Web Tool. We defined the units for analysis as the cut-off points for each assignment to compare students’ interactive behaviour (Shaffer et al., 2016). The ENA algorithm uses a moving window to construct a network model for each post in the data, showing how codes in the current line are connected to codes that appear within the same temporal context. The moving window in our study was defined using four lines (one line plus the three previous lines) within a given conversation. The resulting networks are aggregates of all lines for each unit of analysis in the model. In this model, we aggregated networks using a binary summation in which the networks for a given line reflect the presence or absence of the co-occurrence of each pair of codes. Our ENA model included the following codes: acculturating, planning, organising, writing text, writing arguments, using resources, social bonding, applying feedback, and reflecting. We defined conversations as all lines of data associated with a single value of a thread. A thread is a post with all the replies and interactions to that post. The ENA model normalised the networks for all units of analysis before they were subjected to dimensional reduction. Networks were visualised using network graphs where nodes correspond to the SRL tactics, and edges reflect the relative frequency of co-occurrence, or connection, between those tactics. We used ENA methods to compare students' interactive behaviour for the three assignments (cf. Shaffer, 2019).

To reveal the sequential aspects of contemporaneous events data were grouped into sessions, where one session refers to an uninterrupted sequence of events where the time gap between any two consecutive events is below the chosen threshold. The threshold was set to a time gap of one hour of inactivity between two events and was considered a cut-off value, corresponding to the 75th percentile of the dataset. The session duration was calculated as the total time between the first and last event in a session. These learning sessions were converted into sequences of coded interactions using the Traminer package (Gabadinho et al., 2011).

Differential sequence mining was implemented by comparing the high and low achievers using the sequence distribution plots (Kinnebrew, 2013; Kinnebrew & Biswas, 2012). SPM was implemented using the Chi-square test and plotted to show the differentiating sequences i.e., the statistically significant sequences that occurred in a higher frequency than expected in each group.

To shape the longitudinal timeline of events a temporal post-reply network was constructed for the coded interactions by examining the edges between the reply and the replied-to messages. The timestamp was considered as the onset of interactions, and the last time the post was replied to was considered the offset. We demonstrated the longitudinal relations between the coded interactions using a proximity timeline, which uses a combination of layouts to visualise the relational process longitudinally. The algorithm slices the temporal network at each time point (creating 49 networks), and then implements multidimensional scaling to cluster closely-related nodes according to their geodesic distance on a vertical timeline. A spline is then drawn, connecting the nodes along their position. This results in a timeline where closely related nodes are rendered together throughout the plot (Bender-deMoll & Morris, 2016).

To understand the transition between coded interactions two types of PM approaches were applied. First, relative frequency-based PM offered by the R package BupaR was used (Janssenswillen et al., 2019). Bupar packages offer sequential process maps that highlight the flow and frequencies of examined tactics. Second, to construct the process maps, the timestamp of each student interaction was used as the event time; the coded tactic was used as the ‘event’ and the students’ IDs as the case IDs. The node metrics of the process map represent the relative frequency of the implemented tactic; the edges represent the associative internode relative frequencies.

Results

Aggregate frequencies and relations

To demonstrate how often different SRL tactics were used by the students, as well as to depict how these tactics are interrelated, a distribution graph and an aggregate network graph were created (Fig. 1). These graphs show how often the three strategic SRL phases are represented in the peer interaction process. Looking at overall distribution, the most frequently used tactics were those related to the strategic performance phase, followed by the strategic forethought phase and, lastly, the strategic reflection and evaluation phase. Acknowledging, writing text and writing arguments were the most frequently used tactics. Tactics that are part of strategic forethought such as acculturating and organising followed suit. The third forethought tactic, i.e., planning, was something students almost never engaged in. Reflecting, similarly, is a tactic that is not frequently used by students throughout the peer review and peer interaction process.

Fig. 1

A distribution graph of students’ SRL tactics and an aggregate social network graph in which the nodes represent students’ SRL tactics and the thickness of the edges is proportional to the frequency of connections between them. Nodes with similar colours occur more frequently together

In order to see how students’ actions online connected to one another and, in doing so, to determine the distribution of tactics in the network of peers, an aggregate network graph was created. The strongest connections students made throughout the entire course were between organising and acknowledging, writing arguments and acknowledging, and writing text and acknowledging. Other noteworthy connections can be found between tactics that are part of the strategic performance and forethought phases such as writing arguments and acculturating, acculturating and acknowledging, organising and acculturating, and organising and social bonding. The different colours in the network represent three different clusters of tactics, showing that these tactics tend to be used in the same posts, comments and conversation threads frequently. For the overall network, the strategic performance tactics writing text, using resources and acknowledging tend to tightly link to the strategic forethought tactic organising, as well as to reflecting, while writing arguments and applying feedback link to acculturating. In the smallest cluster of this network graph, social bonding and planning tend to link up most often.

Frequencies and relations

While an aggregate network gives us an idea of the overall distribution of SRL tactics used by students, breaking down students’ use of SRL tactics per assignment gives us a more accurate view of the dynamics and longitudinal temporal aspects of the peer review and interaction process at the task level (Fig. 2).

Fig. 2

A Frequency plot of students’ use of SRL tactics during the three assignment phases and three network graphs on the links between students’ SRL tactics for each assignment. Nodes with similar colours occur more frequently together

First of all, it can be observed that both writing arguments and reflecting were very prominent tactics in the first assignment phase. In fact, about 65% of all posts and comments that revolved around formulating argumentation and about 46% that revolved around reflecting were made during this first assignment. During assignments two and three, their numbers decreased considerably. Secondly, as shown in the network graphs of Fig. 2, acknowledging is strongly connected to writing text, writing arguments and organising during the first assignment phase. Writing arguments, similarly, is strongly connected to acculturating. During the second assignment, student activity that revolved around discussing argumentation dropped considerably, all the while discussions on applying feedback increased together with acculturating, acknowledging, organising, social bonding and using resources. Applying feedback and acculturating show the strongest and almost only noteworthy connection during this assignment. During the third assignment, stronger connections emerge between organising and social bonding on the one hand, and between organising and reflecting on the other, as well as between applying feedback and reflecting. During the three assignment phases, it can be observed that different clusters tend to form, and that different tactics often find themselves in different clusters over time. Acknowledging and applying feedback are the only two tactics that always cluster together during the three assignment phases while all other tactics can be found in different clusters as time goes by.

Contemporaneous events

In order to capture the sequential aspects of students’ use of SRL tactics, SM was used to analyse the events within these contemporaneous time slots. Sequence maps group similar sequences of SRL tactics together that occur over a limited period of time. The different sequences of events in Fig. 3 show that students tend to start conversation threads using a range of different tactics, with writing text, writing arguments and acculturating as the main tactics. With 1045 occurrences, the sequence map illustrates how often different tactics tend to be used further on in conversation threads, where we can observe that there is a lot of room for acknowledging. The PM graph further highlights that acknowledging was the most frequently used tactic in this cluster overall (used 30% of the time), followed by writing text (23%) and writing arguments (20%). Students are observed to follow different pathways during their interactions, with about 46% of the time initiating conversations by discussing argumentation, about 18% of the time using elements of acknowledgement in their initial posts, and about 17% initiating conversations by discussing writing text. The PM graph also illustrates how acknowledgement increasingly finds its way into conversations as they progress, being linked to all other tactics in the process.

Fig. 3

Sequence mining map (to the left) and process map (to the right) of most dominant contemporaneous SRL tactics in the peer interaction process

Differential sequence mining (i.e., making comparisons across groups or clusters) can help identify important patterns that show differences among groups of learners e.g., high and low achievers (Kinnebrew, 2013; Kinnebrew & Biswas, 2012). In this study, we use a combination of SM and SPM to show the insights that can be obtained from such a technique. Figure 4A shows a comparison of distribution of the sequences between the high and low achievers (top 50% of students versus bottom 50% according to grades). The most notable differences can be found in the observations that high achievers are prone to discuss writing arguments more thoroughly, which is also the case for issues revolving around acculturation. SPM using frequent differentiating sequence patterns in Fig. 4B, shows that sequences of applying feedback were relatively less common in the low achiever group, which is a statistically significant observation.

Fig. 4.

a Differential sequence mining shows a comparison of low and high achievers, b shows sequence pattern mining, comparing frequent sequences of high and low achievers

Transitions between tactics

While the sequence map illustrated the contemporaneous proximity of different SRL tactics to one another, process maps give us necessary insights into the transitions between these tactics during the three assignment phases (Fig. 5). In the first assignment, students most often started conversations by posting about writing arguments (about 56%). These interaction sequences often stayed on topic and included comments on argumentation (about 50%) or several other tactics such as writing text (about 15%), acculturating (about 11%) and acknowledging (about 8%). During the second assignment, a shift can be observed, where students tend to initiate conversations by addressing issues that revolve around writing text (about 36%, compared to 12% in assignment one) while writing arguments drops significantly and falls off the chart. The conversation threads that started with writing text tend to stay on topic (about 32%) and most often include acknowledging steps (about 27%) or comments on acculturation (about 11%). While in the third assignment phase writing text is still quite prominent as a conversation starter (about 30%), it is taken over by organising (about 42%) as the most popular tactic to initiate a conversation with. Organising is most commonly followed by acknowledging (about 28%), comments on organising itself (about 19%) or writing text (about 17%) during this period.

Fig. 5

Process maps of the key SRL tactics per assignment and the progression of the SRL process

Co-temporal events

Epistemic networks were created for all three assignment phases to capture the temporal proximity between tactics and determine the rate of temporal co-occurrence. ENA graphs (Fig. 6) illustrate which tactics are central in the interaction process as they systematically co-occur in interaction sequences. For the first assignment, it can be observed that there is a strong connection between writing arguments and acknowledging, indicating they tend to be in close proximity to one another in interaction threads, as well as between acknowledging and writing text, reflecting, and applying feedback. In the second assignment, the connection between writing arguments and acknowledging fades away, a trend which continues over time in the third assignment as well. It can be seen that the connections between writing arguments and all other tactics become very faint too. Acknowledging does stay a strongly connected tactic throughout the overall interaction process and tends to co-occur with applying feedback, using resources and acculturating more strongly than before during the second assignment. It also still co-occurs with writing text, though to a lesser degree compared to assignment one. In the third assignment, the strongest connection that can be observed is between applying feedback and acknowledging, more so than in the previous two assignment phases. Acknowledging still tends to co-occur with writing text and using resources, yet to a lesser degree compared to the second assignment, while it has become more strongly connected to organising over time.

Fig. 6

Epistemic network graphs of the central SRL tactics per assignment phase

Longitudinal events

While all the social and epistemic networks above focused on aggregated events over a period of a couple of weeks (in accordance with the assignments deadlines), the timeline below (Fig. 7), shows a continuous tracing of the longitudinal relations between tactics. The timeline plot offers a sequential, relational and temporal map of longitudinal events, showing how these three aspects can be visualised without losing on complexity and readability.

Fig. 7

Timeline of the longitudinal relations between SRL tactics during the three assignment phases

The timeline was created by slicing the network of interactions into separate networks for each single day of the course. The timeline traces the trajectory of the relationships between the tactics. It shows when a tactic is closely related to other tactics, when and for how long they stay related. In doing so, the visualisation shows the longitudinal progression of the interrelation between the tactics.

In the first assignment, we can observe that after a few days, all the tactics are closely connected, except for planning and using resources. These two categories tend to diverge from other tactics and, therefore, often find themselves isolated. The tactics in assignment one stay closely connected until the deadline for that assignment approaches. By the end of this first phase, writing arguments and applying feedback start to diverge, indicating that they start to appear in separate conversations more consistently. During the second assignment phase, planning and using resources find themselves isolated again. What is most notable in the second phase is that in the middle of the sequence, almost all tactics tend to converge, indicating they tend to co-occur throughout the conversation threads at this time, and thus that conversation threads become quite diversified. During the time before and after this period (i.e., the time students received instructions for the new assignment and the time just before the deadline), it can be seen that the tactics diverge again. The third assignment phase lasted longer and, overall, more divergence can be observed. Over the course of this assignment, writing text, acknowledging, acculturating, reflecting and, for the first time, using resources, largely converge. Social bonding, applying feedback, writing arguments, organising and planning, on the other hand, tend to be less converging. During this period, it can also be observed that there are bursts of convergence and divergence, a tendency we could already start to observe in the second assignment phase. This timeline illustrates that the use of SRL tactics in the peer interaction process is subject to change, and that several factors such as the onset or offset of new tasks and assignments, the time students have to complete assignments or how familiar they are with the types of tasks they have to fulfil, might influence the dynamics and co-occurrence of SRL tactics in online environments over time.

Discussion

The analysis of relational and temporal aspects of self-regulation

This study aimed to examine how and why selected LA methods could help reveal the relational and temporal dynamics of SRL in the CSCL context of academic writing. In this paper, the distinction was made between co-temporal, contemporaneous, and longitudinal aspects of SRL, analysed using social, temporal and epistemic network analysis as well as process and sequence mining. In accordance with recent advances in LA research on sequential and temporal models of collaboration (Swiecki et al., 2019a, 2019b) and SRL (Saint et al., 2020), this paper has reconsidered the concept of temporality, suggesting a new classification of the concept, and has brought together methods in the field to analyse them, in accordance with the first research question.

While, within the measurement of SRL, it has become clear that we can augment the more conventional frequency analyses with SNA, ENA and PM (Saint et al., 2020), SM has proven to analytically and visually strengthen the analysis of the dynamic temporal nature of learners’ SRL behaviour. Slicing up the entire period in which learners were active has, additionally, provided us with more information on the ways SRL tactics are used over time and how useful it is to look at partial temporal aspects, rather than the whole aggregate picture. Since Swiecki et al. (2019b) have argued for “visualizations that effectively summarize team performance” (p. 148), it is our belief that the combination of sequential, temporal and relational measures can shed more light on the ways teams collaborate and work towards common goals. As Swiecki et al. (2019a) have pointed out in another paper, it is crucial to strive for methods that can combine the identification of relevant connections with dimension reduction and visualisation, which can be found in the ENA graphs. What is more, the timeline that is presented in Fig. 7 might be superior in this regard as it enables us to trace the longitudinal relations between tactics, distinguish levels of co-occurrence as well as temporal dimensions of SRL.

To answer the second research question about how LA methods can be used to uncover the evolution of learners’ use of SRL tactics to manage their academic writing process, our study has demonstrated that SRL unfolds and develops on different temporal levels, and that students' SRL activities interrelate at different rates on these temporal levels. Looking at the tasks, our findings show that different SRL tactics are used at different time intervals. As an example, writing argumentation was a tactic that was used significantly more often, and was well-connected to feedback, acculturation and acknowledgment, during the first assignment phase. Students were also more likely to start their discussions by bringing up issues that revolved around writing argumentation in the initial stages of the course and this tactic used to be a prominent co-temporal component of conversation threads during the first assignment. Nevertheless, this tactic fades into the background over time. The aggregate network of Fig. 1 does not show this decline and depicts writing argumentation as a prominent tactic throughout, which is not the case. Looking at the longitudinal timeline, we see more details about the evolution of argumentation, as it starts well-aligned with other tactics until around day 12 where it diverges for most of the second assignment and throughout the third assignment.

These findings further build on Teng and Zang’s (2016) work on monitoring self-regulation for academic writing in a foreign language, where the authors were able to show the interplay of SRL processes on both individual, behavioural, and environmental levels. The writing strategies for SRL they described (i.e., goal-oriented monitoring and evaluating, idea planning, peer learning, feedback handling, course memory, interest enhancement, emotional control, text processing and motivational self-talk), are represented in the SRL tactics in this study, with the addition of a thorough description of the relational, co-temporal, contemporaneous and longitudinal dynamics of the process. The different LA methods that were used have uncovered a major shift in students’ SRL behaviour for academic writing over time, with a transition from tactics which focus on developing and monitoring content and ideas to more form-specific and socio-cultural ones. This further highlights the need for more detailed, comprehensive analyses of SRL for academic writing to improve our classroom practice, our curriculum design and our support systems (Strobl et al., 2019); even more since research has shown that a majority of learners struggle to accurately calibrate their own learning process in this regard (Azevedo et al., 2019; Dunlosky & Lipko, 2007; Viberg et al., 2020a, b, c).

Choosing the right LA methods

SRL is a complex, multi-faceted process which can unfold in different ways and on several temporal levels, and therefore the analysis of SRL events required us to combine different methodologies (Reimann, 2009; Saqr et al., 2020a, b). The choice of methods and the scale at which to implement them depends on the tasks at hand, the granularity of the data and the research questions.

On the basic level, frequency analysis offers a snapshot that is devoid of temporal dimensions. While proven useful, it should only be used when temporal aspects are not relevant for analysis, or when this information is not available. When aspects of time are relevant, the study of individual events can benefit from using time series and trend analysis methods. These methods offer a general overview of the linear progression of events. The temporal dynamics of multiple events can further be studied using temporal networks (Vector auto-regression) as they offer a relational and temporal picture of SRL events, as well as an overview of how these events interrelate.

The transition between events can be studied using process mining, where process models offer a detailed mapping of each transition, including details on time and frequency (Gašević et al., 2017; Peeters et al., 2020; Reimann, 2009). Co-temporal events can be mapped using network analysis methods, more specifically ENA. ENA allows us to study and compare different time windows based on the task at hand. Such flexibility can help researchers better understand the dynamics of different tasks and how students approach them over time (Csanadi et al., 2018; Shaffer, 2019).

Analysing ordered successions of events or sequences (Kinnebrew, 2013; Kinnebrew & Biswas, 2012) is typically done by using SM. SM can map the distribution of sequences, the transition probabilities between events, as well as an event’s entire trajectory from beginning to end. Moreover, SM offers different clustering algorithms that take into account temporality and that can group different temporal events together, also known as differential sequence mining. SPM can be used to study frequent sub-sequences and allows us to differentiate between subgroups. In doing so, SM offers a wealth of methods that can shed light on the different temporal dimensions of SRL.

This study has offered a solid example of how students’ continually evolving SRL tactics can be effectively traced with the help of LA methods. To obtain a more holistic picture of the students’ SRL process, the next step will be to examine how these tactics are linked to other constructs (i.e., SRL strategies and meta-strategies) and dimensions (i.e., cognitive, affective and socio-cultural interactive) of the S2R model (Oxford, 2017). Other methods can be explored such as psychological networks (Saqr & Lopez-Pernas, 2021; Saqr et al., 2021) to reveal the complex relational and temporal aspects of SRL. This will offer a sound ground for designing and providing relevant SRL support and tools (cf. Viberg et al., 2020a, b, c). Such tools can be oriented towards learners to trace their actions, towards teachers to enable them to teach relevant SRL strategies and intervene in time, as well as towards researchers to provide them with a better picture of the complex nature of the SRL process.

Conclusion

The combination of different LA methods has helped us map the temporal, sequential and longitudinal aspects of SRL. To better understand SRL, various methods can be used to highlight the temporal patterns as well as the interrelation between different temporal aspects of the process. Analyses can (1) study these aspects as an aggregate (counts, frequencies or distribution) of different actions or sequences over a long period of time, (2) focus on the relationships between actions or sequences using social networks and epistemic networks, (3) focus on the transitions between each action or sequence using PM and SPM, or (4) focus on the longitudinal timeline of events using temporal network plots or longitudinal SPM.

The contribution of this paper is twofold: we first defined a grouping framework for temporal events so that we can describe and communicate such events more accurately. Without a common language, research is less likely to be communicated clearly. As Ifenthaler and Yau (2020) put it, “learning analytics research and development need to clearly define standards for reliable and valid measures” (p. 1981). Our work can thus be seen as a next step in defining temporal measurement levels.

The second contribution of our work is how we mapped the temporal levels of the self-regulated learning process. We extended previous efforts by exploring the longitudinal timeline of events. Our approach continuously traces the trajectory of events and the relationships between the examined self-regulated learning tactics. It has clearly shown when each tactic correlated with others, when they dissociated from others, and which tactics were used at each point in time during the process. In doing so, it has contributed to mainly uncharted territory, i.e., mapping the longitudinal evolution of events in self-regulated learning. While our study was set in academic writing context, the results apply in contexts that have computer-supported collaborative learning or collaborative learning and serve as an example of how SRL data can be explored temporally.