Vahid Aryadoust

Though significant discussions in writing assessment literature focus on understanding the relationship between the quality of second language (L2) students’ texts measured by human judges and linguistic features identified by automated rating engines such as Coh-Metrix, little attention (if any) has been given to assessing reflective essays presented as individual student blog posts in a tertiary level communication course. The present study examines the relationship between the linguistic features of the reflective blog posts of Asian university learners enrolled in a professional communication course as measured by Coh-Metrix and these posts’ quality as assessed by human raters in discrete assessments. Rather than using traditional linear regression methods, the data was subjected to classification and regression trees (CART) to address this specific research question as follows:

How might Coh-Metrix indices of linguistic features including lexical diversity, syntactic complexity, word frequency, and grammatical accuracy relate to the assessment of these reflection essays made by the instructor?

This study uses the data from 104 tertiary students enrolled in a communication module. They completed four writing tasks at four time points (i.e., Pre-Course, Mid-1-Course, Mid-2-Course, and End-Course), yielding 416 essays, which were marked holistically by both human raters and analyzed via Coh-Metrix. 84 linguistic features for each essay (including vocabulary sophistication, lexical diversity, syntactic sophistication, and cohesion statistics) were recorded.
            A description of the nature of the reflective blog posts will be presented, along with the rationale for this study, more details on the methodology used for analyzing each post and preliminary findings. It will also be argued that using CART modeling to predict essay quality from linguistic features is novel. Unlike linear regression models, CART relaxes normality assumption, optimizing the predictive power of the analysis.