Jack Burston
Computer-Mediated Feedback in
Composition Correction*
Jack Burston
Temple University
ABSTRACT
The purpose of this paper is to describe theoretical and practical considerations related to the provision of feedback in the written compositions of
advanced level foreign language learners, as exemplified by second year
(semester 7-8) students of French. The paper begins by discussing the
approach taken to teaching and assessing writing skills of students at
Monash University. It then proceeds to a consideration of how using a
computer-based composition annotation program, Markin32, can contribute to the reduction of correction loads for instructors and the improvement of the quality and usefulness of composition feedback for students.
KEYWORDS
Advanced Level Writing, Computer Feedback, Automated Annotation,
Focus-On-Form Approaches
INTRODUCTION
As part of a long-term strategy to improve the effectiveness of the teaching of writing skills in advanced level (semester 7-8) second-year French
language courses at Monash University, a two-pronged pedagogical approach has been adopted. The first, reported previously in the CALICO
Journal, concentrates on the problem of morphosyntactic accuracy and
ways to remedy it (Burston, 2001). The second focuses on higher level
*This project was made possible through the generous support of an Australian National
Teaching Development Grant, for which I would like to express my sincere appreciation.
The research described here also owes much to the collaboration of my colleague Patrick
Durel and graduate student Eugene Mogilevski whose assistance and enthusiasm are equally
appreciated.
© 2001 CALICO Journal
Volume 19 Number 1
37
Computer-Mediated Feedback
syntactic, semantic, and lexical accuracy as well as discourse coherence
and cohesion. It is this latter aspect of advanced writing skills that is the
topic of this article.
The acquisition of foreign language writing skills at advanced levels is a
challenging task for students, one for which dominant teaching methodologies that focus on oral communicative competency provide little preparation. For English speaking students of French, the task is made all the
more difficult by the norms of formal discourse in French, which are frequently more demanding than those expected in their native language.
Learner-centered approaches to the development of writing skills quite
justifiably focus on the process of writing as opposed to the product of
writing. To be effective, however, learner-centered approaches necessarily
involve a cyclical process of drafting and feedback. From the teacher’s
perspective, marking essays can be enormously time consuming. From the
student’s viewpoint, the more detailed the corrective comments, the more
difficult and off-putting they can be to assimilate. Practical problems, such
as the legibility and consistency of marginal and interlinear annotations,
further contribute to student difficulties. Moreover, as is well known, little
is to be gained from the correction of compositions if feedback is not
systematically linked to the process of rewriting. As traditionally practiced, multiple correction of essays is as burdensome for students as it is
impractical for teachers and, hence, not practiced nearly as much as it
should be.
THE PROCESS AND PRODUCT OF WRITING AT MONASH UNIVERSITY
Within the framework of second language acquisition theory, advocates
of “Focus-on-Form” approaches (Daughty & Williams, 1998) stress the
need to direct the attention of language learners to formal problems within
a meaningful linguistic environment, as opposed to decontextualized grammar exercises. While there are many ways to achieve this objective, the
guiding principle of such approaches is to disrupt the flow of communication as little as possible and to get learners to “notice the gap” between
what they have produced and what is required by the target language. In
essence, the approach is intended to foster discovery learning. When applied to composition correction, such an approach can also be used effectively to reduce marking loads substantially thereby making cyclical correction and rewriting possible.
In the case of our students, the process and product of writing are dealt
with separately during the term. In previous years, students wrote three
essays (250-300 words), each marked only once, plus a final exam composition. Students now write only two essays but submit them for correc38
CALICO Journal
Jack Burston
tion two or even three times. The former final 90 minute examination
composition, the correction of which students usually never saw, has now
been replaced by two essays written in class under test conditions (i.e.,
completely closed book). The first TST (Travail sur Table as it is called in
French) is given at midterm and, like homework essays, is returned to
students within a week. Aside from the linguistic feedback it provides, the
corrected TST also serves as an early warning to students of what to expect on the second TST given during the final week of classes. The second
TST, too, is corrected and returned to students within a week, that is,
prior to the final exam. Whether the focus is on the writing process or
product, an important part of our pedagogical strategy is making writing
activities during the term count substantially in course assessment: 15%
for the two homework essays plus 10% for each of the TST, a total of
35%. To encourage students to take their writing seriously from the beginning, first drafts account for 80% of the homework essay mark.
Compared to previous practice, where feedback was only provided on
three pieces of written work, students now receive feedback on their writing nine times during the term. Of particular importance, because of the
use of focus on form techniques, the goal of nine corrections has been
achieved without significantly increasing workloads for students or staff.
First drafts of homework assignments are corrected with minimal intervention from instructors who use two types of mark-up devices.
Morphosyntactic mistakes (e.g., errors in spelling, accents, and gender
agreement) are simply highlighted. Higher level syntactic and semantic
difficulties (e.g., vocabulary, tense usage, and phrasal constructions) receive equally minimal treatment the first time around: a maximum of 20
errors are simply underlined without comment. Only in the case of lexical
problems is the nature of the difficulty identified by the symbol “Voc.”
Using this technique, an instructor can usually mark a first draft in about
5 minutes rather than the 20-25 minutes usually required for a full correction.
As part of normal classroom instruction, students are taught how to
spot and eliminate simple mistakes on their own. They are also taught
how to use a French grammar checker (Antidote) to verify the accuracy of
their own corrections. In any event, nothing is to be gained from commenting on simple mistakes in marking students’ work and even less from
correcting them. From a theoretical as well as a practical point of view, all
that can be done is to draw attention to the gap between students’ level of
writing proficiency and that required in French and place responsibility
on students to eliminate it. Since 20% of their first draft mark (i.e., the
equivalent of a full letter grade) is determined by basic grammatical accuracy, students quickly get the message and submit work reasonably free of
fautes bêtes ‘low level errors.’ (For discussion of the issue of low level
errors, see Burston, 2001.)
Volume 19 Number 1
39
Computer-Mediated Feedback
The simple underlining of other types of difficulty in homework essays
serves the same purpose of focusing students’ attention on errors and requiring them to reflect on what they have written. Even at an advanced
level, the number of such problems can run into the dozens in a typical
250-300 word composition. Restricting the number of underlinings to 20
errors has important advantages for teachers and learners alike. For the
instructor, it reduces the amount of correction required and helps to ensure that areas targeted for treatment are those that are the most relevant
to the curriculum. For students, it significantly stems the usual flow of red
ink thereby helping to keep the “affective filter” down. Analysis of second
draft compositions has shown that about three quarters of underlined problems are adequately addressed by the students themselves, either on their
own or through out of class discussions with their instructors. Admittedly, this approach occasionally leads to avoidance strategies and some
inappropriate corrections, but, overall, second drafts are substantially
improved with minimal teacher intervention.
The correction of second drafts, though more comprehensive, is nonetheless again kept to as much of a minimum as possible in order to engage
the maximum attention and reflection of students. As with first draft corrections, minimal commentary has the advantage of reducing marking
workloads, focusing on the most relevant problem areas, and not overwhelming student compositions with annotations. The workload involved
in marking second drafts is of course considerably reduced by the prior
elimination of morphosyntactic mistakes and other errors. To facilitate
correction, a standardized code of some two dozen short annotations is
employed (e.g., Réf (ambiguous reference), Tps (wrong tense), Ord (incorrect word order), Cv (faulty verbal construction), and Con (discourse
connector needed). Faulty sentences are not rewritten for students and
individual comments are again intentionally kept to a minimum in order
to focus as much as possible on the positive features of the substance of
the compositions. As a rule, second drafts usually take no more than 10
minutes each to mark in this fashion and can be quickly returned to students.
Because of time and workload limits, only one of the two term essays
undergoes a third draft. However, knowing that they must produce two
compositions in class under test conditions, students are inclined to look
over their returned second draft carefully (whether or not they are required to resubmit it), which was demonstrably not the case when they
only submitted a single draft of essays and faced a distant final exam.
Given the writing-feedback cycles which precede third drafts, final correction can focus on sentence remodeling and more discourse related difficulties. While the emphasis remains on the process of writing, the final
product is also very much in evidence at this point. In terms of correction
time, it can still take 10-15 minutes to mark the third draft of an essay; but
40
CALICO Journal
Jack Burston
the time is productively spent on higher order matters, and the results are
well worth the effort. Since only half the term essays are corrected a third
time, overall correction workloads remain essentially the same as before.
As previously indicated, the product of student writing is separately
assessed by two TSTs, one at midterm the other in the last week of classes.
The TSTs are marked in exactly the same manner as the first drafts of
homework essays in the form of highlighted morphosyntactic mistakes
and underlined syntactic/semantic errors. Although instructors have to
mark two TST, compared to the previous one exam essay, minimal correction results in comparable marking workloads. The midterm TST is returned to students, who then resubmit it for 20% of the mark. The final
TST is the only composition submitted just once. A detailed analysis of
TST results from the second semester of 1999 (when they were first introduced) showed a marked improvement in grammatical accuracy of the
midterm TST compared to the final exam essay of the first semester: 6.5
errors per 100 words versus 8.1 errors per 100 words. Likewise, the end
of term TST showed an equally large improvement compared to the first
TST (4.8 errors per 100 words vs. 8.1 errors per 100 words). The results
from the first semester 2000 TSTs are even more encouraging. A diagnostic TST was administered during the second week of classes. Analysis revealed an underlying error rate of 10.1 errors per 100 words, which was
essentially what had been observed in previous years’ examinations. In
comparison, the midterm TST results were 5 errors per 100 words, an
improvement of more than 50% and very nearly what it took a whole year
to achieve in 1999. Analysis of this TST data set with regard to higher
order syntax, vocabulary, and discourse structure has not yet been completed, but early results also appear encouraging.
MARKIN32
In seeking to improve composition feedback without increasing marking workloads, our attention was drawn to an inexpensive shareware program called Markin32 which allows automated annotation of essays. (For
a review of Markin32, see Burston, 1998.) Markin32 operates only on
Windows95/98 but produces annotated compositions in ASCI, RTF, and
HTML. Consequently, work can be returned to students in a format compatible with both PC and Macintosh platforms. Likewise, since Markin32
accepts as input texts in either ASCI or RTF, it can accommodate student
essays written with virtually any word processor regardless of platform.
From the instructor’s viewpoint, Markin32 is extremely easy and flexible to use. The work space of Markin32 is essentially that of a word
processor with familiar File and Edit options in the top toolbar menu.
Although texts can be created directly in Markin32, they are normally
imported as ASCII or RTF files for composition correction.
Volume 19 Number 1
41
Computer-Mediated Feedback
The heart of Markin32 is the system of automated annotation buttons
(see Figure 1).
Figure 1
Markin32 Annotation Buttons
As can be seen, the buttons display only a very abbreviated annotation,
but when the cursor is placed over them a fuller description is revealed.
Markin32 was originally developed for ESL students and so is delivered
with a set of default annotation buttons in English. The buttons of course
need to be adapted for foreign language work, which can be easily done by
typing required annotations into three text fields (see Figure 2).
42
CALICO Journal
Jack Burston
Figure 2
Adapting Markin32 to French
Once the changes are made, the buttons can be saved as an external set
and made the default for the program. It is possible to create any number
of annotation button sets and use them to meet different requirements
(e.g., advanced level discourse analysis or content commentary). Using
the annotation buttons is equally simple; students just highlight the portion of the text to annotate and press the appropriate button. Annotations
can be undone by selecting the annotation and then clicking the delete
annotation button.
In addition to the standard annotation buttons, Markin32 also allows
for the insertion of free-form comments. The same procedure is used to
insert comments; the instructor just highlights the portion of the text to
be commented and then presses the appropriate button (see Figure 3).
Volume 19 Number 1
43
Computer-Mediated Feedback
Figure 3
Instructor Comments in Markin32
Another useful feature of Markin32 is the tabulation of error statistics
which can be included in returned work, if desired.
Corrected compositions can be returned to students in two text formats
as well as in the form of a web page, the selection of which is just a matter
of clicking on the corresponding icon. As shown in Figure 4, work returned in RTF appears essentially as it does within the Markin32 editing
window.
44
CALICO Journal
Jack Burston
Figure 4
RTF Text from Markin32
Work returned in HTML format displays only the annotation tags; however, these are in fact hyperlinked to fuller explanations.
The advantages of using Markin32 should be reasonably apparent.
• Standard annotations can be quickly inserted and changed as
required;
• Free form comments are readily accommodated.
• Annotation is legible and unobtrusive.
While Markin32 can be used in principle for any kind of annotation, a
word of caution is in order regarding the purposes to which it is put and
the danger of overwhelming students with feedback. As previously mentioned, within the composition correction framework established for our
students, basic morphosyntactic mistakes are noted but never commented
on. Consequently, the use of Markin32 for the correction of first drafts
would serve no purpose, the whole point of such correction being to focus
on form with minimal teacher intervention. The marking of second drafts
is an entirely different matter, however, and the kind of abbreviated feedVolume 19 Number 1
45
Computer-Mediated Feedback
back provided by Markin32 is particularly appropriate for the intended
purpose. Because of prior corrections, second drafts are largely free of
low level errors. Accordingly, the annotation code used with Markin32
essentially concentrates on higher order problems. Were this not the case,
and correction attempted to deal with lower level errors at the same time,
students’ compositions would typically be festooned with annotations. As
it is, even with a self-imposed limit of 20 syntactic/semantic annotations
per essay, care needs to be taken not to overwhelm students with negative
feedback and always to include some positive reinforcement, which can
be done either by a standard annotation button or a free from comment.
Benefiting from Markin32 requires a certain number of adaptations to
traditional composition writing/correction practice. First, students must
of course submit work in word processed form. In situations such as in
our course, where multiple drafting is required and where access to word
processors is readily available on campus as well as in the home, getting
students to use a word processor poses no problem. Notwithstanding,
anyone attempting to integrate the use of Markin32 into the curriculum
will need to deal with two practical problems: file format, and procedures
for the submission and return of work.
Unless compositions are saved in ASCI or RTF, Markin32 will be unable to import them. While file formatting is easy enough to do, most
students need to be taught how to do it. Inevitably, especially at the beginning of the course, some students will submit their work without properly
formatting it, but this difficulty usually takes care of itself very quickly.
Moreover, all industry standard word processors, like Word or
WordPerfect, can import multiple versions of their own and each others’
file formats. As a result, the instructor can usually reformat a student
essay before importing it into Markin32, if necessary. Should worse come
to worst, the essay can always be returned to students for resubmission in
the required format.
More problematic than file formatting is the issue of how students should
submit their work and how it should be returned to them. After some
experimentation, we discovered that the best solution was for second and
third drafts to be sent and returned as e-mail attachments. This process
has several advantages.
• Unlike ordinary e-mail messages which, especially on Wintel
systems, can preclude the use of diacritics or make access to
them very cumbersome (i.e., typing in ANSI codes), file attachments can use any fonts supported by the originating word processor.
• Students can submit their work at virtually any time from a
large number of points on campus as well as from home if they
have an Internet connection. The same applies to the electronic
retrieval of corrected assignments.
46
CALICO Journal
Jack Burston
• The instructor has proof of submission and, thanks to the time/
date stamp accompanying all e-mail messages, can substantiate
whether or not deadlines have been respected.
• E-mail messages themselves can facilitate communication between students and the instructor, in particular allowing for more
personal interchanges (e.g., explanations why an assignment was
late, general words of encouragement, and admonitions).
• Since attached files remain with original messages even after
they have been extracted, there is always a backup copy in reserve should disaster strike.
• The submission of essays in digital form creates a potentially
rich database for future research into writing skills development.
As with Markin32 itself, the use of e-mail distribution of student essays
also has its special requirements. If instructors retrieve e-mail or extract
attachments at more than one location, some system needs to be devised
to combine extracted student essays into one database or otherwise keep
track of them. Needless to say, up-to-date antivirus protection is an absolute necessity, preferably one which operates automatically when attachments are extracted from e-mail messages.
The electronic correction of student essays of course can only take place
when connected to a computer. Instructors used to taking home a pile of
compositions and correcting them on the fly as circumstances permit (e.g.,
on the train or while waiting for a dental appointment) are likely to find
that the technological overhead outweighs the potential gain. On the other
hand, computer-based composition correction makes possible the use of
some very powerful ancillary tools such as on-line grammar checkers and
dictionaries.
The ability of Markin32 to produce HTML versions of corrected work
also opens up the possibility of integrating the results into a course web
site. This potential is further extended by the ability of Markin32 to link
annotations to external HTML sources such as web sites. For students in
our course, this feature has been exploited to make available a context
sensitive on-line reference grammar. For the instructor, linking Markin32
annotations to an appropriate grammar reference is simply a matter of
clicking on a menu selection (see Figure 5).
Volume 19 Number 1
47
Computer-Mediated Feedback
Figure 5
Markin32 Linking Procedures
Likewise, to consult a grammatical reference, all students need to do is
click on an designated option (see Figure 6).
48
CALICO Journal
Jack Burston
Figure 6
Markin32 Grammatical Reference
CONCLUSION
As should be obvious, Markin32 offers many ways of facilitating composition correction and improving feedback to students. Just as obviously,
its successful use requires some technological literacy on the part of teachers
and learners. Preliminary reactions from students in an earlier pilot group
were very positive, and the end-of-year results pointed to improvements
in writing skills above those in the control group. One must remember,
however, that a number of interdependent factors come into play when
introducing pedagogical innovations like the use of Markin32 into the
curriculum, not the least of which is the instructor’s enthusiasm. For a
teacher who is already at ease with instructional technology and accustomed to correcting essays in electronic form, Markin32 can save time
and energy, at least enough to compensate for the technological overhead
its use entails. Whether or not Markin32 can be integrated into an entire
course, to be used by all instructors, would of course depend on the common denominator of technological literacy among those called upon to
teach the subject. For those willing to experiment on their own, it is certainly worth its shareware price of $30.
Volume 19 Number 1
49
Computer-Mediated Feedback
REFERENCES
Burston, J. (1998). Review of Markin32 (Ver. 1.2). CALICO Journal, 15 (4), 6774.
Burston, J. (2001). Exploiting the potential of a computer-based grammar checker
in conjunction with self-monitoring strategies with advanced level students of French. CALICO Journal, 18 (3), 499-515
Daughty, K., & Williams, J. (1998). Focus on form in classroom second language
acquisition. Cambridge: Cambridge University Press.
AUTHOR’S BIODATA
Jack Burston (B.A., M.A. in French; Ph.D. in Linguistics) is the Director
of FLIT at Temple University. He is particularly interested in authoring
languages and systems for courseware development and is currently completing a multimedia CD-ROM/WWW project on French sociolinguistics.
His major research interest is in second language acquisition in instructed
environments. Since 1996 he has been the Software Review Editor of the
CALICO Journal.
AUTHOR’S ADDRESS
Dr. Jack Burston
Director of FLIT
College of Liberal Arts
524 Anderson Hall
Temple University
1114 W. Berks Street
Philadelphia, PA 19122
Phone: 215/204-3678
Fax:
215/204-3731
E-mail: jburston@astro.temple.edu
50
CALICO Journal