Feedback Follow Up: the Influence

ofTeacher Comment
on Student WritingAssignments

by Deborah W. Dunsford

Introduction

Writing improvement is becoming an increasingly important topic at most

universities. Feedback from potential employers and research repeatedly shows that
college students' writing abilities are below expectations (Lindner et al., 2004).
Several universities have implemented writing-intensive course requirements for
undergraduate students that will ultimately require faculty in all
disciplines toprovide additional writing opportunities in their curriculums (Univ. of
Florida, 2004. The Gordon Rule; Texas A&M Univ., 2004. Writing Intensive
Courses at Texas A&M Univ.; Martin and Burnett, 2003).

For agricultural education and communication programs, this focus frequently takes
the form of service courses that teach writing skills (Kansas State Univ., 2004; The
Ohio State Univ., 2004; Texas A&M Univ., 2004. Core Curriculum; Univ. of
Florida, 2004. Agricultural Education). As the demand for seats in courses that
teach writing skills continues to grow, instructors try to balance the need to provide
students with feedback on their writing assignments with the amount of time it
takes to provide that feedback. While writing instructors from all disciplines
generally agree that revision is one of the best ways to encourage
students to improve their papers, few know what comments or what type of
comments are most likely to help their students revise successfully.

Research into revision and how and why students revise their texts has long been
part of composition literature. So has research into teacher comment
on student texts. However, there is little work that brings the research areas together.
This study may provide a link between these two important areas of research.

Composing a piece of written discourse has long been considered a non-linear,

recursive process (Britton, 1975; Rohman and Wlecke, 1964). Later researchers
built on this model describing composing as a continuous loop where any element
may follow any other element (Bridwell, 1980; Faigley and Witte, 1981; Flower et
al., 1986; Sommers, 1980).

Although the recursive nature of the process is not in question, an actual definition
for revision is less clear. Several definitions use only the etymological definition of
"seeing again" (Boiarsky, 1980). Sommers (1980, p. 380) defines revision as "... a
sequence of changes in a composition changes which are initiated by cues and occur
continually throughout the writing of a work." Drawing on all these definitions, the
operational definition for revision used in this study refers to the additions,
deletions, substitutions, and rearrangements of units of meaning that students make
in their texts in an effort to convey better their intended meaning to an audience.

Teacher comment is another key area of composition research. Much research

shows that teacher response can have a major impact on a student's attitude toward
the text and toward writing in general. De Beaugrande (1979) claimed that if
students see grammar, punctuation and spelling as priorities in teacher comment,
then those are the errors they will repair. Miller (1982) suggested two separate sets
of teacher comments one on content and the other on writing problems. Murray
(1979) advocated doing away with comment completely and using one-on-one
conferences toprovide feedback to students. Peterson et al. (2004) suggest that the
type of paper plays a role in the type of comments teachers provide. Narrative
papers receive a greater percentage of editing-related comments and persuasive
papers tend to receive a greater percentage of revision-related comments (Peterson
et al., 2004).
Besides types of comments, other research examines the quality of those comments.
Lynch and Klemans (1978) surveyed students about their
responses to teacher comment and found that students responded more
positively to comments that not only told them what was wrong with a paper, but
why. Straub (1996) explored directive versus facultative comments on student texts
and the potential control the comments represented. In general composition
researchers agree that the goal of teacher comment on papers is to wean students
away from criticism from the teacher and toward forming their own ability to review
and revise their texts.

Purpose

The purpose of this study was to determine how the types and location
of teacher feedback on a group of student texts influenced the revision choices that
group of students made to their texts. The research objectives were to determine if
the location of teacher feedback influenced students' revision choices and if the type
of content or type of teacher comment influenced those choices.

Methods

The subjects in this study were 62 students enrolled in media writing classes at a
land grant institution in the South. Each of the four classes studied had enrollment
limits of 16 students and the students were primarily juniors and seniors majoring in
journalism or agricultural journalism. A few students were either minoring in
journalism or planned to obtain teaching certification in journalism. The majority of
the students were female (69%), which reflects a nationwide trend in
communication departments. The classes met four times per week during the 16-
week term. The Monday and Wednesday lecture sessions covered techniques in
news writing and textbook material on different types of news stories, as well as
some basic information on methods of writing. The other two weekly class meetings
were 75-minute lab sessions. The students used Microsoft Word software for all of
their assignments. Besides the lab periods, students had access to the computer-
equipped classroom through the work day and had access to their own computers or
other campus computer labs throughout the semester.

Data was collected from the students' four major writing assignments. The four
assignments were as follows: (1) Write a pair of short, one-paragraph leads from a
choice of assigned fact sets; (2) write a news story from a short speech and question-
and-answer session presented by a guest speaker; (3) write a news story about a
coming event on campus or other item of the student's choice that quotes at least one
source (i.e., they had to go interview someone and write the story); and (4) write a
short feature story on a topic of their choice that quoted at least two sources and
required additional background sources. Students wrote for both print and broadcast
media. Theteacher comments and the revisions students made on these assignments
provided the data for this study.

Students had the option to revise one of the two versions of each of the major
assignments. If the students opted to revise one of their papers, the grade they
received on the original paper counted as two-thirds of the grade on the final paper.
The grade on the revised paper counted as the remaining one third. This method
encouraged students to make their best effort on the original paper. The students'
grades on the revised papers would not be lower than the original grade they
received, although the grade could remain unchanged. For purposes of this study,
only the papers that the students opted to revise were analyzed.

Each of the four classes received four different methods of instructor feedback with
a different method used on each of their four major assignments. The comment
methods were marginal and concluding written comments on their papers, marginal
comments only, concluding comments only, and only oral comments to the class as
a group. When revising their papers, the students were required to return the graded
original paper along with the revised version.
To protect the students' identities and to eliminate any chance of bias associated
with any particularstudent, each student was assigned a random number, and an
uninterested third-party placed this number on the students' texts and then cut off the
students' names and course section numbers. Topreserve the regular classroom
environment during the study, the students were not told about the study until the
end of the semester, after the last paper had been turned in. The students received a
written explanation of the study and the use of their texts (anonymously). At this
time they were offered the chance to have their papers removed from the study.
None of the students selected this option. This study met all university requirements
for human studies research and all necessary forms are on file with the university's
research office.

After the student texts had been collected they were sorted by assignment
and teacher comment type (marginal and end, marginal only, end only and oral only
comment). The texts were sorted numerically for ease in coding and an index card
was established for each student number. These cards provided a method of tallying
the number and types of revisions on each text. The data from these cards provided
the basis for the statistical analysis in this study.

Structural revisions made by the students in a second, revised paper were

compared to their original, graded papers. Structural revisions in this study were
additions, deletions, substitutions, and rearrangements (Sommers, 1980). These
structural revisions were examined at the level of units of meaning that may or may
not correspond to the physical division of paragraphs within the text.
According to Rodgers (1967), paragraph divisions frequently do not correspond
with units of meaning within a text, and he suggests that a "stadia of discourse" is a
better unit than the somewhat arbitrary paragraph indention. The "stadia,"
according to Rodgers, is a sentence or group of sentences that contain a single topic
which may or may not be contained in a single paragraph. This idea is particularly
important when working with journalistic writing. Paragraphs in a newspaper or on
an audio script are frequently shorter to accommodate the requirements of the
newspaper's narrow columns or the readability for a television or radio reporter. The
variable of interest for this study was units of meaning, sentences or groups of
sentences that share a common topic.

An ANOVA was performed on the revision data that, in effect, combined the four
classes into a single group for statistical purposes (Ott, 1988). This method is
appropriate because the students were not assigned randomly to the classes used in
the study. The analysis examined the four treatments (marginal and end comment,
marginal comment only, end commend only and oral comment only) and the four
revision types (additions, deletions, substitutions and rearrangements) to determine
if there were any significant differences between the treatments and the outcomes.
In the analysis, differences with p <.10 are considered significant. This significance
level was used to help offset Type II error that could easily result from the relatively
low number of subjects, the imprecise measurement methods, and the exploratory
nature of this research (Lauer and Asher, 1988). Next, using percentages and graphs,
the data were analyzed for similarities and differences among the combination of
treatments and the resulting revisions. A naturalistic inquiry method was used to
examine the relationship between specific instructor comments and the specific
revisions that resulted from that comment (Lincoln and Guba, 1984). To analyze the
data, teacher comment that was written on the student texts or given in the oral
comments were recorded and written on individual index cards. The cards were
sorted into groups of those with similar meanings using Lincoln and Cuba's ( 1984)
method. The groups were then cross checked and again collated into groups by
meaning or the problem they addressed. Seven groups were established, each of
which addressed a different aspect in the texts (Table 1). At this stage in the study
there was no differentiation made between oral and written comments, as those
distinctions were covered in the quantitative phase of the study.

Results and Discussion

Analysis of the teacher comment types resulted in seven comment types: positive
comments, overall quality of all or a section of text, material that does not belong in
the text, material that is out of place, wordiness or over length, wording or sentence
needs work, and meaning is unclear. Studentresponses were examined based on
each comment type.

"Positive comments," as might be expected, did not result in a lot of revisions,

although some students did revise some of these sections of their texts. All of the
revisions resulting from these comments were improvements. Comments on the
"overall quality of all or a section of the text" asked for big changes.
Generally student response to these comments was deleting or substituting material
in their texts, which is not surprising because the comments frequently
related to coherence or focus. Again, the student revisions associated with this
comment were generally improvements, but in some cases deleting material
weakened the story.

Student responses to the comment that to "material that did not belong in the text"
also resulted in deletions. Students tended to act more frequently on some of these
comments than others because a few of the comments offered more specific
instructions ("Doesn't go in the story" vs. "How does this fit?"). Therefore, some of
the comments were not acted on by the students, probably because of uncertainty of
how to solve the problem (Flower et. al., 1986). Generally revisions made in
responseto this comment group improved the student texts. "Material that is out of
place" comments relatedto organization problems in student texts and usually
suggested that the material belonged in the story, just not where the student had
placed the information. As expected, students generally optedto rearrange their texts
in response to this comment. Specific comments in this category resulted in more
revisions that improved the texts than did less instructive comments ("Move Up" vs.
"Out of place"). Students responded frequently to the less specific comments by
removing the material.

Comments on "wordiness" and "wording or sentence needs work" problems

frequently resulted in students deleting material from their texts. However, many
students did a good job of combining sentences and paragraphs to tighten up the text
and reduce the paper's overall length. Getting just the right word can be a particular
problem for student writers and comments in this area included "awkward,"
"choppy," or "vary sentence structure." Responses to these comments frequently
resulted in fine tuning rather than fixing structural or coherence problems. Many
revisions in response to these comments included combining sentences and altering
sentence structure. A few resulted in deletions, but there were more substitutions
used in response to this comment than toother comments. Again, the more specific
the instructions, the more often the students revised successfully. "Unclear meaning"
comments usually refer to the need for more information including specific detail or
other clarifications. Some of the comments went so far as to ask for specific
numbers or other specific information, others were "vague" and "confusing."
Revisions resulting from this comment group varied including deletions and
additions. The quality of the students' revisions also varied.

Results from the qualitative portion of this study indicate that the more directive
the teachercomment on student texts, the more successful student revisions will be
on the text. Students tended to respond to teacher comment if they knew
how to make the requested change or improvement. If they did not know
how to make the change or how to improve the text, they frequently deleted the
material or ignored the comment. According to Spandel and Stiggins (1990),
students frequently misread instructors comment and fail when they are
trying to revise their texts. Occasionally students would substitute material, which
ultimately resulted in a few additions. There were few rearrangements and those
changes were usually in response to a specific comment toalter the order of ideas in
paragraphs.

In response to one of the main questions of this study, teacher comment "Does
influence the choices students make in revising their texts," and a second question
"Does the lack of teachercomment influence student revision?" Indications from the
qualitative portion of this study are that students are even more likely to make
revisions in the absence of written comment when oral only comment is presented.
As with the other student responses to teacher comment, students perceive a benefit
from revising their texts based on the incentive of an improved grade.

The F test included all 64 of the students in the study combined into one large
group. This option was chosen to maintain the natural classroom environment as
much as possible. The data were coded by treatment and by revision outcomes.
Based on this analysis, the only significant outcome at p <.10 was deletions. A
Scheffe S test showed that marginal comment and oral comment only treatments
were similar for deletions, as were marginal and end comment and end comment
only treatments. However, marginal comment and oral comment only treatments
were significantly different than marginal and end comments and end comment only
treatments. This means that the students' responses to each pair of treatments were
similar, but that they responded differently to the treatments not contained in each
pair.

The significance of deletions and the relationship between the two pairs of
treatments provides several options for interpretation. Flower et al. (1986) suggest
that if students do not know how toaddress a problem, they will frequently delete the
material. That is likely the case in this study. second, the similarities between
responses to marginal and end comment and end comment only suggest that
students may be reading and interpreting these comments in much the same way.
The same should be said for the other pair of treatments, marginal only comment
and oral comment. Examining the means of these two treatments, .38 for marginal
only and .51 for oral only, indicates that students made fewer deletions on average
in response to these two comments than for the other comment pair. Deletion means
for marginal and end comment and end comment only were .74 and .81
respectively. The students made more deletions based on these two treatment types.
One obvious similarity between thee two treatments is that they both include
comments on the students' texts. This may indicate that students either read these
comments more often or that they somehow responded to these comments
differently than they did to marginal only comment or oral only comment at least
when it came to making deletions in their texts.

Although the F test showed a significant difference for only deletions, the
descriptive statistics associated with the various treatment totals are worth
discussion. Of the 307 total revisions by type made by students in this study, 109, or
35.62%, were deletions; 85, or 27.28% were substitutions; 59, or 19.28%, were
additions and 55, or 17.32%, were rearrangements. Total revisions broken down
by teacher comment location (Table 2 and Figure 1) were marginal and end
comment, 83; end only comment, 80; oral only, 83 and marginal only, 61. The
primary difference is in the revision types, with deletions showing a much higher
incidence than substitutions; additions and rearrangements are fairly even at the
lower end of the range. The high number of deletions is not unexpected (Flower, et
al., 1986)

Comparing comment location by revision type also provides an interesting

discussion. For marginal only comment, there was a relatively low overall number
of revisions and an even distribution (16 additions, 16 deletions, 15 substitutions,
and 14 rearrangements a range of 2). One possibility for this relatively low number
of revisions with this comment location relates to the lack of space available for
providing feedback. Another reason may be that students do not read these
comments. Bolker ( 1978) suggested that students dissociate themselves
from teacher comment because they fear disappointment. Also, while end comment
often points out problems in a text, it is frequently tempered with positive comment
and is sometimes less directed at a specific point or error in the text (Smith, 2004).

Oral only comment elicited a relatively large number of revisions ( 18, 24, 24, and
17 a range of 7). This number of revisions is somewhat higher than anticipated at
the outset of the study. One of the theories was that students receiving only oral
comment on their texts would revise less because of the somewhat fleeting nature of
the feedback. However, information on the audio tapes of the oral comment sessions
suggests one reason for the unexpected strength of this response. The written notes
for these sessions look, for the most part, like a laundry list of what was wrong (and
occasionally right) with the class' texts. However, the audio tapes of these sessions
include not only the comments on the problems, but usually examples of all or most
of the problems pulled fromstudent papers. The instructor did not return the
students' texts until the end of the class period in an attempt to keep the students'
attention on that day's material. Therefore, when the instructor went through the
comment list, the texts were readily available for discussion. No student names were
mentioned, but the students did ask questions and apparently, from the number of
revisions on their texts, were able to make use of the information.

Conclusions and Implications

Based on the results of this study, teacher comment influences student revision
choices and the more directive the teacher comment, the better chance the students
will revise their texts successfully. This agrees with Flower et al. (1986), Newkirk
(1981) and Shuman (1975), but this study builds on their work by providing specific
illustrations of teacher comment that offers problem identification and revision
strategies paired with actual student revisions.

The placement of written teacher comment does have some influence

on student revisions. In this study, there were more total revisions associated with
oral only comment than the other three types. The previously mentioned audio tapes
indicate that the oral comment sessions frequently included multiple examples of a
problem and multiple solutions. These additional examples may be part of the
reason for the additional revisions. Another possibility may be Bolker's ( 1978)
suggestion that students fear teacher comment and, because the oral comment is less
direct, it is therefore less threatening and students are more apt to listen.
Oral feedback may help build a sense of community rather than force
students to view problems in the texts as theirs alone.

The combination of research methods used in this study added strength to the
conclusions of both portions of the research. For example, deletions were the only
statistically significant response in the experimental study. This outcome could be
explained more clearly using results from the naturalistic inquiry portion of the
study. Matching specific teacher comments with specific revisions revealed that
many of the comments suggested or hinted at deletion as a revision option. Also, the
results of both portions of the study pointed to the importance of more
detailed teacher comment either in the form of more revisions associated with
concluding comments on the texts or the more frequent and more successful
revisions from specific comments on the texts.

Several alterations would enhance future studies using this method. First, the use of
Rodger's (1967) stadia of discourse for identifying changes in the texts was a little
too coarse for the revisions involved. Reviewing changes at the word or phrase level
would be potentially more accurate. Identifying a way to limit the variety
of teacher comment statements by using a check sheet or other method would better
focus the naturalistic inquiry portion of the study and help further identify which
comments elicited successful revisions from the students.

Implications for future research include further examination of oral comment as

a feedback method on student papers. The potential time savings
for writing instructors, as well as the possibilities of greater improvement
in student writing make this an important area for future study. Continued
evaluation of written comment would also be extremely valuable. Careful evaluation
of written comment location and content could lead to better writing and better use
of instructor's time. Finally, one of the major goals of providing comment
on student texts is to help the students learn tointernalize the ability to evaluate their
own texts. Identifying feedback methods that can help students learn to evaluate
 their own writing more successfully will enable them to become better writers.

Agricultural Communications programs frequently offer writing courses as either

part of their curriculum or as a service to their colleges. Providing efficient and
timely feedback on these papers is an increasing challenge as faculty work toward
tenure or promotion with ever-growing studentdemand. Refining our methods of
providing students with feedback on their papers will ultimately improve our
students' writing ability while still making the most efficient use of faculty member
time and resources the best of both worlds for all concerned.

-1-
Questia, a part of Gale, Cengage Learning. www.questia.com

Publication Information: Article Title: Feedback Follow Up: the Influence of Teacher Comment on Student
Writing Assignments. Contributors: Deborah W. Dunsford - author. Journal Title: NACTA Journal. Volume: 50.
Issue: 2. Publication Year: 2006. Page Number: 12+. © 2006 North American Colleges and Teachers of
Agriculture. Provided by ProQuest LLC. All Rights Reserved.

Examining the Effect of Feedback inBeginning L2

Composition

by Carolyn Gascoigne

Introduction

While reviewing research on English-as-a-second-language (ESL) writing, Goldstein

(2001) articulated the following simple, yet unresolved question: "What role
does teacher commentary playin helping students become more efficient writers?" (p.
78). Regardless of the context for writing (Ll, ESL, or L2), "teachers and students
alike intuit that written responses can have a great effect on student writing and
attitude" (Leki, 1990, p. 58). Some of us even prefer to believe that
studentwriting improves "in direct proportion to the amount of time [we] spend on
their papers" (Hariston, 1986, p. 117).

The illusion that the time we invest in correcting and commenting on

student writing has a perfect and positive correlation to the quality of a student's final
product is now under attack. Indeed, years of working as "composition slaves"
(Hariston, 1986) has not produced the results that the countless hours of reading,
correction, and commentary would demand. Instead of a simple equation wherein a
given amount of feedback equals a predictable amount of improvement, Leki (1990)
painted a more realistic picture in which she described writing teachers as those who
"behave with the same combination of a sense of responsibility and a sense of
helplessness as a coach of a football team, booing and cheering while
pacing the margins of the student's paper ... or like a coroner diagnosingthe cause of
death" (p. 57).

Given that providing written feedback and writing evaluative commentary is one
of the "great tasks" (Gonners & Lunsford, 1993, p. 200) both quantitatively in terms
of sheer number of hours, and qualitatively in terms of personal investment, one might
think it would also be a central area of examination. Although there has been a
growing body of literature devoted to the impact of peer response on ESL student
revision, studies of teacher response and its effects on revision have been few (Ferris,
1997). According to Leki (1990), there may be a fairly large amount of information
examining the type of teacher response in Ll writing, yet "examples of feedback and
subsequent student action are rare" (p. 64) and studies of teachers' responses in the L2
setting are "practically nonexistent" (Zamel, 1985, p. 83). For example, while
reviewing all published investigations of teachers' written commentary on rhetoric and
content in ESL and L2 settings, an area of research that did "not really begin
until the 1990s" (p. 75), Goldstein (2001) uncovered a paltry 15 studies. Of these 15,
only 4 looked at the relationship between teacher-written commentary and either
subsequent student revision or essay scores. The 11 others examined student
perceptions of commentary or the type of teacher feedback on final drafts.

In an attempt to address this lacuna, the present study replicated one of the few ESL
investigations that examined the effect of teacher feedback on subsequent composition
revisions. The major distinction, however, was that the present study focused its
attention on a beginning L2 writingpopulation. Results of this investigation will help
determine whether or not the ESL findings are unique to that particular population, or
are more universal in nature.

Type of Feedback

In the undeniably few studies examining teacher feedback, there are several trends
that emerged. First, there has been considerable debate over
where teacher/commentator attention should be placed: on form or content (Hall,
1990). To help inform this debate, Fatham and Whalley (1990) compared revision
scores on both form and content among four groups of ESL writers: Group 1 received
no feedback whatsoever; Group 2 received grammar or formfocused feedback only;
Group 3 received content feedback only, and; Group 4 received feedback on both
grammar and content. Theauthors expected that students would focus on different
aspects of their compositions depending upon the type
of feedback that the teacher provided (p. 182). After scoring original compositions
and revisions, the authors found that students made statistically significant
improvements in grammatical accuracy only when they were given
explicit feedback on grammar. More surprising, however, wasthe finding that all
groups significantly improved the content of their compositions irrespective of thetype
of feedback received. In other words, students improved the content of their revisions
even when teachers provided no feedback concerning the content of the original essay.
Specific feedbackon grammar errors appeared to have a greater effect on grammar
revisions than general content comments had on revisions of content.
Nevertheless, the authors concluded that feedback on grammar and content, whether
given alone or simultaneously, both had a positive effect on revision (p. 185).

Despite these encouraging findings, many researchers have lamented the focus that L2
teachers tend to place on form, or surface-level features, rather than content. For many
of us, the practice of calling attention to error is still the most common procedure for
responding to ESL and L2 writing(Cohen & Cavalcanti, 1990; Ferris, 1995; Leki,
1991; Zamel, 1985). For Gumming (1983),

Error identification appears to be ingrained in the habitual practices of L2 teachers

who perhaps by reason of perceiving their role solely as instructors of the formal
aspects of language, therefore restrict their activities to operations exclusively
within the domain of formal training, rather than that of cognitive development (p. 6).

Zamel (1985) believed that this trend is due to the fact that ESL (and L2) teachers
overwhelmingly view themselves as language teachers, rather than writing teachers.

A second major criticism found in the literature

concerned the manner in which feedback (form or content focused) is supplied. Even
among Ll studies, the research has criticized teachers for being "too general (rubber-
stamping students' papers with remarks like 'be specific'), for being too specific
(giving students advice that is so text specific that they cannot use
it in subsequent writing), and for focusing too heavily on surface features" (Goldstein,
2001, p. 60). Others, such as Burkland & Grimm (1986), lamented the futility of
providing feedback because many Ll students "read the grade and simply
discard the paper" (p. 62).

Summarizing the situation, Zamel (1985) provided an extensive list of

typical feedback problems:

(1) teachers respond to most writing as if it were a final draft, thus reinforcing an
extremely constricted notion of composing;
(2) teachers' marks and comments usually take the form of abstract and vague
prescriptions and directives that students find difficult to interpret;

(3) teachers (especially ESL and L2 teachers) are often more concerned
with language-specific errors and problems and rarely expect students to revise
beyond the surface level;

(4) marks and comments are often confusing, arbitrary, and inaccessible;

(5) teachers appropriate students' texts and impose their own purpose and ideas on
students;

(6) teachers send mixed messages to students by addressing minor surface features
and larger issues of rhetoric and context in the same version of a text. For example,
mechanical errors might be pinpointed at the same time that students are asked to
elaborate upon an idea; and

(7) teachers often fail to provide explicit, text-specific directions, guidelines, and
revising strategies, (p. 79-82)

Hearing the resounding criticism of both the type and the shape of feedback given to
student writers, one must begin to question the value of composition correction
and feedback altogether. (Burkland & Grimm, 1986; Hillocks, 1982; Knoblauch &
Brannon, 1981; Leki, 1990).

On a more optimistic note, surveys of L2 writers (Cohen, 1987; Cohen & Cavalcanti,
1990; Ferris, 1995; McCurdy 1992) have revealed that L2 writers are generally
"happy with the feedback they receive, and claim that they pay attention to it and find
it helpful" (Ferris, 1995, p. 36). ESL writers inFerris's study, for example, reported
wanting and paying the most attention to comments on grammar (67%), with attention
to content-oriented feedback close behind (63%) (p. 40). This
enthusiastic feedback by ESL writers is much more positive than that revealed by Ll
surveys. Indeed, Leki (1991) found that her ESL students equate
good writing in English with error-free writing and both "want and expect their
teachers to correct errors" (p. 203), whereas Ll students reported not paying much
attention to teacher commentary, not understanding it, or feeling some hostility.
Inspired by these encouraging findings concerning ESL and L2 students' self-
perceived acceptance of and positive attitudes toward teacher feedback, Ferris (1997)
sought to examine more closely theinfluence that teacher commentary actually had on
ESL student revisions. To this end, Ferris examined 1,600 marginal and end
comments written on 110 first drafts of papers by 47 advanced ESL students. She then
examined the 110 revised drafts in an attempt to measure the influence, if any,
that the commentary had on subsequent revisions, and to see whether the changes
actually led to improvements. Her specific research questions were:

(1) What characteristics of teacher commentary appear to influence student revision?

and

(2) Do revisions influenced by teacher feedback lead to substantive and effective

changes instudents' papers? (p. 317)

Two sets of analyses were completed. The first examined the feedback provided
by the teacher andthe second measured the effect the feedback had on the revision
process. The comment analysis specifically targeted feedback length, feedback type
(i.e., making a request or giving information),the use of hedges (e.g., "maybe,"
"please," "might"), and text-based versus general commentary.The effect
of the comments on revision was assessed according to an original 0-6 rating scale (O
= no discernible change, 6 = substantial positive change).

Ferris found that marginal requests for information, general requests (regardless of
syntactic form), and summary comments on grammar led to the most substantive
revisions. The use of hedges did not inhibit effective revisions as anticipated. In fact,
they seemed to encourage positive change. Less influential were questions or
statements that provided information to the students and positive comments. Length
of feedback appeared to be an important variable. In general "longer comments and
those which were text specific were associated with major changes more than were
shorter, general comments" (p. 330). Moreover, when either minor or substantive
changes occurred, they overwhelmingly resulted in an improvement of the students'
papers. Very few of the changes (less than 5%) were found to have a negative impact.
Finally, text-specific comments resulted in more positive changes than general
comments.

Ferris concluded that not all feedback is equal and certain forms may be particularly
difficult for students to interpret. She also found that although students generally
addressed comments givenin the form of a question, the changes that resulted had
mixed effects on the revisions. This suggested "that although the students appeared to
understand from the comment that something was required of them, they were less
clear about how to incorporate the requested changes successfully" (p. 331). A sec ond
problematic type of comment was the "give-information comment" (e.g., "Iowa law
favors parental rights. Michigan and California consider the best interests of thechild."
p. 321). Often, this type of comment did not lead to change or produced mixed
results in therevision when it did. Ferris indicates that the give-information comment
is less effective because it does not explicitly instruct the writer to
incorporate the information that is supplied.

Given that the manner in which teachers comment and, more important, the effect of
comments on subsequent revisions has gone largely unexplored (Conners & Lunsford,
1993; Goldstein, 2001; Leki, 1990), and that this lacuna is especially
acute in the nonnative writing environment (Ferris, 1997; Zamel, 1985), Ferris's study
was an essential one. However, Ferris examined an advanced ESL writing population
only. Would her findings still hold in an L2 context, in particular an introductory L2
environment? Or are the effects of feedback context specific? If Ferris's findings were
not applicable to an L2 context, then a host of studies targeting a range of abilities and
languages would need to be run before firm generalizations could be made. The
purpose of the present study, therefore, was to examine the effect of feedback on
subsequent revisions in the beginning L2 environment. The results of this effort
should be of practical interest to the L2 teacher, as well as help delineate or reinforce
the applicability of Ferris's findings.

Method

Instead of looking at the influence of teacher commentary in advanced ESL students'

revisions, the present study attempts to measure the influence that commentary has on
subsequent composition revisions in a beginning L2 French class at the postsecondary
level. Thus, the primary research questions are:

(1) What characteristics of teacher commentary appear to have the greatest effect on
beginning L2 composition revisions, and

(2) Do revisions prompted by teacher feedback lead to substantive and effective

changes inbeginning L2 students' papers?

Participants

Twenty-two of the 25 subjects were freshmen; 19 were female and 6 were male. all
participants were native speakers of English who had either no formal exposure to
French prior to this course, or were placed into the beginning course as the result of
their score on a standardized placement exam.

The Course

The introductory French course was part of a four-semester language requirement.

The course focused on speaking, writing, reading, listening comprehension, culture,
and grammar. The Vis-a-Vis (Amon, Muyskens & Omaggio Hadley, 2000) first-year
textbook program was used. This course met for 80 minutes, three days a week for 16
weeks. Other than the formal writing activities discussed below, writing was primarily
used as a support skill (i.e., filling in blanks for grammar activities, creating
vocabulary lists).

The Writing Tasks

Writing activities (one per chapter, eight chapters total) came directly from the
textbook so that students were exposed to the vocabulary, structures, and themes
essential to the writing task. Each in-class writing assignment began with a prewriting
activity consisting of 3-5 guiding questions. Next, students were to draft a short
composition (average composition length was 92 words) based upon the ideas
generated in the prewriting activity. Students were mainly asked to compose
descriptive texts (e.g., describe your best friend; describe the house of your dreams;
talk about your family; describe your eating habits). Students had fifty minutes to
complete the prewriting andwriting tasks.

Procedure

Only compositions from chapters 4-8 were selected for examination. The first
three writing activities, which averaged only 37 words, were found to be too short for
meaningful analysis. Ultimately, 516 marginal and end comments written on 114 first
drafts of papers by 25 beginning French languagestudents at the University of
Nebraska at Omaha were examined and catalogued by the investigator and
a second independent reader according to the following features identified by Ferris
(1997):

(1) Comment length in number of words;

(2) Comment type (pragmatic intent and syntactic form) ;

(3) The use of hedges (e.g., please, maybe, perhaps), and;

(4) Whether the comment was text based or general, (p. 320)
Comment type was further broken down into the following categories:

(a) Ask for information/question (e.g., Did you consult any sources?)

(b) Make a request/question (e.g., Can you provide an example here?)

(c) Make a request/statement (e.g., This would be better earlier in the essay.)

(d) Make a request/imperative (e.g., Add a citation.)

(e) Give information/question (e.g., The first case was in 1899. Does this change your
view?)

(f) Give information/statement (e.g., The first case was in 1899, not 1919.)

(g) Make a positive comment, statement, exclamation (e.g., This is a great start.)

(h) Make a grammar/mechanics comment, question, statement, or imperative, (p. 322)

Papers with comments were returned to students for revision. Students had two days
out of class to revise their compositions. The 114 revised drafts were then examined'
to measure the influence of the commentary. Revisions were assessed by the
investigator according to Ferris's (1997) scale:

O No discernible change made by student in response to comment;

1 Minimal attempt to address comment, effect generally negative or negligible;

2 Substantive change in response to comment, effect generally negative or negligible;

3 Minimal attempt in response to comment, effect mixed;

4 Minimal attempt to address comment, effect generally positive;

5 Substantive change in response to comment, effect mixed;

6 Substantive change in response to comment, effect generally positive, (p. 322)

A 10% sample was verified by a second independent reader. Interrater reliability was
96%.

Findings

Feedback Type

Although no feedback or correction code was used, there was relatively little
variation in the type and shape of the teacher's comments. First, all feedback was
given in the Ll (English). second, nearly all comments were direct and succinct with
an average comment length of only four words. Third, 398 of the 516 comments were
text based rather than general, and there were only three cases where a hedge was
used. As for comment type, there were no "asking for information" questions, no
requests in the shape of a question, and no requests for information in the shape of a
statement. Instead, there were 64 examples of a request in the shape of an imperative.
For example, a request for detail was directly stated, "Add more detail here," rather
than politely requested, "Can you add more detail here?" or "This would be better with
more detail."

There were no cases where factual information was provided to the writer either in the
form of a question or a statement. There were 118 examples of a positive comment,
statement or exclamation (e.g., "This is great!"), and 334 "comments, questions,
statements, or imperatives" (Ferris, p. 322) focusing on grammar or mechanics, nearly
all of which were supplied as an imperative. Examples of form-
focused feedback include "Pay attention to verb endings," or "Don't forget
agreement." There were no codes, symbols, or systems used in conjunction with the
commentary. However, an arrow was often drawn to link feedback to the phrase or
sentence in question.

Comment Effect

Ferris's (1997) scale was used to measure the influence of the 64 "requests/imperative"
comments. Using the 0-6 scale, the average rating was 4.4, or "minimal attempt to
address comment, effect generally positive." There were two cases where "no
discernable change was made by the studentin response to the comment," and two
cases where there was a "minimal attempt to address the comment, with an effect that
was generally negative or negligible." There were 12 cases where the revisions
showed evidence of "minimal attempt in response to commentary with a mixed effect"
and 14 examples of a "minimal attempt to address comments with a generally positive
effect." There were 16 cases where there was "substantive change in response to
comment with mixed effect," and 18 cases where there was "substantive change with a
generally positive effect."

The 118 positive comments elicited no change whatsoever in any of the

revisions. In contrast, the 334 comments devoted to grammar and mechanics had a
profound effect. Eighty-eight percent of all such comments led to a successful
correction, 8% led to an incorrect change, and a mere 3% were ignored by the
students.

Discussion

Several differences in the outcome of the present study and that of Ferris (1997) are
worth noting. First, Ferris examined 110 sets of first and second drafts producing
1,600 marginal and end comments. The present study targeted 114 sets of first
and second drafts that yielded a mere one third (516) of the marginal and end
comments produced in the Ferris study. The much smaller number of comments in the
L2 context is likely due to the brevity of the L2 compositions. However, this is only
speculation given that the average length of the ESL compositions is not
known. second, whereas Ferris found that the use of hedges led to positive
change in the students' revisions, no conclusions or comparisons stemming from the
present study can be drawn due to the rare use of hedges (three total). Similarly, Ferris
found that length of feedback correlated positively with the success of the
revision. In the present study, all comments were remarkably terse (the average length
was four words), yet feedback still led to a large number of successful revisions.
Finally, no comparisons of comment type can be made among the present data or to
those of the Ferris study because nearly all comments took the shape of a grammar-
focused imperative or a positive comment, statement, or exclamation.

Similarities among the outcomes of the two studies exist as well. For example, just as
Ferris found that textspecific comments produced more positive changes, so did the
present investigation. Also, Ferris found that less than 5% of the student revisions led
to a negative change. Similar results were found in the present study where 8% of the
changes were incorrect.

Conclusion

The copious occurrence of brief form-focused comments in this study unwittingly

served to reinforce the stereotype of the L2 teacher as one concerned
with language rather than composition, and form over meaning. In the words of Zamel
(1985) "teachers (especially ESL and L2 teachers) are often more concerned
with language-specific errors and problems and rarely expect students to revise
beyond the surface level" (p. 79). Certainly, the level of L2 ability examined here
(first semester) is one at which one expects to find a great deal of mechanical errors. It
is also a context in which students typically "want and expect teachers to correct their
errors" (Leki, 1991, p. 203). The clear focus on grammar and mechanics and the
simple and direct feedback provided is not entirely surprising, yet it does limit the
extent to which comparisons with the Ferris study can be drawn. Unfortunately, this
investigation did not yield meaningful data concerning the effect of contentfeedback-
other than general positive comments, statements, and exclamations-or the effect of
different comment types (statement, question, request, or provision of information). It
did, however, reveal that beginning L2 writers
incorporate teacher feedback concerning grammar and mechanics and that they tend to
do so successfully. Even at the earliest stages of E2 instruction, teachers need not feel
that they must supply surface-level corrections to students during the revision process.
Indicating that a mechanical error has taken place through the use of a short and direct
imperative or statement appears to suffice.

Although further study is still desperately needed concerning less form-

focused feedback, the present study demonstrates encouraging findings concerning the
effectiveness of what-right or wrong-is still perhaps the most ubiquitous type of L2
composition feedback. For those who view themselves as language teachers rather
than writing teachers, these findings imply that L2 students do successfully
incorporate short and direct form-focused feedback. Moreover, knowing that students
are capable of such revisions, and that even first-semester L2 students can
successfully revise at the surface level, should encourage teachers to expand
their feedback repertoire even further.

-1-
Questia, a part of Gale, Cengage Learning. www.questia.com

Publication Information: Article Title: Examining the Effect of Feedback in Beginning L2 Composition.
Contributors: Carolyn Gascoigne - author. Journal Title: Foreign Language Annals. Volume: 37. Issue: 1.
Publication Year: 2004. Page Number: 71+. © 2004 American Council on the Teaching of Foreign Languages.
Provided by ProQuest LLC. All Rights Reserved.

Effects of Response Mode and Time Allotment on

College Students' Writing.

by Benjamin J. Lovett , Lawrence J. Lewandowski , Cassie Berger , Rebecca A. Gathje

Written essays area common feature of classroom and high stakes standardized tests at
many age levels. However, little is known about how small alterations in the writing task
affect students' writing, an issue made more important by the increasing use of task
alterations for students with disabilities. In the present study, 140 college students
completed a standardized assessment of writing ability under one of two timing
conditions (10 minutes, 15 minutes) and with one of two response formats (handwritten,
word-processed). Students who used a word processor wrote significantly more than did
students who handwrote their essays. However, the extended time allotment only
increased students' amount ofwriting in the word processor condition. Only small
differences between the groups' essays were found in spelling, grammar, and related
characteristics. Implications of these results for future research and writingassessment
policies are discussed.

Traditionally, two types of test items have been used in educational assessment: items in
which thestudent selects the correct response from a set (as seen in multiple-choice, true-
false, and matching items) and items in which the student constructs a response on his of
her own (as in short answer or essay items). The latter type of test item has become more
common in large-scale standardized testing for several reasons. First, writing samples are
thought by many to be the best method of assessing writingability (Conlan, 1986; Linn &
Miller, 2005). Second, computer programs have been developed to scorewriting samples
(Dikli, 2006), reducing the financial and logistical challenges associated with this type of
assessment. Third, multiple-choice tests have continued to raise concerns about the
ability to measure complex reasoning and problem solving (for a review, see Phelps,
2003).

Although essay tests have certain acknowledged advantages over selected-response tests
(e.g., utility in assessing certain higher-order learning objectives), essay tests also have
potential limitations. One such limitation is that the response mode of the test may
significantly affect examinees' scores; composing an essay using a computerized word
processor program may lead to a different score than composing an essay by hand
(Russell, 1999). A second limitation is that time limits, which determine the amount of
text that students can compose, may significantly affect students' scores, since the amount
of text written is a robust correlate of holistic measures of essay quality (Hopkins, 1998;
Powers, 2005). In the present study, we explore these issues empirically, asking how time
limits and response modes interact to affect students' essay composition.

Before reviewing relevant literature on writing assessment, this study briefly discuss one
of the motivations behind the study. An increasing number of students in higher
education have diagnoses of common disabilities (e.g., learning disabilities, attention
problems, psychiatric disorders, etc.) that may adversely impact their scores on
standardized tests (Konur, 2002). In many countries, alterations are made to the
administration of the tests (for example, testing accommodations) in the hopes ofgiving
these students a fair chance at showing their skills (Bolt & Thurlow, 2004; Hampton &
Gosden, 2004). Extending test time limits and allowing examinees to use
computers to write are among the most common accommodations offered, but little is
known about how these accommodations affect essay examinations.

An appropriate test accommodation should mitigate performance obstacles of students

with disabilities (e.g., large print for a student with visual limitations), while having less
of an effect on the performance of non-accommodated students (Fuchs & Fuchs, 2001).
However, many testing accommodations provide at least some benefit to students both
with and without disabilities (Sireci, Scarpati, & Li, 2005). Indeed, some work shows that
students who have poor academic skills but no disability diagnoses benefit more from
accommodations than those with official diagnoses do (e.g., Elliott & Marquart, 2004).
The present study, then, was conducted in part for its potential implications for the use of
extended time and word processor accommodations, both for students with and without
disability diagnoses.

Response Mode and Writing Performance

A number of studies have compared writing produced with and without the aid of a word
processor. In a recent meta-analysis, Goldberg, Russell, and Cook (2003) examined 26 of
these studies published between 1992 and 2002 using students in K-12 educational
settings. These investigators concluded that word processors lead reliably to writing in
greater quantities (with a weighted effect size of d = .50) andto writing of better quality
(judged using a variety of measures, depending on the study; weighted effect size of d = .
41). These positive effects are consistent with results from earlier research (summatized
in Bangert-Drowns, 1993), as well as the few studies that have been published since
Goldberg and colleagues' meta-analysis (e.g., Burke & Cizek, 2006).

Variability in word processing's effects has led researchers to search for potential
moderators and mediators. Student age appears to be one moderator: Goldberg et al.
(2003) found, in regard to both quantity and quality of writing, that middle and high
school students benefited more from the use of word processors than elementary school
students. Computer skills, a related variable, may also determine students' degree of
benefit; Russell (1999) found that middle school students with above-average
keyboarding skills benefited substantially from word processing, whereas students with
below-average skills actually showed a decrease in essay performance. Similarly, a recent
study by Burke and Cizek (2006) found a set of complex interactions
between response mode and self-perceived computer skills on the essays written by sixth
graders in which higher computer skills were generally associated with more benefit from
word processing.

One issue in measuring the effects of response mode on writing performance has been
rater bias in the assessment of writing composed using different modes; that is, it is
reasonable to suspect that judges who score essays may be more generous to essays that
are either handwritten of word-processed. Research has generally shown that judges are
more generous to essays that are handwritten. In a landmark study on the topic, Powers,
Fowles, Farnum, and Ramsey (1994) asked college students to write two essays on
different topics, one in handwritten form and one in word-processed form. The
handwritten essays were then word-processed exactly as originally written, without
correcting for misspellings and grammar, and the word-processed essays were written
down on paper. These investigators found that when original handwritten essays were
word-processed and scored again, the scores decreased significantly, and when the
computer-produced essays were handwritten and scored again, the mean increased. In
qualitative assessments, raters reported that word-processed essays appeared to be shorter
and claimed to give handwritten essays the "benefit of the doubt," especially when the
handwriting was poor. Other studies on the topic have found similar results (e.g., Arnold
et al., 1990; MacCann, Eastment, & Pickering, 2002), suggesting that all essays should be
presented in the same way (either typed or handwritten) to scorers to avoid rater bias
effects.

Time Allotment and Writing Performance

The substantial relationship between essay length and quality is well known (e.g.,
Hopkins, 1998; Powers, 2005). Time limits clearly have the potential to serve as ceilings
in determining essay length. However, few studies have directly examined the effects of
time allotment on writing. Many studies have focused on the speededness of tests--that is,
the degree to which scores on tests are the product of working quickly (Lu & Sireci,
2007)--but the tests in these studies are rarely measures of writing skill. In their review of
the small extant literature on time allotment and writing quality, Powers and Fowles
(1996) noted that time allotment has generally, but not always, been
found to increase writing performance. In their own comprehensive study, these
investigators assigned college students randomly to 40-minute or 60-minute conditions,
finding that scores of the examinees in the 60-minute conditions were approximately one
half of a standard deviation higher than scores of examinees in the 40-minute conditions;
extended time allotments, then, had a meaningful effect on scores.

In a more recent study, Crawford, Helwig, and Tindal (2004) examined several features
of fifth- and eighth-grade students' compositions when the students were given either 30
minutes or three days tocomplete a composition assignment. Each student completed one
assignment under each of the two time conditions. Crawford and colleagues found that
although fifth-graders' three-day compositions were superior to their 30-minute
compositions, no such effect was present for eighth-graders, suggesting that age may be a
moderator of timing effects. In addition, time had a greater effect on the scores of
students who were receiving special education services, even though the nondisabled
fifth-grade students' essays also showed a (smaller) benefit. Of course, giving examinees
three days to complete an assignment raises other concerns, since examinees will clearly
not be spending the entire interval working on the assignment. In our study, we felt it was
important to keep all time intervals brief enough that examinees could be expected to be
working steadily throughout the intervals.

The Present Study

If response mode and time allotment have been found to affect the writing performance
of students of varying ages and education levels, it is reasonable to ask how these factors
might interact. That is, does the effect of time depend on the response mode, and vice
versa? This question is important because many students with disabilities are given
alterations in test administration that include simultaneous
changes to both response mode and time allotment. In addition, more generally, high
stakes tests (e.g., college and graduate admissions tests) and classroom assessments both
vary substantially in time allotment and response mode. The present study investigated
this issue in a sample of college students whose writing was evaluated under varying
conditions.

Method

Participants

The participants were 140 college students (73 females, 67 males) taking ah introductory
psychology course at a large, private university in the northeastern United States. They
ranged in age from 18 to 22 years old (M = 19.1), and most (75%) were either first- or
second-year students. The participants' college grade point averages (GPAs) ranged from
2.0 to 4.0 (M = 3.2). They completed testing in small group sessions. Each session was
randomly assigned, using one of four conditions: hand writing for 10 minutes (HW-10),
hand writing for 15 minutes (HW-15), word-processed writing for 10 minutes (WP-10),
and word-processed writing for 15 minutes (WP-15). This assignment resulted in four
groups of participants (ns ranging from 29 to 42) that did not differ significantly with
respect to gender composition ([chi square] = 1.04, p = .80), age (F = 1.38, p = .25), year
in school (F = .30, p = .83), or college GPA (F = .84, p = .48; for more details on these
data, see Table 1).

Measures

The study included three primary measurement tools: one assessed participants' motor
speed in either typing or hand writing, a second tool assessed general essay writing skill,
and a third tool assessed participants' ability to compose brief sentences
quickly. To measure each of these three qualities, we sought tasks that could be scored
objectively, and, where relevant, tasks that had evidence of validity showing that the
tasks measured the desired qualities.

Writing/typing speed task. Students were asked to type or hand write (depending on the
experimental condition) the word forms of numbers (i.e., "one, two, three ...") for one
minute as quickly as possible. Each character correctly typed/written was counted to form
a total score used to measure motor speed.

Assessment of spontaneous writing skills. The spontaneous writing materials from Form
B of the Test of Written Language, Third Edition (TOWL-3; Hammill & Larsen, 1996)
were used to assess participants'writing quality. Students were shown a picture and given
a set length of time to write a story about the picture. The TOWL-3 essay rating
procedures yield three separate scores: one for "contextual conventions" (spelling and
mechanics), one for "contextual language" (vocabulary, grammar, and syntax), and one
for "story construction" (prose, plot, and organization); these three scores are typically
summedto generate a "spontaneous writing composite" score. Additionally, the number
of words that participants wrote were counted to obtain a writing quantity measure.

The TOWL-3 is one of the most commonly used standardized measures of writing skill
(Salvia & Ysseldyke, 2007), and although it is typically administered to students at the
high school level and below, most of the dimensions of writing that it assesses are widely
recognized as being important at the college level and beyond (e.g., Jones, 1994). In any
case, the present study did not use the TOWL-3 norm-referenced scores to make absolute
judgments of students' writing skill levels, but rather used the scores to compare groups
that completed the test under different conditions. In addition, although the essay task is
brief (10-15 minutes), this is a common length of time allotted for individual essays on
both high-stakes and classroom assessments.

Writing fluency measure. The writing fluency subtest from the Woodcock-Johnson Tests
of Achievement, Third Edition, Form A (WJ-III; Woodcock, McGrew, & Mather, 2001)
was used to assess students' ability towrite simple expository text quickly. For each item
on this test, students were asked to write a sentence that contained all three of the words
shown for that item (e.g., dog, big, is); students were given seven minutes to complete as
many items as they could. This test has shown adequate reliability ([rho] = .88) and there
is substantial evidence of construct validity, based on correlations with other WJ-III
subtests as well as with other measures of writing.

Procedures

We conducted all sessions in the same small, quiet classroom, with no more than 10
students being tested at a time. After completing informed consent forms, we gave
participants one minute to complete the motor speed task, and when the minute was over,
we passed out the TOWL-3 stimulus picture. We gave participants in the hand written
(HW) sessions pencils and lined paper to write with, whereas participants in the word
processor (WP) sessions used a word processing program (Microsoft WordPad Version
5.1) to type their compositions on laptop computers (Dell Inspiron 1405 models with
standard keyboards and a 14-inch monitor) that we provided. This program does not have
spell-checking or similar aids. After completing the TOWL-3 essay assignment, all
students completed the WJ-III Writing Fluency subtest, using a hand
written response format. We then asked students to complete a brief questionnaire,
including demographic information as well as questions about computer usage.

For scoring, we double-spaced and printed all WP essays, and we typed, double-spaced,
and printed all HW essays, so that judges would not allow the typed format to influence
their ratings of writing quality. We trained two judges to score TOWL-3 essays, and each
of the essays was scored by one of the two judges, with 20% of the essays double-
scored to ensure reliability across judges. The inter-rater reliability (using a Pearson
correlation) for number of words written was 1.0 (100%); the coefficients for the
contextual conventions, contextual language, and story construction subtests from the
TOWL-3 were .90, .88, and .76, respectively, and the coefficient for the TOWL total
score was .88.

Results

We present mean scores in Table 1 for demographic and performance variables of all four
groups (WP-10 minutes, WP-15 minutes, HW-10 minutes, and HW-15 minutes). There
were no significant differences on demographic variables. The differences on
performance variables were generally unsurprising; students in the WP conditions wrote
more words in their essays and wrote more words during the speed task than students in
the HW conditions did, although no significant differences in quality (i.e., TOWL scores)
were present in these analyses.

A second group of analyses examined correlations among writing measures for the HW
groups separately from the WP groups. Because time allotment had very little effect on
correlations, data was collapsed across time conditions. Table 2 displays the correlation
matrices for both response modes, and in some ways the correlations are quite different.
For example, the writing/typing speed score correlated significantly with
all writing measures for the WP conditions, but did not correlate significantly with any
quality measures for the HW groups. Similarly, the number of words written by the WP
groups correlated significantly with all writing measures, yet much lower correlations
were found between these measures for the HW groups. Writing speed and fluency, then,
were better predictors of quality (and rice versa) when essays were completed using a
word processor.

Table 3 presents analysis of variance (ANOVA) models that examined the interaction
between responsemode and time allotment. Of most interest were the Total TOWL score
(quality) and essay length (quantity) measures. With regard to quality, there were no
differences found for writing format/responsemode or time condition and no interaction
(p-values ranging from .08 to .95). However, when considering quantity, we found an
interaction of time and response mode conditions, F(1,139) = 10.73, p < .01. The main
effects for both time and response mode were also significant (p < .01). Further analysis
revealed that the WP groups wrote significantly longer essays than the HW groups, and
additional time increased essay length only in the WP conditions. The WP-15 group (M =
365 words) significantly outperformed the WP-10 group (M = 261 words), yet there was
no difference in essay length between HW-10 (M = 215 words) and HW-15 (M = 216
words) groups.

The three remaining TOWL scores were of less interest in that they are constituents of
the Total TOWL score already analyzed for writing quality. However, analyses of these
variables yielded a small effect for time on the Contextual Conventions subtest, with
slightly better scores for the groups receiving 10 minutes to write. We also found a small
effect of response mode on the Contextual Language subtest score, with those in the WP
condition outperforming the HW groups. These analyses are also presented in Table 3.

Discussion

We examined the effects of two common test administration variables, time allotment
and response mode, on the length and quality of written essays in college students. The
results in this study were mixed. No effect of test conditions was found
for writing quality, while the word processor proved to be better than hand
written response mode for essay length (quantity). The effect was larger when students
had more time to write. Increasing the time allotment had no effect when students were
hand writing essays, but the same increase in time allotment had a greater effect when
essays were completed using a word processor. Moreover, there was no relationship
between essay length and quality (r = .03) for hand written essays, yet this relationship
was quite strong (r = .61) for word-processed essays. These and other correlations among
measures for both response modes suggest that the relationship betweenwriting speed
and writing quality is sensitive to whether compositions are typed or hand written.

The findings in this study are consistent with many other studies examining the effects of
administration conditions on writing performance. For instance, Russell and Haney
(1997) found, as did we, that longer essays were composed on a computer rather than by
hand. Our finding that essay length correlated significantly with quality is also congruent
with past work (e.g., Espin, Scierka, Skare, & Halverson, 1999; Mattern, Camara, &
Kobrin, 2007), although only for the word processor response mode. Finally, our finding
that word-processed essays were longer but not of higher quality recalls the results of
Truell, Alexander, and Davis (2004), who found that marketing students who used word
processors completed a classroom test more quickly but did not obtain higher scores. In
sum, our study coincides with others in the literature, showing that word processing
affords an opportunity to be more time efficient and/or more generative in a fixed time
period, and that it may or may not improve writing quality, depending on the task,
scoring procedures, sample under study, and other features.

However, two discrepancies between our findings and previous work raise interesting
questions. First, why did increased time help only students who word-processed their
essays? Composition is generally understood to be a high-effort academic task (for a
review, see Torrance & Galbraith, 2006), and if students experienced hand writing essays
as requiring more effort than word processing them (given that our students were
used to writing using computers), students may have been more motivated to expand and
revise their essays with the extra time when using a word processor, whereas students in
the hand written conditions may have been satisfied with a minimally adequate essay
product.

A related question is raised by our correlations between essay length and quality: why
was a positive correlation between these variables only found in the word processor
conditions? Part of the answer may lie in the fact that the variables exhibited higher
variability in the word-processor conditions than in the hand written conditions; the
standard deviation for essay length was smaller for all hand written essays than for word-
processed essays, and lack of variability restricts the possible values of the correlation
coefficient (Nunnally & Bernstein, 1994). Another possible explanation is that word
processing makes it easier to revise text by deleting text that is repetitive, irrelevant, or
that otherwise detracts from the essay (MacArthur, 2006), and so word-processed essays
may be less likely to contain additional text that does not contribute to the essays'
purposes. If hand written essays had more irrelevant text (owing in part to the higher
effort required to revise), any relationship between length and quality would be
attenuated significantly.

Implications for Testing Accommodations

Two of our findings appear to have implications for the provision of extended time and
word processor testing accommodations. First, when combined, the longer time allotment
and word processor use led tostudents writing substantially longer essays, and so if length
of composition is important on a writing test, test users and administrators should know
that students with disabilities who are given extended time and word processor
accommodations are given a very substantial benefit over their classmates, at least in
terms of essay length. Since the present study showed that the effects of extended time
and word processor are found in students without disabilities, there may be little rationale
for restricting these accommodations to students with disabilities (see Zuriff, 2000, for
more on the logic of restricting accommodations).

A second implication of our findings for testing accommodations comes from the
interaction between time allotment and response mode. Our finding that extended time
benefited only the fluency of students who word-processed their essays suggests that
providing extended time alone as ah accommodation may not be beneficial to students
on writing tests, unless a word processor is also provided. Since word processor
accommodations may lower the requirements for examinee effort and self-motivation,
more students may take advantage of any extra time that is provided.

A new trend in the testing accommodations arena is the design of tests that MI examinees
can access, so that accommodations are unnecessary, a concept called universal test
design (Thompson, Johnstone, Anderson, & Miller, 2005). A writing test with universal
design might be taken using a word processor and under conditions of generous time
limits, equivalent to providing the accommodations to all examinees. With regard to word
processors, computer-based writing is becoming so ubiquitous that some test
administrators (e.g., those of the GRE) have moved to completely computer-based
testing. Alternatively, word processors may be given as ah option to any examinees that
request them. However, the "optional" approach should be used cautiously, as the present
study has shown that correlations between different essay scores are quite different for
essays that are word-processed.

Limitations and Directions for Future Research

The present study has several limitations, each of which suggests future research that
would be beneficial. First, although we were interested in the effects of common testing
accommodations, our lack of a group of participants with disabilities precluded a
complete test of appropriateness. Although some scholars (e.g., Phillips, 1994; Zuriff,
2000) would consider our finding that nondisabled students benefited from
accommodations to be sufficient evidence against their being restricted to students with
disabilities, other scholars (e.g., Fuchs & Fuchs, 2001; Sireci, Scarpati, & Li, 2005) argue
that as long as students with disabilities benefit more from an accommodation than
nondisabled students, the accommodation is appropriate, and as such, future work should
include students with disabilities.

A second extension would involve manipulating the test administration variables as

within-subject factors,to determine whether individuals'

scores actually improved as a function of time allotment and response mode. Examinees
could complete one essay under each of several conditions, and for time allotment, they
could even be stopped after a certain length of time, and then given an extension to see
how the same essay changes. When extended time has been studied in the context of
reading and mathematics performance (e.g., Lewandowski, Lovett, Parolin, Gordon, &
Codding, 2007), this procedure has been useful, although writing is a somewhat different
task, and changes to ah essay may be difficult to record.

A third extension relates to our writing stimulus (the TOWL-3 stimulus), which was not
normed on college samples of designed specifically for college students, and was only a
single, brief essay prompt. Given the importance of interest in motivating
students to write, various writing prompts with a range of time demands should be
tested to determine which stimuli lead to the greatest motivation, including allowing
students choice in selecting a topic to write about. Of course, it should be noted that our
own task was of a similar length to those found on many standardized writing tests,
suggesting a degree of generalizability to our findings.

The present study, then, should open the door to future research that examines the effects
of task characteristics on the writing performance of students with and without
disabilities, and with a range of academic skills. On a more practical note, test
administrators and users might take away two messages from the present study's results.
First, use of a word processor may be necessary for extended time
accommodations to exert their full effects. Second, many college students appear to use
word processing as their default mode of composition, suggesting that writing tests that
require hand written responses may put these students in an unfamiliar situation,
compromising the validity of the assessment. Therefore, in high-stakes testing situations,
word processors should be considered as ah element of universal test design (Thompson
et al., 2005), furthering the goal of appropriate assessment for all students.

References

Arnold, V., Legas, J., Obler, S., Pacheco, M. A., Russell, C., & Umbdenstock, L. (1990).
Do students get higher scores on their word-processed papers? A study of bias in scoring
hand-written vs. word-processed papers. Whittier, CA: Rio Hondo College. Retrieved
from ERIC database (ED 345818).

Bangert-Drowns, R. L. (1993). The word processor as an instructional tool: A meta-

analysis of word processing in writing instruction. Review of Educational Research, 63,
69-93.

Bolt, S. E., & Thurlow, M. L. (2004). Five of the most frequently allowed testing
accommodations in state policy. Remedial and Special Education, 25, 141-152.

Burke, J. N., & Cizek, G. J. (2006). Effects of composition mode and self-perceived
computer skills on essay scores of sixth graders. Assessing Writing, 11, 148-166.

Conlan, G. (1986). "Objective" measures of writing ability. In K. L. Greenberg, H. S.

Wiener, & R. A. Donavan (Eds.), Writing assessment: Issues and strategies (pp. 109-
125). New York: Longman.

Crawford, L., Helwig, R., & Tindal, G. (2004). Writing performance assessments: How
important is extended time? Journal of Learning Disabilities, 37, 132-142.

Dikli, S. (2006). An overview of automated scoring of essays. Journal of Technology,

Learning, and Assessment, 5(1), 1-36.

Elliott, S. N., & Marquart, A. M. (2004). Extended time as a testing accommodation: Its
effects and perceived consequences. Exceptional Children, 70, 349-367.

Espin, C. A., Scierka, B. J., Skare, S., & Halverson, N. (1999). Criterion-related validity
of curriculum-based measures in writing for secondary school students. Reading
& Writing Quarterly, 14, 5-27.

Fuchs, L. S., & Fuchs, D. (2001). Helping teachers formulate sound testing
accommodation decisions for students with learning disabilities. Learning Disabilities
Research and Practice, 16, 174-181.

Goldberg, A., Russell, M., & Cook, A. (2003). The effect of computers
on student writing: A meta-analysis of studies from 1992 to 2002. Journal of Technology,
Learning, and Assessment, 2(1), 1-51.

Hammill, D. D., & Larsen, S. C. (1996). Test of Written Language. (3rd ed.). Austin, TX:
Pro-Ed.

Hampton, G., & Gosden, R. (2004). Fair play for students with disability. Journal of
Higher Education Policy and Management, 26, 225-238.

Hopkins, K. D. (1998). Educational and psychological measurement and evaluation. (8th

ed.). Boston, MA: Allyn & Bacon.

Jones, E. A. (1994). Defining essential writing skills for college graduates. Innovative
Higher Education, 19, 67-78.

Konur, O. (2002). Assessment of disabled students in higher education: Current public

policy issues. Assessment & Evaluation in Higher Education, 27, 131-152.

Lewandowski, L., Lovett, B. J., Parolin, R., Gordon, M., & Codding, R. S. (2007).
Extended time accommodations and the mathematics performance of students with and
without ADHD. Journal of Psychoeducational Assessment, 25, 17-28.

Linn, R. L., & Miller, M. D. (2005). Measurement and assessment in teaching. (9th ed.).
Upper Saddle River, NJ: Merrill.

Lu, Y., & Sireci, S. G. (2007). Validity issues in test speededness. Educational
Measurement: Issues and Practice, 26(4), 29-37.

MacArthur, C. A. (2006). The effects of new technologies

on writing and writing processes. In C. A. MacArthur, S. Graham, & J. Fitzgerald (Eds.),
Handbook of writing research (pp. 248-262). New York: Guilford.

MacCann, R., Eastment, B., & Pickering, S. (2002). Responding to free examination
questions: computer versus pen and paper. British Journal of Educational Technology,
33, 173-188.

Mattern, K., Camara, W., & Kobrin, J. L. (2007). SAT Writing: An overview of research
and psychometrics to date (Research Note RN-32). New York: College Board.

Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory. (3rd ed.). New York:
McGraw-Hill.

Phelps, R. P. (2003). Kill the messenger: The war on standardized testing. Edison, NJ:
Transaction.

Phillips, S. E. (1994). High-stakes testing accommodations: Validity versus disabled

rights. Applied Measurement in Education, 7, 93-120.

Powers, D. E. (2005). "Wordiness": A selective review of its influence, and suggestions

for investigating its relevance in tests requiring extended written responses (ETS
Research Memorandum RM-04-08). Princeton, NJ: Educational Testing Service.

Powers, D. E., & Fowles, M. E. (1996). Effects of applying different time limits to a
proposed GRE writingtest. Journal of Educational Measurement, 33, 433-452.

Powers, D. E., Fowles, M. E., Farnum, M., & Ramsey, P. (1994). Will they think less of
my handwritten essay if others word process theirs? Effects on essay scores of
intermingling handwritten and word-processed essays. Journal of Educational
Measurement, 31, 220-233.

Russell, M. (1999). Testing on computers: A follow-up study comparing performance on

computer and on paper. Education Policy Analysis Archives, 7(20). Retrieved January
28, 2008 from http://epaa.asu.edu/epaa/v7n20.

Russell, M., & Haney, W. (1997). Testing writing on computers: An experiment

comparing studentperformance on tests conducted via computer and via paper-and-
pencil. Education Policy Analysis Archives, 5(3). Retrieved January 28, 2008 from
http://epaa.asu.edu/epaa/v5n3.html

Salvia, J., & Ysseldyke, J. E. (2007). Assessment in special and inclusive education.
(10th ed.). Boston, MA: Houghton Mifflin.

Sireci, S. G., Scarpati, S., & Li, S. (2005). Test accommodations for students with
disabilities: An analysis of the interaction hypothesis. Review of Educational Research,
75, 457-490.

Thompson, S. J., Johnstone, C. J., Anderson, M. E., & Miller, N. A. (2005).

Considerations for the development and review of universally designed assessments
(NCEO Technical Report No. 42). Minneapolis, MN: National Center on Educational
Outcomes.

Torrance, M., & Galbraith, D. (2006). The processing demands of writing. In C. A.

MacArthur, S. Graham, & J. Fitzgerald (Eds.), Handbook of writing research (pp. 67-80).
New York: Guilford.

Truell, A. D., Alexander, M. W., & Davis, R. E. (2004). Comparing postsecondary

marketing studentperformance on computer-based and handwritten essay tests. Journal of
Career and Technical Education, 20, 69-78.

Woodcock, R. W., McGrew, K. S., & Mather, N. (2001). Woodcock-Johnson Tests of

Achievement. (3rd ed.). Itasca, IL: Riverside Publishing.

Zuriff, G. E. (2000). Extra examination time for students with learning disabilities: An
examination of the maximum potential thesis. Applied Measurement in Education, 13 (1),
99-117.

Benjamin J. Lovett, Ph.D., is an assistant professor of psychology at Elmira College,

where his research focuses on conceptual and psychometric issues in psychoeducational
assessment and psychiatric diagnosis. Lawrence J. Lewandowski, Ph.D., is Meredith
Professor of Teaching Excellence in the psychology department at Syracuse University.
Cassie Berger, M.S., is a Ph.D. candidate in school psychology at Syracuse University.
Rebecca A. Gathje, Ph.D., is a postdoctoral fellow at the Munroe-Meyer Institute and a
clinician working out of the Grand Island Behavioral Health Clinic in Grand Island,
Nebraska.

Table 1
Group Means on Demographic and Performance Rst Variables
Handwritten Word-processed

Handwritten Word-processed

Measure 10 min. 15 min. 10 min. 15 min.

(n = 29) (n = 30) (n = 39) (n = 42)

Age (years) 19.5 19.0 19.0 19.0

College GPA 3.3 3.1 3.2 3.3
WJ-111 Fluency 29.5 27.4 29.1 25.7
Writing Speed Task 136.6 137.7 199.7 181.2
Words Written 215 216 261 365
Context. Conven- 11.1 10.5 11.5 9.8
tions
Context. Language 20.2 19.5 21.0 21.3
Story Construction 9.9 10.3 11.1 11.7
TOWL Total 41.2 40.3 43.5 41.8

Measure F [[eta].sup.2]

Age (years) 1.39 .03

College GPA .838 .02
WJ-111 Fluency 3.61 * .07
Writing Speed Task 28.58 *** .39
Words Written 22.04 *** .33
Context. Conven- 1.96 .04
tions
Context. Language 2.23 .05
Story Construction 1.49 .03
TOWL Total 1.17 .03

* p <.05
** p < .01
*** p < .001

Table 2
Performance Tusk Intercorrelations
 Variable 1 2 3 4

1. Speed Task -- .31 * .05 .11

2. WJ-III Fluency .25 * -- .18 .19
3. Context. Conventions .52 ** .33 ** -- .19
4. Context. Language .46 ** .15 .52 ** --
5. Story Construction .46 ** .24 * .59 ** .63 **
6. TOWL Total .57 ** .28 * .83 ** .85 **
7. Words Written .34 ** .18 .40 ** .54 **

Variable 5 6 7

1. Speed Task .09 .13 .28 *

2. WJ-III Fluency -.24 .00 .25
3. Context. Conventions .16 .69 ** -.04
4. Context. Language -.05 .45 ** .29
5. Story Construction -- .72 ** -.09
6. TOWL Total .88 ** -- .03
7. Words Written .60 ** .61 ** --

Note. Correlations above the diagonal were observed in students in

the hand written conditions (n = 59); those below the diagonal were
observed in the word-processed condition (n = 81).

* p <.05

** p < .01

Table 3
Two-Way Analyses of Variance for TOWL-3 Essay Scores and
Numbers of Words Written

Variable and Source MS F [[eta].sup.2.sub.p]

Contextual Conventions
Response Mode .44 .04 .00
Time 45.69 4.25 * .03
Response Mode x Time 9.30 .87 .01
Contextual Language
Response Mode 52.0 5.57 * .04
Time .78 .09 .00
 Response Mode x Time 9.23 .99 .01
Story Construction
Response Mode 55.08 3.77 .03
Time 9.27 .63 .00
Response Mode x Time .24 .02 .00
TOWL Total Score
Response Mode 195.12 3.17 .02
Time 21.16 .34 .00
Response Mode x Time .23 .01 .00
Number of Words Written
Response Mode 325338.44 38.67 *** .22
Time 93458.25 11.11 ** .08
Response Mode x Time 90251.87 10.73 ** .07

* p < .05

** p <.01

*** p < .001

Questia, a part of Gale, Cengage Learning. www.questia.com

Reading and Responding to Student Writing: a

Heuristic for Reflective Practice

by Richard E. Straub

In "Composition Theory in the Eighties: Axiological Consensus and Paradigmatic

Diversity," Richard Fulkerson laments the wide spread fragmentation and confusion in
contemporary composition theory. He is particularly troubled by the ways we have
come to tangle our talk about the goals of teaching writing-what he calls our different
"philosophies" or "axiologies"-and the various means we have devised to achieve
them. By 1990, he claims, compositionists had come to some consensus about their
commitment to a rhetorical emphasis in writing instruction. At the same time,
however, he sees "a growing complexity and conflict over means of reaching it"
(410). The field, Fulkerson suggests, is in a knot, the means and ends of composition
caught in a giant snarl.

I believe, with Fulkerson, that compositionists have come to some general agreement
about the importance of audience and, more broadly, to appreciate the perspectives
afforded by a social view of writing. I agree that there is a deep confusion about
means and ends in the teaching of writing, and I suspect the problem has gotten worse,
not better, in the past ten years, as we have expanded the range of discourses we
study, learned more about the ways that context and community bear on reading and
writing, gotten more interested in looking at students individually, at work in
particular settings, and (regrettably) given less and less attention in our scholarship to
questions of classroom practice. But I'm not so sure that we have come to a consensus
about the goals of writing instruction or about the nature of good writing. Having
spent some time in the 1980s sorting through the evaluative criteria espoused by the
teachers in William Coles and James Vopat's What Makes Writing Good? for a
dissertation and then studying the responding practices of recognized teachers for
Twelve Readers Reading in the 1990s, I have come to see a great diversity both in our
goals for teaching and in our views toward what makes writing good. In fact, I think a
good bit of confusion has been caused by a general failure to define our values for
writing and make them consistent with our goals, theories, and classroom practice.

In this essay I would like to help teachers clarify their values for student writing, get
their assumptions and classroom methods in dialogue with their goals, and then
suggest a way to deal with this disciplinary confusion. To do so, I will:

1 construct a protocol of a teacher reading and responding to a student essay, to show

the complexity we run into as teacher-- readers and all the things we have to sort
through, focus on, and consider when we read and respond to student writing;

2 provide a map of the various concerns we can take up in our reading, and use it to
examine reading and response in relation to the larger contexts of the writing class;

3 create and model a heuristic for us to reflect on our own reading and
responding practices in light of our philosophy, assumptions, and practice.

Whereas Fulkerson takes up the problem of ends and means at the disciplinary level,
as a problem for composition scholars, I am concerned first of all with the problem
such theoretical conflicts have practically for individual teachers and actual classroom
practice. Instead of looking to theory to address the problem from top down, I will
approach the problem from bottom up by calling onteachers to work through the
problem in terms of their own commitments and problems, with the help of theory.

A PROTOCOL OF TEACHER RESPONSE: THE SCENARIO

Imagine the following situation. It's halfway through a first-year writing course at a
large state university, in a writing program staffed mainly with graduate teaching
assistants who teach two courses a semester. Writing program administrators favor a
process-based expressivist approach to teaching writing, employing no standard
textbook and requiring students to write six essays, most of which are
based on personal experience, most of them to be taken through a series of drafts and
revisions. Teachers are encouraged to privilege the writer's individual voice, be liberal
about the use of narrative, value details and examples from the student's personal
experience, and encourage experimentation with organization and style. They are
urged to emphasize the development of the writer over the quality of the written
product and to respect the needs of the individual student.

The teacher is an experienced teaching assistant who has taken the required training
course for new teachers and, to satisfy an emerging interest in teaching, one other
course in composition theory. She has found herself invigorated by the expressivist
emphasis on personal voice and writing that is close to the self, and she has gone into
her teaching with genuine excitement. At the same time, she has developed a real
commitment to the importance of writing for readers. She sees herself as a hands-
off teacher intent on making the classroom student-centered. A fairly
successfulstudent with a history of doing well in her own writing, she does not
underestimate the importance of clarity and focus, an authoritative stance, and tight
organization in academic writing. In fact, she wonders if more attention shouldn't be
given to such concerns, as well as to grammar and mechanics, in first-year
composition. Over the semester, she has tried, with less success than she had hoped
for, to spend no more than 15-20 minutes reading and
commenting on each studentpaper. Nevertheless, she plans on giving this batch
of papers slightly fuller comments than previouspapers because she sees it as a crucial
point in the class, a culmination of their work to this point.

This student essay is the final draft of the fourth paper. The teacher has talked
informally with students about their plans for the paper, and the students themselves,
guided by a set of questions, have responded to earlier drafts of the paper. In class,
students have been doing a lot of smallgroup workshops to help one another invent,
draft, revise, and edit each of their papers, and once a week the teacher has selected a
sample or two from the course reading anthology to analyze and discuss specific
strategies of writing, for instance, how to recognize different voices, how to start a
story or set up an introduction for an essay, and how to format and punctuate dialogue.
Over the past two weeks, in anticipation of this latest paper, the class has been
examining the ways writers like Toni Morrison and Annie Dillard use detail not only
to describe something but also attach some value or emotional sense to the object.
Here is the assignment students have been given:

Most of you are off at school and in a new place, away from the people and settings
you have become accustomed to and attached to. Similarly, those of you who are from
Tallahassee have likely not had the time or the opportunity to visit some of the old
places that are special to you. Choose some place, atmosphere, or situation that you
miss from home-or, if you are at home, that you have not had the chance to experience
for some time, and miss. Depict this scene, mood, or setting in a way that will allow
your reader-someone who does not know about it-to see the significance it has for
you. Remember that since your aim is to give readers a sense of place, you will do
well to use specific details.

Across the course students have been urged to look at the assignments as a rough
guide for what they might do in their own writing; if they prefer to pursue a somewhat
different focus than is presented in the assignment, they are encouraged to go their
individual route so long as they find a way to deal with the same general subject.
Although this is the final draft and students are not required to take this paper through
another rewrite, they have the option of rewriting and submitting, for a change in
grade, one or two of their course papers by the end of the term. The writer is a
confident, perhaps even a cocky, student who comes across (in the view of
the teacher) as someone who thinks he is a better writer than he is. However, his
confidence is by no means groundless. The teacher has been both disappointed by and
taken with aspects of his previous writings.

A PROTOCOL OF TEACHER RESPONSE: THE READING

The teacher picks up the first paper. David. "The Four Seasons." All right, she thinks,
might as well plunge right in. Let's see if he walks the walk this time. Gets more than
a few good lines in here and there. Lives up to his own billing. Okay, 20 minutes per
paper. Don't comment on everything. Focus. She starts to read.1

The Four Seasons

I like Tallahassee very much. The heat and sunshine almost everyday makes each day
very pleasant.

He actually, she thinks, likes this relentless heat? He obviously hasn't been here in the
summer.

I intend to spend my next four and one half years here, but I miss my other home,
Syracuse, New York. One thing that I truly miss about Syracuse is the four seasons.
Each season is distinct and clear in its own way. I will do my best to describe each
season to you, but remember that my description cannot compare to experiencing each
season for itself.

An okay start, she thinks. A good thesis statement. But he's slow getting into what he
has to say. Try to grab the reader's attention, she jots in the margin. And this escape
hatch won't do. It's a cop out. Why the disclaimer? she writes, and reads on.

In the Spring the ground is soft from the melting snow. You can feel the moist ground
wanting to seep into your shoes.

Good, she thinks, a really nice image. Evocative. She puts a check next to the line in
the margin and writes, Good detail. I can feel the squishing. She's about to underline
the "you" and write, Shift in point of view, but she doesn't want to take away from the
praise she's just given-he needs the applause-and decides for the moment to let it go.
She wonders whether to mark the capital S onSpring or tick-mark it and let him find
it. She puts a slash through the letter, and continues reading.

As the ground begins to dry, the trees begin to blossom and the faint smell of pollen
lingers in the air. The flowers work their way out of the ground and bloom for another
year.

Good, some more vivid description. She thinks of the cherry trees that used to
blossom each spring in her backyard. But it's pretty generic: a composite of a mythical
spring day up north. He could get a lot more specific. Which trees? What kind of
flowers? she writes, then picks up with her reading again.

The familiar sound of geese is heard overhead as you look into the sky and see a
"V"formation travelling north for the summer. A long winter's nap has ended for the
bears, squirrels, rabbits and other hibernating animals. After they awake, their
chattering conversations ramble through the forest.

She's not sure she likes "'V' formation," and she's still not sure about this use of "you."
Why doesn't he use "I," she wonders, or stick with the third person? But she's clearly
troubled by the string of general language, the stereotyped ideas: "The familiar sound
of geese"? "A long winter's nap"? "Chattering conversations"? He's just blowing
through these descriptions, not thinking about them. Coasting. She sharpens her
pencil, drops her hand to the page, and next to the last two sentences writes, Really ?
Squirrels and rabbits hibernating? Bears and rabbits chattering? He's getting careless
here, she thinks, relying far too much on cliches about spring. But she catches herself.
She wants to keep it constructive. She reads back over the sentences. Maybe it's just
the way it sounds, a matter of voice or tone. Maybe he's thinking this is the way a
description of spring is supposed to be: nice and light and homey. Hold off, she tells
herself. Let's see what he does next.

Not only do the animals come out of their shelter in the springtime, but also people.
Many people have a tendency to "hole up" in the wintertime. All your neighbors, that
you thought had died, open up their houses to allow the spring breeze to come along
and carry away that musty air that built up during winter. You can hear voices and
lawnmowers everywhere as people are outside doing their springtime yard work.
Wives are planting new flowers while husbands are raking and mowing the lawn.
Spring is the season of awakening where everything becomes refreshing.

Following Spring is the season that most people look forward to, that is Summer.
Summer is the time of the year when kids are everywhere, because school has been let
out. You can hear their voices and giggles fill the atmosphere. People are always
outside in the summertime because the sun beats down onto the earth and warms
everything up. There are enormous amounts of families going to the beach for the
weekend or going on vacation for a week. As you look down the road, you can see
heat waves resting on the pavement. The foliage is green and spirits are high. There is
a feeling of warmth amongst neighbors, friends, and family.

The wheels, she thinks, have started to spin. Where to start? What to take up? There
are some good moments. Here at the end: The heat resting on the pavement. Good
concrete detail, she writes next to it. It's an image that will lead your readers to
imagine how hot it gets in Syracuse. This is the kind of detail I've been looking for.
And up here at the start of the paragraph: People holing up, hibernating. That's
different, she thinks. Something worth saying. Though the sentences are awkward.
Good idea, she writes, but you can clean up the sentence structure. And the last two
lines. The warmth outside connected with a warmth among neighbors. In contrast to
the hibernating that he says people tend to do when it's cold. Nice.

But she's concerned about all these muddy generalizations. People air out their houses
and do yard work. All of them are wives or husbands, and they plant or they rake and
mow. Don't, she finds herself writing in the margins, the women in your neighborhood
ever cut the grass? Does Mr. Sworin ever plant the flowers? Spring, we find, is the
season of awakening. Spring is followed by summer. Things warm up and people get
back outside, go to the beach, and go on vacation. Not much here, she thinks. He's not
telling us anything we don't already know. He's got to get more focused, get more
specific detail. What happened to all that work we did last week? So much for Annie
Dillard. Where is he in all this description? What significance do the seasons in
Syracuse have for him? Did he read the assignment? She goes to the paragraph and
underlines, in successive sentences, "kids," "voices," "people," "earth," "everything,"
"families," and "foliage." Don't attack, she thinks. Keep it upbeat. She puts a line
down the length of the paragraph and writes: These are all general terms. Let's hear
some details. He's done some sharp writing for short stretches in his earlier paperstoo,
but he's not been able to sustain it. He can do better than this. She takes a breath, about
to move on, but her eye is pulled back to the page. She writes Mechanical transition
next to the first sentence of the paragraph about summer, squiggles a line under
"enormous amounts of families," and puts her pencil down. He's stuck and time's
flying. A quarter after. She's got to move on.

She goes through the rest of the paper, making sure to acknowledge other sharp
descriptions, pointing out places where generalizations steer him away from making
the descriptions his, quickly noting places where a sentence might be smoothed out,
and snagging him at the end just as he is about to slip into the same escape hatch he
devised in the introduction. This, she thinks, might be the one he goes back to for the
last revision. She leans back and tries to settle on her overall sense of the paper. She
gets something positive in mind for a start, and writes her end note to David:

Pretty good, David. You have some fine descriptions of the four seasons in here. But
I'm not sure how much you've experienced them or let us experience them as readers.
You're relying too muchon cliches: geese flying north in the spring, summers at the
beach, raking leaves in the fall, and sitting by the fire with that special someone in the
winter. They're descriptions that we're all familiar with and they don't give any insight
into how the four seasons are unique to you. You seem more intent on capturing some
homey images of spring than in sharing with us your own views of the seasons in
Syracuse. (I wonder: Are you trying to take on too much here with all four seasons?)
Try focusing on particular scenes and specific events you have seen or, better yet, that
have happened to you. Maybe it'd help to sketch out lists of things that you associate
personally with these seasons, select the ones that are most vivid and representative,
and work those into your essay. This might be one you choose to revise.

She debates for a moment and writes "C" at the bottom of the page. Then she thinks of
his sharp descriptions and, next to it, slashes a B. Nothing to write home about, but
better than he's been doing. One down.

MAPPING THE CRITERIA FOR READING STUDENT WRITING

This is a full and fairly complex reading and response.' The teacher seems intent not
simply onlooking at the writing summatively, to assess it, but using her response as an
occasion for leading the writer back into revision and teaching him about writing. It
may not be adequately focused. It may not be as coherent as it could be. It may be too
evaluative in its posture, putting too great an emphasis on the teacher's role as critic
and judge. And it may or may not be just what the studentneeds at this time. But it has
a depth and ambitiousness that are worthy of praise. It is a rich pastiche of a reading,
marked by a criss-cross of impulses and purposes that we routinely experience as we
read student writing, and the comments do a fairly good job of expressing theteacher's
concerns to the student. In constructing this case, I want to suggest the rich and often
bewildering complexity that is involved in reading, evaluating, and responding
to student writing,3 and dramatize the need for finding some way to give order to the
great variety of concerns we may take up as teachers of writing. I also want to capture
something of the dynamic relationship between reading and response-that is, between
reading and evaluating student texts, on the one hand, and communicating with
the student about his writing, on the other. In doing so, I hope to explain why reading
and response are so demanding and show how they might be made more manageable
and used more effectively. I have selected this sample student writing, originally
presented in Twelve Readers Reading, to tie this analysis to the ways that the well-
recognizedteacher-scholars featured in that study respond to student writing, extend
the analysis Ronald Lunsford and I do there of the relationship between reading and
response, and examine response more fully in relation to the larger classroom context
and the teacher's approach to composition.4 Although I might have highlighted a
reading from any of a number of pedagogical perspectives-- a rhetorical pedagogy, a
social constructionist pedagogy, a post-process pedagogy, or some critical pedagogy-I
have chosen an expressivist pedagogy because of its popularity in the classroom and
its ability to accommodate a broad spectrum of features. In the section ahead, I'll chart
the various qualities we might look for when we read student texts. Instead of an
exhaustive list or a survey, I'd like to provide a map and a compass for reading,
evaluating, and responding to student writing: some instruments that might help us as
teachers figure out where we are and make a plan for where we want to go.

TEXTUAL FEATURES

What, then, can we learn by looking back on this response to "The Four Seasons" as a
case study-one instance of a teacher (in this case, a hypothetical teacher) caught in the
act, as it were, of reading student writing? This teacher takes up a variety of issues at a
variety of levels. She looks at the text formally, in terms of what the writer has to say
and how he presents it on the page. She gives a fair amount of attention to local
matters such as correctness, usage, sentence structure, and grammatical point of view.
But she does not emphasize these superficial textual matters over more substantial
matters of content and form. She considers the focus, scope, and overall shape of the
essay; she considers David's use (or lack) of specific detail; and she considers the
quality and substance of his descriptions:

* An okay start, she thinks. A good thesis statement, but he's slow getting into what he
has to say. Are you trying to take on too much here with all/our seasons?

* People holing up, hibernating. That's different, she thinks. Something worth saying.

* But she's clearly troubled by the string of general language, the stereotyped ideas:

* "The familiar sound of geese?" "A long winter's nap?" "Chattering conversations?"

These are the kinds of concerns-matters of correctness, style, organization, and ideas-
we usually talk about when we talk about qualities of student writing. They deal with
the text and the immediate construction of that text from the words on the page.
Although they do not "reside" in the text, we talk about them, for practical purposes,
as formal concerns, as features based in, and recoverable from, the words on the
page.5 They may be plotted out as features of the written text, the meeting ground
where, as Wolfgang Iser notes, the implied author and implied reader come together,
in a carefully negotiated process, to make meaning.6

When monitoring our own criteria, it's important not only to determine the extent to
which we privilege local matters versus global matters but to get at exactly how we
understand these terms. What constitutes "substantive content"? What do we mean
when we say a paper is "well organized"? What makes for our sense of an "informal
voice" or an "effective style"? The map lays out general categories for reading and
evaluating writing; we have to give them local habitation, not just a name. For the
protocol teacher, it is clear, for instance, that writing is not merely a matter of
managing sentences and paragraphs or tending to the superficial properties of
discourse; it's a matter of getting and shaping something to say. Her reading and
response suggest that it is the content of writing-and the forming of that content-that
makes writing good or bad, makes it worth or not worth reading. Further, she clearly
privileges (at least for this type of writing) a particular kind of content: original
writing that grows out of and reflects the student's own experience and perceptions.

CONTEXTUAL CONCERNS

The close attention this teacher gives to the text, especially the emphasis she
places on content, tells us something important about her values as a writing teacher.
But it only begins to account for the depth of her reading and the range of concerns
that we might address when we read studentwriting. In fact, for every time
this teacher focuses on some isolated quality closely tied to the text, she also brings to
her reading some concern that goes beyond the text, beyond the words on the page,
and invests it with the meanings, values, and perspectives of some broader context.
Consider the following instances:

* Try to grab the reader's attention.

* Maybe he thinks this is the way a description of spring is supposed to be: nice and
light and homey.

* What happened to all that work we did last week? So much for Annie Dillard.

* Really? Squirrels and rabbits hibernating? Bears and rabbits chattering?

* Where is he in all this description? What significance do the seasons in Syracuse

have for him?

* He's just blowing through these descriptions, not thinking about them. Coasting.

In each of these comments the teacher expands her orientation from the text to some
larger context of the writing.' The first response looks at the organization of the
writing in terms of the rhetorical situation. The second considers the assumptions
about genre that the student may be bringing to this kind of writing. The third looks at
the text in relation to the work in class. The fourth looks at the writer's statements
explicitly in terms of their accuracy against a larger social reality. The fifth and sixth
view the writing in terms of the student behind the text, his experience and
involvement. All of these contexts are brought into play in her reading. These "texts"
that go with the text, these contexts, inform and influence how she views the paper,
but they also mark the various areas of writing that she attends to in her reading and
response. "Context" in this sense is not only a necessary condition for language and
meaning, as Deborah Brandt argues; it is also a mental construct, a set of expectations
that we use to interpret writing, analyze its strengths and weaknesses, and talk about
how texts work.8 We "bring" certain sets of concerns-what Kenneth Burke would call
certain "terministic screens"-to our reading even as the text elicits other sets of
concerns, other contexts, as we read. By sorting out the various qualities and contexts
brought into play in the protocol teacher's reading of "The Four Seasons" and charting
them on a map, we might get a clearer feel for the geography of reading and the
concerns that figure into our own ways of reading, evaluating, and responding
to student writing.

The Rhetorical Context

The "text" beyond the text that has come to draw most of our attention is the rhetorical
context-the circle of concerns that unites writer, text, and reader in some common
purpose, in response to some situation or need. A student writes a paper, it is
presumed, with some rhetorical purpose: to inform, explain, entertain, persuade, or
generally achieve some effect on readers. The teacher is then called to read the paper
in terms of how the text may affect readers and how well it achieves the writer's
intentions. Teachers who attend to the rhetorical context most obviously focus on the
way the text meets the demands of an audience. How well does David describe the
seasons of Syracuse for those who are unfamiliar with them? What does he do to
engage readers in his discussion? They also focus on the purpose of the writing and
assess how well the text is working to realize the writer's intentions. Does David want
to show his audience how the seasons in Tallahassee are no match for the seasons in
Syracuse? Will his descriptions enable readers to see the significance the seasons have
for him?' Some teachers, looking to get a firmer grasp of what the student is trying to
accomplish in a piece of writing, have come to call on the writer to identify his own
intentions. In statements that accompany the paper, they have students define their
aims, assess their progress, and raise questions for readers to consider. Such "process
memos" (J. Sommers) or "reflective cover letters" (Yancey) provide an additional
resource for reading student writing-and an additional set of concerns to address in our
reading and response. Teachers read in terms of the rhetorical context, in addition,
when they address the tone of the writing and the writer's voice, persona, and ethos-all
of which deal somehow with the ways authors construct themselves as speakers and
establish certain relationships with prospective readers.

The protocol teacher actively invokes the rhetorical context in her reading of "The
Four Seasons," viewing the writing as an instance of someone saying something to
someone else, a writer addressing a reader for some purpose:

* You have some fine descriptions of the four seasons in here. But I'm not sure how
much you've experienced them or let us experience them as readers.

* It's an image that will lead your readers to imagine how hot it gets in Syracuse.

* Not much here, she thinks. He's not telling us anything we don't already know.

* You seem more intent on capturing some homey images of spring than in sharing
with us your own views of the seasons in Syracuse.

* And this escape hatch won't do. It's a cop out.

In the first three instances, the teacher considers the writing in terms of the audience,
in the last two, in terms of the writer's intentions and persona.

The rhetorical context, then, may be plotted as a dynamic set of concerns encircling
and interacting with the text.

When we view student writing as a rhetorical activity, a meaningful act of

communication, we read in a way that, according to Fulkerson, has come to dominate
contemporary composition studies. We look to help the writer see where the writing
achieves and does not yet achieve what he (presumably) set out to accomplish and
thereby dramatize the need for revision. As Nancy Sommers notes, "As writers we
need and want thoughtful commentary to show us when we have communicated our
ideas and when not, raising questions from a reader's point of view that may not have
occurred to us as writers. We want to know if our writing has communicated our
intended meaning and, if not, what questions or discrepancies our reader sees that we,
as writers, are blind to" (148).

The Classroom Context

When we read student writing, we are not, to be sure, just readers reading a piece of
writing. We are teachers who read. We've decided on the kinds of writing that
students are to do. To a large extent, we've determined the qualities that are to be
valued in their writing. We may not warm up to the ubiquitous "What do you want?"
but no question that students in a writing course ask makes more sense. In any class,
even when we allow them to choose their own topics and genres, students are
somehow left to give us what we call for. They are not writers so much as they are
writers in apprenticeship: students. And as students they are imbricated in a set of
power relations that always gives someone else a great deal of control over their
writing. So when it comes time to read student writing, we invariably approach it as
more than just readers. In addition to processing the text we also have to diagnose the
strengths and weaknesses of the writing, evaluate thestudent's performance in terms of
the work of the class, and use the writing to teach the studenthow to write better. As
Patricia Murray writes, "We need to recognize that we bring with us to a reading
of student texts an inevitable evaluation that stems from a discourse community that
more often than not is different from that of the students. We need to look at how our
responses as teachers of composition affect what we advise our students..." (84). If we
are readers, we are always, to one extent or another, teacher-readers,
reading student writing in terms of the assignment, the genre of writing, the work of
the course, and our teacherly expectations.

The Assignment and Genre of the Writing

Much of what we value in student writing is determined by what we ask students to do

in the assignment and, by extension, by the kind of writing we've assigned.
Assignments, in effect, perform the function of designating, among other things, the
genre of writing to be pursued and the features of writing to be highlighted. They link
the writing to a set of established forms and practices. Different assignments and
different genres, of course, impose different demands on writers and elicit different
expectations from readers. A personal narrative will lead us to look for different things
than an informative report or an argumentative essay. A memoir will lead us to expect
something different than a parable or a historical narrative. Assigning writing that is
meant simply to display knowledge will lead us to look for something different than if
we assigned writing to learn (Larson). Assigning freewriting exercises or other kinds
of "low stakes assignments" will likely lead us to engagestudent writing differently
than "high stakes assignments" (Elbow "High Stakes"). Holding students firmly to the
demands of the assignment or, alternately, allowing them the freedom to depart from
the assignment will establish different reading postures and lead us to look for
different features when we read (Jolliffe). In the sample case, the writing assignment
and the genre constraints of the personal descriptive essay clearly affect the
protocol teacher's way of reading "The Four Seasons." She expects the writing to have
an abundance of concrete detail that will enable readers to picture what the seasons are
like in Syracuse and, further, that reflect the special meaning these seasons have for
the writer. She also expects the writing not just to rehash general descriptions but to
tell readers something they probably do not know about the seasons in this
northeastern city:

* Spring, we find, is the season of awakening. Spring is followed by summer. Things

warm up and people get back outside, go to the beach, and go on vacation. Not much
here, she thinks. He's not telling us anything we don't already know.

* What significance do the seasons in Syracuse have for him? Did he read the
assignment?

* They're descriptions that we're all familiar with and they don't give any insight into
how the four seasons are unique to you .... Try focusing on particular scenes and
specific events you have seen or, better yet, that have happened to you.

Given the way contexts spill over into one another and collide, we should expect
certain conflicts to come about. And they do. How much are we to view the writing in
terms of the assignment, the purposes or "intentions" we designate for the writer to
take up? How much are we to let students themselves decide the course of their
writing and the genres they employ?'O In reading "The Four Seasons," we have to
decide how much we are going to allow, or even encourage, David to shape the
writing in his own ways. Are we going to approve his commonplace descriptions of
the seasons? Are we going to call him on the fact that he doesn't adequately treat the
significance that the Syracuse seasons have for him? Are we going to contest his
mechanical, self-conscious opening paragraph as inappropriate for the kind of writing
we have assigned? In the protocol, the teacherseems inclined to allow the student his
intentions-up to a point. She is willing to have him go with his broad treatment of the
four seasons. But she is not so interested in having him pursue his own intentions that
she is willing to forego her interest in seeing more concrete descriptions, seeing him
adopt a new way of getting in and out of the essay, and seeing more of his experiences
and views of the seasons in Syracuse. Her way of reading the writing is clearly
influenced by the assignment and what she expects this kind of writing to do.
The Work in Class

Beyond reading within the rhetorical context, beyond reading in light of the
assignment or a particular genre, we must somehow read student texts with an
eye on the larger pedagogical setting. Invariably, the work we do in class and the
expectations we bring to student texts conspire to make us strong readers, determined
to privilege certain features of discourse. If we have been emphasizing introductions
and conclusions, we tend to look for how students begin and end their own writing. If
we have spent several class periods introducing conventions of reasoned evidence in
argumentative writing, we usually look for such features in student papers. The goals
we set out to achieve in the course also influence our reading and response. If we are
determined to lead students to come up with polished written products, we will be
more attentive to matters of unity, coherence, clarity, and correctness. If we are
intent on giving students practice in their writing processes, we will emphasize certain
strategies of writing and probably place less emphasis ontextual completeness. If we
are committed to preparing students for their future courses, we will be more attuned
to the conventions that are practiced in academic discourse.

The protocol response to "The Four Seasons" illustrates how elements from the
classroom context routinely become the focus of our reading. The teacher, struck by
David's generic descriptions, thinks that all the work they did with detail over the past
couple of weeks has been lost on him. The lessonon Annie Dillard's meticulous
observation is called to mind and prompts an emotional response that leads her to look
all the more expectantly for signs that David has or hasn't taken it to heart. A moment
later, she is taken aback enough by the writing that she wonders if he even read the
assignment. Soon after, she is taken by his image of the heat rising off the pavement.
"This, " she tells him, is the kind of detail I've been looking for. " Optimally, our
reading is an extension of the larger classroom conversation, our emphases as teachers
reflecting the instruction and goals of the course (Knoblauch and Brannon; O'Neill
and Mathison Fife; Straub, "Classroom Instruction").
These contexts-the assignment, the work in class, and the teacher's emphases and
expectations-form some of the most powerful influences in our reading
of student writing. They may be plotted as part of the larger classroom contexts that
inform the ways we read.

These contexts are not neat or mutually exclusive. One context spills over into other
contexts in the way that bays, inland waterways, and oceans spill into one another:
although they are at some point, from some perspective, clearly distinguishable as
entities on their own, with their own distinct properties, they flow into each other and
merge. A single comment, in fact, may invoke several contexts simultaneously, for
example, the rhetorical context and the classroom context, as in the following
response: "Good sharp detail [the text]. It's the kind of `concrete naming' [classroom
context] that will help readers picture the scene [rhetorical situation]." It's not
important to know just when one gives way to another. What's important is that, when
we read, we immerse ourselves in these waters.

The Individual Student

The revived interest in the teaching of writing in the 1960s and 1970s, the New
Rhetoric that would eventuate in contemporary composition studies, was powered by
two major concerns: a concern for viewing all writing, even student writing, as
rhetorical; and a concern for invention and, with it, an interest in composing
processes. Both concerns would lead compositionists to an inevitable interest in
the student writer as an individual: her experiences, her ideas, her composing
methods, and struggles as a learning writer. If writing is, more than clearly and
correctly laying out an argument, a way of saying something to someone
else, student writers must be guided to find something worth saying. If they are to find
something to say, they must learn to investigate their experiences. If they are to form
these ideas more aptly in texts, they must be given practice in drafting and revision.
Since students are at different stages of development as writers, coming from
sometimes vastly different backgrounds and having very different discursive
experiences, educators have looked more and more to gear instruction to the
individual student. The field has devoted so much attention to students as individuals
that "the student" has become a context of its own in the teaching of writing-not
unlike the study of the author's life and work developed into a context of its own in
traditional literary scholarship. The next three contexts of student writing bring this
"individualstudent" into the picture and prompt us to view the writing in light of her
identity and experience.

The Student's Writing Process

As teachers, we are in the rather special position of seeing not just finished texts, but
writing on the way to realization. We get to see the writing as it unfolds, help students
as they draft and revise, and teach the act of composing. Our ways of
reading student texts are dramatically influenced by the extent to which we view
writing as a process and allow the drafting to determine what we look for and
consequently what we see. When we read with such an orientation, we look at a piece
of writing as part of a larger process, viewing early drafts with an eye toward revision
and later drafts in relation to what the writer has come up with before. We generally
read first for overall content and organization, and deal with matters of paragraph
structure, sentence style, and word choice only when the larger conceptual matters
have been worked through (though not necessarily worked out) or as the final draft
approaches. We sometimes even look at final drafts, not summatively as finished
products, but in terms of how they may be revised.

Viewing writing as a process, then, colors the lenses that we bring to student writing.
Once we decide whether to encourage David to continue work on "The Four Seasons"
or call the writing to a halt, we make one giant step in determining what we look for
and how we read. As soon as the protocol teacher, early in her response, asks the
writer "which trees? What kind of flowers?" she is already orienting her responses
toward revision. She's not looking back at writing that is complete; she's implicitly
looking forward to writing that is to be done, considering the writing in terms of how
it might be improved. She also anticipates revision when, on noting David's over-
reliance ongeneralizations in the fourth paragraph, she remarks: "Let's hear some
details. " Her commentarybecomes explicitly concerned with guiding
the student toward revision half way through her end comment:

* Try focusing on particular scenes and specific events you have seen or; better yet,
that have happened to you. Maybe it'd help to sketch out a few lists of things that you
associate personally with these seasons, select the ones that are most vivid and
representative, and work those into your essay. This might be one you choose to
revise.

By taking a process approach to our reading and viewing student writing as work in
progress, we don't have to deal with everything at once. We can slow down,
focus on certain issues at certain times, and deal more fully with the concerns we do
take up (Krest; Sommers; Straub, "TeacherResponse"). We can guide students more
purposefully through their work as writers and use our responses to teach, not just to
critique or grade.

The Student's Work in the Course

The ways we read student writing are also influenced by how we envision the "text"
of the course. How much are we going to look at the student's writings discretely, as
separate projects? How much are we going to look at the writing in terms of
the student's ongoing work in the course in light of his other writings, the strategies he
has been working on, and his development as a writer? How does this paper stack up
against the other papers the student has written? Is there some quality of writing that
he's been working on that should be addressed? Is there any evidence of progress with
some composing strategy? In the sample case, is David's voice more distinctive than
in his earlierpapers? Are his sharp, imagistic descriptions in this paper notably better
than what he has come up with before? These prior texts form yet another context,
another "text beyond the text," that we may attend to in our reading and response. The
protocol teacher makes several evaluations that indicate that she takes an expansive
view of "the text." She looks at "The Four Seasons" as a discrete paper and as a work
in progress, yet she also views the writing in terms of the student's overall work in the
course. She has evidently seen some potential in David's writing and has come to
expect a certain level of work as a result: "He's done some sharp writing for short
stretches in his earlier papers too, but he's not been able to sustain it. He can do better
than this." Paying attention to the student's evolving text across the course is one of
the surest ways of individualizing instruction and using our comments to meet
the student where he is in his work as a learning writer (Krest; Onore; Phelps; Straub,
"Classroom Instruction").

The Student Behind the Text

Our evaluation of student writing is also shaped by our "reading" of the student-our
sense of thestudent behind the text. Our reading of a paper like "The Four Seasons"
may well be affected by our sense of David as a writer and as a student in the class:
his attitudes, efforts, and capabilities. We may also view the words on the page in light
of our ideas about David's experience and identity, whether as an independent agent
or, increasingly now, as a socially constructed subject, defined (among other things)
by race, gender, class, sexual preference, and religion. How fully has he captured the
kind of winters he has experienced? What personal associations does he have of
spring in Syracuse? How may his perceptions of Syracuse and the seasons be seen as a
function of his race, gender, or class? When we read student writing, more often than
not we read the text and the student or person behind the text. More than evaluate,
rank, and judge, we diagnose, coach, guide, explain, and teach. Whom we are
coaching, guiding, and teaching is as important as what we are teaching. If
the student has admittedly had a struggle writing about a subject that is too close to
him, we might steer him toward a different topic. If we know he lacks confidence or
has a difficult history with writing, we might be more inclined to look for successes to
build on. If he has struggled before and is now making progress, we might look at his
writing more forgivingly and play up his accomplishments in our response. If he has
done solid work on the first three papers, we might well expect more from him on the
fourth. "Students use and learn from our comments when we monitor their writing
rather than simply evaluate their final papers," Margie Krest says. "At each point in
our monitoring we must strive to be sensitive to our students as writers so that our
comments foster positive attitudes about writing" (38-39).

The protocol teacher is sensitive to the writer and the person behind this text. Even
before she begins her reading, she has clicked in to certain ways of seeing the writing
in terms of her take onDavid as a student writer. She picks up the first paper, pauses
when she sees that it is David's, decides nevertheless to forge ahead, and quips to
herself: "Let's see if he walks the walk this time." Not much later, she reads the text
against the background of his attitudes toward writing, stroking his confidence even as
she uses this confidence to push him to work harder: "she doesn't want to take away
from the praise she's just given-he needs the applause-and decides for the moment to
let it go." She also makes some inferences from the text, for better or worse, about
David's motives as a writer:

* He's getting careless here, she thinks, relying far too much on cliches about spring.

* He's just blowing through these descriptions, not thinking about them. Coasting.

But she's careful not to confront the student with such presumptions about his motives
and processes, and reminds herself to keep the commentary constructive. Not
surprisingly, when she looks at the writing in terms of David's personal experience she
sees this experience in psychological terms, according to his own individual
perspectives, not in terms of any social identities.

In addition to looking at the text in terms of the larger rhetorical and pedagogical
context, then, we may view it in terms of some image of the student behind the text,
his particular experiences and views, building this image on information we get
directly from our interactions with the student or indirectly from the student's text. In
doing so, we read-not just the writer's texts-but the writer. These writerly concerns
may be plotted as a general context that, like the classroom context, provides a broad
interpretive frame for reading the text and its immediate rhetorical context.

Classroom writing never comes as a text on its own. It always implies a rhetorical
context, a pedagogical context, and a particular student writer. These contexts
routinely come into play in our reading, as a matter of course overlapping,
complementing, and competing with one another. Such interanimation of contexts is
one of the things that makes reading so resourceful. It is also one of the things that
makes reading student writing so complex and demanding. Problems occur when we
do not acknowledge these various contexts as focuses for our reading. In pursuing
other treasures, we read right over them. Other problems occur when the contexts
come into play (as they will, spontaneously) and we do not know how to attend to
them. We take up one, we take up another, as we will, but with no clear focus or
purpose. Still other problems occur when the contexts we do address do not match, in
kind or degree, the emphases of our classroom instruction. We spend two weeks
considering how writers should adapt writing to the audience, and then when we read
we focus almost exclusively on organization and detail-or we read from our own
specialized position as writing teachers and do not reach beyond this role to give the
writer a sense of how the writing might be read by prospective members of its target
audience. As teacher-readers, we have to develop a repertoire of concerns and a knack
for reading student writing in ways that will complement and reinforce the work we
are doing in class. To do so, we have to reflect on our day-to-day teaching, critically
examine our instruction in terms of our goals, articulate the contexts and concerns that
are most important to the course, and focus on them when we read our students'
writing. As we ask students to keep certain key features of discourse in mind as they
write, we should keep certain features of writing-and certain contexts-in mind as we
read and respond.

OTHER CONTEXTS THAT INFLUENCE OUR READING

The text, the rhetorical context, the classroom context, and the student context are the
most obvious frames that inform our reading and establish the features we look for
when we readstudent writing. But there are other contexts that are less obvious but
often no less powerful in the effects they have on our reading: our approach to
teaching writing, our interpretive theories, the academic setting, the writing program,
the larger institutional interests, our own prior experiences with teacher response, the
immediate circumstances of our reading. In fact, these concerns may be so ingrained
that it may be difficult to see them as sets of expectations, as contexts and constraints,
that influence (and make possible) our reading.11 We don't focus on features
associated with these contexts in student texts. We rarely acknowledge them as we
read, and we almost never make reference to them in our comments. Nevertheless,
they are so imbricated in what we do that, even though they usually remain tacit, they
shape what we look for and hence what we see. These concerns, which are brought to
the text in the form of teacher expectations, may be plotted as a backdrop for the more
immediate contexts that inform our reading, illustrating the full complexity of the
interests and influences that define us as teacher-readers.

Approach to Teaching Writing

In "Composition Theory in the Eighties," Fulkerson looks to sort out the various
components that make up a theory of composition, or what I am calling here an
approach to teaching writing.12 A full theory of composition, he posits, must have at
least four components, each of which, to avoid confusion, must be kept distinct:

* a "philosophy" made up of one's goals for instruction and a set of criteria for good
writing;

* a conception of the writing processes students might employ in order to reach these
goals;

* a set of classroom practices that are designed to bring about these goals;
* an epistemology, a conception about what counts for knowledge.

Each of these theoretical components has some real effect on how we read and what
we read for.

There is no theoretical commitment, no matter how explicit or tacit it is, that more
profoundly influences our ways of reading than our goals for instruction. Is the
writing class a service course? Does it exist primarily to clean up student writing and
perform a gatekeeping function for the university? Is it to give students practice in
their composing processes and help them become more confident writers? Is it to help
students learn to write better, to use writing to learn to think more independently and
more critically, and see what value writing might have for them in their lives? Is it
designed to prepare students for the writing they will be expected to do in other
courses-or somehow prepare them for the writing they may have to do when they
take on a job? Is it to help students develop powers of critical thinking, introduce them
to the latest electronic media, engage them in the examination of their culture, or
expose them to other cultures? These questions about the aims of composition
turn on the broader issue of how we envision the larger goals of college education. Is
an undergraduate degree meant to secure students a good job? To broaden their ideas
about the world and understand their place in it? To lead them to become good
citizens? To mobilize them to resist oppressive social structures? These matters are
often decided for us, at least ostensibly, by the writing program, by the department, by
the college, or by the teaching approach we adopt. But they are emphasized or de-
emphasized according to the decisions and actions of the individual teacher. How we
envision these larger goals of instruction will influence how we read and what we look
for when we read and evaluate student writing.

Our reading will also be influenced by how we view knowledge and envision the
nature of the writer's content and meanings (Anson, "Response Styles"). Are they to
come primarily from some fixed common knowledge? Are they to be appropriated
from the statements of some prior discourse? Are they to come from the writer's own
circle of experience-- his own views and personal knowledge? The content
of student writing depends to a large extent on the assignment and the genre of
writing, but it also depends on the writer's specific subject matter. The more we lean
toward an expressivist view of writing, the more we'll value the writer's discovery of
ideas based on, and giving shape to, his own experience. The more we lean toward a
social view of writing, especially some recent social constructionist views, the more
we'll privilege subject matter outside the self and look at invention as a matter of
discerning what has already been said and negotiating various voices and perspectives
in prior texts (Bartholomae; Bizzell; Lu).

Our ways of reading are also affected by our pedagogical theories. If we see writing
instruction as a matter of simply giving students plenty of time to write in different
genres and along the way get feedback from others in the class, we will be more
inclined than we otherwise might to give students a lot of room to make their own
decisions as writers (Elbow, Writing Without Teachers; Moffett; Murray). Instead of
specifying changes we'd like to see made, we will provide reader responses, make
suggestions or frame our comments as questions for the student to take up as he will,
in his own way. If we believe that class time is best used to provide students direct
instruction in writing or engage them in hands-on practice with certain composing
activities like observing, planning, defining, and arranging, we will be more inclined
to look for instances where students make, or might make, use of these strategies
(Hillocks, Berthoff). If, in an effort to emphasize the social dimension of writing, we
look to turn our classroom into a microcosm of a discourse community and establish
students themselves as the sole audience of all course writing-- and perhaps even as
arbiters of what is to count as good writing-we might find ourselves in the curious
position of having to resist many of the comments that come naturally to us as
teachers (Cooper). We might not have recourse, for instance, to question a text that
lacks unity, relies too much on cliches, or is utterly informal.

The protocol teacher appears to adopt an approach that is largely expressivist. She
assumes an expressivist axiology, a subjective epistemology, an expressivist view of
the composing process, and a mixed pedagogy. She posits expressivist goals inasmuch
as she looks to foster the development of the student's own thoughts, the development
of his own voice, and his development both as a writer and as an individual. But she
seems driven not simply by an interest in the student's self-expression but also by a
keen interest in writing for readers, the defining trait of a rhetorical axiology. She
clearly assumes that knowledge and truth are located in personal experience and
perception. She enacts a vision of the composing process as a fairly individualistic
activity involving a series of recursive activities of planning, drafting, and revising.
She seems to shuffle between ateacher-centered and a student-centered pedagogy. She
relies on the presentational mode of analyzing model texts and makes frequent use of
writing workshops, including collaborative peer-response workshops. At times, she
seems intent on directing David to make specific changes. At other times, she seems
content to raise an issue for him to take up as he chooses.

These theoretical commitments find their way into this teacher's reading of "The Four
Seasons." It is not enough that David describe the seasons, he must describe them
through his own experience. He must do this, as the assignment reminds him, in ways
that reveal the significance the Syracuse seasons have for him and, what's more, that
show how these descriptions are unique to him. She focuses primarily on the content
of the writing, but she also deals a good bit with correctness, sentence structure, and
the formal shaping of the writing, including the use of such formal devices as thesis
statements and transitions. For all her attention to matters beyond the text, she still
sees the writing primarily in terms of a cognitive view toward text making.13 If her
work with David's essay here is typical, this teacher, it might be said, is fairly
formalistic and rhetorical in her ways of reading student texts.

But ultimately, with all the evaluative remarks she makes, both negative and positive,
she seems to assume a fairly authoritative stance in her encounter with David's
writing. Although a theory of composition more often than not remains tacit,
the teacher typically acting intuitively according to quiet promptings, it exerts a
powerful influence over our ways of reading and responding to studentwriting.

Interpretive Theory

Our ways of reading are often colored by our literary theory (or theories). The
strategies that we bring to literature spill over into our reading of student writing,
leading us to look at texts in certain ways, with certain emphases, for particular kinds
of meanings. A lot of us, steeped in Modernism and raised on the legacy of New
Criticism, look to perform close readings of the text, appreciating statements that
"show" rather than "tell," seeking out patterns of imagery and theme, and demanding
coherence. Many of us, informed by reader-response criticism and attuned to the
process of interpretation, look to dramatize our ways of negotiating meaning and point
to productive or troublesome gaps in student writing. Others of us don the lenses of
structuralism and attend to various conventions of discourse or look through a
Bakhtinian lens for the voices and intertextual struggles at work in a text. More
recently, under the pervasive influence of poststructural literary theory, we may find
ourselves looking, among other things, to reveal the conflicting voices, hidden
ideologies, and unresolved tensions that are inscribed in the words of our students'
texts. Although evidence of such theoretical influence is not always readily noticeable,
the pull they may exert over our reading is no less forceful.

An even more forceful, if only slightly more obvious, influence in our ways of
reading student writing is exerted by the roles we bring to the text as readers, among
them:

1 Teacher-reader as evaluator (focused on assessment)

2 Teacher-reader as educator (focused on instruction)

3 Teacher-reader as universal audience (representing readers in general)

4 Teacher-reader as target audience (representing a specific group of readers)

5 Teacher-reader as a common reader (presenting herself as an individual reader)

6 Teacher-reader as implied reader (assuming the role suggested by the text)

The traditional default mode for teachers has been to read the text new critically as an
evaluator, or in what Alan Purves calls the role of judge, with the teacher-reader
looking primarily to assess the strengths and weaknesses from a privileged position
above the rhetorical context, typically without any reference to the audience. A second
customary role for teachers to assume is that of instructor, akin to what Purves calls
education's version of the diagnostician/therapist. This teacher-reader also establishes
a certain distance from the text and positions herself outside the immediate rhetorical
context. But instead of looking to critique the writing or fix it, she looks to read the
writing from the perspective of someone primarily interested in helping or improving
the writer.

Increasingly, however, as Fulkerson notes in his survey, contemporary teachers

insinuate themselves into the rhetorical situation and assume the role of some
prospective audience. But here again there is more than a little complexity.
The teacher, we have come to see, may take on several different roles as "a reader."
She may assume the role of an abstract, universal audience, and speak for how the
writing works for "the reader" or "readers," without any specification of who these
readers might be. She may act as a representative of the paper's target audience,
whether it is one specified by the assignment or somehow indicated by
the student writer himself-that is, in the role of some actual audience whom the writer
is trying to address, what Ede and Lunsford call an "audience addressed." She may
read the writing as an individual, common reader, showing students how she
experiences, understands, and reacts to their writing, whether she highlights her own
subjectivity or masks it in favor of some less personal, less partial perspective (Elbow
Writing Without Teachers; Gibson; Macrorie; Moffett). Or she may resist bringing a
predetermined role to the text and look instead to pick up on the readerly role that the
text implicitly invites readers to adopt what Ede and Lunsford call an "audience
invoked." A teacher looking to discern the reader implied in much of "The Four
Seasons," for instance, might find the easy-going, accepting audience of popular
magazines or local newspaper columns. These roles are not distinct or airtight.
A teacher does not simply adopt one to the exclusion of the others. A
given teacher may emphasize one role or another, but she will likely assume different
roles in a single reading-and perhaps all of them over the course of a full set
of student papers. The object, again, is to match our reading roles with our instruction.

The protocol teacher adopts a number of these reader roles in her response. She is
evidently at ease in the role of the teacher as evaluator. Most of her interactions with
the text, in fact, are made from this perspective, among them:

* An okay start, she thinks. A good thesis. But he's slow getting into what he has to
say.

* He's just blowing through these descriptions, not thinking about them. Coasting.

She very frequently casts herself in the role of teacher as educator: Don't attack, she
thinks.

* Keep it upbeat.

* He's done some sharp writing for short stretches in his earlier papers too, but he's
not been able to sustain it. He can do better than this.

But she also assumes various roles as a teacher-reader. Try to grab the reader's
attention, she tells David at the start of the paper, invoking the needs of readers in
general. Much more often, though, she assumes the role of the target audience:

It's an image that will lead your readers to imagine how hot it gets in Syracuse.

* These are all general terms. Let's hear some details.

From time to time the protocol teacher even takes on the role of a subjective reader.
"Good," she thinks as she reads about the moist ground wanting to seep into your
shoes, "a really nice image," and then tells David, " I can feel the squishing. Moments
later, reading the description of the trees beginning to blossom and the faint smell of
pollen lingering in the air, she thinks of the cherry trees that used to grow in her yard.
Both reactions register impressions that the writing has elicited from her as a real
reader. This is the kind of response Peter Elbow extols when he speaks about
providing students with "movies of the reader's mind," that is, the ways
the teacher processes the text moment-to-moment as an everyday reader ("Options").

Some of us may like to think that we leave our subjectivity behind when we teach a
class or read astudent paper, but nothing could be further from the truth. Our work as
teachers, like our students' work as students, is saturated with our own experiences,
interests, and values. Response and judgment, Robert Schwegler reminds us, "are
shaped by the reader's knowledge; ideology (personal, social, cultural); social
situation; and interpretive strategies (often shaped themselves by social and cultural
ideology and class) as well as by textual features and content" (212). We readily
challenge sexist remarks about men mowing and raking while women are left to plant
flowers. We overlook stylistic flaws when a student writes about the sudden death of
her father. We are easily enamored by an essay about Yellowstone National Park if we
have just made our first trip out there ourselves. No matter which roles we adopt
as teacher-readers, no matter how much we are wont to admit or try to suspend our
own views, two things are clear: (1) what we read for and how we read are invariably
affected by our own assumptions, interests, and values, and (2) just how much we
admit-or try to suspend-our personal values and beliefs in our reading greatly
influences how we read our students' writing. Any monitoring of what we look for
when we read student writing must involve an examination of our roles as teacher--
readers (Cowan; P. Murray; Purves; Straub and Lunsford), the ways our values and
experiences may come to bear on our ways of reading (Fish; Freund; Tompkins), and,
as some suggest, even our personalities (Thompson).
The Academic Setting

Regardless of the type of writing we pursue or the kind of community we look to

create in class, we are always somehow influenced by the fact that we are dealing with
writing in an academic setting. We might, for instance, place greater value than we
otherwise would on thoughtfulness and reasoned evidence. We might privilege
analysis over emotion or seriousness over casualness and humor. Lil Brannon and C.
H. Knoblauch have shown how readers, when they invoke the perspective of the
academy, foist on student texts standards and values that are peculiarly academic-and
that wouldn't be stressed in other settings. They cleverly use the example of how our
bias against allowing students to rely on emotional argument might well be different if
we were acting in another discourse community, say, in a court room and not a
classroom. Similarly, Joseph Williams has shown how teachers' harsh, bionic eyes for
error in student writing are made all too human when they don the lenses of
nonteachers and read other kinds of writing.

As members of the academic community, we are primed to look for a certain formal
tone in student writing, pay special attention to correctness, and look for
other conventions and discursive practicesthat are valued in school. The
protocol teacher's response shows the pressures of these constraints. In spite of her
expressed priorities, even as she sees that the draft in front of her is not yet ready for
word-by-word editing, she has trouble resisting taking note of David's problems with
transitions and inappropriate usage. She assumes that writers should have something
fresh and distinctive to say-or at least have something to add to the communal
storehouse of knowledge. These concerns are fundamental to our expectations as
writing teachers. They may be so familiar and obvious, in fact, that it may be difficult
to see them as expectations at all. But when our expectations as teachers become so
rote and generalized that we fail to recognize the particular context of the writing or
become so strong that they override the student's choices and purposes, we impose our
own idealized text on students, appropriate student writing, and ultimately discourage
students from giving themselves to their work as writers (N. Sommers; Brannon and
Knoblauch).

Writing Program Constraints, Institutional Constraints, and Grades

There are still other contexts that quietly but routinely shape the ways we read student
texts. Some writing programs exert an obvious influence over the teaching staff,
requiring the use of certain textbooks, emphasizing process or product, supporting or
not supporting collaborative learning, pushing one kind of writing or another, and, in
doing so, placing a premium on some features and contexts of writing and
downplaying others. The protocol teacher's decision to resist comments about wording
and transitions or to hold back on critical commentary could each derive from the
influence of the writing program. The larger institution itself may also exert a palpable
influence. Some institutions create an atmosphere where students are seen as mere
clients or consumers, and teaching is seen as just another task to check off a much
larger schedule. Other institutions create a more positive environment, where students
and teaching are the heart of the matter. These overarching views toward instruction
surely have an impact on how teachers see themselves as writing teachers, how they
see their classes (and how much they put into them), and how they read their students'
writing.

One of the most obvious and powerful institutional constraints that affect our reading
of student writing is the requirement to give grades-an issue that is always bubbling at
the surface of new teachers' concerns and simmering just below the surface when
any teacher reads student writing. I'm not interested here in the question of how we
decide on grades-or if we should put grades onindividual papers. I'm interested in
noting how putting grades on student papers affects the ways we read and respond to
student writing-sometimes significantly. Consider the case of the protocolteacher.
She's reading what's intended as a final draft of "The Four Seasons." She sees, as she's
reading, that it is not realized enough to warrant closing the project down and treating
it as a finished essay. She responds to the paper as if it will be revised. When she
decides to slash her original C into a cross between a C and a B, I would suggest, she
makes a choice that affects her final take on the writing and that affects her response.
With this generous grade, she allows David-and herself-- more room to rest content
with the paper. She can back away from it cleanly and move on, giving the student a
wide berth in whether or not to go back to it and revise. But if instead, let's say, she
had opted to give more play to the skeptical side of her reading and gave the paper a C
or C-, her posture would change, her reading would change, and her response would
almost certainly change along with them. Her response would likely get more
evaluative, more defensive.14 Putting grades on student papers-or refraining from
putting grades on papers, for that matter-invariably marks our ways of reading and
responding to student texts (Elbow, "Ranking"). It is another powerful constraint to
consider when we consider our work as readers of student writing.

The Teacher's Past Experience with Teacher Response

It is a basic principle of response, especially when we first start out as teachers: we do

unto others what has been done unto us. We make the kinds of comments we
remember getting on our own writing: they are what we are familiar with, what we
have come to associate with response. So our first ways of reading, our default
response style, is usually some version of traditional teacherresponse: rather
abbreviated if not terse in tone and style, and usually evaluative and controlling.
Whatever stylistic features we develop later are often echoes of comments that have
come from teachers in our past who have made an impression. How often have I
started out an end comment with the kind of high praise I remember once getting from
Joan Bobbitt on a graduate paper I wrote on Thomas Hardy: This is a herculean effort!
How often have I acknowledged a student's taking on a formidable task the way I
remember Jim Phelan acknowledging my own attempt at the formidable task of
coming to terms with his first book: Although I can't say that I'm finally persuaded by
your argument, I admire and appreciate the careful reading you give this work. How
often have I taken heart at feeling pressed to be upfront and critical in a comment,
knowing that I could try to frame the criticism in a way that was both sharp and
helpful, as Barnett Gutenberg did a long time ago for me when he wrote on an
undergraduate paper I'd written about Jay Gatsby and Thomas Sutpen: Writing is
meant to express, not impress. Too often you go for the flashy word instead of the
right word. Try to say more simply and clearly what you have to say, in language that
you can control. How much has my own upbeat, encouraging response style evolved
as a reaction to the burning humility and anger I have felt on receiving comments that
were insensitive and (as Nancy Sommers so aptly puts it) mean-spirited? I am
convinced that the voice and fullness of my own comments as ateacher are a direct
result of the long line of teachers I've had, from high school through grad school, who
respectfully and generously gave themselves to nurturing my work through their
comments. And I believe all of us, as Janet Emig and Robert Parker suggest, have
such influences from our pasts at work in the ways we read and respond to our own
students' writing.

The Immediate Circumstances of Reading and Responding

The pull of all these contexts may take effect only as another context-our immediate
circumstances as we read-allows them to take hold. The number of papers we have,
the mood we're in, the order in which we read them, the number of papers we've
already worked through, the number left to go, the other work we have to do before
the next class, all the things that come to mind, fill our hearts, and would pull us away
from our reading. And, of course, time. How much needs to be said about how time
itself affects-or downright determines-how we read and respond to our students'
writing? The way our readings get fuzzier, our comments shorter and grumpier, as we
spend more and more time on a set of papers and get more tired, less patient, and more
than a bit less forgiving? The way our comments get astonishingly more efficient as
class time rapidly approaches-and remarkably more benign if we hand papers back a
week late or a day or two after all the others? Will the protocol teacher have the time
to respond this fully to a Glassful of twenty-four students? Will she be as involved and
clear-headed on the twelfth or twenty-fourth paper as she is with David's, her first?
Will the rote introduction in this paper trigger reactions to other such introductions in
laterpapers? Will David's sharp images and easy cliches lead her to be less accepting
of another student's cliches? Just how much sway these immediate circumstances
exert on our reading will depend, of course, on how clearly we are able to keep in
mind our vision of what we want to accomplish. The more contexts that we bring or
that enter into our reading, the fuller and more textured our response. But the more
contexts we bring into play-or, worse, that come into play unbidden-the greater the
potential for distractions or confusion. One of the main difficulties in learning to read
and respond to student writing is just this: we have to decide, from a dazzling array of
options, on the terms that we are to read by and the concerns we will most value in our
reading. Further, we have to figure out how to match these criteria to our individual
teaching styles and our particular classroom goals. And, further still, we have to
discover what is most important on this paper, for this student, at this time. And, yes,
always, always, working against time. It's no wonder, then, why we learn quickly to
put our best battle faces on when we bring home a new batch ofpapers. We know it
can easily turn into a struggle.

ANALYSIS OF THE PROTOCOL TEACHER'S READING

Fulkerson grants a major status to the values that teachers bring to their reading of
writing-right up next to their goals for instruction. In fact, he says the most obvious
inconsistency within a giventeacher's theory of composition "is subscribing to one
axiology and then evaluating from another" (422). He does little to define the nature
of this crucial component of composition theory or elucidate its importance-they are
not central to his project. It is my purpose to do both because they are central to mine.
A teacher's concept of what makes writing good is the most sensitive and vulnerable
contact zone between her theory and her classroom practice-and the best place to
begin to reflect on one's work as a teacher. In the following section, I will analyze the
protocol teacher's reading of "The Four Seasons" and consider (as best as I can given
the limitations of this scenario) how the values she enacts in her reading square with
her commitments, theories, and practice. In doing so, I'd like to help teachers see how
they might use this map of criteria to examine their own ways of reading, clarify their
own values and goals, and reconcile their own theories and practice.

The protocol teacher does well to reflect her theories and values in her reading of
David's essay. We would expect such a teacher to emphasize content over
organization, style, and correctness, and she does. Fully half of the fifty interpretations
she makes across this reading protocol focus on the writer's ideas and descriptions.
Fittingly, almost all of them are concerned with how well David makes these
descriptions of the seasons vivid and personal, putting into practice her subjectivist
concern for the writer making his own meanings. Another ten interpretations focus on,
or look at some textual quality in terms of, the individual student writer: his
experience, his involvement in the writing, and his earlier work in class. Such routine
evocation of the "student" context puts into practice her emphasis on looking to
improve the student writer over simply improving the writing. Six additional
interpretations view the writing in terms of the rhetorical situation. Only nine of the
matters she takes up in her reading deal with local matters, and over half of all her
interpretations invoke some larger context of writing.

Identifying the concerns a teacher takes up is only a start. Other questions also have to
be considered: How does she define her key criteria? What, in her view, constitutes
good content? Has she been emphasizing such a view in her day-to-day instruction?
What sort of voice does she most value? Is such a valuing consistent with this type of
writing and her pedagogy? What makes organization "effective"? When she calls on a
student to make his style "more direct" or "more informal," what exactly does she
mean? Has she made these meanings clear to students? We never simply apply a
prepackaged set of criteria wholesale to our students' writing. If our terms, as Bakhtin
reminds us, always come shot through with prior meanings, they also get filtered
through our own experiences and perceptions. We give them our own particular twist,
our own individual take, emphasizing one connotation or another. As teachers, it's our
job to unpack these meanings, talk with students about what we mean when we use
them, and explain how they fit in with the methods and goals of the course. For the
protocol teacher, for instance, good content depends to a large extent on the originality
and distinctiveness of the writer's descriptions. Such a definition fits in well with the
type of writing the class is working on and her expressivist goals. A writer's voice, in
order to be effective, must be informal and personal-- a voice that speaks casually
with readers and captures something of the writer's individual way of seeing the
world. Such an understanding also seems to go well with the descriptive, personal
writing she calls for on this assignment, and it is in line with her student-centered,
expressivist pedagogy.

There are, however, some problems and inconsistencies. She seems to think that
student writers should be fairly explicit in structuring their writing. They would do
well to announce their focus in a thesis statement, forecast topics to come, and make
explicit transitions. But are explicit thesis statements necessary or useful in the context
of such personal writing? Can such thesis statements typically be found in the
personal essays of published writers? Are they consistent with her expressivist
pedagogy? Even if these values might be reconciled with her theories, there is another
related issue: has she done enough to talk about the nature of such formal strategies of
organization in class?

Her concern with transitions also seems misguided. She reads the paragraph about
summers in Syracuse and sees a problem with an overly mechanical transition. But it
seems more apt-and more theoretically consistent-to see what happens here as a matter
of the writer failing to get beyond the obvious statements one might make about the
seasons: not a formal problem so much as a problem of invention or a rhetorical
problem involving the writer's failure to make the writing responsive to readers. Her
concern with David's use of the second person "you"-presumably because it is too
informal-is also problematic. The trouble is that the writer's use of "you" seems well
within the bounds of the genre and not unsuited to the assigned audience for the
assignment: those (common readers) who are unfamiliar with this place. The writer's
use of "you" is also not inconsistent with the conventions of personal writing, which
are more relaxed than the constraints of academic discourse. At any rate, like the other
instances above, it seems an unnecessary concession to formal academic discourse,
given her pedagogical allegiances.

There are more significant problems. When David fails to provide more descriptions
based on his own experience and views, she chides him for not following the
assignment. Yet her student-centered pedagogy-- and her own classroom policy-
allows students great latitude in interpreting and following the demands of the
assignment. (To her credit, she does not express this concern to David in her
comments. She also doesn't insist that he focus his writing on only one or two seasons
if he takes up a revision-or that he go back and revise this paper.) More troublingly,
she criticizes David for not adopting the voice of an expert on the Syracuse seasons
and establishing his authority as a writer. The fact .that he is writing a personal essay
and the fact that he is writing for an audience of everyday readers both mitigate
against her expecting David to adopt the voice of an expert. Such an authoritative
voice might be appropriate for an academic audience. It might even be appropriate for
some magazine audiences. It might be rightly expected in a course based on a social
constructionist axiology or a social epistemology. But it is not necessary for the
circumstances of this assignment or the framework of this class. There is no reason
David cannot effectively meet the demands of the situation laid out for him by
employing the casual personal voice he adopts here-or, for that matter, the second
person "you" perspective.

This is a method any writing teacher might use to examine her own theories
and practices from the ground up, starting with what she actually looks for in her own
students' writing and then trying literally to come to terms with them. Do you
emphasize the content of student writing as much as you think you do? As much as
your pedagogy calls for? Do you deal with organization in ways that are consistent
with the genre of writing and your theories? Do the contexts you address reflect your
values and classroom practice? If you are an expressivist, does your reading deal in
some detail with the student's own experience and views? If you espouse a rhetorical
axiology, do the majority of your responses focus on the writer's persona, purpose,
and audience? If you are a social constructionist or if you employ a cultural pedagogy,
does your reading emphasize the social side of the map and deal, in particular, for
instance, with the conventions of the discourse community or the cultural assumptions
that are underneath the writer's claims? How do you understand the various terms you
employ? Do you define "substantive content" in ways that are consistent with your
expressivist epistemology? Do you talk about organization in ways that reflect your
allegiance to a rhetorical axiology? Does your emphasis on a particular kind of voice
go with our social constructionist agenda? Do you read student texts from the
perspective of the audience that you ask students to address in their writing? Do you
talk about writing as a recursive process in class but then expect students to always be
at a certain stage of writing-finished by now with their discovery and planning, done
with their drafting, focusing now only on local revisions? By sorting out our criteria
and goals, we can crystallize our theoretical commitments, bring our theory to bear
more fruitfully on our practice, enrich our instruction, and sharpen our response to
student writing. Moreover, we can make our theory and teaching and response work
together to achieve what we are most committed to achieving in the writing
classroom.

ANALYSIS OF THE PROTOCOL TEACHER'S RESPONSE TO THE WRITER

Our sense of what makes writing good is the nerve center that connects and operates
our theories and our classroom practice. Everything we assume about the teaching of
writing and want to achieve as writing teachers comes down, I believe, to what we
value and look for when we read our students' papers. This is where it all comes
together: our goals, our commitments about good writing, our views toward the
writing process, and our pedagogy. It is how we actually read our students' writing,
page by page, paper by paper, student by student, that gives breath to our assumptions
and that makes or breaks our instruction. But reading and evaluating our students'
texts is only half the work. The other half is what we choose to call to the student's
attention and how we present our readings to students in the form of comments. As
important as our ways of reading student texts are, they are finally only as useful as
they inform our work in class, are manifested in our comments, engage students in
reenvisioning their writing, and teach them something meaningful about their work as
writers. I'll make use of a simple heuristic to analyze the protocol teacher's comments
and evaluate how well she turns her reading of "The Four Seasons" into a response to
David.

The soundness of a teacher's commentary depends on how well she accomplishes four
general tasks and how well they mesh with her theory and classroom practice.15

1 How many comments does the teacher make? How well does she engage the issues
of the writing? Does she take on too much in her response? Does she make enough
comments to engage the writer in revision?

In this protocol the teacher writes 29 comments to Davidessentially 29 distinct

statements, individually and in clusters: 16 in the margins, another 13 in her end note.
It is a generous helping of comments, well above the average of 22 comments that
informed writing teachers employ in their responses (Straub and Lunsford). The
number of comments enables her to give a fairly close reading to the writing and get
into a meaningful exchange with the writer.

2 How well does the teacher specify and communicate her comments?

The comments are written out in statements that are, for the most part, text-specific
and detailed, and the teacher does a good job in expressing her points to David. In
several instances, she even goes on to explain or illustrate some initial response in one
or more follow-up comments:

* Good detail. I can feel the squishing.

* Good concrete detail... It's an image that will lead your readers to imagine how hot it
gets in Syracuse. This is the kind of detail I've been looking for.

* You're relying too much on cliches: geese flying north in the spring, summers at the
beach, raking leaves in the fall, and sitting by the fire with that special someone in the
winter. They're descriptions that we're all familiar with and they don't give any insight
into how the four seasons are unique to you.

This is not to say that her comments are always well-stated or that the response
couldn't do with a couple more explanatory comments. The student may have some
trouble making sense of her asking in the first paragraph, Why the disclaimer? He
may not get the gist of her saying Shift in point of view in the second paragraph.
The teacher would also do well to add some follow-- up explanation of what she's
looking for when, just a short time later, she writes Which trees? What kind of
flowers? or when she circles the general terms in the next paragraph and says Let's
hear some details. What kind of details is she hoping to see? What effect would they
have? Her terse Mechanical transition could also use some explanation.

3 How well does the teacher capture her key concerns and invoke what she sees as the
most important contexts in her responses? Are her responses focused and coherent?

The protocol teacher does well on this score. Eighteen of her twenty-nine comments
are focused on the content of writing, in this case, on David's descriptions. Another
dozen go beyond the formal text and invoke some larger context: the rhetorical
situation, the writer's personal experience, and his work as a writer. Only three
comments deal with local matters-all of them in marginal comments. The end note is
reserved for dealing in greater depth with large conceptual and rhetorical matters. The
response is particularly apt, given this teacher's expressivist assumptions and the
assignment, in looking at the writing in terms of David's own experience with the
seasons of Syracuse.16 It reflects what seems to be a strong rhetorical emphasis. And
it does well to provide some direct process-based advice that the writer might follow
to help him come up with the kind of substantive content that this teacher is looking
for in the writing. The protocol teacher also does a fairly good job of keeping David's
attention on the areas she considers most important at this time. She smartly avoids
presenting a number of her initial reactions in the form of comments to David-her
seeing his opening gambit as a cop out, her sense that he is getting careless and just
blowing through his descriptions, coasting. For all her good choices about staying
focused on what's most important to her at this stage of the writing, however, she
might do well to resist commenting on transitions, usage problems, and minor errors,
as her emphasis on student development rather than on completed texts would advise.
There is much more substantive revision for David to work on here before he attends
in any detail to such local matters. The teacher's marginal comments are not overly
intrusive on the student's text, and her end note is laid out well, in some detail. The
two complement one another. Her end note, further, does not merely repeat the same
concerns she raises in the margins, but focuses on certain key concerns and, for the
most part, explains them more fully.

4 How does the teacher construct herself as a teacher-reader through the ways she
frames her comments-and what kind of relationship does she establish with the
student?

The protocol teacher's comments are far more evaluative and directive than they might
be, especially for an expressivist with a student-centered pedagogy and a commitment
to giving students greater authority over their decisions and purposes as writers.
Twelve of her twenty-nine comments are presented in the form of evaluations, seven
of them negative. Much to her credit, she has a penchant for making note of what is
praiseworthy and looking to build on the writer's strengths. She compliments David
several times on his vivid descriptions. At times she even goes beyond general praise
and explains what she likes. Still, her response reveals the limits of our grammar of
praise. She likes David's way of looking at Syracuseans coming out of hibernation
after a long winter, and tells him so. Good idea, she writes. But she says nothing in her
comment about what in her eyes makes the idea good: how it is different from the
usual descriptions of people getting outdoors in the spring, how it consequently might
capture a reader's attention. She also likes the way he pursues this metaphor and sees
the emerging warmth of the season as stirring a parallel warmth of feeling among
neighbors. But again she offers only a quick note of praise-Nice-and moves on, failing
to make the most of the opportunity. And while she starts her end note with two praise
comments, she leaves both of them unspecified and unelaborated, allowing them to be
read merely as a gesture, a conventional pat on the back at the start of a teacher's end
note. How different the comment would read if she specified her praise here in the
way she specifies her criticism in the ensuing comments:

* Pretty good, David. You have some fine descriptions of the four seasons in here.
The moist ground seeping into your shoes in the spring. The heat resting on the
pavement in the summer. The aroma of dried leaves in the fall. The soft scrunching of
snow in the winter.

Her comments are made more critical by several additional comments that she
presents in the form of closed questions, all of which are more evaluative than
interrogatory:

* Why the disclaimer?

* Really? Squirrels and rabbits hibernating? Bears and rabbits chattering?

* Don't... the women in your neighborhood ever cut the grass?

To soften some of this evaluative edge, the teacher might employ more reader-
responsecommentary, the kind of commentary she provides to David's description of
the moist ground in spring: Good detail. I can feel the squishing. She might explain,
for example, how the reference to geese goes only so far to help her imagine how
nature announces the return of spring in Syracuse and to identify the kinds of trees she
associates with spring. Such specific moment-by-moment reactions and reader
responses might also help establish more of the kind of reader-to-writer relationship
she seems to value and help her put into practice her commitment to lead students to
assume authority over their writing choices. Still, overall she does a solid job of
transforming her reading into a response, using her comments to reinforce the work in
class, and guiding David's ongoing work as a writer.
CONCLUSION

Writing at the turn of the last decade, Richard Fulkerson seems hopeful that the
confusion he saw in composition studies could be substantially reduced and the
consensus he saw emerging could be developed into wide-spread agreement. Such
development, he seems to suggest, would come about mainly at the level of theory,
through the work of composition scholars. I have no doubt that theory can help us
better understand the nature and goals of composition and help reduce the confusion
between ends and means that seems rampant both in our scholarship and in our day-
to-day teaching of writing. But I also believe that theory and disciplinary knowledge
can go only so far in these efforts. The real work of clarifying our values and goals
and connecting our theory and classroom practice is a practical problem that must be
addressed first of all by teachers, individually.

In this article I have looked to meet the problem head on by calling on teachers to cast
themselves in the role of reflective practitioners and critically examine what is at the
heart of our theory and practice: our concept of what makes writing good. To facilitate
such a project, I have laid out and ordered the range of concerns that we take up as
teachers when we read and respond to student writing. I have looked, in turn, to devise
a heuristic we can use to clarify our goals, define our values and get them in line with
our theories and practice, and test how well our criteria and goals are realized in our
comments to students. More than our philosophy or theories, more than the kinds of
writing we assign, more than the classroom activities we employ, more than the
strategies we ask students to practice, it is our concept of writing and the evaluative
criteria that we bring to our day-to-day reading of student texts that enable us to merge
our theory and practice, define us as one sort of teacher or another, and determine the
strength of our teaching.

By sorting through our values, we can clarify our own commitments and develop a
sharper vision for our work as writing teachers. We can become better readers of our
students' texts, become more able to adapt our responses to specific students in
 specific situations, and, quite simply, improve our instruction. Moreover, by starting
with our work as teachers and interrogating our own theories and
instructional practices, we will also be in a better position to contribute as teacher-
scholars to the disciplinary knowledge of composition and even perhaps, over time, to
the kind of disciplinary continuity that Fulkerson and others hope to achieve. This is
the key to working through the inconsistencies that frustrate our attempts to think
clearly about the teaching of writing. This is where our energy is best spent in trying
to work out our theory and practice and make our teaching more consistent, more
purposeful, and ultimately more effective: with the teacher in the classroom looking at
her own values, methods, and goals.

Tallahassee, Florida

-1-

The Effect of Audio

and Written TeacherResponses on EFL Student
Revision.

by Ana Maria Morra , Maria Ines Asis

Providing feedback is one of the foreign language writing teacher's most important
tasks (Ferris, 2007). However, there is less certainty about the techniques that should be
used (K. Hyland & F. Hyland, 2006). This article reports on research that investigated the
effects of two types of teacher feedback, on-tape and written, and of the
absence of feedback on students' (n = 89) error correction. A comparison of the
number of macro (content, organization) and micro (vocabulary, grammar, mechanics)
errors before and after the experiment yielded a statistically significant reduction in the
number of mistakes in final drafts.Students perceived the type of response received
(either taped or written) had been helpful in revising their papers and considered the most
beneficial aspect of teacher feedback to have been a focus on micro errors. This study
offers insights into an ecclectic approach to teaching writing in similar EFL contexts.

Over the last twenty-five years, the focus of writing instruction has changed from product
to process, from seeing students' writing less as a finished product than as a piece of work
perfectible through multiple drafting with feedback between drafts (K. Hyland, 2003; K.
Hyland & F. Hyland, 2006). Whether the process approach to writing instruction per se
has brought about positive results in foreign language pedagogy has provoked much
controversy, resulting in a thrust towards a more balanced view of process and form
(Applebee, 2000; Bromley, 2003). No doubt the process approach has had a major
impact onwriting research and instruction (K. Hyland, 2003). It continues to be applied in
the teaching Of English as a foreign language (EFL) because by placing considerable
emphasis on revising and responding to writing, it allows teachers and students more
meaningful interaction. Process feedback, with its emphasis on the recursive
nature of writing, has emerged as an essential component of the approach and has stayed
in the forefront of instructional practice.

Review of Literature

Several studies have shown the positive results of teacher feedback. These studies have
focused onfeedback on form (Ferris, 1997) and content (Fathman & Whalley 1990),
others on different means ofdelivery, such as electronic (Greenfield, 2003; Warschauer,
2002; Williams, 2002) and conferencing (Goldstein & Conrad, 1990; Patthey-Chavez &
Ferris, 1997). Additional research, however, has questioned
the effect of teacher feedback on language errors (Goldstein & Conrad, 1990; K. Hyland
& F. Hyland, 2006). A few studies have shown no clear
signs of improvement of students' writing accuracy after response, so the debate has
continued between those who have believed in the
effectiveness ofcorrective feedback on form and those who have not (Guenette, 2007).
The most adamant argument against grammar correction came from Truscott (1999,
2007), whose position has provoked claims for additional research before the controversy
over the effectiveness of error correction on learners' ability to write accurately is settled
(Ferris, 2004). Still other researchers have not found significant differences onstudent
revision and editing involving the explicitness of teacher response (direct, indirect) or the
means used (written, verbal) (Ferris, 1995, 2006; K. Hyland, 2003).

Both a paucity of research on teacher response to student writing and, in some cases,
contradictory results, have left many key questions only partially answered (K. Hyland &
F. Hyland, 2006). One aspect in particular has remained virtually unexplored:
the effect of taped feedback on learners' revisions, hence the need for further
research on teacher feedback in EFL contexts.

Purpose of the Study

The present study aimed at contributing to efforts to improve EFL writing by

investigating the effects oftwo different
modes of teacher response: written comments on the margins and recorded feedback.
Reformulations made by students who did not receive any teacher feedback during the
process were also included in the data for analysis.

Context and Rationale of the Study

This study was carried out at the School of Languages, National University of Cordoba,
Argentina, where teachers were concerned about students' poor performance in the
composition component of their exams. Many explanations could have reasonably
accounted for these poor results: overcrowded classrooms, a deficit of student reading
and writing strategies, lack of prior knowledge of content and rhetorical structures, or a
flaw at some stage of the teaching practice, among others. These considerations led to an
examination of the feedback students were receiving and their reactions to it. Teachers
annotatedstudents' written assignments by writing observations on the margins or by
making general comments at the bottom of students' papers. Remarks concentrated
heavily on language use rather than on content and global organization.

Because learners complained about their not understanding teachers' marginal comments,
especially those on macro aspects such as subject matter and relevance of ideas, teachers
questioned their approach and began to explore alternatives. A verbal response such as
face-to-face conferencing was unfeasible in this context due to the large
number of students per class and the lack of school facilities for teachers and students to
meet for the sessions. A good choice seemed to be the "taped commentary" suggested by
K. Hyland (1990, 2003). On-tape feedback required the teacher to record his or her
response on a tape cassette and to write a number on the student's paper to signal what the
observation referred to (K. Hyland, 2003, p. 182). As previously said, this
means of providing feedback has been virtually unexplored except for K. Hyland's study
(1990).

In spite of the scant support from empirical research, on-tape feedback appeared
intuitively attractive to instructors in the EFL Department. A comparison of the
advantages and disadvantages of this type offeedback and the one traditionally used in
our classes yielded the following results (see Table 1).

This study then was undertaken to investigate the traditional written response (with
codes on margins) that the staff had been using in their courses for a decade, the
innovative on-tape response tried out for two semesters, and the lack of teacher response
throughout a multiple draft writing cycle. The researchers of this study proposed the
following alternative hypotheses:

1. On-tape feedback may be more effective than written feedback in

helping students reformulate theirwritten work at the macro level (content, relevance,
organization, coherence, logic).

2. Written feedback using codes on margins may be more effective than on-
tape feedback in helpingstudents correct errors at the micro level (surface sentence errors,
grammar, use of lexical items, spelling, punctuation).

Method

Setting and Subjects

The study was carried out in a five-year teacher training and translation studies
undergraduate program at a college in Argentina. The study involved six
groups of students taking EFL courses at a post-intermediate level of proficiency
according to the criteria established by the Association of Language Testers in Europe.
Three of these groups had enrolled in the Teacher Training program and the other three in
the Translation Studies stream. For each orientation there were two experimental groups
(the innovative one with taped response and the traditional one
with written codes on margins), and one control group (no teacher response). Eighty-
nine students, producing a total of 267 pieces of writtenwork, participated in the study.
Two teachers also participated, each in charge of the three groups in one
program of studies (either Teacher Training or Translation).

Materials and Procedures

The two experimental groups and the control group in each program of studies received
the same writinginstruction for 6 weeks. On week 7, the groups were given a topic and
were asked to write a short opinion essay. They produced two drafts and a final version.
Between drafts, the innovative group received taped feedback, and the traditional group
received written feedback with codes. Previously, the researchers and the two teachers
providing feedback had had several meetings for standardization ofcriteria. In these
meetings, instructors received detailed instructions and training on how to use a
taxonomy of errors designed by the researchers (see Table 2 for this taxonomy and the
codes used forwritten feedback). Teachers were asked to concentrate on the macro
aspects of writing (content and organization) in their feedback on the first draft
and on macro and micro (vocabulary, grammar, mechanics) components in their
comments on the second draft. No feedback was given to the control groups except for a
general instruction for students to revise their compositions and make any changes the
writers considered appropriate to improve their written work. All the groups had
deadlines for first and second drafts and for their final versions.

When this writing cycle was over, students answered a semi-structured questionnaire (see
Appendix) about the learners' reactions to the type of feedback they had received or the
absence of feedback.

Data Analysis and Results

Working on the initial drafts, we identified, charted, and then attributed each teacher-
taped comment orwritten mark to one of the 19 categories shown in Table 2 plus an
additional category (used only in the analysis stage for flaws and errors not classified by
the taxonomy employed). When a comment or mark did not correspond to one of the
categories on the list, we classified the observation ourselves according to the 20 types
established. We proceeded in the same way on the final drafts. Last, we compared the
number of macro and micro errors in the first draft and in the final version for each
student and, eventually, for each group.

The coding stage was long and complex. Teachers had not applied
the feedback categories consistently. For example, in written feedback we found marks
with no codes attached or with codes not listed in the taxonomy agreed upon. For
taped feedback, we judged some comments as "vague," "unnecessary," or "misleading."
In both cases, we found instances of incorrect feedback, that is, teachers had suggested a
change be made even though there was no error. We spent long hours analyzing and
coding the data, going over ambiguous comments and, in cases of disagreement,
appealing to a third party, a colleague thoroughly acquainted with our taxonomy.
Instances of students' unsuccessful revisions (incorrect reformulations, no change made
in spite of teacher comment) were set apart for future analysis.

The data were analyzed using the non-parametric Wilcoxon's Paired Signed Rank Test to
investigate, by means of a unilateral analysis, the statistical significance of the difference
between the number of macro and micro errors before and after the taped or
the written feedback. Wilcoxon's test was also used to analyze control-group data to
investigate whether the absence of feedback had any impact on the number of errors
found in the final versions of the students' essays (see Table 3).

The results of the statistical tests indicated that the null hypothesis could be rejected in all
cases and that the differences found after treatment (taped, written feedback,
no feedback) were statistically significant. The results also showed that these changes had
been positive both at the macro and micro level in all but one group. The Translation
Studies group, which had received taped feedback, showed negative changes at the micro
level--that is, after treatment, this group registered an increase in the number of micro
errors. In regards to the two control groups, the differences found were also statistically
significant and positive in all cases. In sum, five of the six groups that participated in the
study showed a statistically significant reduction in the number of errors in the final
drafts of their essays.

Results of the questionnaire revealed that 87% of the subjects in the experimental groups
were able to understand their teacher's comments. Moreover, the students stated that the
type of response received (either taped or written) had helped them revise and rewrite
their papers. Also, 56% of the subjects in the control groups acknowledged that the
opportunity to review and reformulate their papers even without any observation or
comment on the part of the teacher had been beneficial.

When asked about the features of their written work reformulated after feedback or after
being given the opportunity to revise and rewrite, 88% of students said that they could
correct errors in grammar, lexis, spelling, and punctuation (micro errors); 44% also stated
that they had succeeded in overcoming weaknesses in content, coherence, relevance, and
meaning (macro errors).

As to the type of feedback preferred, 96% of those who received on-tape comments chose
this type offeedback; 88% of all students who completed the questionnaire
ticked written feedback as their preference, and only 14% of those who did not
get feedback said they had enjoyed reviewing and rewriting their compositions without
any kind of teacher response.

Discussion and Conclusions

Responding to student writing appears to have a positive effect on learners' written work
regardless ofthe means used (written or taped). In this study, the number of students'
errors and weaknesses at both the micro and the macro level decreased significantly in
the final version except for one group in the Translation Studies program. The
implications of this result will be discussed below.

Another important finding of the study is that the mere opportunity for students to revise
their compositions on their own, without teacher feedback, resulted in a reduction of the
number of flaws. This outcome emphasizes the importance of self-assessment,
contributes to research that shows improvement of students' end-products after rereading
and rewriting their own papers without any feedback (Ferris and Roberts 2001), and
supports findings of limited teacher response to the progress students make in
their writing (Graham, 1983, cited by Fathman & Whalley, 1990; Truscott, 2007). Thus,
it would probably be beneficial, as Fathman and Whalley (1990) suggest, to encourage
teachers to make a place for more independent writing in their classes (such as diaries,
which demand no teacher intervention) and to promote more self-guided revision at
home, thus paving the way for greater autonomy in assessment and revision.

Furthermore, the perception of 88% of our subjects that the most beneficial aspect
of teacher feedback or of the opportunity to revise their writing on their own was linked
to formal aspects of language use suggests polemic issues for teachers of EFL writing at
the university level, at least in the context of where this study was conducted: What
should be the main objective of a writing class? How much attention must be paid to
language accuracy in the composition class? Should teachers be satisfied with a spotless
composition even if it is weakened by poor argumentation or insubstantial content? Or
should they teach strategies for focusing on relevance and richness of ideas as well
as on language forms? The students in this study seemed to adhere to a documented
belief among EFL writers that places high value on a composition free of surface errors
(K. Hyland & F. Hyland, 2003; Leki, 1990; Morra, Badenes & Musso, 2003). This
conviction may grow out of a major concern in EFL academic contexts about the
learner's language proficiency being far from a near-native operational command.
The students involved in this study are expected to improve their accuracy as they
progress in their studies because they will be graduating as teachers of EFL or becoming
translators. Probably as a result of this, learning to write is synonymous with acquiring
and rehearsing the use of lexical items, syntactic patterns, cohesive devices, and other
language forms, a practice which focuses on accuracy and clear exposition before
content. EFLwriting teachers might choose, then, to help learners develop awareness that
producing effective texts also involves substantive content and information-gathering
strategies. Further research is needed in similar EFL contexts to find out whether
the feedback modes studied here result in improved writingperformance.

This study also supports research in EFL feedback that suggests that students do take
teachers' observations seriously and do not feel upset by them (Ferris, 1995; Ferris &
Roberts, 2001; Leki, 1990). Such an attitude contrasts with evidence provided by studies
of feedback in English as the native language (Ferris, 2007). Our classroom experience
indicates that EFL writers are aware of their language limitations and, consequently,
expect and want their teachers to correct errors. A similar attitude in second-language
learners is reported by F. Hyland (1998).

In regard to the mode of teacher feedback preferred, the taped response was chosen by
almost 100% of the students who experienced this type of feedback in the study and
could thus compare it with the more familiar written type. These students stated that the
recordings' "shortened distance" with the teachermade them feel as if they were actually
"talking" with the teacher even if they could not participate in a face-to-face dialogue.
They reported a sense of commitment that led them nearly always to hand in their revised
papers back on time. But perhaps a more important point students made was that, with
their papers in hand, they felt they could grasp the teacher's comments easily. They could
clearly see what she or he was aiming at, where ambiguity impaired comprehension, and
where there were gaps in the line of thought. Similar student reactions have been reported
by K. Hyland (1990).

As mentioned earlier, one of the groups that received taped feedback showed an increase
in the number of micro errors after treatment. K. Hyland (1990) suggested that both those
who give and those who receive taped feedback tend to concentrate more on macro
aspects of writing. In fact, the final version of the essays of the students belonging to this
group revealed more clarity in the expression of ideas despite the number of language
errors.

To conclude, most of our subjects expressed preference for either written feedback or
taped feedback, and a small percentage found self-reviewing effective. Thus, student
response to feedback may be more or less effective depending on the mode
of teacher response. A possible pedagogical implication for teachers could be to
offer students a taste of different types of feedback for them to choose from, thus
responding to students' individual needs. Future studies should attempt to investigate this
further with a single group of students; this was not feasible in this study. Also, it would
be interesting to complement the quantitative nature of the present piece of research with
a qualitative one to capture teachers' perception of the effectiveness of the type
of feedback they provide.

Appendix

Survey

Thank you for participating in our research project. As a last step, we would like you to
answer this questionnaire. Please, choose the answer(s) that best represent(s) your
experience and complete the blank spaces provided.

A. Which of the following types of feedback (comments made by the teacher) were you
familiar with before the experiment? (Note: You can choose more than one answer here.)

1. Use of a minimum "abstract" code, e.g., underlining of words or phrases; use of signs
such as ? ! [] ^ [??].

2. Words or letters written on the margins, such as ST, Agr, Sp, WO, etc.

3. Longer and more detailed comments and suggestions written on the margins

4. and at the end of the written piece.

5. Tape-recorded feedback.

6. Feedback sent by e-mail.

7. Face-to-face conferences with the teacher.

8. Oral or written comments made by peers.

9. Other (please specify briefly): --

10. None of them.

B. What type of feedback were you given during this experiment?

1. Comments written on the margins (with codes).

2. Tape-recorded feedback.

3. None.

1. If you received teacher feedback, how helpful was the feedback when re-writing your
essay?

a. Very helpful.

b. Quite helpful.
c. Rather helpful.

d. Not helpful at all.

2. If you had no teacher feedback, how helpful was the chance to revise on your own?

a. Very helpful.

b. Quite helpful.

c. Rather helpful.

d. Not helpful at a11.

C. In which of the following aspects was the feedback you received or the chance to
revise withoutfeedback most helpful? (Note: You can choose more than one answer
here.)

1. To express your ideas more clearly, e.g. emphasizing what is important and discarding
secondary ideas.

2. To organize your writing in a better way.

3. To improve your grammar.

4. To improve your vocabulary.

5. TO improve your spelling.

6. To improve your punctuation.

7. Other (please, specify) --

D. How many times did you submit your essay during the experiment?
1. Only once.

2. Two times.

3. Three times.

E. If you did not comply with the three scheduled submissions, why didn't you?

1. You didn't think rewriting your work was going to help you improve
your writing skills.

2. You were not able to understand the comments made and, therefore, you were not able
to correct them.

3. You could not meet the stipulated deadlines.

4. Other (please specify briefly): --

F. Which of the types of feedback you are now familiar with have helped you the most to
improve yourwriting skills?

1. Oral or written comments made by peers.

2. Comments written on the margins (with codes).

3. Longer and more detailed comments and suggestions written on the margins and at the
end of the essay.

4. Minimum "abstract" code (underlining, use of signs such as? ! [] ^ [??]).

5. Tape-recorded feedback.

6. Face-to-face conferences with the teacher.

7. Feedback sent by e-mail.

8. Other (please specify briefly):

9. None of them.

Thank you.

References

Applebee, A. N. (2000). Alternative models of writing development. In R. Indrisano & J.

R. Squire (Eds.), Perspectives on writing: Research, theory and practice (pp.90-111).
Newark, DE: International Reading Association.

Bromley, K. (2003). Building a sound writing program. In L. Mandel Morrow, L.

Grambell, & M. Pressley (Eds.), Best practices in literacy instruction (pp. 143-166). New
York: The Guildford Press.

Fathman, A. K. & E. Whalley. (1990). Teacher response to student writing:

Focus on form versus content. In B. Kroll(Ed.),. Second language writing. Cambridge:
Cambridge University Press.

Ferris, D. (1995). Student reactions to teacher response in multiple-draft composition

classroom. TESOL Quarterly, 29(1), 33-53.

Ferris, D. (1997). The influence of teacher commentary on student revision. TESOL

Quarterly, 31(2), 315-339.

Ferris, D. (20041). The "grammar correction" debate in L2 writing: Where are we, and
where do we go from here? (and what de we do in the meantime ...?). Journal of Second
Language Writing, 13, 49-62.

Ferris, D. (2006). Does error feedback help student writers? New evidence on the short-
and long-term effects of written error correction. In K. Hyland & F. Hyland
(Eds.), Feedback in second language writing(pp. 81-104). Cambridge: Cambridge
University Press.

Ferris, D. (2007). Preparing teachers to respond to student writing. Journal of Second

Language Writing, 16, 165-193.

Ferris, D., & Roberts B. (2001). Error feedback in L2 classes: How explicit does it need
to be? Journal of Second Language Writing, 10, 161-184.

Goldstein, L., & Conrad, S. (1990). Student input and negotiation of meaning in
ESL writing conferences. TESOL Quarterly, 24(3), 443-460.

Greenfield, R. (2003). Collaborative e-mail exchange for teaching secondary ESL: A case
study in Hong Kong. Language Learning and Technology, 7(1), 46-70.

Guenette, D. (2007). Is feedback pedagogically correct? Journal of Second

Language Writing 16(1), 40-53.

Hyland, F. (1998). The impact of teacher-written feedback on individual writers. Journal

of Second Language Writing, 7(3), 255-286.

Hyland, K. (1990). Providing productive feedback. ELTJournal, 44(4), 279-285.

Hyland, K. (2003). Second language writing. Cambridge: Cambridge University Press.

Hyland, K., & Hyland, F. (2006). Feedback in second language writing. Cambridge:
Cambridge University Press.

Leki, I. (1990). Coaching from the margins: Issues in written response. In B. Kroll, (Ed.),
Second languagewriting (pp. 57-68). Cambridge: Cambridge University Press.

Morra, A. M., Badenes, G., & Musso, E. (2003). Error correction by undergraduate and
postgraduatestudents. Advances in ESP research in Latin America. Proceedings of the
VIII Latin American ESP Colloquium (pp. 105-112). Salta, Argentina: Comunicarte
Editorial

Oxford, R. (2001). Cognitive processes and learning strategies: A seminar to integrate

theory and practice in Argentina. Unpublished manuscript, National University of
Cordoba, Argentina.

Patthey-Chavez, G., & Ferris, D. (1997). Writing conferences and the weaving of
multivoiced texts in college composition. Research in the Teaching of English, 31, 51-90.

Reid, J. (1995). Learning styles in the ESL/EFL classroom Boston, MA: Heinle &
Heinle.

Truscott, J. (1999). The case for "the case for grammar correction in L2 writing classes":
A response to Ferris. Journal of Second Language Writing, 8, 111-122.

Truscott, J. (2007, in press). The effect of error correction on learners' ability to write
accurately. Journal of Second Language Writing.

Warschauer, M. (2002). Networking into academic discourse. Journal of English for

Academic Purposes, 1, 45-48.

Williams, J. (2002). Undergraduate second language writers in the writing center. Journal
of Basic Writing, 21(2), 73-91.

Dr. Ana Maria Mona is Chair of English at the School of Languages, National University
of Cordoba, Argentina, and Head of the Master Program in Spanish as a Foreign
Language at the same university. Her e-mail address is anamorra@gmail.com. Maria Ines
Asis (MA) is a lecturer in English for Speakers of other Languages (ESOL) and
curriculum leader for the additional learning support division at Southgate College,
London, UK. Email: ines.asis@southgate.ac.uk.

Table 1. Advantages and disadvantages of written and taped commentary.

Type of
 feedback Advantages Disadvantages

Written on * More viable * Vague and confusing

margins * Effective if well given * Too much focus on
* Comments on use of editing surface errors,
language very helpful root problems overlooked
* Risk of teacher's
appropriation of student
work
* Impersonal
* Time consuming

Recorded * Engages students into * Cannot be done anywhere

on tape reformulating their * Students may concentrate
papers to reward the more on listening
teacher for the than on comments
individual attention * Students with poor
paid to their papers listening skills may be
* Helps teacher at a disadvantage
concentrate on content
and meaning more than on
use of language
* Comments on students'
ideas very helpful
* Contributes to
self-monitoring
* Helps less skilled
writers
* Consumes less time
* Motivates more

Table 2. Taxonomy of errors used by teachers providing feedback and

codes for written response.

Codes
Error type and description (written feedback)

1. Content

Relevance, richness of information RI

2. Organization, Coherence, Cohesion

 Thesis/Topic sentence(s) TS
Supporting sentences SS
Concluding sentence(s) CS
Logical development of ideas LD
Use of transitional signals (one-word Trans S
connectors, transitional phrases, and sentences)
Referencing (shifts in pronouns, usage of Pr Ref
"it," "this," vague or ambiguous reference)

3. Lexis

Collocations Colloc
Word form WF
Choice of word (specificity in relation to WC
topic)

4. Grammar

Subordination Sub
Double subject D Subj
Omission of subject O Subj
Word order (order of imbedded questions, WO
subject-verb inversion, others)
Verb (tense, form) V
Agreement (subject-verb, singular-plural, Agr
excluded pronoun reference)
Prepositions Prep

5. Mechanics

Punctuation (commas: subordinate, main clauses, Punct

period, semicolon between main clauses,
punctuation with connectors)
Spelling SP
Others (used only for analysis of data)

Table 3. Results of the Non-Parametric Wilcoxon's Paired Signed Rank

Test.

Groups in the Translation Studies Program

 Macro Errors

Written feedback SS 16
[SIGMA] + 136 [SIGMA]-0
T 35

Taped feedback SS 16
[SIGMA] + 134 [SIGMA]-3
T 35

No feedback SS 12
[SIGMA] + 78 [SIGMA]-0
T 17

Groups in the Teacher Training Program

Macro Errors

Written feedback SS 20
[SIGMA] + 185 [SIGMA]-5
T 53

Taped feedback SS 13
[SIGMA] + 72.5 [SIGMA]-17.5
T 21

No feedback SS 12
[SIGMA] + 66 [SIGMA]-12
T 17

Micro Errors

Written feedback SS 16
[SIGMA] + 124 [SIGMA]-12
T 35

Taped feedback SS 16
[SIGMA] + 34.5 [SIGMA]-101.5
T 35

No feedback SS 12
[SIGMA] + 71 [SIGMA]-4.5
 T 17

Groups in the Teacher Training Program

Micro Errors

Written feedback SS 20
[SIGMA] + 192 [SIGMA]-28
T 60

Taped feedback SS 13
[SIGMA] 60 [SIGMA]-17
T 17

No feedback SS 12
[SIGMA] + 90 [SIGMA]-1
T 17

Feedback Follow Up: the

Influence ofTeacher Comment on Student WritingAs
signments

by Deborah W. Dunsford

Introduction

Writing improvement is becoming an increasingly important topic at most

universities. Feedback from potential employers and research repeatedly shows that
college students' writing abilities are below expectations (Lindner et al., 2004). Several
universities have implemented writing-intensive course requirements for
undergraduate students that will ultimately require faculty in all disciplines to provide
additional writing opportunities in their curriculums (Univ. of Florida, 2004. The Gordon
Rule; Texas A&M Univ., 2004. Writing Intensive Courses at Texas A&M Univ.; Martin
and Burnett, 2003).

For agricultural education and communication programs, this focus frequently takes the
form ofservice courses that teach writing skills (Kansas State Univ., 2004; The Ohio
State Univ., 2004; Texas A&M Univ., 2004. Core Curriculum; Univ. of Florida, 2004.
Agricultural Education). As the demand for seats in courses that teach writing skills
continues to grow, instructors try to balance the need to
provide students with feedback on their writing assignments with the amount of time it
takes to provide that feedback. While writing instructors from all disciplines generally
agree that revision is one of the best ways to encourage students to improve their papers,
few know what comments or what type of comments are most likely to help
their students revise successfully.

Research into revision and how and why students revise their texts has long been
part ofcomposition literature. So has research into teacher comment on student texts.
However, there is little work that brings the research areas together. This study may
provide a link between these two important areas of research.

Composing a piece of written discourse has long been considered a non-linear, recursive
process (Britton, 1975; Rohman and Wlecke, 1964). Later researchers built on this model
describing composing as a continuous loop where any element may follow any other
element (Bridwell, 1980; Faigley and Witte, 1981; Flower et al., 1986; Sommers, 1980).

Although the recursive nature of the process is not in question, an actual definition for
revision is less clear. Several definitions use only the etymological definition of "seeing
again" (Boiarsky, 1980). Sommers (1980, p. 380) defines revision as "... a
sequence of changes in a composition changes which are initiated by cues and occur
continually throughout the writing of a work." Drawing on all these definitions, the
operational definition for revision used in this study refers to the additions, deletions,
substitutions, and rearrangements of units of meaning that students make in their texts in
an effort to convey better their intended meaning to an audience.
Teacher comment is another key area of composition research. Much research shows
that teacherresponse can have a major impact on a student's attitude toward the text and
toward writing in general. De Beaugrande (1979) claimed that if students see grammar,
punctuation and spelling as priorities in teacher comment, then those are the errors they
will repair. Miller (1982) suggested two separate sets of teacher comments one on content
and the other on writing problems. Murray (1979) advocated doing away with comment
completely and using one-on-one conferences to provide feedback to students. Peterson
et al. (2004) suggest that the type of paper plays a role in the type of comments teachers
provide. Narrative papers receive a greater percentage of editing-related comments and
persuasive papers tend to receive a greater percentage of revision-related comments
(Peterson et al., 2004).

Besides types of comments, other research examines the quality of those comments.
Lynch and Klemans (1978) surveyed students about their responses to teacher comment
and found thatstudents responded more positively to comments that not only told them
what was wrong with a paper, but why. Straub (1996) explored directive versus
facultative comments on student texts and the potential control the comments
represented. In general composition researchers agree that the
goal of teacher comment on papers is to wean students away from criticism from
the teacher and toward forming their own ability to review and revise their texts.

Purpose

The purpose of this study was to determine how the types and
location of teacher feedback on a group of student texts influenced the revision choices
that group of students made to their texts. The research objectives were to determine if
the location of teacher feedback influenced students' revision choices and if the
type of content or type of teacher comment influenced those choices.

Methods

The subjects in this study were 62 students enrolled in media writing classes at a land
grant institution in the South. Each of the four classes studied had enrollment
limits of 16 students and thestudents were primarily juniors and seniors majoring in
journalism or agricultural journalism. A fewstudents were either minoring in journalism
or planned to obtain teaching certification in journalism. The
majority of the students were female (69%), which reflects a nationwide trend in
communication departments. The classes met four times per week during the 16-week
term. The Monday and Wednesday lecture sessions covered techniques in
news writing and textbook material on different types of news stories, as well as some
basic information on methods of writing. The other two weekly class meetings were 75-
minute lab sessions. The students used Microsoft Word software for all of their
assignments. Besides the lab periods, students had access to the computer-equipped
classroom through the work day and had access to their own computers or other campus
computer labs throughout the semester.

Data was collected from the students' four major writing assignments. The four
assignments were as follows: (1) Write a pair of short, one-paragraph leads from a
choice of assigned fact sets; (2) write a news story from a short speech and question-and-
answer session presented by a guest speaker; (3) write a news story about a coming
event on campus or other item of the student's choice that quotes at least one source (i.e.,
they had to go interview someone and write the story); and (4) write a short feature
story on a topic of their choice that quoted at least two sources and required additional
background sources. Students wrote for both print and broadcast media.
Theteacher comments and the revisions students made on these assignments provided the
data for this study.

Students had the option to revise one of the two versions of each of the major
assignments. If thestudents opted to revise one of their papers, the grade they
received on the original paper counted as two-thirds of the grade on the final paper. The
grade on the revised paper counted as the remaining one third. This method
encouraged students to make their best effort on the original paper. The students'
grades on the revised papers would not be lower than the original grade they received,
although the grade could remain unchanged. For purposes of this study, only the papers
that the students opted to revise were analyzed.
Each of the four classes received four different methods of instructor feedback with a
different method used on each of their four major assignments. The comment methods
were marginal and concluding written comments on their papers, marginal comments
only, concluding comments only, and only oral comments to the class as a group. When
revising their papers, the students were required to return the graded original paper along
with the revised version.

To protect the students' identities and to eliminate any chance of bias associated with any
particular student, each student was assigned a random number, and an uninterested third-
party placed this number on the students' texts and then cut off the students' names and
course section numbers. To preserve the regular classroom environment during the study,
the students were not told about the study until the end of the semester, after the last
paper had been turned in. The students received a written explanation of the study and the
use of their texts (anonymously). At this time they were offered the chance to have their
papers removed from the study. None of the students selected this option. This study met
all university requirements for human studies research and all necessary forms are on file
with the university's research office.

After the student texts had been collected they were sorted by assignment
and teacher comment type (marginal and end, marginal only, end only and oral only
comment). The texts were sorted numerically for ease in coding and an index card was
established for each student number. These cards provided a method of tallying the
number and types of revisions on each text. The data from these cards provided the basis
for the statistical analysis in this study.

Structural revisions made by the students in a second, revised paper were compared to
their original, graded papers. Structural revisions in this study were additions, deletions,
substitutions, and rearrangements (Sommers, 1980). These structural revisions were
examined at the level ofunits of meaning that may or may not correspond to the physical
division of paragraphs within the text. According to Rodgers (1967), paragraph divisions
frequently do not correspond with units ofmeaning within a text, and he suggests that a
"stadia of discourse" is a better unit than the somewhat arbitrary paragraph indention. The
"stadia," according to Rodgers, is a sentence or groupof sentences that contain a single
topic which may or may not be contained in a single paragraph. This idea is particularly
important when working with journalistic writing. Paragraphs in a newspaper or on an
audio script are frequently shorter to accommodate the requirements of the newspaper's
narrow columns or the readability for a television or radio reporter. The
variable of interest for this study was units of meaning, sentences or groups of sentences
that share a common topic.

An ANOVA was performed on the revision data that, in effect, combined the four classes
into a single group for statistical purposes (Ott, 1988). This method is appropriate
because the students were not assigned randomly to the classes used in the study. The
analysis examined the four treatments (marginal and end comment, marginal comment
only, end commend only and oral comment only) and the four revision types (additions,
deletions, substitutions and rearrangements) to determine if there were any significant
differences between the treatments and the outcomes. In the analysis, differences with p
<.10 are considered significant. This significance level was used to help offset Type II
error that could easily result from the relatively low number of subjects, the imprecise
measurement methods, and the exploratory nature of this research (Lauer and Asher,
1988). Next, using percentages and graphs, the data were analyzed for similarities and
differences among the combination of treatments and the resulting revisions. A
naturalistic inquiry method was used to examine the relationship between specific
instructor comments and the specific revisions that resulted from that comment (Lincoln
and Guba, 1984). To analyze the data, teacher comment that was written on the student
texts or given in the oral comments were recorded and written on individual index cards.
The cards were sorted into groups of those with similar meanings using Lincoln and
Cuba's ( 1984) method. The groups were then cross checked and again collated into
groups by meaning or the problem they addressed. Seven groups were established, each
of which addressed a different aspect in the texts (Table 1). At this stage in the study
there was no differentiation made between oral and written comments, as those
distinctions were covered in the quantitative phase of the study.

Results and Discussion

Analysis of the teacher comment types resulted in seven comment types: positive
comments, overall quality of all or a section of text, material that does not belong in the
text, material that is out ofplace, wordiness or over length, wording or sentence needs
work, and meaning is unclear. Student responses were examined based on each comment
type.

"Positive comments," as might be expected, did not result in a lot of revisions, although
somestudents did revise some of these sections of their texts. All of the revisions
resulting from these comments were improvements. Comments on the "overall
quality of all or a section of the text" asked for big changes. Generally student response to
these comments was deleting or substituting material in their texts, which is not
surprising because the comments frequently related to coherence or focus. Again, the
student revisions associated with this comment were generally improvements, but in
some cases deleting material weakened the story.

Student responses to the comment that to "material that did not belong in the text" also
resulted in deletions. Students tended to act more frequently on some of these comments
than others because a few of the comments offered more specific instructions ("Doesn't
go in the story" vs. "How does this fit?"). Therefore, some of the comments were not
acted on by the students, probably because ofuncertainty of how to solve the problem
(Flower et. al., 1986). Generally revisions made in response to this comment group
improved the student texts. "Material that is out of place" comments related to
organization problems in student texts and usually suggested that the material belonged in
the story, just not where the student had placed the information. As
expected, students generally opted to rearrange their texts in response to this comment.
Specific comments in this category resulted in more revisions that improved the texts
than did less instructive comments ("Move Up" vs. "Out ofplace"). Students responded
frequently to the less specific comments by removing the material.

Comments on "wordiness" and "wording or sentence needs work" problems frequently

resulted instudents deleting material from their texts. However, many students did a good
job of combining sentences and paragraphs to tighten up the text and reduce the paper's
overall length. Getting just the right word can be a particular problem for student writers
and comments in this area included "awkward," "choppy," or "vary sentence structure."
Responses to these comments frequently resulted in fine tuning rather than fixing
structural or coherence problems. Many revisions in response to these comments included
combining sentences and altering sentence structure. A few resulted in deletions, but
there were more substitutions used in response to this comment than to other comments.
Again, the more specific the instructions, the more often the students revised
successfully. "Unclear meaning" comments usually refer to the need for more
information including specific detail or other clarifications. Some of the comments went
so far as to ask for specific numbers or other specific information, others were "vague"
and "confusing." Revisions resulting from this comment group varied including deletions
and additions. The quality of the students' revisions also varied.

Results from the qualitative portion of this study indicate that the more directive
the teachercomment on student texts, the more successful student revisions will be on the
text. Studentstended to respond to teacher comment if they knew how to make the
requested change or improvement. If they did not know how to make the change or how
to improve the text, they frequently deleted the material or ignored the comment.
According to Spandel and Stiggins (1990),students frequently misread instructors
comment and fail when they are trying to revise their texts. Occasionally students would
substitute material, which ultimately resulted in a few additions. There were few
rearrangements and those changes were usually in response to a specific comment to alter
the order of ideas in paragraphs.

In response to one of the main questions of this study, teacher comment "Does influence
the choices students make in revising their texts," and a second question "Does the
lack of teachercomment influence student revision?" Indications from the qualitative
portion of this study are thatstudents are even more likely to make revisions in the
absence of written comment when oral only comment is presented. As with the other
student responses to teacher comment, students perceive a benefit from revising their
texts based on the incentive of an improved grade.
The F test included all 64 of the students in the study combined into one large group. This
option was chosen to maintain the natural classroom environment as much as possible.
The data were coded by treatment and by revision outcomes. Based on this analysis, the
only significant outcome at p <.10 was deletions. A Scheffe S test showed that marginal
comment and oral comment only treatments were similar for deletions, as were marginal
and end comment and end comment only treatments. However, marginal comment and
oral comment only treatments were significantly different than marginal and end
comments and end comment only treatments. This means that the students' responses to
each pair of treatments were similar, but that they responded differently to the treatments
not contained in each pair.

The significance of deletions and the relationship between the two pairs of treatments
provides several options for interpretation. Flower et al. (1986) suggest that if students do
not know how to address a problem, they will frequently delete the material. That is
likely the case in this study. second, the similarities between responses to marginal and
end comment and end comment only suggest that students may be reading and
interpreting these comments in much the same way. The same should be said for the
other pair of treatments, marginal only comment and oral comment. Examining the
means of these two treatments, .38 for marginal only and .51 for oral only, indicates
that students made fewer deletions on average in response to these two comments than
for the other comment pair. Deletion means for marginal and end comment and end
comment only were .74 and .81 respectively. The students made more deletions
based on these two treatment types. One obvious similarity between thee two treatments
is that they both include comments on the students' texts. This may indicate
that students either read these comments more often or that they somehow responded to
these comments differently than they did to marginal only comment or oral only
comment at least when it came to making deletions in their texts.

Although the F test showed a significant difference for only deletions, the descriptive
statistics associated with the various treatment totals are worth discussion. Of the 307
total revisions by type made by students in this study, 109, or 35.62%, were deletions; 85,
or 27.28% were substitutions; 59, or 19.28%, were additions and 55, or 17.32%, were
rearrangements. Total revisions broken down by teacher comment location (Table 2 and
Figure 1) were marginal and end comment, 83; end only comment, 80; oral only, 83 and
marginal only, 61. The primary difference is in the revision types, with deletions showing
a much higher incidence than substitutions; additions and rearrangements are fairly even
at the lower end of the range. The high number of deletions is not unexpected (Flower, et
al., 1986)

Comparing comment location by revision type also provides an interesting discussion.

For marginal only comment, there was a relatively low overall number of revisions and
an even distribution (16 additions, 16 deletions, 15 substitutions, and 14 rearrangements a
range of 2). One possibility for this relatively low number of revisions with this comment
location relates to the lack of space available for providing feedback. Another reason may
be that students do not read these comments. Bolker ( 1978) suggested
that students dissociate themselves from teacher comment because they fear
disappointment. Also, while end comment often points out problems in a text, it is
frequently tempered with positive comment and is sometimes less directed at a specific
point or error in the text (Smith, 2004).

Oral only comment elicited a relatively large number of revisions ( 18, 24, 24, and 17 a
range of 7). This number of revisions is somewhat higher than anticipated at the outset of
the study. One of the theories was that students receiving only oral comment on their
texts would revise less because of the somewhat fleeting nature of the feedback.
However, information on the audio tapes of the oral comment sessions suggests one
reason for the unexpected strength of this response. The writtennotes for these sessions
look, for the most part, like a laundry list of what was wrong (and occasionally right)
with the class' texts. However, the audio tapes of these sessions include not only the
comments on the problems, but usually examples of all or most of the problems pulled
from student papers. The instructor did not return the students' texts until the end of the
class period in an attempt to keep the students' attention on that day's material. Therefore,
when the instructor went through the comment list, the texts were readily available for
discussion. No student names were mentioned, but the students did ask questions and
apparently, from the number of revisionson their texts, were able to make use of the
information.

Conclusions and Implications

Based on the results of this study, teacher comment influences student revision choices
and the more directive the teacher comment, the better chance the students will revise
their texts successfully. This agrees with Flower et al. (1986), Newkirk (1981) and
Shuman (1975), but this study builds on their work by providing specific illustrations
of teacher comment that offers problem identification and revision strategies paired with
actual student revisions.

The placement of written teacher comment does have some influence on student
revisions. In this study, there were more total revisions associated with oral only
comment than the other three types. The previously mentioned audio tapes indicate that
the oral comment sessions frequently included multiple examples of a problem and
multiple solutions. These additional examples may be part of the reason for the additional
revisions. Another possibility may be Bolker's ( 1978) suggestion
that students fear teacher comment and, because the oral comment is less direct, it is
therefore less threatening and students are more apt to listen. Oral feedback may help
build a sense of community rather than force students to view problems in the texts as
theirs alone.

The combination of research methods used in this study added strength to the conclusions
of both portions of the research. For example, deletions were the only statistically
significant response in the experimental study. This outcome could be explained more
clearly using results from the naturalistic inquiry portion of the study. Matching
specific teacher comments with specific revisions revealed that many of the comments
suggested or hinted at deletion as a revision option. Also, the results of both portions of
the study pointed to the importance of more detailed teacher comment either in the form
of more revisions associated with concluding comments on the texts or the more frequent
and more successful revisions from specific comments on the texts.
Several alterations would enhance future studies using this method. First, the use of
Rodger's (1967) stadia of discourse for identifying changes in the texts was a little too
coarse for the revisions involved. Reviewing changes at the word or phrase level would
be potentially more accurate. Identifying a way to limit the variety of teacher comment
statements by using a check sheet or other method would better focus the naturalistic
inquiry portion of the study and help further identify which comments elicited successful
revisions from the students.

Implications for future research include further examination of oral comment as

a feedback methodon student papers. The potential time savings for writing instructors, as
well as the possibilities of greater improvement in student writing make this an important
area for future study. Continued evaluation of written comment would also be extremely
valuable. Careful evaluation of writtencomment location and content could lead to
better writing and better use of instructor's time. Finally, one of the major goals of
providing comment on student texts is to help the students learn to internalize the ability
to evaluate their own texts. Identifying feedback methods that can helpstudents learn to
evaluate their own writing more successfully will enable them to become better writers.

Agricultural Communications programs frequently offer writing courses as either part of

their curriculum or as a service to their colleges. Providing efficient and
timely feedback on these papers is an increasing challenge as faculty work toward tenure
or promotion with ever-growing student demand. Refining our methods of
providing students with feedback on their papers will ultimately improve
our students' writing ability while still making the most efficient use of faculty member
time and resources the best of both worlds for all concerned.

The Importance of Feedback

by William Conerly
I'm a sucker for factory tours. Though this plant was noisy and humid, I learned a little
about extruding plastic film and a lot about the value of feedback. The plastic film
made by this firm let ions flow from one side to the other, with nothing else flowing
either way. That's all the chemistry that I understood, despite the owner's best efforts
to rectify the time I had spent snoozing in high school.

The company's customers specified the quality of the film in terms of defects per
roll. The rolls were large, with hundreds of feet of yard-wide film, and the defects
were microscopic. The customers were demanding no more than 4,000 defects per
roll, and producers had been having difficulty meeting that quality benchmark
consistently.

The company I visited had a defect counter. The film came out of the plastic extruder
onto a drum. When the drum was full, it was carted over to the defect
counter. The film came off the drum, through the defect counter, and onto another
drum. The good rolls were shipped, the bad rolls were tossed out. It obviously
pained the owner to toss out rolls because of poor quality. But that was theold way, he
explained.

Someone had the bright idea of saving a bit of work. The defect counter could be
mounted on theoutflow end of the extruder, just as the film was going onto the drum.
That would eliminate a step inthe production process, saving a few minutes of time for
each roll. The defect counter was sophisticated in its detection of flaws, but low-
tech in its counting of them: it had a mechanical counter, like the ones that are
sometimes used to count customers entering a store. Each defect registered with a
little "click."

Like anyone would be, the extruder operators were bothered when the clicker started
sounding off. Their job was to continuously monitor the conditions
inside the machine, such as temperature, pressure, and humidity. Each important
condition had its own meter, with a minimum and maximum noted on the dial,
and the operators had controls to change these factors. They soon learned that a long
series of clicks meant that something was out of line, and they would quickly adjust
their controls. Pretty soon the counter was generally quiet. The operators began
reacting when they heard just a few clicks. The dials would show that everything was
within tolerances, but perhaps not at the ideal levels. After a few months the operators
learned that if they reacted as soon as they heard a click, they could reduce successive
clicks. They also learned the "sweet spot" on each dial to prevent clicks. It required
them to monitor their gauges more closely, but it also made the job more fun. Friendly
competition developed to see who would have the fewest defects per roll.

By the time I visited, the number of defects had fallen from 4,000 per roll to less than
20. That staggering improvement came about because the operators were getting
immediate feedback on their performance. No foreman was berating them to do better.
No bonus system offered incentives. They just wanted to do good work, and now they
had the feedback they needed to do their jobs well.

Before the defect counter was installed next to the extruder, an engineer might have
been able to predict that with excellent operators defects should be as low as 20 per
roll. But the operators would have scoffed at the ivory tower approach. "He just
doesn't know what it's like to run theextruder all day long," they would likely have
said. And they would have been right. It's very difficult to do one's job at the peak
level without feedback.

Feedback and the Marketplace

Feedback often sounds like an engineering concept, but it's central to the working of a
competitive economy. A business uses various resources to produce a finished
product. Is that endeavor worthwhile? If the value of the finished product is greater
than the value of the resources used, then the company has created additional value.
It's doing something right and should be given positive feedback. In a free-market
economy, that feedback is called profit.
Similarly, a company that uses highly valued resources to produce a product that is
not highly valued should get negative feedback. It's called a loss. The case
of the plastics extruder is fairly easy. A defect is a defect. It can be counted, and it is
tied to the operator's attentiveness. Not all desired results are so easily
quantified. The principle of feedback improving quality, however, still applies.

Like many business owners, I find that the best feedback I get is from my profit-and-
loss statement. Black ink is good, red ink bad. But feedback is available in more ways
than aggregate profits.

Although the bulk of my work is economic research for businesses, I also give
speeches to trade associations and other groups. The price I set for my speeches a year
ago about matched the value of the time that I put into preparation and presentation.
There was no extra value created directly, but I was hoping for some indirect value
from the speeches as a marketing tool. That is, I hoped for more consulting projects
because of my exposure to more business people. Over time, however, it seemed that
all I was getting from the speeches were more speeches. But as a break-even
proposition, there wasn't much reason to sell more speeches.

In other words, the feedback suggested that the resources I was putting
into the activity just equaled the value of the activity. It wasn't harmful, but neither
was it generating extra value.

Then at the suggestion of a professional speaker, I tried raising my price. (Funny that
an economist should get pricing information from a motivational speaker, but
sometimes an outside opinion helps.)The first feedback I watched was sales level. It
turned out that the volume of speeches didn't change. (It is possible that the volume
would have grown in the absence of the price increase, but at least it didn't fall.)
And the second feedback was significant: the time I was spending giving speeches
was definitely more valuable than other possible uses of my time. I was adding value.
That feels good in the abstract and especially good in the wallet.
Although we often think that low prices are good, prices that are too low generate
improperfeedback to producers. They don't realize just how valuable their goods or
services are, so they don't produce as much.

Feedback is important not just because it gets us producing the right products
at the right prices, but because the world is changing. Demand goes up, demand goes
down. Technology improves, theprice of key supplies changes, human abilities
change, and only a feedback process can get our activities in line with the new reality.

Distorting Feedback

Writing a tribute to feedback may seem as useful as a tribute to gravity or the beauty
of rainbows. But feedback can be stymied, to the great detriment of the people who
consume and work. In theplastics company it would not help at all to
muffle the clicker. It would make for a quieter work environment, but product quality
would certainly fall. Neither would it help to have extra clicking noises added when
there were no defects. That would lead operators to adjust their equipment at the
wrong time, possibly lowering quality. In other words, the feedback signal has to be
undistorted.

In my business, as in all other businesses, there are various government-mandated

distortions of the feedback process. First, the taxes I have to pay on my income tell me
that my time isn't very valuable to others. After federal and state taxes are paid, I get
only $60 for every $100 that others are willing to pay me. When I compare the value
of my time to other people against the value of the time I spend sitting on my boat, the
tax system gives me improper feedback. Thus, I spend more time on my boat than I
should (which is exactly the conclusion my wife reached without economic analysis).

Price controls on the services I sell would similarly distort the feedback I get; controls
would lead me to think that my services were not highly valued. Were I to receive a
subsidy, the distorted feedbackwould indicate the resources I used were not very
 valuable. Thus I would overuse those resources.

Businesses are going to great lengths to increase and speed up feedback. New
information systems are getting data to the employees responsible for the results. At
many companies, sales people get immediate information on new orders; corporate
buyers get immediate information about what products customers are purchasing;
inventory managers get immediate information about what products need to be
restocked. The value of feedback has helped fuel a large increase in business's use of
information technology.

Unfortunately, we have not been so fast to eliminate the distortions to feedback caused
by government interference in the economy. But there is yet one
other feedback mechanism at work. Countries whose governments have less
involvement in the economy tend to grow faster, making their populations wealthier.
That feedback process may come to dominate global economics. That sounds as good
as a silent clicker.

-1-

Give Positive Feedback

by Peggy Simonsen

When we hear of the need for feedback from managers to employees, we tend to think only of
corrective feedback-improving performance problems, for example. Performance problems are
barriers that must be overcome of course, for success on the present job as well as for future
opportunity. But feedback also needs to be positive to reinforce good behavior. It
is the combination of both positive and corrective feedback that results in employees changing
their behavior.

I once had a manager say to me, "Why should I have to give positive feedback? They're only
doing their job. That's what they're paid to do!" My response was, "Do you want them to
continue doing it?"
Positive Feedback Motivates Positive feedback supports employee motivation. It is always nice
to hear, but is essential when employees have gone the extra mile, overcome a crisis, or
put in unusual effort to achieve a goal or meet a deadline.

You could argue that is what they are paid to do; but if you value the behavior,
positive feedbackrewards and reinforces it. Just as the steps to giving
corrective feedback make it more valuable and actionable, so do the steps to
positive feedback.

A "good job" kind of comment is better than no response; but specific, targeted comments are
more meaningful. For example, if you want to acknowledge an employee's willingness to work
overtime when needed, you might just say thank you. But taking a few minutes to say
specifically what you appreciated about the employee's efforts, the impact it has on the team
or the output of work, plus a comment about the future, make positive feedback much more
meaningful. Taking time to provide that kind of positive feedback reinforces the very behavior
you value (See side bar).

Sometimes positive reinforcement is needed after the employee demonstrates the corrected
behavior.

Let's look at an example: Joe creates a negative climate in the work group. He's a complainer,
particularly about company systems and procedures. His manager recognizes that Joe's
attitude and complaints have a negative impact on the team, and must be addressed.

As a first step, the manager identifies specific occasions where she observed Joe's behavior
that she wants to address. In preparation for giving feedback, she identifies the impact
of the negative behavior on the team and on Joe's reputation in the firm. She prepares
examples of the changed behavior she wanted Joe to work on and then met with Joe to
discuss it. Her purpose in addressing what some might consider "personality characteristics" is
to help Joe change the behavior that was getting in the way of the team's work and Joe's
success.

The Second Step. Joe's manager knows the first step in changing behavior is the awareness of
it as a problem, and so the need for corrective feedback. The second step is willingness to
work on theproblem, and her approach in giving feedback helps Joe accept the problem
instead of becoming defensive. Together they identify actions that could contribute to Joe's
efforts to change, because behavior change requires practice of the new, desired behavior.
As the manager, she is willing to deal with some of the procedures that cause Joe's frustration
and negativism. She knows also that positive reinforcement would be needed each time she
observed the desired behavior by Joe.
The second step of the feedback process is identifying the impact of the behavior
on the employee's goals or the team's success. Career discussions with an employee,
help the supervisor know what his or her goals are and what must be done to address a
behavior change accordingly. It becomes a clear case of "what's in it for me?"

In the case of Joe, the impact of the negative attitude might be stated something like this.
"Joe, you have identified a goal to have more opportunity to work on cross-functional teams to
broaden your base of expertise. If your reputation for complaining precedes you, other team
leaders will be less likely to select you. Even with your technical expertise, you may not be
seen as a team player."

In stating the desirable behavior, Joe's manager might say something like, "What teams need
is someone who can help analyze the pros and cons of a situation without sounding critical of
others. Let's discuss what you need to do to change your approach."

Behavior as a Controllable Factor

Giving corrective or positive feedback requires the manager to be aware of individual

employee's behavior. It means approaching behavior as a controllable factor, not writing it off
as an unchangeable character trait. After providing corrective feedback and
reinforcing the employee's attempts to change, if the desired change hasn't
occurred, the manager has a choice to make. Should he or she persevere or put his or her
energy into developing someone who is more responsive?

The manager is not being asked to be a therapist just to make sure his or her employees are
clear about expectations and provide some guidance to move in the right direction. A good
manager is one about whom people say, "he or she always gave me good feedback. He or she
acknowledged when I did something well and corrected me when I needed it."

Reprinted with permission of Insight, the magazine of the Ilinois Society of CPAs

-1-

The Effects of Hands-On Experience onStudents'

Preferences for Assessment Methods.
by Katrien Struyven , Filip Dochy , Steven Janssens

Although changes, invention, creation, design, progress, in short, innovation, are

concepts that represent the key issues of educational policy, they are often a
source of anxiety and concern for many teachereducators and teachers themselves (Hall,
1979; Van den Berg & Ros, 1999). They are reluctant to readdress, redesign, and
reorganize the teaching practices with which they feel comfortable. Consequently;
proposals for educational innovation are critically appraised and a negative disposition
toward the unknown is apparent (Hargraeves, 2000).

There is often professional unease about unknown subjects, situations, or innovations. It

might be an atavistic fear of change, anxiety about chaos, or professional concern for
conserving established values and standards (Cuban, 1998; Kelchtermans, 2005; Waugh
& Punch, 1987). It is only when familiarity grows with these subjects or situations that
fears are allayed and the new, changed situation is accepted (Van den Bergh,
Vandenberghe, & Sleegers, 1999; Van de Ven & Rogers, 1988). Fullan (2001) describes
the process as follows:
Real change, whether desired or not, represents a serious
personal and collective experience characterized
by ambivalence and uncertainty, and if the change
works out it can result in a sense of mastery, accomplishment
and professional growth. The anxieties of
uncertainty and the joys of mastery are central to the
subjective meaning of educational change, and to
success or failure thereof. (p. 32)

Positive experiences with the change, embodied in the sense of mastery,

accomplishment, and professional growth, define the success of the change and its
continuation in practice. Moreover, ifstudents are thought of as "participants in a
process of change and organizational life" rather than as "potential
beneficiaries of change" (Fullan, 2001, p. 151), involving students when studying
change and understanding educational innovation comes in natural. In fact, during the
process of change, studentsmight suffer similar feelings of uncertainty and ambivalence
at the start and joys of mastery, accomplishment, and academic growth when change has
proven to work. In this respect, student teachers in teacher training programs are
interesting subjects. On one hand, they are students in the process of change when
experiencing new teaching methods or assessment modes; on the other hand, they are to
serve the function of teachers implementing change in practice.

In addition, empirical evidence repeatedly has shown that teachers tend to teach as they
were taught (Johnson & Seagull, 1968), often failing to use the innovative techniques
that have been advocated in training (Grippin, 1989). Rather than delivering information
about engaging and innovative teaching practices through traditional approaches,
modeling the use of these teaching methods serves the purpose (Loughran & Berry,
2005).

Combining the abovementioned arguments, the assumption is made that the modeling
and use of new assessment techniques in teacher training might generate initial fearful
dispositions with student teachers toward the changes, fears that might be allayed when
familiarity grows and the changes tend to work out, with feelings of mastery,
accomplishment, and professional/ academic growth as consequences, defining the
(possible) adoption of the change in student teachers' current and future teaching
practices. Hence, the effects of student teachers' hands-on (read: actual) experience with
new modes of assessment in teacher training on their preferences for evaluation methods
in general, and the experienced method in particular, are investigated. This study not
only examines the dynamics ofstudents' preferences with respect to four assessment
methods that follow a standardized course onchild development but also aligns
instruction and assessment as a traditional learning environment is compared to a
student-activating type of instruction that is followed by one of four assessment types. In
contrast to former research on students' assessment preferences (Birenbaum, 1997;
Darling-Hammond & Snyder, 2000; Sambell, McDowell, & Brown, 1997; Zoller &
Ben-Chaim, 1997), studies that have adopted case-study designs, qualitative
methods of data gathering, or survey inquiries at one moment in time, the present study
is unique for its use of a quasi-experimental design by means ofquestionnaires
administered in three different occasions, that is, prior to, during, and after student
teachers have had experience with a particular assessment method.

In recent years, the repertoire of assessment methods in use in higher education has
expanded considerably. New modes of assessment have enriched the conventional
evaluation setting, formerly characterized by both the multiple-choice examination and
the traditional evaluation by essay (Sambell et al., 1997). More recently, portfolios, self-
and peer assessment, simulations, and other innovative methods were introduced in
higher educational contexts. The notion of alternative assessment is in this respect often
used to denote forms of assessment that favor the integration of assessment, teaching,
and learning; the involvement of students as active and informed participants;
assessment tasks that are authentic, meaningful, and engaging; assessments that mirror
realistic contexts; and assessments that focus on both the process and
products of learning and move away from single test scores toward a descriptive
assessment based on a range of abilities and outcomes (Sambell et al., 1997). The
present study compares four assessment methods: one conventional multiple-choice test
and three alternative assessment methods, namely, case-based evaluation, peer
assessment, and portfolio assessment, and investigates (the dynamics in) student
teachers" assessment preferences.

A profound study of educational literature on students' assessment preferences

confronted us with three types of studies. First, there are the studies that
require students to express their preferences concerning two contrasting evaluation
methods. These studies often relate student characteristics to preferences for particular
assessment types. For example, Zeidner (1987) concludes that both female and male
high school students prefer the multiple-choice format to the essay type of examination.
Also Traub and MacRury's (1990) scholastic students report more positive attitudes
toward multiple-choice tests on the grounds that these examinations seem easier to
prepare, easier to take, and may produce higher relative scores. Nevertheless, these
results do not apply to the entire group of students. Birenbaum and Feldman (1998)
discovered that university students (in social sciences and arts) with good learning skills,
who have high confidence in their academic ability, tend to prefer the essay
type ofassessment to multiple-choice examinations. Conversely, students with poor
learning skills, who tend to have low confidence in their academic ability, prefer
multiple-choice testing to the constructed-response type of assessment. Results also
show that low test anxiety measures were related to positive attitudes toward the essay
format. Students with high test anxiety have more unfavorable attitudes toward open-
ended format assessments and a preference for the choice-response type. In contrast to
Zeidner (1987), this study also indicated gender differences, with men having more
favorable attitudes toward the choice-response format than do women (Birenbaum &
Feldman, 1998; Struyven, Dochy, & Janssens, 2005).

The second category of studies also investigates students' assessment preferences for
several evaluation methods. In reference to asking students to indicate their
appreciation of a whole range ofassessment methods, Birenbaum's (1997)
results on university students' assessment preferences demonstrate that the highest
preference was for teacher-guided test preparation. Next in order came nonconventional
assessment types and student participation, followed by higher order thinking tasks and
integrated assessment. The least favored category was oral exams. Similarly, Zoller and
Ben-Chaim (1997) found that oral examinations of all types were rated as least preferred
by college students in both the United States and Israel. The higher stress level
associated with oral exams, and the consequent poor performance, are the main reasons
for students' dislike of this type of evaluation. College science students' preferred
assessment method is nonconventional, written exams in which time is unlimited and
any materials are allowed. Moreover, American students like the traditional writtenexam
significantly more than do their Israeli counterparts (Zoller & Ben-Chaim, 1997).
Characteristic ofboth categories of studies is that students' assessment preferences are
not related to a particular teaching context in students' higher education programs.
Moreover, preferences are measured at one moment and are treated as student
characteristics that are fairly stable over time.

However, there is evidence to suggest that students' preferences change due to hands-
on experience with a particular teaching environment, which is the focus of the present
study. Many researchers have interrogated students about their assessment preferences
after experiencing alternative methods ofassessment. By means of the case study
methodology, Sambell et al. (1997) tried to unveil universitystudents' interpretations,
perceptions, and behaviors when experiencing different forms of alternative assessment.
Broadly speaking, they found that students often reacted negatively when they discussed
what they regarded as "normal" or traditional assessment. Exams had little to do with the
more challenging task of trying to make sense and understand their subject. In contrast,
when studentsconsidered new forms of assessment, their views of the educational
worth of assessment changed, often quite dramatically. Alternative assessment was
perceived to enable, rather than pollute, the quality of learning achieved (Sambell et al.,
1997). However, students repeatedly express that a severe workload tends to alter their
efforts in studying (Sambell et al., 1997). Similarly, Slater (1996) found thatstudents in
their first year of university physics education had come to like portfolio
assessment.Students thought that they would remember much better and longer what
they were learning, compared with material learned for other assessment formats,
because they had internalized the material while working with it, thought about the
principles, and applied concepts creatively and extensively for the duration of the
course. Students enjoyed the time they spent on creating portfolios and believed it
helped them learn. Segers and Dochy (2001) found similar results in students'
perceptions about self- and peer assessment in a problem-based learning environment
setting at university (with law and educational science students). Students reported that
these assessment procedures (or should they be called pedagogical tools) stimulate deep-
level learning and critical thinking. In addition, Macdonald (2002) finds evidence
for students' preferences being affected by course experiences at the Open University. In
contrast to the feedback received from the undergraduate students, all of the
postgraduate students exposed to a project as an end-of-course assessment expressed
satisfaction with the project as a final examinable component (Macdonald, 2002).
Although it is clear from these results that students' assessment preferences are affected
by their experience with a particular assessment method, the lack of pretest measures
makes it hard to pinpoint the effect of the experience with evaluation on students'
changes in assessment preferences.

The purpose of this study is to provide evidence about the effect of student teachers'
hands-onexperience of a particular mode of assessment (e.g., multiple-choice test, case-
based exam, peer assessment or portfolio) on their assessment preferences using a three-
wave longitudinal design. The research project attempts to test the following hypotheses,
reported in this article: (a) unknown assessment methods are regarded negatively; (b) as
familiarity with an assessment method grows, assessment preferences will change
positively; and (c) students' preferences are congruent withstudents' perceptions of the
appropriateness of the assessment methods.

RESEARCH DESIGN

Sample

The investigation had a quasi-experimental design because of the authentic educational

context in which the study was conducted. Learning materials on the content of child
development (a course book, a booklet of assignments, and four evaluation formats)
were developed for a set of 10 lessons. These were delivered in five research conditions
involving 664 students in their 1st year of the elementaryteacher training
program of eight participating institutions in the Flemish part of Belgium (N =
664).Students were primarily women (83%) age 18 to 20 years. Lecturers, responsible
for the instruction ofthe course on child development in diverse teacher education
institutions in Flanders, were invited to participate in the experiment. Of these, 20 highly
motivated, qualified ([greater than or equal to] 5 years of experience in teaching)
lecturers participated in the study. Conversations with (the teams of) lecturers in each
institution, prior to the experiment, revealed that their interest in active teaching,
preliminary (positive) hands-on experiences in class, and the prospects of gaining insight
into the effects of student-activating teaching methods and assessment on student
learning and perceptions were the foremost intrinsic motives mentioned. The clear-cut
student-activating teaching/learning materials to be received was the primary extrinsic
motive. During this conversation, each team ofteachers within an institution expressed
their preferences toward the treatments in the study. First choices of assessment-by-
instruction could be assigned to the teacher training lecturers. Student teachers,
following the course on child development, were randomly distributed between the
classes and lecturers. Apart from the lecture-taught students who followed instruction in
large groups (approximately 70 students in each group), class sizes in the activating
conditions were limited to a maximum of 40. However, due to absenteeism and the
phenomenon of dropout in the 1st year of higher education, class sizes were usually
smaller (see Table 1).

Research Conditions

In all, there are five experimental groups in this investigation: one lecture-based setting
and four student-activating learning environments characterized by one of four
assessment modes. Student-activating teaching methods, as defined in this study,
challenge students to construct knowledge by means of authentic assignments (a) that
literally require their active participation in the learning/teaching process to incorporate
the available information by means of discovery learning (b) and with the assistance of a
scaffolding teacher (c) who is continuously available to help and coach the student
teachers. These three characteristics are essential features of the teaching methods that
are used to activate students in this study. The experimental conditions are as follows:

Le: lecture-based learning environment + multiple-choice examination (N = 131)

Mc: student-activating learning environment + multiple-choice examination (N = 119)

Ca: student-activating learning environment + case-based assessment (N = 126)

Pe: student-activating learning environment + peer/ co-assessment (N = 174)

Po: student-activating learning environment + portfolio assessment (N = 114)

The first group of preservice teachers (Le) was instructed in the content of the child
development course within a traditional lecture-based learning environment
characterized by formal lectures and assessed by means of a multiple-choice
examination format. The other four groups (Mc, Ca, Pe, Po) learned in the same student-
activating learning environment, characterized by the same learning content and teaching
methods. However, the assessment mode that accompanied this learning setting
distinguished the conditions from one another. Each group was assigned to a different
assessment mode, namely, (a) a multiple-choice test (Mc), (b) a case-based assessment
(Ca), (c) a peer/coassessment (Pe), and (d) a portfolio assessment (Po).
Because teacher education has a fairly uniform structure in Flanders and a similar
intake of 1st-year students (e.g., gender, socioeconomic status, ethnicity, etc.), each
institution for teacher education was assigned one of these learning environments.
Consequently, all participating student teachers within a particular school were treated
the same way by means of standardized learning materials. This procedure precludes
bias in the results due to students in different classes comparing one treatment to
another. A lack of differences in pretest measures ratifies that groups might be
considered comparable (see also Struyven, Dochy, Janssens, & Gielen, 2006). The
course on child development is compulsory within the 1st year of the program for
preservice elementary school teachers. Table 1 provides a numeric representation of the
matching procedures of research conditions, schools, lectures, classes, and student
teachers involved in the present study.

Learning Materials for the Course on Child Development

The learning materials consist of three parts: a course book, a booklet of student-
activating assignments on the contents studied in the course book, and the four
assessment methods that were used to evaluate student performance in the course.

The course book. The content of the course book (Struyven, Sierens, Dochy, & Janssens,
2003) concerns the development of the child, from conception throughout the school age
period until adolescence. Each developmental stage in the book was treated in the same
way, by a thematic sequence of the domains that characterize the growth of each child
(e.g., linguistic development, physical development, motor development, and cognitive
development).

The student-activating assignments. Apart from the lecture-based learning environment,

all students in the experiment were instructed in the content by means of student-
activating assignments (e.g., problem-based learning tasks, case studies, teamwork, etc.),
in which students were challenged to become active learners who construct their own
knowledge. The assignments required students to select, interpret, acquire, and manage
the information available, aimed at its application in real-life cases or solutions to
authentic, relevant problems. The teachers' role within the student-activating learning
environment was restricted to the supervision and the coaching of the students' learning
processes while they were tackling these tasks. The assignments were collaborative in
nature (6-8students) and required shared responsibilities from students. Ten lessons,
each of approximately 1.5 hours, grouped the assignments and were the same in the four
student-activating groups. Detailed instructions with the assignments were provided to
direct both students and teachers. As well as the pre-constructed, standard
learning/teaching materials for students and teachers, randomly selected observations in
the classes of participating teachers ensured the intended, standardized implementation
of the treatments.

The assessment method. Bearing in mind the importance of alignment between

instruction and assessment (Biggs, 1996; Segers, Dochy, & Cascallar, 2004), the lecture-
based setting was only followed by the multiple-choice test, whereas the case-based
evaluation and peer and portfolio assessment required working with the student-
activating assignments. With the exception of one group (Le), students were largely
unfamiliar with the four assessment methods and the majority of studentshad no prior
hands-on experience of the methods. All methods address the content(s) (and
assignments) of the course on child development and aim to measure similar domains of
knowledge from acquisition and insight to application and problem-solving skills, the
emphasis lying on the latter. Dependent on the sole summative function (Le, Mc, and
Ca) or the combined formative function of assessment (Po, [Pe.sup.1]), the purposes of
the assessment methods are, respectively, to measure individual differences and to assist
learning in combination with the assessment of individual achievement (Pellegrino,
2002).

Each assessment method is discussed in detail and an overview of the characteristics of

the tools and procedures is presented in Table 2. The upper half of the table illustrates
the characteristics of the end-of-course examination or assessment conversation, whereas
the lower part emphasizes the assessment procedure as it is embedded in the course.

Multiple-Choice Test

The multiple-choice test followed the set of classes and included 20 questions with four
multiple-choice answers. Only one choice offers the correct solution to a question. To
avoid pick/guess answers, a correction for guessing was used, that is, wrong answers
were punished by a withdrawal of 1/3 point (correct answer = 1 point, blank answer = 0
points). Items were equally divided into four categories relating to knowledge, insight,
application, and problem solving. Whereas the first category only aims to measure
knowledge acquisition, the other categories surpass this level, aiming at the
measurement of knowledge construction (Birenbaum, 1996). The examination was a
closed book evaluation.

Case-Based Assessment

The case-based assessment in this study was developed according to the principles of the
OverAll Test (Segers, 2003) and concerned the case of Miss Ellen, who is a teacher in
the 3rd year of elementary school. The case material involved a set of information and
documents on the class and pupils, including the following: the class report for the first
trimester, the detailed planning of a week, the thematic planning of the whole school
year, the floor plan of the classroom and its play materials, a medical record on Robert,
and a letter from Cindy's mother. Students received the case materials after the final
lesson so they could use them to prepare for their examination. The examination
questions remained confidential and all concerned this case study, which
required students to apply the theories concerning child development to this
case. Students were allowed to use all resources available (e.g., course book, case
documentation, articles, etc.) to complete their examination.

Peer/Coassessment

Peer assessment has been used to differentiate between the individual contributions to
small group projects (Topping, 2003) and is a cooperative process of peer and teachers'
involvement. In particular, three problem-based assignments related to the contents of
child development were the subjects of the peer evaluation. These assignments required
the preservice teachers to work together in groups of six to eight individuals. The peer
assessment involved group members scoring their peers, and themselves, on the learning
and collaborative processes within the group and the contribution of each group member
to the task's solutions. Scoring was anonymous. Student teachers had to indicate for each
individual in the group, including themselves, whether he or she contributed more (score
3), equal to (score 2), or less (score 1) than average or if the student made no
contribution to the group's work at all (score 0). A calculation of these scores resulted in
an individual peer factor (i.e., an approximate score of 1) for each of the members of the
group. A peer factor score lower than I indicates studentteacher's efforts that are below
average and affect achievement negatively, whereas a score higher than 1 suggests
above-average work and affects the student teacher's performance positively. As already
mentioned, the peer assessment method in the present study is a cooperative method of
student teachers and the lecturer. The starting point of student teacher's assessment is the
product of the group, namely, its solutions to the authentic problem set in the
assignment, which is scored by theteacher. Depending on the individual peer factor (cf.
approximate score of 1), this teacher's score was increased or decreased, which could lie
above or below the average score of the group, resulting in an individual score for the
student teacher that was not only dependent on the student's efforts and contribution to
the group's work but also was dependent on the quality of the group's work. Correction
for favoritism was introduced by eliminating the highest and lowest peer score for each
student. To conclude the peer assessment procedure, an oral group assessment
conversation on the group's assignments was arranged.

Please note that student teachers were to be informed of their individual peer factor (cf.
the approximate score of 1) after each of the three assignments so peer assessment would
serve formative assessment purposes as well. However, due to time restrictions and
excessive workload, the lecturers were not able to provide student teachers with
formative feedback on their peer factor, opportunities of assisting preservice teachers'
learning for future tasks being missed.

Portfolio Assessment

The portfolio assessment consists of a portfolio, including the portfolio-constructing

process, and an assessment interview of the student by the teacher. The portfolio
document contained a selection of activating assignments tackled during classes,
elaborated by students" reflections on their own experiences and learning (Davies &
LeMahieu, 2003; Janssens, Boes, & Wante, 2001). For example, remarkable experiences
during teaching practice, or from their own childhood, had to be applied to the content
studied in the selected assignment. The evaluation of the portfolio served both formative
(during the lessons by means of intermediate feedback sessions) and summative
purposes (during the final portfolio conversation; Black & William, 1998). Only after
the final assessment conversation between student and teacher on the definitive handed-
in portfolio was a final score given to the quality of work and effort that the student had
invested.

The scores for the course on child development were solely determined by the
score on the examination in the multiple-choice (Le and Mc) and the case-based (Ca)
conditions, whereas students' work on the group assignments during the lessons
and on the portfolio was included in students' final marks for the peer (Pe) and portfolio
conditions (Po). Apart from the multiple-choice examination (Le and Mc), all
assessment methods took place by means of an open-book format that allowed the use of
resources. The multiple-choice examination (Le and Mc) and case-based evaluation (Ca)
were individual writtenexaminations, whereas both the peer (Pe) and portfolio
assessments (Po) were accompanied by an oral assessment conversation, in groups and
individually, respectively. These assessment conversations mainly concerned the
submitted work, namely, the problem-based assignments and the definitive portfolio.

Although the assessment formats included in this study are new to the students, it would
be incorrect to conclude that exposing students to unfamiliar assessments in the final
exam is an acceptable practice. Thorough preparations were, of course, made
for students to become informed and skilled examination participants. For instance,
information on the assessment method was given during the first lesson of the course
and sample questions were distributed beforehand. The methods, including formative
assessment, portfolio (Po), and peer assessment (Pe), also included informative
directions from the teacher and a try-out session was run.

Although the goals to be achieved and the skills needed during the assessment processes
might differentiate between the methods, the assessment methods concern similar
contents and assess comparable domains of student knowledge. Notwithstanding the fact
that assessment methods are not interchangeable objective-wise, abstraction is made as
the influence of each assessment mode onstudents' assessment preferences is central to
this study. However, these differences between the assessment methods need to be
carefully born in mind given the purposes they serve for explanation.

Longitudinal Measurements

Data collection on student assessment preferences was obtained using a longitudinal

design by means of questionnaires. Three moments of measurement were selected: (a) at
the end of the first lesson when students had been briefly introduced to the learning
environment and evaluation method for the course (pretest 1, Lesson 1), (b) at the end of
the 10th and final lesson before preparing for the examination (pretest 2, Lesson 10), and
(c) immediately after students had experienced their expected mode of assessment
(posttest, exam). Items on the questionnaires were either closed or open-ended. To
control for social desirability in responses, student teachers attending the course on child
development were reassured of the anonymous processing of the questionnaires.
Moreover, their lecturers would not have access to their personal responses. The
comment was given to student teachers that there were no wrong or correct answers to
the items, only honest or dishonest answers. Obviously, the honest answers represented
the desirable kinds that are aimed for.

Students' preferences for different assessment methods were measured at the end of
Lesson 1 and after experiencing their assessment method. On both
occasions, students were given an identical list of 13 assessment methods, for each of
which they had to indicate their appreciation on a 5-point Likert-type scale ranging from
I really like this method (score = 5), I quite like this method (score = 4), I have no
explicit opinion, I'm undecided (score = 3), I slightly dislike this method (score = 2), to I
really dislike this method (score = 1). The presence of at least five response categories is
required for ordinal-treated-as-interval measurement scales (Likert, 1932).
Because students may be unfamiliar with or have no distinct preferences for some
assessments, a neutral response category was included. The 13 evaluation methods
included in the list are as follows: (a) multiple-choice examination, (b)
individualwritten (closed book) examination (i.e., typical open questions with short
answers), (c) individual oral (closed book) examination, (d) individual (written) open
book examination, (e) take-home examination, (f) practical examination (e.g.,
simulation, internship-exercises, etc.), (g) case-based examination, (h) paper/essay, (i)
portfolio assessment, (j) oral examination in a group, (k) peer assessment, (l) self-
assessment, and (m) cooperative assessment. The unconventional or alternative
assessment methods (such as 5, 6, 7, 9, 10, 11, 12, and 13) were associated with a basic
one-sentence definition to avoid misinterpretations and to make sure student teachers
understand the methods in the same and intended way.

At the end of Lesson 10, students were asked for their perceptions of the expected
assessment method, that is, before they had hands-on experience during the examination
period. Students had to explain how they felt about the method of assessment that
accompanied the course. They had to tick one of the following answers: very good (++,
score 5), good (+), moderate (+/-), weak (-), or very weak (- -, score 1). Blank space was
provided for optional, additional comments.

After the examination, student teachers were not only asked about their assessment
preferences by indicating the extent to which they liked or disliked the methods from the
list but students' perceptions of the appropriateness of the assessment method they had
experienced (i.e., one of the four examination formats mentioned above) were examined.
Student teachers had to indicate whether they found the assessment mode they had
experienced appropriate (yes) or inappropriate (no). In addition, if students thought their
method to be inappropriate, they could suggest a more suitable assessment format in the
blank space provided.

RESULTS

The results are reported in three discrete sections, corresponding to the hypotheses being
tested. Descriptive statistics, (nonparametric) table statistics, and ANOVAs are applied,
including Bonferonni comparisons and effect sizes ([R.sup.2] and Cohen's d). Before
discussing the three hypotheses central to this article, an overall ranking of students'
preferences for the various assessment methods is given to provide an overview.
Generally, students prefer the assessment modes in the order represented in Table 3.

According to the students in this study, the most familiar and convenient assessment
mode, namely, the individual closed book written examination, is generally most
popular. This is followed by the practical examination and then the individual closed
book oral examination. Preferences were scored on a 5-point Likert-type scale:
Consequently, only the methods with a score of less than 3, such as the oral group exam
and the peer assessment format, are actually disliked by students. Although most
alternative assessment modes and methods that allow for the use of resources at home or
at the examination moment are considered to reduce stress and test anxiety (Zoller &
Ben-Chaim, 1988), they score only moderately (approximately score 3) in the list of
student preferences. Please note that high standard deviations (> 1.06) are reported,
suggesting that student teachers' opinions are (more) widespread.
Hypothesis 1: Unknown assessments, unloved assessments. Are unknown assessments
regarded negatively?

When student teachers were asked to evaluate their course on child development during
the final lesson, the answers for assessment method reflect the distribution among the
research conditions shown in Table 4. Table 4 clearly shows that the lecture-
based students (Le), as well as their fellows in the active learning environment with
portfolio assessment (Po), are most satisfied with their expected mode of assessment. In
contrast, student teachers in the student-activating learning environments with multiple-
choice testing (Mc), case-based examination (Ca), or peer assessment (Pe) think
negatively about their forthcoming evaluation method. The Kruskal-Wallis test for k
independent groups reveals statistically significant results ([chi square] = 37.34, df = 4, p
= < .0001) and the Bonferroni mean comparisons replicate the differences between the
lecture-based setting (Le) and three other conditions in the study (Mc, Ca, Pe) and the
activated portfolio condition (Po) and the peer assessment condition (Pe).

The additional comments associated with this item, and the abovementioned values,
reveal interesting arguments in both positive and negative directions. Because the
question for additional information was optional, only about half of
the students expressed their opinion and their answers might not reflect the group's
perceptions accurately. Therefore, the choice was made not to quantify students'
additional comments. Nonetheless, these arguments provide valuable information with
regard to the characteristics of the tools that are preferred or disliked. Caution is
recommended with respect to the generalization of these results.

An important positive argument that associates the alternative assessment methods is

that there is no real examination or exam that is related to cramming and learning by
heart in the perception ofstudents (e.g., "cramming for the exam is unnecessary" [Po],
"no 'real' examination" [Pe, Po], "too much information available to study/cram" [Ca]).
Another positive argument sounded in both the alternative and the traditional assessment
conditions is that theory and (own) practice are interrelated (e.g., "theories need to be put
into practice/applications of theory are more important" [Po, Ca], "stress
is onapplications" [Le], "compatible with my interests and experiences" [Po]). Both
arguments might have given rise to the experienced instructive and informative nacre of
the assessments in the present study (e.g., "instructive/ informative" [Po, Pe, Ca], "open
book format, looking up information is fascinating" [Ca], "intellectually challenging
questions" [Le], "questions cover the range of the course, you know whether you really
understand everything" [Le]). In addition, several answers are relegated to the rest
category that pinpoints particular features of the diverse methods in the study. For
example, for the peer assessment, student teachers mentioned that you learn how to deal
with the evaluations of your work by others than the teacher (Pe)
and students appreciated in the multiple choice format that "littlewriting is needed"
(Mc/Le) and "the correct answer is already shown" (Mc/Le). These arguments contrast
highly with the negative comments that were given to ratify poor scale scores. Perhaps
the lack of a real examination in the alternative assessment method conditions has led
student teachers to feelings of unattained mastery of the contents (e.g., "Do I really know
everything about the contents in the course?"/"I did not grasp the information on child
development" [Po], "I never actually study the theories and information in the course
book" [Ca, Pe, Po]). Moreover, other negative arguments tend to associate the particular
procedures of the method of assessment. For example, some student teachers in the
portfolio condition (Po) thought working on the portfolio required "much efforts" and
associated with "high workloads." Peer assessment student teachers (Pe), on the other
hand, were more concerned about the "fairness" of the procedure, about the difficulties
they experienced assessing other peers, and reported the problem of absent team
members for adequate peer assessment execution (e.g.,
"subjective"/"favoritism"/"appropriate evaluation method?" [Pe], "difficult" [Pe], "not
allstudents attend classes regularly" [Pe]). These students tend to experience unease
about the peer assessment scoring procedure and how it is going to affect their scores
(e.g., "what is my final score going to be like" [Pe]). This comment might have been
strengthened by the poor(er) implementation of the peer assessment procedure because
due to time restrictions and high workloads of the lecturer, formative
interim feedback on the peer factor was not given to student teachers while
working on the assignments. With regard to the case-based examination (Ca), some
student teachers considered the method "easy" but regretted that their efforts on the
assignments during the course did not count in their final scores (e.g., "the assignments
in class should count for our final score" [Ca]). Finally, the multiple-choice test (Le/Mc)
also generated negative arguments that associated poor attitudes toward this evaluation
format, such as "answers resemble each other" (Le), "no additional comments and
support are allowed" (Mc/Le), and the large amount of contents covered by the course
made studentsfeel that there was simply "too much information to study" (Mc/Le).

It should be noted that the portfolio (Po) and peer assessment (Pe) conditions had prior
hands-onexperience with the formative part of the evaluation method and the lecture-
taught group (Le) experienced a partial multiple-choice format examination at the time
this question needed to be addressed. Consequently, Hypotheses I and 2 that unknown
assessments are unloved tend to be true, with the exception of peer assessment (Pe),
which received the lowest appreciation scores despite some hands-on experience
(possibly because the formative function was inadequately served?).

Hypothesis 2: As familiarity with an assessment method grows, students' preferences for

it change positively.

Evidence for this hypothesis is provided by means of the pre- and postmeasurements of
student teachers' preferences for the listed assessment methods. If students' appreciations
of the methods between the first lesson and after their examination are compared by
research condition, each experienced assessment format benefits from the experience of
the respective group of students, as demonstrated in Table 5. Paired t tests show the
statistical significance of the differences. Cohen's d embodies the effect size of the
comparisons and the results of the Bonferroni method are included in the notes for Table
5. The actual experienced assessment mode, as well as the related methods, are
underlined. Positive t values demonstrate gains in student teachers' preferences, whereas
negative scores represent losses of students' appreciation.
The results are remarkable and the four assessment methods and related modes display
(significantly) higher preference scores after the student teachers have had hands-
on experience with them during the course on child development. For example, the
multiple-choice test at the end of the course is remarkably more favored, compared to
the beginning of that same course in the multiple-choice conditions Le and Mc. In effect,
the multiple-choice examination becomes the most popular method in the student-
activating multiple-choice condition (Mc) and second best, behind the individual written,
closed book exam, in the lecture-taught group (Le). The same results are found in the
case-based condition (Ca), where both the open book format and the case-based
examination benefit from the hands-on experience of students and become the most
popular assessment formats for these students. Although self, peer, and cooperative
assessments did not reach the top 3 of preferred evaluation methods according to the
peer students (Pe), they all improved in preference scores, as did the oral examination in
groups that these students experienced. The related open book format and the take-home
exam also gained popularity in this condition. Only the portfolio students (Po) did not
express statistically significant gains in preference for the experienced method.
Nonetheless, student teachers' preference means rose and the portfolio assessment
achieved second place in evaluation preferences behind the individual oral examination,
which would approximate to the format of the final assessment conversation on the
definitive portfolio were it to allow the use of resources.

The preference levels of custom evaluation methods, such as number 2 and 3 in the list,
have fallen. In particular, the appreciation for the individual written, closed book
examination suffered losses in all research conditions. The preference level of the
individual oral (closed book) examination also fell significantly in the lecture-taught
(Le), the active multiple-choice (Mc), and case-based (Ca) conditions. A decrease in
preference is revealed for the practical examination.

Furthermore, Table 5 reveals significant changes in preference that might be provoked

by experiences outside the course on child development. For example, lecture-taught
students (Le) seem to feel less favorable toward the case-based examination, the
paper/essay type of exam, and portfolio assessment, whereas this trend is not apparent in
the active multiple-choice condition (Mc). These latter students tend to like the open
book format more at the end of the academic year but tended to dislike peer assessment.
Similarly, explanations for the shift in preferences for the paper, portfolio, and
cooperative assessment in the case-based (Ca) condition remain unknown. On top of
that, students in the peer assessment (Pe) condition tend to prefer the multiple-choice
format significantly more after the examination, whereas this trend is not apparent
elsewhere. These results are inconsistent with the assumptions and are different for the
various research conditions. Therefore, it is likely that other experiences, in other
courses, during the student's 1st year of elementary teacher training might explain these
phenomena.

This table also suggests the first hypothesis, that unknown assessments are negatively
regarded, to be true. Additional evidence is provided by the ANOVA analysis of the four
concerning assessment methods by research condition for both moments of
measurement, represented in Table 6.

The ANOVAs clearly demonstrate the increase in preferences after student teachers'
hands-onexperience of the assessment mode during the child development course.
Whereas at the start of the lessons (L1) students in different research conditions tend to
think similarly about the four assessment methods in this study, their preferences after
their evaluation experience (Ex) differ between the research conditions. One exception is
the active multiple-choice (Mc) students, who tend to dislike their assessment method
beforehand more strongly than their fellow students in other conditions, whereas
afterward, students consider the multiple-choice test to be their most preferred method.
Comparable a posteriori likeliness is shown by the lecture-taught students (Le), who
favor this method more highly than their fellow students in the case-based (Ca) and
portfolio (Po) conditions. The assessments in the other three conditions (Ca, Pe, Po)
gained popularity after students' experiences with that method to the same extent,
compared to the initial preference scores. Results are statistically significant. The
increasing [R.sup.2]s between Lesson I and the examination moment also support
the effect of the hands-on experience of students' preferences for that particular
assessment method. In addition, similarities to the peer assessment (Pe) condition are
detected only for the portfolio condition (Po).

Hypothesis 3: Students' preferences are congruent with students' perceptions of the

appropriateness of the assessment methods.

If familiarity grows and preferences become more positive (see Tables 5 and 6) after
student teachers' hands-on experience, then it is hypothesized that students' preferences
are congruent with their perceptions of the appropriateness of the assessment method.
Positive experiences of an assessment will compel positive perceptions of its
appropriateness, whereas negative experiences with an assessment method might
produce negative perceptions of its appropriateness. Students were asked whether they
considered the assessment method that they experienced to be appropriate. Table 7
represents students' responses. Blank space was provided if students wanted to
expand on their answer or felt like suggesting a more appropriate assessment format.

After the examination, the majority of the students think of their experienced assessment
method as appropriate or well suited to the course. Notwithstanding, the Pearson chi-
square test of independence reveals significant differences between the research
conditions (dr= 4, [chi square] = 51.30, p < .0001) and the Bonferroni comparisons show
that the peer assessment (Pe) condition and, to some extent, the lecture-taught (Le)
group, are least positive about the appropriateness of the assessment format for this
course. Of interest, the lecture-taught students (Le) seem to be not as satisfied with the
multiple-choice test as their colleagues in the student-activating condition. When
students suggested a different format of assessment as more appropriate, almost all
answers suggest the customary methods of the individual written and oral examination
formats. Sometimes suggestions for changes to the experienced assessment method were
proposed, such as blank space to give additional explanations for the choice of answer in
the multiple-choice test or a combination of multiple-choice items with open-ended
questions.
DISCUSSION AND CONCLUSIONS

In conclusion, the three hypotheses of the study are largely supported by the results,
namely, (a) unknown assessments are unloved assessments; (b) as familiarity grows with
the assessment tool, students' preferences will change positively; and (c) students'
perceptions of the appropriateness of the assessment method will be congruent with their
preferences. Of interest, the conclusions apply to both traditional tests, such as multiple-
choice testing, and new modes of assessment, such as case-based assessment, peer
evaluation, and portfolio assessment.

Unknown assessments are negatively regarded assessments and, prior to this study,
students' assessment preferences did not differ between research conditions. One
exception was the low preference scores of the multiple-choice test in the active Mc
condition. Perhaps the realization that the multiple-choice test was going to be their
assessment method somehow frightened students and made them think more negatively
about their expected examination. However, these critiques faded after the actual
experience of the multiple-choice test, with the active Mc students considering this
format to be their most preferred method and regarding it more positively than their
fellow students, who did not experience this multiple-choice test (Ca, Pe, Po).
Notwithstanding these positive attitudes of student teachers, multiple-choice tests have
been the target of severe public and professional attack during recent years (Sambell et
al., 1997; Zeidner, 1987). Moreover, university students in Sambell et al.'s (1997) study
often reacted negatively when they discussed conventional evaluation methods. Exams
had little to do with the more challenging task of trying to make sense of, and
understand, their subject. In fact, students claimed that traditional evaluation had a
detrimental effect on their learning process. Hence, the results of this study are rather
surprising. The a priori unfamiliarity with this tool, and the possible good use of the
assessment method (cf. student teachers' comments, Hypothesis 1, e.g., good coverage of
the contents, emphasis on application) and its orientation toward knowledge
construction, including the skills to apply content to practice-relevant situations and to
solve real-life problems, might serve to explain these findings. Inversely, the probable
poor use of the peer assessment, in which the formative function (cf.
interim feedback sessions on the peer factor) was not met by the lecturers due to time
constraints and high workload, tends to give an explanation for the moderate to low
effects of this format. Hence, opportunities for adjustments in learning behavior and
group processes were missed and students lacked information on the performance
consequences of the peer assessment procedure (cf. student teachers' comments,
Hypothesis 1). Moreover, student teachers' comments tend to be consistent with the
results of Clifford (1999), who found students to resent staff not taking responsibility
and feeling unsure about their own, and their peers', abilities to grade work. Similarly,
possibly because student teachers lacked interim feedback from the lecturer and
therefore insight in the consequences of the peer assessment procedure for their scores,
preservice teachers in the present study also highlight reservations about favoritism for
friends and subjectivity or incompetence in scoring of their peer assessors. This finding
contrasts with other results (e.g., Segers & Dochy, 2001) that have used the same peer
assessment procedure but with the interim feedback adequately carried out.
Consequently, the use and quality of the assessment method elicit an important
limitation of the present study: Conclusions only apply to the instrument or tool that was
used to make the assessment method operational. Caution in generalizing the findings to
the assessment method is recommended. Replications of this study and verification of
the results are needed. In fact, findings could be extended and illuminated further by
assessing the quality of implementation of the methods in the study. Moreover, the
results in this study consider the participating student teachers grouped in each research
condition as a whole. No additional differential effects or covariables were investigated.
For example, what are the (intermediate) effects of gender, student performance,
individual teachers, and students' approaches to learning, among other factors? Future
research that addresses these questions will be valuable.

Notwithstanding these reservations, the purpose of this study was to investigate the
dynamics of students' preferences for assessment methods, triggered by actual hands-
on experience with assessments, and convincing evidence that is in line with educational
literature (Aday & Campbell, 1995; Feigenbaum & Friend, 1992) was found. In
particular, the assessment methods of each of the research conditions are highly valued
by student teachers after their examinations. This finding seems to replicate and extend
previous findings of Macdonald (2002) and is consistent with research on students
changing their dispositions (e.g., Sambell et al., 1997). However, would it be correct to
advocate that the effects are caused by the hands-on experience and the associated
growing familiarity with the assessment method in the study? An alternative explanation
might be that the results are influenced and biased by social desirability in responses
(e.g., "the teacher expects this answer from me"). The interim findings at the end of the
final lesson of the course tend to support a causal relationship to the biasing effect.
Significant differences are found between the unknown assessment methods and the
familiar modes. In particular, the lecture-taught students (Le) and their portfolio
condition (Po) colleagues were already highly satisfied with their assessment modes.
The lecture-taught students had already experienced a multiple-choice test after the
autumn semester. Likewise, the portfolio students, who were intensively working on the
completion of their definitive portfolios at the time of the Lesson 10 measurement,
already had firsthand experience of working on a portfolio, possibly resulting in Fullan's
(2001) notion of mastery, achievement, or academic growth with student teachers.
Moreover, students also had participated in a formative assessment conversation with
their lecturer, so the routine for their final examination conversation with the teacher was
already known. Please note that the portfolio assessment method provides preservice
teachers with more feedback, support, and direction before the end-of-course assessment
during the examination period than the solely summative assessments in the study. That
is, the formative function that the portfolio serves might have assisted student teachers'
learning (Black & William, 1998; Sadler, 1989) and may have affected students'
attitudes toward this assessment method positively. Of interest, these arguments of
familiarity and of the beneficial formative function of the method were expected to apply
to the peer assessment method as well; however, it did not, as explained above (possibly
caused by a poor[er] implementation of the method). This finding also supports the
causal relationship thesis better than the social desirability explanation.

From this research, we have learned that hands-on experience of assessment does make a
difference and implications for educational practice are indicated. Students' perceptions
of learning and assessment are not stable, unchangeable characteristics. This message is
encouraging in the midst of the accumulating literature on the resistant nature of prior
beliefs and the failure of teacher education programs to produce changes (Kagan, 1992;
Pintrich, Marx, & Boyle, 1993). However, if we want to change the traditional
perceptions of student teachers and teachers, this study provides evidence that hands-
on experience with new assessment tools serves as an incentive for such a change to
occur. This is supported by the finding of Willcoxson (1998) that most academics
mentioned their own experiences of being taught as key factors influencing the way they
themselves teach. Consequently, university instructors working with preservice teachers
ought to be encouraged to use a variety of assessment methods with students in their
courses. Student teachers need to be informed of the properties of the assessment
methods as early as possible so that familiarity will grow fast. However, changes in
perceptions might occur only after actual experience with the assessment and merely
giving informationon the method itself will not be sufficient (Richardson, 1990). In fact,
continuous improvement, action research, and experimentation are characteristics of
successful innovative schools (Little, 1982; Sagor, 2000). Consequently, the
recommendation to expose student teachers to many new assessment method
experiences, as well as teaching method experiences, is apparent. This type of modeling
would benefit the teacher candidates once they have embarked on their teaching careers
as they would have more experience with, and inclination to use, a wider variety of
classroom assessment options (e.g., formative and/or summative, open vs. closed book,
individual vs. group, knowledge acquisition or knowledge construction purposes, etc.).
In this way, a wide range of prospective elementary school children will be
accommodated and diverse learning outcomes are achieved. Moreover, the use of
formative assessments has the ability to raise standards in achievement and in classroom
teaching (Black & William, 1998; Sadler, 1989; Stiggins, 2002). Finally, hands-
on experience with unknown assessment methods is an important incentive to trigger
changes in students' perceptions and assessment preferences. Hence, teacher educators
are strongly recommended to make the most of the assessment experimentation
opportunity in class and constructively contribute to student learning.

In addition, the present study shows that students do not prefer a single method of
assessment while disliking all others. Instead, students tend to prefer a mix of modes and
this mix of preferences shows similarities to, but also considerable differences from, the
findings in educational literature on students' assessment preferences. For example, if
students' preferences for multiple-choice tests and open-ended questions or examinations
are compared, the results of the present study contradict the results of Traub and
MacRury (1990) and Zeidner (1987). The explanation underlies the hypothesis of this
study that unknown assessment methods are regarded negatively. The use of multiple-
choice tests in Belgian (Flemish) Secondary Education (12-18 years) is rare.
Consequently, student teachers in the study enter higher education with little knowledge
of this assessment mode (Van den Berg & Ros, 1999) and, consequently, low preference
scores are obtained. Contrary to the findings of Sander, Stevenson, King, and Coates
(2000) and Macdonald (2002), these students do not favor coursework assessment, such
as portfolio assessments and papers/ essays, to traditional examinations such
as written and oral closed book examinations, at least not initially or prior to actual
experiences with the methods. Likewise, research findings that alternative assessment
methods are favored above conventional assessment methods (Birenbaum, 1997;
Sambell et al., 1997) do not apply to these results. They are more consistent with the
finding of the Zoller and Ben-Chaim (1997) study, in which American students were
particularly interested in the traditional written exam as a known means to achieve
grades. The findings of Birenbaum (1997) and Zoller and Ben-Chaim (1997) that the
oral examination is the least favored also does not account for the students in this
study. On the contrary, the individual oral examination is in the top 3 of preferred
assessment methods. Perhaps student teachers are more talkative than their colleagues in
other higher education programs. The oral group examination, on the other hand, does
not appeal to the students; moreover, students dislike this assessment method as much as
the peer assessment method. Presumably--and the additional comments of student
teachers in the study (cf. Hypothesis 1) and Clifford's (1999) results tend to support this
statement--the dependence of individual scores on others, apart from the teacher, makes
students feel uncomfortable and uneasy. Please note that comparison of the results of the
present study to the findings of previous studies on students' assessment preferences is
hardly warranted. Often, students' prior experiences with assessment are not taken under
investigation, pretest measures are lacking, and students' assessment preferences are
treated as fairly stable learner characteristics. However, the finding that student teachers
like a variety of assessment methods asserts teacher education educators to adopt an
assortment of evaluation formats. Diverse objectives might be set such as helping to
develop positive student teachers' perceptions regarding alternative assessments,
satisfying a broad range of pre-service teachers with distinct preferences (and associated
learner characteristics), or attempting future elementary school pupils to be assisted in
their learning by means of new methods of assessment. Whatever the aims,
enriching teacher education programs with alternative (and formative) assessment
methods in addition to conventional methods might serve the purpose.

REFERENCES

Aday, R. H., & Campbell, M. J. (1995). Changes in nursing students' attitudes and work
preferences after a gerontology curriculum. Educational Gerontology, 21 (3), 247-260.

Biggs, J. (1996). Enhancing teaching through constructive alignment. Higher Education,

32(3), 347-364.

Birenbaum, M. (1996). Assessment 2000: Towards a pluralistic approach to assessment.

In M. Birenbaum & F. Dochy (Eds.), Alternatives in assessment of achievements,
learning processes and prior knowledge: Evaluation in education and human services
(pp. 3-29). Boston: Kluwer Academic.

Birenbaum, M. (1997). Assessment preferences and their relationship to learning

strategies and orientations. Higher Education, 33(1), 71-84.

Birenbaum, M., & Feldman, R. A. (1998). Relationships between learning patterns and
attitudes towards two assessment formats. Educational Research, 40(1), 90-97.
Black, P., & William, D. (1998). Inside the black box: Raising standards in classroom
assessment. Phi Delta Kappan, 80(2), 139-148.

Clifford, V. A. (1999). The development of autonomous learners in a university setting.

Higher Education Research and Development, 18(1), 115-128.

Cuban, L. (1998). How schools change reforms: Redefining reform success and failure.
Teachers College Record, 99(3), 453-477.

Darling-Hammond, L., & Snyder, J. (2000). Authentic assessment of teaching in

context. Teaching andTeacher Education, 16(5-6), 523-545.

Davies, A., & LeMahieu, P. (2003). Reconsidering portfolios and research evidence. In
M. Segers, F. Dochy, & E. Cascallar (Eds.), Optimising new modes of assessment: In
search of qualities and standards (pp. 141-170). Dordrecht, the Netherlands: Kluwer
Academic.

Feigenbaum, E., & Friend, R. (1992). A comparison of freshmen and upper division
students' preferences for small and large psychology classes. Teaching of Psychology,
19(1), 12-15.

Fullan, M. (2001). The new meaning of educational change (3rd ed.). New York:
Teachers College Press.

Grippin, P. C. (1989, February). Using research on teaching excellence to re-

model teacher education. Paper presented at the annual meeting of the Eastern
Educational Research Association, Savannah, Georgia.

Hall, G. E. (1979). The concerns-based approach to facilitating change. Educational

Horizons, 57(4), 202-208.

Hargraeves, A. (Ed.). (2000). Learning to change: Teaching beyond subjects and

standards. San Francisco: Jossey Bass.
Janssens, S., Boes, W., & Wante, D. (2001). Portfolio: een instrument voor toetsing en
begeleiding [Portfolio: An instrument for evaluation and coaching]. In F. Dochy, L.
Heylen, & H. Van de Mosselaer (Eds.), Assessment in onderwijs [Assessment in
education] (pp. 203-224). Utrecht, the Netherlands: Lemma.

Johnson, J. L., & Seagull, A. A. (1968). Form and function in the affective training of
teachers. Phi Delta Kappan, 50(3), 166-170.

Kagan, D. M. (1992). Implication of research on teacher beliefs. Educational

Psychologist, 27(1), 65-90.

Kelchtermans, G. (2005). Teachers' emotions in educational reforms: Self-

understanding, vulnerable commitment, and micropolitical literacy. Teaching
and Teacher Education, 21 (8), 995-1006.

Likert, R. (1932). A technique for the measurement of attitudes. Archives of Psychology,

140, 55.

Little, J. W. (1982). Norms of collegiality and experimentation: Workplace conditions of

school success. American Educational Research Journal, 19(3), 325-340.

Loughran, J., & Berry, A. (2005). Modelling by teacher educators. Teaching

and Teacher Education: An International Journal of Research and Studies, 21(2), 193-
203.

Macdonald, J. (2002). Getting it together and being put on the spot: Synopsis,
motivation and examination. Studies in Higher Education, 27(3), 329-338.

Pellegrino, J. W. (2002). Knowing what students know. Issues in Science and

Technology, 19(2), 48-52.

Pintrich, P. R., Marx, R. W., & Boyle, R. A. (1993). Beyond cold conceptual change:
The role of motivational beliefs and classroom contextual factors in the process of
conceptual change. Review of Educational Research, 63(2), 167-199.

Richardson, V. (1990). Significant and worthwhile change in teaching practice.

Educational Researcher, 19(7), 10-18.

Sadler, R. (1989). Formative assessment and the design of instructional systems.

Instructional Science, 18, 119-144.

Sagor, R. (2000). Guiding school improvement with action research. Alexandria, VA:
Association for Supervision and Curriculum Development (ASCD).

Sambell, K., McDowell, L., & Brown, S. (1997). "But is it fair?" An exploratory study
of student perceptions of the consequential validity of assessment. Studies in
Educational Evaluation, 23(4), 349-371.

Sander, P., Stevenson, K., King, M., & Coates, D. (2000). University students'
expectations of teaching. Studies in Higher Education, 25(3), 309-323.

Segers, M. (2003). Evaluating the Over All Test: Looking for multiple validity
measures. In M. Segers, F. Dochy, & E. Cascallar (Eds.), Optimising new modes of
assessment: In search of qualities and standards (pp. 119-140). Dordrecht, the
Netherlands: Kluwer Academic.

Segers, M., & Dochy, F. (2001). New assessment forms in problem-based learning: The
value-added of the students' perspective. Studies in Higher Education, 26(3), 327-343.

Segers, M., Dochy, F., & Cascallar, E. (Eds.). (2004). Optimising new modes of
assessment: In search of qualities and standards. Dordrecht, the Netherlands: Kluwer
Academic.

Slater, T. F. (1996). Portfolio assessment strategies for grading first-year university

physics students in the USA. Physics Education, 31(5), 329-333.

Stiggins, R. J. (2002). Assessment crisis! The absence of assessment FOR learning. Phi
Delta Kappan, 83, 758-765.

Struyven, K., Dochy, F., & Janssens, S. (2005). Students' perceptions about evaluation
and assessment in higher education: A review. Assessment and Evaluation in Higher
Education, 30(4), 331-347.

Struyven, K., Dochy, F., Janssens, S., & Gielen, S. (2006). On the dynamics of students'
approaches to learning: The effects of the learning/teaching environment. Learning and
Instruction, 16(4), 279-294.

Struyven, K., Sierens, E., Dochy, F., & Janssens, S. (2003). Groot worden: De
ontwikkeling van baby tot adolescent. Handboek voor toekomstige leerkrachten
[Growing: The development from baby to adolescent. Course book for prospective
teachers]. Leuven, the Netherlands: LannooCampus.

Topping, K. (2003). Self and peer assessment in school and university: Reliability,
validity and utility. In M. Segers, F. Dochy, & E. Cascallar (Eds.), Optimising new
modes of assessment: In search of qualities and standards (pp. 55-88). Dordrecht, the
Netherlands: Kluwer Academic.

Traub, R. E., & MacRury, K. (1990). Multiple-choice vs. free response in the testing of
scholastic achievement. In K. Ingenkamp & R. S. Jager (Eds.), Test und trends 8:
jahrbuch der padagogischen diagnostik [Test and trends 8: Yearbook of educational
diagnostics] (pp. 128-159). Weinheim und Base, Germany: Beltz Verlag.

Van den Berg, R., & Ros, A. (1999). The permanent importance of the subjective reality
of teachers during educational innovation: A concerns based approach. American
Educational Research Journal, 36, 879-906.

Van den Berg, R., Vandenberghe, R., & Sleegers, P. (1999). Management of school
innovations from a cultural-individual perspective. School Effectiveness and School
Improvement, 10, 321-351.
Van de Ven, A. H., & Rogers, E. M. (1988). Innovations and organizations: Critical
perspectives. Communication Research, 15(5), 632-651.

Waugh, R. F., & Punch, K. F. (1987). Teacher receptivity to systemwide change in the
implementation stage. Review of Educational Research, 57(3), 237-254.

Willcoxson, L. (1998). The impact of academics' learning and teaching

preferences on their teaching practices: A pilot study. Studies in Higher Education,
23(1), 59-70.

Zeidner, M. (1987). Essay versus multiple-choice type classroom exams: The student's
perspective. Journal of Educational Research, 80(6), 352-358.

Zoller, U., & Ben-Chaim, D. (1988). Interaction between examination-type anxiety state
and academic achievement in college science: An action-oriented research. Journal of
Research in Science Teaching, 26(1), 65-77.

Zoller, U., & Ben-Chaim, D. (1997). Examination type preferences of college science
students and their faculty in Israel and USA: A comparative study. School Science and
Mathematics, 97(1), 3-12.

Katrien Struyven

Filip Dochy

Steven Janssens

University of Leuven, Belgium

Katrien Struyven is a postdoctoral research fellow at the Centre for

Research on Teaching and Training at the University of Leuven (KULeuven), Belgium.
She achieved her doctoral degree on the topic of "The Effects of Student-Activated
Teaching/Learning Environments (Instruction and Assessment) on Students' Perceptions,
Student Performance, and Preservice Teachers' Teaching" in May 2005. Her research
interests lie within the field of didactics and assessment and their respective effects on
student performance, students' approaches to learning, and students' perceptions of the
learning environment.

Filip Dochy is professor of teaching and training and corporate training at the Centre for
Educational Research on Lifelong Learning and Participation and the Centre for
Research on Teaching and Training at the University of Leuven (KULeuven), Belgium.
His research concentrates on new learning and training environments, new modes of
assessment on teaching, and lifelong learning of trainers inteacher training and corporate
training settings. He is past president of the European Association for Research on
Learning & Instruction (EARLI, www.earli.org) and editor of the Educational Research
Review.

Steven Janssens is professor at and director of the Centre for Research on Teaching and
Training at the University of Leuven (KULeuven), Belgium. His research focuses on
subjective theories of teachers, micropolitics in the teacher-pupils relationships, and the
use of video and portfolio in teacher training.

TABLE 1 Overview of Conditions and Number of Schools, Lecturers,

Classes, and Student Teachers Involved in the Study

Treatment

Lessons Assessment [N.sub.schools]

Lectures+ Multiple-Choice Test (Le) 1

Student-activating Multiple-choice test (Mc) 1
assignments+ Case-based exam (Ca) 2
Peer assessment (Pe) 2
Portfolio (Po) 2

Treatment [N.sub.Lecturers]

Lessons Assessment S1 S2

Lectures+ Multiple-Choice Test (Le) 2 /

 Student-activating Multiple-choice test (Mc) 5 /
assignments+ Case-based exam (Ca) 3 1
Peer assessment (Pe) 5 1
Portfolio (Po) 2 1

Treatment [N.sub.classes]

Lessons Assessment S1 S2

Lectures+ Multiple-Choice Test (Le) 2 /

Student-activating Multiple-choice test (Mc) 5 /
assignments+ Case-based exam (Ca) 3 3
Peer assessment (Pe) 5 3
Portfolio (Po) 4 1

Treatment [N.sub.students]

Lessons Assessment S1 S2

Lectures+ Multiple-Choice Test (Le) 131 /

Student-activating Multiple-choice test (Mc) 119 /
assignments+ Case-based exam (Ca) 54 72
Peer assessment (Pe) 87 87
Portfolio (Po) 86 28

NOTE: S1, S2 = School 1 and School 2 in each condition; Le =

lecture-based learning environment + multiple-choice examination; Mc =
student-activating learning environment + multiple-choice examination;
Ca = student-activating learning environment + case-based assessment;
Pe = student-activating learning environment + peer/co-assessment; Po
= student-activating learning environment + portfolio assessment. In
some cases, condition equals school, school equals teacher, and/or
teacher equals class. Although an inevitable feature of
quasi-experimental research, the overlaps in categories make it hard
to directly address the effects of school, lecturer, or class.
Numbers of students are based on the number of students who
administered the questionnaires. In reality, more student teachers
might be subscribed to the course than students involved in the
present study.

TABLE 2 Overview of the Characteristics of (and Differences Between)

the Four Assessment Methods in the Study
 Assessment Multiple-Choice Case-Based
Characteristics Test Exam

End-of-course
exam or
assessment
conversation
Open/closed book Closed Open
examination
Individual/exam Individual Individual
in group
Written/oral Written Written
examination
Questions in the Information in Case study (in relation
exam deal course book to course book info)
directly with?
Assessment
procedure
as part of the
course on child
development
Preparation of Lesson 1 Instructions
students for Interim lessons /
the assessment
method during
the course? Lesson 10 Example
questions
Formative/ Summative Summative
summative
use of the
evaluation
Interim feedback No No
(formal
sessions)
Course work No No
calculated in
final score?
Final score Exam Exam
determined by?

Assessment Peer Assessment Portfolio

 Characteristics

End-of-course
exam or
assessment
conversation
Open/closed book Open Open
examination
Individual/exam Group Individual
in group
Written/oral Oral Oral
examination
Questions in the Group tasks (in relation Portfolio (in relation
exam deal to course book info) to course book info)
directly with?
Assessment
procedure
as part of the
course on child
development
Preparation of Instructions Instructions + Instructions
students for Case study on Try-out Interim feedback
the assessment adolescence (Interim peer feed- (1 session)
method during back, 3 sessions) (a)
the course? Case to go (home) Example Example
+ Example questions questions questions
Formative/ (Formative) (a) + Formative +
summative Summative Summative
use of the
evaluation
Interim feedback (Yes, after each Yes, 1 interim session
(formal groups paper) (a) (and on demand)
sessions)
Course work Yes Yes
calculated in
final score?
Final score Group tasks (N = 3) + Group tasks, including
determined by? peer assessments + individual reflections
exam (portfolio) + exam

(a) Due to time restrictions, teachers were unable to provide students

with the interim peer feedback information.
TABLE 3 Simple Statistics for the List of Assessment Preferences
at the Start of the Course on Child Development

Assessment Method N M SD Ranking

Individual written exam 668 4.24 0.73 1

Practical examination 667 4.10 0.84 2
Individual oral exam 665 3.78 1.12 3
Portfolio assessment 662 3.66 1.14 4
Take home examination 660 3.38 1.20 5
Open book examination 661 3.29 1.20 6
Paper/essay 663 3.29 1.19 6
Self assessment 668 3.27 1.17 8
Cooperative assessment 665 3.22 1.25 9
Multiple-choice test 669 3.11 1.27 10
Case-based examination 661 3.03 1.06 11
Oral group exam 667 2.70 1.23 12
Peer assessment 666 2.67 1.24 13

TABLE 4 The Overall Appreciation of the Expected Assessment Method

at the End of Lesson 10 by Research Conditions, Reported by Means of
Percentages (%) (N = 561)

Total
Condition N M (SD) + + + +/- - - - (%)

Le 113 3.48(0.81) 6.2 47.8 35.4 8.8 1.8 100

Mc 99 2.92(1.10) 7.1 24.2 33.3 24.2 11.1 100
Ca 92 2.95(1.03) 2.2 34.8 27.2 27.2 8.7 100
Pe 154 2.69(1.17) 5.2 23.4 25.3 27.9 18.2 100
PO 103 3.21(1.14) 9.7 39.8 21.4 20.4 8.7 100

NOTE: very good = + + (score 5), good = +, moderate = +/-, weak =-,
very weak = - - (score 1). Le = lecture-based learning environment
+ multiple-choice examination; Mc = student-activating learning
environment + multiple-choice examination; Ca = student-activating
learning environment + case-based assessment; Pe = student-activating
learning environment + peer/co-assessment; Po = student-activating
learning environment + portfolio assessment. Kruskall-Wallis test for
research condition revealed statistically significant results ([chi
square] = 37.34; df = 4; p < .0001). If the scale scores are used, the
Bonferonni comparisons reveal significant differences ([alpha] = .05)
for Le > Ca, Mc, Pe and Po > Pe.

TABLE 5 Students' Preference Values for Each Method From the List at
Both Moments of Measurement by Research Condition, Including the
Values of the Paired t Tests of Examination Minus Lesson 1

Lesson 1 Exam
Condition Evaluation Method M (SD) M (SD)

Le (N = 115) Multiple-choice exam * 3.31(1.25) 3.68(1.24)

Indiv + written exam 4.16(0.71) 3.93(0.90)
Indiv + oral exam 3.82(1.08) 3.47(1.27)
Open book exam 3.46(1.25) 3.42(1.21)
Take home exam 3.38(1.24) 3.21(1.30)
Practical exam 4.24(0.68) 3.38(1.17)
Case-based exam 3.07(1.08) 2.63(1.07)
Paper/essay 3.34(1.24) 2.72(1.32)
Portfolio 3.52(1.12) 3.02(1.35)
Group + oral exam 2.72(1.22) 2.58(1.19)
Peer assessment 2.66(1.21) 2.49(1.31)
Self-assessment 3.34(1.12) 3.03(1.19)
Cooperative assessment 3.28(1.22) 3.08(1.27)
Mc (N = 99) Multiple-choice exam * 2.61(1.27) 3.95(1.20)
Individual + written exam 4.43(0.61) 3.57(1.23)
Individual + oral exam 3.91(1.14) 2.88(1.34)
Open book exam 3.35(1.13) 3.80(1.23)
Take home exam 3.43(1.08) 3.65(1.23)
Practical exam 4.13(0.96) 3.24(1.18)
Case-based exam 2.94(0.99) 2.64(1.21)
Paper/essay 3.23(1.23) 2.78(1.27)
Portfolio 3.51(1.25) 3.01(1.38)
Group + oral exam 2.93(1.24) 2.70(1.41)
Peer assessment 2.94(1.21) 2.37(1.21)
Self-assessment 3.38(1.16) 3.12(1.33)
Cooperative assessment 3.33(1.35) 2.93(1.30)
Ca (N = 91) Multiple-choice exam 3.10(1.19) 3.11(1.38)
Individual + written exam 4.15(0.74) 3.87(0.97)
Individual + oral exam 3.87(1.10) 3.34(1.17)
Open book exam * 3.40(1.22) 4.20(0.91)
Take home exam * 3.59(1.14) 3.58(1.26)
Practical exam 4.10(0.85) 3.48(1.11)
 Case-based exam * 2.94(1.11) 3.97(1.01)
Paper/essay 3.04(1.14) 2.55(1.23)
Portfolio 3.75(1.09) 2.79(1.39)
Group + oral exam 2.83(1.20) 2.82(1.26)
Peer assessment 2.65(1.27) 2.48(1.19)
Self-assessment 3.33(1.22) 3.06(1.12)
Cooperative assessment 3.31(1.25) 2.73(1.28)
Pe (N = 152) Multiple-choice exam 3.11(0.77) 3.42(1.21)
Individual + written exam 4.25(0.77) 3.94(1.06)
Individual + oral exam 3.66(1.13) 3.64(1.15)
Open book exam * 3.07(1.18) 3.83(1.04)
Take home exam * 3.33(1.20) 3.79(1.08)
Practical exam 3.97(0.48) 3.79(1.13)
Case-based exam 3.09(1.14) 2.95(1.20)
Paper/essay 3.45(1.08) 3.25(1.24)
Portfolio 3.77(1.13) 3.46(1.15)
Group + oral exam * 2.50(1.15) 3.26(1.25)
Peer assessment * 2.74(1.29) 3.17(1.26)
Self -assessment * 3.14(1.18) 3.55(1.13)
Cooperative assessment * 3.19(1.23) 3.69(0.95)
Po (N = 58) Multiple-choice exam 3.04(1.17) 3.19(1.14)
Individual + written exam 4.16(0.82) 3.18(1.21)
Individual + oral exam * 4.09(0.84) 4.12(0.65)
Open book exam * 3.39(1.00) 3.35(1.19)
Take home exam * 3.18(1.19) 3.20(1.31)
Practical exam 4.16(0.83) 3.77(1.11)
Case-based exam 2.96(1.02) 3.30(1.10)
Paper/essay 3.30(1.21) 3.05(1.11)
Portfolio * 4.05(1.07) 3.96(1.11)
Group + oral exam 2.82(1.24) 2.81(1.26)
Peer assessment 2.86(1.22) 2.56(1.18)
Self-assessment 3.45(1.16) 3.11(1.23)
Cooperative assessment 3.35(1.31) 2.98(1.25)

Paired t Tests
(Exam - L1)

Condition Evaluation Method df t

Le (N = 115) Multiple-choice exam * 104 2.44

Indiv + written exam 114 -2.49
Indiv + oral exam 114 -3.30
 Open book exam 111 -0.37
Take home exam 109 -1.53
Practical exam 114 -7.66
Case-based exam 113 -3.52
Paper/essay 112 -4.46
Portfolio 114 -3.72
Group + oral exam 114 -1.24
Peer assessment 114 -1.35
Self-assessment 114 -2.75
Cooperative assessment 110 -1.43
Mc (N = 99) Multiple-choice exam * 92 7.98
Individual + written exam 98 -5.89
Individual + oral exam 98 -7.55
Open book exam 94 3.62
Take home exam 96 1.62
Practical exam 98 -6.64
Case-based exam 95 -2.11
Paper/essay 96 -3.24
Portfolio 95 -3.12
Group + oral exam 96 -1.58
Peer assessment 96 -3.99
Self-assessment 97 -2.07
Cooperative assessment 96 -2.82
Ca (N = 91) Multiple-choice exam 79 0.07
Individual + written exam 90 -2.69
Individual + oral exam 89 -3.50
Open book exam * 90 5.44
Take home exam * 87 -0.08
Practical exam 89 -4.98
Case-based exam * 89 7.37
Paper/essay 90 -3.57
Portfolio 90 -6.42
Group + oral exam 89 -0.07
Peer assessment 87 -1.12
Self-assessment 89 -2.16
Cooperative assessment 88 -4.20
Pe (N = 152) Multiple-choice exam 139 3.02
Individual + written exam 150 -3.11
Individual + oral exam 150 -0.25
Open book exam * 149 7.27
Take home exam * 147 3.70
Practical exam 149 -2.06
 Case-based exam 145 -1.18
Paper/essay 145 -1.70
Portfolio 145 -2.88
Group + oral exam * 151 5.95
Peer assessment * 151 3.26
Self -assessment * 151 3.77
Cooperative assessment * 151 4.17
Po (N = 58) Multiple-choice exam 51 0.83
Individual + written exam 56 -5.42
Individual + oral exam * 57 0.31
Open book exam * 56 -0.21
Take home exam * 55 0.09
Practical exam 55 -2.26
Case-based exam 56 1.71
Paper/essay 56 -1.33
Portfolio * 55 -0.50
Group + oral exam 56 -0.10
Peer assessment 56 -1.93
Self-assessment 55 -1.88
Cooperative assessment 54 -1.99

Paired t Tests
(Exam - L1)
Condition Evaluation Method p d

Le (N = 115) Multiple-choice exam * 0.016 * 0.297

Indiv + written exam 0.014 * -0.284
Indiv + oral exam 0.001 * -0.297
Open book exam 0.715 -0.033
Take home exam 0.130 -0.134
Practical exam <0.001 * -0.899
Case-based exam 0.001 * -0.409
Paper/essay <0.001 * -0.484
Portfolio <0.001 * -0.403
Group + oral exam 0.216 -0.116
Peer assessment 0.179 -0.135
Self-assessment 0.007 * -0.268
Cooperative assessment 0.157 -0.160
Mc (N = 99) Multiple-choice exam * <0.001 * 1.085
Individual + written exam <0.001 * -0.886
Individual + oral exam <0.001 * -0.828
Open book exam 0.001 * 0.381
 Take home exam 0.109 0.190
Practical exam <0.001 * -0.827
Case-based exam 0.037 * -0.271
Paper/essay 0.002 * -0.360
Portfolio 0.002 * -0.380
Group + oral exam 0.117 -0.173
Peer assessment <0.001 * -0.471
Self-assessment 0.041 * -0.208
Cooperative assessment 0.006 * -0.302
Ca (N = 91) Multiple-choice exam 0.948 0.008
Individual + written exam 0.009 * -0.325
Individual + oral exam 0.001 * -0.467
Open book exam * <0.001 * 0.743
Take home exam * 0.937 -0.008
Practical exam <0.001 * -0.627
Case-based exam * <0.001 * 0.971
Paper/essay 0.001 * -0.411
Portfolio <0.001 * -0.769
Group + oral exam 0.945 -0.008
Peer assessment 0.265 -0.138
Self-assessment 0.033 * -0.231
Cooperative assessment <0.001 * -0.473
Pe (N = 152) Multiple-choice exam 0.003 * 0.252
Individual + written exam 0.002 * -0.324
Individual + oral exam 0.803 -0.018
Open book exam * <0.001 * 0.683
Take home exam * <0.001 * 0.403
Practical exam 0.041 * -0.181
Case-based exam 0.241 -0.120
Paper/essay 0.091 -0.172
Portfolio 0.005 * -0.272
Group + oral exam * <0.001 * 0.633
Peer assessment * 0.001 * 0.337
Self -assessment * <0.001 * 0.355
Cooperative assessment * <0.001 * 0.455
Po (N = 58) Multiple-choice exam 0.410 0.130
Individual + written exam <0.001 * -0.948
Individual + oral exam * 0.755 0.040
Open book exam * 0.837 -0.036
Take home exam * 0.926 0.016
Practical exam 0.028 * -0.398
Case-based exam 0.092 0.321
 Paper/essay 0.188 -0.215
Portfolio * 0.620 -0.083
Group + oral exam 0.923 -0.008
Peer assessment 0.058 -0.250
Self-assessment 0.066 -0.284
Cooperative assessment 0.051 -0.289

NOTE: Le = lecture-based learning environment + multiple-choice

examination; Mc = student-activating learning environment +
multiple-choice examination; Ca = student-activating learning
environment + case-based assessment; Pe = student-activating learning
environment + peer/co-assessment; Po = student-activating learning
environment + portfolio assessment. The underlined assessment methods
represent either the evaluation method that is experienced within the
research condition or is part of, or similar to, the assessment method
(e.g., the portfolio assessment includes an open book exam; the peer
assessment procedure also involves self- and cooperative assessment).
Cohen's d is used as effect size. Bonferroni comparisons for the list
of methods at the examination show significant differences ([alpha]
= .05) between the research conditions for (a) multiple-choice exam:
Le > Ca and Mc > Ca, Pe, Po; (b) individual + written exam: Ca >
Po, Le > Po, and Pe > Po; (c) individual + oral exam: Le > Mc, Pe >
Mc, and Po > Ca, Le, Mc; (d) open book exam: Ca > Le, Po, and Pe
> Le; (e) take-home exam: Pe > Le, Po; (f) practical exam: Pe > Le,
Mc; (g) case-based exam: Ca > Le, Mc, Pe, Po, and Po > Le, Mc; (h)
paper/essay: Pe > Ca, Le, Mc; (i) portfolio: Pe > Ca and Po > Ca, Le,
Mc; G) group + oral exam: Pe > Le, Mc; (k) peer assessment: Pe >
Le, Mc, Ca, Po; (l) self-assessment: Pe > Ca, Le; (m) cooperative
assessment: Pe > Ca, Le, Mc, Po. At the start of the course,
differences are found for multiple-choice exam: Le > Mc and Pe > Mc
and for portfolio: Po > Le, Mc. Only the scores are used of the
participants for whom pre- and posttest are available (different from
Table 4).

* p < .05.

TABLE 6 One-Way ANOVA the Four Assessment Methods of This Study,

Abstracted From the List of Methods, by Research Condition for
Both Moments of Measurement (MoM)

Evaluation MoM [df.sub.model] [df.sub.error]

Method (Condition)
 Multiple-choice L1 4 664
test (Me and Le) Ex 4 588
Case-based L1 4 656
examination (Ca) Ex 4 637
Portfolio L1 4 657
assessment (Po) Ex 4 636
Cooperative L1 4 660
assessment (Pe) Ex 4 631

Evaluation F p [R.sup.2]
Method (Condition)

Multiple-choice 6.01 <0.0001 * 0.0349

test (Me and Le) 9.04 <0.0001 * 0.0579
Case-based 0.56 0.6907 0.0034
examination (Ca) 31.54 <0.0001 * 0.1653
Portfolio 2.29 0.0588 0.0137
assessment (Po) 12.50 <0.0001 * 0.0729
Cooperative 0.31 0.8729 0.0019
assessment (Pe) 13.19 <0.0001 * 0.0772

Evaluation Significant
Method (Condition) Differences

Multiple-choice Mc < Le, Ca, Pe, Po

test (Me and Le) Me > Pe, Po, Ca
Le > Po, Ca
Case-based /
examination (Ca) Ca > Po, Pe, Me, Le
Po > Me, Le
Pe > Le
Portfolio /
assessment (Po) Po, Pe > Me, Le, Ca
Cooperative /
assessment (Pe) Pe > Po, Le, Me, Ca

NOTE: MoM = Moments of Measurement, L1 = at the end of lesson 1;

Ex = immediately after the examination; Le = lecture-based learning
environment + multiple-choice examination; Me = student-activating
learning environment + multiple-choice examination; Ca =
student-activating learning environment + case-based assessment;
 Pe = student-activating learning environment + peer/co-assessment;
Po = student-activating learning environment + portfolio assessment.
Bonferroni (Dunn) t tests for each couple of conditions ([alpha] =
.05).

TABLE 7 Students' Perceptions on the Appropriateness of

the Assessment Method, After Completing the
Examination on Child Development (N = 570)

Condition N Yes No Total (%)

Le 125 71.2 28.8 100

Mc 107 83.2 16.8 100
Ca 96 89.6 10.4 100
Pe 181 56.9 43.1 100
Po 61 88.5 11.5 100

NOTE: Le = lecture-based learning environment + multiple-choice

examination; Mc = student-activating learning environment +
multiple-choice examination; Ca = student-activating learning
environment + case-based assessment; Pe = student-activating learning
environment + peer/co-assessment; Po = student-activating learning
environment + portfolio assessment. Pearson chi-square test is
statistically significant: df = 4, [chi square] = 51.30, p < .0001.
Bonferonni comparisons for proportions by research condition show
statistically significant differences ([alpha] = .05) for yes: Ca >
Le, Pe, Mc > Pe, and Po > Pe, and for no: Le > Ca, and Pe > Ca, Mc,

Po.
Comments to Comments: Teachers and
Students in Written Dialogue about
Critical Revision

by Christyne A. Berzsenyi

Effective teacher feedback increases students' awareness of the choices they can make and
have made in a piece of writing and enables them to discuss those choices with others. Like
many writing teachers, I continue to search for the most effective methods of teaching
students revision strategies through my feedback to their writing. However, I have felt that
the unidirectional nature of the traditional teacher feedback and student revision of drafts
process produces limited results in terms of actively involving students in rhetorical analysis
that results in more effective text. As the technical writing student notes in the epigraph
above, students are not typically required to articulate a rationale for their choices or offer an
explanation, defense, justification and reconsideration of those choices. In turn, they are not
taught to critically analyze their texts, on which successful revision practices are based. Even
the most provocative and sensitive teacher comments generally ask students to comply with
the teacher's evaluations and suggestions in revised texts, often without a genuine
understanding of the intent of the teacher's feedback. Simply put, there is no meaningful
dialogue about the paper between teacher and student, which means that students do not
learn the internal dialogue of self-critique needed for performing critical revision on their own.
Instead, students associate revision with dependence on a teacher's authoritative evaluation.

Like many teachers, I've used student-teacher conferences to discuss revision with students.
However, I've had moderate success engaging them in active, critical discussion. Students
have shown difficulty talking freely, specifically, and spontaneously in real-time conversation
about revising their texts. Without a conceptual vocabulary to "talk" about writing, students
can not critically discuss their writing in terms of thesis statements, topic sentences, language
conventions, support, audience, appropriateness, purpose, and so forth. Once students
understand this language, they can put this new literacy into practice in intercommunication
with teachers and their peers. Creating what Peter L. Mortensen calls a "talk-back" form of
interaction between student-writer and teacher-reader could give students the voice and
agency to respond to the traditional authority figure. Also, such interaction with teachers
engages students more actively and critically in their own writing processes, an important
component of successful writing, as Pamela Gay, Kathryn Evans, Andrea Lunsford, Helen
Rothschild Ewald, and Richard Beach have noted. However, just as with learning any new form
of literacy, students need exposure and practice with the language of revision before they feel
confident and proficient enough in the discourse to use it in oral communication.

With the goal of developing students' revision literacy and practices, I developed a dynamic,
critical revision method that allows students some time to reflect on their texts and to think
about responses to my feedback before entering into discussion and collaboration with me.
This essay describes the Comments to Comments assignment, an asynchronous, written
collaboration between teachers and students that is designed to teach students to develop,
analyze, articulate, reconsider, and explain their revision ideas. Comments to Comments
begins when students write their first drafts and I write feedback directly on their papers. The
feedback provides marginal and end comments to identify and discuss specific rhetorical
strategies, organization, syntax, semantics, and mechanics issues in their texts-both global
and local revision concerns. Second, I return student papers with feedback and explain the
Comments to Comments assignment requirements with the aid of a handout of instructions,
which illustrates how to respond to my feedback in a dialogue format (see Appendix A). Third,
after students transcribe verbatim all of my feedback on their papers, they must carefully
consider and responsively reply to each prompt. More specifically, students respond with
discussion about their choices, justification for agreeing or disagreeing with my interpretation
of their writing, interpretations of the assignment, questions, challenges, and so forth. What
results is a written document that resembles a play script of dialogue between teacher and
student about the student's text. To encourage active participation in the written dialogue, I
assign students a grade for completing Comments to Comments. Finally, the fourth step
involves teacher's second response in writing to the student's reply comments. Here,
Comments to Comments ends but oral conversations and collaboration could continue.
Granted, the assignment requires a bit more consideration, time, and writing from both
students and teachers in order to achieve a rich and focused discussion and revision process.
However, performing Comments to Comments with students once at the beginning of the
semester has saved me time later in the course because oral conversations, peer revision, and
my feedback are clearer and more meaningful to students as they revise subsequent
assignments. While I consider this assignment to be useful for teaching revision to less
experienced students, it has not been equally successful in all writing classes. I will discuss the
long, tedious process of trial and error that I went through as I refined and adapted the
assignment.

TEACHERS IN WRITTEN DIALOGUE WITH STUDENTS

While Pamela Gay has argued for a similar approach to written dialogue with students in her
1998 essay, "Dialogizing Response in the Writing Classroom: Students Answer Back," my
approach to inviting students into a dialogue of critical revision differs. More specifically, Gay
argues that basic writing students initially need to "vent" their feelings as a structured form of
response in addition to providing a general reaction to the teacher's evaluation of their writing.
Similarly, I agree that venting is important for students to move emotionally beyond their
initial reactions of frustration, disappointment, and even anger into more productive emotional
and mental states for performing critical rhetorical revision. This is why I do not begin revision
dialogue with students immediately after returning papers to them with grades. I wait a class
session or two, after they've had some time to reflect on their writing and my feedback before
I discuss their grades, my comments, or the evaluations. While engaged in writing Comments
to Comments, students respond to my comments and discuss their own texts directly and
specifically in a less emotion-focused revision dialogue than Gay suggests-one with conviction
about rewriting and collaborating.

On the one hand, I do agree with Gay that students should have the opportunity to express
their concerns, confusion, and frustration about an assignment and their performance of it. On
the other hand, I think such dialogue should be a part of the general discourse of the writing
classroom throughout the writing process. Instead of an initial reaction to the paper's grade, I
call students to provide responses of disagreement to specific points of my feedback along
with reasoning to articulate a developed, critical discussion. If, after writing many points of
reasoned disagreement a student feels that my evaluation was too severe, they are invited to
make an argument for it. In fact, students have persuaded me to change my assessment
through substantiated reasoning, which amounts to successful, critical dialogue about their
writing.

THE ROUGH EVOLUTION OF COMMENTS TO COMMENTS

Over the past six years, my approach to giving feedback in Comments to Comments has
developed significantly in accordance with my goals, my evaluations of students' written
dialogues, and with students' anonymous critiques of the revision assignment. As Nancy
Sommers explains, teacher feedback should motivate students to revisit their texts with
curiosity and involvement: "The challenge we face as teachers is to develop comments which
will provide an inherent reason for students to revise; it is a sense of revision as discovery, as
a repeated process of beginning again, as starting out new, that our students have not
learned" (156). Further, feedback should strive to invigorate students' inquiry into concerns of
audience, purpose, terminology, conventions, genre, and form to comprise a critical revision
process, as James Porter suggests. In addition, teachers should address the language
mechanics and format issues. Also, feedback must be given with care to produce generous
responses that are specific, reflective, challenging, and critical as opposed to vague, overly
general, and ambiguous, as Chris Anson and Michael Robertson have argued in their essays.
In turn, students are challenged to think and dialogue about their writing in a substantive
manner. The following lists present types of feedback and response that develop into the
written interaction of Comments to Comments:

Types of Teacher Feedback

* Questions that suggest expansion, clarification, explanation, persuasion

* Remarks which reveal instructor's understanding of the students' texts so that student
writers can evaluate whether that response is in line with their purposes or not

* Identification of a mechanical problem in a specific sentence with a request to explain the

error of standard English and to correct the sentence

* Praise about what is working well accompanied by questions that ask students to explain
why that element works well
Types of Student Responses

* Revised words or sentences accompanied by an explanation or the student's agreement with

the suggestion or interpretation of the text

* Reasoning for disagreeing with instructor's suggestion or interpretation of the text

* Discussions of successful writing strategies that respond to instructor's praise

* New directions for revision that were not initiated by the instructor

While these lists represent the current dialogue guidelines, when I began teaching Comments
to Comments, my feedback did not elicit a critical reflection on student writing. I will address
several of these problems that interfered with productive dialogue and how I modified the
assignment to overcome the obstacles.

Instructor's Handwriting

One problem with written dialogue involved my handwriting. In an effort to be more time
efficient while grading papers, I quickly wrote comments to student's writing because I had so
much I wanted to share with students. However, what resulted was sloppy handwriting that
students could not always read. In addition to hindering students' ability to dialogue with me,
carelessly written communication hurt my ethos. In fact, it was quite embarrassing as one
business writing student referred to my handwriting as "an ancient Egyptian sign system."
Bottom line, my written remarks needed to be more carefully written so that my students
could decipher them without translation from me. In turn, students ask for clarification
occasionally instead of regularly.

Students with Low Revision Literacy

Students also had trouble understanding my feedback when they lacked a working knowledge
of revision literacy. In other words, students must be able not only to read the words but also
to understand the meaning of the questions, rhetorical concepts, and suggestions for revision
in order to respond to them appropriately and completely. After having gotten some responses
such as "I'm not sure what you mean by this" and "I'm not sure how to respond to this
comment," I decided to provide a class session for students to begin the assignment. During
this period, I explain the requirements thoroughly and provide a handout of instructions and
examples of productive revision dialogue. Further, I review overhead slides that present
several types of comments and responses from previous students to illustrate teacher-student
collaboration. Encouraging all students to read through each of my comments, I then address
each student individually, asking if they have any questions and if I can clarify any aspect of
my feedback. I agree with David L. Wallace; it is important to emphasize that students learn
to write purposefully, evaluate their performance in terms of that purpose, and reconsider
their strategies to produce more successful writing for a target audience. Ideally, the teacher
feedback should help students to assess their rhetorical effectiveness and to revise their
writing with purpose and audience in mind.

Marginal or End Comments?

While I have debated over the placement of feedback on student papers, as other scholars
have, I decided that marginal comments are necessary for basic writers, while end comments
are effective with more advanced writers, who have fewer sentence-level revision concerns
than less experienced writers. For example, those basic writers who write sentence fragments,
run-ons, and comma splices have difficulty identifying them on their own in their writing when
I wrote end comments. End comments would vaguely indicate the presence of mechanical
errors somewhere in the paper without giving enough guidelines for inexperienced writers to
revise them. (I'll address revision dialogue about language mechanics a bit later in this essay.)
Also, I found that I wrote superfluous description just trying to identify or reference the
section of their text to which I had a comment. It was an inefficient use of feedback time and
space. The clarity of marginal comments stems from their specificity, and students no longer
give me the responses of confusion such as "what part of my essay are you talking about?" or
"I'm not sure where you mean."

A Teacher Learning to Let Go of Authority

Trying to helpful, I tended to be too directive about what I thought students should do to
revise their work, instead of letting them think it through. In turn, I limited students' critical
thinking processes for revision. Further, I wasn't prompting students to perform rhetorical
criticism of their texts; rather, I was making their decisions for them and then telling them
what to do, which completely undermined my desire to encourage critical thought. To address
this problem, I started to write feedback that was inquisitive and interpretative rather than
directive. For example, I commanded a revision regarding focus by writing, "provide a thesis
statement." As a change, I have articulated a prompt about thesis statements for a female
composition student by writing, "I don't see a thesis statement that indicates the main point of
your paper. Could you write one or revise a sentence that you think suggests your main
point?" With this shift in sentence function, I received much more elaborate responses from
the student who could explain what she thought was her thesis statement and add a new or
revised thesis statement to convey the focus of her paper. My goal here was to encourage the
student to dialogue with me about what a clear thesis is and why it is clear or not clear for a
specific audience in a specific text. Given this, I was able to write a prompt that engaged the
student in a critical decision about how effectively she was communicating the main point of
her essay instead of placing her in a passive subject position.

Students Not Elaborating

When students did not elaborately respond to my feedback, I couldn't gauge whether or not
they understood the assignment, conventions of essay writing, mechanical rules, or the like.
Further, I could not understand what rationale or learned practice was underlying their
decisions, which would help me respond more appropriately to their revision strategies.
Therefore, my follow-up reply wasn't very responsive to why the student made specific
choices. For example, when I praised students for their writing, students often responded with
a simple "thank you." More specifically, when I used the vague praise of "good detail," I
couldn't be sure that the writer was consciously providing details that supported topic
sentences. The point is that my comment of "good detail" did not evoke a conversation. In
fact, it stifled conversation in its definitive evaluation and lack of inquiry into student views or
decisions behind the inclusion of a given detail. Now, I follow-up my praise with a question
that asks students to explain how an element functions in the essay. For example, "good
detail" is followed up with "Why are good details important to topic sentences?" As a result,
students provide reasoning that reflects their understanding of how details illustrate and
support the sub-points provided in a topic sentence. In another example of praise about the
inclusion of a clear thesis statement, I asked a male writing student, "What does your thesis
statement let your popular magazine audience know about what your essay argues?" Here,
the student must rethink the power and clarity of his thesis statement in terms of the
audience, emphasizing a rhetorical perspective. Through this inquiry process about writing and
rewriting, I can reinforce conventions of composition and strategies of argumentation to make
writing more conscious for each student.

Learning to Phrase Questions so That Students Explain

After I learned to phrase suggestions as questions, I still received responses that expressed
uncritical compliance. For example, I would ask a composition student, "how about including
an example of the benefits of training with a professional to be more persuasive?" To this
question, the student responded with "ok." Clearly, I have not engaged the student in an act
of critical discussion about how and why to revise the text. Therefore, I now add to my
suggestions a follow-up question or questions such as "Why do you think I've made that
suggestion" or "Do you agree or disagree and why?" More specifically, I provide a suggestion
for revision and request a rationale for such a revision strategy. For example, I wrote to a
technical writing student on a cover letter for a resume, "Why might explaining the
significance of your work experience here help support your topic sentence and increase your
credibility?" Instead of supplying students with my rationale for a suggestion, I ask students to
critically think through possible strategies, effects, and reasons for revising the work, which is
how I characterize critical revision. As an appropriate and complete response, students must
provide verbatim what additions they would include, and make clear to me that they are not
simply complying with my suggestion. Rather, they must show conviction through elaboration
and reasoning that the decision to revise in a specific manner is a result of their own critical
thinking process with their own goals in mind. Without these inquiries, I was inadvertently
emphasizing the finished product of writing rather than the process of critical revision. The
following are examples of original feedback, student responses, and my follow-up comments:

Instructor's Feedback: Perhaps you could make this paragraph more meaningful to less
experienced readers by saying what the pieces or some of the pieces of equipment do in terms
of the process of beer brewing. What do you think?

Student Response: I definitely think it would make the paper more interesting and flow like a
well-developed paragraph containing more explicit information. Ex. The first piece of
equipment that you might find in your kitchen is a brewpot. A brewpot needs to be made of
stainless steel or enamel-coated metal that is not chipped. (nontraditional basic writing
student) Instructor Follow-up: Excellent, thorough revision discussion here!

Instructor Feedback: Where's your thesis? Would you write one?

Student Response: Weather forecasting will always be a challenging occupation, but with the
help of radar, Doppler Radar, and computers, predicting weather has become more accurate
over the years.

Instructor Follow-up: Excellent thesis with clarity about where you are heading with this
essay.

Having to rationalize a revision decision will increase the internalization of revision inquiry and
strategies. In other words, students learn by doing the work of critical revision.

Inviting Students to Disagree

A few years ago, a couple of students expressed disagreement with my feedback and did so in
an intelligent, critical response. After recognizing the value of encouraging this kind of critical
thought, I began to invite students to disagree with my feedback, if that's how they felt. While
a few students lacked good reasons for their disagreements, most disagreements resulted in
productive dialogue when students thought critically outside of my suggestions and in terms of
their own rhetorical goals. While disagreement is invited, substantiation or justification for the
student's alternate position is required, just as an instructor's comments need justification.
Instead of simply expressing compliance and telling me what they think I want to hear, I
encourage students to critically think through what they want for their own writing, what they
mean, and how they want readers to think about their subject. Further, such justification and
clarification enables me to better understand a student's purpose and reasoning, which may
not have been obvious while reading the paper. In some cases, students' disagreement has
convinced me that their reasoning makes more sense rhetorically than what I had in mind. In
other cases, the students' disagreement indicates some misunderstanding of the assignment
or my comment. The following are examples of my original feedback on their papers, their
disagreements with my feedback, and my follow-up comments:

Instructor's Feedback: Why did you change paragraphs? You continue to talk about
regulations regarding basketball baskets in the second paragraph. Explain. Agree/disagree?

Student Response: I thought I should change paragraphs because I was talking about the past
and then I started to talk about the present. (Basic writing student)

Instructor's Follow-up: Ok. Good explanation. For revision, make sure that your topic
sentences reflect your change in time frame.

Instructor's Feedback: Performance effect of lowering a truck? Does it give a smoother ride or
the like? That would mean more to someone like me who doesn't know much about
customizing trucks-your target audience.

Student Response: I answered the question later in the essay. Therefore, there's no need for
revision. (Basic writing student) Instructor's Follow-up: Ok. I see that you do address this
later. My comment anticipated the significance of the process of lowering a truck, which you
do address successfully at a later point.

Instructor's Feedback: Why change paragraphs? Explain why or why not.

Student Response: The reason for changing paragraphs is because I got this information from
two different sources. Not only that, the one paragraph shows the effects on male smokers
and the other shows the effects on female smokers. After your explanation in class, I now
know that I need to do some revision on my topic sentences so that I clearly explain that one
paragraph is about men and the other is about women. Also, I need to show more details on
the subjects "male" and "female" in each paragraph. (Composition student)

Instructor's Follow-up: exactly!

Instructor's Feedback: Could you elaborate on this theory [of how the mysterious contents in
the briefcase might be an Academy Award Oscar statuette]? It seems more realistic.
Student Response: I don't believe in this theory at all. I refuse to believe that the briefcase,
something that boggled the greatest movie minds contained an Oscar. It's not something that
could cause people to get killed over. (Basic writing student)

Instructor's Follow-up: Ok. That's logical reasoning provided here. You sound firm in your
belief with a strong voice that is missing from other parts of the essay. Put this in the paper to
provide a strong refutation against this theory; it's persuasive.

What's a pleasure for me to see is that student disagreements are generally written in a
strong voice and reflect students' commitments to their writing and to their rhetorical goals.
On the one hand, some disagreements reinforce in students what their goals are and how to
achieve them better. On the other hand, other disagreements express a misunderstanding
about the assignment or my comment, which I could then clarify for them by referring to the
assignment requirements or by explaining what I intended by my comment. Regardless, in
disagreement, students show they care enough about their text to disagree with me and
negotiate the rewriting, reshaping, and re-imagining of their work.

Dealing with Mechanics

In terms of discussing the revision of language mechanics, I realized that I needed to address
conventions and guidelines with inexperienced writers. As Robert Connors, Mina Shaughnessy,
and others have argued, mechanical errors are rhetorically significant in the construction of
ethos as writers meet discourse community expectations. Years ago, I would make "x"
markings to indicate some form of mechanical error in that sentence without blatantly
identifying the error so that the student would have to think about it. However, in response, I
received some reactions of confusion that never developed into critical dialogue such as "I'm
not sure what's wrong here" or blind guesses such as "Is it a fragment?" Then, I came to
believe that inexperienced student writers need more specific feedback than vague circles
around an error or markings without information about the mechanical error. However, they
also need more active revision work than what comes with the instructor's correction of an
error, which students gloss over or completely ignore, and, therefore, do not learn to
recognize or correct.

As a new strategy, I decided to name the mechanical error in an identified sentence. However,
inexperienced writers would still be confused about revising since many don't have the
vocabulary for understanding some mechanical errors by name. For example, on one student's
essay, I indicated that a sentence was a comma splice, asking the student to edit the
sentence. When the student responded to my prompt, he wrote, "I don't know what this is,"
and provided a new sentence that produced another comma splice error. Clearly, I had to
address at least some of the common grammatical errors instead of just assuming or
expecting that writing students know what they are and how to correct them. In fact, I began
introducing revision by reviewing some of the common mechanical errors. Then, if they still
made the errors in their papers, I would state the error in the sentence and ask students to
review by referencing that type of error in their style manuals, explain why it is
ungrammatical, and rewrite the sentence with the correction. For further assistance in this
endeavor, I suggest going to my office hours and/or to the Writing Center. The following is an
example of dialogue with a basic writing student about language mechanics:

Instructor's Feedback: Why is this a fragment? Please revise the sentence.

Student Response: The subject was missing which is necessary for a complete sentence. The
sentence should read, "Baking in the sun all day, I got very burned."

Instructor's Follow-up: Good work.

Having students take the time to reference resources on mechanics and explain why their
writing reflects an error in mechanics helps students to recognize that kind of error and learn
the appropriate conventions for the first time. The following is an example of a revision
dialogue with a basic writing student who did not fully answer my prompt and, in turn, also
didn't really learn what was ungrammatical about his sentence:

Instructor's Feedback: Why is this a run-on? Revise the sentence.

Student Response: DNA is a perfect tool for law enforcement agencies to have especially with
the criminals getting more advanced it gives the police an advantage.

Instructor's Follow-up: This is still a run-on. Let's work in my office on how to correct run-on
sentences, which combine two or more sentences without proper punctuation.

As with the above example, the dialogues about mechanics can reveal a lack of understanding
about the type of error and provide an opportunity to develop skill in that area, which might
have gone neglected by the student without such prompting from me. Other students may
have typos, which simply show a lack of proofing and editing. Generally, students respond to
typo comments by bashfully acknowledging a "stupid mistake" that was overlooked in error.
Then, they must correct the typo. Calling attention to these surface level mistakes helps
reinforce good proofing and editing practices that should occur during the final stages of
revision to build a credible ethos.

Learning to Convey Suggestions Clearly

When students occasionally gave thin and unreflective responses, I realized that my
comments mistakenly conveyed a neutral attitude about a missing, key element. For example,
the following exchange with a composition student illustrates my miscommunication:

Instructor's Feedback: I'm not sure where your thesis is. Student Response: It's in my first
sentence, second paragraph. What I meant by this vague comment was that it was a problem
of clarity, purpose, and focus that I could not find this essential feature of an essay, the
thesis. The second sentence of his essay was not an arguable claim. What I was hoping the
student would have done with my comment was to realize that he had not written a clear
thesis statement and that he would provide one for the revision dialogue and subsequent
draft. However, the lack of direction and nonchalant tone of my comment did not facilitate
such a response. I had not clearly communicated my assessment, suggestions for revision, or
my expectations for further dialogue. With some embarrassment, I responded to the student
by acknowledging my lack of clarity, requesting revision of the thesis, and stating its
importance in an essay:

Instructor's Follow-up: I realize now that my comment is vague. What I should have said here
is that the fact that I can't identify a thesis means that I need you to clarify what you are
arguing by writing a clearer thesis. Why do you think that your lay magazine readers should
easily understand the main point of your essay in terms of persuasiveness?

In this follow-up comment, I admit my error to the student, which I believe creates a less
authoritative relationship and, therefore, further enables more genuine collaboration between
us. Also, I ask him to revise so that he would better achieve his own goals of readers'
understanding of and adherence to his main argument-a rhetorical analysis of the student text
with the student.

Praise, Praise, Praise!

An important counterpart to discussing weaknesses in a student's writing is to identify

strengths as well, a strategy I incorporated into Comments to Comments. Since praise
encourages students to overcome writing apprehension (Daiker 105) and offers "the
psychology of positive reinforcement" (Irmscher 150), students develop more positive
attitudes about their writing. Also, praising student writing lets students know that they are
doing things well, which reinforces effective writing strategies, as Macrorie, Hirsch,
Shaughnessy, and Diederich have argued. However, my early forms of praise were typically
vague comments of "good" or "yes" in the margins. After receiving simple expressions of
appreciation for the compliment rather than a critical dialogue, it was obvious to me that
students need to develop an understanding of what is "good" writing within specific contexts
so that they can use that strategy again in future writing. In turn, I modified my praise
feedback by identifying the strength of the text with comments such as "clear thesis."
However, in response, I consistently received unreflective responses such as "thanks" or "The
book shows us to do it this way." This student demonstrates attention to the book as a
reference to disciplinary writing but does not express an understanding of how or why this
strategy operates for the community or purpose. Thus, I needed to modify my praise in order
to stimulate critical thought about why a particular sentence or strategy works well in a
student's paper. In turn, I started to include a question along with the praise to stimulate
discussion of the reasons the rhetorical element served the paper's effectiveness. However,
responding to praise often perplexes students because they don't see a need to reply to a
compliment with which they are in agreement. Therefore, I had to explain that even my praise
necessitated their analytical responses. Addressing my praise of their written work directly
helps students to realize how good a paragraph, detail, or the like actually is. I want students
to recognize when and how they have moved closer to their rhetorical goals with effective
writing, conclusions they may not have realized during the drafting of their papers. Clearly,
the realization of one's strengths as a writer enables greater confidence in that writing ability.
The following exchanges about praise are with three basic writing students:

Instructor's Feedback: This scene came to life for me because of your vivid description of the
cabin fire using active verbs and expressive adjectives. How did you decide to write these
details?

Student's Response: I carefully selected these words in an attempt to recreate for the reader
the energy that was apparent at the time and which caused me to act with fear, anger, and
finally action.

Instructor's Follow-up: Clear reasoning!

Instructor Feedback: Good topic sentence! Why are topic sentences important to paragraphs
and essays?

Student Response: Topic sentences let the reader know that the subject is changing and also
keeps them interested.

Instructor Follow-up: Yes!

Instructor Feedback: Great Title! Why are great titles important?

Student Response: The title got your attention and it stated what the essay was going to be
about. It is also a fun one!

Instructor Follow-up: yes! I agree!

Such student responses indicate analysis of rhetorical strategies, which lead to an
understanding of how to produce particular effects for readers in subsequent writing tasks.
Also, focusing on strengths celebrates what students have accomplished, which helps to
change the instructor's role from authoritative corrector of mistakes to writing collaborator and
encourager.

What's the Final Word?

As a revision strategy, instructors can use Comments to Comments either as the final act of
revision on a paper or in conjunction with final rewrites to be submitted for a grade. The
decision to follow-up Comments to Comments with a graded, revised draft depends on
instructors' time constraints and on their desires to achieve or to do as Julie Jung suggests,
disrupt closure to students' revision processes. Worth noting, students typically prefer to
revise the draft after Comments to Comments in order to achieve the finished, improved text.
In fact, when I have not asked students to turn in a revised essay after Comments to
Comments, they report less satisfaction with the assignment and have criticized me for not
requiring the follow-up revisions in the text. Several students associate improvements in their
writing when they see the evidence before them in their final papers as compared to earlier
drafts. As a result of this critique, I started to include a second draft in the project, which
further reinforces the importance of revision as an ongoing process. After all, if students feel
good about their writing, they will write more and become more confident.

COMMENTS TO COMMENTS: MIXED REVIEWS

For the past six years, I have taught Comments to Comments in a wide variety of writing
courses that included basic writing, standard composition, honors composition, technical
writing, and business communication. However, I adjust my feedback and expectations to suit
the course and the level of student writing, which produces different forms of written dialogue,
collaboration, and revision. In fact, my method of adapting the assignment for each course
was developed in part by anonymous student feedback on the assignment. Students from the
full range of writing courses answered the question, "Has Comments to Comments been an
effective method of learning to revise your own writing? Why or why not?" In the introductory
technical writing classes, students overall found the assignment to be useful and could
complete the assignment without much difficulty. With more advanced writers, the assignment
can feel like busy work because they have already interiorized much of the kind of revision
dialogue in which I would engage them. The advanced business writing students, for example,
completed Comments to Comments with thorough elaboration about their writing choices and
with a greater percentage of disagreements with me than less experienced writers provide.
Both of these factors reveal that they have had more experience with writing and that they
have more direction in terms of how they wish to formulate their documents and why.
Therefore, I no longer use Comments to Comments with the senior level students. Instead,
I've found that Comments to Comments makes a discernable difference with less experienced
writers. Discussions of subsequent drafts and assignments reveal experimentation and
practice with their new literacy and elevations in their confidence and interest in their writing.
Also, in the technical writing classes, students learn the discourse of communities of science
and technology, which is unfamiliar, formal rhetorical territory. I find that Comments to
Comments calls students to apply the technical writing concepts from our textbook in written
dialogue, and, therefore, incorporate those terms into their own work. It's exciting to see
student writers' anxieties about writing diminish as they learn to talk about the assignments,
ask questions about writing for an audience, and discuss how they plan to fulfill the
requirements and their own purposes.

While students continue to have criticisms about the assignment, they generally fall into one
of two types and are few in numbers. A small percentage of students expressed a preference
for speaking with me rather than writing with me their dialogues about revision. Among the
basic writing students surveyed, only four students out of eighty either considered Comments
to Comments to be as useful as or less useful than oral conferencing. I encourage these more
face-to-face communication-- oriented students to come to my office for individual
conferencing at any point in the revision process. The second criticism is that Comments to
Comments takes a long time to complete, which is accurate if the student is responding
specifically and elaborately in revision. For example, one student identified having trouble
referencing the style manual to learn about a mechanical rule and explain how to revise the
sentence. Another student stated that there were a lot of comments to which to respond,
which made the assignment lengthy. Despite these relatively few criticisms, other students
reported that Comments to Comments "wasn't that hard," was a "good learning tool," and
"was very fun talking about something I know about [myself]."

Overall, student feedback suggests that I continue to use Comments to Comments for
teaching revision in classes with less experienced writers. The following kinds of positive
feedback clearly prevailed among the students' responses and motivated my continued
application and modification of the assignment:

Yes, I think it makes you think about why you do certain things and how to do them right.
Many times you don't know why you are making certain revisions but this makes you think
about why you are doing them. Doing this allows you to understand revising better, therefore,
being better at it in the future. (Basic writing student on essay assignment)

Instead of just reading and probably ignoring your comments, I had to analyze what you were
saying, think about the effect of the comment on my document and then decide how to go
about either implementing the comment or telling you why I felt the comment wasn't good for
me and my purpose. It made me reason out my decisions for putting certain things in my
document. I think this helped me to develop a more readable, scannable, and attractive
technical document. (Technical writing student on resume assignment)

I believe I learned more from doing the Comments to Comments than I have from just
revising rough drafts. It made me think of more than one technique of revising. Instead of
coming out and stating how to fix the work, the students are told what is wrong and they must
figure out how to fix it. (Basic writing student on essay assignment)

I do believe that the Comments to Comments assignment is a good idea because it gives you
a good amount of time to answer the questions with a good response as well. I also think that
it might be a good idea to revise the resume and have it graded a second time to make it an
even better copy. (Technical writing student on resume writing)

This feedback gave me the sense that students understood the goals of the assignment as well
as the benefits of revision on their writing. Also, they wrote that they enjoyed talking about
themselves and in "writing the play script." Further, students indicated satisfaction with the in-
depth exchange about their writing, which focused on aspects of persuasion and convention.

Overall, the main difference between how I've used Comments to Comments in the various
writing courses is the kind of feedback I give and, in turn, students' responses. On the one
hand, in sophomore writing classes such as the technical writing classes, my feedback is
focused on rhetorical issues-meeting audience expectations and document-form conventions,
and achieving writing objectives. On the other hand, in firstyear writing courses, my feedback
addresses assignment requirements, essay features, paragraphing, mechanical errors, and the
use and documentation of research sources. As student writers across courses, cultures, and
writing contexts have varying needs for improving their rhetorical effectiveness, even within a
given course, I have to adjust the revision dialogue to meet those needs. Those needs become
apparent to me as I get to know them in class with writing assignments, one-on-one
conferencing, and in-class interaction. For example, with ESL students in my technical writing
classes, my comments focused on their language mechanics related to writing in a second
language, which differs from basic writing language mechanics issues. In sum, how I practice
and implement Comments to Comments in writing classes is always under review and
adjustment to individual students and their writing.

More on Comments to Comments

Future work on this pedagogy includes methods for assessing changes in students' awareness
and application of revision. At this point, while I have considered setting up test classes as
control groups, too many variables make any attempt to correlate changes in their papers with
their completion of Comments to Comments inconclusive. As it continues to evolve, this
creative and critical pedagogy will focus on students' awareness of how they write, why they
write, to whom they write, and ways that their writing can become more effective.

As Robert Probst argues, teacher feedback should involve students in a "shared commitment-
they are not opponents sparring in a linguistic ring, the student attempting to slip confusions
and inadequacies past the teacher, and the teacher attempting to catch, label, and castigate
all the flaws" (70). Accordingly, Comments to Comments is a collaborative model of student-
teacher revision; students are more likely to perceive instructors as being on their side,
working on their behalf, rather than as an obstacle to overcome, psyche out, figure out, or
manipulate in order to earn the desired grades. The sense of team spirit becomes a strong
part of the complex power relationship between students and instructors. Further, teachers'
feedback should suggest that student writing matters enough to warrant a collaborative
revision endeavor. Calling students to actively dialogue with instructors about their writing
makes it difficult for even resistant students to be apathetic. Such repeated readings are
encouraged by the revision dialogue of inquiry, discussion, and negotiation between teacher
and student. Through this experience with revision, students develop a literacy that makes
face-to-face conferencing and peer review more productive and meaningful. Furthermore, with
practice, the language of revision dialogue becomes the internal dialogue of self-- critique,
therefore, making a more resourceful writer.

Echoing Dawn Dreyer's purposes in teaching revision with students, I designed Comments to
Comments to invigorate in students an attitude toward writing that involves self-awareness,
effective communication with others, and interest in their own writing-all of which fuel
repeated readings, reconsideration of original plans, and assessment of current and new
directions in writing.

Lehman, Pennsylvania

-1-