Course: Language Assessment - Principles and classroom practices

Chapter 7 and 9
Summarized by: Nguyễn Thị Mỹ Duyên - Student ID: 1967 012 049
Class : 19NHGD1

CHAPTER 7 SUMMARY: ASSESSING SPEAKING

1. SPEAKING SKILL

• Definition of Speaking
Speaking is the real-time, productive, aural/oral skill (Bailey, 2003:48). It is real
time because the other interlocutor is waiting for the speaker to speak right then and
the speaker cannot revise his response as he might do in writing. It is productive
because the language is directed outward. It is aural because the response is
interrelated with the input often received aurally and it is oral because the speech is
produced orally.

• Levels of Speaking
From the highest to its lowest level, speaking can be dissected into text, utterance,
clause, phrase, word, morpheme and phoneme (van Lier, 1996). Success in speaking
means being able to communicate message using accurate and acceptable use of
language throughout these levels. Knowing these levels shall help test maker
understand what to expect from test taker’s performance.

• Types of spoken Language

Spoken Language can be in the form of monologue or dialogue. A monologue
can be planned or impromptu while dialogue is almost always unplanned while
dialogue can be interpersonal or transactional; each can be either familiar or
unfamiliar.

• Micro- and Macroskills of Speaking

Brown (2004:142-143) suggests a list of micro- and macroskills of speaking to
help determine test maker as what to assess (whether to assess on smaller chunks of
language or speaking’s larger elements) as follows:

Microskills
• Produce differences among English phonemes and allophonic variants.
• Produce chunks of language of different lengths.
• Produce English stress patterns, words in stressed and unstressed positions,
rhythmic structure, and intonational contours,
• Produce reduced Forms of words and phrases_
• Use an adequate number of lexical units (words) in order to accomplish
pragmatic purposes.
• Produce fluent speech different rates of delivery.
• Monitor one’s own oral production and use various strategic devices—pauses,
fillers, self-corrections, backtracking—to enhance the clarity of the message.
• Use grammatical word classes (nouns, verbs, etc.), systems (e.g., tense,
agreement, pluralization), word order, patterns, rules, forms.
 • Produce speech in natural constituents— in appropriate phrases, pause groups,
breath groups, and sentences
• Express a particular meaning in different grammatical forms.
• Use cohesive devices in spoken discourse.
Macroskills
• Appropriately accomplish communicative functions according to situations,
participants and goals.
• Use appropriate styles, registers, implicature, redundancies, pragmatic
conventions, conversation rules, floor –keeping and –yielding, interrupting, and
other sociolinguistic features in face to face conversations.
• Convey links and connections between events and communicate such relations
as focal and peripheral ideas, events and feelings, new information and given
information, generalization and exemplification.
• Convey facial features, kinesics, body language, and other nonverbal cues along
with verbal language
• Develop and use a battery of speaking strategies, such as emphasizing key
words, rephrasing, providing a context for interpreting the meaning of words,
appealing for help, and accurately assessing how well your interlocutor is
understanding you.

2. ASSESSING SPEAKING
• Challenges in Assessing Speaking
Hughes (1984:101) believes that that successful interaction involves both
comprehension and production. For that reason, he believes it is essential that a task
elicit behavior (or performance) which actually represent the test taker’s speaking
competence. In addition to selecting the appropriate assessment, O’Malley (1996:58)
also mention determining evaluation criteria as another major challenge. Much in the
same tone, Brown (2004:140) describes two major challenges in assessing speaking:
(1) the interaction of listening and speaking (e.g. the use of much clarification) can
make it difficult to treat speaking apart, (2) the speaker’s strategy to dodge certain
form to convey meaning may make it difficult for test makers to design a solid
elicitation technique (one that can result in the expected target form).

• Basic Types of Speaking Assessment Tasks

Brown (2004:141) provides 5 types of Assessment Tasks. The headings below are
Brown’s proposed categories but the tasks in each category come also from the
descriptions by Heaton (1988), Hughes (1989) and O’Malley (1996). In the past it is
agreed that speaking leaves no tangible product to be assessed (unlike writing), yet
today technology has make it possible to record the speech in every type of the task. A
 challenge of this sort has little relevance to today’s practice. Therefore, albeit
unmentioned, the following types of Task may involve recording the test taker’s
speech.
1 Imitative: repeating a small stretch of language and focused on pronunciation.
Test maker considers using this type of assessment if he is not interested in test
taker’s competence in understanding and conveying meaning or in getting
involved in interactive conversation. The competence assessed is that of purely
phonetic, prosodic, lexical and grammatical (pronunciation).

2 Intensive
2.1. Reading Aloud
Heaton (1988:89) and Hughes (1989:110) maintains that the use of reading aloud
may not be appropriate because of the difference in processing written input from that
of spoken one. However, a check on stress-pattern, rhythm and pronunciation alone
may be conducted using reading aloud. Brown (2004:149) suggests that we use
reading aloud as a companion for other more communicative tasks.
2.2. Directed Response Task (e.g response to a recorded speech)
One of the most popular Task of speaking for its practicality and mass lab- use,
despite its mechanical and non-communicative nature, DRT is beneficial to elicit a
specific grammatical form or a transformation of a sentence which requires minimal
processing (microskills 1-5, 8 & 10) (Brown, 2004:147).

2.3. Sentence/Dialogue Completion

Heaton (1988:92) warns us the fact that this type may provide illogical flow of
conversation given that the sentence or dialogue completion is done in lab (which is
what normally administered). Therefore, this type will probably be beneficial only for
assessing test taker’s microskill of providing the right chunks of language and other
pronunciation features.
However, as Brown (2004:151) exemplifies, a more responsive-type of
sentence/dialogue completion may actually be free of said caveat and keep us from the
risk of judging a test taker’s competence as insufficient caused by aural
misunderstanding in processing the input. SDC helps measure speaking competence
apart from its interrelatedness to listening.
 2.4. Translation up to simple sentence level (interpreting-game)
Interpreting, as Hughes (1989:108) describes, may involve the test-proctor acting
as native speaker of test taker’s first language and the test taker interpreting the
utterance into English. It is believed that because speaking is negotiation of intended
meaning (O’Malley, 1996:59), interpreting-game can be used to measure test-taker
competence in conveying his message into the target language (Brown, 2004:159).
2.5. Limited picture-cued Task (including simple sequence)
Pictures are mostly convenient to elicit description (Hughes, 1989:107). In
addition to describing comparison, order of events, positions and location, a more
detailed picture may be used to elicit test taker’s competence in telling a plan,
directions and even opinions (Brown, 2004:151-158).

3 . Responsive:
Small dialogue, response to spoken prompt (simple greeting, request &
comments)

3.1. Question and Answer

Questions at responsive level tend to be referential (as opposed to intensive,
display question) (Brown, 2004:159). Referential question requires test takers to
produce meaningful language in response. Such questions may require an open-ended
response or a counter-question directed to the interviewer (Brown, 2004:160).

3.2. Giving Instruction and Direction

In this type of task, test takers are elicited their performance in describing a how-
to description. A five- to six-sentence response may be sufficient to be required either
from an impromptu question or a-minute planning prior to the instruction (Brown,
2004: 161).

3.3. Paraphrasing
Oral Paraphrasing can have written or aural input with the latter being more
preferable. A paraphrase as a speaking assessment should be conducted with caution
because test taker’s competence may be mistakenly judged by their short-term
memory and listening comprehension instead of their speaking production.
4 Interactive (larger dialogue on Transactional and Interactional Conversation)

4.1. Interview
Interview can be face-to-face, one-on-one or two-on-one each with its advantage
and disadvantage. A two-on-one interview may save time and scheduling and provide
authentic interaction between two test takers, although it pose a risk of one test taker
domination the other.
Hughes (1989:105) proposes 11 rules to conduct an interview:
- Make the oral test as long as feasible
- Include as wide a sample of specified content as is possible in the time
available
- Plan the test carefully
- Give the candidate as many ‘fresh start’ as possible
- Select interviewers carefully and train them
- Use a second tester
- Set only a tasks and topics that would be expected to cause candidates
no difficulty in their own language
- Carry out the interview in a quiet room with good acoustics
- Put candidates at their ease
- Collect enough relevant information
- Do not talk to much (the interviewer)
In addition to Hughes’ proposal, Canale (1984) proposes four main steps to
follow to conduct, in this case, an oral proficiency test.
2. Warm Up : small talk about identity, origin and the like
3. Level-Check :wh-questions, narrative without interruption, read a
passage aloud, tells how to make or do something, a brief guided role-
play
4. Probe :field-related questions
5. Wind-down : easier questions pertaining test taker’s feeling about the
interview
The challenge with an interview is how the open-ended response is scored.
Creating a consistent, workable scoring system to ensure reliability has been one of
the major challenge in designing an interview as means to assess speaking (Brown,
2004:171). There are at least two solution to this problem: one is using an analytical
scoring rubric and the other is a holistic one. Rescoring the performance later from the
tape can be an alternative, too (O’Malley, 1996:79).
 - Drama-like Task
O’ Malley (1996:85) divides drama-like task into three sub-types: improvisations,
role play and simulation. The difference of each is respectively the preparation and
scripting. Improvisation give very little opportunity for test taker to prepare the
situation and may incite creativity in using the language. Role play provides slightly
longer time to and test taker can prepare what to say although scripting is highly
unlikely. Meanwhile, simulation (including debate) requires planning and decision
making. Simulation may involve real-world sociodrama which is the pinnacle of
speaking competence.
Like interview, drama-like task may evoke unpredictable response. Similar care
used to tackle interview may be useful for this type of task as well.

- Discussions and Conversations

Discussions and Conversations (Brown, 2004: 175) provide somewhat similar
difficulties in terms of predictability of the response hence consistency of the scoring
to that of interview and drama-like tasks. Test makers seem to choose this type of task
as informal assessment to elicit and observe test taker’s performance in:
• starting, maintaining and ending a topic
• getting attention, interrupting and controlling
• clarifying, questioning and paraphrasing
• signaling for comprehension (e.g nodding)
• using appropriate intonation patterns
• using kinesics, eye contact and body language
• being polite, being formal and other sociolinguistic situation

- Games
It is nearly impossible to list all games, but virtually all games that can elicit
spoken language objectively can be used as informal assessment for speaking. Brown
(2004:176) warns us that using games may go beyond assessment and adds that a
certain perspective need to be maintained in order to keep it in line with assessment
principles.
Some examples of games which Brown (2004:175-176) mentions (tinkertoy,
crossword puzzle, information gap, predetermined direction map) can all fall in the
umbrella of information-gap activities by O’Malley (1996:81)’s
 standpoint as he explains that an information gap is an activity where one student
is provided information that another (e.g his pair) does not know but need to. An
information gap activity involves collecting complete information to restructure a
building, sequence a picture into order or simply find the differences between two
pictures. To score an information gap activity, O’Malley (1996:83) suggest test maker
to consider the speaker’s “accuracy and clarity of the description as well as on the
reconstruction.”
5 Extensive (monologue)
The following are monologues which take longer stretch of the language and
requires extensive (multi-skills) preparations. The terms are self-explanatory and
some may actually possess some characteristics with some types previously
explained only with longer and broader scope of language use.
5.1. Speech (Oral Presentation or oral report)
It is commonly practiced to present a report, paper or design in school setting. An
oral presentation can be used to assess a speaking skill holistically or analytically.
However, it is best used for intermediate or advanced level of English focusing on
content and delivery (Brown, 2004:179).
5.2. Picture-cued Story Telling
Similar to the limited version, at this level the main consideration of using a
picture or a series of pictures is to make it into a stimulus for longer story or
description; a six-picture sequence with enough details in the settings and character
will be sufficient to test, among others, vocabulary, time relatives, past tense irregular
verbs and even fluency in general (Brown, 2004:181)
5.3. Retelling a Story, News Event
Different from paraphrasing, retelling a story takes longer stretch of discourse
with different, preferably narrative, genre. The focus is usually on meaningfulness of
the relationship of events within the story, fluency and interaction to audience
(Brown, 2004:182)
5.4. Translation (Extended Prose)
In this type of task, a longer text preferably in written form which is presented in
he test taker’s native language is to be studied prior to interpreting the text with ease
in the actual testing. The text can cover a dialogue, procedure, complex directions,
synopsis or a play script. Caution should be made concerning with this type of task
because this particular type
 requires a skill not intended for every speaker of a language. Therefore, if this type
is to be used a degree of confidence should be made sure (as in the case whether the
test takers are in pursuit of a bachelor degree!) (Brown, 2004:182).

• Scoring Rubric
An effective assessment should follow this rule (Brown, 2004:179):
1. Specific criteria
2. Appropriate task
3. Elicitation of optimal output
4. Practical and reliable scoring procedures
Scoring remains the major challenge in assessment. There are at least two types of
known scoring rubric for speaking: (1) holistic and (2) analytical. A holistic rubric
range, for example, from 1 to 6 each reflecting unique capacity of the speaker with 6
being normally native-like traits and 1 a total misuse of language which incite
misunderstanding. An analytical rubric, on the other hand, scores performance in
different subcategories such as grammar, vocabulary, comprehension, fluency,
pronunciation and task completion. There are two common practice regarding the
latter:
4. the total score is summed in average to reflect an overall score or (2) each
categories is given a different weight sometimes without the necessity to sum up the
total score.
O’Malley (1996:65) suggests several steps in developing rubric:
4.1. Set criteria of task success
4.2. Set dimensions of language to be assessed (grammar, vocabulary,
fluency, pronunciation .etc)
4.3. Give appropriate weight to each dimension (if omission is possible, do)
4.4. Focus of what test taker can do, instead of what they cannot.
Which rubric is better? Whichever is used, if high accuracy is the goal, multiple
scoring is required (Hughes, 1989:97) Since test taker’s speech can now be recorded
for second-time scoring by different rater, a balance between holistic and analytical
rubric (i.e use two types of rubric for the same task whenever possible) is
recommended (O’Malley, 1996:66).
3. CONCLUSION
The key of assessing speaking skill is understanding the continuum of (1)
spoken language, (2) task types and (3) scoring rubric. This non-rigid separation between
one level of competence and another requires time and effort in specifying the criteria of
speaking, task to elicit particular behavior and in developing practical yet representative
scoring rubric. The variety of task types will help test maker to decide which one is
appropriate for the wide array of the continuum of this particular skill.

10
 CHAPTER 9 SUMMARY: ASSESSING WRITING
The assessment of writing is not an easy task. Before assessing writing, we have to
make clear our objective or criterion. In this case we have to decide what we want to test. Is it
handwriting ability? Correct spelling? Writing sentences that are grammatically correct?
Paragraph construction? or Logical development of a main idea? These tests have different
objectives and each objective can be assessed through a variety of tasks.

Before looking at specific tasks, we have to scrutinize the different of written language,
types of writing, and micro and macro skills of writing. Therefore, this writing are going to
discuss how to assess writing based on the different of written language, types of writing, and
micro and macro skills of writing. Also the writer will provide some examples of writing tests
and how to score the test.

1. Genres of Written Language

Generally, there are three kinds of genres of written language (Brown, 2004: 219). They
are academic writing, job-related writing and personal writing. The following table will
show to us the examples for each genres of written language.

Genres of
No. Examples
Writing
1. paper and general subject reports
2. essays, compositions
Academic 3. academically focused journals
1.
Writing 4. short-answer test responses
5. technical report (e.g., lab report)
6. theses and dissertations
1. messages (e.g., phone messages)
2. letters/emails
3. memos (e.g., interoffice)
Job-related 4. reports (e.g., job evaluation, project
2.
Writing reports
5. schedules, label, and signs
6. advertisement announcement
7. manuals
1. letters, emails, greeting card, and
invitations
2. messages and note
3. calendar entries, shopping lists, and
Personal reminders
3.
Writing 4. financial documents
5. forms, questioners, medical reports,
6. diaries, and personal journals
7. fiction (e.g., short stories and
poetry)

2. Types of Writing Performance

Brown (2004: 220) classifies types of writing performance into four classifications.
They are:

11
 a. imitative : This category includes the ability to spell correctly and to perceive
phoneme-grapheme correspondences in the English spelling system. At
this stage, form is the primary if not exclusive focus, while context and
meaning are of secondary concern.
b. intensive : As one may think, Intensive writing as described here has nothing to do
with writing intensively, but controlled. Under this definition, students are
supposed to copy sentences and words, rewrite texts and passages, order
sentences among others.
c. responsive : It requires learners to perform at a limited discourse level, connecting
sentences into a paragraph and creating a logically connected sequence of
two or three paragraphs. The writer has already mastered the fundamentals
of sentence-level grammar and is more focused on the discourse
conventions that will achieve the objectives of the written text.
d. extensive : It implies successful management of all the processes and strategies of
writing for all purposes. Writers work focusing on the achievement of a
purpose. Organizing ideas logically, using details to support or illustrate it
and demonstrating syntactic and lexical variety.

3. Micro- and Macro-skills of Writing

The elements of micro- and macro-skills of writing are:
a. Micro-skills of writing
1) produce writing at an efficient rate of speed to suit the purpose
2) produce an acceptable core of words and use appropriate word order patterns
3) use acceptable grammatical systems (e.g., tense, agreement, pluralization), patterns,
and rules
4) express a particular meaning in different grammatical forms
5) use cohesive devices in written discourse.

b. Macro-skills of writing
1)  use the rhetorical forms and conventions of written discourse
2) appropriately accomplish the communicative functions of written texts according to
form and purpose
3) distinguish between literal and implied meanings when writing
4) convey links and connections between events, and communicate such relation as
main idea, supporting idea, new information, given information, generalization, and
exemplification.
5) correctly convey culturally specific references in the context of the written text.
6) Develop and use a battery of writing strategies, such as accurately assessing the
audience’s interpretation, using prewriting devices, writing fluency in the first
drafts, using paraphrases and synonyms, soliciting peer and instructor feedback, and
using feedback for revising and editing.

The taxonomy of micro- and macro-skills will assist us to in defining the ultimate
criterion of an assessment procedure. Micro-skills of writing apply more appropriately to
imitative and intensive types of writing task, while the macro-skills of writing are essential
for the successful mastery of responsive and extensive writing.

4. Designing Assessment Tasks

12
a. Imitative Writing
1) Tasks in [Hand] Writing Letters, Words, and Punctuation
It aims to increase the use of personal and laptop computers and handheld
instruments for creating written symbols. Handwriting has the potential of becoming
a lost art as even very young children are more and more likely to use keyboard to
produce writing. Handwriting remains a skill of paramount importance within the
larger domain of language assessment.

a) Copying
There is nothing innovative or modern about directing a test-taker to copy letters
or words. For example:
Handwriting letters, words, and punctuation marks
The test-taker reads: Copy the following words in the
spaces given:
car cat bat go

sin did sit pin

b) Listening cloze selection tasks

The test sheet provides a list of missing words from which the test-taker must
select. For example:

Test-takers hear:
Write the missing word in each blank. Below the story is a list
of words to choose from.

Have you ever visited San Francisco? It is a very nice city. It is

cool in the summer and warm in the winter. I like the cable cars
and bridges.

Test-taker see:
Have ever visited San Francisco? It a very nice . It is
in summer and in the winter. I the cable cars
bridges.

is you cool city like and warm the

13
 c) Picture-cued tasks
The test-taker will be displayed a poster or picture such as car, cat, watermelon,
or pen, and they are asked to write the word that picture represents. For example:

1. 2. 3.

d) Form completion tasks

A variation on pictures is the use of a simple form (registration, application, etc.)
that asks for name, address, phone number, and other data. For example:

The test-taker will be asked to fill out this application form based on their data.

e) Converting numbers and abbreviation to words.

In this test, the test-takers are asked to write down abbreviation such as numbers,
hours of the day, dates, or schedules into words. This task can serve as a
reasonably reliable method to stimulate handwritten English. For example:

The test-takers hear: Fill in the blanks with words:

Test-takers see:
9:00 : 3:35 :
11/11/1985 : 11.04 :

2) Spelling Tasks and Detecting Phoneme – Grapheme Correspondences

A number of task types are in popular use to assess the ability to spell words
correctly and to process phoneme-grapheme correspondences.
a) Spelling tests
In a traditionally, old-fashioned spelling test, the teacher dictates a simple list of
words, one word at a time followed by the word in a sentence, repeated again,
with a pause for test-takers to write the word. Scoring emphasizes correct
spelling.

b) Picture-cued tasks

14
 Pictures are displayed with the objective of focusing on familiar words whose
spelling may be unpredictable. Items are chosen according to the objectives of
the assessment, but this format is an opportunity to present some challenging
words and words pairs: boot/book, read/reed, bit/bite, etc.

c) Multiple-choice techniques
Presenting words and phrases in the form of a multiple choice task risks crossing
over into the domain of assessing reading, but the items have a follow-up writing
component. For examples:

Test-takers read:
Choose the words with the correct spelling to fit the
sentence, then write the word in the space provided.
1. He washed his hands with
a. soap c. sop
b. sope d. soup
2. I tried to stop the car, but the din’t work.
a. braicks c. brakes
b. brecks d. bracks

d) Matching phonetic symbols

If the test-takers have been familiar with the phonetic alphabet, they could be
shown phonetic symbols and asked to write the correctly spelled word
alphabetically. For example:

b. Intensive Writing
1) Dictation and Dicto-Comp
a) Dictation
Dictation is an assessment of the integration of listening and writing. It is simply
the retention in writing of what one hears aurally, so it could be classified as an
imitative type of writing, especially since a proportion of the test-taker’s
performance centers on correct spelling.

b) Dicto-Comp
This kind of tests, the teacher will read a paragraph at normal speed, usually
twice or three times; then the teacher asks the students to rewrite what they have
been read by their teacher.

2) Grammatical transformation tasks (practical, reliable)

15
 Even though it is an old technique in assessing writing, but it has positive side;
grammatical transformation tasks are easy to administer and are therefore practical,
quite high in score reliability, and arguably tap into knowledge of grammatical
forms that will be performed through writing. Numerous versions of the task are
possible:
 Change the tenses in a paragraph.
 Change statements to yes/no or wh-questions.
 Change question into statements.
 Combine two sentences into one using a relative pronoun.
 Change from active to passive voice.

3) Picture-Cued tasks
The main advantage in this technique is in detaching the almost ubiquitous reading
and writing connection and offering instead a nonverbal means to stimulate written
responses. Numerous types of this task are:
a) Short sentences
A drawing of some simple action is shown: the test-taker writes a brief sentence.
For example:

Test-takers see the following pictures

Test-takers read: What is the man doing?

What is the boy doing?
What is the baby doing?
What is the man doing?
Test-takers write:
1. …………………………………………………………………………
2. ………………………………………...................................
3. ………………………………………....................................
4. …………………………………………………………………………

b) Picture description

16
 In this test, the teacher will show a picture to the students and then ask them to
describe the picture. For example: based on the presented picture, the test-takers
are asked to describe the picture using four of the following presentations: on.
over, under, next to, and around.

Test-takers see the following pictures

Describe the picture using four of the following

presentations: on. over, under, next to, and around!

c) Picture sequence description

A sequence of three to six pictures depicting a story line can provide a suitable
stimulus for written production. The picture must be simple and unambiguous
because an open-ended task at the selective level would give test takers too
many options. For example:

Test-takers see:

Test-takers read:

4) Vocabulary Assessment Tasks

Most vocabulary study is carried out through reading. A number of assessments of
reading recognition of vocabulary are multiple-choice technique, matching, picture-
cued identification, cloze techniques, guessing the meaning of a word in context.
The major techniques used to assess vocabulary are defining and using a word in a
sentence. Here is the example:

17
 Test-takers read:
1. Write good sentence using the following
words!
A. Communicate
…………………………………………
…………
B. Utilize

5) Ordering Tasks
In this kind of test, the teacher gives scrambled sets of words to the students and
then asks the students to reorder the scrambled sets of the words into a correct
sentence. Here are the examples:
Test-takers read:
Put the words below into the correct order to make a
sentence:
1. beautiful / tonight / my / to / the / come / will / home /
girl /
2. goes / mother / My / everyday / market / the / to /
3. a / They / song / do / sing / to / not / want /

Test- takers write:

1. …………………………………………………………
………………………………………

6) Short-answer and Sentence Completion Tasks

In this case, the test-takers are asked to write short-answer and complete sentences.
For examples:
Test-takers see:
1. Sukirman : ………………………………………….?
Agus : I am studying English.
2. You are in the kitchen helping your roommate cook.
You need to ask questions about quantities. Ask a
question using how much (2a) and a question using
how many (2b), using nouns like sugar, pounds, flour,
onions eggs, and cups.
2a. :
……………………………………………………………

c. Responsive and Extensive Writing

1) Paraphrasing
The initial step in teaching paraphrasing is to ensure that learners understand the
importance of paraphrasing: to say something in one’s own words, to avoid
plagiarizing, to offer some variety in expression.

18
2) Guided Question and Answer
Another lower-order task in this type of writing is a guided question and answer
format in which the test administrator poses a series of question that essentially
serve as an outline of the emergent written test. Here is the example of this kind of
test:

Guided writing stimuli

1. Where did this story take place? (setting)
2. Who are the people in the story? (characters)
3. What happened first? and then? and then?
4. Why did ………………… do
…………………….? (reasons, causes)
5. What did…………………do……………think
about ……? (opinion)
6. What happened at the end? (climax?

3) Paragraph Construction Tasks

Assessment of paragraph development takes on a number of different forms:
a) Topic sentence writing
This tasks thereof consists of:
- specifying the writing of a topic sentence
- scoring points for its presence or absence, and
- scoring and/or commenting on its effectiveness is stating the
topic.
b) Topic development within a paragraph
Four criteria are commonly applied to assess the quality of a paragraph:
- the clarity of expression of ideas
- the logic of the sequence and connections
- the cohesiveness or unity of the paragraph
- the overall effectiveness or impact of the paragraph as a
whole.
c) Development of main and supporting ideas across paragraphs
The elements which can be considered in evaluating this kind of task are:
- addressing the topic, main idea, or principal purpose
- organizing and developing supporting ideas
- using appropriate details to undergrid supporting ideas
- showing facility and fluency in the use of language
- demonstrating syntactic variety,
4) Strategic Option
Developing main and supporting ideas is the goal of the writer attempting to create
an effective text. A number of strategies are commonly taught to second language
writer to accomplish their purposes. Aside from strategies of free-writing, outlining,
drafting, and revising, writers need to be aware of the task that has been demanded
and to focus on the genre of writing and the expectations of the genre.

a) Attending to task
- In responsive writing, the context is seldom completely open-ended: a task
has been defined by the teacher or test administrator, and the writer must

19
 fulfill the criterion of the task. Even in extensive writing of longer texts, a
set of directives has been stated by the teacher or is implied by the
conventions of the genre.
- Four types of tasks are commonly addressed in academic writing courses: 1)
compare/contrast, 2) problem/solution, 3) pros/cons, and 4) cause/effect.
Depending on the genre of the text, one or more of these task types will be
needed to achieve the writer’s purpose.
- Assessment of the fulfillment of such tasks could be formative and informal,
but the product might also be assigned a holistic or analytic score.

b) Attending to genre
Assessment of the more common genres may include the following criteria,
along with chosen factors from the list in item of main and supporting ideas.
- Reports
- Summaries of Readings/Lecture/Videos
- Responses to Readings/ Lecture/Videos
- Narration, Descriptive, Persuasion/Argument, and Exposition
- Interpreting Statistical, Graphic, or Tabular Data
- Library Research Report

5. Scoring Methods for Responsive and Extensive Writing

Writing can be assessed in different modes, for example analytic scoring, holistic scoring,
and primary trait scoring. If evaluating the same piece of writing, each mode of scoring
should result in similar "scores," but each focuses on a different facet of L2 writing.

a. Holistic Scoring
Holistic scoring results in a more general description for categories, but includes the
different elements of writing implicitly or explicitly. The result is usually a global
grade, such as A, B, C, D, E (see appendix 1)

The advantages and disadvantages of holistic scoring

1) Advantages
- Fast evaluation
- Relatively high inter-rater reliability
- Applicability to writing across many different disciplines
- The fact that scores tend to emphasize the writer’s strengths (Cohen, 1994: 315)
2) Disadvantages
- One score masks differences across the sub-skill within each score
- No diagnostic information is available (no washback potential)
- The scale may not apply equally well to all genres of writing

b. Primary Trait Scoring

Primary Trait Scoring is used when the teachers focus on a particular component of the
writing but may include a secondary trait which may receive less weight. For example,
students may be asked to demonstrate the ability to use a variety of words to describe,
they may be asked to show mastery of subject-verb agreement, or they may be rated on
their ability to organize their thoughts. With this kind of scoring, only the identified
features are assessed. The other errors are ignored.

20
 The advantage and disadvantage of primary trait scoring
1) Advantages
The main advantage of primary trait scoring is that it focuses the students on one
particular aspect of the oral or written task.
2) Disadvantages
The advantage cited above can also be a disadvantage in that it ignores the other
elements of speaking and writing that are important to the speaking and composing
processes. Primary trait rubrics may be especially useful for early drafts or writing-
to-learn activities.

c. Analytic Scoring
In this mode, students' writing is evaluated based on detailed grades for elements of
writing such as vocabulary, grammar, composition, or mechanics. Results are based on
multiple sub-grades (e.g., 4 out of 5 on vocabulary, plus 3 out of 5 on grammar plus 4
out of 5 on content (see appendix 2)

The advantages and disadvantages of analytic scoring

1) Advantages
- The instructor can give different weights to different dimensions. This allows the
instructor to give more credit for dimensions that are more important to the
overall success of the communication task. For example, in a writing rubric, the
dimension of content might have a total point range of 30, whereas the range for
mechanics might be only 10.
- They provide more information to students about the strengths and weaknesses
of various aspects of their language performance.
2) Disadvantages
- It takes more time than holistic scoring.
- Concentration on the different aspects may divert attention from the overall
effect of the piece of writing. Inasmuch as the whole is often greater than the
sum of its parts, a composite score may be very reliable but not valid (Hughes
1989: 93-94)

21