Unit 6 Test Design II

Have you ever used or designed a rubric to test your students´ speaking skills? IN this unit, you will
develop the skills needed to successfully design rubrics to assess communicative skills. Enjoy
the unit!
Competencies:
By the end of this Unit, students will be able to:
 Design a range of alternative assessment procedures and assess their
effectiveness in assessing students’ performance.
 Define comprehensible performance criteria and/or rubrics that clearly
lay out the specific expectations for particular tests sections or tasks;
and
 Use performance criteria and rubrics to provide learners with timely and
detailed feedback

Content of the Unit:

1. Introduction
2. Rubrics and performance criteria
3. Testing communication skills
4. Summary and Conclusions
5. Assessment Plan

Estimated Time

 10 hours.

Assessment Plan

In order to successfully complete your work on this Unit you will be

asked to do two assessment tasks:

Unit 6 Final Task; this task represents 5 points of the overall grade.
You will submit this task through the UAS platform.

Unit 6 Discussion FORUM, this forum represents 5 points of the

overall grade. You will complete this forum on the UAS Platform.
1. Introduction

In this unit we will discuss issues related to the use of rubrics and
performance criteria to test communication skills, specifically, how to
assess the speaking skill.

Read the following questions as they will help you get ready for this unit:
 What is alternative assessment?
 Have I used alternative assessment?
 What is a rubric?
 Have I designed and use a rubric?
 Are there different types of rubrics?
 What are the elements to consider in designing a rubric?

By the end of this unit, you will be able to answer the questions above.

2. Rubrics and Performance Criteria

Alternative assessment or authentic assessment has gained popularity in

the field of second language teaching in the past few years. As we
discussed in previous units, “the main purpose of alternative assessment is:

“To assess students beyond the traditional testing and encouraging

students to participate in open about standards and criteria of successful
performance” (Brown & Abeywickrama, 2010).

García and Pearson (1994, p. 357) stated that goal of alternative

assessment is to gather evidence about how students are approaching,
processing, and completing real-life tasks in a particular domain”.
According to Grabin (2007), tasks and activities given to students in
alternative assessment should have specific criteria (whether in the form of
a rubric, a rating scale, or a checklist) to guide students in the development
of their products and performances.
Authors have stated the importance of using rubrics to assess students´
performance more effectively, but what is a Rubric?

Task 1)
Step 1) Answer the following question: what is a Rubric?
You are free to do some research in order to provide an answer

You probably found that generally, rubric is described as a set of

criteria/rules or guidelines that can be used to assess one´s work or
success of a tasks (Berger, 2011).

Berger (2011), Brookhart (2013), Walvoord (2010), Orliech, (et. Al, 2010)
distinguish two primary components of a rubric: criteria and standards (also
known as scoring scales).
- Criteria: describes what is evaluated
- Standards: describe the level of achievement and task
involvement in teaching that level.

Therefore, the major purpose in using rubrics is to assess performance. In

Assessing students´ performance the examiner can whether observer the
process of doing something (for example, oral communication or
presentation) or observe the product that is the result of students` work (for
example, an essay or a project) (Brookhart, 2013).

Task 2)
Step 1) Answer the following question: What type of rubrics do you know?
Rubrics can be designed in different shapes and sizes, but generally, there
are four main types of assessment rubrics, according to Berger (2011):

 Checklist: is a simple list of items designed for both assessor and

student to check if every item is presented.
 Rating Scale: is very similar to checklist, because it consists of a
list of items, but in addition to list, it has a scale, ranging from
Strongly Agree to Strongly Disagree.
 Analytic/descriptive rubric: it works on each criterion separately,
provides scales for a list of components and a description for each
rating. This type of rubric is used to assess important tasks where
each component needs to be evaluated separately.
This is an example of an incomplete analytic rubric that you can try to fill
out**

This is an example of an analytic rubric for composition tasks:

  Holistic rubric: is the one that is used to assess a project or
product as a whole. It describes the performance by applying all
the criteria at the same time and making possible an overall
judgement about the quality of the work.

This is an example of a holistic oral presentation rubric for class

presentation
Task 3)
Step 1) Answer the following question: Can you identify advantages in
using rubrics as part of your assessment? Any disadvantages?

Some advantages of using assessment rubrics are:

 It provides teachers and learners with information about:
o What will be assessed
o Standards that need to be met
o Where students are in relation to goals.
 It increases consistency / objectivity in the way the teacher rates
students` performance.
 It helps teachers to give feedback to their students.

Some disadvantages are:

 It requires practice to apply it well.
 You can not use the same rubric for all your tasks, you have to
design one according to the task.
 Sometimes rubrics are too complex and difficult for students to
understand.
 It is time consuming

Just like in everything, there are positive attitudes and negative attitudes to
the use of rubrics. However, we would like to highlight the fact that rubrics
are useful for both teacher and students in using them as an assessment
tool, as well as usage of assessment rubrics as instructional tools
(Walvoord, 2010).
You need to consider that if an assessment rubric is used, it is essential to
share them with students before they begin their tasks or test, so they can
use the rubric as a guide to a better performance. Using rubrics in teaching
and learning has the potential to positively affect students` attribution of
success or failure, removing students` “mystery” of how assessment is
evaluated, and it provides clear directions for performance that are within
the control of the student.

Task 4)
Step 1) Answer the following question: Have you ever designed a rubric?
What do you think should be included? What steps did you follow to create
it?

Berger (2011), suggests starting from looking for existing rubrics (you can
do it for your end of unit task) in any web search engine and find some
ready-made rubrics with ease. If you analyze rubrics made by other
teachers there is an opportunity for you to find one that is mostly suitable
for your learners` needs, learning task and program; however, you can
always design your own.

In order to design a rubric, or find the appropriate rubric for your course,
there is a set of questions that you need to answer.

 What is the purpose of assessment? It is very crucial to determine

the purpose for the assessment. For example, if students are
assessed to see if they have enough knowledge to pass to the
next level rubrics will be different from the assessment performed to
identify instructional needs of individual group of students.

 What are the objectives of the assessment? It is necessary to

determine what students are expected to know or be able to do to
assess them effectively. Assessors must decide if they want to
concentrate on one aspect (for example, speaking), or they want
to assess several aspects (for example, listening and speaking) at
the same time.

 What is the most appropriate task for assessment? The task is

what students actually perform to show their knowledge and
abilities. It would be more beneficial and fairer if the selected task
can be accomplished at many levels (full range of students in the
classroom).

 What is the procedure of assessment? Students feel themselves

more confident if they are familiar with the format/procedure of the
assessment, so anxiety of the assessment will be reduced, and
the result will be more objective. In addition, this provides face
validity to the task.

 What are the most appropriate scoring criteria for assessing

students’ performance? Criteria must be designed according to
the assessment objectives. It helps teachers/students to
understand the requirements and gives them opportunity to
develop necessary strategy to reach the goals (Brookhart, 2013).

 What are the most appropriate grading scales for the scoring
criteria? Each criterion may be given different weights. The
detailed description of the scales helps assessor to grade the
students’ performance objectively on one hand, and on the other
hand, gives students opportunity to predict the total score and
later, based on information from the rubric, have clear understanding of
their weakness and strengths.

 How to interpret the results of the assessment? The meaning of

the assessment results must be explained according to the
objectives of assessment, so it can be used by
students/parents/teachers to focus on possible improvement.
Analyzing the performance according to the rubrics, gives
opportunity to identify the weakness and strength of a
student/performance and to determine the focus of future activities
(Walvoord & Anderson, 2011).

In order to successfully design a rubric all of the aspects listed above

should be carefully consider.

3. Testing Communication Skills

Let`s explore at the use of rubrics for assessing speaking as they are
generally distinguished between holistic and analytic.

The holistic method of assessment is used to assess learner´s

performance as a whole, without judging the component parts (criteria)
separately (Mertler, 2001). This is commonly used when you as a teacher
need to judge students´ performance very quickly. This type of method is
time saving; however, it does not provide specific feedback for
improvement as it does not specific what was not truly satisfactory.

The analytical method is used to judge students´ performance first

assessing individual parts separately, then the results of individual scores
are summed to calculate a final total score. This method provides useful
information for students and teachers about areas of strength and
weakness; however, this method is time-´consuming, even if you have a
good rubric, it takes time to assess every part individually.
Speaking is the most difficult skill to assess objectively and reliably, due to
the fact that, a speaker may produce all the right sounds but not make any
sense, or have great difficulty with phonology and grammar and yet be
able to get the message across (Kitao & Kitao, 1996, p. 2). Another issue
to consider is that it is judged on the spot during face-to-face interaction or
online or voice recorded if it were the case.

Task 5)
Step 1) Answer the following question: What aspects will you include in a
rubric for assessing speaking? List them in the space below.

Knight (1992) provided a list of possible speaking assessment criteria that

shows a variety of criteria from which teachers can choose the most
appropriate ones for particular contexts (students´ needs, curriculum or
course objectives, etc.).

Task 6)
Step 1) As you read Knight (1992) list, highlight the criteria that you find
appropriate for your teaching context (your language classroom). This
might become the foundation for the rubric you will create as part of the
end of unit task.

 Grammar (range and accuracy).

Grammar is one of the most often used criteria in assessing
students' speaking, which is usually described as a set of rules by
which the language is created, or a set of rules used for
 combining sentences. Students may also be assessed for the
range of grammar they use in their oral performance, which
stands for the number of grammatical rules and how
accurately/correctly students are able to use them. Following the
grammar rules does have an impact on the comprehensibility of
the utterance, so the quality of communication largely depends on
it. On the other hand, there is a misleading concept that
knowledge of grammar rules and ability to use them in speech is
considered as mastering the language itself, however,
communication is more than simply following the grammar rules.

 Vocabulary (range and accuracy).

Vocabulary is another very popular criteria that is used to assess
students' speaking skill, which refers to a number, range, and
accuracy (choosing the right words and forms of the words
according to the particular context) of the words used in a
particular subject. Vocabulary accuracy assessment criteria,
however, can be incomplete, if it is used separately from
sociolinguistic appropriateness of words’ selection (i.e.,
variant/dialect, style, jargon, etc.).

 Pronunciation (individual sounds, stress, rhythm, intonation, and

linking/elision/assimilation).
Assessing students’ pronunciation accuracy is usually compared
against the native speaker’s natural speech production. English is
quickly becoming a language of international communication, so
more and more people from different countries use it as their first
language. That makes it very difficult to set the standards to apply.
That is why teachers should focus on only major peculiarities of
English language pronunciation (comprehensibility in the first
place).

 Fluency (speed of talking, hesitation while speaking, hesitation

before speaking).
 Fluency is one another popular criteria in assessing students' oral
performance, which is often mentioned in contrast to accuracy.
When assessment focuses on fluency, assessors judge students'
ability to speak naturally without worrying too much about being
completely accurate. The sub-criteria (speed, hesitations) should
be decoded correctly because they may vary in the natural flow of
speech as well and they may be features of the fluency (for
example, fluency is rather the ability to know how and when to
hesitate, or when to speak slowly). Moreover, fluency is not
something that stands by itself, it depends on the level of
language skills - it is hardly possible to produce meaningful
speech without appropriate vocabulary stored in their mind, and
knowledge of the grammar rules to combine them into sentences.

 Conversational skill (topic development, initiative, and

conversation maintenance).
Conversational skills criteria are described as the ability to
maintain a conversation. It assesses speakers' ability to maintain
the coherence of his own utterances and with the speech of the
interlocutors, taking turns in conversation, using pauses and
fillers, correcting oneself, asking for further information and details
to develop the topic of the conversation. In addition, the rules of
etiquette (politeness norms) may also influence the conversational
skills.

 Sociolinguistic skill (distinguishing register and style, use of

cultural references).
This criterion refers to sociolinguistic appropriateness of a
speaker, which involves ability to select the right registers and
styles, (such as formal and informal) and the balanced use of
cultural references. Assessment according to sociolinguistic skills
focuses on students' ability to evaluate situations and decide what
the right thing to say is in terms of manners, politeness,
tactfulness, etc. to be contextually appropriate. Mastering these
 skills is as important as mastering grammar, vocabulary,
pronunciation, fluency, etc. This criterion is not easy to assess,
however, its close interconnection with lexical accuracy may be
taken into consideration while observing students' sociolinguistic
skills.

 Non-verbal (eye-contact and body language).

Speaking involves not only producing the words, but also the
message expressed with the help of body (posture, gestures,
facial expressions, eye contact) or what is also called non-verbal
communication. The message delivered verbally is supported by
the nonverbal signs to provide more natural, vivid and clear
understanding of the speech.

 Content (relevance of arguments and ideas).

This criterion refers to logic and consistent arrangement of
arguments, ideas, comments, etc. to the speakers' point, leading
to better and easier understanding.

 Pragmatic competence.
Assessment of pragmatic competence aims to measure the ability
of a learner to communicate his/her intended message clearly, on
the other hand, assess the ability to interpret the received
message as it was intended by the interlocutor.

Stop and Think: Which elements did you highlight? Are they the most
appropriate for your teaching context? Design a speaking activity for your
students to perform considering the selected elements.

Let`s go to the final section of this unit.

4. Summary and Conclusion

Rubrics are often used by teachers to grade student work; however, rubric
can serve a major purpose; for instance, as an instrument to provide
students with feedback on what they did and didn´t do well. Rubrics can
provide students with specific areas to improve and combined with your
feedback and guidance, students can become better language users.

In this short unit, we defined rubrics and analyze different types of rubrics
and we ended up considering the use of rubrics to assess the speaking
skill.

5. Assessment Plan

Welcome to the end of Unit Task. Remember, this task should be uploaded
to the UAS Platform before Sunday at 10:00pm.

Instructions:

Step 1) Download the sample speaking activity from the UAS Platform.

Step2) Create a speaking activity to be used in your language classroom.

You won´t teach it.
 Plan the steps you “would use” to implement the activity. Make
sure your instructions are well-sequenced and clear.

For instance:
o T starts the class by greetings students.
o T explains the activity and gives the following instructions to
students:
 Ok guys, in this activity you will….do this, and this.
o T checks instructions (ICQs):
 What are you going to do?
 How much time?
 Bla bla bla.
o T starts the activity
  OK, are you ready? Let´s start
o T monitors students´ performance by…
o Bla bla bla.
o T marks the rubric.
o T marks the end of the speaking activity.

Step 3) Choose a type of rubric that best suits the nature of your speaking
activity.
Step 4) Develop a rationale describing why you chose that type of rubric
and the elements in the rubric. Remember, the type of rubric and elements
should be aligned to the speaking activity you designed.

Step 5) Submit in a word document:

 Cover page
 The sequence of steps you would follow to implement the activity.
 The sample speaking activity + materials.
 The rubric.
 The rationale of why you chose the rubric and the element in the
rubric.
 A conclusion including:
o What you liked about your activity.
o What you liked about the rubric.

Step 6) Submit before Sunday at 10:00pm.

Your tutor,
Heidy Paredes
Welcome to this FORUM.
Instructions:
Step 1) In this FORUM you will share the product of your end of unit task.
include the following elements:
A paragraph with information of the context / class profile where the activity
was/would be implemented. 
 For example, the following "role-play" was/would be implemented
in a class with 20 students, bla bla. 
A paragraph with an overall description of your task. 
 This is a role-play in which students will make use of vocabulary of
X, Y and Z to do some grocery shopping. 
A paragraph with an overall description of task sequence: 
 This speaking task consists of 3 key moments: 

o A pre-task in which the teacher reviewed/introduced some

vocabulary about this and this and this.
o The creation of the role-play (individually / in pairs).
o Students presentation of their product (role-play).
o T marking the rubric.
o Feedback from the teacher. 
A description of the elements you included in the rubric and why.
 I decided to use a rubric including elements such as this and this
and this. The first element addresses the following factors X, Y
and Z. The second element, bla bla. 
  You can find the complete speaking task attached to this post
along with the materials. 
Post Due date Tuesday. 
Step 2) Analyze one of your peers speaking activity, you can download the
document they attached to get a better perspective on the speaking task,
and reply with: 
 Suggestions on how to improve the quality of the speaking activity.
 Comments on how the rubric could be improved and why. 
Note: The purpose of this step is for your peers to have a better speaking
activity and rubric. Due date Thursday. 
Step 3) Analyze the replies received and offer additional information (if
requested) or thank, acknowledge, etc. the suggestions your received. Due
date Saturday.
Step 4) React to the summary, Due date Monday

Your tutor,
Heidy Paredes