The document discusses various aspects of evaluating educational programs and materials. It describes two major approaches to evaluation - objective model and responsive evaluation. It also outlines different purposes, focuses, scopes, types of information, and roles of evaluators. Timing of evaluation is discussed as formative or summative. Steps in conducting an evaluation of a task and planning are presented. The document also discusses evaluating teachers' needs and wants from course books.
2. Dimension of the Evaluation
Approach: Objective model and responsive
evaluation are two major approaches.
The objective model belongs to the
psychological tradition of educational
research.
Here, the achievement can be determined
and the learners capacity and learning
outcomes can be measured.
3. Responsive Evaluation…
It belongs more to the sociological
tradition of educational research and its
ideographic in approach.
Its aim to illuminate the complex nature of
the teaching learning process.
4. Purpose…
Two major purposes... …accountability
and development.
The purpose of the development is to
determine whether the destined
destinations of the program have been
met.
The purpose of the development may be
either to improve the curriculum or to
foster teacher teacher-development or
both.
5. Focus… …
Effectiveness and Efficacy.
The programme is effective or not for the
needs of the learners.
Does the programme fulfill the needs of the
learners more effectively than some other
alternative program?
The evaluator needs to compare with what
the learners knew and were able to do
before starting the program and with what
they know and able to do at the conclusion
of the program.
6. Scope…
The scope of the evaluation can be
internal or external.
It uncritically accept the goals of the
program .
It submits the goals of the programme
properly.
7. Evaluators…
It is a matter of consideration that who is
the evaluators of the programme.
It should be restricted from the outside so
that both inside and outside evaluators
can evaluate the programme properly.
8. The Timing… …
A formative evaluation is takes place as the
programme is being developed and
taught.
A summative evaluation takes place at the
end of the programme.
Relevant to micro and macro evaluation.
9. Types of Information….
An evaluation based on objectives model
Approach traditionally relies on one type of
information: a)learners’ test scores.
A responsive evaluation will require a
greater variety of information including
a)Documentary information
b)Information collected by means of self-
report
c)Information collected from classroom
observation.
10. Conducting an evaluation of
a task…
Description of a task: The way we
evaluate a task depends upon a precise
description of the task given.
To achieve this, content of the task should
be specific as follows
Input
Procedure
Language proficiency
Outcome
11. Planning the evaluation…
It demands the prior consideration
because without prior planning it may
prove impossible to achieve a systematic
and principled evaluation.
12. Collecting information
A task evaluation will need to consider
three types of information.
1.information about how the task was
performed
2.Information about what learning took
place
3.Teacher’s and learner’s opinion about
the work.
13. Analysis of the information
collected….
Qualitative analysis and quantitative
analysis.
While the former involves the use of
umber to determine rating numerically,
the later is a more holistic and
impressionistic which evaluate tasks in the
light of representative responses.
15. “Teachers’ needs and wants
from course books”
Needs analysis has featured prominently in
the literature of language teaching in the last
20 years but most of what has been written
has focused on learner’s communicative
needs.
Teachers’ needs are treated as a part of
situation analysis. In literature, needs are
defined in 3 terms.
They are: a) ownership b) kinds c) sources.
16. When teachers are asked
what their needs from a
course book
1. Teachers’ perception of administrative
needs. e.g. The school is under-resourced
and a very strict syllabus is set which the
teachers’ are expected to obey.
2. measured learners’ needs’:
e.g. The teacher has administered a
diagnostic test at the beginning of the
course and is aware of the learners’
communicative needs.
17. 3. Teachers’ perception of learners needs: The
teacher believes that Japanese students are
quiet shy and thus require special training in
speaking.
4. Teachers’ wants: Even though ELT experts
recommend a learner-centered approach
these days and the other colleagues of the
language centre follow the trend, the
teacher prefers and also secretly believes in
the value of a teacher-centred approach for
certain learners.
18. Teachers’ needs would consist of two
general areas: one deriving from personal
traits such as their age, sex, cultural and
educational background the other from
their professional traits such as areas and
levels of expertise, length and types of
teaching experience.
19. a) Self-perceived needs – the needs which are
reported y the teacher. These are what teachers
themselves can articulate.
b) Needs perceived by others – the needs of the
teachers which they are not aware of and thus
cannot articulate themselves and which are
identified by others (e.g. colleagues, teacher-
trainers, researchers) in response to qualitative
data(e.g. observation of the teachers’ teaching,
analyzing the tendency in interview and
questionnaire responses of the teacher).
20. c) Objectively measured needs – the
needs which are identified in objective
studies in which quantified data is
collected , analysed and interpreted by
a third party who tries to be detached,
unbiased and accurate.
21. Teachers’ wants: In many cases, what
may be identified by the teachers
themselves and by a third party as their
‘needs’ could be their ‘wants’ as well.
22. The study of teachers’ wants may reveal
that teachers’ commitment and
involvement due to their preference for
materials and methods are keys to
effective language learning.
23. Teachers - an endangered
species
Exploring teachers’ needs and wants is crucial
when the role division between the materials
producers (e.g. professional materials writers
and publishers) and the users (e.g. teachers,
educational administrators and learners)
seems to be becoming more and more
evident. Remarkable technical advancement
has brought sophistication and a great
proliferation of ESL/EFL coursebooks but it has
also created a wider role division between
materials producers and materials users.
24. The division between the producers and
users has also affected the coherent
linear sequence of curriculum
development/course design processes to
the level that concerns are being
expressed that the materials could carry
the threat of deskilling teachers by
reducing the teachers’ role.
25. Various different models of the
process of course design (e.g.
Johnson 1989; Dubin and
Olshtain 1986; Richards 1990)
Firstly, the teachers and administrators
draw up a very general framework of a
particular class and learners. In this
framework, the characteristics of the
learners are defined in terms of the
learners’ preference for a course and the
levels of their proficiency based on the
tests administered at the beginning of the
course. The goal of teaching is usually
represented in the name of the course.
26. Materials selection holds an assertive position
in the second stage in the sequence; the
teachers and administrators select from
commercially available coursebooks the one
suitable for the class defined in the initial
stage. In fact producers provide prospective
selectors (e.g. administrators and teachers)
with information as to the target learners,
objectives and methodology in the blurbs or
in the introductions of the books. They may
also provide a syllabus map indicating ho
units are integrated into a coherent course.
27. The materials writer and the publisher who
produced the materials have more direct
control of the course design processes
than the teachers. The degree of
dominance depends on how closely the
teachers choose to follow the coursebook
or hoe much teachers take initiative in
making flexible use of the materials.
28. Empowerment of teachers
Four areas are recognized in which teachers
needs and wants could be reflected in the
development and use of materials.
Quality assessment of published course books:
a) The needs for objective measurement of the
quality of published course book. According to
Seldon several ways of realizing these including:
˃ Desirability and introducing a Which? Magazine
for ELT course books.
˃ Improvement and innovation in ELTJ reviews in
order to enhance their validity, usefulness and
availability.
29. b) Stricter and more systematic materials selection
procedures.
Many checklists are now available which are designed to
enable systematic selection of materials. It would be useful
to have a survey review comparing these checklists.
C) Establishing methods and feedback routes of user’s
evaluation. Three kinds of evaluation according to Rea-
Dickins are:
˃ Pre - use evaluation which can be done prior to use of a
course book.
˃ in - use evaluation
˃ Post use evaluation, measured in terms of learners
performance.
30. d) Establishing system for teachers need and
wants to be reflected in the production process.
e) Wider perspectives in teacher development.
It includes teaching development components of
materials evaluation, adaption and writing. It also
includes of researches methodology training in
post graduate courses in TEFL/TESL and applied
linguistics. Besides, institutions could make more
use of staff meetings to provide opportunities for
teacher’s development.
32. Opportunities for Change…
Tomlinson stresses that evaluators need to
develop their own principled criteria which
take into consideration the context of the
evaluation and their own beliefs. He claims
that evaluation criteria should be developed
before materials are produced, and used to
make decisions about the approach,
procedures and activities to be adopted as
well as to monitor their development and
subsequent use.
33. Tomlinson differentiates between universal and local
criteria, the former being those that can be used to
evaluate materials for any learner anywhere. To generate
these criteria he advises evaluators to brainstorm a list of
principled beliefs that they hold about how languages are
most effectively acquired and then convert these beliefs
into criteria for evaluating materials, such as ‘Are the
materials likely to achieve affective engagement?’
34. Tomlinson recognizes that evaluation is
inevitably subjective, that it ‘focuses on the
users of the materials’ and attempts to
measure the potential or actual effects of the
materials on their users. In contrast, analysis
focuses on the materials and aims to identify
what they contain, what they ask learners to
do and what they say they are trying to
achieve, aiming to provide an objective
account of the materials, though the
selection of questions is inevitably subjective
and there is often a hidden agenda which it is
hoped the revealed data will support.
35. we need textbooks to save time and
money and many teachers want a
coursebook which provides everything
they need in one source.