UNIT 9

-- - - -- - --

Structure
9.0 Introduction
9.1 Objectives
9.2 Evaluation :And1 Pervasive Process
9.2.1 Scope and Purposes of Evaluation
9 2 2 Types of Evaluation- Formative and Surnmative I

9.23 Evaluation:An Integral Component of Educatilonal Procsss

i
9.3 Evaluation at Different Stages

F 9.3.1
9.3.2
9.3.3
Curriculum Planning Stage
Instruction/Course Design Stage
Courseware Development Stage
9.3.4 Cumculum Evaluation/Renewal Stage

9.4 MajorAreas of Evaluation

9.4.1 Courseware/ProgrammeEvaluation
9.4.2 Evaluation of Instructional Methods and Processes
9.4.3 Learner Assessment

9.5 Evaluation: SomeGeneral Concerns

,9.6 Evaluation-A Means, Not an End in Itself
I

i 9.8 Unit End Activities

i
I -9.9 References and Suggested Readings

9.10 Clues to Check Your Progress I

9.0 INTRODUCTION 1

Evaluation generally refers to the evaluation of learning outcom&sof learnersbut it

can also be done for educational programmes or any product. 1dthis unit we shall
discuss the evaluationofeducationalprogrammes that involvetecwlogy. M d o n a l
~ e c h w l o is
~ ayoung
y discipline.~ udue t to its immensepotential td facilitateteaching
and l e a m i n g k is a p w i n ginterestin this field. Electronic suped@mys7 skyways,
virtual classrooms and similar other concepts have paved the ay for the use of
$
tec)mology inclassroomsituation.Today it is the age ofthe tec ology mediated
education. Nevertheless training the learners in evaluation of such an education is a
challenge. It is in this respect that the unit would help you. Evaluatbn ofeducationa!
technology is importantbecause the teachers7W s t r a t p r s and +her stakeholders
want to know whether the investment in new technology isjustifia.
As a teacher you will have to equip yourself with knowledge and k l l s required for
evaluatingand assessingthe impact of educationaltechnology on t e a c h and learning.
In this unit, you will learn the importance of evaluation of educatibnaltechnology.
You will understand and will be able to implementthe prgcess of $valuation of the
use of technology in schools and classrooms.
 Evaluation Concepts
9.1 OBJECTIVES

I After studying this unit, you should be able to:

1
I define the concept of evaluation;
1
understand the importance of evaluation of the use of educational technology;
1
I
8 describe the process of evaluation of technology integrated curriculum,
instruction and the learningmaterial;

I discuss skills of evaluating instructionalmethods used in the classroom; ,a

I differentiatethe methods of evaluationof hardware and software components;

9.2 EVALUATION: AN ALL PERVASIVE -PROCESS

Education is a continuous process because knowledge never cedses to grow.
Evaluation is important in every walk of life and more so in education. There cannot
be education without evaluation. The term evaluation is often used interchangeably
with the term measurement. But it is more comprehensive and involves the placing
of an o v e d valuejudgement on leaming. To evaluate is to appraise, assess, calculate,
estimate, gauge, judge, rank, rate and value. The thing evaluated could be an
object/ contenaearner behaviourJetc. Evaluation has been generally described as
'the process ofjudging the value or worth of something or of a trait possessed by a
person. For instance, evaluation is carried out to determine how well aprojecv
educational program has achieved the predetermined goals. It is a simple way of
examining, monitoringand checking the outcome ofthe project. It is basicallyjudging
the worth of a project or a product or a programme Evaluation includes assessment
and the more specific measurement. Measurement may for instance, involve
quantification, say the number of learners ,passing the programme.

9.2.1 Scope and Purposes of Evaluation

Communication technology has made education, training and learning more
accessible. It has also changed the method of teaching and leaming. The use of
educational technology in enhancing learning experiencehas been hotly debated in
academic circles. Studies confirmetheawesome power of TV and computers to
impart education. The conventional method of teaching is gradually giving way for
the technologically mediated education. With the mind-boggling array of new
technological options in the classroom, formal evaluation of technology becomes
inevitable.
Evaluation of technology has to be undertaken because St has changed the very
model ofteaching. Performance of the students depends on many things. One of the
visible things that makes major difference in the classroom is the use of new
technology. Technology like computers, internet, video, audio, etc. are vying with
books and black boards in the classroom. Therefore,there is need to evaluate the
convergenceofinstruction and technology. Evaluation will answer the broad question
whether technological intervention in classroom is better than no technology.
Evaluation also helps the policy makers decide, if adopting a technology will help
accomplish educational goals.New technology is regarded as a boon to improve
student performance. Continuous evaluation of the impact of such technological
interventions will help decision-makersto expand such services in other situations.
Evaluation helps rationalize the huge investments and resources needed. Evaluation
also helps in revising the curriculum and in understanding how technologyaffects the
role and expertise of teachers.
 9.2.2 Types of Evaluation - ~ o r m a t i v eand bummative Evaluation :ABroad Concept
I

There are two major types of evaluation. They a& formative and summative
evaluations. I I

Formative Evaluation: Formative evaluation is m4nly used in the early stages of

implementation of iprogramme. It does not measure the outcbme of a programme
-or a project but is very useful in identifying the major or milnor bottlenecks in a
programme and the aspects that need to be improvqd beforf delivery. You may
recall that we have discussed this in the previous units ofthe eplier block.
P
For example, a school wants to introduce a comput$r orientdtion for
the teachers after school hours on weekdays. The prdgramm&is extremely useful
for the teachers and the hands on experiences are offaked free bf cost. Nevertheless
the programme fails to attract teachers in large numbed. Early ebluation reveals that
the teachers find it extremely difficult to attend the programme eerthe school hours
'as they will be totally exhausted and will be longingta go home. The results of such
formative evaluation can therefore be used to make chanaes and improve the
programme, which will improve the impact or the effect o f t . study. The focus of
formative evaluation is on monitoring an ongoing project or aprogrammeto collect
feedback. This helps in bringing changes or improvements in tple project.
I
I
Formative evaluation is usem:
I

in identifying the problems in the early stages of the proj(xt;

in improvingthe impact or effect of the programme;
in making d v e evaluation more meaningfd; I
I
in improving the planning, management and implrmentatibn of a
!
I
in monitoring an ongoing project for improvement;
in fine-tuning the implementation of the project; ~
I I
in guiding futureactivities ofthe project; I
I I
SummativeEvaluation: unlike formative evaluatiod in s d a t i v e evaluation the
data are collected after the programme has been implemented We have discussed
t this in the earlierunits ofthe previous block. It followsthe ttansaJ'on ofthe curriculum

I
r"
for evaluating learningoutcomes. It can be in the f m of achie ment tests, used for
grading and certification. Summativeevaluation is basically an impact study, which
enablesus to understand cause and effect relationship. It provide3evidence of whether
the programme has accomplished the objectives, ornot. It i$volves judging the

4
overall impact of the project on a target population, Did th project achieve its
startedgoals? Is it worth continuing?These are some of the majo questions answered
by summative evaluation. !
~
For example in a school the students are provided access to *b based lessons in
additionto classroom teaching to improve their perfomlance. Stpnmativeevaluation
ofthis technological intervention in the classroom is &ne at d o levels. Firstly, the
test grades (pretest) and basic data ofthe students prior to the idplementation of the
programme are collected in the early stages similar to formatbve evaluation. The
second stage of the evaluation is done after the instrudtionshate been delivered to

to the use of technology. 4

evaluate the impact of the programme (posttest). In this exam le, the test scores of
the studentsdeterminethe impact of the educationaltachnolo programme in this
example. Improvement in the performance of the studentsif fo d, may be attributed
I

I
7
Evaluation Concepts Summativeevaluation is usefulin:
determining the impact and the long term impacts of any programme;
in getting immediate solutionto the problem.

9.2.3 Evaluation: An Integral Component of Educational

Process
Evaluation is an integral component of education. The feedback fiom evaluation can
be used to improve the curriculum, textbooks, method of teaching, facilities for
learning, etc. Evaluation is required at every stage of introducing and integrating
technology in education.Evaluationbegins even before the conceptualization of the
project or programme. Take for example the project on computer literacy
programme. In this project, the evaluation must begin with need assessment i.e.
determining the need for making the target group computer literate. It helps in
planning the programme as per the needs and interests of the learners. -
Later, when the project of computer literacy moves from conceptualizationto the
stage of implementation, feedback following evaluation reveals its effectiveness.One
of the factorsattributed to the success of the project is its proper implementation.In
the computer literacy project for example, questions like whether the computers
were accessible to the learners and whether all ofthem in the target group participated
will guide the evaluators to monitor the implementation ofthe project.
Basically evaluationis conducted to know whether the objectives ofthe project are
accomplished. In this project on computer literacy, evaluation looks for evidence of
impact oftechnologically mediated education on the learners. Therefore, it is clearly
evident that evaluationhelps to determinewhether aproject is necessary; if so how
long it should be run? Will it be worthwhile to repeat or extend the programme to
other areas? The feasibility, reliability, credibility and utility of a programme are
determined by evaluation. Evaluative studies at the end of the project answer the
hdamental question of whether the project produced desired positive outcome
and if so does itjustifl continuation and extension. Evaluationhelps assessthe worth
and progress of the project and in meeting the vision and goals of educationprocess.
Evaluation is the basis for planning and implementation of educational technology.
As the agenda of education today, is to implement and integrate technology in all
areas of educational environments,evaluation assumes importance in the process of
education.
Check Your Progress 1
Answer the following questions briefly.
1) How does evaluation differ h m measurement?
2) What is formative evaluation?
3) What is summative evaluation?

9.3 EVALUATION AT DIFFERENT STAGES

Educationaltechnology offers wide range of tools, materials and devices to be used
in education. Technological intervention in designing the courses in terms of hard
ware and softwareneeds to be evaluated. Finally the significantelementsofeducational
process, which need to be evaluated, are instructional methods and procedures in
which technology is used. Evaluation can be conducted at different stages. We shall
8 discuss them now.
9.3.1 Curriculum Planning Stage Evaluation :ABroad Concept

Curriculum is basically an organized plan to guide ledrning experiencesand to attain

teaching and learning goals. Curriculum laced with hbvy coniponent of educational
technology needs to be specially evaluated. Today it is felt sat technology should
be integrated in the curriculum to improve student learning. lhis calls for providing
technology and software resources to educational iqstitutioris and for professional
development of teachers. Nevertheless, it is nectssary tc) plan what and how
technology will actually be used in curriculum dependin&on the needs of the
educational institutions and the teachers.
The firstrequirementin a curriculum design as you have a l 1 - 4read
~ is to formulate
the objectives. It should state clearly the expected end product of education. The
objectives should reflect the needs of the students as well as the needs of the
society. The objectives should determine the process. ~ h next k stage is that of
content selection and organizationi.e. designingthe content. Co~tentbasidly consists
of informationthat has to be learnt and the related skills that bave to be developed.
Content shouldbe adequate both in terms of quality and quantity. Content consists
of facts, skills, values, etc. The content should contributeto the achievement of the
stated objectives. The educational technology seleclied shou@also be accordingto
the nature of the content. i
P
Deciding the method of imparZing instruction is the next step. t determines the tools
and resowces needed for delivering instruction.The use of nstructional media is
planned at this stage. Finally there is implementation of the(curriculum. It is then
evaluated to determine its effectiveness. I

9.3.2 Instruction/Course Design Stage

The presence of teacher in the classroom makes learning intepctivethrough a two-
way communication process. Educationaltechnology too has bpact on the methods
of instruction. Moderntechnologies canprovide instructions through multimedia and
can make instructions interactive. Thanks to technology, I e T g is becoming more
and more individualisticand therefore, learnercentric. The tecwology selected should
be evaluated to check if it fdfds these conditions.i4ny design df instruition should be
based on the instructional objectives of the cuniculmm.~ec&ologyshould aid the
llfillment of objectives. Does this computer softwa& help sthdentsto acquire skills
of problem solving in a better way? Does this softwdre checkthepercentage of low
performing students?These could be the questionsqat need @wen. Evaluation of
technology certainly reveals whether it is need-basekl or not. (
The quality of instructiondepends on teachers and orther resobces. re the teachers
technology literate? Do they frequently underglo profejsional development
programmes? These questions have to be answered before imitegratinga particular
technology. Technology integration in the course requires rekwces. Making a list
of the type of resources needed and access to thkm is nebessary for planning
instruction. Media cannot be isolated from the insbction. heref fore we should
design the method with the instruction and media cbmbination to accomplish the
objectives. We may plan a learner centric method in die followingmanner (Figure 1).
l
Media I

,
Fig. 1 I
I
Evaluation Concepts 9.3.3 Courseware Development Stage
A courseware generally refers to content to be delivered through educational
technology. Curriculum planners and teachers today have a choice of wide range of
materials and facilitiesfor teaching- learning. The development of courseware is
much more complicated than the development of textbooks and other print materials.
As a teacher you must know what tools to use in the classroom and how to use them
effectively. The teacher has to understand and analyze the impact ofthese new tools
on the learning process. First we must think whether the use of technology is usell
for instructions.If it is good then the teacher should find out how it has improved
learning. If it is not so then teacher should probe why the use oftechnologyhas not
produced desired results. Therefore evaluation is important at every stageoflearning. ,

First the appropriate technology has to be selected. It may be audio, video,

multimedia CDs, online content, etc. It is not necessary that all the technologies are
required for a course. Theyneed to be evaluated in terms ofvalidity- both instructional
and technical validity. Questionslike the following may be asked for establishingthe
validity.
Is the technology sound enough to enhance learning experiences?
What is the budgetary allocation for materials?
Does the material justify the huge sums of money spent by the sponsors1
governrnent/school?
Does the technology help to meet the instructional goals?
Is the technology flexible? Can we use it for multi-purposes?
Is the technology relevant to the lessons?
Is the technology obsolete?
\
Are teachers/supervisorstrained to handle the technology?
These questions confront the curriculum designers. One can conduct apilot test to
evaluate the courseware in terms of above-mentioned criteria. Thereafterevaluation
of performance of the students will make us understand the role of instrudtional
materials in learning. Evaluation of the courseware reveals its effectiveness.We may
use standard tests, check the marks sheets, class records, report cards to determine
whether positive performance can be attributed to the use of new technology devices
in the instructional process.

9.3.4 Curriculum Evaluation/Renewal Stage

Curriculum validation refers to critical assessment of curriculum by subject experts.
Validation is evaluation or assessment and approval of curriculum. The relevance,
credibility, viability and flexibility of the curriculum are the criteria for validation.
Once the c~uriculumis approved, it moves on to the next stage i.e., implementation
and improvement.After the curriculum is adopted, it calls for monitoring to determine
whether it is producing desired results or not.
The curriculumis then evaluated for its improvement and renewal. The effectiveness
of the course is assessed in terms of several factors viz. what kind of competencies
are the students equipped with following the successful completion of the course? Is
the coursecompatibleto the rapid changes in technology? Is technologicalintervention
good or bad for the exit profile of the students?Asurvey will help collect information
10 to know the needs and demands of the students.
 I

can technology be adopted and integrated into the durriculw? Many teachers and
studentshave great difliculties in adjusting to new instructia technologies. New
technology like multi-media, video lessons, audiovisuals, irpteractive learning and
others sound great but they should be useM and awssible to the users viz., teachers,
students, and thus beneficial to the stakeholderssuch as parent&school administrators
and educationists. The other important component of evaluationis relevance. Is the
technology-based curriculum relevant for the target studentis? What should be the
exit profile ofthe students?Does this curriculum meet those expectations? Validity
of curriculum is the next important criteria of evaluation. Checking the validity of
learningmaterials, which may be softwares, video lessons odaudio CDs is required.
Lastly assessment of the curriculum in terms of feasibility ha$ to be undertaken. The
practicality ofthe curriculum design has to be examined. Soine ofthe goals of new
technology are ambitious. The realization of these goals requires huge investments,
infrastructure and skilled human power. If computer literacy,is the main goal of the
curriculum,then the student-computerratios and the teacher+omputer ratios should
be adequate to meet the instructional objectives. Therefore, i\ is necessary to check
the feasibility ofthe curriculum. I

practices and students' performance.

' What kind of media is appropriate?

Is the hardware technically sound?

Is the software relevant to the curriculum? i

I!
Evaluation Concepts Can the'programmeshelp achievethe instructionalobjectives?
Do you thinkthe software has affected learning?
Would you recommend it to a fiiend or a colleague?
How do students learn in technology mediated classroom?
How often did the students use the programme?
What kind of skills did students develop in the programme?
b Can you attribute the quality and effectiveness of teaching to the use of
-technology?
These are some of the common evaluative questions concerning hardware and
software.

9.4.2 Evaluation of Instructional Methods and Processes

Educational technology has transformed the methods of instruction by widening the
horizon of subject matter and the teaching methods. Technology has come to the aid
of classroom teacher as well as distant teachers. They have created more resources
for teachers as well as students. New technology has also paved the way for
transformation of instructional practices. The shift from conventional method of
instruction to technology based instructional techniques certainly needs evaluation.
Quality and effectiveness of instruction delivered through technology must be
evaluated in terms of its end results i.e. student learning and this has to be canied out
through standardized tests. The comparison of the test scores before and after the
introduction of technology initiative is one way of such evaluation.However, the
accuracy of these measures depends on the reliability and validity of the tests.
Evaluation of instructional method provides data about the instructional potential of
the technology in the teaclung learningprocess. Genemllythe evaluationof instructional
methods and processes includethe following:
The statement ofinstructional objectives.
Need assessment is conducted. For example, what is the need for introducing
word processing software? Does the softwareimprove students' writing skills?
One has tojustify the use of the software in the instruction through evaluation.
Performance of technology as a tool for providing instruction. For this one may
need to evaluate the effectiveness ofthe software.
Performance of technology, operational difficulties, need for trained teachers,
etc. One may mod@ the instructions on the basis ofthe findings and administer
the programme after plugging the loopholes.
The end product of the programme determined through summative evaluation.
The students must be evaluated for the quality and effectivenessof instruction
through standardised tests. The comparison of the test scores before and after
the introductionoftechnology initiative is one way of assessment.Nevertheless,
the accuracy of these measures depends on the reliability and validity of the
tests. Grades awarded to students for their reports and presentations and their
test scores are some of the considerationsfor surnrnative evaluation. Further
classroom bbservation and teacher surveys are other means of evaluation.
The instructional activities. We have to collect qualitativedata on the behaviour
ofthe students.The measures ofattitudeof the studentsare importantto remove
12 some mistaken notions about technology.
 I
I Evaluation :ABroad Conced
9.4.3 Learner Assessment I
I

Evaluation of learner performance can be done thhugh achievement tests, based

upon the courseware delivered through technolagy. The ~urnberof successful
candidates forms the basis for assessment of thg effectiveqessofthe technology.
For instancethose taught without the technological interveritions and those taught
through technology may be compared through aclpievemebt tests to evaluate the
impact of technology on learners' achievements.

I Answer the following questions briefly. I

I

I
I 1) What is hardware? I

I
1 3) Mention some of the criteria for evaluation of technolop.

9.5 EVALUATION: SOME GENERAL CONCERNS

Educationis undergoing rapid changes due to the influence of technology in general
1 and information and communicationtechnology(ICT) in particular. Todayinfomation
technology is becoming an integralpart of education.These new technologies have
changed the perception rather the very concept of a classrooin and have increased
1 opportunities for studentsand teachers to have access to abhndant and variety of
resourceshitherto not available. However, the inc-ing use of ICT in education is
leading to certain concerns. Some of these are :
What tools or devices should be used?
Does technology improve professional development of teachers?
Do studentsperform better with technological interventibn?
These are some of the general concerns of the edwcators and evaluators. Let us
discuss some of these issues.
Why to evaluate?
We all know that technology is changing rapidly and it requires great efforts to keep
pace with technology. Technology gets outdated rapidly. The mind-boggling array
of technological devices has changed the ways and means ofteaching and learning.
Technology althowgh should be consideredas a natural acadehic input in education
like textbooks and blackboard but the choice and use of technology in education
demand evaluation of the technologybefore it is chosen. Educationaltechnology is
no'rnagic wand to solve all the problems of education. So evaluation is necessary to
remove any misconceptions regarding them and adopt a pragmatic approach
regarding the potential oftechnology. Evaluation findings prbvide sound basis for
making anyjudgments.Evaluation is dependablea s it is based on accurate, relevant
and comprehensive information. Data from evaluation can ~ u i d us e as to which
technologiesreally enhance the learning experience significantlkand which do not. It
helps us to assess their strengths and weaknesses. Therefore we can retain some
componentsand discard others that fall short of our expectations.
Evaluationhelps us to monitor a programme and modiQ it if necessary. For example
I
learnersmay have opted to enroll as online learnersbut later may find it difficult to
continue. Evaluation of the programme, including the study of the impact on the
target group can point out the problems of learners in accessingonline facilities on
Evaluation Concepts a continuedbasis .While improving the programme, the alternate technologies can
be sought. If you don't modifjr the inputs (technologies integrated),the whole project
will fall flat.
Evaluation gives dictions for improvement and without it we will have to grope in
dark. Are the computers working? Are students really learning with the help of
computers? Have students made any improvement after they were introduced to
computer lessons? These are some of the queries denoting the need for assessment
of any programme. Evaluation is a useful means to strengthena project.
Use of technology demands huge investments. Many educational institutionshave
been spendinghuge sums of money on technology like computers and networking
to increase productivity in terms of learning.The funding agenciesneed data that can
show that the investment has paid dividends. They need to be reassured that the
funds are producing desired results and are being used for V I O I - ? ?causes.
~ Evaluaiion
is necessaryto understand what impact this investment is having on pupils. Evaluation
makes everyone involved accountable for kis or her actions.A5 part of professional
development of teachers, for example, a project expects the teachers to usz computer
for teaching. How fiequentlyor to what extent they use computes? Are they working
hard or are they casual in their approach? Evaluation answersthese kinds of questions.
The mainjustifisatiofi for evaluation in a classi'oom is to know the learning outcomes
ofthe students.Effectivelearning depends largely on continuousevaluation. There
cannot be any worthwhile teaching without evaluation as accomplishingeducational
outcomes is the purpose of teaching and learning.
Activity
You may prepare a worksheet to address the question of why we need to
evaluate. Consider the differentparticipants &d users of evaluation studies to
prepare the worksheet. You may prepare it in the following manner:
Users of Evaluation data
1. Funding agency/sponsors
What do t h y want to know?
Does investment in computer literacy
I
programme improve achievement?
2. Government bodies Does computer literacy programme
increase access to quality education and
bridge the digital divide among school
ckildren ?
3. School administrators Does the computer literacy programme
check low attendance and motivate
students?
4. Educationists What impact the computer literacy
progmmme will have on the quantity and
quality of student work in the classroom?
5. Teachers Does computer literacy programme
improve the performance of the
students?

1
6. Parents Does the computer literacy programme
impart related skills and prepares
students for future challenges?
Whatfwhom to evaluate? Evaluation :ABroad Concept

Ifevaluationis inevitablethen the important questidn is what and whom to evaluate?

Technology-mediated education includes computers, networks, printers, scanners,
file servers, CD-ROM drives, Internet, the variohs softwares, etc. The focus of
evaluationis to understand the impact of these devices on pupils. For example, did
the computer aided teaching of biology lesson improve the performance of the
students?Did the multimedia lesson using 3-D image of heart and its hctioning
help students perform better? These questions arb important. The proper use of
instructionaltechnology is to know what one has to evaluate.
We need to define clearly what to evaluate and dhom to evaluate. In scientific
terminology,you have to evolveyour indicators, benchmarksand measures.Indicators
are statements that reflect specific gods that can b& used to gauge progress. They
suggest the activity one has to take to achieve theIgoals. Benchmark is a target,
which specifies the final outcome that the programmlt aimsto achieve. It is a point of
reference, a yardstick or a norm and a final target for that goal. Benchmarktells us
what the programme is trying to achieve. Measure is evidence in either quantitative
or qualitative form. It may be in the form of number, percentage,'mtio or qualitative
statements.
Study the followingworksheet consisting of evaluativecomponentsto understand
the processes of collecting information: I

Goal Indicator Benchmark Measures

Computer I Good number of By the end of the ' Assessment of
literacy students will be school year, high student computer
among trained to become school students literacy by
students computer literate in government conducting skills
schools in test demonstrating
districts wil be ,their ability to
computer literate perform basic

2. Check Computer literacy By the end of the

dropout programme will school year, the and dropout rate
rates in increase attendance rate
government attendance in will increase by efore and after
schools. government 10 percent. he introduction of
schools and the omputer literacy
number of dropouts rogramme.
will decline. -

In this manner one has to workout the indicators, benchmark and measures for
each goal to know what type of information is needed fbr evalu
How to evaluate?
The use oftechnology in education is a big challenge for
considerable time, investment, training and skill to
education. There are two stages at which
stage is to plan evaluation along with the
early evaluation helps the school to plan
basic data about the number ofusers,
which are essentialfor planning
an evaluation after the programme has been implemented.
Goals and questions: First it is required to formulate Questio
technology. Define the goals of your study. What is that you
clear about what you want to find out from the study The tools
 Evaluation Concepts on the purpose of evaluation.Write down the questions that you want to be m e r e d ?
Collect information accordingly.
Second step is to select the respondents. This can be done through the standard
sampling techniques (Refer unit 12).Nevertheless, in some situations it may not be
possible to followsarnplingtechniques.For example ifyou have to studythe computer
use by the students belonging to particular language or caste group, you have to
study only these students. In non-sampling studies your frndings speak only for that
particular school or group of students and cannot be generalized for all the schools
or for all the students.
Third step is wherever possible make the study comparative. Compare the
performance of the students using Internet, for example, with the group of non-
users.You might compare:
the performance of two groups of students, one who received and the other
who did not receive technology based biology lessons;
computer users and non users both belonging to low performing group;
computer literacy of studentsbefore and after training;
performance of boys with thatof girls in the use of technology in the classroom;
the duration and type of computer use of students of government school to that
of private school.
When you compare, make sure that both the groups are similar in all the aspects as
far as possible.
The fourth step is to collect information. The source of information depends on the
subject of evaluation.Nevertheless a checklist will give you an idea about what type
ofinformation to collect. Read this example now:
Checklist of information in case of students exposed to video clippings in biology
lesson:

I The total number of students;

Data following a studyof the curriculum to know the content which can absorb
technology inputs;

I What type of technology can be used;

I Data on access to computers;

I
I
Data on the earlier test scores of the students;
Data on the learning difficulties ofthe students;
Number of girls and boys in the study group;
Type of school-government, aided or private.
Thus a mixture of qualitative and quantitative informationhas to be collected for any
typeofevaluativestudy.As faraspossible we should collect informationb m multiple
sources, as it will strengthen the finding. Your sourcesof informationmay be school
administrators,teachers, students, parents, media persons, financial experts, and
manufacturers of hardware and software. Collect information about previous
evaluation studies to avoid duplication of studies and to overcome pitfalls. The
information that you collect must be accurate.Also make sure that the source of
self-reportingtechniques and tools like rating scales ( an in$mnents that elicitsthe

combined if objectives are multi-pronged.

ysis (Refer Unit 12)

may be done. The content is analy
ones).After the analysis one has to
n it is required to take

17
Evaluation Concepts You may prioritize your goals and answer the above questions on the basis of
informationcollected.

9.6 EVALUATION: A MEANS, NOT AN END IN ITSELF

Evaluation of the use of educational technology will help teachers and school
administrators to make better judgments. However, a teacher shouid not believe
that all the learningproblems ofthe students could be solved through it. Evaluationis
not a magic wand to solve the problems of education. It is not an isolated element.
It is a continuous process. It does the following:
guides students towards achievementand enhances learning experience ;
understands and analyzesthe learningdficulties;
helps students to overcome the problems;
identifies the strengths and weakness of the students, programme and the
technology used ;
Determines costquality-effectiveness ratio of programmes.
To achievethe above mentioned points, evaluation should be timely. It should predict
long term impact and should generate information that leads to improvement of the
programme.
Utilization of Evaluation Findings
Evaluation gives us information that is credible and objective. The duration of
evaluatibn depends on the nature of the educational programme. However, as
technology is evolving at a very rapid rate, the studies cannot be lingered for years
as the findings may become irrelevant. Nevertheless you can organize the useful
evidences for immediate use in the following manner.
Use of technology by students: This wiil include information on use of technology
by students. Data on issues such as time spent on computerlothertechnologies,
the purpose for which technology like computer and Internet are used, whether
\
they are used under supervision or independently,qualitative information about the
students' response to new technology viz., whether technology motivated them to
learn better, whether they learnt the skills of operating the technological devices,
etc. are to be compiled and organized.
Teachers' use of technology: You may gather information about the type of skills
teachers have developed following training in handling technology.To what extent
they have incorporated technology in their lessons, whether they are innovative in
using technology, whether they require M e r training, etc. are to be collected and
organized under this head.
Data can also be collected from the drop out sample i.e. those learners who quit the
programme before completing it, the administrators, the prospective employers,
etc.
Check Your progress 3
Answer the following questions briefly:
1) What are the sources of information for evaluation oftechology ?

18
 I Evaluation :ABroad Concept
2) Define the followingterms;
i
9 Indicator I
I

I I

Benchmark ! I
!

iii) Measure I 1
I

9.7 SUMMARY 1

rI In this unit you have learnt to define the conceptof evalLon

the importance of evaluating the use of
you have understood
in classroom. You
I have also studied about the process of
instruction and the lesson material.
skills ofevaluating instructional
I also learntto differentiatethe

9.8 UNIT END ACTIVITIES I

I

1) Find out the for a computer literacyprogtamme dat has been introduced
in a school accessible to you.
I
2) P ~ p a r e
a table of goals, indicator, benchmark and mebures for multi-media
programmes in any subject for any grade. I I

3) Prepare a plan for the evaluation of a programme bn the introduction of .

computer literacy for students in a governmerit school]

9.9 REFERENCES AND SUGGESTED READING

You may visit websites on the to$cs discussed in this unit, like the following:
retrieved on 28.8.07
htrp:/lwww.wmich.edulevalctr/jc~~~~~~~~-~~~.h~
http://www.b~~hessballs.~~m/~gpmgramevaluation.
retrieved on 28.8.07
on 28.8.07

Check Your Progress 1

$
2) Fomtive evaluation is mainly done during the devel pment ofthe programme
or in early stages of implementation. It is useful for i proving the programme
before its final delivery.
I
3) Sumrnativeevaluation is basically an impact study, provides evidence,
whether the programme has accomplishedthe obje
I
Check Your Progress 2

1)
I
The physical equipment itself like cornpujer, TV,
I
$lie ,etc. used to runthe
Evaluation Concepts 2) The programme run on the hardware.
3) Infrastructure required to operate it, cost involved, operational difficulties,
pedagogic strengths and weaknesses, etc. .
Check Your Progress 3

1) Learners, dropout sample, teachers, educational ~drninistrators,employers,

etc.
2) 9 Indicator -statementsthat reflect specific goals that can be used to gauge
progress. It sqggests the activity for achieving the goals.
Benchmark is a target, which specifiesthe finaloutcome that the proprune
aims to achieve. It's a point of reference, a yardstick or a norm to be
achieved through the educational programme.

i Measure is evidence i.e. data in either quantitative or qualitativeform. It

may be in the form of number, percentage, ratio or qualitative statements.

20