Models of curriculum

Evaluation

Dr. G.A.RATHY
Assistant Professor
Electrical Engineering
Department
NITTTR, Chennai
Product Evaluation

• Looks at results of a program after its

implementation
• Focuses on summative question techniques
• Examination for accuracy or obsolescence
Five Evaluation Models

1. Objectives Approach (Tyler)

2. CIPP (Stufflebeam)
3. Stakes Model
4. Goal-Free (Scriven)
5. Hierarchy of Evaluation (Kirkpatrick )
Concept of a model
Theory : Explains a process
(Why?)
Model : Describes a process
(How?)
Model is a representation of reality presented with
a degree of structure and order.
 Why do we need a model for curriculum
evaluation?

To provide a conceptual framework for

designing a particular evaluation depending
on the specific purpose of the evaluation.
 Tyler’s Model of Curriculum Evaluation

Propounded by Ralph Tyler in 1949.

One of the earliest curriculum experts.
Tyler ’s model is also referred to as “the
goal attainment model of curriculum
evaluation”
 Tyler’s Model (Cont’d)

Tyler ’s model emphasize more on goals

formulation through a
detailed analysis of feedback from the
students, society and subject matter.
The model measures the extent to which
the educational goals of a program have
been attained (Singla & Gupta, n.d).
 Tyler’s Model (Cont’d)

Tyler model proceeds from the broader

view of a concept and narrow it down to
the specific.
For instance, it examine the needs of the
society and then narrow it to stating the
specific objectives to be achieved in order
meet the societal need. It is however said
to be deductive in nature (Oliva &
Gordon, 2012).|
 Tyler’s Rationale

What educational purpose should the school seek to

attain (i.e. educational aims, goals and objectives)?
What educational experiences (learning experiences)
can be provided that are likely to bring about the
attainment of the set purpose?
 Tyler Rationale (Cont’d)

How can these educational experiences

be effectively organized (organization of
learning experiences)?
How can we determine whether these
purposes are being attained (evauation)?
(Omoniyi, 2009)
 Tyler’s Model

Selection of Selection of Organization Evaluation

educational learning of learning of learning
purposes experiences experiences experiences
 Tyler’s Model: Short Comings

Linear Framework
Leaves evaluation towards the tail end of
the framework
 1. Tyler’s Model
Key Emphasis:
• Instructional Objective
Purpose:
• To measure students progress towards objectives
Method:
1. Specify Instructional Objectives
2. Collect performance Data
3. Compare performance data with the objectives/standards
specified
Limitation of Tyler’s Model
1. Ignores process
2. Not useful for diagnosis of reasons why a
curriculum has failed
Tyler’s Planning Model(1949)

What educational goals

Objectives
should the school seek to
attain?

How can learning experiences

Selecting learning experiences be selected which are likely to
be useful in attaining these
objectives?

Organising learning How can learning experiences

experiences be organised for effective
instruction?

How can the effectiveness of

Evaluation of students learning experiences be
performance evaluated?
[Print, M. (1993) p 65]
 CIPP Model of Curriculum Evaluation

Propounded in 1971
The CIPP model came as a consensus of
the Phi-Delta Committee that was chaired
by Daniel Stufflebeam in 1971. CIPP
focused more on decision making at each
stages of the curriculum processes.
 CIPP Model of Curriculum Evaluation

C – Context
I – Input
P – Process
P – Product
Evaluation Objective Method Application to Decision making

Context  To determine the operating context By comparing the actual and the intended For deciding upon settings to be served
• To identify and assess needs and opportunities in and outputs • For changes needed in planning
context  Needs of Industry, Society
• To diagnose problems underlying the needs and  Future Technological developments
opportunities  Mobility of the students

Input To identify and assess system capabilities, Analyzing resources, solution strategies, For selecting sources of

available input strategies and designs for procedural designs for relevance, feasibility and support solution strategies and procedural

implementing the economy designs for

strategies structure changing activities

• Entry behavior of students
• Curriculum Objectives
• Detailed contents
• Methods and media
• Competencies of teaching faculty
• Appropriateness of teaching / learning
resources

Process To identify process defects in the procedural By monitoring the procedural barriers and For implanting and

design or its implementation remaining alert to unanticipated ones and refining the programme design and procedure

describing the for

actual process effective process control

Product To relate outcome information to objectives Measurement Vs Standards interpreting the For deciding to continue, terminate,
and to context input and process information outcome modify, build or refocus a change of activity.
 CIPP: Short Coming

However, despite several promises of excellence and

potential in moving in the region of formative and
summative evaluation CIPP model, is too concerned
with how the process is supposed to be rather than
the reality on ground (Pradinata, 2012).
 2. CIPP Model

The CIPP model of evaluation concentrates on:

Context of the programme
Input into the programme
Process within the programme
Product of the programme
CIPP Model
 CIPP Model
Intended Actual
ENDS Context Evaluation Product Evaluation Qn:Have
Qn: What? we?
Attainments
Environment &
Needs Outcomes - both quality and
significance

Input Evaluation Process Evaluation Qn: Are

MEANS Qn: How? we?
Procedural Designs Procedures in use
Strategies & Monitoring & implementation
Resources
 CIPP Model
Stufflebeam’s model of evaluation relies on both formative
and summative evaluation to determine the overall
effectiveness a curriculum programme
CIPP
 CIPP Model

Context Evaluation
o Most basic kind of evaluation
Objective:
• To determine the operating context
• To identify and access needs and opportunities in the
context
• To diagnose problems underlying the needs and
opportunities
• Identify population
 CIPP Model

Context Evaluation

Method:

system analysis, survey, document review, hearing,

interview, tests, Delphi (Wiseman technique)
 CIPP Model

Context Evaluation
Relation to decision making:
• Decide upon settings
• Goals and objectives
• Provides rationales for determining objectives
• For changes needed in planning
 CIPP Model

Context Evaluation
Target Group for Collecting Information
Industry
Society
Working Technician Engineers
AICTE
 CIPP Model
Input Evaluation
Objective:
• Identify and assess system capabilities
• Alternative strategies
• Implementation design
• Budget

Method: resources analysis, feasibility analysis, literature research, exemplary

program visits and pilot projects
 CIPP Model

Input Evaluation

Relation to decision making:

• Selecting sources
• Structuring activities
• Basis for judging implementation
 CIPP Model

Input Evaluation

• Entry behavior of students

• Curriculum Objectives
• Detailed contents
• Methods and media
• Competencies of teaching faculty
• Appropriateness of teaching / learning resources
 CIPP Model

Process evaluation:
Objectives: To identify process defects in the procedural design
or its implementation

Method: monitoring, describing process, interacting, observing

 CIPP Model
Process evaluation
Relation to decision making:
• For implementing and refining program design and
procedures
• Process control
• Provides periodic feedback to those responsible for
implementation
 CIPP Model

Process evaluation

Feedback to judgement

• The effectiveness of teaching –learning methods

• Utilisation of physical facilities
• Utilisation of teaching learning process
• Effectiveness of system of evaluation of students
performance
 CIPP Model

Product evaluation:

Objectives: To relate outcome information to objectives and to

context input and process information

Method: Measurement Vs Standards interpreting the outcome

Relation to decision making: For deciding to continue,

terminate, modify, build or refocus a change of activity.
 CIPP Model
Product evaluation:

• Employability of technician engineers

• Social status of technician engineers
• Comparability of wage and salary structures
• Job adaptability and mobility
 STUFFLEBEAM’S CIPP Model(1971)
Context Input Process and Product evaluation

• Key Emphasis : Decision-making

• Purpose : To facilitate rational and continuing
decision-making
• Strengths : a) Sensitive to feedback
b) Rational decision making among
alternatives
• Evaluation : Identify potential alternatives,set up
activity quality control systems
Strengths of CIPP
It has the potential to act in a formative, as well as summative
way, helping to shape improvements while the project is in
process, as well as providing a summative or final evaluation

Limitations of CIPP
Top-down or managerial in approach
CIPP View of Institutionalized Evaluation
Steps in CIPP model :

• Focus the evaluation

• Collect information
• Organize information
• Analyze information
• Report information
• Administration of the evaluation report
 3. STAKE’s COUNTENANCE MODEL

- Antecedent is any condition existing prior to teaching

and learning which may relate to outcome.
- Transactions are the countless encounters of students
with teacher, student with student, author with reader,
parent with counsellor
- Outcome include measurements of the impact of
instruction on learners and others
 Description Matrix Judgement Matrix

Rationale Intents Observation Standards Judgement

Antecedents

Transactions

Outcomes
ANTECEDENTS

• Conditions Existing prior to Curriculum Evaluation

 Students interests or prior learning
 Learning Environment in the Institution
 Traditions and Values of the Institution
Methods

• Time table
• Syllabus and textbooks
• Interview
TRANSACTIONS
Interactions that occur between:
TEACHERS STUDENTS
STUDENTS STUDENTS
STUDENTS CURRICULAR MATERIALS
STUDENTS EDUCATIONAL ENVIRONMENT
TRANSACTIONS = PROCESS OF EDUCATION
Methods

• Activity Records
• Observations of class
• Self-report by teachers
• Self-report by pupils
OUTCOMES
• Learning outcomes
• Impact of curriculum implementation on
 Students
 Teachers
 Administrators
 Community
• Immediate outcomes Vs Long range outcomes
Methods

• Test and written work

• Questionnaires
• Interviews
Three sets of Data
1. Antecedents
• Conditions existing before implementation
2. Transactions
• Activities occurring during implementation
3. Outcomes
• Results after implementation
• Describe the program fully
• Judge the outcomes against external standards
How did Stake’s model help us to analyse data?

intentions observations issues

antecedents Staff and students Not all had these Time

would acquire skills or chose to
skills develop them

transactions All students would IP address Copyright

have access to restrictions denied
video access to some

outcomes Most students About half used What about others?

would use video video and one third
enjoyed it
 STAKE’s Model
Key Emphasis:
Description and judgement of Data
Purpose:
To report the ways different people see curriculum
Focus is on Responsive Evaluation
1. Responds to audience needs for information
2. Orients more toward program activities than results
3. Presents all audience view points(multi perspective)
 STAKE’s Model
Limitations:
1. Stirs up value Conflicts
2. Ignores causes
GOAL FREE EVALUATION (1973)

Proponent : Michael Scriven

Goals are only a subset of anticipated effects

• External evaluator unaware of stated goals and
objectives

• Determine value and worth of program based

on outcomes or effects and quality of those
effects
• GF evaluator avoids learning the stated purpose/
goals/ intended achievements, of the program
prior to or during the evaluation.

• Instead, the GF evaluator observes and measures

actual processes and outcomes; and interviews
program consumers.

• This prevents tunnel vision, or only looking at the

program as it pertains to the intended goals at the
risk of overlooking many positive and/or negative
unintended side-effects.
GFE
• GFE evaluator asks: What does the program actually
do? Rather, what does the program intend to do?

• “Merit is determined by relating program effects to the

relevant needs of the impacted population, (Scriven,
1991. p. 180).”

• A comprehensive needs assessment is conducted

simultaneously with data collection.
• It may identify unintended positive and negative
side-effects and other context specific
information.
• It is adaptable to changes in needs or goals.
• It is reversible; an evaluation may begin goal
free and later become goal-based using the goal-
free data for preliminary investigative purposes.
Comparison of GFE Goals and Stated Goals
Goal Free Evaluation
 Created by Michael Scriven (1972)
 The evaluation occurs without the external evaluator
knowing the goals of the program
 This is done so that:
◦ The study topic is not automatically narrowed
◦ Removes negativity to unanticipated discoveries
◦ Eliminate bias created by knowing the goals
◦ Evaluator remains independent
Goal Free Evaluation
 Looks at actual effects
 Can be done in parallel with goals-based evaluation
ASSESSING TRAINING EFFECTIVENESS often entails using
the four-level model developed by
Donald Kirkpatrick.
Kirkpatrick's Four Levels of Evaluation

In Kirkpatrick's four-level model, each successive

evaluation level is built on information provided by
the lower level.
• According to this model, evaluation should
always begin with level one, and then, as
time and budget allows, should move
sequentially through levels two, three, and
four. Information from each prior level
serves as a base for the next level's
evaluation.
ASSESSING TRAINING EFFECTIVENESS often entails using
the four-level model developed by
Donald Kirkpatrick.
 Level 1 - Reaction

 Evaluation at this level measures how participants in

a training program react to it.

 It attempts to answer questions regarding the

participants' perceptions - Was the material relevant to
their work? This type of evaluation is often called a
“smilesheet.”

 According to Kirkpatrick, every program should at

least be evaluated at this level to provide for the
improvement of a training program.
 Level 2 - Learning

 Assessing at this level moves the evaluation

beyond learner satisfaction and attempts to assess
the extent students have advanced in skills,
knowledge, or attitude.
To assess the amount of learning that has occurred due to a training
program, level two evaluations often use tests conducted before training
(pretest) and after training (post test).
Level 3

Evaluation - Transfer

 This level measures the transfer that has occurred in

learners' behavior due to the training program.

• Are the newly acquired skills, knowledge, or

attitude being used in the everyday
environment of the learner?
Level 4

Evaluation- Results
 This level measures the success of the program in
terms that managers and executives can
understand -increased production, improved
quality, decreased costs, reduced frequency of
accidents, increased sales, and even higher profits
or return on investment.
Level four evaluation attempts to assess training in terms of
business results. In this case, sales transactions improved steadily
after training for sales staff occurred in April 1997.
Methods for Long-Term Evaluation

 Send post-training surveys

 Offer ongoing, sequenced training and coaching

over a period of time

 Conduct follow-up needs assessment

 Check metrics to measure if participants achieved
training objectives

 Interview trainees and their managers, or their

customer groups
STEPS INVOLVED IN CURRICULUM EVALUATION
Specify the objectives of the evaluation

Choose an appropriate evaluation design

/ method
Identify the sources of information / data

Construct instruments for data collection

Conduct pilot scale evaluation

Select or develop strategies for

data collection

Conduct Large Scale evaluation

Analyse the data

Prepare reports and

feedback to decision makers
The Delphi method is a systematic, interactive forecasting method
which relies on a panel of independent experts.

The carefully selected experts answer questionnaires in two or more

rounds. After each round, a facilitator provides an anonymous summary
of the experts’ forecasts from the previous round as well as the reasons
they provided for their judgments.

Thus, participants are encouraged to revise their earlier answers in light

of the replies of other members of the group.