Expertise, Consumer-Oriented, and Program-Oriented Evaluation Approaches
•Download as PPTX, PDF•
3 likes•15,049 views
All information referenced from: Fitzpatrick, J., Sanders, J., & Worthen, B. (2011). Program evaluation: Alternative approaches and practical guidelines (4th ed.). Upper Saddle River, N.J.: Pearson Education.
1 of 20
More Related Content
Expertise, Consumer-Oriented, and Program-Oriented Evaluation Approaches
2. Expertise-Oriented Approach
•The oldest type of formal evaluation
•Relies on professional expertise to
judge the quality of an institution,
program, product, or activity.
4. Formal Review Systems
•Mostly used in accreditation
•Examine existing structure of organization
•Examine published standards
•Follow a specified schedule
•Increasing emphasis on outcomes
•Use the opinions of multiple experts
•Status of those being evaluated is
affected by the results
5. Informal Review Systems
•Used primarily for evaluations that
lack published standards or follow a
specified review schedule.
•Use multiple reviewers
•Status of those being reviewed is
affected by results
•Examples: Peer review of articles,
theses or dissertation committees
6. Ad Hoc Panel Reviews
•Occur at irregular intervals when
circumstances demand
•Reviews not related to
institutionalized evaluation or
standards
•Usually one-shot evaluations
prompted by a particular, time-bound
need for evaluative information
7. Examples of Ad Hoc Panels
•Panels to develop standards
•Funding agency review panels
•Blue ribbon panels
8. Ad Hoc Individual Reviews
•Review of any entity by an individual
selected for his/her expertise
•Usually to judge value or make
recommendations
•Example: Employment of a
consultant to review an educational,
social, or commercial program
9. Consumer-Oriented Evaluation
•Much like expertise-oriented
•Helps inform decisions on what to
purchase or trade
•Judge the quality of something, establish
value, merit or worth of a product
•Audience is broader: The purchasing
public, and they are not known to the
evaluator.
10. Consumer-Oriented, cont.
•Developed by Scriven
•Much like needs assessments
•Functional analysis of product
•IMPORTANT to:
•IDENTIFY the criteria correctly
•DEVELOP standards to judge those criteria
•COLLECT data
•SYNTHESIZE information to make a final
judgment
13. Program-Oriented Approaches
•Focus on learning some key features
of the program, and then serve to
help the evaluator decide which
questions should be asked.
•Most common type: The objectives-
oriented evaluation
15. Tylerian Evaluation
•Steps:
•Establish broad goals or objectives
•Classify the goals or objectives
•Define the objectives in behavioral terms
•Find situations in which achievement of objectives can be shown
•Develop or select measurement techniques
•Collect performance data
•Compare performance data with behaviorally stated objectives
•Generally discredited today as
oversimplifying a complex system
16. Provus Discrepancy Evaluation
•Steps:
•Agree on standards (objectives)
•Determine whether a discrepancy
exists between performance and
standards
•Using that information, decide
whether to improve, maintain, or
terminate the program or some
aspect of it
17. Theory-driven evaluation
•Program theory is “the construction
of a plausible and sensible model for
how a program is supposed to work.”
•Programs fail to achieve goals for
two different reasons:
•The program isn’t delivered as planned and, therefore, isn’t
ready to be tested (implementation failure)
•The program is delivered as planned, and the results
indicate that the program theory was incorrect (theory
failure)
18. Theory-driven evaluation, cont.
•Steps:
•Engage relevant stakeholders
•Develop a first draft of program theory
•Present a draft to stake holders for
discussion, reaction, and input
•Conduct a plausibility check
•Communicate findings to key stakeholders
•Probe arrows for model specificity
•Finalize program impact theory
19. Goal-Free Evaluation
•The evaluator purposely avoids becoming aware
of the program goals
•Predetermined goals are not permitted to
narrow the focus of the evaluation study
•Goal-free evaluation focuses on the actual
outcomes rather than intended program
outcomes
•The goal-free evaluator has minimal contact
with the program manager and staff
•Goal-free evaluation increases the likelihood
that unanticipated side effects will be noted
20. Reference
•Fitzpatrick, J., Sanders, J., &
Worthen, B. (2011). Program
evaluation: Alternative approaches
and practical guidelines (4th ed.).
Upper Saddle River, N.J.: Pearson
Education.