ORIGINAL ARTICLE

Evaluation of the Master of Science in Epidemiology

(Clinical Epidemiology) Curriculum

Erlyn A. Sana1, Melflor A. Atienza1 , Jose Alvin P. Mojica1, 2 , Lupe F. Abarquez1 and Nemuel S. Fajutagana1
1
National Teacher Training Center for the Health Professions, University of the Philippines Manila
2
Department of Rehabilitation Medicine, College of Medicine and Philippine General Hospital, University of the Philippines Manila

and international sponsors. Alumni and students appreciated the

Master of Science in Epidemiology effectiveness of their teachers and complementing instructional
(Clinical Epidemiology) Program Updates resources. The range of work by its graduates, the networks
established, and the expansion of the Department of Clinical
Since the completion of the evaluation in 2002, the de- Epidemiology (DCE, the UP department offering the MSE-CE
partment has held four curriculum workshops to address program) prove that the MSE-CE is a sustainable post-graduate
the findings of the study. In 2004, the revised curriculum program.
has been presented and ratified by the University Council
and was subsequently approved by the Board of Regents. Conclusion: The MSE-CE institutionalized clinical epidemiology as
The major changes are: 1) reduction of core courses, 2) a distinct discipline among medical colleges in the Philippines.
increase in major courses, 3) re-classification of elective
courses into major courses, and 4) creation of new elec- Keywords: clinical epidemiology, curriculum evaluation, and context,
tive courses. The new curriculum has been in place for input, process and product (CIPP) evaluation
the past 5 years. Presently, we accept 8-11 students a
year, many of whom can avail of scholarships from the Introduction
Philippine Council for Health Research and Develop- Institutions genuinely interested in effecting change must
ment. Future plans to further refine the curriculum and subject themselves to continuous self-examination. It was with
other aspects of the program will be focused towards 1) this in mind that the Department of Clinical Epidemiology
a stepladder program 2) tracking options for biostatistics, (DCE) commissioned the National Teacher Training Center
health social science and clinical economics 3) offering the for the Health Professions (NTTCHP) in 2002 to evaluate
program independent from the program of the College of its 10-year-old Master of Science in Epidemiology-Clinical
Public Health 4) and development of learning modules Epidemiology (MSE-CE) curriculum.
with a goal to offer courses (and eventually the program) As external evaluator, NTTCHP gathers evidence that
on-line. the DCE can use in planning the future directions of the
program. Since the DCE serves as primary stakeholder, it was
Jacinto Blas V. Mantaring III, MD, MSc appropriate to use a management-oriented evaluation design,
MS Program Director, Department of Clinical Epidemiology,
specifically the context, input, process and product evaluation
UP College of Medicine
(CIPP) developed by Stufflebeam in 1969 and 1983.1,2 Context
evaluation helps program managers make planning decisions
such as what goals and objectives the curriculum should
Abstract have. Input evaluation deals with structuring decisions
Objectives. To evaluate the relevance, usefulness, applicability, specifically for determining the usefulness and relevance of
responsiveness, acceptability, efficiency, overall impact and the curriculum and the various courses offered, choosing
sustainability of the Master of Science in Epidemiology-Clinical which instructional resources must be available and which
Epidemiology (MSE-CE) curriculum. alternative strategies must be considered, and determining
the acceptability of program policies. Process evaluation
Methods. The context, input, process and product evaluation (CIPP) helps program administrators implement decisions like
design was used. Of 106 alumni and students, 100 were contacted determining how well the program is being conducted, what
and 80 participated in a survey. Key informant interviews, direct might threaten its success and what revisions are needed.
observation and focus group discussions with faculty members, Product evaluation deals with recycling decisions like what
present and past administrators, selected alumni and students, results the curriculum yielded, how well the needs were
and review of data were done. The results were analyzed using reduced and what should be done after the curriculum has
descriptive statistics and content analysis. run its course.
Results. Context evaluation revealed that MSE-CE responded to Objectives
the need to train physicians to use research evidence in clinical The MSE-CE curriculum was evaluated in terms of the
decision-making. Despite some reservations among other UP following: (1) relevance of the program to health research
colleges, the program pushed through due to support from local and clinical practice, (2) usefulness and applicability of each
course offered in relation to research and clinical decision
making, (3) responsiveness of program management to
Corresponding author: Erlyn A. Sana, PhD students’ academic and psychosocial needs and faculty
National Teacher Training Center for the Health Professions requirements, (4) relevance and acceptability of the policies
University of the Philippines Manila
Telephone: +632 521-0899 Telefax: +632 526-4259
and guidelines related to admissions, graduation, student
Email: erlynsana@yahoo.com.ph retention, thesis and other administrative matters, (5)

Vol. 43 N0. 4 2009 ACTA MEDICA PHILIPPINA 35

Evaluation of the Master of Science in Epidemiology Curriculum

teaching competencies and efficiency including adequacy Table 1. Evaluation matrix

of academic materials, teaching aids and other resources, (6)
Evaluation Objects CIPP Data needed Data
overall impact of the program on the students in terms of:
Domain collection proce-
(a) adequacy of skills in qualitative and quantitative research
dure
designs and critical appraisal, (b) attitudes and utilization
1. Relevance of the Context • Records and • Review of
of evidence-based clinical practice, (c) number and type of
program health statistics records
clinical or public health research involved in, (d) number of
• Perceptions of • Key informant
research consultancies and other related extension services,
faculty, alumni interview
(e) number of publications, (f) number of teaching/training
and students • FGD
programs in clinical epidemiology involved in as trainer and
2. Usefulness and appli- Input • Perceptions of • Review of the
(7) sustainability in terms of (a) the needs of research and
cability of each course faculty, alumni written cur-
educational institutions, teaching hospitals, and industry
offering and students of riculum of the
for training in clinical epidemiology, (b) needs for clinical
the usefulness MSE-CE
epidemiology training in Southeast Asia, and (c) donor
and applica- • Survey
interest in the program.
bility of the
program
Methods
3. Responsiveness of • Perceptions of • Review of poli-
The study involved two population groups, namely,
program management faculty, alumni cies on admis-
the faculty and students–both alumni and those currently
to students’ academic & students of sion, retention
enrolled. The 24 faculty members came from DCE, University
and psychosocial the relevance and graduation
of the Philippines College of Medicine (UPCM) and the
needs and faculty and accepta- of students
Department of Epidemiology and Biostatisics of the UP
requirements bility of policies • Review of
College of Public Health (UPCPH), and included program
4. Relevance and accept- policies on
administrators as well. Two were on leave reducing the
ability of the policies recruitment and
faculty population to 22. As of the first semester of 2002-2003,
and guidelines related promotion of
DCE had admitted a total of 106 students, six (6) of whom
to admissions, gradua- faculty
did not have any recorded forwarding addresses, thereby
tion, student retention, • Review of
reducing the accessible population to 100, of whom 80 agreed
thesis and other library collec-
to participate.
administrative matters tions and other
Data collection included the use of a survey questionnaire
instructional
(for alumni and students) on perceptions on the program’s
resources used
policies, courses, teaching-learning activities, resources
• Survey
used, assessment scheme to determine achievement, and
5. Teaching competen- Process • Performance • Review of
management. A series of key informant interviews (KIIs),
cies and efficiency ratings of performance
direct observation of classes, focus group discussions (FGDs)
including adequacy students and ratings of
and review of documents were used to triangulate the
of academic materials, faculty faculty
quantitative data. There were six FGDs and 24 KIIs completed
teaching aids and • Perceptions of • Key informant
with selected alumni, students, faculty, administrators and
other resources students and interview
leaders of partner agencies. The actual study took place from
alumni of their • FGD
June to December 2002. The names of the respondents were
experiences in • Direct observa-
coded for confidentiality.
the program tion
Data were analyzed using descriptive statistics and content
• Actual teaching • Survey
analysis. Patterns of response, frequency of occurrence
strategies used
and intensity of statements were considered in the content
in class
analysis. Table 1 presents the summary of the evaluation plan
6. Overall impact Product • Perceptions of • Review of
according to the objectives and CIPP design of the study.
7. Sustainability faculty, alumni, graduation rate
partner agen- • Analysis of
Results
cies, sponsors performance
Context evaluation
& students on ratings of stu-
The roots of MSE-CE curriculum were traced from the
the impact & dents & faculty
introduction of clinical epidemiology as a separate discipline
sustainability • KII
distinct from epidemiology, medicine and public health. In
the late 1960s, the Rockefeller Foundation (RF) concluded
that despite several initiatives from various institutions, the Centers (CERTCs) in developed countries with strong clinical
world health situation had not significantly improved.3 The epidemiology (CE) programs, and a network of Clinical
RF concluded that there was a need for physicians to directly Epidemiology Units (CEUs) in developing countries. INCLEN
focus on studying the course of diseases, the background organized the CERTCs and CEUs following the assumption
of their patients and the various factors contributing to the “that the establishment of CEUs in schools of medicine will
occurrence of diseases and use these for clinical decision have a favorable impact on the provision of effective and
making. These areas served as the distinguishing features of efficient systems of health care which are appropriate for
clinical epidemiology. In July 1979, RF created the International the health status of the population served by those medical
Clinical Epidemiology Network (INCLEN), the body that schools by:
would carry out the task of developing clinical epidemiology 1. Educating within a clinical setting, physicians to use
as a science. interventions proven to be efficacious,
Then, by virtue of a board resolution drafted by the RF 2. Educating within a clinical setting, physicians to
in 1982, INCLEN created and established partner agencies, establish arrangements for providing effective care
namely, the Clinical Epidemiology Resource and Training efficiently and

36 ACTA MEDICA PHILIPPINA Vol. 43 N0. 4 2009

 Evaluation of the Master of Science in Epidemiology Curriculum

3. Encouraging (as a result of 1 & 2) a more rational Table 2. Professional roles and minimum terminal competencies of
approach to the allocation of resources for medical care MSE-CE graduates
in relation to the health status of the population.”3
INCLEN established the Clinical Epidemiology Unit of the Professional role Minimum terminal competencies
University of the Philippines College of Medicine (UPCM) 1. Clinician Deliver the best health care and relate to individual
in 1984. One respondent recalled the process as follows: patients
“In the beginning, Rockefeller wanted the unit (referring to 2. Researcher Show competence in research proposal writing,
CEU) to work with clinicians—people who practice medicine, grant application, project implementation and pres-
those who really see patients … because they are the ones entation in written (technical report, journal article,
who should be given this expertise so that they can effect the report), or verbal forms (speaker in conferences)
change they want. Then initially, there were 26 units all over 3. Consultant Make critical review of research proposal for grants
the world in all major continents. These 26 units are CEUs. application and technical committees; implements
The long-term plan was to let them evolve into CERTCs so project and trials for drug companies and other
that they can propagate it (referring to CE) themselves and agencies; serves as judge in research contests, journal
they would be responsible to train the next generation of editor or reviewer; advocate in health reforms within
clinical epidemiologists. All of these happened, actually.” the levels of affiliations with a hospital, specialty so-
CEU in UPCM evolved as a unit under the Office of the ciety or department
Dean of UPCM from 1984 to 1989. The RF transformed it into 4. Administrator / Serve as administrator, manager or coordinator of
a CERTC in 1991. With the offering of the MSE-CE in 1992, Manager / any private or public bureaucracy
the new CERTC based in UPCM became a degree-granting Coordinator
unit of the University. Then in 1998, the UP Board of Regents 5. Educator Serve as teacher in clinical epidemiology and any
approved the change of its name to the Department of Clinical medical specialty in both the undergraduate and
Epidemiology. graduate levels of medical education; adviser in edu-
As the RF and CEU in UPCM had to strongly justify with cational reform or curriculum development projects
the external environment and the international academic 6. Learner Assume responsibility for own continuing educa-
community the need for a separate discipline to the point of tion
labeling it as a “schism” from public health epidemiology, 7. Leader / initiator Advocate for the promotion of clinical epidemiology
the MSE-CE as expected went through some form of testing. as distinct field of study
The program was perceived as a redundancy of the Master
of Science in Public Health Major in Epidemiology (MSPH- 5. Appreciate the roles of both economics and social
Epi) which was a curricular offering of the Department sciences in making health interventions more efficient
of Epidemiology and Biostatistics (DEBS) of UPCPH. and acceptable.
But cognizant that the offering of MSE-CE had the full The relevance of the MSE-CE at DCE was strengthened
endorsement of the Chancellor, CPH gave way. when full-time scholarships were pledged by the Rockefeller
When the Board of Regents approved the MSE-CE, CPH’s Foundation through INCLEN and the Philippine Council
MSPH-Epi program was abolished and the college was tasked for Health Research and Development (PCHRD) since the
to reorganize it into the MSE-Public Health Epidemiology beginning of the program.
(MSE-PHE) track. To ensure that the new MSE-CE would
not be a redundancy of the MSE-PHE, the then CEU, in Input Evaluation
collaboration with other CERTCs in Southeast Asia, met for The MSE-CE Curriculum. This program is a 32-unit
a workshop in 1993 and agreed that MSE-CE graduates will curriculum composed of 11 units of core courses, nine (9)
play the professional roles and competencies listed in Table units of major, six (6) units of electives and six (6) units of
2.4,5 thesis. CPH handles nine (9) out of the 11 units of core
During the same 1993 conference, these terminal because, as explained by five CPH faculty members “these are
competencies were broken down into the following general all CPH courses.” DCE handles only one core course: CE 201:
course objectives:3 Fundamentals of Health Economics, Health Social Science and
1. Demonstrate skills in the application of epidemiologic Research Ethics. Students can take the six (6) units of electives
concepts and principles to the resolution of clinical and from CPH, School of Economics or the Statistical Center in UP
public health problems; Diliman Campus.
2. Identify, plan, undertake, analyze and interpret clinical In 1997, the National Graduate School for the Health
or public health research projects; Sciences (NGSHS) was established and facilitated the
3. Deliver technical services to clinicians or public health application procedure to the program, the documentation of
workers on how to: the progress of students and the completion of requirements
a. Properly identify factors in disease causation; for graduation. In 2002, MSE-CE was being handled by
b. Evaluate the reliability and validity of instruments; the department led by a program director. The faculty is
c. Determine the efficacy and effectiveness of composed of one full-time faculty member and 24 part-time
interventions; faculty members from UP Manila’s Colleges of Medicine
d. Plan strategies for disease control/prevention; and Arts and Sciences, the National Institutes of Health, and
e. Devise methods for evaluating health technology/ the Philippine General Hospital; and a consultant from the
programs and; Philippine Health Insurance Corporation. Out of the 24, five
f. Provide guidelines for research activities whose had original appointments with the department; the rest were
results could be the basis for health policy cross-appointees. There were no official recruitment policies
formulation for faculty members. Support personnel included one full-
4. Develop a critical attitude in evaluating scientific time secretary and one messenger.
literature and information in the management of health Policies. Students and graduates were asked to rate the
problems and policies of the program in terms of their perceived degree of

Vol. 43 N0. 4 2009 ACTA MEDICA PHILIPPINA 37

Evaluation of the Master of Science in Epidemiology Curriculum

acceptability considering their current professional positions Table 4. Courses identified by students and alumni to be most
and clinical responsibilities in their respective institutions. A and least helpful
rating scale from 1 (not acceptable) to 4 (most acceptable) was
Area Helped Most Helped least
used. Table 3 presents the ratings. The figures showed that the
Research CE 211: Fundamentals of Pharma 250: Introduc-
policies on student admission, retention, thesis and graduation
undertaking Clinical Epidemiology tion to Pharmacoepi-
were all highly acceptable. The high level of acceptability
CE 212: Basic Research demiology
of the policies for admission and retention showed that
Methodology Epi 201:Fundamentals
students and alumni already appreciated the unique place of
Biostat 201: Fundamentals of of Epidemiology
physicians engaging in clinical epidemiology work. On the
Biostatistics I
other hand, the policy on maximum residence rule (MRR)
Epi 204: Study Designs in
was rated 2.82. All master’s degree students in UP Manila
Epidemiology
are allowed a maximum of five school years to complete their
Clinical decision CE 213 : Critical Appraisal of Pharma 250: Introduc-
programs. Those who reach their fifth year without finishing
making Medical Literature tion to Pharmacoepi-
all the requirements for graduation are labeled as “on MRR”.
CE 211: Fundamentals of demiology
These students are required to apply for an extension of their
Clinical Epidemiology Biostat 202: Fundamen-
MRR and take a three-unit refresher course.
CE 201 : Fundamentals of tals of Biostatistics II
Instructional facilities and resources. Facilities for instruction
Health Economics, Health
were supplied by the UPCM and generous grants from the
Social Science and Research
RF and INCLEN. As of 2002, DCE library had a rich collection
Ethics
of local and international materials on clinical epidemiology,
Evidence based Medicine
statistics and health social science. The library subscribed to
Teaching/training CE 213: Critical Appraisal of Pharma 250: Introduc-
16 international journals and updated their collections from
activities Medical Literature tion to Pharmacoepi-
the World Health Organization (WHO). All the instructional
CE 212: Basic Research demiology
facilities like overhead projectors, slide projectors and tape
Methodology
recorders were supplied by INCLEN. There were nine
CE 211: Fundamentals of
computers provided by UPCM, INCLEN and other donors.
Clinical Epidemiology
Students were asked to name the most and least useful
subjects. The responses are summarized in Table 4.
fixed schedule of activities for accepting application forms and
Relevance to students’ needs. Courses were evaluated by
screening applicants, including administering the qualifying
students according to the degree to which students agreed
examination, conducting the panel interview and notifying
with given statements (1 for strongly disagree to 5 for strongly
applicants of admission on a given school year. Students
agree). Most courses were rated favorably by the students.
and alumni were asked their perceptions on the MSE-CE
In all of these, the highest rating was consistently awarded
program management. A scale of 1 (least favorable) to 5 (most
to the relevance of the subject matter to the students’ needs.
favorable) was used. Table 5 presents these perceptions.
This is especially true with CE 211: Fundamentals of Clinical
The lowest rating was given to responsiveness to students’
Epidemiology, where the mean rating for relevance was 5.0.
psychosocial needs. Graduates explained in the FGD that
the schedule of classes was rigid and the MRR requirements
Table 3. Perceptions of alumni and students of the acceptability of
MSE-CE policies (n=80)
added more burden and pressures on them.
Students. Since 1992, a total of 106 students have been
Policies Perceived degree of acceptability admitted to the program. At the time of admission, their ages
Mean (SD) Minimum- ranged from 26 to 68 years old (mean = 41.56), which places
Maximum ratings them at the mid-career and mid-adulthood stages. Thirty-five
1. Qualification for admission 3.51 (0.57) 2-4 (43.8%) are male and 45 (56.3%) are female. All graduates
2. Requirements for admission 3.50 (0.60) 2-4 have a medical degree except one student who is a BS Nursing
3. Criteria for selection of students 3.58 (0.55) 2-4 degree holder. Thirty-seven percent (37%) financed their
4. Grade requirements 3.39 (0.68) 1-4 own education; the rest were financed by PCHRD, INCLEN,
5. Coursework of 35 units 3.43 (0.71) 1-4 and other agencies. With the withdrawal of support from
6. Thesis as a requirement for graduation 3.17 (0.87) 1-4 INCLEN/SEACLEN in 1998, most students started to finance
7. Option for diploma 3.35 (0.77) 1-4 their own education. Although the lack of funding for their
8. MRR students to take additional units 2.82 (0.95) 1-4 tuition was not a problem, lack of budget for their thesis work
was cited by some alumni as one difficulty they faced.
Actual delivery of the curriculum. Direct observation revealed
On the other hand, generally low ratings (Mean=3.76) were that classes in CEU involved less than ten students of the MSE-
given to CE 201, which was composed of Health Economics, CE program and the format was still 75 to 100 percent lecture.
Health Social Science and Research Ethics. The faculty has These followed the outlines provided by the syllabus.
difficulty handling this particular course as each part is In some classes, students were also asked to work on cases
only briefly covered. The students felt that CE 216: Medical and discussion followed. Student output would be presented
Informatics, which received the lowest rating (Mean=3.86), before the class and the other students would give their
was relevant but did not meet the objectives. The students’ critique. Every lecturer passed around the attendance sheet
assessment of their courses was lower during the first years of and asked the students regarding their interests in research.
the program. In general, they rated their courses in 2001-2002 This suggests that teachers who happened to be the senior
favorably. ones were not familiar with the students, probably since they
met the class only for specific topics and not regularly for the
Process evaluation entire quarter, unlike preceptors.
Program management. With the guidance of and in As for the courses taken in the College of Public Health,
coordination with the NGSHS, the department followed a MSE-CE students joined the students of Public Health; hence,

38 ACTA MEDICA PHILIPPINA Vol. 43 N0. 4 2009

 Evaluation of the Master of Science in Epidemiology Curriculum

Table 5. Means and SDs of perception of MSE-CE Program Figures show that students and alumni have generally
Management (n=80) favorable perceptions of the adequacy and accessibility of
Items on program management Mean (SD)
basic instructional facilities in DCE. Modules and handouts
are rated highest because as respondents described: “they are
1. Responsiveness to students’ academic needs 4.03 (0.75)
already provided for in separate folders per subject for specific
2. Responsiveness to students’ psychosocial needs 3.80 (0.89)
prices.” This also explains why the books had relatively lower
3. Execution of basic procedures, e.g., admission, etc. 4.01 (0.77)
ratings. Respondents explained: “journals are adequate but
4. Administrative staff easy to deal with 4.58 (0.66)
only a few are accessible because most of them are not in the
5. Program director easy to deal with 4.55 (0.62)
UPCM library; what are available are only the abstracts,”
and “computers were lacking.” They explained further that
there were more than fifty students in each class. Lectures during small group activities, “after the grouping, those from
were held in the College of Public Health Auditorium. Again, PH [Public Health] go back to PH and use CPH computers,”
these were traditional lectures during which any student and “those of us from CE go back to CEU.”
could ask questions. Laboratory exercises were answered in
separate rooms where they grouped themselves in 20s and Product evaluation
solved problems individually. Answers to exercises were Overall impact. As of the end of 2002, 37 students had
discussed later. graduated and 69 students were still enrolled. Out of the 106,
According to alumni, prior to the start of classes, a pretest 5.66% graduated within four years; the rest of the alumni
was given to students to determine their level of competence finished in five to six years. Alumni and students currently
in statistics. Those who were deemed in need of closer enrolled were asked to assess their own levels of competence
guidance were grouped as “intensive care unit” and tutorials in terms of 23 research skills. The response format was a
were regularly held for these students outside of class hours. four-point Likert-type rating scale ranging from 1 (needs
This was a welcome thing for these students since they were improvement) to 4 (outstanding). Table 7 lists those areas
aware that they needed help. Due to the discrepancy in the where respondents reported mean ratings of ≥3.0 indicating
competence of students, some of them found lessons boring average competence.
because the teachers were adjusting to the slower students. Items 2, 3, 6, 8, 10 and 13 in Table 7 compose the construct
The students also found the teaching resources and activities on research conceptualization and design. The overall mean
appropriate to the objectives. Computer work, lectures and rating is 3.26. Items 1, 5, 9 and 12 compose the second construct
small group discussion were the most extensively used on data analysis and interpretation with an overall mean
strategies. Exercises, workshops, small group discussions and rating of 3.28. Items 4, 7 and 11 for research management and
computer work were the most useful to them. They, however, ethics yielded the lowest rating of all average competencies
suggested more hands-on experience in browsing the net, with a mean of 3.21.
use of handheld or portable computers and constructing More than half of all the respondents were attached to
a database could improve the course. This was echoed by a medical school as faculty members. In particular, alumni
alumni and even faculty members who felt students lacked from the University of Santo Tomas, the De La Salle Health
exercises with software especially in nonparametric statistics. Sciences Institute and the Cebu Institute of Medicine became
Students were not looking for a lighter or an easier time as the pioneer members of their respective institutions’ CEUs.
they went through the program. They were in fact asking Table 8 presents contrasting results. Alumni and
for more exercises to hone their skills in protocol writing, students reported the areas where they thought they needed
statistical analysis, presenting a paper and teaching research. improvement.
One problem identified was the wide gaps in the schedule Sustainability. From 35 accomplished curriculum vitae, a
of classes. During the interviews, respondents expressed total of 71 research papers completed by students and alumni
the seeming lack of coordination between the content and were recorded. Respondents also played various roles in
handling of the course in CPH and DCE. each of these researches, namely: being chair or member of
Teachers. Students rated their teachers’ characteristics Table 6. Perceptions of graduates and students on the adequacy
and performance from 1 to 5, the latter being the highest and accessibility of instructional resources in the MSE-CE program
rating. Respondents perceived their teachers to possess the (n=80)
competence necessary to teach their respective courses in
Adequacy Accessibility
epidemiology (mean = 4.46, SD = 0.68). Lower ratings were
given for availability for consult (mean = 3.75, SD = 0.87), Mean (SD) Mean (SD)
Books 3.50 (1.09) 3.87 (1.08)
punctuality (mean = 4.07, SD = 0.73), attendance (mean
Journals 3.21 (1.18) 3.36 (1.17)
= 4.09, SD = 0.77) and being inspiring (mean = 4.04, SD =
Classrooms 4.10 (1.07) 4.34 (0.80)
0.87). The numerous other responsibilities of teachers might
Audio-visual facilities 3.83 (1.13) 4.07 (1.03)
explain why they were not always around for their students.
Handouts 4.34 (0.81) 4.53 (0.71)
The student ratings of faculty for academic year 2001-2002
Modules 4.22 (0.85) 4.40 (0.76)
also reflected these findings. During the FGDs, respondents
Computers 3.22 (1.19) 3.41 (1.13)
complimented the CPH faculty members as experts since they
were chronologically more senior and described DCE faculty
members as novices in an equally new program. their institution’s or specialty society’s research committees;
Instructional resources. Alumni and students currently editor or member of the editorial staff of research-oriented
enrolled were asked to rate the instructional resources using publications; director, assistant director or member of their
two constructs, namely, perceived degrees of adequacy school’s CEU; resident research coordinator; technical
and accessibility. The rating scale used ranged from 0 (not consultant of the PCHRD; clinical research coordinator
applicable), 1 (not adequate/not accessible) to 5 (adequate/ or faculty members of research or clinical epidemiology.
accessible). The mean ratings, range of scores, and standard However, a DCE faculty and a DCE student also admitted
deviations (SD) are presented in Table 6. that a great number of students and alumni remained users

Vol. 43 N0. 4 2009 ACTA MEDICA PHILIPPINA 39

Evaluation of the Master of Science in Epidemiology Curriculum

rather than doers of research. DCE students, during the Table 8. Research skills at which respondents need improvement
FGD admitted that “We don’t have time to concentrate in (n=80)
research … we are primarily physicians and our immediate
Research skills Mean (SD)
preoccupation is taking care of our patients.”
Alumni and students were asked to rate the MSE-CE 1. Conduct of key informant interviews, FGDs, observa- 2.87 (0.72)
curriculum in terms of overall relevance. On a scale of 1 to 5 tions and other qualitative methods
(least to most favorable score), the MSE-CE was found to be 2. Cultural appropriateness of interventions 2.80 (0.80)
relevant to the health needs of the country (mean = 4.68, SD 3. Planning of strategies for disease control / provision 2.75 (0.81)
= 0.57), to health practice (mean = 4.83, SD = 0.41), useful in 4. Use of appropriate statistical analysis 2.66 (0.81)
making sound clinical decisions (mean = 4.83, SD = 0.41) and 5. Device methods for evaluating health technology / 2.65 (0.91)
useful in the conduct of health research (mean = 4.88, SD = programs
0.40). 6. Designing questionnaire 2.63 (0.75)
Stakeholders and partner agencies were asked about the 7. Statistical computations (using at least one computer 2.61 (0.99)
sustainability of the MSE-CE program. The Department of software)
Health (DOH) confirmed that the said curriculum “is a much 8. Determination of sample size 2.53 (0.78)
needed one.” Over the last 10 years that the MSE-CE program 9. Formulation of health policies based on evidence 2.53 (0.81)
was being offered, DCE has been coordinating with the 10. Cost analysis of health programs and interventions 2.31 (0.88)
following institutions: The PCHRD in both national and local
levels, the National Research Council of the Philippines, DOH, to slowly build the professional roles and competencies of
research committees and CEUs in three different sites, the physicians to become evidence-based in their practice, and, at
Philippine, Asian and regional CEUs and CERTCs and most of the same time, become advocates, trainers and teachers of the
the students’ and alumni’s research committees and societies. value of research in their respective institutions.
While the initiative and push to introduce the MSE-CE
program came from the outside, it became part of the existing
Table 7. Research skills with average competence by self-assessment
(n=80)
structure in the University of the Philippines Manila. Most
of the core courses were taken from the College of Public
Research skills Mean (SD) Health, administrative and faculty items were taken from
1. Proper identification of factors in disease causation 3.66 (0.66) other colleges and units, while facilities and physical plant
2. Identification of the research problem 3.53 (0.66) for instruction did not really entail new capital outlays and
3. Knowledge and synthesis of related literature 3.45 (0.57) fresh releases of funds and the existing university rules
4. Advocate health policies 3.42 (0.85) surrounding graduate programs were adopted. The College
5. Determine the efficacy and effectiveness of interventions 3.35 (0.72) of Medicine added a new program but basically worked on
6. Doing a research proposal 3.33 (0.63) the same structures.
7. Knowledge of ethical issues in research and clinical 3.26 (0.63) Specific management decisions could be drawn out from
practice the CIPP evaluation findings. Context evaluation proved
8. Knowledge of appropriate research design 3.25 (0.54) the program’s determined position to prepare clinicians for
9. Critically evaluate research information 3.20 (0.08) decision-making, thereby affirming the need for the program
10.Developing a conceptual framework 3.11 (0.69) to continue and recruit as many students as possible. Input
11.Research management 2.95 (0.83) evaluation provided evidence that, internally, within
12. Evaluation of the reliability and validity of measure- 2.93 (0.76) the college and the university, the program served as an
ments additional program and operated on the same existing
13. Knowledge of sampling procedures 2.90 (0.64) structures. External support was adequate to recruit students,
enough for them to temporarily leave their respective clinical
All these organizations expressed their unconditional support practice. This implies sensitive management decisions should
to the MSE-CE program. A case in point is the case of PCHRD the external support be terminated. Process evaluation
that introduced the “twinning project.” This schema matches showed that the MSE-CE was appreciated because of the
a center of excellence (the older twin) with a region-based combined dedication and competence of the faculty, staff,
satellite institution (younger twin) so that the former may administrators and their external partners. However, there is
help the latter devote its research expertise.a A DCE alumna plenty of room left to improve the actual research competency-
explained that her institution’s CEU conducted seminars and building of the students not only in terms of appreciating
workshops on health research for clients in Region 6. Same the program, judging research contests and being users
alumna explained: “The experience was a different kind of of research evidence. There was an apparent need to make
high because we get the chance to be like our teachers.” After the candidates consistent doers of research as well. Product
10 years of implementation, it could then be said that the MSE- evaluation proved to be the most significant area where the
CE is multiplying itself and is now succeeding to produce the MSE-CE program could claim success. The program did not
next generation of the country’s clinical epidemiologists. just create an academic discipline; it also paved the way for
other service agencies in government to be evidence-based.
Discussion By the constant supply of research material provided by both
The introduction of the Master of Science in Epidemiology- faculty and students, the Departments of Health, and Science
Clinical Epidemiology track in the University of the Philippines and Technology through the Philippine Council for Health
College of Medicine was an offshoot of the development of Research and Development, among others, admittedly grew
clinical epidemiology as a distinct academic discipline in the and blossomed as mature institutions in terms of handling
Philippines. It was founded on the clear need for physicians data and formulating relevant policies.
to use research evidence in their clinical decision-making.
And because the target students were busy adult learners, Conclusions and Recommendations
the curriculum was especially designed to be competency- The Department of Clinical Epidemiology introduced and
based. The courses and the program of study were organized nurtured Clinical Epidemiology as an academic discipline

40 ACTA MEDICA PHILIPPINA Vol. 43 N0. 4 2009

 Evaluation of the Master of Science in Epidemiology Curriculum

through its MSE-CE curriculum. Perceived at the start as a recommended that the DCE conduct periodic curriculum
redundancy of public health epidemiology, it grew into a evaluation. Courses identified in this study as relevant or
unique field of specialization through the combined support of otherwise, interesting or otherwise, should be reconsidered.
faculty, students, UP Manila, the Rockefeller Foundation and Among others, it is important to monitor retention, attrition
other partner agencies like the Philippine Council for Health and graduation among students. Since faculty advising and
Research and Development. The program was appreciated for thesis support are critically important to ensure quality, it is
building the competence of physicians in appraising evidence further recommended that continuous resource generation
for clinical decision-making. Amidst their busy schedules as is done to have additional faculty and staff items, and thesis
clinicians, the faculty, graduates and students of the program grants.
have become researchers and program managers ensuring
the sustainability of the discipline. References
The MSE-CE curriculum proved to be a modest success of 1. Sana, EA, Atienza, MA, Abarquez, LF, et. al. Teaching and Learning in the
institution building. Started as a vision of an administrator Health Sciences. In Press.
who challenged the status quo, the program became a 2. Worthen, BR and Sanders, JR. Educational Evaluation: Alternative
Approaches and Practical Guidelines. New York: Longham, 1987.
reflection of how various educational ingredients of planning 3. White, KL. Healing the Schism: Epidemiology, Medicine and the Public’s
with a particular context, generating appropriate and adequate Health. New York: Springer-Verlag, 1991.
inputs, maintaining competent and committed faculty and 4. Tan-Torres, TL. “Formative Evaluation of the Master of Science in
staff, and regular monitoring of outcomes and products should Epidemiology-Clinical Epidemiology.” Major Project for the Degree of
Master in Health Professions Education. National Teacher Training Center
be blended at various stages of program development. As an for the Health Professions, University of the Philippines Manila. 1995.
educational product, it is undergoing a continuous process of 5. Philippine Council for Health Research and Development. Annual Report,
refinement, slowly gaining ground as an academic discipline. 1999.
To ensure continuous program quality and relevance, it is

Dreams can come true

A Commentary on the Evaluation of the Master of Science in Epidemiology

(Clinical Epidemiology) Curriculum
By
Professor Richard F. Heller

Professor Richard Heller served as Director of the Centre for Clinical Epidemiology and Biostatistics
(CCEB) of the University of Newcastle, Australia for more than 15 years. The CCEB is one of the first clini-
cal epidemiology and research training centers (the only one in Australia) established by the Rockefeller
Foundation in the early 80’s. Prof. Heller is currently Emeritus Professor of the Universities of Manchester,
UK, and Newcastle, Australia and Coordinator of the Peoples-uni (http://peoples-uni.org;http://courses.
peoples-uni.org/)-- an open access education initiative.

In the late 1970’s, the Rockefeller Foundation realised that clinicians were important instruments of change in health
policy, but were not trained in the population sciences so unable to provide a Public Health perspective to their policy
advice. The International Clinical Epidemiology Network (INCLEN) was born, to train clinicians in what subsequently
became Evidence Based Practice. The University of the Philippines, Manila, was selected as a site for the training of mid-
career clinicians in the first round of training, and a number of bright young academics went overseas to take Masters
courses in Clinical Epidemiology. The dream was that these people would then become change agents themselves, both
in the practice, advocacy, policy role and teaching. This paper shows how the dream has come true in the Philippines.

The genesis of a new Masters course is described carefully and the paper shows how best practice evaluation was
performed using a ‘Context, Input, Process and Product evaluation (CIPP)’ design. Interviews, observations and focus
groups amongst the relevant players provided information which not only led to course improvements, but was able
to be used to demonstrate the value of the course. As a result, as shown by the postscript, the University has supported
the course, which is now sustainable. Because of this, the high quality of those initially trained, and through the training
of the next cadre of clinical epidemiologists, the whole discipline has become institutionalized as a distinct discipline
among medical colleges in the Philippines. Graduates play important roles in health policy formulation for the Philip-
pines and internationally. This story is important to be told, and I applaud the authors and the Journal for this publica-
tion.

Vol. 43 N0. 4 2009 ACTA MEDICA PHILIPPINA 41

Evaluation of the Master of Science in Epidemiology Curriculum

Mapping of MS Epidemiology (Clinical

Epidemiology) graduates and students
The M.S. Epidemiology-Clinical Epidemiology has trained more than 150 health professionals
from various health institutions in the Philippines, Indonesia and China. The following map
shows their areas of practice in the Philippines. While there are a number who are practicing
in the provinces (including the provinces in Visayas and Mindanao), majority (82%) are based
in Metro Manila. To address this inequity, the Department of Clinical Epidemiology at the
University of the Philippines is partnering with the Philippine Council for Health Research and
Development to attract health professionals based in areas outside of Metro Manila. One major
hurdle identified is the difficulty of uprooting a clinician from his area of practice for a year (the
program requires one year of course work in Manila). It is within this context that the long-
term plan of offering selected courses and eventually the program by distance learning was
envisioned.

42 ACTA MEDICA PHILIPPINA Vol. 43 N0. 4 2009