E-portfolio based learning: Implementation and evaluation Nahid Zarifsanaiey1, 2, Samieh Etemadi3, Rita Rezaee4* 1 Department of e-learning, virtual school, center of excellence for e-learning in medical sciences, Shiraz University of medical sciences, Shiraz, Iran. 2 virtual university of medical Science, Tehran, Iran. 3 Student research committee, Shiraz University of medical sciences, Shiraz, Iran. 4 Clinical education research center, health human resources research center, Shiraz university of medical sciences, Shiraz, Iran Correspondence: Rita Rezaee, clinical education research center, health human resources research center, Shiraz university of medical sciences, Shiraz, Iran. Email: rita_rezaee@yahoo.com ABSTRACT Across the various study programs, portfolios serve many functions and purposes. A customizable electronic portfolio (e-portfolio) that facilitates sharing different parts of e-portfolio and provides across-referencing capabilities was developed at the Shiraz University of Medical Sciences (SUMS) in 2015. This paper describes the design of the e-portfolio and its effects on the learning outcomes and attitude of the students towards evaluation by e-portfolio. Material and Method This was a single-group quasi-experimental study with pretest-post-test. The sample size was 33 students of the postgraduate Medical students. A pretest was done to evaluate the students’ knowledge level. The students were evaluated for two months during the semester by the electronic portfolio. At the end of the semester, post-test was performed and their attitudes toward the evaluation by electronic portfolio were assessed. Validation of the electronic portfolio was assessed by 10 experts on technical, functional and educational criteria’s by a questionnaire. Responses from the experts indicated that the quality of electronic portfolio was acceptable. The obtained results of analysis of pair t-test revealed that the post-test scores were significantly more than the pretest (p=0.027). Results showed that the student's had a positive attitude towards evaluation through electronic portfolio. There was no significant linear relationship between the level of knowledge and attitude of research samples. The correlation between the students’ reflection on learning and post-test scores was significant, but in other areas (pre-test and attitude), there were no significant correlation (p<0.001). Conclusion Use of e-portfolio can have a significant impact in improving the education system and e-learning courses. Keywords: E-Portfolio, Evaluation, Design, Knowledge, Attitude. Introduction In the 21st century, new and innovative teaching approaches are crucial and vital elements in developing high quality human capital[1, 2]. The use of portfolio as a potential tool could help to develop higher order thinking skills and revolutionizes the process of teaching[3, 4]. McGraw-Hill higher education webpage defines portfolio as: "Objects and methods of assessment ". As objects, they are a place for holding materials such as; papers, photographs, or drawings that are representative of students' work and progress. As methods of assessment, portfolios provide ways for teachers to continuously collect and assess student work[5]. Access this article online Website: www.japer.in E-ISSN: 2249-3379 How to cite this article: Nahid Zarifsanaiey, Samieh Etemadi, Rita Rezaee. E-portfolio based learning: Implementation and evaluation. J Adv Pharm Edu Res 2018;8(S2):170-175. Source of Support: Nil, Conflict of Interest: None declared. In most traditional classes students are asked to work and learn within a very limited time frame[6]. This provides little or no opportunity to think deeply and reflect on what they have learned. Students are forced to submit their assignments without any reflective practices. Thus, a portfolio provides the opportunity to reflect on what has been learned, and both students and teachers can evaluate the progress during a course. Paulson and Meyer view portfolio as an laboratory experiment to construct meaning[7], which engage students in selfreflection[8]. Cooper and Love described two types of portfolios: formative portfolio (focus on the process of learning for each student), and summative portfolio (focus on learning outcomes)[9]. The first one demonstrates changes over a period of time and the second one contains proof of a student’s skills. In the past decade application of e-portfolio in higher education has increased, especially in the area of education with emphasis on learning and assessment[10]. Electronic portfolio is a learning and assessment tool. It is a personalized web based collections of content and reflection[11]. Using e-portfolio in health care education as a learning and assessment method, help students to reflect on their This is an open access journal, and articles are distributed under the terms of the Creative Commons Attribution-Non CommercialShareAlike 4.0 License, which allows others to remix, tweak, and build upon the work non-commercially, as long as appropriate credit is given and the new creations are licensed under the identical terms. © 2018 Journal of Advanced Pharmacy Education & Research | Published by SPER Publication 170 Nahid Zarifsanaiey and et al.: E-portfolio based learning: Implementation and evaluation experiences and could be used as an administrative tool to organize their work[12]. Reflection is a cyclic process of self-regulation that students revise and think about their work and actions[13]. Stimulating reflection needs some certain conditions such as good introduction, clear purpose and appropriate structure, it means students learn from their own experiences[14]. In general, the purpose of educational portfolio is to systematically organize and manage evidences at three levels or domains: curriculum of the program, instruction and assessment, and students’ reflection on learning. Like any other institution, Shiraz University of Medical Sciences is faced with several challenges due to increasing enrollment and need to support student’s learning. As Batson has argued eportfolio has great potential to alter higher education[15]. In this study we used e-portfolio as a digital tool to collect work (artifacts), which demonstrates students’ ongoing effort and growth (development) in research methodology area. Methodology Study design The study employed a quasi-experimental single group design (before-after-design) in order to examine students’ improvement in specific defined learning outcomes through eportfolio. Since random selection was not possible this design was chosen. Total of 33 students were assigned to the course by the e-learning department based on their needs to present a research proposal in order to graduate. The students experienced learning through a student-center approach using e-portfolio, which allowed them to take control of their learning process. The students had access to the free online course management tool (Mahara, which is localized based on the university LMS and is explained bellow), and the instructor directed them to various online lessons (subject matters). Each student created their own learning artifacts and had the opportunity to reflect on them. The educator acted more as a guide or facilitator rather than someone who delivers information and provided feedback on students’ artifacts (assignments). Participants In this study a total of 33 students, 26 female and 7 male from postgraduate medical education students who were in their final semester and were enrolled in Research in Education course. Data Collection Data sources included pre-test and post-test scores. All students received the pre-post tests. The researcher graded the pretest in order to assess the students’ knowledge of the subject. Then the researcher administered the instruction and provided feedback. Finally, the post-test was conducted and graded by the same researcher. It is worth noting that the results were anonymous. Content validity of the tests was confirmed by 5 experts in the field of research methodology and Kuder Richardson index (0.78) confirmed its reliability. Means and standard deviation of the pre-post test scores is reported in table 1. Design stage of e-portfolio At the design phase some value-added features were considered and added. These features consist of: customizable, multiple structure, ability to share and interact with supervisors and peers, ability to search, integration to LMS, ability to transfer data, download records, back up and reduce storage requirements. These features facilitate sharing different parts of e-portfolio and provided cross-referencing capabilities. The system that seemed to meet our requirements was Mahara, which was compatible with the university virtual settings. Mahara is an open source e-portfolio. However, there were running costs such as maintenance and upgrades. Customization of the e-portfolio was provided after major challenges and improvement in resources. Finally, it was integrated into the university virtual learning environment (LMS). At the second phase, a structured student centered learning outcomes were designed. In this approach the teacher’s role was to help students think through by using the following steps: 1) The learner identified and described the experience (assignment), and the teacher listened actively to encourage, not judge or make any interpretation. 2) The learner reflected on what was learned and the teacher clarified and summarized. 3) The learner identified new learning needs and their plan to meet these needs, and the teacher facilitated their plans. At the end of the program all intended and unintended learning outcomes were reviewed. The evaluations of the e-portfolios were based on the following criteria: - Educational: • • value added to students’ learning Sharing material and learning resources • • Learning activity Reflective work (blogs, comments on assessments, response to peers and faculty reviews) - Functional: • • Ease of use for students and supervisors Ability to search keywords • Ability to bookmark educational materials - Technical: • Integration and ability to transfer data from and to the university LMS Evaluation of e-portfolio (content validity) was confirmed by 10 experts in electronic learning, medical education and IT (CVI= 0.7, CVR=0.8). All students were informed about the purpose of using eportfolio. It was emphasized that e-portfolio is being used as a formative teaching tool and focuses on students’ development through the use of self evaluation and reflection. Journal of Advanced Pharmacy Education & Research | Oct-Dec 2018 | Vol 8 | Issue S2 171 Nahid Zarifsanaiey and et al.: E-portfolio based learning: Implementation and evaluation Interventional stage In this stage researcher played the role of facilitator to provide necessary instructions and encouraged them on reflection. The course consisted of four 3-h sessions related to the subject matter. During each session different part of the subject was discussed and at the end of the course, students were asked to complete four learning activities (assignments). Students could record and reflect on their learning in writing proposal skills. Researcher provided feedback on assignments and checked their progress. In this study, the following ethical issues were taken into consideration: After obtaining a written permission from the college officials, the training program began at the Education Development Center, Shiraz University of Medical Sciences. At the beginning of the training program after the researcher had introduced herself, she explained the objectives and the need to implement e-portfolio to the students, and a written informed consent was obtained from the participating students. The students were also assured that all information collected will remain confidential. This study was approved by the Ethics Committee of Shiraz University of Medical Sciences (ECSUMS). Approved Code of Ethics Committee for Research Project was CT: 94-170 Results Among the 33 virtual medical education students who participated in the study, 26 were female (79%) and 7 were male (21%). Overall mean of participants’ ages was (37 ± 5.79) with the range of 26-50.Altogether, 79% of the subjects were married and 21% were single. The first specific purpose of this research included the evaluation of e-portfolio for validity by experts. For this purposes the e-portfolio was evaluated by 10 experts on technical, functional and educational criteria’s. Our findings revealed that the points for all features evaluated were positive, and the quality of e-portfolio was acceptable. Responses from experts indicated that the e-portfolio was very useful to medical education students; easy to use, safe, and assisted in learning and evaluating the students. The second specific purpose of this research was the comparison of the participants' learning toward “research in education” before and after intervention. Therefore, the research samples filled the pre-test questionnaires before and 8weeks after learning (Table 1). Table 1.Comparison of knowledge scores before and after the intervention (0-20) Knowledge score Before intervention after intervention Mean±SD Minimum Maximum 13.36±2.396 5 17 10 Table 2. The attitude of the research samples toward evaluation through e-portfolio Attitude No Minimum Maximum Means 33 95 145 120.97±12.045 The results showed that the student's had a positive attitude towards evaluation through e-portfolio. The correlation between the level of knowledge and attitude of the research samples were also examined (Table 3). Table 3. The correlation between the level of knowledge and attitude of research samples knowledge Pre -test Post-test The correlation coefficient 0.069 -0.093 P value 0.704 0.608 No 33 33 Attitude According to table 3, there was no significant linear relationship between the level of knowledge and attitude of research samples. The fourth purpose was to determine the correlation between student’s reflections and teachers’ feedbacks with the level of knowledge and attitude of the students (Table 4). Table 4. The correlation between students’ feedbacks with their level of knowledge and attitude Variables Pre-test Post-test The number of feedback The correlation coefficient 1.000 0.282 0.069 -0.118 P value 0.0 0.112 0.704 0.514 The correlation coefficient 0.282 1.000 -0.093 0.373* P value 0.112 0.0 0.608 0.032 0.069 -0.093 1.000 0.036 0.704 0.608 0.0 0.841 -0.118 0.373* 0.036 1.000 0.514 0.032 0.841 0.0 The correlation Attitude Pre-test Post-test Attitude coefficient P value The number The correlation of students’ reflection and coefficient teacher P value feedback P value 0.027 14.42±1.714 post-test scores were significantly more than the pre-test. The pair t-test revealed that there was a significant difference between before and after intervention (p=0.027). The third specific goal of this research was to determine the attitude of the research samples toward evaluation through eportfolio (Table 2). 17 As is evident from the above table, the correlation between the students’ reflection on learning and post-test scores was significant, but in others areas (pre-test and attitude), there were no significant correlation( p<0.001). It was shown in the above table that there was a meaningful relationship between pre-test and post-test scores, and the 172 Journal of Advanced Pharmacy Education & Research | Oct-Dec 2018 | Vol 8 | Issue S2 Nahid Zarifsanaiey and et al.: E-portfolio based learning: Implementation and evaluation Discussion The aim of this study was implementation and evaluation of eportfolio amongst postgraduate medical students. The eportfolio framework was designed for educating and selfreflecting of the medical education students. The experts evaluated the e-portfolio in three areas (technical, functional, and educational) as a qualified tool for supporting teaching and learning process. There was a significant increase in total mean scores of students learning after using the e-portfolio (p=0.027). This result revealed that the e-portfolio was a useful tool for education and evaluation. This finding is in line with results obtained in some studies. Pincombe et al. (2010) investigated the effects of eportfolio on learning and satisfaction of midwifery students. They showed that the e-portfolio increased the student participation in learning process and helped them in the application of the principles and theory in clinical education[16]. In another research carried out by Fong et al. in "Digital teaching portfolio in higher education", it was revealed that the e-portfolio had a positive effect on teaching and learning[17]. Moreover, in a study by Garrett et al. (2013) evaluated the implementation of an e-portfolio to assess the clinical competence in nursing program. Their results revealed that eportfolio was accessible, convenient and transparent method of assessment[18]. The results of this study are not in line with some other studies, for example; Currant et al. in their research concluded that there were no differences between using e-portfolio with learner autonomy and levels of learning [19]. Chi-Cheng et al. also studied the effects of using a web-based portfolio on student's self-regulated learning and their academic achievement. The results showed that web-based portfolio did not have any significant effect on students' progress[20]. Seals also studied the impact of using portfolios as a learning tool in student achievement in a pilot project study. The results showed that there was no significant difference between the control and experimental groups [21]. It seems that one reason for lack of differences in learning in some researches, measuring e-portfolio is at an early stage and the principles and criteria in scoring do not exist. Furthermore, the educational goals must be appropriate with using the electronic e-portfolio. E-portfolio is more compatible for measuring school achievement of learners in real situations and overtime. In summary, it can be concluded that by using e-portfolio, students are encouraged to have higher order of thinking; selfregulated learning and self-evaluation through attaining academic achievement. One important factor in accepting educational methods is the student attitude. Consequently, other specific goal of this research was to determine the attitude of the research samples towards evaluation by the e-portfolio. As the result of this study has suggested, most of the research subjects had a positive attitude toward this approach. This finding is in line with the results obtained in studies carried out with Notari et al., Naumann et al., Currant et al., and McKenna et al.[22-24]. These studies emphasized that since e-portfolio helped the students to think about their progress, analyzed their work, recognized their strengths and weaknesses and got feedback, therefore, they had a positive attitude towards e-portfolio. These factors were also identified as the important factors to imposed positive attitude in our research. In general, learning strongly depends on learner’s experiences and theirs reflection on the learning process. Besides, the teachers must bring up issues for discussions in order to enhance group interactions, and promote self-reflection among students[25]. Therefore Students’ reflection and teacher feedbacks are the critical components in e-portfolio. The eportfolio without reflection is like collecting facts without asking for the reasons. As the results showed, there was a significant correlation between the students’ self-reflection and teacher feedbacks with the level of learning (p<0.001). Many researchers have confirmed the close relationship between students’ reflection and learning in e-portfolios, for example, Ching et al., in a study investigated the relationship between eportfolios, organization, and Reflection. They concluded that students’ reflection was directly related to student success[26]. In a qualitative case study, Landis et al., examined the role of reflection in e-portfolio and concluded that reflection had comprehensive effects in teaching and learning process[27]. In conclusion the use of e-portfolio can have a significant impact in improving the education system and e-learning courses. Consent for publication Not applicable Availability of data and materials Data are accessible by contacting the corresponding authors. Competing interests The authors declare that they have no competing interests. Funding No funding was provided for this work. Authors’ contributions NZ devised the study concept, designed the study, supervised the intervention, data collection and analysis, participated in the coordination of the study, and critically revised the manuscript. RR collected data, ran the study intervention, participated in the study concept, performed the analyses and revised the manuscript SE contributed to the design and analysis of the study data, and drafted the manuscript. Acknowledgement The present article was extracted from the MSc Medical Education thesis by Mrs. Samie Etemadi under the guidance of Dr. Nahid ZarifSanaiey with the consultation of Dr. Rita Journal of Advanced Pharmacy Education & Research | Oct-Dec 2018 | Vol 8 | Issue S2 173 Nahid Zarifsanaiey and et al.: E-portfolio based learning: Implementation and evaluation Rezaee (Grant No:94-170). The authors would like to thank the Vice-chancellor of Shiraz University of Medical Sciences for supporting this research. The authors thank the Clinical Research Development Center of Shiraz University of Medical Sciences for the statistical analysis and also thank the president of e-learning center and the president of Education Developmental center (EDC) of Shiraz University of Medical Sciences for their patience and hard work invested in learning and using the web-based e-portfolio system. The authors wish to thank the Research Consultation Center (RCC) at Shiraz University of Medical Sciences for their invaluable editorial assistance. References 1. Zarifsanaiey N, Amini M, Saadat F. A comparison of educational strategies for the acquisition of nursing student’s performance and critical thinking: simulationbased training vs. integrated training (simulation and critical thinking strategies). BMC Med Educ. 2016;16(1):294. 2. Rezaee R, Shokrpour N. Performance assessment of academic departments: CIPP model. European Journal of Social Sciences. 2011;23(2):227-36. 3. McGraw-Hill. Higher Education2003 Available from: http://highered.mcgrawhill.com/sites/0072486740/student_view0/chapter9/gl ossary.html. 4. Zarshenas L, Keshavarz T, Momennasab M, Zarifsanaiey N. Interactive Multimedia Training in Osteoporosis Prevention of Female High School Students: An Interventional Study. Acta Med Iran. 2017 Aug 1;55(8):514. 5. Paulson FL. What Makes a Portfolio a Portfolio? Educational leadership. 1991;48(5):60-3. 6. Rezaei R, Mehrabani G. A comparison of the scorings of real and standardized patients on physician communication skills. Pak J Med Sci. 2014;30(3):664. 7. Zarifsanaiey N, Ghorbani N, Rezaee R. A Comparison Of Mobile-Based Learning And Lecture-Based Learning In An Adult Health Care Workers Course. Iioab Journal. 2016 Jan 1;7:466-71. 8. Borman GD, Dowling NM. Longitudinal achievement effects of multiyear summer school: Evidence from the Teach Baltimore randomized field trial. Educ Eval Policy Anal. 2006;28(1):25-48. 9. Cooper T, Love T. Online portfolios: issues of assessment and pedagogy. AARE. 2001:417-26. 10. Cotterill S, Skelly G, McDonald A. Design and integration issues in developing a managed learning environment which is responsive to changing curriculum and policy requirements. Network Learning. 2004;2004:409-16. 11. Greenberg G. The digital convergence: Extending the portfolios Model, EDUCASE Review. EDUCAUSE review. 2004;39(4):28-36. 174 12. Rees C, Sheard C. Undergraduate medical students' views about a reflective portfolio assessment of their communication skills learning. Medical Education. 2004;38(2):125-8. 13. Korthagen FA, Kessels J, Koster B, Lagerwerf B, Wubbels T. Linking practice and theory: The pedagogy of realistic teacher education: Routledge; 2001. 14. Wade RC, Yarbrough DB. Portfolios: A tool for reflective thinking in teacher education? Teach Teach Educ 1996;12(1):63-79. 15. Batson T. The Electronic Portfolio Boom: What's It All About? Syllabus. 2002;16(5):n5. 16. Pincombe J, McKellar L, Weise M, Grinter E, Beresford G. ePortfolio in midwifery practice:“the way of the future”. Women Birth 2010;23(3):94-102. 17. Fong RW-t, Lee JC-k, Chang C-y, Zhang Z, Ngai AC-y, Lim CP. Digital teaching portfolio in higher education: Examining colleagues' perceptions to inform implementation strategies. Internet High Educ. 2014;20:60-8. 18. Garrett BM, MacPhee M, Jackson C. Evaluation of an eportfolio for the assessment of clinical competence in a baccalaureate nursing program. Nurse education today. 2013;33(10):1207-13. 19. Currant N, Haigh J, Higgison C, Hughes P, Rodway P, Whitfield R. Designing eportfolio based learning activities to promote learner autonomy. Final Report to the Fourth Cohort of the Inter/National Coalition for Research into Electronic Portfolios, University of Bradford. 2010. 20. Chi-Cheng C, Pao-Nan C. Effects of reflection category and reflection quality on learning outcomes during webbased portfolio assessment process: A case study of high school students in computer application course. TOJET. 2011;10(3). 21. nseals G, J. the effects of portfolio use as a learning tool on aljebra 2 students achievement and their attitudes toward mathematics: mississippi; 2001. 22. Notari M, Sobko T, Churchill D. Using wearable technology to improve the acquisition of new literacies A new pedagogical approach of situated individual feedback coming from the Activity Trackers and reflected upon in the ePortfolio. International Mobile Learning 2016. 23. Naumann F, Yang J-L, Thai T, Ford C, Polly P. Virtual patient consultations and the use of an ePortfolio assessment to support student learning of integrated professional skills. Focus on Health Professional Education: A Multi-disciplinary Journal. 2016;17(3):69. 24. McKenna G, Baxter G, Hainey T. E-portfolios and Personal Development: A Higher Educational Perspective. Journal of Applied Research in Higher Education. 2017;9(1). 25. Rezaee R, Ebrahimi S. Clinical Learning Environment at Shiraz Medical School. Acta Med Iran. 2013;51(1):62. 26. Ching Y-H, Yang D, Baek Y, Baldwin S. Enhancing Graduate Students’ Reflection in E-Portfolios Using the Journal of Advanced Pharmacy Education & Research | Oct-Dec 2018 | Vol 8 | Issue S2 Nahid Zarifsanaiey and et al.: E-portfolio based learning: Implementation and evaluation TPACK Framework. Australasian Journal of Educational Technology. 2016. 27. Landis CM, Scott SB, Kahn S. Examining the Role of Reflection in ePortfolios: A Case Study. International Journal of ePortfolio. 2015;5(2):107-21. Journal of Advanced Pharmacy Education & Research | Oct-Dec 2018 | Vol 8 | Issue S2 175