A Model to Assess Online Learning: Analysis and Proposal

_________________________________________________________________________________________ ISSN: 2414-0325. Open educational e-environment of modern University, № 2 (2016) UDK Laura Alonso-Díaz PhD Professor in the Education Studies Department Teacher Training Faculty of the University of Extremadura in Caceres (Spain) laulonso@unex.es Rocío Yuste Tosina PhD Professor in the Education Studies Department Teacher Training Faculty of the University of Extremadura in Caceres (Spain) rocioyuste@unex.es Gemma Delicado Puerto PhD Professor in the English Department Teacher Training Faculty of the University of Extremadura in Caceres (Spain) gdelpue@unex.es Sixto Cubo Delgado PhD Professor in the Education Studies Department Education Faculty of the University of Extremadura in Badajoz (Spain) sixto@unex.es Prudencia Gutiérrez Esteban PhD Professor in the Education Studies Department Education Faculty of the University of Extremadura in Badajoz (Spain) pruden@unex.es Juan Arias Masa PhD Professor in the Telematic Engineering Department University Centre, University of Extremadura, in Mérida (Spain) jarias@unex.es A MODEL TO ASSESS ONLINE LEARNING: ANALYSIS AND PROPOSAL Abstract. An assessment system for e-learning methods is analyzed in this study. The system proposed used videoconference to assess students. The model tested has been used in three different countries. Teachers, students and external evaluators were involved in the process. Using a qualitative methodology, interviews, focus groups and questionnaires were used. The results confirm that a viable, efficient and innovative educational model have been developed for higher distance education. Also, videoconference and Synchronous Virtual Classrooms have proved to be efficient tools to evaluate the e-assessment method in virtual learning spaces. Keywords: E-learning, e-assessment, assessment, videoconferencing, higher education, teaching method innovations, synchronous environments, qualitative research Introduction. One area of e-learning in which we have detected more difficulty in terms of innovation is that of assessment; and while the research done in recent years has increased notably throughout the world, developing new models and objects of assessment, there is still much that can be done to increase innovation in e-assessment, given that, as Bevitt (2014) affirms, there is an significant gap in the literature. Along this line, more and more researchers feel that qualitative methods can be especially effective in the study of virtual learning environments. Agostinho (2005) has proposed using naturalistic inquiries for e-learning research; Fitzgerald, Hackling & Dawson (2013) and Steinmetz (2012) have applied ethnography to online training processes; while Postma et al. (2013) have contributed to the use of grounded theory in online discussion forums. Studies on formative assessment have remarkably increased at an international level. The concept of assessment itself has aroused many interests not only from a pedagogical point 1 _________________________________________________________________________________________ ISSN: 2414-0325. Open educational e-environment of modern University, № 2 (2016) of view, but also strategic and even economic. For the last few years, these interests have favoured the correction and precision of its definition. In our study we focus on online formative assessment, where, to start with, we find elements that are identical to those that arise in any assessment method and which have to be contextualized according to the specific learning situation to be observed, measured and improved. In their studies Gikandi, Morrow & Davis (2011) state that assessment (whether formative or summative) in online learning contexts includes distinctive characteristics in comparison with face-to-face contexts, especially due to the asynchronous nature of the participant’s interactivity. Therefore, educators must rethink pedagogy in virtual settings in order to achieve effective formative assessment strategies. Those who have delved more deeply into the design of a formative assessment model, such as Meyen et al. (2002), have concentrated their greatest efforts on the effectiveness of technology and teaching/content design, but without transcending traditional formulas of distance learning. Daly et al. (2010) made a step forward by considering formative assessment in an ulterior manner; i.e. by identifying the processes that take place around it in which technology plays a significant role. Hew et al. (2004) observed that, despite the new concept of e-learning, traditional methods such as tests, final exams and final projects were still being used for assessment. Alternative methods, however, would also begin to appear, such as portfolios, which for Klenowski (2007) are collections of assigned projects that highlight individual achievements, foment self-assessment and offer an alternative focus to the assessment process. At the same time, another tool has appeared which allows synchronous communication with the student while retaining the non-situational characteristics of e-learning. Thus, videoconferencing has gradually begun to take centre stage for specific activities in the learning process. According to Cabero & Prendes (2009), initial assessment (of prior knowledge), as well as processive (monitoring of student interaction) and final (e. g. oral presentations and examinations) assessment can be done effectively through videoconferencing. Meanwhile, we maintain that such a classroom would be an environment that integrates audio and video tools, screen sharing, projection systems, remote control and so on, making distance learning possible “at any time and in any place” and including both visual and auditory interaction. Interested in trying an innovative formative e-assessment system, our research team designed a specific proposal for a Secondary Education Specialist online course aimed at graduates from Spanish, Portuguese and Latin American universities wishing to be trained to be secondary education teachers. Our virtual learning proposal considered an environment designed to be flexible, based on access to original and varied sources, for students considered as active participants in their own training, accompanied by a team of coordinated and interfunctional teachers who shared with the students, both individually and in group, the responsibility for the process. A process in which the interaction formulas were negotiated and where priority was placed on problem solving, which alternated individual and collaborative work, and which offered a rich diversity of materials (media), and continuous assessment, formative and comprehensive, with dialogue as main premise. Throughout the course, it was intended that students acquired knowledge and skills necessary to reflect, analyze and criticize the main contents that shape the formative aspects of a secondary or middle school teacher in different countries of the Western world and train them to fulfil their functions in that level of education. Methods. The study here presented examines the assessment in virtual learning settings in a University Specialist Course for students who want to train as secondary education teachers. Following these steps: 2 _________________________________________________________________________________________ ISSN: 2414-0325. Open educational e-environment of modern University, № 2 (2016) – Preliminary step: This stage arises from the research’s conceptual theoretical framework. The field to be studied is defined in this stage, together with the different stages of the design of qualitative research, describing the object to be studied, the triangulation, the data collection techniques and tools and the analysis to be developed. – Field work: It consisted in the data collection, which was carried out in two temporal spaces, at the beginning and once the Specialist course had finished: The participants of the study were eight teachers characterized by their university and psycopedagogic training; the students, graduates from different university degrees; and experts’ group was made up of four teachers. The istruments used were questionaries, focus groups and interviews. Results. A category system was developed as described below (Fig. 1) – GAA: General Aspects of the Assessment. – AIA: Assessment of Individual Activities Requested to the Students. – AGA: Assessment of Group Activities Requested to the Students. – AVID: Assessment via Videoconference. – TOO: Tools used for the Assessment. Fig. 1. Chart of sorts Regarding the first category, General Aspects of the Assessment (GAA), the students surveyed (15 of the 17 questionnaires analyzed) was overall pleased with the type of assessment proposed: "From my point of view, the assessment includes all four dimensions of the assessment process, that is: a previous design of the criteria, a comparison of information to obtain a balanced judgment, a decision-making process and the communication of results" (student). Regarding the Assessment of Individual Activities (AIA), overall, the students consider the individual activities requested by the teachers as a very effective tool to ensure the quality of the course’s assessment system. As for the Assessment of Group Activities (AGA), students have not paid much attention to this category, but those who have answered (3 out of 17) have given a very positive judgment, as one student says: "With the final project I was able to consolidate the contents dealt within the course, and to study in depth other related issues". 3 _________________________________________________________________________________________ ISSN: 2414-0325. Open educational e-environment of modern University, № 2 (2016) Regarding the assessment of Individual Activities Requested to the Student (AIA), the interviewee values positively the fact that more value or weight is given to the activities, since she considers that "it is the best way to learn". As for the Assessment of Group Activities (AGA), the views are varied: 60% of the teachers argue that no work was carried out 100% collaboratively, because the students simply divided the parts and then joined them together. Other teachers-tutors, however, argued that the group activities had truly given them the criteria to evaluate and know the students better. Regarding the Assessment via Videoconference (AVID), one of the experts focused on its reliability: "Many times in a face-to-face setting we tell the students to start creating an electronic portfolio, in the end they will submit a project or a research report. What we know is that the students have given it to us physically, but we don’t have the mechanism, we have it when we are talking to them. This can be done online, but you have to find another criterion and the criterion is that the student-tutor ratio cannot be very high" (expert). In addition, another expert clarifies that "If you are doing an online course properly you know your students well, and it would be impossible for them to 'cheat' in their work. The important thing is that if it is well designed I assure you they do not lie" (expert). Finally, experts have also called attention to the need to establish coherence between the training model and the assessment system, so that the necessary means to enable online and distance assessment are arbitrated whenever the formative model follows these parameters. Discussion. In this study we have presented an online assessment model that does not require the presence of the students, based on a constructive consideration of knowledge where learning can and should be assessed and evaluated throughout the training process itself, with tasks that can be assessed from the perspective of individual and group learning. The assessment of this study has helped us establish the following conclusions: Progress has been made in achieving an innovative model for feasible and effective e-assessment. The assessment we propose is formative. Most students consider the assessment followed as a highly motivating method since, in addition to the different techniques and tools used, the assessment is considered to be part of the teaching-learning process and not only an activity that takes place at the end of the course. The results of this study converge the intended teaching innovation in universities with e-learning as agent for educational change. In general, the implementation of this online assessment method has been positive, especially when focusing on continuous assessment throughout the activities, projects and interviews (via videoconference), together with a tutorial model that ensures the supervision of the student’s learning progress, enhanced by a manageable ratio of five students per tutor. In turn, this is reinforced by the distinctiveness of the pilot, where the teachers were selected for their high participation and motivation. REFERENCES Agostinho (2005). Naturalistic inquiry in e-learning research. International Journal of Qualitative Methods, 4(1), 813-26. Bevitt, S. (2014). Assessment innovation and student experience: a new assessment challenge and call for a multi-perspective approach to assessment research. Assessment & Evaluation in Higher Education. Cabero, J. & Prendes, M.P. (2009). La videoconferencia. Aplicaciones a los ámbitos educativo y empresarial. Sevilla: MAD. Daly, C., Pachler, N., Mor, Y. & Mellar, H. (2010). Exploring formative e‐assessment: using case stories and design patterns. Assessment & Evaluation in Higher Education, 35(5), 616-636 4 _________________________________________________________________________________________ ISSN: 2414-0325. Open educational e-environment of modern University, № 2 (2016) Fitzgerald, A., Hackling, M. & Dawson, V. (2013). Through the Viewfinder: Reflecting on the Collection and Analysis of Classroom Video Data. International Journal of Qualitative Methods, 12, 52-64. Gikandi, J.W.; Morrow, D. & Davis, N.E. (2011). Online Formative Assessment in Higher Education: A Review of the literature. Computers & Education 57, 2.333-2.351. Hew, K., Liu, S., Martinez, R. Bonk, C. & Lee, J. Y. (2004). Online education evaluation: what should we evaluate? Taken from: ERICDatabase. Klenowski, V. (2007). Desarrollo de portafolios para el aprendizaje y la evaluación. Madrid: Narcea. Meyen, E., Aust, R., Bui, Y., Ramp, E. & Smith, S. (2002). The Online Academy formative evaluation approach to evaluating online instruction. The Internet and Higher Education, 5, 89-108. Postma, L., Blignaut, A. S., Swan, K. & Sutinen, E. (2013). Reflections on the use of grounded theory to uncover patterns of exclusion in an online discussion forum at an institution of higher education. Journal of Qualitative Methods, 12, 529-550. Steinmetz (2012). Message received: virtual ethnography in online message boards. International Journal of Qualitative Methods, 11(1), 26-39. 5