_________________________________________________________________________________________
ISSN: 2414-0325. Open educational e-environment of modern University, № 2 (2016)
UDK
Laura Alonso-Díaz
PhD Professor in the Education Studies Department
Teacher Training Faculty of the University of Extremadura in Caceres (Spain)
laulonso@unex.es
Rocío Yuste Tosina
PhD Professor in the Education Studies Department
Teacher Training Faculty of the University of Extremadura in Caceres (Spain)
rocioyuste@unex.es
Gemma Delicado Puerto
PhD Professor in the English Department
Teacher Training Faculty of the University of Extremadura in Caceres (Spain)
gdelpue@unex.es
Sixto Cubo Delgado
PhD Professor in the Education Studies Department
Education Faculty of the University of Extremadura in Badajoz (Spain)
sixto@unex.es
Prudencia Gutiérrez Esteban
PhD Professor in the Education Studies Department
Education Faculty of the University of Extremadura in Badajoz (Spain)
pruden@unex.es
Juan Arias Masa
PhD Professor in the Telematic Engineering Department
University Centre, University of Extremadura, in Mérida (Spain)
jarias@unex.es
A MODEL TO ASSESS ONLINE LEARNING: ANALYSIS AND PROPOSAL
Abstract. An assessment system for e-learning methods is analyzed in this study. The system
proposed used videoconference to assess students. The model tested has been used in three
different countries. Teachers, students and external evaluators were involved in the process. Using
a qualitative methodology, interviews, focus groups and questionnaires were used. The results
confirm that a viable, efficient and innovative educational model have been developed for higher
distance education. Also, videoconference and Synchronous Virtual Classrooms have proved to be
efficient tools to evaluate the e-assessment method in virtual learning spaces.
Keywords: E-learning, e-assessment, assessment, videoconferencing, higher education, teaching
method innovations, synchronous environments, qualitative research
Introduction. One area of e-learning in which we have detected more difficulty in
terms of innovation is that of assessment; and while the research done in recent years has
increased notably throughout the world, developing new models and objects of assessment,
there is still much that can be done to increase innovation in e-assessment, given that, as
Bevitt (2014) affirms, there is an significant gap in the literature. Along this line, more and
more researchers feel that qualitative methods can be especially effective in the study of
virtual learning environments. Agostinho (2005) has proposed using naturalistic inquiries for
e-learning research; Fitzgerald, Hackling & Dawson (2013) and Steinmetz (2012) have
applied ethnography to online training processes; while Postma et al. (2013) have contributed
to the use of grounded theory in online discussion forums.
Studies on formative assessment have remarkably increased at an international level.
The concept of assessment itself has aroused many interests not only from a pedagogical point
1
_________________________________________________________________________________________
ISSN: 2414-0325. Open educational e-environment of modern University, № 2 (2016)
of view, but also strategic and even economic. For the last few years, these interests have
favoured the correction and precision of its definition.
In our study we focus on online formative assessment, where, to start with, we find
elements that are identical to those that arise in any assessment method and which have to be
contextualized according to the specific learning situation to be observed, measured and
improved. In their studies Gikandi, Morrow & Davis (2011) state that assessment (whether
formative or summative) in online learning contexts includes distinctive characteristics in
comparison with face-to-face contexts, especially due to the asynchronous nature of the
participant’s interactivity. Therefore, educators must rethink pedagogy in virtual settings in
order to achieve effective formative assessment strategies.
Those who have delved more deeply into the design of a formative assessment model,
such as Meyen et al. (2002), have concentrated their greatest efforts on the effectiveness of
technology and teaching/content design, but without transcending traditional formulas of
distance learning. Daly et al. (2010) made a step forward by considering formative assessment
in an ulterior manner; i.e. by identifying the processes that take place around it in which
technology plays a significant role. Hew et al. (2004) observed that, despite the new concept
of e-learning, traditional methods such as tests, final exams and final projects were still being
used for assessment. Alternative methods, however, would also begin to appear, such as
portfolios, which for Klenowski (2007) are collections of assigned projects that highlight
individual achievements, foment self-assessment and offer an alternative focus to the
assessment process.
At the same time, another tool has appeared which allows synchronous communication
with the student while retaining the non-situational characteristics of e-learning. Thus,
videoconferencing has gradually begun to take centre stage for specific activities in the
learning process. According to Cabero & Prendes (2009), initial assessment (of prior
knowledge), as well as processive (monitoring of student interaction) and final (e. g. oral
presentations and examinations) assessment can be done effectively through
videoconferencing. Meanwhile, we maintain that such a classroom would be an environment
that integrates audio and video tools, screen sharing, projection systems, remote control and
so on, making distance learning possible “at any time and in any place” and including both
visual and auditory interaction.
Interested in trying an innovative formative e-assessment system, our research team
designed a specific proposal for a Secondary Education Specialist online course aimed at
graduates from Spanish, Portuguese and Latin American universities wishing to be trained to
be secondary education teachers. Our virtual learning proposal considered an environment
designed to be flexible, based on access to original and varied sources, for students considered
as active participants in their own training, accompanied by a team of coordinated and
interfunctional teachers who shared with the students, both individually and in group, the
responsibility for the process. A process in which the interaction formulas were negotiated
and where priority was placed on problem solving, which alternated individual and
collaborative work, and which offered a rich diversity of materials (media), and continuous
assessment, formative and comprehensive, with dialogue as main premise. Throughout the
course, it was intended that students acquired knowledge and skills necessary to reflect,
analyze and criticize the main contents that shape the formative aspects of a secondary or
middle school teacher in different countries of the Western world and train them to fulfil their
functions in that level of education.
Methods. The study here presented examines the assessment in virtual learning settings
in a University Specialist Course for students who want to train as secondary education
teachers. Following these steps:
2
_________________________________________________________________________________________
ISSN: 2414-0325. Open educational e-environment of modern University, № 2 (2016)
– Preliminary step: This stage arises from the research’s conceptual theoretical
framework. The field to be studied is defined in this stage, together with the different stages
of the design of qualitative research, describing the object to be studied, the triangulation, the
data collection techniques and tools and the analysis to be developed.
– Field work: It consisted in the data collection, which was carried out in two temporal
spaces, at the beginning and once the Specialist course had finished:
The participants of the study were eight teachers characterized by their university and
psycopedagogic training; the students, graduates from different university degrees; and
experts’ group was made up of four teachers.
The istruments used were questionaries, focus groups and interviews.
Results. A category system was developed as described below (Fig. 1)
– GAA: General Aspects of the Assessment.
– AIA: Assessment of Individual Activities Requested to the Students.
– AGA: Assessment of Group Activities Requested to the Students.
– AVID: Assessment via Videoconference.
– TOO: Tools used for the Assessment.
Fig. 1. Chart of sorts
Regarding the first category, General Aspects of the Assessment (GAA), the students
surveyed (15 of the 17 questionnaires analyzed) was overall pleased with the type of
assessment proposed: "From my point of view, the assessment includes all four dimensions of
the assessment process, that is: a previous design of the criteria, a comparison of information
to obtain a balanced judgment, a decision-making process and the communication of results"
(student).
Regarding the Assessment of Individual Activities (AIA), overall, the students consider
the individual activities requested by the teachers as a very effective tool to ensure the quality
of the course’s assessment system. As for the Assessment of Group Activities (AGA),
students have not paid much attention to this category, but those who have answered (3 out of
17) have given a very positive judgment, as one student says: "With the final project I was
able to consolidate the contents dealt within the course, and to study in depth other related
issues".
3
_________________________________________________________________________________________
ISSN: 2414-0325. Open educational e-environment of modern University, № 2 (2016)
Regarding the assessment of Individual Activities Requested to the Student (AIA), the
interviewee values positively the fact that more value or weight is given to the activities, since
she considers that "it is the best way to learn".
As for the Assessment of Group Activities (AGA), the views are varied: 60% of the
teachers argue that no work was carried out 100% collaboratively, because the students
simply divided the parts and then joined them together. Other teachers-tutors, however,
argued that the group activities had truly given them the criteria to evaluate and know the
students better.
Regarding the Assessment via Videoconference (AVID), one of the experts focused on
its reliability: "Many times in a face-to-face setting we tell the students to start creating an
electronic portfolio, in the end they will submit a project or a research report. What we know
is that the students have given it to us physically, but we don’t have the mechanism, we have
it when we are talking to them. This can be done online, but you have to find another criterion
and the criterion is that the student-tutor ratio cannot be very high" (expert). In addition,
another expert clarifies that "If you are doing an online course properly you know your
students well, and it would be impossible for them to 'cheat' in their work. The important
thing is that if it is well designed I assure you they do not lie" (expert).
Finally, experts have also called attention to the need to establish coherence between the
training model and the assessment system, so that the necessary means to enable online and
distance assessment are arbitrated whenever the formative model follows these parameters.
Discussion. In this study we have presented an online assessment model that does not
require the presence of the students, based on a constructive consideration of knowledge
where learning can and should be assessed and evaluated throughout the training process
itself, with tasks that can be assessed from the perspective of individual and group learning.
The assessment of this study has helped us establish the following conclusions: Progress has
been made in achieving an innovative model for feasible and effective e-assessment. The
assessment we propose is formative. Most students consider the assessment followed as a
highly motivating method since, in addition to the different techniques and tools used, the
assessment is considered to be part of the teaching-learning process and not only an activity
that takes place at the end of the course. The results of this study converge the intended
teaching innovation in universities with e-learning as agent for educational change. In general,
the implementation of this online assessment method has been positive, especially when
focusing on continuous assessment throughout the activities, projects and interviews (via
videoconference), together with a tutorial model that ensures the supervision of the student’s
learning progress, enhanced by a manageable ratio of five students per tutor. In turn, this is
reinforced by the distinctiveness of the pilot, where the teachers were selected for their high
participation and motivation.
REFERENCES
Agostinho (2005). Naturalistic inquiry in e-learning research. International Journal of
Qualitative Methods, 4(1), 813-26.
Bevitt, S. (2014). Assessment innovation and student experience: a new assessment
challenge and call for a multi-perspective approach to assessment research. Assessment &
Evaluation in Higher Education.
Cabero, J. & Prendes, M.P. (2009). La videoconferencia. Aplicaciones a los ámbitos
educativo y empresarial. Sevilla: MAD.
Daly, C., Pachler, N., Mor, Y. & Mellar, H. (2010). Exploring formative e‐assessment:
using case stories and design patterns. Assessment & Evaluation in Higher Education, 35(5),
616-636
4
_________________________________________________________________________________________
ISSN: 2414-0325. Open educational e-environment of modern University, № 2 (2016)
Fitzgerald, A., Hackling, M. & Dawson, V. (2013). Through the Viewfinder: Reflecting
on the Collection and Analysis of Classroom Video Data. International Journal of Qualitative
Methods, 12, 52-64.
Gikandi, J.W.; Morrow, D. & Davis, N.E. (2011). Online Formative Assessment in
Higher Education: A Review of the literature. Computers & Education 57, 2.333-2.351.
Hew, K., Liu, S., Martinez, R. Bonk, C. & Lee, J. Y. (2004). Online education
evaluation: what should we evaluate? Taken from: ERICDatabase.
Klenowski, V. (2007). Desarrollo de portafolios para el aprendizaje y la evaluación.
Madrid: Narcea.
Meyen, E., Aust, R., Bui, Y., Ramp, E. & Smith, S. (2002). The Online Academy
formative evaluation approach to evaluating online instruction. The Internet and Higher
Education, 5, 89-108.
Postma, L., Blignaut, A. S., Swan, K. & Sutinen, E. (2013). Reflections on the use of
grounded theory to uncover patterns of exclusion in an online discussion forum at an
institution of higher education. Journal of Qualitative Methods, 12, 529-550.
Steinmetz (2012). Message received: virtual ethnography in online message boards.
International Journal of Qualitative Methods, 11(1), 26-39.
5