Networks in the assessment of groupwork

Peer assessment has a double goal in higher education: it is both used to enhance learning of difficult topics or complex skills like working in projects and higher learning competences on the part of students , and to support, deepen or broaden the assessment by tutors and assessors of group work and project outcomes of team-based or online learning projects. Research on peer assessment splits into two areas, too. On the one hand, empirical research on peer assessment is mainly focussed on the design, evaluation and improvement of peer assessment practices in order to enable tutors to offer students the best possible peer assessment environment and experience. On the other hand, methodological research on peer assessment primarily deals with problems and principles of measurement and related technical issues which guarantee that the quantitative results of peer assessments are meaningful and satisfy several other qualitative criteria specific to the learning objectives and in line with regulations and restrictions imposed by the administrative staff or the educational quality assurance department. In this paper we will take up the latter-methodological-perspective. The primary reason is that since the pioneering work of Lejk et al. [1] in the 90's and a single publication of Sharp [2] more than 10 years ago, no substantial progress has been made in this area beyond some attempts to implement those early, mainly statistical approaches to peer assessment [2]. What is indeed painfully missing is a systematic study of the ways in which peer ratings, i.e. the mutual judgements of student performance in a group, can be aligned and combined with tutor's judgment about the student project's outcomes in the form of a team score, such that each student gets a representative , well-balanced and fair individual score, mark or grade. In short: there is still no theory of peer assessment based on sound concepts and principles. Therefore, we will start our systematic study with the development of an algebraic theory of educational metrics, edumetrics for short, which is rich enough to formulate explicitly all concepts and principles needed for peer assessment. We want to be able to talk about different scoring scales (2), scoring models (2), scoring classes, or types (3) and-altogether-scoring rules (2 * 2 * 3 = 12). We will clarify why and how the latter differ and we will point out their respective strengths and weaknesses. Furthermore, we show that all scoring rules can be generalized by introducing a so-called impact factor , which allows peer ratings to have more or less effect on the team score. This kind of weighting parameter offers tutors a systematic way to tailor peer assessment to whatever (unforeseen) contex-tual factors force them to rethink and redo the impact that peer ratings may have on the team score. Finally, we will explain the most important peer assessment principle or property we discovered and which we dubbed Split-Join-Invariance (SJI): an unbiased, well-balanced and fair scoring rule for peer assessment will be such that the mean of (final) student scores will be equal to the (initial) team score given by the tutor. Violation of this principle may lead to unpleasant arguments between students and tutor why the team score should deviate from a representative summary (mean) of the students scores. Most of the scoring rules which we constructed satisfy SJI, but some of them (including two called "assessment by adjustment"-a kind of mixed additive-multiplicative scoring procedure sometimes used by tutors) lack this property, which makes them somewhat suspect and deficient, though they are definitely better than the majority of scoring rules found in practice.