Scaling up Assessment for Learning

Scaling up Assessment for
Learning
CITERS 2017, June 9
Professor David Carless
University of Hong Kong
The University of Hong Kong

Overview
• Assessment for learning (AfL)
• AfL & MOOCs
• Peer feedback & technology
• Leadership & communities of practice

Chapter 1: Scaling up AfL: Progress & prospects
(Carless)
Chapter 14: How does Technology enable Scaling
up AfL? (Dawson & Henderson)

AfL definition
Assessment for which the
first priority is promoting students’
learning (Black et al., 2004)
Cf. Formative assessment
Learning-oriented assessment
(Carless, 2007, 2015a)

Interlinked AfL strategies
1. Productive assessment design
2. Students making judgments
3. Effective feedback processes
4. Developing student appreciation of quality
(Carless, 2017)

Scaling up (Coburn, 2003)
Spread
Depth
Sustainability
Shifts in ownership

Why scale-up AfL?
Research evidence
(Black & Wiliam, Hattie)
Dissatisfaction with current assessment &
feedback practices

Black & Wiliam (1998)
Formative assessment & learning gains
Effect sizes: 0.4 - 0.7
… But implementation is generally weak

Hattie: Visible Learning

MOOCs & AfL

MOOC Assessment design
Alignment of goals, activities & low-stakes
assessment
MC quizzes + automated
feedback
Higher order thinking?

Peer assessment
Peer reviewed assignments + detailed
rubrics as key feature of MOOCs
(Admiraal et al, 2015; Huisman et al., 2016)

Reliability of PA
Moderate reliability of peer assessment
(Admiraal et al., 2015)
Use of multiple peer assessments to
mitigate variance in judgments (Hew, 2016)

Peer feedback for learning
Learners gain more from composing than
receiving peer feedback (Nicol et al., 2014)

Peer & Self-evaluation
Students did 5 peer reviews then self-
evaluated own work
(Hew, 2016)

FEEDBACK PROCESSES &
TECHNOLOGY

Audio & Video feedback
Rapport
Nuance
Personalisation
Monologue or Dialogue?
Time saver?

Peer video feedback
Peer-to-peer video feedback
delivered via Facebook
Hung (2016)

Personalised feedback at scale
Using learning analytics to
scale the provision of
personalised feedback
https://www.ontasklearning.org/

Sustainable feedback
Students generating & using feedback from
peers or self as part of self-regulated
learning (Carless et al., 2011)

DEVELOPING STUDENT
APPRECIATION OF
QUALITY

MOOC limitation?
MOOCs do not systematically develop
student understanding of quality
(Dawson & Henderson, 2017)

Using exemplars
Exemplars convey messages
that nothing else can
(Sadler, 2002)

Appreciating quality
Students need to debate nature of quality &
develop capacities in making judgments

SCALING UP POSSIBILITIES

Enhancing design
Adding technology to existing design &
expecting improvements is flawed
Invest in improved assessment & feedback
designs to leverage gains from technology
(Dawson & Henderson, 2017)

Leadership
Resourcing & support
Incentives & rewards

TELI
Technology-Enhanced Learning Initiative
Associate Vice-President (Teaching
and Learning) Ricky Kwok

Incentives
Teaching Innovation Award
Teaching Feedback Award

Communities of practice (CoPs)
Surfacing and sharing AfL practices
(Hounsell & Zou, 2017)

Staff development
Good CoPs
Dialogue & communication

Scaling up Assessment for Learning

References
Admiraal, W., Huisman, B., & Pilli, O. (2015). Assessment in massive open online
courses. The Electronic Journal of e-Learning, 13(4), 207-216.
Black, P., Harrison, C., Lee, C., Marshall, B., & Wiliam, D. (2004). Working inside the
black box: Assessment for learning in the classroom. Phi Delta Kappan, 86(1), 8-21.
Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in
Education, 5(1), 7-74.
Carless, D. (2007). Learning-oriented assessment: Conceptual basis and practical
implications. Innovations in Education and Teaching International, 44(1), 57-66.
Carless, D. (2015a). Exploring learning-oriented assessment processes. Higher
Education, 69(6), 963-976.
Carless, D. (2015b). Excellence in University Assessment: learning from award-winning
teachers. London: Routledge.
Carless, D. & K.K.H. Chan (2016). Managing dialogic use of exemplars. Assessment and
Evaluation in Higher Education, http://dx.doi.org/10.1080/02602938.2016.1211246
Carless, D., Salter, D., Yang, M., & Lam, J. (2011). Developing sustainable feedback
practices. Studies in Higher Education, 36(4), 395-407.
Coburn, C. (2003). Rethinking scale: Moving beyond numbers to deep and lasting
change. Educational Researcher, 32(6), 3-12.

References
Hattie, J. (2009). Visible learning. London: Routledge.
Hew, K. F. (2016). Promoting engagement in online courses: What strategies can we
learn from three highly rated MOOCS. British Journal of Educational Technology,
47(2), 320-341.
Hounsell, D. & Zou, T. (2017). Surfacing and Sharing Advances in Assessment: A
Communities-of-practice approach. In D. Carless, S. Bridges, C.K.W. Chan & R.
Glofcheski (Eds.), Scaling up Assessment for learning in Higher Education. Singapore:
Springer.
Huisman, B., Admiraal, W., Pilli, L., van de Ven, M., & Saab, N. (2016). Peer assessment
in MOOCs: The relationship between peer reviewers’ ability and authors’ essay
performance. British Journal of Educational Technology. Doi: 10.1111/bjet.12520.
Hung, S.-T. A. (2016). Enhancing feedback provision through multimodal video
technology. Computers & Education, 98, 90-101.
Nicol, D., Thomson, A., & Breslin, C. (2014). Rethinking feedback practices in higher
education: a peer review perspective. Assessment & Evaluation in Higher Education,
39(1), 102–122.
Sadler, D. R. (2002). Ah! … So that’s ‘quality’. In P. Schwartz & G. Webb (Eds.),
Assessment: Case Studies, Experience and Practice from Higher Education (p.130-
136). London: Kogan Page.

THANK YOU

Scaling up Assessment for Learning

More Related Content

Scaling up Assessment for Learning