research-article

Open access

Paper or Online?: A Comparison of Exam Grading Techniques

Authors:

Soohyun Nam Liao,

Rick OrdAuthors Info & Claims

ITiCSE '19: Proceedings of the 2019 ACM Conference on Innovation and Technology in Computer Science Education

Pages 99 - 104

https://doi.org/10.1145/3304221.3319739

Published: 02 July 2019 Publication History

Abstract

As computer science enrollments continue to surge, exam grading requires significant instructional resources. Online grading platforms have been developed in recent years and have been adopted at a number of institutions; however, their effectiveness compared with traditional grading on paper is not fully known. This study is the first in CS to compare online and paper grading. Comparing overall time to grade, including all factors, online grading doesn't show a consistent advantage compared with paper grading. We observed that online grading is much faster during the actual grading phase, but some of this benefit is offset by the additional overhead prior to grading (e.g., scanning) for online grading. Examining student and grader preferences based on feedback, both groups show a strong preference for the online format, predominately due to the convenience of the online platform. Graders report being able to grade more accurately online with the ability to modify rubrics, but that grading on paper tends to result in more social interactions among graders.

References

[1]

R. E. Bennett. Formative assessment: a critical review. Assessment in Education: Principles, Policy & Practice, 18(1):5--25, 2011.

[2]

J. Biggs. Enhancing teaching through constructive alignment. Higher Education, 32(3):347--364, 1996.

[3]

P. Black. Formative assessment: raising standards inside the classroom. School Science Review, 80(291):39--46, 1998.

[4]

A. Bloomfield and J. F. Groves. A tablet-based paper exam grading system. SIGCSE Bull., 40(3):83--87, June 2008.

Digital Library

[5]

A. Cain and M. A. Babar. Reflections on applying constructive alignment with formative feedback for teaching introductory programming and software architecture. In ICSE 2016.

Digital Library

[6]

Y. Cao and L. Porter. Evaluating student learning from collaborative group tests in introductory computing. In SIGCSE 2017.

Digital Library

[7]

K. Chebrolu, B. Raman, V. C. Dommeti, A. V. Boddu, K. Zacharia, A. Babu, and P. Chandan. Safe: Smart authenticated fast exams for student evaluation in classrooms. In SIGCSE 2017.

Digital Library

[8]

N. R. Council et al. How people learn: Brain, mind, experience, and school: Expanded edition. National Academies Press, 2000.

[9]

CRA. Computing Research Association, The Taulbee Survey. https://cra.org/wp-content/uploads/2018/05/2017-Taulbee-Survey-Report.pdf. Accessed Jan. 15, 2019.

[10]

K. Dewey, P. Conrad, M. Craig, and E. Morozova. Evaluating test suite effectiveness and assessing student code via constraint logic programming. In ITiCSE 2017.

Digital Library

[11]

F. Dochy, M. Segers, D. Gijbels, and K. Struyven. Assessment engineering. In Rethinking assessment in higher education: learning for the longer term., pages 87--100. Oxford: Routledge, 2007.

[12]

M. L. Epstein, A. D. Lazarus, T. B. Calvano, K. Matthews, R. A. Hendel, B. B. Epstein, and G. M. Brosvic. Immediate feedback assessment technique promotes learning and corrects inaccurate first responses. The Psychological Record, 52:187--201, 2002.

[13]

J. Forbes, D. J. Malan, H. Pon-Barry, S. Reges, and M. Sahami. Scaling introductory courses using undergraduate teaching assistants. In SIGCSE 2017.

Digital Library

[14]

B. J. Fraser, H. J. Walberg, W. W. Welch, and J. A. Hattie. Syntheses of educational productivity research. International Journal of Educational Research, 11(2):187--212, 1987.

[15]

G. Gibbs, C. Simpson, P. Gravestock, and M. Hills. Conditions Under Which Assessment Supports Students' Learning. Learning in Teaching in Higher Education, 1(1):3--31, 2004.

[16]

C. Gregg and C. M. Lewis. Working with undergraduate teaching assistants: Best practices and lessons learned. In SIGCSE 2015.

Digital Library

[17]

S. Gulwani, I. Radivcek, and F. Zuleger. Feedback generation for performance problems in introductory programming assignments. In FSE 2014.

Digital Library

[18]

W. Harlen and M. James. Assessment and learning: differences and relationships between formative and summative assessment. Assessment in Education: Principles, Policy & Practice, 4(3):365--379, 1997.

[19]

B. Harrington, M. Ahmadzadeh, N. Cheng, E. H. Wang, and V. Efimov. TA Marking Parties: Worth the Price of Pizza? ICER 2018.

Digital Library

[20]

P. Ihantola, T. Ahoniemi, V. Karavirta, and O. Sepp"al"a. Review of recent systems for automatic assessment of programming assignments. In Koli Calling 2010.

Digital Library

[21]

H. Keuning, J. Jeuring, and B. Heeren. Towards a systematic review of automated feedback generation for programming exercises. In ITiCSE 2016.

Digital Library

[22]

S. N. Liao, W. G. Griswold, and L. Porter. Impact of class size on student evaluations for traditional and peer instruction classrooms. In SIGCSE 2017.

Digital Library

[23]

T. MacWilliam and D. J. Malan. Streamlining grading toward better feedback. In ITiCSE 2013.

Digital Library

[24]

D. J. Nicol and D. Macfarlane-Dick. Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in higher education, 31(2):199--218, 2006.

[25]

H. L. Roediger and J. D. Karpicke. The power of testing memory: Basic research and implications for educational practice. Perspectives on Psychological Science, 1(3):181--210, 2006.

[26]

A. Singh, S. Karayev, K. Gutowski, and P. Abbeel. Gradescope: A fast, flexible, and fair system for scalable assessment of handwritten work. In L@S 2017.

Digital Library

[27]

B. R. Snyder. The hidden curriculum. New York: Knopf, 1970.

[28]

J. A. Stone. Using reflective blogs for pedagogical feedback in CS1. In SIGCSE 2012.

Digital Library

Cited By

Wulandari W(2024)The Use of Anagram Technique to Improve Student’s Vocabulary Mastery in SMPN 04 DedaiJurnal Riset dan Inovasi Pembelajaran10.51574/jrip.v4i2.15584:2(1141-1152)Online publication date: 12-Aug-2024
https://doi.org/10.51574/jrip.v4i2.1558
Agrawal ABhadhouriya SPandey ABhadoriya SShrivastava DDubey GDubey MSengar HShakywar KShrivastava V(2024)Transforming Student Assessment in Higher EducationImproving Student Assessment With Emerging AI Tools10.4018/979-8-3693-6170-2.ch013(363-386)Online publication date: 29-Nov-2024
https://doi.org/10.4018/979-8-3693-6170-2.ch013
Moshayedi ARoy AKhan ZLan HLotfi HZhang X(2024)EMTIHAN: Design of a novel approach to secure online examination challenge in remote learningEducation and Information Technologies10.1007/s10639-024-13170-4Online publication date: 4-Dec-2024
https://doi.org/10.1007/s10639-024-13170-4
Show More Cited By

Index Terms

Paper or Online?: A Comparison of Exam Grading Techniques
1. Social and professional topics
  1. Professional topics
    1. Computing education

Recommendations

Reliability, equivalence and respondent preference of computerized versus paper-and-pencil mental health questionnaires

The purpose of this study was to examine the reliability, equivalence and respondent preference of a computerized version of the General Health Questionnaire (GHQ-12), Symptom Checklist (SCL-90-R), Medical Outcomes Study Social Support Survey (MOSSSS), ...
Assessing tenth-grade students' problem solving ability online in the area of Earth sciences

This study examined tenth-grade students' (n=263) problem solving ability (PSA) online through assessing students' domain-specific knowledge (DSK) and reasoning skills (RS) in Earth sciences as well as their attitudes toward (AT) Earth sciences related ...
Participation in online discussion environments: Is it really effective?

This study aimed to develop a rubric to assess participation of students in online discussion environments. For this purpose, the study included 168 students who participated in a course offered online during the spring semester of the 2015---2016 ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ITiCSE '19: Proceedings of the 2019 ACM Conference on Innovation and Technology in Computer Science Education

July 2019

583 pages

ISBN:9781450368957

DOI:10.1145/3304221

General Chairs:
Bruce Scharlau
University of Aberdeen, UK
,
Roger McDermott
Robert Gordon University, UK
,
Program Chairs:
Arnold Pears
KTH Royal Institute of Technology, Sweden
,
Mihaela Sabin
University of New Hampshire, USA

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCSE: ACM Special Interest Group on Computer Science Education

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 July 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ITiCSE '19

Sponsor:

SIGCSE

ITiCSE '19: Innovation and Technology in Computer Science Education

July 15 - 17, 2019

Aberdeen, Scotland Uk

Acceptance Rates

Overall Acceptance Rate 552 of 1,613 submissions, 34%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
1,253
Total Downloads

Downloads (Last 12 months)312
Downloads (Last 6 weeks)42

Reflects downloads up to 25 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Wulandari W(2024)The Use of Anagram Technique to Improve Student’s Vocabulary Mastery in SMPN 04 DedaiJurnal Riset dan Inovasi Pembelajaran10.51574/jrip.v4i2.15584:2(1141-1152)Online publication date: 12-Aug-2024
https://doi.org/10.51574/jrip.v4i2.1558
Agrawal ABhadhouriya SPandey ABhadoriya SShrivastava DDubey GDubey MSengar HShakywar KShrivastava V(2024)Transforming Student Assessment in Higher EducationImproving Student Assessment With Emerging AI Tools10.4018/979-8-3693-6170-2.ch013(363-386)Online publication date: 29-Nov-2024
https://doi.org/10.4018/979-8-3693-6170-2.ch013
Moshayedi ARoy AKhan ZLan HLotfi HZhang X(2024)EMTIHAN: Design of a novel approach to secure online examination challenge in remote learningEducation and Information Technologies10.1007/s10639-024-13170-4Online publication date: 4-Dec-2024
https://doi.org/10.1007/s10639-024-13170-4
Naeem AGad AAlhalabi MYaghi MKhelifi AGhazal M(2023)AI-Based Mobile Paper Grading: Trends and Challenges2023 10th International Conference on Future Internet of Things and Cloud (FiCloud)10.1109/FiCloud58648.2023.00065(404-409)Online publication date: 14-Aug-2023
https://doi.org/10.1109/FiCloud58648.2023.00065
Hahn MNavarro Sde-La-Fuente-Valentin L(2022)LUD: An Automatic Scoring and Feedback System for Programming Assignments2022 International Conference on Advanced Learning Technologies (ICALT)10.1109/ICALT55010.2022.00118(384-386)Online publication date: Jul-2022
https://doi.org/10.1109/ICALT55010.2022.00118
Lujak MSlavkovik MLebis AVermeulen MDoniec A(2022)Objective Tests in Automated Grading of Computer Science Courses: An OverviewHandbook on Intelligent Techniques in the Educational Process10.1007/978-3-031-04662-9_12(239-268)Online publication date: 16-Jun-2022
https://doi.org/10.1007/978-3-031-04662-9_12
Gorgani HShabani S(2021)Online exams and the COVID-19 pandemic: a hybrid modified FMEA, QFD, and k-means approach to enhance fairnessSN Applied Sciences10.1007/s42452-021-04805-z3:10Online publication date: 28-Sep-2021
https://doi.org/10.1007/s42452-021-04805-z
van Heerden DGoosen L(2021)The Future of Information and Communication Technology CoursesIntelligent Computing10.1007/978-3-030-80129-8_36(507-526)Online publication date: 6-Jul-2021
https://doi.org/10.1007/978-3-030-80129-8_36
Motogna SMarcus AMolnar AVanderose BHenry JFrenay BDevroey X(2020)Adapting to online teaching in software engineering coursesProceedings of the 2nd ACM SIGSOFT International Workshop on Education through Advanced Software Engineering and Artificial Intelligence10.1145/3412453.3423194(1-6)Online publication date: 9-Nov-2020
https://dl.acm.org/doi/10.1145/3412453.3423194
Kennedy CLawson AFeaster YKraemer EGiannakos MSindre GLuxton-Reilly ADivitini M(2020)Misconception-Based Peer Feedback: A Pedagogical Technique for Reducing MisconceptionsProceedings of the 2020 ACM Conference on Innovation and Technology in Computer Science Education10.1145/3341525.3387392(166-172)Online publication date: 15-Jun-2020
https://dl.acm.org/doi/10.1145/3341525.3387392

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents