research-article

Public Access

TMOSS: Using Intermediate Assignment Work to Understand Excessive Collaboration in Large Classes

Authors:

Chris PiechAuthors Info & Claims

SIGCSE '18: Proceedings of the 49th ACM Technical Symposium on Computer Science Education

Pages 110 - 115

https://doi.org/10.1145/3159450.3159490

Published: 21 February 2018 Publication History

Abstract

As computer science classes grow, instructor workload also increases: teachers must simultaneously teach material, provide assignment feedback, and monitor student progress. At scale, it is hard to know which students need extra help, and as a result some students can resort to excessive collaboration--using online resources or peer code--to complete their work. In this paper, we present TMOSS, a tool that analyzes the intermediate steps a student takes to complete a programming assignment. We find that for three separate course offerings, TMOSS is almost twice as effective as traditional software similarity detectors in identifying the number of students who exhibit excessive collaboration. We also find that such students spend significantly less time on their assignment, use fewer class tutoring resources, and perform worse on exams than their peers. Finally, we provide a theory of the parametric distribution of typical student assignment similarity, which allows for probabilistic interpretation.

References

[1]

A. Aiken. 2014. A System for Detecting Software Plagiarism. Online. (2014). Retrieved October 25, 2016 from https://theory.stanford.edu/~aiken/moss/

[2]

T. Batane. 2010. Turning to Turnitin to Fight Plagiarism among University Students. Educational Technology & Society Vol. 13, 2 (2010), 1--12.

[3]

R. Caldarola and T. MacNeil. 2009. Dishonesty deterrence and detection: How technology can ensure distance learning test security and validity. In Proceedings of the 8th European Conference on E-Learning. 108--115.

[4]

CodeHS. 2016. Project: Breakout. Online. (2016). Retrieved October 25, 2016 from https://codehs.com/library/course/1/module/469

[5]

J. Forbes, D. J. Malan, H. Pon-Barry, S. Reges, and M. Sahami. 2017. Scaling Introductory Courses Using Undergraduate Teaching Assistants. Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education (SIGCSE '17). ACM, New York, NY, USA, 657--658.

Digital Library

[6]

P. H. Fröhlich. 2016. Department of Computer Science at The Johns Hopkins University: 600.111: Python Scripting. Online. (2016). Retrieved October 25, 2016 from http://gaming.jhu.edu/~phf/2011/fall/cs111/assignment-breakout.shtml

[7]

D. Gitchell and N. Tran. 1999. Sim: A Utility for Detecting Similarity in Computer Programs. The Proceedings of the Thirtieth SIGCSE Technical Symposium on Computer Science Education (SIGCSE '99). ACM, New York, NY, USA, 266--270.

Digital Library

[8]

D. Gries, L. Lee, S. Marschner, and W. White. 2016. CS 1110: Introduction to Computing Using Python Fall 2016. Online. (2016). Retrieved October 25, 2016 from http://www.cs.cornell.edu/courses/cs1110/2016fa/

[9]

E. J. Gumbel. 1935. Les valeurs extrêmes des distributions statistiques. Ann. Inst. Henri Poincaré Vol. 5, 2 (1935), 115--158.

[10]

E. Lau, X. Liu, C. Xiao, and X. Yu. 2004. Enhanced user authentication through keystroke biometrics. Computer and Network Security Vol. 6 (2004).

[11]

A. Maas, C. Heather, C. T. Do, R. Brandman, D. Koller, and A. Ng. 2014. Offering Verified Credentials in Massive Open Online Courses: MOOCs and technology to advance learning and learning research (Ubiquity symposium). Ubiquity, Vol. 2014, May (2014), 2.

Digital Library

[12]

S. Mahdi and M. Cenac. 2005. Estimating Parameters of Gumbel Distribution using the Methods of Moments, probability weighted Moments and maximum likelihood. Revista de Matemática: Teoría y Aplicaciones, Vol. 12, 1--2 (2005).

[13]

D. L. McCabe. 2005. Cheating among college and university students: A North American perspective. Intl. Journal for Educational Integrity Vol. 1, 1 (2005), 10--11.

[14]

N. Parlante, S. A. Wolfman, L. I. McCann, E. Roberts, C. Nevison, J. Motil, J. Cain, and S. Reges. 2006. Nifty Assignments Proceedings of the 37th SIGCSE Technical Symposium on Computer Science Education (SIGCSE '06). ACM, New York, NY, USA, 562--563.

Digital Library

[15]

A. Petersen, M. Craig, J. Campbell, and A. Tafliovich. 2016. Revisiting Why Students Drop CS1. In Proceedings of the 16th Koli Calling International Conference on Computing Education Research (Koli Calling '16). ACM, New York, NY, USA, 71--80.

Digital Library

[16]

C. Piech, M. Sahami, D. Koller, S. Cooper, and P. Blikstein. 2012. Modeling How Students Learn to Program. In Proceedings of the 43rd ACM Technical Symposium on Computer Science Education (SIGCSE '12). ACM, New York, NY, USA, 153--160.

Digital Library

[17]

J. Pierce and C. Zilles. 2017. Investigating Student Plagiarism Patterns and Correlations to Grades. Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education (SIGCSE '17). ACM, New York, NY, USA, 471--476.

Digital Library

[18]

L. Prechelt, G. Malpohl, and M. Phlippsen. 2000. JPlag: Finding plagiarisms among a set of programs. Technical Report. Universitat Karlsruhe.

[19]

E. Roberts, J. Lilly, and B. Rollins. 1995. Using Undergraduates As Teaching Assistants in Introductory Programming Courses: An Update on the Stanford Experience. In Proceedings of the Twenty-sixth SIGCSE Technical Symposium on Computer Science Education (SIGCSE '95). ACM, New York, NY, USA, 48--52.

Digital Library

[20]

S. Schleimer, D. S. Wilkerson, and A. Aiken. 2003. Winnowing: Local Algorithms for Document Fingerprinting. Proceedings of the 2003 ACM SIGMOD International Conference on Management of Data (SIGMOD '03). ACM, New York, NY, USA, 76--85.

Digital Library

[21]

J. Schneider, A. Bernstein, J. vom Brocke, K. Damevski, and D. C. Shepherd. 2016. Detecting Plagiarism based on the Creation Process. CoRR Vol. abs/1612.09183 (2016). http://arxiv.org/abs/1612.09183

[22]

J. G. Spohrer and E. Soloway. 1986. Analyzing the high frequency bugs in novice programs Papers presented at the first workshop on empirical studies of programmers on Empirical studies of programmers. Ablex Publishing Corp., Norwood, NJ, USA, 230--251. http://portal.acm.org/citation.cfm?id=28897

Digital Library

[23]

G. Whale. 1988. Plague: Plagiarism Detection Using Program Structure Tech. Rep. 8805.

[24]

M. J. Wise. 1996. YAP3: Improved Detection Of Similarities In Computer Program And Other Texts SIGCSEB: SIGCSE Bulletin (ACM Special Interest Group on Computer Science Education). ACM Press, 130--134.

Digital Library

[25]

J. Young. 2012. Dozens of plagiarism incidents are reported in Coursera's free online courses. The Chronicle of Higher Education (Aug. 2012). http://www.chronicle.com/article/Dozens-of-Plagiarism-Incidents/133697

Cited By

Haji Amin Shirazi SPang AKnight ASalloum MVahid FStone JYuen TShoop LRebelsky SPrather J(2025)Midterm Exam Outliers Efficiently Highlight Potential Cheaters on Programming AssignmentsProceedings of the 56th ACM Technical Symposium on Computer Science Education V. 110.1145/3641554.3701883(437-442)Online publication date: 12-Feb-2025
https://dl.acm.org/doi/10.1145/3641554.3701883
Sağlam THahner SSchmid LBurger ERoychoudhury APaiva AAbreu RStorey MGama KSiegmund J(2024)Automated Detection of AI-Obfuscated Plagiarism in Modeling AssignmentsProceedings of the 46th International Conference on Software Engineering: Software Engineering Education and Training10.1145/3639474.3640084(297-308)Online publication date: 14-Apr-2024
https://dl.acm.org/doi/10.1145/3639474.3640084
Liu EYuan DAhmed ACornwall EWoodrow JBurns KNie ABrunskill EPiech CFinn CStephenson BStone JBattestilli LRebelsky SShoop L(2024)A Fast and Accurate Machine Learning Autograder for the Breakout AssignmentProceedings of the 55th ACM Technical Symposium on Computer Science Education V. 110.1145/3626252.3630759(736-742)Online publication date: 7-Mar-2024
https://dl.acm.org/doi/10.1145/3626252.3630759
Show More Cited By

Index Terms

TMOSS: Using Intermediate Assignment Work to Understand Excessive Collaboration in Large Classes
1. Social and professional topics
  1. Professional topics
    1. Computing education
      1. Computing education programs
        Computer science education
        CS1
      2. Student assessment

Recommendations

The PyramidSnapshot Challenge: Understanding Student Process from Visual Output of Programs
SIGCSE '19: Proceedings of the 50th ACM Technical Symposium on Computer Science Education

In the ideal CS1 classroom, we should understand programming process---how student code evolves over time. However, for graphics-based programming assignments, the task of understanding and grading final solutions, let alone thousands of intermediate ...
Building a Community of First Year Students Improves Student Retention and Performance in Computing Courses: (Abstract Only)
SIGCSE '18: Proceedings of the 49th ACM Technical Symposium on Computer Science Education

First-year, college-level computing courses are gateway courses with low passing rates, resulting in student attrition and transfers out of computer science degrees. Learning Communities (LC) are a group of students who enroll in two or more courses, ...
Active Academic Integrity
SIGCSE '21: Proceedings of the 52nd ACM Technical Symposium on Computer Science Education

Even in an introduction to computer science course, students commit academic misconduct. Traditional plagiarism detection software like MOSS fail to give great results for assignments whose solutions are only a few lines long. We change program variable ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGCSE '18: Proceedings of the 49th ACM Technical Symposium on Computer Science Education

February 2018

1174 pages

ISBN:9781450351034

DOI:10.1145/3159450

General Chairs:
Tiffany Barnes
North Carolina State University, USA
,
Daniel Garcia
University of California, Berkeley, USA
,
Program Chairs:
Elizabeth K. Hawthorne
Union County College, USA
,
Manuel A. Pérez-Quiñones
UNC Charlotte, USA

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCSE: ACM Special Interest Group on Computer Science Education

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 February 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Science Foundation (http://dx.doi.org/10.13039/100000001)

Conference

SIGCSE '18

Sponsor:

SIGCSE

SIGCSE '18: The 49th ACM Technical Symposium on Computer Science Education

February 21 - 24, 2018

Maryland, Baltimore, USA

Acceptance Rates

SIGCSE '18 Paper Acceptance Rate 161 of 459 submissions, 35%;

Overall Acceptance Rate 1,787 of 5,146 submissions, 35%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

22
Total Citations
View Citations
704
Total Downloads

Downloads (Last 12 months)121
Downloads (Last 6 weeks)13

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Haji Amin Shirazi SPang AKnight ASalloum MVahid FStone JYuen TShoop LRebelsky SPrather J(2025)Midterm Exam Outliers Efficiently Highlight Potential Cheaters on Programming AssignmentsProceedings of the 56th ACM Technical Symposium on Computer Science Education V. 110.1145/3641554.3701883(437-442)Online publication date: 12-Feb-2025
https://dl.acm.org/doi/10.1145/3641554.3701883
Sağlam THahner SSchmid LBurger ERoychoudhury APaiva AAbreu RStorey MGama KSiegmund J(2024)Automated Detection of AI-Obfuscated Plagiarism in Modeling AssignmentsProceedings of the 46th International Conference on Software Engineering: Software Engineering Education and Training10.1145/3639474.3640084(297-308)Online publication date: 14-Apr-2024
https://dl.acm.org/doi/10.1145/3639474.3640084
Liu EYuan DAhmed ACornwall EWoodrow JBurns KNie ABrunskill EPiech CFinn CStephenson BStone JBattestilli LRebelsky SShoop L(2024)A Fast and Accurate Machine Learning Autograder for the Breakout AssignmentProceedings of the 55th ACM Technical Symposium on Computer Science Education V. 110.1145/3626252.3630759(736-742)Online publication date: 7-Mar-2024
https://dl.acm.org/doi/10.1145/3626252.3630759
Sağlam TBrödel MSchmid LHahner SRoychoudhury APaiva AAbreu RStorey M(2024)Detecting Automatic Software Plagiarism via Token Sequence NormalizationProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639192(1-13)Online publication date: 20-May-2024
https://dl.acm.org/doi/10.1145/3597503.3639192
Syndergaard CEdwards J(2024)Revisiting Plagiarism Deterrence in CS1 Through Keystroke Data2024 Intermountain Engineering, Technology and Computing (IETC)10.1109/IETC61393.2024.10564457(371-376)Online publication date: 13-May-2024
https://doi.org/10.1109/IETC61393.2024.10564457
Karnalim O(2024)Sensitive Similarity on Programming Assessments Expecting Highly Similar Submissions2024 IEEE World Engineering Education Conference (EDUNINE)10.1109/EDUNINE60625.2024.10500603(1-5)Online publication date: 10-Mar-2024
https://doi.org/10.1109/EDUNINE60625.2024.10500603
Toba HKarnalim O(2024)Machine Learning Models to Detect AI-Assisted Code Anomaly in Introductory Programming CourseAdvanced Technologies and the University of the Future10.1007/978-3-031-71530-3_11(163-181)Online publication date: 17-Dec-2024
https://doi.org/10.1007/978-3-031-71530-3_11
Wortzman BStephens-Martinez KMinnes MOla OBlank ADoyle MStephenson BDorn BSoh LBattestilli L(2023)Who's Cheating WhomProceedings of the 54th ACM Technical Symposium on Computer Science Education V. 210.1145/3545947.3569609(1210-1211)Online publication date: 1-Mar-2023
https://dl.acm.org/doi/10.1145/3545947.3569609
Allen WBelsky SKelly BBarela JPeveler MCutler BMerkle LDoyle MSheard JSoh LDorn B(2022)Metrics for Student Classroom Engagement and Correlation to Software Assignment PlagiarismProceedings of the 53rd ACM Technical Symposium on Computer Science Education V. 210.1145/3478432.3499133(1141-1141)Online publication date: 3-Mar-2022
https://dl.acm.org/doi/10.1145/3478432.3499133
Du JSong YAn MAn MBogart CSakr MMerkle LDoyle MSheard JSoh LDorn B(2022)Cheating Detection in Online Assessments via Timeline AnalysisProceedings of the 53rd ACM Technical Symposium on Computer Science Education - Volume 110.1145/3478431.3499368(98-104)Online publication date: 22-Feb-2022
https://dl.acm.org/doi/10.1145/3478431.3499368
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten