research-article

Public Access

Using Programming Process Data to Detect Differences in Students' Patterns of Programming

Authors: Adam Scott Carter, Christopher David HundhausenAuthors Info & Claims

SIGCSE '17: Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education

Pages 105 - 110

https://doi.org/10.1145/3017680.3017785

Published: 08 March 2017 Publication History

Abstract

Analyzing the process data of students as they complete programming assignments has the potential to provide computing educators with insights into their students and the processes by which they learn to program. In prior work, we developed a statistical model that accurately predicts students' homework grades. In this paper, we investigate the relationship between the paths that students take through the programming states on which our statistical model is based, and their overall course achievement. Examining the frequency of the most common transition paths revealed significant differences between students who earned A's, B's, and C's in a CS 2 course. Our results indicate that a) students of differing achievement levels approach programming tasks differently, and b) these differences can be automatically detected, opening up the possibility that they could be leveraged for pedagogical gain.

References

[1]

Ahadi, A. et al. 2015. Exploring Machine Learning Methods to Automatically Identify Students in Need of Assistance. Proceedings of the Eleventh Annual International Conference on International Computing Education Research (Omaha, NE, USA, 2015).

Digital Library

[2]

Altadmri, A. and Brown, N.C.C. 2015. 37 Million Compilations: Investigating Novice Programming Mistakes in Large-Scale Student Data. Proceedings of the 46th ACM Technical Symposium on Computer Science Education (Kansas City, MO, USA, 2015), 522--527.

Digital Library

[3]

Baker, R.S.J. and Siemens, G. 2014. Educational data mining and learning analytics. The Cambridge Handbook of the Learning Sciences. Cambridge University Press. 253--274.

[4]

Bergin, S. et al. 2005. Examining the role of self-regulated learning on introductory programming performance. Proc. 2005 ACM International Computing Education Research Workshop. ACM Press. 81--86.

Digital Library

[5]

Bransford, J. et al. eds. 1999. How people learn: Brain, mind, experience, and school. National Academy Press.

[6]

Carter, A.S. et al. 2015. The Normalized Programming State Model: Predicting Student Performance in Computing Courses Based on Programming Behavior. Proceedings of the Eleventh Annual International Conference on International Computing Education Research. ACM. 141--150.

Digital Library

[7]

Carter, A.S. and Hundhausen, C.D. 2015. The Design of a Programming Environment to Support Greater Social Awareness and Participation in Early Computing Courses. J. Comput. Sci. Coll. 31, 1 (Oct. 2015), 143--153.

Digital Library

[8]

Ihantola, P. et al. 2015. Educational Data Mining and Learning Analytics in Programming: Literature Review and Case Studies. Proceedings of the 2015 ITiCSE on Working Group Reports (New York, NY, USA, 2015), 41--63.

Digital Library

[9]

Jadud, M.C. 2006. Methods and Tools for Exploring Novice Compilation Behaviour. Proceedings of the Second International Workshop on Computing Education Research. ACM. 73--84.

Digital Library

[10]

Jadud, M.C. and Dorn, B. 2015. Aggregate Compilation Behavior: Findings and Implications from 27,698 Users. Proceedings of the Eleventh Annual International Conference on International Computing Education Research (Omaha, NE, USA, 2015).

Digital Library

[11]

Jeske, D. et al. 2014. Learner characteristics predict performance and confidence in e-Learning: An analysis of user behavior and self-evaluation. Journal of Interactive Learning Research. 25, 4 (2014), 509--529.

[12]

Leinonen, J. et al. 2016. Automatic Inference of Programming Performance and Experience from Typing Patterns. Proceedings of the 47th ACM Technical Symposium on Computing Science Education (Memphis, TN, USA, 2016).

Digital Library

[13]

Ma, W. et al. 2014. Intelligent tutoring systems and learning outcomes: A meta-analytic survey. Journal of Educational Psychology. 106, 2007 (2014), 901--918.

[14]

Minor, B. et al. 2015. Data-Driven Activity Prediction: Algorithms, Evaluation Methodology, and Applications. Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2015).

Digital Library

[15]

Schunk, D.H. 2012. Learning theories: An educational perspective. Merrill Prentice Hall.

[16]

Slavin, R.E. 2011. Educational psychology: Theory and practice. Pearson Education.

[17]

U.S. Department of Education, Office of Educational Technology 2012. Enhancing Teaching and Learning through Educational Data Mining and Learning Analytics: An Issue Brief.

[18]

Watson, C. et al. 2013. Predicting Performance in an Introductory Programming Course by Logging and Analyzing Student Programming Behavior. Proceedings of the 2013 IEEE 13th International Conference on Advanced Learning Technologies (2013), 319--323.

Digital Library

Cited By

Zhong HNiu JLi J(2024)InsProg: Supporting Teaching Through Visual Analysis of Students’ Programming ProcessesProceedings of the 2024 International Conference on Advanced Visual Interfaces10.1145/3656650.3656668(1-5)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3656650.3656668
Švábenský VPankiewicz MZhang JCloude EBaker RFouh EMonga MLonati VBarendsen ESheard JPaterson J(2024)Comparison of Three Programming Error Measures for Explaining Variability in CS1 GradesProceedings of the 2024 on Innovation and Technology in Computer Science Education V. 110.1145/3649217.3653563(87-93)Online publication date: 3-Jul-2024
https://dl.acm.org/doi/10.1145/3649217.3653563
Brown NMac VWeill-Tessier PKölling MStephenson BStone JBattestilli LRebelsky SShoop L(2024)Writing Between the Lines: How Novices Construct Java ProgramsProceedings of the 55th ACM Technical Symposium on Computer Science Education V. 110.1145/3626252.3630968(165-171)Online publication date: 7-Mar-2024
https://dl.acm.org/doi/10.1145/3626252.3630968
Show More Cited By

Index Terms

Using Programming Process Data to Detect Differences in Students' Patterns of Programming
1. General and reference
  1. Cross-computing tools and techniques
    1. Metrics
2. Social and professional topics
  1. Professional topics
    1. Computing education
      1. Computing education programs
        Computer science education

Recommendations

The Normalized Programming State Model: Predicting Student Performance in Computing Courses Based on Programming Behavior
ICER '15: Proceedings of the eleventh annual International Conference on International Computing Education Research

Educators stand to benefit from advance predictions of their students' course performance based on learning process data collected in their courses. Indeed, such predictions can help educators not only to identify at-risk students, but also to better ...
Computer programming and ninth-grade students' understanding of variable
Investigating the Applicability of the Normalized Programming State Model to BlueJ Programmers
Koli Calling '18: Proceedings of the 18th Koli Calling International Conference on Computing Education Research

It has long been a goal of educators to accurately identify at-risk students early enough in the term to intervene, and the increasing availability of programming process data and learning analytics tools has brought education researchers closer to ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGCSE '17: Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education

March 2017

838 pages

ISBN:9781450346986

DOI:10.1145/3017680

General Chairs:
Michael E. Caspersen
Aarhus University
,
Stephen H. Edwards
Virginia Tech
,
Program Chairs:
Tiffany Barnes
North Carolina State University
,
Daniel D. Garcia
University of California, Berkeley

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCSE: ACM Special Interest Group on Computer Science Education

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 March 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Science Foundation

Conference

SIGCSE '17

Sponsor:

SIGCSE

SIGCSE '17: The 48th ACM Technical Symposium on Computer Science Education

March 8 - 11, 2017

Washington, Seattle, USA

Acceptance Rates

SIGCSE '17 Paper Acceptance Rate 105 of 348 submissions, 30%;

Overall Acceptance Rate 1,595 of 4,542 submissions, 35%

Upcoming Conference

SIGCSE Virtual 2024

Sponsor:
sigcse

1st ACM Virtual Global Computing Education Conference

December 5 - 8, 2024

Virtual Event , NC , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

21
Total Citations
View Citations
781
Total Downloads

Downloads (Last 12 months)123
Downloads (Last 6 weeks)25

Reflects downloads up to 12 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhong HNiu JLi J(2024)InsProg: Supporting Teaching Through Visual Analysis of Students’ Programming ProcessesProceedings of the 2024 International Conference on Advanced Visual Interfaces10.1145/3656650.3656668(1-5)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3656650.3656668
Švábenský VPankiewicz MZhang JCloude EBaker RFouh EMonga MLonati VBarendsen ESheard JPaterson J(2024)Comparison of Three Programming Error Measures for Explaining Variability in CS1 GradesProceedings of the 2024 on Innovation and Technology in Computer Science Education V. 110.1145/3649217.3653563(87-93)Online publication date: 3-Jul-2024
https://dl.acm.org/doi/10.1145/3649217.3653563
Brown NMac VWeill-Tessier PKölling MStephenson BStone JBattestilli LRebelsky SShoop L(2024)Writing Between the Lines: How Novices Construct Java ProgramsProceedings of the 55th ACM Technical Symposium on Computer Science Education V. 110.1145/3626252.3630968(165-171)Online publication date: 7-Mar-2024
https://dl.acm.org/doi/10.1145/3626252.3630968
Gupta AJindal MGoyal A(2024)Identification of Student Programming Patterns through Clickstream Data2024 IEEE International Conference on Computing, Power and Communication Technologies (IC2PCT)10.1109/IC2PCT60090.2024.10486775(1153-1158)Online publication date: 9-Feb-2024
https://doi.org/10.1109/IC2PCT60090.2024.10486775
Dol SJawandhiya P(2024)Systematic Review and Analysis of EDM for Predicting the Academic Performance of StudentsJournal of The Institution of Engineers (India): Series B10.1007/s40031-024-00998-0105:4(1021-1071)Online publication date: 4-Feb-2024
https://doi.org/10.1007/s40031-024-00998-0
Mai TCrane MBezbradica M(2023)Students’ Learning Behaviour in Programming Education Analysis: Insights from Entropy and Community DetectionEntropy10.3390/e2508122525:8(1225)Online publication date: 17-Aug-2023
https://doi.org/10.3390/e25081225
Li YLin SZhen ZHe DQian JXie W(2023)Evaluation Method of GA-BP Neural Network Programming Ability Based on Entropy Weight-Deviation2023 10th International Forum on Electrical Engineering and Automation (IFEEA)10.1109/IFEEA60725.2023.10429436(1183-1187)Online publication date: 3-Nov-2023
https://doi.org/10.1109/IFEEA60725.2023.10429436
Leinonen JCastro FHellas A(2022)Time-on-task metrics for predicting performanceACM Inroads10.1145/353456413:2(42-49)Online publication date: 17-May-2022
https://dl.acm.org/doi/10.1145/3534564
Leinonen JCastro FHellas AMerkle LDoyle MSheard JSoh LDorn B(2022)Time-on-Task Metrics for Predicting PerformanceProceedings of the 53rd ACM Technical Symposium on Computer Science Education - Volume 110.1145/3478431.3499359(871-877)Online publication date: 22-Feb-2022
https://dl.acm.org/doi/10.1145/3478431.3499359
Araya IBeas VStamulis KAllende‐Cid H(2022)Predicting student performance in computing courses based on programming behaviorComputer Applications in Engineering Education10.1002/cae.2251930:4(1264-1276)Online publication date: 25-Apr-2022
https://doi.org/10.1002/cae.22519
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents