research-article

Public Access

Blending Measures of Programming and Social Behavior into Predictive Models of Student Achievement in Early Computing Courses

Authors:

Adam S. Carter,

Christopher D. Hundhausen,

Olusola AdesopeAuthors Info & Claims

ACM Transactions on Computing Education (TOCE), Volume 17, Issue 3

Article No.: 12, Pages 1 - 20

https://doi.org/10.1145/3120259

Published: 28 August 2017 Publication History

Abstract

Analyzing the process data of students as they complete programming assignments has the potential to provide computing educators with insights into both their students and the processes by which they learn to program. In prior research, we explored the relationship between (a) students’ programming behaviors and course outcomes, and (b) students’ participation within an online social learning environment and course outcomes. In both studies, we developed statistical measures derived from our data that significantly correlate with students’ course grades. Encouraged both by social theories of learning and a desire to improve the accuracy of our statistical models, we explore here the impact of incorporating our predictive measure derived from social behavior into three separate predictive measures derived from programming behaviors. We find that, in combining the measures, we are able to improve the overall predictive power of each measure. This finding affirms the importance of social interaction in the learning process, and provides evidence that predictive models derived from multiple sources of learning process data can provide significantly better predictive power by accounting for multiple factors responsible for student success.

References

[1]

A. Ahadi, R. Lister, H. Haapala, and A. Vihavainen. 2015. Exploring machine learning methods to automatically identify students in need of assistance. In Proceedings of the 11th Annual International Conference on International Computing Education Research.

Digital Library

[2]

A. Ahadi, V. Behbood, A. Vihavainen, J. Prior, and R. Lister. 2016. Students’ syntactic mistakes in writing seven different types of SQL queries and its application to predicting students’ success. In Proceedings of the 47th ACM Technical Symposium on Computing Science Education.

Digital Library

[3]

M. Ahmadzadeh, D. Elliman, and C. Higgins. 2005. An analysis of patterns of debugging among novice computer science students. Proceedings of the 10th Annual SIGCSE Conference on Innovation and Technology in Computer Science Education (ITiCSE’05). ACM Press. 84--88.

Digital Library

[4]

A. Altadmri and N. C. C. Brown. 2015. 37 million compilations: Investigating novice programming mistakes in large-scale student data. In Proceedings of the 46th ACM Technical Symposium on Computer Science Education. 522--527.

Digital Library

[5]

I. Arapakis, M. Lalmas, and G. Valkansas. 2014. Understanding within-content engagement through pattern analysis of mouse gestures. In Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management.

Digital Library

[6]

R. S. J. Baker and G. Siemens. 2014. Educational data mining and learning analytics. The Cambridge Handbook of the Learning Sciences. Cambridge University Press. 253--274.

[7]

A. Bandura. 1997. Self-Efficacy: The Exercise of Control. Worth Publishers.

[8]

S. Bergin, R. Reilly, and D. Traynor. 2005. Examining the role of self-regulated learning on introductory programming performance. In Proceedings of the 2005 ACM International Computing Education Research Workshop. ACM. 81--86.

Digital Library

[9]

P. Blikstein, M. Worsley, C. Piech, A. Gibbons, M. Sahami, and S. Cooper. 2014. Programming pluralism: Using learning analytics to detect patterns in novices’ learning of computer programming. International Journal of the Learning Sciences. 23, 4 (2014), 561--599.

[10]

J. Bransford, A. L. Brown, and R. R. Cocking, Eds. 1999. How People Learn: Brain, Mind, Experience, and School. National Academy Press.

[11]

N. C. C. Brown, M. Kolling, D. McCall, and I. Utting. 2014. Blackbox: A large scale repository of novice programmers’ activity. In Proceedings of the 45th ACM Technical Symposium on Computer Science Education.

Digital Library

[12]

A. Bulling, U. Blanke, and B. Schiele. 2014. A tutorial on human activity recognition using body-worn inertial sensors. ACM Computing Surveys. 46, 3 (Jan. 2014).

Digital Library

[13]

T. Busjahn, C. Schulte, B. Sharif, Simon, A. Begel, M. Hansen, R. Bednarik, P. Orlov, G. Shchekotova, and M. Antropova. 2014. Eye tracking in computing education. In Proceedings of the 10th Annual Conference on International Computing Education Research (Glasgow, Scotland, 2014).

[14]

P. F. Campbell and G. P. McCabe. 1984. Predicting the success of freshmen in a computer science major. CommunIcations of the ACM. 27, 11 (1984), 1108--1113.

Digital Library

[15]

A. S. Carter. 2012. Supporting the virtual design studio through social programming environments. In Proceedings of the 9th Annual International Conference on International Computing Education Researc, 157--158.

Digital Library

[16]

A. S. Carter, C. D. Hundhausen, and O. Adesope. 2015. The normalized programming state model: Predicting student performance in computing courses based on programming behavior. In Proceedings of the 11th Annual International Conference on International Computing Education Research. ACM. 141--150.

Digital Library

[17]

A. S. Carter and C. D. Hundhausen. 2015. The design of a programming environment to support greater social awareness and participation in early computing courses. Journal of Computing Sciences in Colleges. 31, 1 (2015), 143--153.

Digital Library

[18]

A. S. Carter and C. D. Hundhausen. 2016. With a little help from my friends: An empirical study of the interplay of students’ social activities, programming activities, and course success. In Proceedings of the 2016 ACM Conference on International Computing Education Research. ACM. 201--209.

Digital Library

[19]

J. Carter and D. Prasun. 2010. Are you having difficulty? In Proceedings of the 2010 ACM Conference on Computer Supported Cooperative Work.

Digital Library

[20]

A. Field. 2013. Discovering Statistics Using IBM SPSS Statistics.

[21]

D. R. Goldenson and B. J. Wang. 1991. Use of structure editing tools by novice programmers. Empirical Studies of Programmers: Fourth Workshop. Ablex. 99--120.

[22]

M. J. Graham, J. Federick, A. Byers-Winston, A. B. Hunber, and J. Handelsman. 2013. Increasing persistence of college students in STEM. Science. 341, 27 (Sept. 2013), 1455--1456.

[23]

M. Guzdial. 1994. Software-realized scaffolding to facilitate programming for science learning. Interactive learning Environments. 4, 1 (1994), 1--44.

[24]

B. Hartmann, D. MacDougall, J. Brandt, and S. R. Klemmer. 2010. What would other programmers do: Suggesting solutions to error messages. In Proceedings of the 28th Conference on Human Factors in Computing Systems. ACM. 1019--1028.

Digital Library

[25]

C. D. Hundhausen, J. L. Brown, S. Farley, and D. Skarpas. 2006. A methodology for analyzing the temporal evolution of novice programs based on semantic components. In Proceedings of the 2006 ACM International Computing Education Research Workshop. ACM. 45--56.

Digital Library

[26]

C. D. Hundhausen, D. M. Olivares, and A. S. Carter. 2017. IDE-Based learning analytics for computing education: A process model, critical review, and research agenda. Transactions on Computing Education (2017).

[27]

C. D. Hundhausen, A. S. Carter, and O. Adesope. 2015. Supporting programming assignments with activity streams: An empirical study. In Proceedings of the 46th ACM Technical Symposium on Computer Science Education. 320--325.

Digital Library

[28]

P. Ihantola, A. Vihavainen, A. Ahadi, M. Butler, J. Borstler, S. Edwards, E. Isohanni, A. Korhonen, A. Petersen, K. Rivers, M. A. Rubio, J. Sheard, B. Skupas, J. Spacco, C. Szabo, and D. Toll. 2015. Educational data mining and learning analytics in programming: Literature review and case studies. In Proceedings of the 2015 ITiCSE on Working Group Reports (New York, NY, USA, 2015), 41--63.

Digital Library

[29]

M. C. Jadud. 2006. An Exploration of Novice Compilation Behaviour in Bluej.

[30]

M. C. Jadud. 2006. Methods and tools for exploring novice compilation behaviour. In Proceedings of the 2nd International Workshop on Computing Education Research. ACM. 73--84.

Digital Library

[31]

M. C. Jadud and B. Dorn. 2015. Aggregate compilation behavior: Findings and implications from 27,698 Users. In Proceedings of the 11th Annual International Conference on International Computing Education Research.

Digital Library

[32]

D. Jeske, C. Stamov-Rossnagel, and J. Backhaus. 2014. Learner characteristics predict performance and confidence in e-Learning: An analysis of user behavior and self-evaluation. Journal of Interactive Learning Research. 25, 4 (2014), 509--529.

[33]

C. M. Kessler and J. R. Anderson. 1986. A model of novice debugging in LISP. Empirical Studies of Programmers. 198--212.

[34]

G. Khodabandelou, C. Hug, R. Deneckere, and C. Salinesi. 2014. Unsupervised discovery of intentional process models from event logs. In Proceedings of the 11th Working Conference on Mining Software Repositories (2014).

Digital Library

[35]

E. Kim, S. Helal, and D. Cook. 2010. Human activity recognition and pattern discovery. Pervasive Computing. 9, 1 (Jan. 2010), 48--53.

Digital Library

[36]

M. Kölling, B. Quig, A. Patterson, and J. Rosenberg. 2003. The BlueJ system and its pedagogy. Journal of Compuer Science Education. 13, 4 (2003), 249--268.

[37]

J. Lave and E. Wenger. 1991. Situated Learning: Legitimate Peripheral Participation. Cambridge University Press.

[38]

J. Leinonen, K. Longi, A. Klami, and A. Vihavainen. 2016. Automatic inference of programming performance and experience from typing patterns. Proceedings of the 47th ACM Technical Symposium on Computing Science Education.

Digital Library

[39]

W. Ma, O. O. Adesope, J. C. Nesbit, and Q. Liu. 2014. Intelligent tutoring systems and learning outcomes: A meta-analytic survey. Journal of Educational Psychology. 106, 2007 (2014), 901--918.

[40]

B. Minor, J. R. Doppa, and D. J. Cook. 2015. Data-driven activity prediction: Algorithms, evaluation methodology, and applications. In Proceedings of the 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2015).

Digital Library

[41]

D. Mujumdar, M. Hallenbach, B. Liu, and B. Hartmann. 2011. Crowdsourcing suggestions to programming problems for dynamic web development languages. CHI’11 Extended Abstracts on Human Factors in Computing Systems, 1525--1530.

[42]

Online Studio-Based Learning Environment. 2012. Retrieved Jan. 14, 2014 from https://osble.org/.

[43]

A. Peterson, J. Spacco, and A. Vihavainen. 2015. An exploration of error quotient in multiple contexts. In Proceedings of the 15th Koli Calling Conference on Computing Education Research.

Digital Library

[44]

M. B. Rosson, J. M. Carroll, and H. Sinha. 2011. Orientation of undergraduates toward careers in the computer and information sciences: Gender, self-efficacy and social support. ACM Transactions on Computing Education. 11, 3 (Oct. 2011), 1--23.

Digital Library

[45]

M. B. Rosson, J. M. Carroll, and H. Sinha. 2011. Orientation of undergraduates toward careers in the computer and information sciences: Gender, self-efficacy and social support. Transactions on Computing Education 11, 3 (2011), 1--23.

Digital Library

[46]

D. H. Schunk. 2012. Learning Theories: An Educational Perspective. Merrill Prentice Hall.

[47]

W. R. Shadish, T. D. Cook, and D. T. Campbell. 2002. Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Houghton Mifflin Company.

[48]

R. E. Slavin. 2011. Educational Psychology: Theory and Practice. Pearson Education.

[49]

J. C. Spohrer. 1992. MARCEL: Simulating the Novice Programmer. Ablex.

[50]

E. S. Tabano, M. M. T. Rodrigo, and M. C. Jadud. 2011. Predicting at-risk novice java programmers through the analysis of online protocols. In Proceedings of the 7th International Workshop on Computing Education Research, 85--92.

Digital Library

[51]

U. S. Department of Education, Office of Educational Technology. 2012. Enhancing Teaching and Learning through Educational Data Mining and Learning Analytics: An Issue Brief.

[52]

K. Verbert and E. Duval. 2012. Learning analytics. Learning and Education. 1, 8 (2012).

[53]

Visual Studio - Microsoft Developer Tools. 2015. Retrieved April 20, 2015 from http://www.visualstudio.com.

[54]

C. Watson, F. W. B. Li, and J. L. Godwin. 2014. No tests required: Comparing traditional and dynamic predictors of programming success. In Proceedings of the 45th ACM Technical Symposium on Computer Science Education. ACM. 469--474.

Digital Library

[55]

C. Watson, F. W. B. Li, and J. L. Godwin. 2013. Predicting performance in an introductory programming course by logging and analyzing student programming behavior. In Proceedings of the 2013 IEEE 13th International Conference on Advanced Learning Technologies (2013), 319--323.

Digital Library

[56]

B. C. Wilson and S. Shrock. 2001. Contributing to success in an introductory computer science course: A study of twelve factors. SIGCSE Bulletin 33, 1 (2001), 184--188.

Digital Library

[57]

A. F. Wise. 2014. Designing pedagogical interventions to support student use of learning analytics. In Proceedings of the 4th International Conference on Learning Analytics and Knowledge, 203--211.

Digital Library

Cited By

Fischer BBarthelmes BPanitz SIwer EDörner R(2024)Seeking Consent for Programming Process Data Collection with Trustee-Based EncryptionProceedings of the 2024 ACM Conference on International Computing Education Research - Volume 110.1145/3632620.3671125(131-142)Online publication date: 12-Aug-2024
https://dl.acm.org/doi/10.1145/3632620.3671125
Liu W(2024)Design and user behavior analysis of an English learning social platform based on digital entertainment content recommendation algorithmEntertainment Computing10.1016/j.entcom.2024.100734(100734)Online publication date: May-2024
https://doi.org/10.1016/j.entcom.2024.100734
Malmi LSheard JSinclair JKinnunen PSimon (2023)Domain-Specific Theories of Teaching Computing: Do they Inform Practice?Proceedings of the 23rd Koli Calling International Conference on Computing Education Research10.1145/3631802.3631810(1-15)Online publication date: 13-Nov-2023
https://dl.acm.org/doi/10.1145/3631802.3631810
Show More Cited By

Index Terms

Blending Measures of Programming and Social Behavior into Predictive Models of Student Achievement in Early Computing Courses

Recommendations

IDE-Based Learning Analytics for Computing Education: A Process Model, Critical Review, and Research Agenda
Special Issue on Learning Analytics

In recent years, learning process data have become increasingly easy to collect through computer-based learning environments. This has led to increased interest in the field of learning analytics, which is concerned with leveraging learning process data ...
Using Learning Analytics to Promote Student Engagement and Achievement in Blended Learning: An Empirical Study
ICEBT '18: Proceedings of the 2018 2nd International Conference on E-Education, E-Business and E-Technology

The emergence of blended learning has huge impact on traditional learning. Blended learning has its own unique characteristics combining the advantages of traditional learning and online learning. However, some problems of blended learning have also ...
Self-regulated learning strategies predict learner behavior and goal attainment in Massive Open Online Courses

Individuals with strong self-regulated learning (SRL) skills, characterized by the ability to plan, manage and control their learning process, can learn faster and outperform those with weaker SRL skills. SRL is critical in learning environments that ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Computing Education

ACM Transactions on Computing Education Volume 17, Issue 3

Special Issue on Learning Analytics

September 2017

116 pages

EISSN:1946-6226

DOI:10.1145/3135995

Editor:
Chris Hundhausen
Washington State University, USA

Issue’s Table of Contents

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 August 2017

Accepted: 01 June 2017

Revised: 01 June 2017

Received: 01 September 2016

Published in TOCE Volume 17, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

National Science Foundation

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

35
Total Citations
View Citations
1,002
Total Downloads

Downloads (Last 12 months)130
Downloads (Last 6 weeks)21

Reflects downloads up to 10 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Fischer BBarthelmes BPanitz SIwer EDörner R(2024)Seeking Consent for Programming Process Data Collection with Trustee-Based EncryptionProceedings of the 2024 ACM Conference on International Computing Education Research - Volume 110.1145/3632620.3671125(131-142)Online publication date: 12-Aug-2024
https://dl.acm.org/doi/10.1145/3632620.3671125
Liu W(2024)Design and user behavior analysis of an English learning social platform based on digital entertainment content recommendation algorithmEntertainment Computing10.1016/j.entcom.2024.100734(100734)Online publication date: May-2024
https://doi.org/10.1016/j.entcom.2024.100734
Malmi LSheard JSinclair JKinnunen PSimon (2023)Domain-Specific Theories of Teaching Computing: Do they Inform Practice?Proceedings of the 23rd Koli Calling International Conference on Computing Education Research10.1145/3631802.3631810(1-15)Online publication date: 13-Nov-2023
https://dl.acm.org/doi/10.1145/3631802.3631810
Hackett PWermelinger MKear KDouce C(2023)Using a Virtual Computing Lab to Teach Programming at a DistanceProceedings of the 7th Conference on Computing Education Practice10.1145/3573260.3573262(5-8)Online publication date: 6-Jan-2023
https://dl.acm.org/doi/10.1145/3573260.3573262
Gómez AMarco-Galindo MMinguillón J(2023)Evaluation of an Intervention on Activity Planning in CS1IEEE Revista Iberoamericana de Tecnologias del Aprendizaje10.1109/RITA.2023.330217418:3(287-294)Online publication date: Aug-2023
https://doi.org/10.1109/RITA.2023.3302174
Wan HLi MZhong ZLuo X(2023)Early Prediction of Student Performance with LSTM-Based Deep Neural Network2023 IEEE 47th Annual Computers, Software, and Applications Conference (COMPSAC)10.1109/COMPSAC57700.2023.00026(132-141)Online publication date: Jun-2023
https://doi.org/10.1109/COMPSAC57700.2023.00026
Alghamdi OClinch SAlhamadi MJay C(2023)Novice Programmers Strategies for Online Resource Use and Their Impact on Source Code2023 IEEE/ACM 16th International Conference on Cooperative and Human Aspects of Software Engineering (CHASE)10.1109/CHASE58964.2023.00018(92-104)Online publication date: May-2023
https://doi.org/10.1109/CHASE58964.2023.00018
Omer UTehseen RFarooq MAbid A(2023)Learning analytics in programming courses: Review and implicationsEducation and Information Technologies10.1007/s10639-023-11611-028:9(11221-11268)Online publication date: 16-Feb-2023
https://dl.acm.org/doi/10.1007/s10639-023-11611-0
Daniels MMalmi LPears ASimon (2023)What is Computing Education Research (CER)?Past, Present and Future of Computing Education Research10.1007/978-3-031-25336-2_2(9-31)Online publication date: 5-Jan-2023
https://doi.org/10.1007/978-3-031-25336-2_2
R JN.A D(2022)A Recent Exploration on Student Performance Analysis using Educational Data Mining MethodsInternational Journal of Innovative Technology and Exploring Engineering10.35940/ijitee.G9188.081192211:9(6-10)Online publication date: 30-Aug-2022
https://doi.org/10.35940/ijitee.G9188.0811922
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Media

Figures

Other

Tables

View Issue’s Table of Contents