research-article

Public Access

Exploring the Value of Different Data Sources for Predicting Student Performance in Multiple CS Courses

Authors:

Soohyun Nam Liao,

Daniel Zingaro,

Christine Alvarado,

William G. Griswold,

Leo PorterAuthors Info & Claims

SIGCSE '19: Proceedings of the 50th ACM Technical Symposium on Computer Science Education

Pages 112 - 118

https://doi.org/10.1145/3287324.3287407

Published: 22 February 2019 Publication History

Abstract

A number of recent studies in computer science education have explored the value of various data sources for early prediction of students' overall course performance. These data sources include responses to clicker questions, prerequisite knowledge, instrumented student IDEs, quizzes, and assignments. However, these data sources are often examined in isolation or in a single course. Which data sources are most valuable, and does course context matter? To answer these questions, this study collected student grades on prerequisite courses, Peer Instruction clicker responses, online quizzes, and assignments, from five courses (over 1000 students) across the CS curriculum at two institutions. A trend emerges suggesting that for upper-division courses, prerequisite grades are most predictive; for introductory programming courses, where no prerequisite grades were available, clicker responses were the most predictive. In concert, prerequisites and clicker responses generally provide highly accurate predictions early in the term, with assignments and online quizzes sometimes providing incremental improvements. Implications of these results for both researchers and practitioners are discussed.

References

[1]

https://www.rdocumentation.org/packages/lmtest/versions/0.9--36/topics/lrtest.

[2]

https://www.rdocumentation.org/packages/stats/versions/3.5.1/topics/glm.

[3]

A. Ahadi, R. Lister, H. Haapala, and A. Vihavainen. Exploring machine learning methods to automatically identify students in need of assistance. In Proceedings of the 11th Conference on International Computing Education Research, pages 121--130, 2015.

Digital Library

[4]

A. Anthony and M. Raney. Bayesian network analysis of computer science grade distributions. In Proceedings of the 43rd Technical Symposium on Computer Science Education, pages 649--654, 2012.

Digital Library

[5]

K. Castro-Wunsch, A. Ahadi, and A. Petersen. Evaluating neural networks as a method for identifying students in need of assistance. In Proceedings of the 48th Technical Symposium on Computer Science Education, pages 111--116, 2017.

Digital Library

[6]

relax CRA Enrollment Committee Institution Subgroup. Generation CS: Computer science undergraduate enrollments surge since 2006. Computing Research Association, 2017.

[7]

C. H. Crouch and E. Mazur. Peer instruction: Ten years of experience and results. American Journal of Physics, 69, 2001.

[8]

H. Danielsiek and J. Vahrenhold. Stay on these roads: Potential factors indicating students' performance in a CS2 course. In Proceedings of the 47th Technical Symposium on Computer Science Education, pages 12--17, 2016.

Digital Library

[9]

D. Hagan and S. Markham. Does it help to have some programming experience before beginning a computing degree program? In ACM SIGCSE Bulletin, volume 32, pages 25--28, 2000.

Digital Library

[10]

L. Lepp"anen, J. Leinonen, P. Ihantola, and A. Hellas. Predicting academic success based on learning material usage. In Proceedings of the 18th Conference on Information Technology Education, pages 13--18, 2017.

Digital Library

[11]

S. N. Liao, D. Zingaro, M. A. Laurenzano, W. G. Griswold, and L. Porter. Lightweight, early identification of at-risk CS1 students. In Proceedings of the 12th Conference on International Computing Education Research, pages 123--131, 2016.

Digital Library

[12]

A. Lishinski, A. Yadav, J. Good, and R. Enbody. Learning to program: Gender differences and interactive effects of students' motivation, goals, and self-efficacy on performance. In Proceedings of the 12th Conference on International Computing Education Research, pages 211--220, 2016.

Digital Library

[13]

M. Natrella. NIST/SEMATECH e-handbook of statistical methods. http://www.itl.nist.gov/div898/handbook, 2010.

[14]

C. G. Petersen and T. G. Howe. Predicting Academic Success in Introduction to Computers. Association for Educational Data Systems, 12(4):182--191, 1979.

[15]

L. Porter, C. Bailey Lee, and B. Simon. Halving fail rates using peer instruction: a study of four computer science courses. In Proceedings of the 44th technical symposium on Computer science education, pages 177--182, 2013.

Digital Library

[16]

L. Porter, D. Bouvier, Q. Cutts, S. Grissom, C. Lee, R. McCartney, D. Zingaro, and B. Simon. A multi-institutional study of peer instruction in introductory computing. In Proceedings of the 47th Technical Symposium on Computer Science Education, pages 358--363, 2016.

Digital Library

[17]

L. Porter, D. Zingaro, and R. Lister. Predicting student success using fine grain clicker data. In Proceedings of the 10th Conference on International Computing Education Research, pages 51--58, 2014.

Digital Library

[18]

K. Quille and S. Bergin. Programming: Predicting student success early in CS1. a re-validation and replication study. In Proceedings of the 23rd Conference on Innovation and Technology in Computer Science Education, pages 15--20, 2018.

Digital Library

[19]

V. Ramalingam, D. LaBelle, and S. Wiedenbeck. Self-efficacy and mental models in learning to program. SIGCSE Bulletin, 36:171--175, 2004.

Digital Library

[20]

A. Robins. Learning edge momentum: A new account of outcomes. Computer Science Education, 20(1):37--71, 2010.

[21]

M. M. T. Rodrigo, R. S. Baker, M. C. Jadud, A. C. M. Amarra, T. Dy, M. B. V. Espejo-Lahoz, S. A. L. Lim, S. A. Pascua, J. O. Sugay, and E. S. Tabanao. Affective and behavioral predictors of novice programmer achievement. In Proceedings of the 14th Conference on Innovation and Technology in Computer Science Education, pages 156--160, 2009.

Digital Library

[22]

V. L. Sauter. Predicting computer programming skill. Computers & Education, 10(2):299--302, 1986.

Digital Library

[23]

M. Vellukunnel, P. Buffum, K. E. Boyer, J. Forbes, S. Heckman, and K. Mayer-Patel. Deconstructing the discussion forum: Student questions and computer science learning. In Proceedings of the 48th Technical Symposium on Computer Science Education, pages 603--608, 2017.

Digital Library

[24]

C. Watson and F. W. Li. Failure rates in introductory programming revisited. In Proceedings of the 19th Conference on Innovation and Technology in Computer Science Education, pages 39--44, 2014.

Digital Library

[25]

C. Watson, F. W. Li, and J. L. Godwin. No tests required: Comparing traditional and dynamic predictors of programming success. In Proceedings of the 45th Technical Symposium on Computer Science Education, pages 469--474, 2014.

Digital Library

[26]

B. C. Wilson and S. Shrock. Contributing to success in an introductory computer science course: a study of twelve factors. SIGCSE Bulletin, 33:184--188, 2001.

Digital Library

[27]

J. R. Wilson and K. A. Lorenz. Short history of the logistic regression model. In Modeling Binary Correlated Responses using SAS, SPSS and R, pages 17--23, 2015.

[28]

D. Zingaro. Peer instruction contributes to self-efficacy in CS1. In Proceedings of the 45th technical symposium on Computer science education, pages 373--378, 2014.

Digital Library

[29]

D. Zingaro, M. Craig, L. Porter, B. A. Becker, Y. Cao, P. Conrad, D. Cukierman, A. Hellas, D. Loksa, and N. Thota. Achievement goals in CS1: Replication and extension. In Proceedings of the 49th Technical Symposium on Computer Science Education, pages 687--692, 2018.

Digital Library

Cited By

Rys-Recker NHao Q(2024)Early Identification of Struggling Students in Large Computer Science Courses: A Replication Study2024 IEEE 48th Annual Computers, Software, and Applications Conference (COMPSAC)10.1109/COMPSAC61105.2024.00022(88-93)Online publication date: 2-Jul-2024
https://doi.org/10.1109/COMPSAC61105.2024.00022
Poitras EDempsey DCrane BSimpson SSiegel A(2023)The Applications of Learning Analytics to Enhance Learning and Engagement in Introductory Programming InstructionPerspectives on Learning Analytics for Maximizing Student Outcomes10.4018/978-1-6684-9527-8.ch005(89-108)Online publication date: 24-Oct-2023
https://doi.org/10.4018/978-1-6684-9527-8.ch005
Bertolini RFinch SNehm R(2023)An application of Bayesian inference to examine student retention and attrition in the STEM classroomFrontiers in Education10.3389/feduc.2023.10738298Online publication date: 14-Feb-2023
https://doi.org/10.3389/feduc.2023.1073829
Show More Cited By

Index Terms

Exploring the Value of Different Data Sources for Predicting Student Performance in Multiple CS Courses
1. Social and professional topics
  1. Professional topics
    1. Computing education
      1. Computing education programs
        Computer science education
        CS1

Recommendations

Exploring studio-based instructional models for computing education
SIGCSE '08: Proceedings of the 39th SIGCSE technical symposium on Computer science education

While the demand for college graduates with computing skills continues to rise, such skills no longer equate to mere programming skills. Modern day computing jobs demand design, communication, and collaborative work skills as well. Since traditional ...
Exploring studio-based instructional models for computing education
SIGCSE 08

While the demand for college graduates with computing skills continues to rise, such skills no longer equate to mere programming skills. Modern day computing jobs demand design, communication, and collaborative work skills as well. Since traditional ...
Guiding Students to Discover CS Concepts and Develop Process Skills Using POGIL
SIGCSE '20: Proceedings of the 51st ACM Technical Symposium on Computer Science Education

This workshop introduces Process Oriented Guided Inquiry Learning (POGIL) to anyone who teaches CS or related subjects. In a POGIL classroom, teams of 3-4 learners work on activities with a particular structure based on learning cycles. Through scripted ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGCSE '19: Proceedings of the 50th ACM Technical Symposium on Computer Science Education

February 2019

1364 pages

ISBN:9781450358903

DOI:10.1145/3287324

General Chairs:
Elizabeth K. Hawthorne
Union County College, USA
,
Manuel A. Pérez-Quiñones
University of North Carolina at Charlotte, USA
,
Program Chairs:
Sarah Heckman
North Carolina State University, USA
,
Jian Zhang
Texas Woman's University, USA

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCSE: ACM Special Interest Group on Computer Science Education

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 February 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Best Paper

Author Tags

Qualifiers

Research-article

Funding Sources

NSF

Conference

SIGCSE '19

Sponsor:

SIGCSE

SIGCSE '19: The 50th ACM Technical Symposium on Computer Science Education

February 27 - March 2, 2019

MN, Minneapolis, USA

Acceptance Rates

SIGCSE '19 Paper Acceptance Rate 169 of 526 submissions, 32%;

Overall Acceptance Rate 1,595 of 4,542 submissions, 35%

Upcoming Conference

SIGCSE TS 2025

Sponsor:
sigcse

The 56th ACM Technical Symposium on Computer Science Education

February 26 - March 1, 2025

Pittsburgh , PA , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

18
Total Citations
View Citations
1,340
Total Downloads

Downloads (Last 12 months)145
Downloads (Last 6 weeks)10

Reflects downloads up to 25 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Rys-Recker NHao Q(2024)Early Identification of Struggling Students in Large Computer Science Courses: A Replication Study2024 IEEE 48th Annual Computers, Software, and Applications Conference (COMPSAC)10.1109/COMPSAC61105.2024.00022(88-93)Online publication date: 2-Jul-2024
https://doi.org/10.1109/COMPSAC61105.2024.00022
Poitras EDempsey DCrane BSimpson SSiegel A(2023)The Applications of Learning Analytics to Enhance Learning and Engagement in Introductory Programming InstructionPerspectives on Learning Analytics for Maximizing Student Outcomes10.4018/978-1-6684-9527-8.ch005(89-108)Online publication date: 24-Oct-2023
https://doi.org/10.4018/978-1-6684-9527-8.ch005
Bertolini RFinch SNehm R(2023)An application of Bayesian inference to examine student retention and attrition in the STEM classroomFrontiers in Education10.3389/feduc.2023.10738298Online publication date: 14-Feb-2023
https://doi.org/10.3389/feduc.2023.1073829
Quille KVidal-Meliá LNolan KMooney A(2023)Evolving Towards a Trustworthy AIEd Model to Predict at Risk Students in Introductory Programming CoursesProceedings of the 2023 Conference on Human Centered Artificial Intelligence: Education and Practice10.1145/3633083.3633190(22-28)Online publication date: 14-Dec-2023
https://dl.acm.org/doi/10.1145/3633083.3633190
Quille KNam Liao SCostelloe ENolan KMooney AShah KBecker BQuille KLaakso MBarendsen ESimon (2022)PreSSProceedings of the 27th ACM Conference on on Innovation and Technology in Computer Science Education Vol. 110.1145/3502718.3524755(54-60)Online publication date: 7-Jul-2022
https://dl.acm.org/doi/10.1145/3502718.3524755
Al Madi NPeng SRogers TMerkle LDoyle MSheard JSoh LDorn B(2022)Assessing Workload Perception in Introductory Computer Science Projects using NASA-TLXProceedings of the 53rd ACM Technical Symposium on Computer Science Education - Volume 110.1145/3478431.3499406(668-674)Online publication date: 22-Feb-2022
https://dl.acm.org/doi/10.1145/3478431.3499406
Hicks ECook AMalasri KZaman APhan VMerkle LDoyle MSheard JSoh LDorn B(2022)Keep It Relevant! Using In-class Exercises to Predict Weekly Performance in CS1Proceedings of the 53rd ACM Technical Symposium on Computer Science Education - Volume 110.1145/3478431.3499357(154-160)Online publication date: 22-Feb-2022
https://dl.acm.org/doi/10.1145/3478431.3499357
Arakawa KHao QDeneke WCowan IWolfman SPeterson AMerkle LDoyle MSheard JSoh LDorn B(2022)Early Identification of Student Struggles at the Topic Level Using Context-Agnostic FeaturesProceedings of the 53rd ACM Technical Symposium on Computer Science Education - Volume 110.1145/3478431.3499298(147-153)Online publication date: 22-Feb-2022
https://dl.acm.org/doi/10.1145/3478431.3499298
Gupta AGarg DKumar P(2022)Mining Sequential Learning Trajectories With Hidden Markov Models For Early Prediction of At-Risk Students in E-Learning EnvironmentsIEEE Transactions on Learning Technologies10.1109/TLT.2022.319748615:6(783-797)Online publication date: 1-Dec-2022
https://doi.org/10.1109/TLT.2022.3197486
Mehdi RNachouki M(2022)A neuro-fuzzy model for predicting and analyzing student graduation performance in computing programsEducation and Information Technologies10.1007/s10639-022-11205-228:3(2455-2484)Online publication date: 18-Aug-2022
https://doi.org/10.1007/s10639-022-11205-2
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents