research-article

Automated Feedback Framework for Introductory Programming Courses

Authors:

Jianxiong Gao,

Bei Pang,

Steven S. LumettaAuthors Info & Claims

ITiCSE '16: Proceedings of the 2016 ACM Conference on Innovation and Technology in Computer Science Education

Pages 53 - 58

https://doi.org/10.1145/2899415.2899440

Published: 11 July 2016 Publication History

Get Access

Abstract

Using automated grading tools to provide feedback to students is common in Computer Science education. The first step of automated grading is to find defects in the student program. However, finding bugs in code has never been easy. Comparing computation results using a fixed set of test cases is still the most common way to determine correctness among current automated grading tools. It takes time and effort to design a good set of test cases that can test the student code thoroughly. In practice, tests used for grading are often insufficient for accurate diagnosis.

In this paper, we present our utilization of industrial automated testing on student assignments in an introductory programming course. We implemented a framework to collect student codes and apply industrial automated testing to their codes. Then we interpreted the results obtained from testing in a way that students can understand easily. We deployed our framework on five different introductory C programming assignments here at the University of Illinois at Urbana-Champaign.The results show that the automated feedback generation framework can discover more errors inside student submissions and can provide timely and useful feedback to both instructors and students. A total of 142 missed bugs were found within 446 submissions. More than 50% of students received their feedback within 3 minutes of submission.

References

[1]

C. Cadar, D. Dunbar, and D. R. Engler. Klee: Unassisted and automatic generation of high-coverage tests for complex systems programs. In OSDI, volume 8, pages 209--224, 2008.

Digital Library

Google Scholar

[2]

J. G. Ganssle. A guide to code inspections, 2001.

Google Scholar

[3]

P. Godefroid, M. Y. Levin, and D. Molnar. Sage: whitebox fuzzing for security testing. Queue, 10(1):20, 2012.

Digital Library

Google Scholar

[4]

I. Huet, O. R. Pacheco, J. Tavares, and G. Weir. New challenges in teaching introductory programming courses: a case study. In Frontiers in Education, 2004. FIE 2004. 34th Annual, pages T2H-5. IEEE, 2004.

Crossref

Google Scholar

[5]

G. Jianxiong. Auto grading tool for introductory programming courses. Master';s thesis, University of Illinois, Champaign, 2015.

Google Scholar

[6]

C. F. Kemerer and M. C. Paulk. The impact of design and code reviews on software quality: An empirical study based on PSP data. Software Engineering, IEEE Transactions on, 35(4):534--550, 2009.

Digital Library

Google Scholar

[7]

K. Sen. Concolic testing. In Proceedings of the twenty-second IEEE/ACM International Conference on Automated Software Engineering, pages 571--572. ACM, 2007.

Digital Library

Google Scholar

[8]

K. Sen and G. Agha. Cute and jcute: Concolic unit testing and explicit path model-checking tools. In Computer Aided Verification, pages 419--423. Springer, 2006.

Digital Library

Google Scholar

[9]

A. Yadin. Reducing the dropout rate in an introductory programming course. ACM Inroads, 2(4):71--76, Dec. 2011.

Digital Library

Google Scholar

Cited By

View all

Hull MPednekar VMurray HRoy NTung ERoutray SGuerin CChen JWang ZLee SRoozbahani MChau D(2024)VG: Automatic Grading of D3 VisualizationsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.332718130:1(617-627)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TVCG.2023.3327181
Syaifudin YYapenrui DNoprianto Funabiki NSiradjuddin IChasanah H(2024)Implementation of Self-Learning Topic for Developing Interactive Mobile Application in Flutter Programming Learning Assistance System2024 ASU International Conference in Emerging Technologies for Sustainability and Intelligent Systems (ICETSIS)10.1109/ICETSIS61505.2024.10459432(1103-1107)Online publication date: 28-Jan-2024
https://doi.org/10.1109/ICETSIS61505.2024.10459432
Ali MRomli RKareem E(2024)Feedback Generation for Automatic Programming Assessment Utilizing AI Techniques: An Initial Analysis of Systematic Mapping StudiesAdvances in Intelligent Computing Techniques and Applications10.1007/978-3-031-59711-4_23(257-272)Online publication date: 30-Jun-2024
https://doi.org/10.1007/978-3-031-59711-4_23
Show More Cited By

Index Terms

Automated Feedback Framework for Introductory Programming Courses
1. Applied computing
  1. Education

Recommendations

Micro-scale Concolic Testing Framework for Automated Test Data Generation Based on Path Coverage
ICSE-Companion '24: Proceedings of the 2024 IEEE/ACM 46th International Conference on Software Engineering: Companion Proceedings

Automated test data generation based on path coverage (ATDG-PC) is an essential task in software testing. However, existing concolic testing approaches typically employ single static analyses or search-based algorithms for different paths without ...
Grey-box concolic testing on binary code
ICSE '19: Proceedings of the 41st International Conference on Software Engineering

We present grey-box concolic testing, a novel path-based test case generation method that combines the best of both white-box and grey-box fuzzing. At a high level, our technique systematically explores execution paths of a program under test as in ...
Automated test data generation and stubbing method for C/C++ embedded projects
Abstract
Automated test data generation for unit testing C/C++ functions using concolic testing has been known for improving software quality while reducing human testing effort. However, concolic testing could face challenging problems when tackling ...

Comments

Information & Contributors

Information

Published In

ITiCSE '16: Proceedings of the 2016 ACM Conference on Innovation and Technology in Computer Science Education

July 2016

394 pages

ISBN:9781450342315

DOI:10.1145/2899415

General Chairs:
Alison Clear
Eastern Institute of Technology, New Zealand
,
Ernesto Cuadros-Vargas
San Pablo Catholic University, Peru
,
Program Chairs:
Janet Carter
University of Kent, UK
,
Yvan Tupac
San Pablo Catholic University, Peru

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 July 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ITiCSE '16

Sponsor:

Bilkent University
SIGCSE

ITiCSE '16: Innovation and Technology in Computer Science Education Conference 2016

July 11 - 13, 2016

Arequipa, Peru

Acceptance Rates

ITiCSE '16 Paper Acceptance Rate 56 of 147 submissions, 38%;

Overall Acceptance Rate 552 of 1,613 submissions, 34%

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

21
Total Citations
View Citations
528
Total Downloads

Downloads (Last 12 months)47
Downloads (Last 6 weeks)4

Reflects downloads up to 10 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Hull MPednekar VMurray HRoy NTung ERoutray SGuerin CChen JWang ZLee SRoozbahani MChau D(2024)VG: Automatic Grading of D3 VisualizationsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.332718130:1(617-627)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TVCG.2023.3327181
Syaifudin YYapenrui DNoprianto Funabiki NSiradjuddin IChasanah H(2024)Implementation of Self-Learning Topic for Developing Interactive Mobile Application in Flutter Programming Learning Assistance System2024 ASU International Conference in Emerging Technologies for Sustainability and Intelligent Systems (ICETSIS)10.1109/ICETSIS61505.2024.10459432(1103-1107)Online publication date: 28-Jan-2024
https://doi.org/10.1109/ICETSIS61505.2024.10459432
Ali MRomli RKareem E(2024)Feedback Generation for Automatic Programming Assessment Utilizing AI Techniques: An Initial Analysis of Systematic Mapping StudiesAdvances in Intelligent Computing Techniques and Applications10.1007/978-3-031-59711-4_23(257-272)Online publication date: 30-Jun-2024
https://doi.org/10.1007/978-3-031-59711-4_23
Rocha HCosta ETedesco P(2023)Helping to provide adaptive feedback to novice programmers: a framework to assist the Teachers2023 18th Iberian Conference on Information Systems and Technologies (CISTI)10.23919/CISTI58278.2023.10212000(1-6)Online publication date: 20-Jun-2023
https://doi.org/10.23919/CISTI58278.2023.10212000
Joyse Barbosa Rocha Hde Barros Costa ECabral de Azevedo Restelli Tedesco P(2023)A Taxonomy to Assist TAs in Providing Adaptive Feedback to Novice Programmers2023 IEEE Frontiers in Education Conference (FIE)10.1109/FIE58773.2023.10343309(1-9)Online publication date: 18-Oct-2023
https://doi.org/10.1109/FIE58773.2023.10343309
Liu XCastellanos HWiese LMagana A(2023)Exploring Machine Learning Methods to Identify Patterns in Students' Solutions to Programming Assignments2023 IEEE Frontiers in Education Conference (FIE)10.1109/FIE58773.2023.10342972(1-6)Online publication date: 18-Oct-2023
https://doi.org/10.1109/FIE58773.2023.10342972
Skalka JDrlík M(2023)Development of Automatic Source Code Evaluation Tests Using Grey-Box Methods: A Programming Education Case StudyIEEE Access10.1109/ACCESS.2023.331769411(106772-106792)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3317694
Hurtig NHollingsworth JBlankenship SKraemer ESitaraman MHallstrom JBecker BQuille KLaakso MBarendsen ESimon (2022)Network Visualization and Assessment of Student Reasoning About ConditionalsProceedings of the 27th ACM Conference on on Innovation and Technology in Computer Science Education Vol. 110.1145/3502718.3524793(255-261)Online publication date: 7-Jul-2022
https://dl.acm.org/doi/10.1145/3502718.3524793
Hurtig NHollingsworth JScrivner O(2022)Visualization of Students’ Solutions as a Sequential Network2022 IEEE Global Engineering Education Conference (EDUCON)10.1109/EDUCON52537.2022.9766502(1189-1194)Online publication date: 28-Mar-2022
https://doi.org/10.1109/EDUCON52537.2022.9766502
Martin FShanley NHite NPugalee DPerez-Quinones MAhlgrim-Delzell LHart E(2022)Professional Development Strategies and Recommendations for High School Teachers to Teach Computer Science OnlineComputers in the Schools10.1080/07380569.2022.212734340:2(133-151)Online publication date: 12-Oct-2022
https://doi.org/10.1080/07380569.2022.2127343
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Micro-scale Concolic Testing Framework for Automated Test Data Generation Based on Path Coverage

Grey-box concolic testing on binary code

Automated test data generation and stubbing method for C/C++ embedded projects

Comments

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Other Metrics

Article Metrics

Other Metrics

Cited By

Login options

Full Access

PDF

eReader

Abstract

References

Cited By

Index Terms

Recommendations

Micro-scale Concolic Testing Framework for Automated Test Data Generation Based on Path Coverage

Grey-box concolic testing on binary code

Automated test data generation and stubbing method for C/C++ embedded projects

Comments

Information

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Get Access

Login options

Full Access

View options

PDF

eReader

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations