research-article

Transformation-Based Diagnosis of Student Programs for Programming Tutoring Systems

Authors:

Yam San CheeAuthors Info & Claims

IEEE Transactions on Software Engineering, Volume 29, Issue 4

Pages 360 - 384

https://doi.org/10.1109/TSE.2003.1191799

Published: 01 April 2003 Publication History

Abstract

A robust technology that automates the diagnosis of students' programs is essential for programming tutoring systems. Such technology should be able to determine whether programs coded by a student are correct. If a student's program is incorrect, the system should be able to pinpoint errors in the program as well as explain and correct the errors. Due to the difficulty of this problem, no existing system performs this task entirely satisfactorily, and this problem still hampers the development of programming tutoring systems. This paper describes a transformation-based approach to automate the diagnosis of students' programs for programming tutoring systems. Improved control-flow analysis and data-flow analysis are used in program analysis. Automatic diagnosis of student programs is achieved by comparing the student program with a specimen program at the semantic level after both are standardized. The approach was implemented and tested on 525 real student programs for nine different programming tasks. Test results show that the method satisfies the requirements stated above. Compared to other existing approaches to automatic diagnosis of student programs, the approach developed here is more rigorous and safer in identifying student programming errors. It is also simpler to make use of in practice. Only specimen programs are needed for the diagnosis of student programs. The techniques of program standardization and program comparison developed here may also be useful for research in the fields of program understanding and software maintenance.

References

[1]

D. Allemang, “Using Functional Modes in Automatic Debugging,” IEEE Expert, vol. 6, no. 6, pp. 13-18, 1991.

Digital Library

[2]

J.R. Anderson R. Farrell and R. Sauers, “Learning to Program in Lisp,” Cognitive Science, vol. 8, pp. 87-129, 1984.

[3]

S.K. Abd-El-Hafiz and V.R. Basili, A Knowledge-Based Approach to Program Understanding. Kluwer Academic, 1995.

Digital Library

[4]

A. Adam and J. Laurent, “A System to Debug Student Programs,” Artificial Intelligence, vol. 15, no. 1, pp. 75-122, 1980.

Digital Library

[5]

M. Elsom-Cook and B. du Boulay, “A Pascal Program Checker,” Artificial Intelligence and Human Learning, J. Self, ed., pp. 361-373, 1988.

[6]

J. Ferrante K. Ottenstein and J. Warren, “The Program Dependence Graph and its Use in Optimization,” ACM Trans. Programming Languages, vol. 9, no. 3, pp. 319-349, 1987.

Digital Library

[7]

T.S. Gegg-Harrison, “Exploiting Program Schemata in an Automated Program Debugger,” J. Artificial Intelligence in Education, vol. 5, no. 2, pp. 255-278, 1994.

Digital Library

[8]

A. Goldberg and D. Robson, Smalltalk-80: The Language and its Implementation. Addison-Wesley, 1983.

Digital Library

[9]

J.S. Hahn and J. Kim, “Automatic Problem Description from Model Program for Knowledge-based Program Tutor,” Automatic Software Eng., vol. 4, pp. 439-461, 1997.

Digital Library

[10]

N. Hattori and N. Ishii, “A Method to Remove Variations in Source Codes,” Information and Software Technology, vol. 38, pp. 25-36, 1996.

[11]

S. Horwitz, “Identifying the Semantic and Textual Differences between Two Versions of a Program,” ACM SIGPLAN Notices, vol. 25,no. 6, pp. 234-245, 1990.

Digital Library

[12]

J.E. Hopcroft and J.D. Ullman, Introduction to Automata Theory, Languages, and Computation. Addison-Wesley, 1979.

Digital Library

[13]

D. Jackson and D.A. Ladd, “Semantic Diff: A Tool for Summarizing the Effects of Modifications,” Proc. Int'l Conf. Software Maintenance, pp. 243-252, 1994.

Digital Library

[14]

W.L. Johnson and E. Soloway, “Proust: Knowledge-Based Program Understanding,” IEEE Trans. Software Eng., vol. 11, no. 3, pp. 11-19, Nov. 1985.

Digital Library

[15]

W. Kozaczynski J. Ning and A. Engberts, “Program Concept Recognition and Transformation,” IEEE Trans. Software Eng., vol. 18, no. 12, pp. 1065-1075, Dec. 1992.

Digital Library

[16]

E. Lemut B. du Boulay and G. Dettori, Cognitive Models and Intelligent Environments for Learning Programming. Springer-Verlag, 1992.

Digital Library

[17]

C. Looi, “Automatic Debugging of Prolog Programs in a Prolog Intelligent Teaching System,” Instructional Science, vol. 20, pp. 215-263, 1991.

[18]

D. Loveman, “Program Improvement by Source to Source Transformation,” J. ACM, vol. 24, no. 1, pp. 121-145, 1977.

Digital Library

[19]

J.D. McGregor B.A. Malloy and R.L. Siegmund, “A Comprehensive Program Representation of Object-Oriented Software,” Ann. Software Eng., vol. 2, pp. 51-91, 1996.

[20]

S.S. Muchnick, Advanced Compiler Design and Implementation. Morgan Kaufmann, 1997.

Digital Library

[21]

W.R. Murray, Automatic Program Debugging for Intelligent Tutoring Systems, Morgan Kaufmann, 1988.

Digital Library

[22]

A. Pettorossi and M. Proietti, “Transformation of Logic Programs: Foundations and Techniques,” J. Logic Programming, vols. 19-20, pp. 261-320, 1994.

[23]

C. Rich and L.M. Wills, “Recognizing a Program's Design: A Graph-Parsing Approach,” IEEE Software, vol. 1, pp. 82-89, 1990.

Digital Library

[24]

E.Y. Shapiro, Algorithmic Program Debugging. MIT Press, 1983.

Digital Library

[25]

G. Thorburn and G. Rowe, “PASS: An Automated System for Program Assessment,” Computers and Education, vol. 29, no. 4, pp. 195-206, 1997.

Digital Library

[26]

H. Ueno, “Concepts and Methodologies for Knowledge-Based Program Understanding-the ALPUS's Approach,” IEICE Trans. Information and Systems, vol. E78-D, no. 9, pp. 1108-1117, 1995.

[27]

W. Yang S. Horwitz and T. Reps, “A Program Integration Algorithm that Accommodates Semantics-Preserving Transformations,” ACM Trans. Software Eng. and Methodology, vol. 1, no. 3, pp. 310-354, 1992.

Digital Library

Cited By

Wang F(2024)A Feasible Study of a Deep Learning Model Supporting Human–Machine Collaborative Learning of Object-Oriented ProgrammingIEEE Transactions on Learning Technologies10.1109/TLT.2022.322634517(413-427)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TLT.2022.3226345
Troussas CKrouska ASgouropoulou C(2022)Double-Layer Controller for Detecting Learners’ Erroneous Knowledge in Database ProgrammingIntelligent Tutoring Systems10.1007/978-3-031-09680-8_20(204-212)Online publication date: 29-Jun-2022
https://dl.acm.org/doi/10.1007/978-3-031-09680-8_20
Keuning HHeeren BJeuring JSherriff MMerkle LCutter PMonge ASheard J(2021)A Tutoring System to Learn Code RefactoringProceedings of the 52nd ACM Technical Symposium on Computer Science Education10.1145/3408877.3432526(562-568)Online publication date: 3-Mar-2021
https://dl.acm.org/doi/10.1145/3408877.3432526
Show More Cited By

Index Terms

Transformation-Based Diagnosis of Student Programs for Programming Tutoring Systems

Recommendations

Tracking pointers with path and context sensitivity for bug detection in C programs

This paper proposes a pointer alias analysis for automatic error detection. State-of-the-art pointer alias analyses are either too slow or too imprecise for finding errors in real-life programs. We propose a hybrid pointer analysis that tracks actively ...
Tracking pointers with path and context sensitivity for bug detection in C programs
ESEC/FSE-11: Proceedings of the 9th European software engineering conference held jointly with 11th ACM SIGSOFT international symposium on Foundations of software engineering

This paper proposes a pointer alias analysis for automatic error detection. State-of-the-art pointer alias analyses are either too slow or too imprecise for finding errors in real-life programs. We propose a hybrid pointer analysis that tracks actively ...
Automatic diagnosis of student programs in programming learning environments
IJCAI'99: Proceedings of the 16th international joint conference on Artificial intelligence - Volume 2

This paper describes a method to automate the diagnosis of students' programming errors in programming learning environments. In order to recognize correct students' programs as well as to identify errors in incorrect student programs, programs are ...

Comments

Information & Contributors

Information

Published In

cover image IEEE Transactions on Software Engineering

IEEE Transactions on Software Engineering Volume 29, Issue 4

April 2003

95 pages

ISSN:0098-5589

Issue’s Table of Contents

Copyright © Copyright © 2003 IEEE. All Rights Reserved.

Publisher

IEEE Press

Publication History

Published: 01 April 2003

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

27
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 15 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wang F(2024)A Feasible Study of a Deep Learning Model Supporting Human–Machine Collaborative Learning of Object-Oriented ProgrammingIEEE Transactions on Learning Technologies10.1109/TLT.2022.322634517(413-427)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TLT.2022.3226345
Troussas CKrouska ASgouropoulou C(2022)Double-Layer Controller for Detecting Learners’ Erroneous Knowledge in Database ProgrammingIntelligent Tutoring Systems10.1007/978-3-031-09680-8_20(204-212)Online publication date: 29-Jun-2022
https://dl.acm.org/doi/10.1007/978-3-031-09680-8_20
Keuning HHeeren BJeuring JSherriff MMerkle LCutter PMonge ASheard J(2021)A Tutoring System to Learn Code RefactoringProceedings of the 52nd ACM Technical Symposium on Computer Science Education10.1145/3408877.3432526(562-568)Online publication date: 3-Mar-2021
https://dl.acm.org/doi/10.1145/3408877.3432526
Keuning HHeeren BJeuring J(2020)Student Refactoring Behaviour in a Programming TutorProceedings of the 20th Koli Calling International Conference on Computing Education Research10.1145/3428029.3428043(1-10)Online publication date: 19-Nov-2020
https://dl.acm.org/doi/10.1145/3428029.3428043
Toll DWingkvist AEricsson M(2020)Current State and Next Steps on Automated Hints for Students Learning to Code2020 IEEE Frontiers in Education Conference (FIE)10.1109/FIE44824.2020.9274053(1-5)Online publication date: 21-Oct-2020
https://dl.acm.org/doi/10.1109/FIE44824.2020.9274053
Wang CZhang WZhao HJin Z(2020)Towards a fictional collective programming scenario: an approach based on the EIF loopEmpirical Software Engineering10.1007/s10664-020-09850-725:5(3671-3710)Online publication date: 1-Sep-2020
https://dl.acm.org/doi/10.1007/s10664-020-09850-7
Marin VRivero CZhu WTao DCheng XCui PRundensteiner ECarmel DHe QXu Yu J(2019)Clustering Recurrent and Semantically Cohesive Program Statements in Introductory Programming AssignmentsProceedings of the 28th ACM International Conference on Information and Knowledge Management10.1145/3357384.3357960(911-920)Online publication date: 3-Nov-2019
https://dl.acm.org/doi/10.1145/3357384.3357960
Perry DKim DSamanta RZhang XMcKinley KFisher K(2019)SemCluster: clustering of imperative programming assignments based on quantitative semantic featuresProceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation10.1145/3314221.3314629(860-873)Online publication date: 8-Jun-2019
https://dl.acm.org/doi/10.1145/3314221.3314629
Bian PLiang BZhang YYang CShi WCai Y(2019)Detecting Bugs by Discovering Expectations and Their ViolationsIEEE Transactions on Software Engineering10.1109/TSE.2018.281663945:10(984-1001)Online publication date: 1-Oct-2019
https://dl.acm.org/doi/10.1109/TSE.2018.2816639
Marin VRivero CBaysal OMenzies T(2018)Towards a framework for generating program dependence graphs from source codeProceedings of the 4th ACM SIGSOFT International Workshop on Software Analytics10.1145/3278142.3278144(30-36)Online publication date: 5-Nov-2018
https://dl.acm.org/doi/10.1145/3278142.3278144
Show More Cited By

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents