research-article

The FCS1: a language independent assessment of CS1 knowledge

Authors:

Allison Elliott Tew and

Mark GuzdialAuthors Info & Claims

SIGCSE '11: Proceedings of the 42nd ACM technical symposium on Computer science education

March 2011

Pages 111 - 116

https://doi.org/10.1145/1953163.1953200

Published: 09 March 2011 Publication History

Get Access

Abstract

A primary goal of many CS education projects is to determine the extent to which a given intervention has had an impact on student learning. However, computing lacks valid assessments for pedagogical or research purposes. Without such valid assessments, it is difficult to accurately measure student learning or establish a relationship between the instructional setting and learning outcomes.

We developed the Foundational CS1 (FCS1) Assessment instrument, the first assessment instrument for introductory computer science concepts that is applicable across a variety of current pedagogies and programming languages. We applied methods from educational and psychological test development, adapting them as necessary to fit the disciplinary context. We conducted a large scale empirical study to demonstrate that pseudo-code was an appropriate mechanism for achieving programming language independence. Finally, we established the validity of the assessment using a multi-faceted argument, combining interview data, statistical analysis of results on the assessment, and CS1 exam scores.

References

[1]

American Educational Research Association, American Psychological Association, and National Council on Measurement in Education. Standards for educational and psychological testing. American Educational Research Association, Washington, DC, 1999.

Google Scholar

[2]

J. Cohen. Statistical Power Analysis for the Behavioral Sciences. Lawrence Erlbaum Associates, Hillsdale, NJ, 2nd edition, 1988.

Google Scholar

[3]

A. M. Decker. How Students Measure Up: An Assessment Instrument for Introductory Computer Science. PhD thesis, University at Bu ffalo (SUNY), Buff alo, NY, 2007.

Google Scholar

[4]

K. Goldman, P. Gross, C. Heeren, G. Herman, L. Kaczmarczyk, M. C. Loui, and C. Zilles. Identifying important and difficult concepts in introductory computing courses using a Delphi process. In SIGCSE '08: Proceedings of the 39th ACM Technical Symposium on Computer Science Education, pages 256--260, 2008.

Digital Library

Google Scholar

[5]

R. K. Hambleton, H. Swaminathan, and H. J. Rogers. Fundamentals of item response theory. Sage Publications, Newbury Park, CA, 1991.

Google Scholar

[6]

D. Hestenes, M. Wells, and G. Swackhamer. Force concept inventory. The Physics Teacher, 30:141--158, March 1992.

Crossref

Google Scholar

[7]

L. C. Kaczmarczyk, E. R. Petrick, J. P. East, and G. L. Herman. Identifying student misconceptions of programming. In SIGCSE '10: Proceedings of the 41st ACM Technical Symposium on Computer Science Education, 2010.

Digital Library

Google Scholar

[8]

M. T. Kane. Validation. In R. L. Brennen, editor, Educational Measurement, pages 17--64. American Council on Education/Praeger Publishers, Westport, CT, 4th edition, 2006.

Google Scholar

[9]

J. C. Libarkin and S. Anderson. Assessment of learning in entry-level geoscience courses: Results from the geoscience concept inventory. Journal of Geoscience Education, 53:394--401, 2005.

Crossref

Google Scholar

[10]

M. D. Miller, R. L. Linn, and N. E. Gronlund. Validity. In Measurement and assessment in teaching, pages 80--99. Pearson Education, Upper Saddle River, NJ, 10th edition, 2009.

Google Scholar

[11]

P. A. Moss, B. J. Girard, and L. C. Haniford. Validity in Educational Assessment. Review of Research in Education, 30(1):109--162, 2006.

Google Scholar

[12]

A. E. Tew. Assessing fundamental introductory computing concept knowledge in a language independent manner. PhD thesis, Georgia Institute of Technology, Atlanta, GA, 2010.

Digital Library

Google Scholar

[13]

A. E. Tew and M. Guzdial. Developing a validated assessment of fundamental CS1 concepts. In SIGCSE '10: Proceedings of the 41st ACM Technical Symposium on Computer Science education, 2010.

Digital Library

Google Scholar

Cited By

View all

Lu KKrishnamurthi S(2024)Identifying and Correcting Programming Language Behavior MisconceptionsProceedings of the ACM on Programming Languages10.1145/36498238:OOPSLA1(334-361)Online publication date: 29-Apr-2024
https://dl.acm.org/doi/10.1145/3649823
Nitta CEiselt KStephenson BStone JBattestilli LRebelsky SShoop L(2024)The First Five Years of a Dual Track Programming SeriesProceedings of the 55th ACM Technical Symposium on Computer Science Education V. 110.1145/3626252.3630947(965-971)Online publication date: 7-Mar-2024
https://dl.acm.org/doi/10.1145/3626252.3630947

Index Terms

The FCS1: a language independent assessment of CS1 knowledge
1. Social and professional topics
  1. Professional topics
    1. Computing education
      1. Computing education programs
        Computer science education
      2. Model curricula

Recommendations

Uses, Revisions, and the Future of Validated Assessments in Computing Education: A Case Study of the FCS1 and SCS1
ICER 2021: Proceedings of the 17th ACM Conference on International Computing Education Research

Validated instruments of knowledge are important for creating an accepted and shared measurement for the most important part of education – student learning. The Force Concept of Inventory from Physics Education Research has had a significant impact ...
Read More
Replication, Validation, and Use of a Language Independent CS1 Knowledge Assessment
ICER '16: Proceedings of the 2016 ACM Conference on International Computing Education Research

Computing education lags other discipline-based education research in the number and range of validated assessments available to the research community. Validated assessments are important for researchers to reduce experimental error due to flawed ...
Read More
Developing a validated assessment of fundamental CS1 concepts
SIGCSE '10: Proceedings of the 41st ACM technical symposium on Computer science education

Previous studies of student programming ability have raised questions about students' ability to problem solve, read and analyze code, and understand introductory computing concepts. However, it is unclear whether these results are the product of ...
Read More

Comments

Information & Contributors

Information

Published In

SIGCSE '11: Proceedings of the 42nd ACM technical symposium on Computer science education

March 2011

754 pages

ISBN:9781450305006

DOI:10.1145/1953163

General Chairs:
Thomas J. Cortina
Carnegie Mellon University
,
Ellen L. Walker
Hiram College
,
Program Chairs:
Laurie Smith King
College of the Holy Cross
,
David R. Musicant
Carleton College

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 March 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SIGCSE '11

Sponsor:

SIGCSE

SIGCSE '11: The 42nd ACM Technical Symposium on Computer Science Education

March 9 - 12, 2011

TX, Dallas, USA

Acceptance Rates

SIGCSE '11 Paper Acceptance Rate 107 of 315 submissions, 34%;

Overall Acceptance Rate 1,595 of 4,542 submissions, 35%

Upcoming Conference

SIGCSE Virtual 2024

Sponsor:
sigcse

SIGCSE Virtual 2024: ACM Virtual Global Computing Education Conference

December 5 - 7, 2024

Virtual Event , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
1,540
Total Downloads

Downloads (Last 12 months)110
Downloads (Last 6 weeks)5

Other Metrics

View Author Metrics

Citations

Cited By

View all

Lu KKrishnamurthi S(2024)Identifying and Correcting Programming Language Behavior MisconceptionsProceedings of the ACM on Programming Languages10.1145/36498238:OOPSLA1(334-361)Online publication date: 29-Apr-2024
https://dl.acm.org/doi/10.1145/3649823
Nitta CEiselt KStephenson BStone JBattestilli LRebelsky SShoop L(2024)The First Five Years of a Dual Track Programming SeriesProceedings of the 55th ACM Technical Symposium on Computer Science Education V. 110.1145/3626252.3630947(965-971)Online publication date: 7-Mar-2024
https://dl.acm.org/doi/10.1145/3626252.3630947

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Uses, Revisions, and the Future of Validated Assessments in Computing Education: A Case Study of the FCS1 and SCS1

Replication, Validation, and Use of a Language Independent CS1 Knowledge Assessment

Developing a validated assessment of fundamental CS1 concepts