Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/2978249.2978252acmotherconferencesArticle/Chapter ViewAbstractPublication PageswipsceConference Proceedingsconference-collections
research-article

Introducing Student Assessments with Evidence of Validity for NYC's CS4All

Published: 13 October 2016 Publication History

Abstract

There are few assessments with strong evidence of validity for collecting data from students studying computer science (CS) education. Many of the assessments focus solely on students' attitudes and beliefs, or on curriculum. With the launch of NYC's CS4All initiative, there is a need for assessments that can be used at scale to measure CS outcomes across curricula and programs. This paper presents scales drawn from Panorama Education for collecting data from students, and the methodology used in creating and modifying those scales. The selected scales were modified, without undermining their evidence of validity, to measure students' perceptions of the pedagogical effectiveness and expectations and rigor of a CS teacher; and students' perceptions of their own interest in CS and engagement in their CS class. Following a discussion of the surveys and their modifications, data from an initial administration of the surveys are presented as a baseline for the community. Overall response data as well as cross-scale correlations are included. The paper concludes with a discussion of findings and how the surveys will be used in the future in NYC.

References

[1]
L. A. Clark and D. Watson. Constructing validity: Basic issues in objective scale development. Psychological assessment, 7(3):309, 1995.
[2]
L. J. Cronbach and P. E. Meehl. Construct validity in psychological tests. Psychological bulletin, 52(4):281, 1955.
[3]
L. A. DeLyser and M. Preston. A public school model of cs education. In Proceedings of the International Conference on Frontiers in Education: Computer-Science and Computer Engineering (FECS), page 233. The Steering Committee of The World Congress in Computer Science, Computer Engineering and Applied Computing (WorldComp), 2015.
[4]
D. A. Dillman, J. D. Smyth, and L. M. Christian. Internet, phone, mail, and mixed-mode surveys: the tailored design method. John Wiley & Sons, 2014.
[5]
A. Elliott Tew, B. Dorn, and O. Schneider. Toward a validated computing attitudes survey. In Proceedings of the ninth annual international conference on International computing education research, pages 135--142. ACM, 2012.
[6]
H. Gehlbach and M. E. Brinkworth. Measure twice, cut down error: A process for enhancing the validity of survey scales. Review of General Psychology, 15(4):380, 2011.
[7]
S. Grover, M. Bienkowski, and E. Snow. Assessments for computational thinking in k-12. In Proceedings of the 46th ACM Technical Symposium on Computer Science Education, pages 700--700. ACM, 2015.
[8]
S. Grover and R. Pea. Computational thinking in k--12 a review of the state of the field. Educational Researcher, 42(1):38--43, 2013.
[9]
S. Grover, R. Pea, and S. Cooper. Designing for deeper learning in a blended computer science course for middle school students. Computer Science Education, 25(2):199--237, 2015.
[10]
M. Kane. Current concerns in validity theory. Journal of Educational Measurement, 38(4):319--342, 2001.
[11]
S. Messick. Validity of psychological assessment: validation of inferences from persons' responses and performances as scientific inquiry into score meaning. American psychologist, 50(9):741, 1995.
[12]
Panorama Education. Validity brief: Panorama student survey. Technical report, Panorama Education, 2015.
[13]
J. W. Pellegrino, M. L. Hilton, et al. Education for life and work: Developing transferable knowledge and skills in the 21st century. National Academies Press, 2013.
[14]
B. Schueler, J. McIntyre, W. Suh, B. Mascio, and H. Gehlbach. Measuring student perceptions of teacher pedagogical effectiveness: The development of a new student survey tool. Association for Public Policy Analysis and Management, 2015.
[15]
C. Schulte and M. Knobelsdorf. Attitudes towards computer science-computing experiences as a starting point and barrier to computer science. In Proceedings of the third international workshop on Computing education research, pages 27--38. ACM, 2007.
[16]
D. Seehorn, S. Carey, B. Fuschetto, I. Lee, D. Moix, D. O'Grady-Cunniff, B. B. Owens, C. Stephenson, and A. Verno. Csta k-12 computer science standards: Revised 2011. 2011.
[17]
M. Smith. Computer science for all. Website, January 2016.
[18]
A. E. Tew and M. Guzdial. The fcs1: a language independent assessment of cs1 knowledge. In Proceedings of the 42nd ACM technical symposium on Computer science education, pages 111--116. ACM, 2011.
[19]
A. Yadav, D. Burkhart, D. Moix, E. Snow, P. Bandaru, and L. Clayborn. Sowing the seeds: A landscape study on assessment in secondary computer science education. Comp. Sci. Teachers Assn., NY, NY, 2015.

Cited By

View all
  • (2025)Evaluating Students' Experience in High School CS Education: A CAPE Framework-Based ApproachProceedings of the 56th ACM Technical Symposium on Computer Science Education V. 210.1145/3641555.3705176(1559-1560)Online publication date: 18-Feb-2025
  • (2024)Teaching Computing in Indigenous Schools: An Early Experience ReportProceedings of the 2024 on RESPECT Annual Conference10.1145/3653666.3656104(201-205)Online publication date: 16-May-2024
  • (2022)Capacity-related factors associated with computer science access and participation in Georgia public high schoolsPolicy Futures in Education10.1177/14782103221081920Online publication date: 14-Apr-2022
  • Show More Cited By
  1. Introducing Student Assessments with Evidence of Validity for NYC's CS4All

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    WiPSCE '16: Proceedings of the 11th Workshop in Primary and Secondary Computing Education
    October 2016
    124 pages
    ISBN:9781450342230
    DOI:10.1145/2978249
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    In-Cooperation

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 13 October 2016

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Assessment
    2. CS4All
    3. K12 CS Education

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Conference

    WiPSCE '16

    Acceptance Rates

    WiPSCE '16 Paper Acceptance Rate 10 of 58 submissions, 17%;
    Overall Acceptance Rate 104 of 279 submissions, 37%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)11
    • Downloads (Last 6 weeks)1
    Reflects downloads up to 20 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2025)Evaluating Students' Experience in High School CS Education: A CAPE Framework-Based ApproachProceedings of the 56th ACM Technical Symposium on Computer Science Education V. 210.1145/3641555.3705176(1559-1560)Online publication date: 18-Feb-2025
    • (2024)Teaching Computing in Indigenous Schools: An Early Experience ReportProceedings of the 2024 on RESPECT Annual Conference10.1145/3653666.3656104(201-205)Online publication date: 16-May-2024
    • (2022)Capacity-related factors associated with computer science access and participation in Georgia public high schoolsPolicy Futures in Education10.1177/14782103221081920Online publication date: 14-Apr-2022
    • (2022)A Decade of Demographics in Computing Education Research: A Critical Review of Trends in Collection, Reporting, and UseProceedings of the 2022 ACM Conference on International Computing Education Research - Volume 110.1145/3501385.3543967(323-343)Online publication date: 3-Aug-2022
    • (2021)The Potential of CR-SE for K-12 Computer Science Education: Perspectives from Two LeadersVUE (Voices in Urban Education)10.33682/3en3-cbgn50:1Online publication date: 2021
    • (2020)Evaluation and Assessment Needs of Computing Education in Primary GradesProceedings of the 2020 ACM Conference on Innovation and Technology in Computer Science Education10.1145/3341525.3387371(124-130)Online publication date: 15-Jun-2020
    • (2020)Self-directed learners’ perceptions and experiences of learning computer science through MIT open coursewareOpen Learning: The Journal of Open, Distance and e-Learning10.1080/02680513.2020.178160637:4(370-385)Online publication date: 18-Jun-2020
    • (2019)Physical Programming for Blind and Low Vision Children at ScaleHuman–Computer Interaction10.1080/07370024.2019.162117536:5-6(535-569)Online publication date: 3-Jul-2019
    • (2017)Interested In Class, But Not In The HallwayProceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education10.1145/3017680.3017722(243-248)Online publication date: 8-Mar-2017

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media