Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3478431.3499382acmconferencesArticle/Chapter ViewAbstractPublication PagessigcseConference Proceedingsconference-collections
research-article

A Study of the Effects of Oral Proficiency Exams in Introductory Programming Courses on Underrepresented Groups

Published: 22 February 2022 Publication History

Abstract

Oral proficiency exams implemented in introductory programming courses at a public, research, minority-serving university provide a way to connect with students one-on-one, keep students accountable on the course material to improve their learning, and help improve feelings of belongingness in the learning environment. This study analyzes data disaggregated based on majority and underrepresented demographic groups by gender, race/ethnicity, and parental education level, to investigate the following three research themes across all groups: (1) to determine the impact of the oral exams on student motivation for independent learning; (2) to compare student stress levels related to the oral exams; and (3) to compare belongingness and final course performance. Student survey data was collected and analyzed following the oral exam intervention. For research theme (1), students across nearly all demographic groups reported that oral exams improved their understanding of the material and encouraged them to do more independent work. For research theme (2), students in some underrepresented groups reported higher stress levels than their majority group peers, and all groups reported a reduction in stress level after completing the oral exam. For research theme (3), students in some underrepresented groups reported lower sense of belonging and had lower final course performance than their majority group peers.

Supplementary Material

MP4 File (SIGCSE22V1fp490v.mp4)
In this video, details of the oral proficiency exam intervention are presented, including a discussion of the core proficiencies assessed and sample exam questions. The three research goals and study design are then introduced, with a brief breakdown of the survey instrument. Tables are presented that summarize the statistical analysis of the survey data with a discussion of the main results that address each of the research questions. The video concludes with recommendations for CS educators based on the findings in this study.

References

[1]
Eric M Anderman. 2007. The effects of personal, classroom, and school goal structures on academic cheating. In Psychology of academic cheating . Elsevier, 87--106.
[2]
Kesina Baral, Jeff Offutt, Paul Ammann, and Rasika Mohod. 2021. Practice Makes Better: Quiz Retake Software to Increase Student Learning. (2021).
[3]
Jo ao Paulo Barros, Lu'is Estevens, Rui Dias, Rui Pais, and Elisabete Soeiro. 2003. Using lab exams to ensure programming practice in an introductory programming course. ACM SIGCSE Bulletin, Vol. 35, 3 (2003), 16--20.
[4]
Shannon T Brady, Geoffrey L Cohen, Shoshana N Jarvis, and Gregory M Walton. 2020. A brief social-belonging intervention in college improves adult outcomes for black Americans. Science Advances, Vol. 6, 18 (2020), eaay3689.
[5]
Jeni L Burnette, Crystal L Hoyt, V Michelle Russell, Barry Lawson, Carol S Dweck, and Eli Finkel. 2020. A growth mind-set intervention improves interest but not academic performance in the field of computer science. Social Psychological and Personality Science, Vol. 11, 1 (2020), 107--116.
[6]
Jannick Demanet and Mieke Van Houtte. 2012. School belonging and school misconduct: The differing role of teacher and peer attachment. Journal of youth and adolescence, Vol. 41, 4 (2012), 499--514.
[7]
Robert Fraser et almbox. 2014. Collaboration, collusion and plagiarism in computer science coursework. Informatics in Education-An International Journal, Vol. 13, 2 (2014), 179--195.
[8]
James Garner, Paul Denny, and Andrew Luxton-Reilly. 2019. Mastery learning in computer science education. In Proceedings of the Twenty-First Australasian Computing Education Conference. 37--46.
[9]
Josephine A Gasiewski, M Kevin Eagan, Gina A Garcia, Sylvia Hurtado, and Mitchell J Chang. 2012. From gatekeeping to engagement: A multicontextual, mixed method study of student academic engagement in introductory STEM courses. Research in higher education, Vol. 53, 2 (2012), 229--261.
[10]
Hasmik Gharibyan. 2005. Assessing students' knowledge: oral exams vs. written tests. ACM SIGCSE Bulletin, Vol. 37, 3 (2005), 143--147.
[11]
Lisa C Kaczmarczyk, Alvaro Monge, Jeff Offutt, Healther Pon-Barry, and Suzanne Westbrook. 2015. You should (and absolutely can) keep diversity in sharp focus during the enrollment surge. In 2015 Research in Equity and Sustained Participation in Engineering, Computing, and Technology (RESPECT). IEEE, 1--3.
[12]
Kathleen J Lehman, Julia Rose Karpicz, Veronika Rozhenkova, Jamelia Harris, and Tomoko M Nakajima. 2021. Growing Enrollments Require Us to Do More: Perspectives on Broadening Participation During an Undergraduate Computing Enrollment Boom. In Proceedings of the 52nd ACM Technical Symposium on Computer Science Education . 809--815.
[13]
Karyn L Lewis, Jane G Stout, Noah D Finkelstein, Steven J Pollock, Akira Miyake, Geoff L Cohen, and Tiffany A Ito. 2017. Fitting in to move forward: Belonging, gender, and persistence in the physical sciences, technology, engineering, and mathematics (pSTEM). Psychology of Women Quarterly, Vol. 41, 4 (2017), 420--436.
[14]
Kevin Lin, Lina Battestilli, and Michael Ball. 2021. Strategies for Authentic Assessments of Mastery in CS Courses. In Proceedings of the 52nd ACM Technical Symposium on Computer Science Education . 1361--1361.
[15]
Catherine Mooney, Anna Antoniadi, Ioannis Karvelas, Lána Salmon, and Brett A Becker. 2020. Exploring Sense of Belonging in Computer Science Students. In Proceedings of the 2020 ACM Conference on Innovation and Technology in Computer Science Education . 563--563.
[16]
Catherine Mooney and Brett A Becker. 2020. Sense of Belonging: The Intersectionality of Self-Identified Minority Status and Gender in Undergraduate Computer Science Students. In United Kingdom & Ireland Computing Education Research conference. 24--30.
[17]
Peter Ohmann. 2019. An Assessment of Oral Exams in Introductory CS. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education. 613--619.
[18]
Claudia Ott, Brendan McCane, and Nick Meek. 2021. Mastery Learning in CS1-An Invitation to Procrastinate?: Reflecting on Six Years of Mastery Learning. In Proceedings of the 26th ACM Conference on Innovation and Technology in Computer Science Education V. 1 . 18--24.
[19]
Keith Quille and Susan Bergin. 2020. Promoting a Growth Mindset in CS1: Does One Size Fit All? A Pilot Study. In Proceedings of the 2020 ACM Conference on Innovation and Technology in Computer Science Education . 12--18.
[20]
Scott J. Reckinger and Shanon M. Reckinger. 2021. Oral Proficiency Exams in High-Enrollment Computer Science Courses. In 2021 ASEE Virtual Annual Conference Content Access. ASEE Conferences, Virtual Conference.
[21]
Gili Rusak and Lisa Yan. 2021. Unique Exams: Designing assessments for integrity and fairness. In Proceedings of the 52nd ACM Technical Symposium on Computer Science Education . 1170--1176.
[22]
Mihaela Sabin, Karen H Jin, and Adrienne Smith. 2021. Oral Exams in Shift to Remote Learning. In Proceedings of the 52nd ACM Technical Symposium on Computer Science Education. 666--672.
[23]
Mariana Silva, Matthew West, and Craig Zilles. 2020. Measuring the score advantage on asynchronous exams in an undergraduate CS course. In Proceedings of the 51st ACM Technical Symposium on Computer Science Education . 873--879.
[24]
Jane G Stout and Jennifer M Blaney. 2017. ?But it doesn?t come naturally?: how effort expenditure shapes the benefit of growth mindset on women's sense of intellectual belonging in computing. Computer Science Education, Vol. 27, 3--4 (2017), 215--228.
[25]
Gregory M Walton and Shannon T Brady. 2017. The many questions of belonging. (2017).
[26]
Gregory M Walton and Geoffrey L Cohen. 2007. A question of belonging: race, social fit, and achievement. Journal of personality and social psychology, Vol. 92, 1 (2007), 82.
[27]
Gregory M Walton and Geoffrey L Cohen. 2011. A brief social-belonging intervention improves academic and health outcomes of minority students. Science, Vol. 331, 6023 (2011), 1447--1451.
[28]
David S Yeager, Carissa Romero, Dave Paunesku, Christopher S Hulleman, Barbara Schneider, Cintia Hinojosa, Hae Yeon Lee, Joseph O'Brien, Kate Flint, Alice Roberts, et almbox. 2016. Using design thinking to improve psychological interventions: The case of the growth mindset during the transition to high school. Journal of educational psychology, Vol. 108, 3 (2016), 374.

Cited By

View all
  • (2024)Socratic Mind: Scalable Oral Assessment Powered By AIProceedings of the Eleventh ACM Conference on Learning @ Scale10.1145/3657604.3664661(340-345)Online publication date: 9-Jul-2024
  • (2024)Virtual pair programming and online oral exams: effects on social interaction, performance, and academic integrity in a remote computer programming courseComputer Science Education10.1080/08993408.2024.2344401(1-41)Online publication date: 5-May-2024

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
SIGCSE 2022: Proceedings of the 53rd ACM Technical Symposium on Computer Science Education - Volume 1
February 2022
1049 pages
ISBN:9781450390705
DOI:10.1145/3478431
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 February 2022

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. belongingness
  2. oral exam
  3. underrepresented groups

Qualifiers

  • Research-article

Conference

SIGCSE 2022
Sponsor:

Acceptance Rates

Overall Acceptance Rate 1,595 of 4,542 submissions, 35%

Upcoming Conference

SIGCSE TS 2025
The 56th ACM Technical Symposium on Computer Science Education
February 26 - March 1, 2025
Pittsburgh , PA , USA

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)40
  • Downloads (Last 6 weeks)4
Reflects downloads up to 02 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Socratic Mind: Scalable Oral Assessment Powered By AIProceedings of the Eleventh ACM Conference on Learning @ Scale10.1145/3657604.3664661(340-345)Online publication date: 9-Jul-2024
  • (2024)Virtual pair programming and online oral exams: effects on social interaction, performance, and academic integrity in a remote computer programming courseComputer Science Education10.1080/08993408.2024.2344401(1-41)Online publication date: 5-May-2024

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media