Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3491102.3501917acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article

AlgoSolve: Supporting Subgoal Learning in Algorithmic Problem-Solving with Learnersourced Microtasks

Published: 29 April 2022 Publication History

Abstract

Designing solution plans before writing code is critical for successful algorithmic problem-solving. Novices, however, often plan on-the-fly during implementation, resulting in unsuccessful problem-solving due to lack of mental organization of the solution. Research shows that subgoal learning helps learners develop more complete solution plans by enhancing their understanding of the high-level solution structure. However, expert-created materials such as subgoal labels are necessary to provide learning benefits from subgoal learning, which are a scarce resource in self-learning due to limited availability and high cost. We propose a learnersourcing workflow that collects high-quality subgoal labels from learners by helping them improve their label quality. We implemented the workflow into AlgoSolve, a prototype interface that supports subgoal learning for algorithmic problems. A between-subjects study with 63 problem-solving novices revealed that AlgoSolve helped learners create higher-quality labels and more complete solution plans, compared to a baseline method known to be effective in subgoal learning.

Supplementary Material

MP4 File (3491102.3501917-video-preview.mp4)
Video Preview
MP4 File (3491102.3501917-talk-video.mp4)
Talk Video

References

[1]
Owen L Astrachan. 2004. Non-competitive programming contest problems as the basis for just-in-time teaching. In 34th Annual Frontiers in Education, 2004. FIE 2004. IEEE, T3H–20.
[2]
Robert K Atkinson, Richard Catrambone, and Mary Margaret Merrill. 2003. Aiding Transfer in Statistics: Examining the Use of Conceptually Oriented Equations and Elaborations During Subgoal Learning.Journal of Educational Psychology 95, 4 (2003), 762.
[3]
Roland Backhouse. 2011. Algorithmic problem solving. John Wiley & Sons.
[4]
John B Biggs and Kevin F Collis. 2014. Evaluating the quality of learning: The SOLO taxonomy (Structure of the Observed Learning Outcome). Academic Press.
[5]
Brian Brost, Yevgeny Seldin, Ingemar J Cox, and Christina Lioma. 2016. Multi-dueling bandits and their application to online ranker evaluation. In Proceedings of the 25th ACM International on Conference on Information and Knowledge Management. 2161–2166.
[6]
Francisco Enrique Vicente Castro and Kathi Fisler. 2020. Qualitative Analyses of Movements Between Task-level and Code-level Thinking of Novice Programmers. In Proceedings of the 51st ACM Technical Symposium on Computer Science Education. 487–493.
[7]
Richard Catrambone. 1998. The subgoal learning model: Creating better examples so that students can solve novel problems.Journal of experimental psychology: General 127, 4 (1998), 355.
[8]
Richard Catrambone and Keith J Holyoak. 1990. Learning subgoals and methods for solving probability problems. Memory & Cognition 18, 6 (1990), 593–603.
[9]
Christian Charras and Thierry Lecroq. 2004. Handbook of exact string matching algorithms. King’s College Publications.
[10]
Michelene TH Chi. 2009. Active-constructive-interactive: A conceptual framework for differentiating learning activities. Topics in cognitive science 1, 1 (2009), 73–105.
[11]
Michael De Raadt, Richard Watson, and Mark Toleman. 2009. Teaching and assessing programming strategies explicitly. In Proceedings of the 11th Australasian Computing Education Conference (ACE 2009), Vol. 95. Australian Computer Society Inc., 45–54.
[12]
Adrienne Decker, Lauren E Margulieux, and Briana B Morrison. 2019. Using the SOLO taxonomy to understand subgoal labels effect in CS1. In Proceedings of the 2019 ACM Conference on International Computing Education Research. 209–217.
[13]
Paul Denny, John Hamer, Andrew Luxton-Reilly, and Helen Purchase. 2008. PeerWise. In Proceedings of the 8th International Conference on Computing Education Research. 109–112.
[14]
Paul Denny, John Hamer, Andrew Luxton-Reilly, and Helen Purchase. 2008. PeerWise: students sharing their multiple choice questions. In Proceedings of the fourth international workshop on computing education research. 51–58.
[15]
Shayan Doroudi, Ece Kamar, and Emma Brunskill. 2019. Not everyone writes good examples but good examples can come from anywhere. In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, Vol. 7. 12–21.
[16]
Elena L Glassman, Aaron Lin, Carrie J Cai, and Robert C Miller. 2016. Learnersourcing personalized hints. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing. 1626–1636.
[17]
Philip J Guo, Julia M Markel, and Xiong Zhang. 2020. Learnersourcing at Scale to Overcome Expert Blind Spots for Introductory Programming: A Three-Year Deployment Study on the Python Tutor Website. In Proceedings of the Seventh ACM Conference on Learning@ Scale. 301–304.
[18]
Mark Guzdial, Luke Hohmann, Michael Konneman, Christopher Walton, and Elliot Soloway. 1998. Supporting programming and learning-to-program with an integrated CAD and scaffolding workbench. Interactive Learning Environments 6, 1-2 (1998), 143–179.
[19]
Minjie Hu, Michael Winikoff, and Stephen Cranefield. 2013. A process for novice programming using goals and plans. In Proceedings of the Fifteenth Australasian Computing Education Conference-Volume 136. 3–12.
[20]
Robin Jeffries, Althea A Turner, Peter G Polson, and Michael E Atwood. 1981. The processes involved in designing software. Cognitive skills and their acquisition 255 (1981), 283.
[21]
Dayu Jiang and Slava Kalyuga. 2020. Confirmatory factor analysis of cognitive load ratings supports a two-factor model. Tutorials in Quantitative Methods for Psychology 16 (2020), 216–225.
[22]
Hyoungwook Jin, Minsuk Chang, and Juho Kim. 2019. SolveDeep: A System for Supporting Subgoal Learning in Online Math Problem Solving. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. 1–6.
[23]
Hassan Khosravi, Gianluca Demartini, Shazia Sadiq, and Dragan Gasevic. 2021. Charting the design and analytics agenda of learnersourcing systems. In LAK21: 11th International Learning Analytics and Knowledge Conference. 32–42.
[24]
Juho Kim. 2015. Learnersourcing: improving learning with collective learner activity. Ph.D. Dissertation. Massachusetts Institute of Technology.
[25]
Kenneth R Koedinger, John C Stamper, Elizabeth A McLaughlin, and Tristan Nixon. 2013. Using data-driven discovery of better student models to improve student learning. In International conference on artificial intelligence in education. Springer, 421–430.
[26]
Andy Kurnia, Andrew Lim, and Brenda Cheang. 2001. Online judge. Computers & Education 36, 4 (2001), 299–315.
[27]
H Chad Lane and Kurt VanLehn. 2005. Teaching the tacit knowledge of programming to novices with natural language tutoring. Computer Science Education 15, 3 (2005), 183–201.
[28]
Lauren E Margulieux and Richard Catrambone. 2019. Finding the best types of guidance for constructing self-explanations of subgoals in programming. Journal of the Learning Sciences 28, 1 (2019), 108–151.
[29]
Lauren E Margulieux, Richard Catrambone, and Laura M Schaeffer. 2018. Varying effects of subgoal labeled expository text in programming, chemistry, and statistics. Instructional Science 46, 5 (2018), 707–722.
[30]
Lauren E Margulieux, Mark Guzdial, and Richard Catrambone. 2012. Subgoal-labeled instructional material improves performance and transfer in learning to develop mobile applications. In Proceedings of the ninth annual international conference on International computing education research. 71–78.
[31]
Lauren E Margulieux, Briana B Morrison, Baker Franke, and Harivololona Ramilison. 2020. Effect of Implementing Subgoals in Code. org’s Intro to Programming Unit in Computer Science Principles. ACM Transactions on Computing Education (TOCE) 20, 4 (2020), 1–24.
[32]
Michael McCracken, Vicki Almstrum, Danny Diaz, Mark Guzdial, Dianne Hagan, Yifat Ben-David Kolikant, Cary Laxer, Lynda Thomas, Ian Utting, and Tadeusz Wilusz. 2001. A multi-national, multi-institutional study of assessment of programming skills of first-year CS students. In Working group reports from ITiCSE on Innovation and technology in computer science education. 125–180.
[33]
Piotr Mitros. 2015. Learnersourcing of complex assessments. In Proceedings of the Second (2015) ACM Conference on Learning@ Scale. 317–320.
[34]
Steven Moore, Huy Anh Nguyen, and John Stamper. 2021. Examining the Effects of Student Participation and Performance on the Quality of Learnersourcing Multiple-Choice Questions. In Proceedings of the Eighth ACM Conference on Learning@ Scale. 209–220.
[35]
Briana B Morrison, Brian Dorn, and Mark Guzdial. 2014. Measuring cognitive load in introductory CS: adaptation of an instrument. In Proceedings of the tenth annual conference on International computing education research. 131–138.
[36]
Briana B Morrison, Lauren E Margulieux, and Adrienne Decker. 2020. The curious case of loops. Computer Science Education 30, 2 (2020), 127–154.
[37]
Briana B Morrison, Lauren E Margulieux, and Mark Guzdial. 2015. Subgoals, context, and worked examples in learning computing problem solving. In Proceedings of the eleventh annual international conference on international computing education research. 21–29.
[38]
Eni Mustafaraj, Khonzoda Umarova, Franklyn Turbak, and Sohie Lee. 2018. Task-specific language modeling for selecting peer-written explanations. In The Thirty-First International Flairs Conference.
[39]
Ha Nguyen, June Ahn, William Young, and Fabio Campos. 2020. Where’s the Learning in Education Crowdsourcing?. In Proceedings of the Seventh ACM Conference on Learning@ Scale. 305–308.
[40]
David N Perkins, Chris Hancock, Renee Hobbs, Fay Martin, and Rebecca Simmons. 1986. Conditions of learning in novice programmers. Journal of Educational Computing Research 2, 1 (1986), 37–55.
[41]
Robert S Rist. 1989. Schema creation in programming. Cognitive Science 13, 3 (1989), 389–414.
[42]
Robert S Rist. 1991. Knowledge creation and retrieval in program design: A comparison of novice and intermediate student programmers. Human-Computer Interaction 6, 1 (1991), 1–46.
[43]
Anjali Singh, Christopher Brooks, Yiwen Lin, and Warren Li. 2021. What’s In It for the Learners? Evidence from a Randomized Field Experiment on Learnersourcing Questions in a MOOC. In Proceedings of the Eighth ACM Conference on Learning@ Scale. 221–233.
[44]
Yanan Sui, Vincent Zhuang, Joel W Burdick, and Yisong Yue. 2017. Multi-dueling Bandits with Dependent Arms. In Proceedings of the Thirty-Third Conference on Uncertainty in Artificial Intelligence.
[45]
Marie Tarrant and James Ware. 2010. A comparison of the psychometric properties of three-and four-option multiple-choice questions in nursing assessments. Nurse education today 30, 6 (2010), 539–543.
[46]
Xu Wang, Srinivasa Teja Talluri, Carolyn Rose, and Kenneth Koedinger. 2019. UpGrade: Sourcing student open-ended solutions to create scalable learning opportunities. In Proceedings of the Sixth (2019) ACM Conference on Learning@ Scale. 1–10.
[47]
Szymon Wasik, Maciej Antczak, Jan Badura, Artur Laskowski, and Tomasz Sternal. 2018. A survey on online judge systems and their applications. ACM Computing Surveys (CSUR) 51, 1 (2018), 1–34.
[48]
Sarah Weir, Juho Kim, Krzysztof Z Gajos, and Robert C Miller. 2015. Learnersourcing subgoal labels for how-to videos. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing. 405–416.
[49]
Jacob Whitehill and Margo Seltzer. 2017. A Crowdsourcing Approach to Collecting Tutorial Videos–Toward Personalized Learning-at-Scale. In Proceedings of the Fourth (2017) ACM Conference on Learning@ Scale. 157–160.
[50]
Joseph Jay Williams, Juho Kim, Anna Rafferty, Samuel Maldonado, Krzysztof Z Gajos, Walter S Lasecki, and Neil Heffernan. 2016. Axis: Generating explanations at scale with learnersourcing and machine learning. In Proceedings of the Third (2016) ACM Conference on Learning@ Scale. 379–388.
[51]
Joseph Jay Williams, Anna N Rafferty, Dustin Tingley, Andrew Ang, Walter S Lasecki, and Juho Kim. 2018. Enhancing online problems through instructor-centered tools for randomized experiments. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1–12.
[52]
Albina Zavgorodniaia, Rodrigo Duran, Arto Hellas, Otto Seppala, and Juha Sorva. 2020. Measuring the cognitive load of learning to program: A replication study. In United Kingdom & Ireland Computing Education Research conference.3–9.

Cited By

View all
  • (2024)GraspUI: Seamlessly Integrating Object-Centric Gestures within the Seven Phases of GraspingProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661551(1275-1289)Online publication date: 1-Jul-2024
  • (2024)CodeTree: A System for Learnersourcing Subgoal Hierarchies in Code ExamplesProceedings of the ACM on Human-Computer Interaction10.1145/36373088:CSCW1(1-37)Online publication date: 26-Apr-2024
  • (2024)Bridging Learnersourcing and AI: Exploring the Dynamics of Student-AI Collaborative Feedback GenerationProceedings of the 14th Learning Analytics and Knowledge Conference10.1145/3636555.3636853(742-748)Online publication date: 18-Mar-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
CHI '22: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems
April 2022
10459 pages
ISBN:9781450391573
DOI:10.1145/3491102
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 April 2022

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Algorithmic problem-solving
  2. Learnersourcing
  3. Subgoal learning

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

CHI '22
Sponsor:
CHI '22: CHI Conference on Human Factors in Computing Systems
April 29 - May 5, 2022
LA, New Orleans, USA

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI '25
CHI Conference on Human Factors in Computing Systems
April 26 - May 1, 2025
Yokohama , Japan

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)78
  • Downloads (Last 6 weeks)10
Reflects downloads up to 09 Nov 2024

Other Metrics

Citations

Cited By

View all
  • (2024)GraspUI: Seamlessly Integrating Object-Centric Gestures within the Seven Phases of GraspingProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661551(1275-1289)Online publication date: 1-Jul-2024
  • (2024)CodeTree: A System for Learnersourcing Subgoal Hierarchies in Code ExamplesProceedings of the ACM on Human-Computer Interaction10.1145/36373088:CSCW1(1-37)Online publication date: 26-Apr-2024
  • (2024)Bridging Learnersourcing and AI: Exploring the Dynamics of Student-AI Collaborative Feedback GenerationProceedings of the 14th Learning Analytics and Knowledge Conference10.1145/3636555.3636853(742-748)Online publication date: 18-Mar-2024
  • (2024)Unlocking Creator-AI Synergy: Challenges, Requirements, and Design Opportunities in AI-Powered Short-Form Video ProductionProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642476(1-23)Online publication date: 11-May-2024

View Options

Get Access

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media