Using Rubrics Integrating Design and Coding to Assess Middle School Students' Open-ended Block-based Programming Projects

A corrigendum was issued for this paper on September 25, 2020. You can download the corrigendum from the supplemental material section of this citation page.

Free-choice, open-ended projects are commonly used to assess student learning in introductory block-based programming (BBP) environments. They are generally assessed in school based on criteria such as the social impact conveyed, whether the projects work without errors, and whether they are creative and engaging. Additionally, researchers have assessed such projects based on the frequency of use of various coding constructs like variables, conditionals, and iterations. This paper presents a novel multi-dimensional rubric for analyzing open-ended BBP projects that integrates assessment of front-end project design and back-end sophistication of use of coding constructs. Further, the novelty of the rubric lies in the fact that instead of relying solely on frequencies, it uses scaled scores based on sophistication of rubric components. Using this rubric, 160 Scratch and App Inventor projects were scored and analyzed. The paper establishes external validity of the rubric and examines what we can learn about student learning from this analysis. Our findings will help K-12 CS educators and curriculum developers recognize what aspects of CS middle school students need most support on, and how to leverage programming environments to provide this support.