Evaluating students’ programming skill behaviour and

personalizing their computer learning environment using “The

Hour of Code” paradigm

Author Name *
Affiliation *
Address *

Author Name *
Affiliation *
Address *

ABSTRACT
One of the most intriguing objectives when teaching computer science in mid-adolescence high school students is
attracting and mainly maintaining their concentration within the limits of the class. A number of theories have been
proposed and numerous methodologies have been applied aiming to assist in the implementation of a personalized
learning environment. In this paper, an empirical study is performed with the support of the “Hour of Code” initiative
which took place for a second consecutive year at the end of previous year. The initiative was presented to a number of
high-school students as a motivation for teaching computer programming to them. An evaluation of the students’
programming skills is attempted with the aim of a questionnaire and a simple personalization framework is presented in
order to adapt to the students’ personal needs.

KEYWORDS
teaching programming, evaluation, Hour of Code, personalization framework, adaptation

1. INTRODUCTION
School as a learning organization [9] but primarily as a social environment [1] and the educator as carrier and
imparter of knowledge are daily facing many challenges in their attempt to shape the “school of the future”,
where knowledge is not only statically presented but transferred as part of a dynamically and evolving
procedure. Within such an environment computer science plays one of the most important roles. Computer
science depicts not only as a static and tiresome lesson, but also as a continuously effort to present and
transfer useful knowledge and moreover assist students in their quest for critical evaluation of the
information presented.
Teaching computer science to high school students clearly should be enlisted to Bruner’s pedagogical
paradigm for constructivist learning approach where students are encouraged to enrich what they learn from
the tutor with previous acquired knowledge from their past, in order to formulate new ideas or concepts [2].
Combining this educational procedure with Piaget’s model for cognitive learning where students are taught
something, think about it and then try to express it in their own words [8], lead to a hybrid teaching model for
computer science in classroom.
Moreover, computational thinking theory as introduced by S. Papert [7] and later presented by J. Wing
[11], advocate that algorithmic thinking is a fundamental skill which everyone can realize (and not only
computer scientists). In addition, J. Wing argued for the significance of bringing computational thinking in
other aspects of life.
Within this educational environment, bringing computer science and teaching programming lessons to
high school students not only encourage their way of understanding through computational thinking and
cognitive learning, but also promotes their personal skills and influence them in an alternative way of
acquiring and using the knowledge learned.
All of these aspects described above are widely recognized and were included and applied in an
innovative idea which took place worldwide during the 2nd week of December (8-14 Dec. 2014) which was
named “Hour of Code” [12]. The “Hour Of Code” initiative firstly introduced in the United States of
America, as an opportunity for mid-adolescence students to be taught “just one hour of code”. The campaign
was quickly spread all over the world and moreover promoted from numerous powerful and influential
people, including the president of the United States, founders of big IT companies and other individuals from
sports and arts. The success of the initiative was significant and students throughout the world were taught
and enjoyed an hour of code for a consecutive year.
In the remainder of this paper, the “Hour of Code” initiative is briefly described in section 2 and the
three-step teaching methodology which we applied in section 3. Furthermore a short evaluation of the
students understanding of simple programming concepts with the aim of an additional example is given in
section 4, along with a questionnaire filled by each student. Finally, we extend our previous work [6] by
proposing a simple framework which aims to help us personalize the computer learning environment for each
student and adapt our teaching methodology to their specific skills and needs.

2. THE HOUR OF CODE

2.1 Description of “The Hour of Code” Initiative

The most significant objective of the “Hour of Code” initiative as this is presented by the originators of the
idea is to bring closer all students to computer science and programming, through the presentation and
accomplishment of the “one hour of code” paradigm. The idea initially originated in the United States of
America from an non-profit organization (code.org) which then turned into a public charity.
Among the most fundamental goals of the initiative as those are presented by the founders are summarized in
the following:

 Introducing Computer Science classes to every high school student United States, particularly in
urban and rural neighborhoods.
 Changing official guidelines in all 50 states in order to classify Computer Science as part of the
math/science "core" curriculum
 Connect the computer science community in order to develop events for Computer Science
education worldwide
 Increase the representation and active participation of women and students in the technology field

The significant importance of the initiative and the successful results which were obtained, are
demonstrated by the numbers presented in the first page of their website. Namely, almost 5.2 billion lines of
code have been written till today and more than 100 million students have learned at least one hour of code.
The campaign of the initiative, as well as the programming examples which are presented, have been
endorsed by great personalities of all the important aspects of social life. Moreover, even the president of the
United States supported actively the project and encourages every student to actively participate and
complete at least an hour of code because “…learning these skills isn’t just important for your future, it’s
important for our country’s future..”[10]. All these factors demonstrate the noteworthy significance of the
initiative and exhibit many of the reasons which made the DG connect European Commission to organize a
similar event within the boundaries of the European Union during the 3 rd week of October 2014 with
excessive success.
3. TEACHING METHODOLOGY

3.1 METHODOLOGY OF TEACHING “Hour Of Code” IN THE

CLASSROOM
Within this context and the “Hour of Code” as the triggering event, we decided to actively participate in the
initiative and teach the very first tutorial of the webpage to 3 different classes of high-school students aged
14-17 years old. Most of the students had no previous experience in the field of computer programming (with
the exception of a few of them who having been learning computer algorithms as part of a course in their
curriculum) and mainly live in rural neighborhoods.
The approach of teaching the computer programming hour is briefly explained in the 3 steps which
follow and important characteristics are highlighted throughout this description.
3.1.1 Introduction – Explaining the importance
The initial teaching approach which was selected in order to evenly introduce the importance of computer
science (and more generally computers) to the students, was a 3-minute brief introduction on the recent
innovations on the computer science field (tablets, touch-screens, mobile devices) and the numerous
applications which nowadays all of us are daily using. In addition the students were encouraged to imagine
their parents’ life 30 years ago in the decade of 80’s when the presence and use of computers were
insignificant.

3.1.2 Career opportunities in the computer science field

Following the initial approach presented to the students for the context of computer science, another step
which introduced career opportunities in the CS field was presented. A brief discussion in the form of
questions-answers took place with the students showing a lot of curiosity for the working conditions and
mainly the annual salary of professions relating to computers. A very short demonstration was made with
pictures taken from the web which illustrated the new offices of Apple Inc., Google and Facebook. Students
were excited to learn the middle IT computer annual salaries and the benefits which arise when working for a
big IT company.
At this point a short video from the “Hour of Code” webpage was showed, where a number of
personalities analyzed the importance of computer science and programming. Students were impressed to
learn that a considerable number of the top-100 wealthiest people on earth are working in the IT industry.

3.1.3 Demonstrating the 20 steps example

The most important step of the teaching methodology followed was the demonstration to each class of the
first example of the webpage with 20 steps. This example was carefully chosen from the organizers of the
initiative to include the basic steps of programming, conditions (if-statements) and loops (repeat and repeat-
until statements).
Students enjoyed the demonstration (in a single PC with the image projected to a board) and the
main reasons for this success were the simplicity of the examples (cartoon characters were chosen for the 20
steps) and the built-in video explanations of each group of paradigms from well-recognized celebrities.
After the first few steps, each of the three classes was able to solve the additional steps roughly
effortless and most of the students found the solutions to each step relatively quickly. The very last steps of
the examples were more difficult and a few number of the students lost their attention throughout the whole
hour.
Each class was able to complete all the steps of the example within the given hour, enjoying the short
video-presentations of conditions and loops. Moreover, the idea of giving a certification for completing the
initiative from the founders excited the majority of the students, demanding a printed copy of it.

4. EVALUATION OF THE INITIATIVE FROM THE STUDENTS

In order to evaluate the student’s perceptive of the programming patterns presented and their basic
programming skills and potential programming talents, we performed a two phase evaluation approach.
Initially, another example (Lightbot) of the initiative was given for each student to complete alone (without
the educator’s assistance) during another teaching hour of the Computer Science course (one pc per student).
The purpose of this phase was for the educator to quietly evaluate the student’s progress in the puzzle, for
the proper usage of condition statements and loops. The example was more difficult than the one presented
and a significant amount of the students did not accomplish it (till the end). Mainly, younger students were
not able to successfully handle loop statements when puzzles converted into more difficult stages with
complex steps.
All these results and the overall progress of each student separately, were recorded in an independently
designed evaluation sheet.
For the second phase of the evaluation, a specifically designed questionnaire was given to each student (in
all 3 classes) with a few questions. The purpose of the questionnaire was for the student to evaluate the
initiative, record their experiences in programming and their interest in computer science. Among the
questions, one was specifically chosen to ask for their opinion of the initiative and give an estimate of their
level of understanding of the programming patterns presented (all of them, many, few and not-at-all). The
results obtained from this question are presented in Table I.

TABLE I. LEVEL OF UNDERSTANDING (ACCORDING TO STUDENTS OPINION)

Number of students (49)
Understanding of 3nd Percentage
each step 1st class 2nd class class
(age 15) (age 16) (age
17)
All 5 6 3 28,57%

Many of them 13 6 7 53,06%

Few of them 4 4 16,33%

None 1 2,04%

Another noteworthy fact which was inferred from the questionnaire given was the wide acceptance of
computer science among the students and their motivation for a future career in the IT field. Almost 3 out of
4 students envision a profession in the computer science field (according to their understanding of computer
science). Among them, there subsist a few which yearn for the hardware industry and few of them in the
computer programming field (Table II below).

TABLE II. POSSIBLE FUTURE CAREER IN THE IT INDUSTRY

Number of students (49)

Career in the IT
field 1st class 2nd class 3nd class Percentage
(age 15) (age 16) (age 17)
Yes 12 9 6 55,1%

No 11 3 8 44,9%

5. PERSONALIZING THE COURSE – ADAPTING TO EACH

INDIVIDUAL CLASSROOM REQUIREMENTS
The assessment performed and the results obtained from both the evaluation sheet and the questionnaire
given, motivated as to start designing a personalized system in order to improve the quality of teaching
programming to these students and adapt our methods to their specific needs.
Our goal was to personalize the learning environment for each individual and provide personalized
material and exercises for the course of computer studies in high school.
 The necessity for a personalized education environment has been pointed out in numerous cases both in
the past [4] as well as nowadays [3], [5]. The principal advantage for a personalized approach from a
student’s perceptive is the ability to create an adapted learning environment with personalized notes and
exercises which will guide the student to learn at its’ own pace. Moreover, there requisites a chief advantage
for the tutor, as he/she will be able to design, prepare and implement a number of educational packages
which slightly vary, fitting each student’s individual needs.
Therefore, our approach was to firstly design and in a later phase implement, a simple personalized
learning environment in order to provide our students an enhanced learning experience. The implementation
of our system is still in a premature phase; consequently we present here snippets of the initial design.
In the first phase of our design of this personalized teaching environment, a simple user profile was
created for each student. The profile comprised of basic characteristics for each student (name, age, gender)
and two other fields (scienceedu and understand) as shown in Figure 1 below. Both of these fields were given
with a decimal numeric score (0 to 1) which was obtained from the evaluation phase earlier described. The
sciencedu field illustrate the student’s tendency to math/science and the understand field our estimate
evaluation of the student’s understanding of programming.

Student’s User
Profile

First Name
ScienceEdu
Last Name
0…1
Age
Gender
Understand
Class
0…1

Figure 1. The design of a student’s profile (initially static)

In the next phase, we are planning to create personalized material and small tests for three categories of
computer programming. The material should be carefully chosen to illustrate specific components of
programming, for three categories (beginners, intermediate and advanced) and adapt to each student’s
requirements.
The main idea of this design is to support each student by assessing his/her skills and automatically
placing them in one of the three categories. This classification will be based on the evaluation results
obtained by the given questionnaire combined with the user profile’s numerical fields.
Finally, a simple automated system will be designed and implemented in order to generate and present
personalized material according to the age/gender of the student and ranking of the student. This material will
have appropriate questions and exercises in all three categories (basic, intermediate, advanced) and the ability
to be enriched manually. The proposed personalized system could initially perform a basic preliminary test
for each student. The results obtained could identify potential weakness and errors that a student has and
automatically generate the appropriate material which a student should study.
The user profile which initially is static could be a dynamically evolving one, by updating the two
numerical fields accordingly. A simple dynamic user profile could provide the system with a better estimate
of understanding for the programming course. A mean value of the two fields could be calculated and
appointed to the automated system in order to generate the appropriate material. Lower values of the mean
value are appointed from the system to present novice material to students, where bigger values could
generate advanced material and tests.
This simple framework is in the process of design and implementation and any further results obtained
should be tested in an experimental environment within the classroom.
6. CONCLUSIONS AND FUTURE WORK
Evaluating teaching performance and students’ learning behavior and skills are very challenging tasks which
require effort and time. The opportunity to teach and assess students’ programming skills was given with the
“Hour of Code” initiative. In this paper, a brief description of the initiative was given, along with the
teaching methodology which was followed.
The evaluation of the students’ learning behavior and the interest they expressed, motivated us with the
idea to adapt our methodology and teach computer programming according to their specific needs. Therefore,
a simple personalized framework is under design and implementation which aim to provide a personalized
learning environment for the computer programming education.
The work presented is an ongoing project with a number of open questions and challenging ideas. The
framework presented just finished the implementation phase and a number of ongoing questions acquired our
attention. The design is simple, but will provide us with enough material to teach for the remainder of the
semester.
Another framework with a different design is under consideration. User profiling is a dynamic concept
which is resupplied with new data. Teaching as a dynamic and evolving process requires material adapted to
each student and personalized tests dynamically created.
Therefore, a new framework will encapsulate all those concepts and a design of an adaptive and personalized
learning management system will assist instructors of algorithmic and programming courses.

REFERENCES
[1] Boocock, S. S. (1973). The school as a social environment for learning: Social organization and micro-social process
in education. Sociology of Education, 15-50.
[2] Bruner, J. (1996). The Culture of Education, Cambridge, MA: Harvard University Press.
[3] Fancsali, Stephen E., and Steven Ritter. "Context personalization, preferences, and performance in an intelligent
tutoring system for middle school mathematics." Proceedins of the Fourth International Conference on Learning
Analytics and Knowledge. ACM, 2014
[4] Fok A.W.P. and H.H.S. Ip, "Personalized education: An exploratory study of learning pedagogies in relation to
personalization technologies." Advances in Web-Based Learning–ICWL 2004. Springer Berlin Heidelberg, 2004,
407-415.
[5] Halstead-Nussloch Richard, Jon Preston, and Han Reichgelt. 2014. Improving student success through personalization
and customization. Proceedings of the 15th Annual Conference on Information technology education (SIGITE '14).
ACM, New York, NY, USA, 77-80.
[6]
[7] Papert Seymour, An Exploration in the Space of Mathematics Educations, International Journal of Computers for
Mathematical Learning,1(1):95–123, 1996
[8] Piaget J.,The Equilibrium of Cognitive Structures, The Central Problem in Cognitive Development 1985, University
of Chicago Press, Chicago, Illinois
[9] Silins H., Mulford B., (2002) "Schools as learning organisations: The case for system, teacher and student learning",
Journal of Educational Administration, Vol. 40 Iss: 5, pp.425 – 446
[10] Stephenson C., Bulding Student Interest in Computing, CSTA ACM Communications
[11] Wing Jeannette, Computational thinking, Communications ACM 49, 3 (March 2006), 33-35
[12] www.code.org, Last accessed 15 Feb 2015