QUT Digital Repository:
http://eprints.qut.edu.au/
Corney, Malcolm Walter and Teague, Donna M. and Thomas, Richard N. (2010)
Engaging Students in Programming. In: 12th Australasian Computing Education
Conference (ACE 2010), 18-21 January 2010, Brisbane. (In Press)
© Copyright 2010 Australian Computer Society, Inc.
Engaging Students in Programming
Malcolm Corney, Donna Teague and Richard N. Thomas
Faculty of Science and Technology
Queensland University of Technology
Brisbane, Australia
[m.corney, d.teague, r.thomas]@qut.edu.au
Abstract
Poor student engagement and high failure rates in first
year units were addressed at the Queensland University of
Technology (QUT) with a course restructure involving a
fresh approach to introducing programming. Students‟
first taste of programming in the new course focused less
on the language and syntax, and more on problem solving
and design, and the role of programming in relation to
other technologies they are likely to encounter in their
studies.
In effect, several technologies that have
historically been compartmentalised and taught in
isolation have been brought together as a breadth-first
introduction to IT.
Incorporating databases and Web development
technologies into what used to be a purely programming
unit gave students a very short introduction to each
technology, with programming acting as the glue between
each of them. As a result, students not only had a clearer
understanding of the application of programming in the
real world, but were able to determine their preference or
otherwise for each of the technologies introduced, which
will help them when the time comes for choosing a
course major.
Students engaged well in an intensely collaborative
learning environment for this unit which was designed to
both support the needs of students and meet industry
expectations. Attrition from the unit was low, with
computer laboratory practical attendance rates for the first
time remaining high throughout semester, and the failure
rate falling to a single figure percentage.
Keywords: introductory programming, IT course, student
engagement, attrition
1
Introduction
Attrition from programming courses is historically high
(Berenson, Slaten et al. 2004; Kinnunen and Malmi 2006;
Biggers, Brauer et al. 2008), particularly in minority
groups for whom there is often poor representation to
begin with (Cohoon 2002; Fisher and Margolis 2002;
Lewis, McKay et al. 2006; Varma 2006; Vilner and Zur
2006).
Introductory programming units in particular have had
an alarming failure rate (Sheard and Hagan 1998; Robins,
Rountree et al. 2003). More than 30% of QUT students
Copyright © 2010, Australian Computer Society, Inc. This
paper appeared at the Twelfth Australasian Computing
Education Conference (ACE2010), Brisbane, Australia, January
2010. Conferences in Research and Practice in Information
Technology, Vol. 103. Tony Clear and John Hamer, Eds.
Reproduction for academic, not-for-profit purposes permitted
provided this text is included.
on average had failed QUT‟s introductory programming
subject since 2003, and for some semesters that
percentage was significantly higher (Teague and Roe
2009).
Much research has focused on this dilemma
culminating in a range of cognitive theories for high
failure rates including:
the difficulty of understanding the purpose of
programs and their relationship with the computer;
difficulty in grasping the syntax and semantics of a
particular programming language (Robins, Rountree
et al. 2003);
misconceptions of programming constructs (Soloway
and Spohrer 1989);
inability to problem-solve (McCracken, Almstrum et
al. 2001); and
inability to read and understand program code
(Lister, Adams et al. 2004; Mannila 2006).
The idea that the failure lies in the ability of some
students to grasp the course content has seen repeated
redevelopment of introductory programming courses with
changes of language, paradigm, and swapping between
breadth and depth of content approaches.
Motivation, however, has been found to be one of the
major reasons for students dropping out of IT courses
(Kinnunen and Malmi 2006). QUT‟s data shows that
there is a correlation between students who fail an introductory programming unit and those who withdraw from
their degree. This has been informally confirmed by
many other institutions. The implicit assumption is that
lack of motivation leads to failure or that poor performance in early assessment tasks leads to poor motivation.
To address student motivation by making learning
more fun, courses and tools have been developed to help
captivate introductory programming students (Lister
2004; Parsons and Haden 2006; Pollard and Duvall 2006;
Davis and Rebelsky 2007; Feinberg 2007).
The „big picture‟ for many students is that
programming is perceived as a solitary occupation, and
one which is conducted in a competitive environment.
This is unwittingly reinforced at university where
faculties demand that programming students be
individually assessed. The generally accepted stereotype
of a programmer is the „geeky‟ young male with dubious
social skills, and it is not surprising that many students
are alienated by this image. This may often negatively
affect their confidence and lead to a subsequent lack of
engagement and interest (Fisher and Margolis 2002).
Collaborative learning establishes an environment
conducive to learning and addresses some of the social
and cultural barriers facing students (Wilson, Hoskin et
al. 1993; Williams and Kessler 2000; McDowell, Werner
et al. 2002; Gehringer, Deibel et al. 2006). It has been
found that students benefit from the peer support while
learning, and at the same time are motivated by peer
pressure and a sense of purpose and belonging
(McKinney and Denton 2006).
Taking the collaborative approach by using pair
programming in the learning environment has been
documented as having significant educational benefits
including active learning and improved retention,
program quality, and confidence in the solution
(McDowell, Werner et al. 2002; Nagappan, Williams et
al. 2003; McDowell, Werner et al. 2006; Mendes, AlFakhri et al. 2006). A more social rather than competitive
environment is established with pair programming which
promotes more interaction and lends twice as much brain
power and an extra set of eyes to a programming exercise
(Simon and Hanks 2007).
In this paper we report on our approach to redesigning
the introductory programming unit at QUT. Our aim was
to focus on the issues of motivation and engagement, as
well as the social and cultural attitudes affecting the way
students engage in their learning to program. We
provided students with an intensely collaborative learning
environment and were better able to engage students in
programming by providing them with a taste of a number
of technologies, each of which relies on programming.
The redesign of this introductory unit came about
within a new overall course structure for the Bachelor of
IT. By providing students with introductions to various
technologies, it was hoped that students with a preference
for a particular area of study could get a brief taste of that
area before embarking on a more serious investment of
time in a particular unit.
The remainder of the paper is structured as follows. In
Section 2, we discuss previous approaches taken in
teaching introductory programming at QUT and the
perceived problems with those approaches. Section 3
presents the approach we have developed for the new
introductory programming unit, describing the unit‟s
structure and how it fits into the new course structure, and
the unit‟s approach to teaching and assessment. An
analysis of the student cohort enrolled in the unit is also
provided. The evidence of our success in improving the
unit is presented in Section 4. Finally we present our
future plans for the unit and our conclusions in Sections 5
and 6.
2
2.1
Historical Units
Introductory Programming Units
The teaching of programming at QUT has undergone an
evolution over the past twenty years as undergraduate
degree course structures were reviewed and redeveloped.
Introductory programming has always been seen as a
requirement for all students enrolled in our IT course by
industry advisers.
QUT has recently offered an
undergraduate degree program in Games and Interactive
Entertainment (GIE) and advisers in this field similarly
agree that programming is necessary.
Over the past eight years many changes were made to
our introductory programming units as the number of
students entering IT degree courses first swelled and then
dwindled for many reasons – the rise and fall of the tech
bubble, post Y2K, programming and programmers being
seen as an outsourced commodity etc.
Rising numbers of students lead to problems in the
quality of teaching and learning inherent with large class
sizes and the overall student experience suffered
accordingly.
A subsequent and dramatic decline in the number of
students entering the course resulted in fewer students
with high university entrance scores which affected the
nature of the student cohort. Methods of teaching which
had previously been seen as successful were no longer
working as well as they had.
Student cohorts typically contained a mixture of
students who could already program to some extent and
those who had never programmed before. As the
introductory units were aimed at the lowest common
denominator, students with programming skills typically
saw the units as too simple and failed to engage with the
content. Other students simply did not cope with learning
to program for any number of reasons, and then failed to
engage as they fell behind in their learning.
Assessment in these units typically consisted of one or
two programming assignments of varying levels of
difficulty and an end of semester exam which was worth
70% of their final grade for the unit. Assignments were
often quite large in the context of novice programming,
were often poorly attempted and were rarely completed
successfully. While formative feedback was provided to
students for assignments, it was typically not useful to the
students in examinations.
Recent offerings of first programming units attempted
to focus on problem solving but this was done with an
emphasis on producing programs, rather than through
categorization of problems into types that can be solved
with different algorithmic approaches.
2.2
Approaches Taken
In previous course designs a number of approaches have
been taken in the teaching of introductory programming
units at QUT. Imperative programming was used initially
with languages such as Pascal and Modula 2. Attempts
were made to introduce object oriented programming
with Java and C#, though this was mainly taught with an
introduction to imperative programming before objects
were introduced. A purely functional approach was also
trialled using Scheme as the language du jour.
2.3
The Impact of Course Structure
The design of many information technology courses has
compartmentalized the teaching of the main building
blocks of IT systems development, i.e. programming,
database design, administration and use; and Web
development. Each of these areas has been taught as a
separate body of knowledge that IT students should learn
in isolation.
The IT course at QUT had been altered after faculty
reviews to address falling student numbers. However,
little was done in these reviews to address the poor
student outcomes for introductory programming.
2.4
Retention
Retention of students in introductory programming units
suffered as the student cohort altered as discussed above.
This was evidenced each semester at census dates by
shrinking class sizes.
Tutorial and/or practical attendances nearly always
declined as the weeks of the semester progressed. Factors
such as workload from other units and external
commitments were often blamed but a major factor may
likely have been lack of engagement with the unit
materials.
Overall results for introductory programming units in
the past have typically had a bimodal distribution with
one mode for those students who either had prior
knowledge of programming or managed to learn it
quickly and another mode for those students who did not
manage to learn to program well at all.
Students who performed poorly in these programming
units, which were normally core to all IT courses, were
unlikely to continue tertiary study and those who were
enrolled in double degrees would often discontinue the IT
degree. As Table 1 shows, semester 1, 2008 saw 19% of
students failing the first programming unit. In the same
semester, 35% of students discontinued their course with
over half of them withdrawing from university altogether
(see Table 2).
Semester 1, 2008
pass
fail
First Programming Unit
81%
19%
Table 1: Fail/Pass Rates
Semester 1, 2008
Changed to other course or
inactive/on leave
Discontinued course enrolment
Withdrew from First
Programming Unit
Attrition Rates
16.2%
18.4%
19.4%
Table 2: Attrition Rates
3
The New Course Structure
The design goal for the first semester core of the new
Bachelor of IT (BIT), introduced in 2009, was to improve
student engagement, and consequently progression, while
maintaining the quality of our graduates. The core
maintains the idea of ensuring that all of our BIT
graduates have a common set of skills and knowledge.
The technical material in the previous core was taught
pretty much the same way for the past 30 years. Many of
the technical units had poor progression rates and
particularly the programming and database units had very
poor progression rates. The Faculty of Science and
Technology (SciTech‟s) approach to teaching the
technical material was similar to most other mainstream
IT degrees and we had similarly poor progression rates.
We needed a different approach to introducing technical
content to see significant changes in progression rates.
The design of the new BIT took into account our
experiences teaching IT, model curricula and research in
pedagogy. In particular we considered the guidelines
provided by the ACM, AIS and IEEE Computer Society
Computing Curricula 2005 (The Joint Task Force for
Computing Curricula ACM/AIS/IEEE-CS 2006) along
with its companion volumes Computer Science (The
Interim Review Task Force ACM/IEEE-CS 2008),
Information Systems (Gorgone, Davis et al. 2002),
Information Technology (Lunt, Ekstrom et al. 2008) and
Software Engineering (The Joint Task Force on
Computing Curricula IEEE-CS/ACM 2004). (In the
remainder of this section we will use the term CC2005 to
refer to Computing Curricula 2005 and all of its
companion volumes.) CC2005 is a content driven view
of curricula. The design of the new BIT took into
account the knowledge areas and recommended topic
weighting from CC2005 but we purposely did not aim to
meet the recommended weightings. Given the breadth of
computing and the need to adequately develop generic
capabilities we found the CC2005 recommendations to be
too heavily biased towards content over general abilities.
We also noted that even the content recommended by
CC2005 was too narrow to adequately cover both the
breadth of information technology and the associated
non-computing knowledge required by graduates. A
recent workshop on redefining computing curricula noted
that in a short period of time they were able to identify
over 100 additional knowledge areas relevant to
computing that were not covered by CC2005 (Isbell,
Stein et al. 2010). In the end, the design of the new BIT
followed current pedagogical research being undertaken
at QUT (QUT 2009). The first semester core units and
units which immediately follow focus on preparing
students for their university studies. The middle part of
the degree focuses on delivering relevant material in
realistic contexts. The final year core units focus on
preparing students for their post-degree careers with an
emphasis on integrating the topics studied earlier in their
degree and on understanding the context in which they
will apply the knowledge and skills.
Consequently the new core has a shallower
introduction to the technical content and introduces this
content using an integrated approach, which matches how
the content is used in practice. We wanted to shift the
focus from didactic teaching to a more constructivist
approach to learning (Bowden and Marton 1999). Our
own experience introducing Problem-Based Learning
(PBL) into programming units (Adams, Clarke et al.
2001) and the experience of MacDonald (1998) in
medicine validates this shallower introduction of
material. The experience from the PBL community is
that a limited and focussed coverage of content provides a
better basis for students to learn material on their own,
rather than trying to cram everything into a course. The
integrated approach to introducing content material was
informed by the framework for Teaching and Assessment
of Software Development (Thomas, Cordiner et al. 2010),
which provides a structure for designing a stream of
technical units that make up a coherent whole. We
believe that if students are engaged in the learning
process and the material, they will learn the fundamental
concepts well giving them adequate skills for the
remainder of their degree, regardless of the area in which
they specialise.
The core is intended to develop well rounded students,
and leave technical depth to a set of specialisation
streams. This reflects industry feedback on requirements
for graduates with better business, communication and
teamwork skills, and also on the expected technical
ability of our graduates. The first semester units are not
meant to eliminate difficult technical material, but to
present it in a more engaging manner which better shows
how the course content inter-relates. The motivation for
this is the number of our students who do not see how the
different aspects of IT fit together (e.g. believing that
working with databases does not require programming
skills).
In student focus groups conducted at QUT, some of
the comments from students were that they wanted more
"hands on" work, more exposure to "real world" projects
and examples, a better introduction to object-oriented
programming, and more industry certification options.
Double degree students commented that SciTech was
better at building student cohorts than other faculties and
that we were better at getting students to work together.
Most students disliked working in teams if some team
members did not "pull their weight". However, we
recognise the importance of students building a network
of friends during their transition to university, as well as
the valuable generic skills that collaborative learning
affords them.
INB104 – Building IT Systems – is one of four core
units offered in the first semester of the first year of study
for Bachelor of IT students. It is also a core unit for
Bachelor of Games and Interactive Entertainment
students and for all students enrolled in double degrees
involving either of these degrees. Double degree students
typically undertake study in INB104 in the second
semester of their course.
The four core units form a coherent group which
expose students to: the breadth of domains in which IT is
used and how IT has changed those domains; advances in
computing devices to introduce hardware architecture,
networking and operating systems; and an introduction to
the profession of IT and the generic skills required by all
IT graduates.
INB104 rounds this set of units out with an
introduction to the basic building blocks of IT systems:
networks, databases, and software – programs, scripts and
Web development. While this may seem like a large
amount of material, the topics are covered without going
into too great a depth.
An aim of the unit is to provide students with some
experience in these building blocks before they choose a
set of follow-on units in second semester, which includes
Programming, Databases, Networks, and the Web. These
units then lead on to the areas in which students may
specialise, such as enterprise systems, software
engineering, Web development or networking.
3.1
Introducing INB104 – Building IT Systems
Combining programming, databases and Web
development into one first year unit allows the students to
gain an earlier understanding of these basic concepts
albeit at a more general level. The main aim of this unit
is to engage the students in these building blocks and to
learn the basics by immersing them in a variety of
interesting tasks that will use one, two or all three of the
technologies. Programming is being used as the glue
between the systems, rather than simply for the sake of
something that must be learnt.
Given the range of topics in INB104, programming
skills are restricted to the fundamental concepts of
sequence, selection, iteration and functions. We decided
to use Python as the introductory language due to our
experience with the language and its simplicity. Python
is open source, involves hassle-free installation and has a
simple syntax and development interface. It can be used
for writing simple scripts, full blown programs and for
the creation of object-oriented systems. It is also possible
to use the functional programming paradigm with this
language.
INB104 focuses on imperative programming using a
top down design approach which motivates the use of
functions. An imperative approach was chosen because it
provides the best approach to teaching the fundamental
concepts covered in the unit. A functional or objectoriented approach would require too much overhead in
the form of other concepts to suit the goals of the unit
when using Python. It is also an approach that suits the
design of scripting programs as well as providing the
underlying algorithmic logic required by object-oriented
programming. Students will encounter both of these
approaches in following units. Python library modules
are utilised for animation (PyGame), image manipulation
(PIL), and database communication (MySQLdb).
MySQL is used for projects which require database
interaction. It provides good GUI based tools for
interaction with a database server, so that SQL queries
can be tested before they are used in Python programs.
The Web development element of the unit covers a
subset of HTML and uses simple text editors in
preference for sophisticated development environments
like DreamWeaver.
INB104 highlights the fact that SQL and HTML are IT
system development languages that can be used to build
interactive systems.
3.2
Teaching Approach
Students are expected to attend a two hour demonstrationstyle lecture and a two hour computer laboratory based
tutorial session each week. The laboratory sessions are
supervised by one tutor for each class of 25 to 30
students. This level of support has been provided to our
first year students for some time now and does not
represent any increase in resources required for delivery
of the unit.
Lecture slides and laboratory worksheets are published
online prior to delivery and audio recordings are made
available following the lectures.
In the first week of semester, students are encouraged
to partake in an animated social discussion in the
practical sessions, aimed at breaking the ice and getting
to know a little about their fellow students. Students then
self-select into pairs. They are asked to choose carefully,
a partner they believe they will be able to work with
effectively throughout semester. Students are encouraged
to consider such things as demographics, culture,
background, motivation, programming experience and
timetabling. They are then introduced to the concept and
protocol of pair programming (Beck 2005) and provided
with background information supporting its use in both
industry and the learning environment. In the university
environment, we believe that the student pairs should be
fixed for the semester as the main aim of the learning
experience is the content matter rather than the pair
programming protocol.
In this and previous pair
programming experiments (Teague and Roe 2009) we
have found that after an initial settling in period, students
prefer to continue with the same partner for the duration
of the semester rather than spend time developing a
workable relationship with someone new.
Paired students are encouraged to actively engage with
their partner to complete practical exercises both during
practicals and at other negotiated times either on or off
campus. Tutors enforce regular swapping of roles
between driver and navigator, reminding them of ways
they can be active members of the pair including asking
questions, offering alternatives, researching syntax etc.
Activities in the weekly computer based practical
sessions are all directly related to the unit content
delivered at the prior lecture and these activities in many
cases feed directly into similar tasks that are required for
assessment. This approach is seen by the unit developers
as vital for providing relevance to the weekly schedule
and engagement with the unit material.
3.3
Assessment
Previous studies (McDowell, Werner et al. 2002; Urness
2009) have found that assignments developed by novice
programmers involving pair-programming result in better
quality code than individual submissions. The same
studies also note that similar exam averages are achieved
by those learning in a pair-programming environment as
those who have worked individually throughout semester.
Assessment for this unit is a combination of a portfolio
of activities undertaken during the semester, a reflective
report comparing initial and final skills in the areas taught
in the unit, and weekly online quizzes.
„Threshold concepts‟ that novice programmers
encounter and often get stuck on (Eckerdal, McCartney et
al. 2006) are made less of an issue with the support of
collaborative learning, and by offering a range of
assessment items that appeal to the students‟ sense of
enjoyment and target their interest areas e.g. gaming,
graphics, and Web development.
The assessment items are described in more detail
below.
3.3.1
Portfolio
Small projects are developed by pairs but teamwork is not
assessed. Student pairs spend some time working on the
project tasks in computer laboratory practical sessions but
are expected to complete the tasks using the same
collaborative protocols at negotiated times and places
with their partner between practicals. The students use a
problem solving framework which provides a scaffold for
developing the required skills. Students are expected to
take an inventory of their current skills at the beginning
of the task and then determine an approach that will
enable them to complete the task.
The tasks are somewhat open ended in their definition
and most include challenge content, allowing those
students who have pre-existing skills to use and challenge
those skills. For those students with little or no prior
knowledge, supporting material is supplied in lectures
and readings, while laboratory exercises supported by
tutors provide the opportunity for them to develop the
required
skills
with
hands-on
practice
and
experimentation. For the first offering of this new unit,
the portfolio was submitted twice in draft form, allowing
the students to receive feedback on their submissions
before finally being graded. The drafts were due for
submission at the end of weeks four and eight, with the
final portfolio submitted at the end of week 13.
Each submission required a selection of two projects
per pair, resulting in a final portfolio containing six
projects overall. The first draft submission required a
choice of practical exercises which students had been
completing in class. These were programming exercises
which entailed writing simple functions with or without
parameters which involved sequential expressions,
Boolean logic, conditions, iteration and/or the use of
Turtle graphics.
The second draft submission of the portfolio included
a selection of more challenging programming projects,
some of which required a little research and development
of skills not covered in class. One example of this was a
project to produce a bouncing balls animation using an
imported PyGame library. Other projects available for
selection at this stage involved using Lindenmayer
systems (Prusinkiewicz and Lindenmayer 1990) for
drawing fractal patterns with Turtle graphics and a
language translator using Python‟s built-in dictionary data
type and its functionality.
The final portfolio stage offered the choice of projects
which involved programming, SQL and/or HTML
development. A programming and SQL project required
students to populate an existing database with the
contents of supplied text files. One project required the
use of all three technologies, in which students produced
a HTML popularity cloud of gathered student data.
Another project had students design a set of static Web
pages to display images captured by traffic cameras
deployed around the South East corner of Queensland in
the form of a traffic camera wall.
3.3.2
Reflective Report
A brief search indicates that few programming courses
incorporate the use of reflective practice by its students.
Zagal and Bruckman (2007) talk about a blogging
environment developed for students as a learning tool, to
reflect on their game-play experiences. Kay et al (2007)
incorporated student reflective practice in their
programming education system which was designed to
facilitate self-assessment and reflection.
The reflective report for this unit required the students
to compare their initial and final skill levels in the areas
of computer programming, database usage and Web
programming. The students were instructed to undertake
a learning style survey (Fleming 2009) and to reflect
upon how that style was manifested during their learning
in this unit.
The development of the report was guided by a series
of questions that asked about knowledge of a topic,
comparison of understanding between the beginning and
end of the semester and whether or not the student
enjoyed learning about the topic. There was also a
further question asking if the student would enrol in
further units related to the topic.
3.3.3
Quizzes
For the first ten weeks of the unit students were required
to complete an on-line quiz, each contributing 1%
towards their final grade.
The first quiz was a
questionnaire designed simply to gather information
about students‟ IT and programming skills as well as
perceptions and attitudes which was later used by them to
help reflect on their development of skills and knowledge
throughout the semester.
The remainder of the quizzes consisted of multiple
choice questions which tested their knowledge of
concepts covered in the previous week‟s lecture and
practical session. Only one attempt per student was
permitted for each quiz, and they were expected to
complete the quizzes in their own time, within a week or
so from being available online. With no real time
restriction for completing a quiz and the freedom to use
whatever resources they felt necessary in order to answer
the questions, each quiz provided the opportunity for
students to reflect on their understanding of technical
content.
3.4
Semester 1
86%
14%
94%
6%
98%
2%
Semester 2
80%
20%
93%
7%
99%
1%
Table 4: Student Cohort Statistics
3.4.2
Learning Style
The VARK Learning Style questionnaire that students
completed as part of their reflective report assessment
provided a profile of their learning preferences in terms
of giving and receiving information (Fleming 2009).
Figure 1 below shows the breakdown of the learning
styles of students in the first offering of INB104.
Cohort Analysis
The cohort for this unit‟s first offering did not consist
entirely of straight Information Technology students.
Many of its students were enrolled in the School of IT‟s
new Bachelor of Games and Interactive Entertainment
(BGIE) degree and many combine their studies in IT with
a second degree. Information on the cohort is reported
below.
We have also surveyed the learning style preferences
of students in the cohort, using an on-line survey
(Fleming 2009). Information gathered from this survey
will be used to ensure that all learning styles are catered
to in future offerings of the unit. The predominant
learning styles of the cohort are also reported below.
3.4.1
Students
Male
Female
Full Time
Part Time
Domestic
International
Course Breakdown
The students undertaking the course come mainly from
the School of IT‟s two main courses, the Bachelor of
Information Technology (BIT) and Bachelor of Games
and Interactive Entertainment (BGIE) and double degrees
paired with these. Table 3 shows the breakdown of
courses in which students undertaking this unit are
enrolled. Further information about the student cohort
undertaking the unit is shown in Table 4.
Figure 1: Student Cohort Learning Styles
4
Our approach in measuring the evidence of the success of
the unit was as follows:
Comparison of student results before and after the
introduction of this unit.
Comparison of student attrition rates before and
after the introduction of this unit.
Comparison of number of instances of plagiarism
before and after the introduction of this unit.
Analysis of formal and informal student feedback
from this unit.
Comparison of attendance rates at computer
laboratory classes before and after the introduction
of this unit.
Analysis of commentary relating to favourite
assessment tasks in reflective reports.
4.1
Course
BIT
BIT/Double
BGIE
BGIE/Double
Other Single Degree
Other Double Degree
Other
Semester 1
34%
19%
39%
1%
4%
1%
3%
Semester 2
26%
42%
4%
18%
8%
1%
2%
Table 3: Breakdown of Degree Course Enrolments
Measures of Success
Final Results and Attrition
The outcome of the assessment regime for the first
offering of the unit was a unimodal distribution centred
around 75%. Approximately 70% of students achieved a
grade of 6 or 7, 20% achieved a grade of 4 or 5 and only
6% of the cohort failed to achieve a passing grade.
Updating Table 1 and Table 2 above gives some
indication of the effect that pass rates may have had on
attrition. Table 5 below shows that in semester 1, 2009
the failure rate dropped from 19% to 6%. Table 6
indicates that the attrition rate from the courses dropped
from 35% to 9% while the attrition rate from the first
programming unit has dropped from 19% to 6%.
Result
pass
fail
First Programming Unit
Sem 1, 2008
Sem 1, 2009
81%
94%
19%
6%
Table 5: Fail/Pass Rates
Withdrawal
Changed to other course or inactive
Discontinued enrolment
Withdrew 1st Programming Unit
Attrition Rates
Sem 1, Sem 1,
2008
2009
16%
4%
18%
5%
19%
6%
Table 6: Attrition Rates
There has been a slight increase in the number of
students who submitted assignments and who undertook
weekly quizzes, indicating that the assessment tasks
chosen for the unit seem to have been engaging to the
student cohort.
4.2
Plagiarism
An unexpected but welcome benefit of the approach
being used has been a reduction in plagiarism of
assignment work. There have been only two instances of
plagiarism so far in the new unit. This is a dramatic
reduction when compared to previous offerings where
there have typically been five to ten cases of plagiarism
detected.
The reason for this has not been fully investigated but
the most likely reason is the collaborative learning
environment.
The unit has been useful in providing me with skills to
develop my programming expertise. The collaborative
approach has been very useful for me personally.
The whole unit is structured well. I personally found it
engaging, as it had a great many ways to learn the topic.
Going to lectures, listening to lectures via recording, the
tutorial exercises and going to the practical for extra
help, you can't lose.
The PASS (Peer Assisted Study Scheme) leader for
this unit also provided interesting feedback about the
collaborations:
I think the [pair] programming team idea is working very
well, I have had a fair few couples come in and talk to me
during pass sessions, and they often manage to figure out
answers as a pair with a little push in the right direction.
From what I have seen it is also stopping people being
lazy...(I've seen some very motivated groups).
4.4
Workshop Attendance
Historically in first year IT subjects at QUT, attendance at
scheduled lectures, tutorials and practicals dramatically
declines through the teaching semester. For example in
2007, an average of 80% of students initially attended
programming practicals, and by the end of semester only
16% of students turned up. A similar pattern has been
recorded for subsequent semesters (Teague and Roe
2009). Figure 2 displays a comparison of practical
attendance rates from previous semesters for the first
programming unit at QUT and for the first two offerings
of INB104 in 2009.
Percentage of Students Attending Practicals
100%
90%
4.3
Student Feedback
To measure the success of the design of Building IT
Systems we have collected formal and informal student
feedback. Some student commentary is reproduced
below. While there has been some feedback from
students that has been negative, the majority of the
comments have praised the unit. This is definitely the
case with students who identified themselves through
their comments as having failed the predecessor unit.
Students also provided positive feedback in their
reflective reports about their learning experiences:
I have definitely changed my mind slightly about several
aspects after completing this course. I thought the
programming part would be extremely hard and boring.
I was very wrong about this and the programming parts
didn’t turn out to be as difficult as I expected and they
were more challenging and exciting than boring.
Students were invited to provide feedback through the
university‟s Learning Experience Survey (LEX). The
response rate from students in the unit was over 37%.
Students were asked a number of questions using a fivepoint Likert scale, while also given the opportunity to
provide free form qualitative comments:
This unit wastes no time in getting right into the fun stuff,
getting you engaged and making you participate. The
problems were complex but solvable and the tutors and
lecturers were always willing to help.
80%
70%
60%
Sem 2, 2007
50%
Sem 1, 2008
40%
Sem 1, 2009
30%
Sem 2, 2009
20%
10%
0%
2
3
4
5
6
7
8
9
10
11
12
13
Week of Semester
Figure 2: Practical Attendance Rates
Attendances were recorded for 50% of the practicals in
both semester 2, 2007 and semester 1, 2008. For the
subsequent semesters in 2009, attendances were recorded
for the entire cohort.
A marked improvement in
attendance rates by students in the new unit has been
evidenced.
In semester 1, 2009, attendances were not collected in
two of the practicals, one in week 10 and the other in
week 12. For these two practicals, the attendance
averages were 24 and 23 students respectively, each
equating to approximately 7% of the entire cohort of
enrolled students. Therefore, the actual attendance
percentages for weeks 10 and 12 was more likely 7%
higher than recorded in the graph. In week 4 of semester
2, 2009 the significant drop in attendances is due to a
public holiday.
The reason for improved attendance at weekly
computer laboratory practical sessions seems to be linked
to the assessment tasks that have been used. This is
discussed further in the next section regarding evidence
collected from the reflective reports.
4.5
Analysis of Reflective Reports
The reflective reports were intended to highlight to the
students the amount of learning that was achieved by
participation in INB104. This was elicited from students
through a series of questions.
The reflective reports have also provided insights from
the students as to what engaged them in the unit. These
insights indicate that the use of the portfolio of activities
as the main assessment item for the unit and allowing
students to choose which project tasks to include in their
portfolios was a major reason for the success of the unit
in reducing attrition and for increasing the pass rate.
Approximately 20% of students made some comment
in their reflective reports about the specific projects that
they undertook in the unit which they found most fun and
engaging. Table 7 below shows that the projects with
some visual feedback component were favoured by the
students in the unit. We believe that this is not
unexpected as the visual result indicates the successful
completion of the project.
Project Topic
PyGame Animation
Database Manipulation
Turtle Graphics
Popularity Cloud (HTML
production by Python with
data from a database)
Preferred Project
36%
10%
41%
33%
Table 7: Students' Preferred Projects
Further analysis of the reflective reports shows that
92% of students made some comment in their reflective
reports about which breadth units they would or would
not study in future semesters of their degree course and
this is shown in Table 8 below.
Area of Study
Programming
Databases
Web
Development
Response
Will Study
Will Not Study
No Indication
Will Study
Will Not Study
No Indication
Will Study
Will Not Study
No Indication
63%
33%
4%
51%
37%
12%
66%
25%
9%
Table 8: Response Rates Regarding Future Units
It is postulated that this will have a positive impact on
the student cohorts in those breadth units. By being
exposed to the major building blocks of IT systems in this
unit, students have gained a better understanding of the
areas and made decisions on whether these areas are of
interest to them in their future careers and further studies.
5
Future Plans
The next iteration of INB104 (this current teaching
period) introduces an end of semester exam which will
contribute 30% to a student‟s final grade. The exam will
attempt to test a working understanding of the basic
concepts introduced during the semester covering
programming, SQL, HTML and how these technologies
inter-relate. Plenty of time will be provided to complete
the exam, and enable them to perform to the best of their
ability and to experience university exam conditions with
low stress.
None of the introductory core units in the new course
initially incorporated an exam as an assessment item. It
was felt that INB104 was the logical unit among these
core units for an exam to be used as a type of assessment
that first year students should be exposed to.
Examinations provide an alternative yet valid method for
measuring learning outcomes in a compressed time frame
with minimal opportunities for plagiarism.
Marking of the portfolio projects was very timeconsuming, with both source code files and often quite
verbose documentation to wade through and markers
were instructed to provide significant and effective
feedback to the students. The current iteration of the unit
will have only one draft portfolio before the final version
is submitted. This will reduce the marking workload on
teaching staff, but still provide students with valuable
feedback in order to both improve their submission, and
prepare for the final exam.
Students loved the „fun stuff‟, and that meant
something different for each student. The Games degree
students tended to prefer Turtle graphics and the task
using the PyGame library, while others found joy in
building a database or creating Web pages using Python.
While the projects will generally be recycled in
subsequent semesters, new projects will also be
developed, providing a wider range of project options for
students. Ideas for new projects include making use of
readily available libraries for Python.
Students were generally very interested in their
learning style profiles provided by the VARK Learning
Style questionnaire. In the current and future offerings of
this unit, students will be required to complete the
learning style questionnaire in week 1, which gives them
the opportunity to immediately take advantage of the
information they gain about their learning preferences
during their study.
6
Conclusions
In response to falling student numbers in the Bachelor of
IT degree at QUT, and high failure rates and high attrition
rates in introductory computer programming units, the
faculty undertook a major course revision which included
significant changes to the first programming unit
encountered by first year students.
We believe that the changes made to the unit have
resulted in better engagement with the material by the
students. Attrition from the unit has been reduced to 6%.
The failure rate in the unit has also been reduced to 6%.
This has been achieved while maintaining introductory
computer programming concepts – statement sequences,
conditional statements, iterative and recursive approaches
to repetition, and functional decomposition using top
down design.
The unit has also introduced some of the basic
concepts of database design and manipulation,
networking, and Web page production using HTML. We
believe that we have covered the core fundamentals of
these topics in this unit, giving students sufficient
grounding in these areas so they are aware of them in
their professional lives, even if they will not directly use
those skills.
Workshops were conducted in a pair programming
mode so that students could learn in a collaborative
manner, being able to support each other‟s learning.
Informal and formal feedback indicates that this has been
well received by the students.
Data collection from student assessment supports our
claim that project tasks have been designed to be
engaging. We believe that this is because many of the
projects have a visual outcome, and that they are
extensible so that students with prior knowledge in the
area can demonstrate their higher level skills.
For assessment, students have undertaken weekly
quizzes and produced a portfolio of the collaborative
project tasks undertaken and have written a reflective
report, outlining the skills and knowledge gained during
the semester. The reflective report has also required
students to articulate their preferences with regard to the
topics covered in the unit. This has afforded them the
opportunity to contemplate the different study paths that
are offered from which they can select topics to study for
the remainder of their course. We believe that this may
also lead to lower attrition rates and lower failure rates for
follow on units, as students have written in their
reflections that they will not enrol in units that they now
know to be personally unappealing.
7
References
Adams, M., Clarke, S. and Thomas, R. (2001): Model of
thinking in the PBL process: Comparison of medicine
and information technology. Proc. Third Asia Pacific
Conference
on
Problem
Based
Learning,
Rockhampton, QLD, Australia.
Beck, K. (2005): Extreme programming explained:
embrace change Boston, MA, Addison-Wesley.
Berenson, S.B., Slaten, K.M., Williams, L. and Ho, C.-W.
(2004): Voices of women in a software engineering
course: Reflections on collaboration. Journal on
Educational Resources in Computing (JERIC) 4(1).
Biggers, M., Brauer, A. and Yilmaz, T. (2008): Student
perceptions of computer science: a retention study
comparing graduating seniors vs. CS leavers. Proc.
39th SIGCSE Technical Symposium on Computer
Science Education, Portland, OR, USA, ACM.
Bowden, J. and Marton, F. (1999): The university of
learning. London, Kogan Page.
Cohoon, J.M. (2002): Women in CS and biology. ACM
SIGCSE Bulletin 34(1): 82-86.
Davis, J. and Rebelsky, S.A. (2007): Food-first computer
science: starting the first course right with PB&J. Proc.
38th SIGCSE Technical Symposium on Computer
Science Education, Kentucky, USA, ACM.
Eckerdal, A., McCartney, R., Moström, J.E., Ratcliffe,
M., Sanders, K. and Zander, C. (2006): Putting
threshold concepts into context in computer science
education. ACM SIGCSE Bulletin 38(3): 103-107.
Feinberg, D. (2007): A visual object-oriented
programming environment. Proc. 38th SIGCSE
Technical Symposium on Computer Science Education,
Kentucky, USA, ACM.
Fisher, A. and Margolis, J. (2002): Unlocking the
clubhouse: the Carnegie Mellon experience ACM
SIGCSE Bulletin 34(2): 79-83.
Fleming, N.: VARK a guide to learning styles.
http://www.vark-learn.com/english/index.asp.
Accessed 6 Aug 2009.
Gehringer, E.F., Deibel, K. and Whittington, K.J. (2006):
Panel: cooperative learning - beyond pair programming
and team projects. Proc. 39th SIGCSE Technical
Symposium on Computer Science Education, Houston,
Texas USA, ACM.
Gorgone, J.T., Davis, G.B., Valacich, J.S., Topi, H.,
Feinstein, D.L. and Longenecker, H.E., Jr. (2002).
Model curriculum and guidelines for undergraduate
degree programs in information systems. Association
for Information Systems.
Isbell, C., Stein, L.A., Cutler, R., Forbes, J., Fraser, L.,
Impagliazzo, J., Proulx, V., Russ, S., Thomas, R. and
Xu, Y. (2010): (Re)defining computing curricula by
(re)defining computing. InRoads SIGCSE Bulletin (in
press).
Kay, J., Li, L. and Fekete, A. (2007): Learner reflection
in student self-assessment. Proc. Ninth Australasian
Computing Education Conference, Ballarat, Victoria,
Australia, ACS.
Kinnunen, P. and Malmi, L. (2006): Why students drop
out CS1 course? Proc. Second International Workshop
on Computing Education Research, Canterbury, United
Kingdom, ACM.
Lewis, S., McKay, J. and Lang, C. (2006): The next wave
of gender projects in IT curriculum and teaching at
universities. Proc. Eighth Australasian Computing
Education Conference, Hobart, Tasmania, Australia,
ACS.
Lister, R. (2004): Teaching Java first: experiments with
pigs-early pedagogy. Proc. Sixth Australasian
Computing Education Conference, Dunedin, New
Zealand, ACS.
Lister, R., Adams, E.S., Fitzgerald, S., Fone, W., Hamer,
J., Lindholm, M., McCartney, R., Moström, J.E.,
Sanders, K., Seppällä, O., Simon, B. and Thomas, L.
(2004): A multi-national study of reading and tracing
skills in novice programmers. ACM SIGSCE Bulletin
36(4): 119-150.
Lunt, B.M., Ekstrom, J.J., Gorka, S., Hislop, G., Kamali,
R., Lawson, E., LeBlanc, R., Miller, J. and Reichgelt,
H. (2008). Curriculum guidelines for undergraduate
degree programs in information technology. ACM and
IEEE-CS.
MacDonald, P.J. (1998): Selection of health problems for
a problem-based curriculum. In The Challenge of
Problem-Based Learning. 2. D. Boud and G. Feletti
(eds). Kogan Page.
Mannila, L. (2006): Progress reports and novices‟
understanding of program code. Proc. Sixth Baltic Sea
Conference on Computing Education Research, Koli
Calling.
McCracken, M., Almstrum, V., Diaz, D., Guzdial, M.,
Haga, D., Ben-David, K.Y., Laxer, C., Thomas, L.,
Utting, I. and Wilusz, T. (2001): A multi-national,
multi-institutional study of assessment of programming
skills of first-year CS students. ACM SIGCSE Bulletin
33(4): 125-180.
McDowell, C., Werner, L., Bullock, H. and Fernald, J.
(2002): The effects of pair-programming on
performance in an introductory programming course.
Proc. 33rd SIGCSE Technical Symposium on
Computer Science Education, Cincinnati, Kentucky,
USA, ACM.
McDowell, C., Werner, L., Bullock, H.E. and Fernald, J.
(2006): Pair programming improves student retention,
confidence, and program quality Communications of
the ACM 49(8): 90-95.
McKinney, D. and Denton, L.F. (2006): Developing
collaborative skills early in the CS curriculum in a
laboratory environment. ACM SIGCSE Bulletin 38(1):
138-142.
Mendes, E., Al-Fakhri, L. and Luxton-Reilly, A. (2006):
A replicated experiment of pair-programming in a 2ndyear software development and design computer
science course. ACM SIGCSE Bulletin 38(3): 108-112.
Nagappan, N., Williams, L., Ferzli, M., Wiebe, E., Yang,
K., Miller, C. and Balik, S. (2003): Improving the CS1
experience with pair programming. Proc. 34th SIGCSE
Technical Symposium on Computer Science Education,
Reno, Nevada, USA, ACM.
Parsons, D. and Haden, P. (2006): Parson's programming
puzzles: a fun and effective learning tool for first
programming courses. Proc. Eighth Australasian
Computing Education Conference, Hobart, Australia,
ACS.
Pollard, S.L. and Duvall, R.C. (2006): Everything I
needed to know about teaching I learned in
kindergarten: bringing elementary education techniques
to undergraduate computer science classes. Proc. 37th
SIGCSE Technical Symposium on Computer Science
Education, Houston, Texas, USA, ACM.
Prusinkiewicz, P. and Lindenmayer, A. (1990): The
algorithmic beauty of plants. New York, Springer
Verlag.
QUT:
Supporting
real
world
learning.
http://www.otq.qut.edu.au/initiatives/projects/.
Accessed 23 Oct 2009.
Robins, A., Rountree, J. and Rountree, N. (2003):
Learning and teaching programming: a review and
discussion. Journal of Computer Science Education
13(2): 137-172.
Sheard, J. and Hagan, D. (1998): Our failing students: a
study of a repeat group. ACM SIGCSE Bulletin 30(3):
223-227.
Simon, B. and Hanks, B. (2007): First year students'
impressions of pair programming in CS1. Proc. Third
International Workshop on Computing Education
Research, Atlanta, Georgia, USA, ACM.
Soloway, E.I. and Spohrer, J.C. (1989): Studying the
novice programmer. Hillsdale, NJ, Lawrence Erlbaum
Associates.
Teague, D. and Roe, P. (2009): Learning to program from pear-shaped to pairs. Proc. First International
Conference on Computer Supported Eductaion, Lisbon,
Portugal, INSTICC Press.
The Interim Review Task Force ACM/IEEE-CS (2008).
Computing science 2008: an interim revision of
CS2001. ACM and IEEE - CS.
The Joint Task Force for Computing Curricula
ACM/AIS/IEEE-CS (2006). Computing curricula
2005: the overview report. ACM and IEEE Computer
Society.
The Joint Task Force on Computing Curricula IEEECS/ACM (2004). Curriculum guidelines for
undergraduate degree programs in software
engineering. ACM and IEEE Computer Society.
Thomas, R., Cordiner, M. and Corney, D. (2010): An
adaptable framework for the teaching and assessment
of software development across year levels. Proc.
Twelfth
Australasian
Computing
Education
Conference, Brisbane, Queensland, Australia, ACS.
Urness, T. (2009): Assessment using peer evaluations,
random pair assignment, and collaborative programing
in CS1. Journal of Computing Sciences in Colleges
25(1): 87-93.
Varma, R. (2006): Making computer science minorityfriendly. Communications of the ACM 49(2).
Vilner, T. and Zur, E. (2006): Once she makes it, she is
there: gender differences in computer science study.
Proc. 11th Annual SIGCSE Conference on Innovation
and Technology in Computer Science Education,
Bologna, Italy, ACM.
Williams, L. and Kessler, R.R. (2000): The effects of
“pair-pressure” and “pair-learning” on software
engineering education. Proc. 13th Conference on
Software Engineering Education & Training, Austin,
Texas, USA, IEEE Computer Society.
Wilson, J.D., Hoskin, N. and Nosek, J.T. (1993): The
benefits of collaboration for student programmers.
Proc. 24th SIGCSE Technical Symposium on Computer
Science Education, Indianapolis, Indiana US, ACM.
Zagal, J.P. and Bruckman, A.S. (2007): GameLog:
fostering reflective gameplaying for learning. Proc.
2007 ACM SIGGRAPH Symposium on Video Games
San Diego, California, ACM.