Teachers Using Classroom Data Well
Teachers Using Classroom Data Well
Teachers Using Classroom Data Well
Final report
December 2013
School of Education
The University of Queensland
Acknowledgements
The project team at the School of Education would like to thank the individuals and
organisations who generously shared their time, experiences, opinions, and artefacts for
the purposes of this project. This project would not have been possible without their
support and participation. We would also like to thank the Queensland College of Teachers
for their ongoing support of this project.
The project team would also like to acknowledge that this research was funded by the
Queensland College of Teachers. The views expressed herein are those of the authors and
are not necessarily those of the Queensland College of Teachers.
2|Page
Table of Contents
Table of Contents 3
List of tables 6
List of figures 6
List of text boxes 6
List of pictures 7
Executive summary 8
Introduction 8
Literature review 8
Methodology 10
Key findings 10
Recommendations for further development and future directions 14
1. For the QCT: 14
2. For principals and school leaders: 15
3. For teachers: 16
4. For dissemination of exemplars of good practice: 17
Introduction 18
Terms of reference 18
Report structure 19
Data collection 21
Section 1: Literature 22
Literature review 22
Part A: Context 22
Theme 1: Data and accountability 22
Vertical accountabilities: Lessons for classroom assessment 23
Horizontal accountabilities: Lessons for classroom assessment 26
Accountability, national testing, and improvement 27
Barriers to effective use of data to improve practice 29
Collecting, interpreting and analysing data 30
Part B: Focus on teachers 32
Theme 2: Data and assessment literacy 32
Formative and summative assessment 34
Assessment feedback 36
Feedback systems 37
3|Page
Authentic assessment 37
Assessment for learning 38
Theme 3: Data and numerate teachers 41
Data-driven systems 41
Types of data systems 43
Data-informed decision making and improving student learning 44
Cycles of continuous improvement 45
Leadership 49
Professional learning 51
Professional judgement 52
Part C: Strategies 53
Theme 4: Using data 53
Data sources 53
Numeracy and literacy data 55
A mathematics-based example 59
Section 2: Descriptive accounts 62
Access to useful data 62
Assessing student learning 65
Alignment 65
Using frameworks 65
Classroom data 67
Diagnostic tools 68
Conversations 69
Providing feedback to students on their learning 70
Making consistent & comparable judgments 70
Accountability 71
Professional judgements 71
In-school support for teachers 72
Mentoring teachers – Coaching academy 72
Education Officers 73
Moderation 74
External moderation 74
Internal moderation 75
Professional learning 77
Interpreting student data 78
Accountability 79
4|Page
Data inquiry models and cycles 79
Attendance, suspension and exclusion data 79
Alignment and differentiation within the P-12 structure 81
Tracking progress 81
Towards the QCE and OP – Feedback to students 81
Towards improved academic outcomes - Feedback to students 83
Towards career pathways 83
Standardised test data 84
For differentiated classes 84
Timing of the release of NAPLAN data 85
Systemic use of NAPLAN data 86
Pedagogy 87
Pedagogy and feedback 88
Pedagogy, assessment and the curriculum 88
Reporting on student achievement 89
Accountability 90
Communication with parents 90
Academic reporting 90
Comment-free reporting 91
Collating, analysing and displaying data on student learning 92
Spreadsheets 92
Warehousing tools 93
Section 2: Summary of key ideas 98
Section 3: Cases of strategies and practices for using classroom data 101
Case 1: Collecting and using diagnostic data focusing on reading 105
Case 2: Using data in visual displays 115
Case 3: Data-informed planning cycles 121
Case 4: Data for tracking progress and mentoring students 127
Case 5: Data and differentiated learning 134
References 140
Appendix A: Annotated bibliography 149
Appendix B: Alphabetical list of strategies and practices for using classroom data
192
5|Page
List of tables
List of figures
List of pictures
Picture 1: Spreadsheet display on the left and Australian curriculum on the right ............. 109
Picture 2: Group composition for the week based on children’s understandings (Red, green,
white) .................................................................................................................................... 111
Picture 3: Colour-coded Data wall ........................................................................................ 117
Picture 4: QTAC Guide (http://www.qtac.edu.au/OtherServices/Publications.html) ......... 131
7|Page
Executive summary
Introduction
The Queensland College of Teachers (QCT) initiated this report as an investigation of the
practices associated with Standard 5 of the Australian Professional Standards for Teachers
(AITSL, 2011). Related to professional practice, Standard 5 refers to “assess, provide
feedback and report on student learning.” This Standard incorporates school executive and
teacher understandings of how to collect, analyse, interpret and use systemic and
classroom data to support and improve students’ learning. It is argued that the
development of teachers’ knowledge and skills in these areas will assist with informing and
generating improved student outcomes. Simultaneously the acquisition of such knowledge
and skills will empower teachers as they become inquiry-minded and data-literate.
Literature review
An extensive review of the literature related to the topics of: assessment, data collection,
analysis, interpretation, and data use was undertaken. The literature review was organised
into four themes that emerged from an inductive analysis of the literature.
The first theme was identified as: data and accountability. Vertical and horizontal
accountabilities are mentioned in the literature. At present in systems and schools there is
a preoccupation with vertical accountabilities, which refer to the use of externally
mandated, high stakes testing. These types of assessment/testing are almost solely used to
audit and monitor schools for external purposes and are often associated with the
performance of schools and their teachers. This first part of the literature review canvases a
range of lessons for classroom assessment that can be drawn from an understanding of
vertical and horizontal accountabilities. In addition, a close analysis of the literature in this
section reveals that increasingly both vertical and horizontal accountabilities are being used
8|Page
more appropriately, and more usefully, to help improve students’ learning outcomes. This
section concludes with a discussion of the barriers to the effective use of data and to how
systems and schools might break down these barriers.
The second theme was identified as: data and assessment literacy. Assessment literacy
refers to the ability to select appropriate assessment techniques, to assess students’
learning for summative and formative purposes, to provide feedback to students on their
learning and to use effective strategies to make consistent and comparable assessment
judgements. Assessment literacy is associated with the assessment culture in a school that
promotes professional dialogue around assessment and a common language for discussing
and analysing the characteristics of effective assessment practice. The literature review in
this section described the key findings related to formative and summative assessment,
assessment feedback, and an understanding of authentic assessment. Considerable
attention in this section is paid to assessment for learning and refers to influential
researchers in this area, such as Stiggins (2002) who argues that teachers and policy makers
should lay importance on assessment for learning where information gained from
assessment is used to inform teachers about students’ learning and to improve it.
The third theme that emerged from the review of the literature was: data and numerate
teachers. This theme refers to the need for teachers to understand what data is, be able to
interpret it carefully, be aware of the limitations of data, and evaluate whether their
decisions based on data are justified. Specifically teachers need to become knowledgeable
about data and to be able to use it competently and confidently in order to make
instructional decisions. The literature review highlights the types of knowledge and skills
that teachers need to develop as they engage with various data-driven systems. A number
of models of the processes of becoming data numerate are described. These include the
Inquiry Circle - A Philosophy of Continuous Improvement (Barnes, 2004) and the Data Wise
Improvement Process (Boudett, City, & Murnane, 2006). The importance of data teams and
of leadership to develop a school’s capacity to be data numerate are highlighted. In the last
part of this section of the literature review the role of professional learning in becoming
data numerate and the importance of active reflection in professional learning are
highlighted.
The fourth theme referred to: using data. The fourth theme describes the strategies
mentioned in the literature that can be used to source and interpret data. Initially in this
section of the literature review various researchers’ and writers’ accounts of the uses of
data are presented. Next, examples are provided from two Australian states describing how
literacy and numeracy data, namely NAPLAN data, are used. In addition, an example of the
use of diagnostic data in numeracy is provided. These examples serve to highlight how data
can be used. This project sought to reveal other practices of data use.
9|Page
Methodology
The first data set in this report was collected through consultation interviews with senior
staff in various key organisational stakeholder groups. These groups comprised educational
authorities from the government, Catholic and Independent sectors in Queensland. The
purpose of this data collection was to identify examples of effective practices of data
collection and use. Consultations also addressed issues arising from the literature review,
such as the barriers to effective data use. Therefore, stakeholders were asked about the
challenges they faced implementing Standard 5 of the Australian Professional Standards for
Teachers. Stakeholders were also asked to comment on assessment and data use in the
current national and state educational policy contexts. For example, top-down or vertical
accountability to governments and schooling systems has dominated assessment practices
in recent years, therefore, stakeholders were asked about accountability pressures they
may have felt or encountered associated with implementing Standard 5. The stakeholders
were also asked about their expectations related to how teachers might exemplify the
Australian Professional Standards for Teachers, with particular reference to Standard 5.
Another data set was collected through interviews with key informants, such as principals,
heads of departments, and teachers. These interviews were conducted during school visits
across the three sectors. This data set also included the collection of artefacts and
exemplars that were thought to demonstrate effective practice. Additional data was
gathered from public documents such as annual reports, and education and school
websites.
General titles will be used when referring to participants. For example, ‘education officer’
refers to individuals employed at a district or regional level within an education authority.
Some of these individuals are in roles other than those of education officers, but are
identified using the collective title. The sector (government, Catholic or Independent) has
not been identified for reasons of confidentiality. “Principal”, “head of department” and
“teacher” will be used to identify the participants’ role, but again, the sector will not be
mentioned. In total, there were five individuals identified with the title “education officer”,
two as “principal’, one as “head of department”, and eight as “teacher”.
Key findings
There are eight key findings. Each finding is discussed in turn in the next section.
The findings of this investigation reveal that the main focus of the descriptive accounts
from stakeholders was on accountabilities and alignment. That is, the vast majority of the
stakeholders’ remarks were dominated by references or inferences related to “being
10 | P a g e
accountable”, “having to be accountable” or “expecting others to be accountable”. In
addition, the stakeholders made references to the need for alignment between the
purposes for which data were collected and the uses of the data.
The descriptive accounts indicated that the education systems and schools were inundated
with data, which they had created or was provided to them by educational authorities. The
stakeholders suggested that this increased emphasis on measurement data served
primarily as a tool for improving vertical accountability. In commenting on this
phenomenon, Lingard and Sellar (2013) have suggested some of this measurement data, in
particular the NAPLAN data, act as “catalyst data” and are pivotal to school and system
accountability practices. Indeed such data seem to “launch” even more data collection.
There was concern that some data were not interpreted and left as raw data rather than
being interpreted and used to good effect in schools.
The stakeholders spoke of the recent and very common practice of making visual displays
of the data collected. The data were presented via online portals that were accessible by
computers.
One key finding related to such visual displays was that it became apparent that access to
data was related to the position of the individual within the education system. Often only
specific key people (e.g., the Principal, Deputy Principal, a Head of Departments or Head of
Curriculum) had access to the data and “released” it to the teachers. While this provides an
ease of access, it is also a mechanism of control. Increasingly, within societies of control,
schools rely more and more on numerical codes and data, and the utilisation of socio-
technological mechanisms such as passwords that create gatekeepers and users (Deleuze,
1995). Consequently, institutions such as schools are becoming corporate systems that are
increasingly required to maintain copious record keeping of measurement and
11 | P a g e
performance data in order to enable auditing (Power, 1999) and at the same time privilege
only certain people with the data and/or the release of the data.
The incidence of online warehousing tools and the utilisation of dashboards was evident
across two of the sectors. These data displays provided teachers, schools and education
systems with tools for comparison of individual students, classes, and schools across a
state, nationally and internationally, as well as comparisons of teachers based on
extrapolated student data. These warehousing tools also scaffolded the data interpretation
process for the teachers by providing pre-determined formats and visual displays. However,
Australian and international studies (Hayes, Mills, Christie, & Lingard, 2006; Sahlberg, 2007)
have indicated the limited usefulness of these data and cautioned against the use of
simplistic analyses and international comparisons derived from these data because these
data fail to take into account the many underlying characteristics, such as socio-economic
status or family background, that may explain the comparative performance of schools.
The stakeholders identified a tendency towards the use of other visual and public displays
of data, such as the use of spreadsheets and data walls. Often the stakeholders spoke
about such public displays in the context of becoming sites of professional conversations
among teachers about assessment and student improvement. However, conversations
surrounding these data walls were generally focused on data obtained from diagnostic or
other classroom assessments rather than high-stakes NAPLAN data, or conversations about
assessment practices and pedagogy.
The displays, while public, were generally reserved for the teacher cohort, rather than
being made available to students and their parents. However, conversations between
teachers that were based on data from the visual displays also led to conversations with
students and their parents.
The descriptive accounts focused on the multiple purposes of data. The stakeholders were
of the opinion that effective use of data was dependent upon alignment between the
purposes for which data were collected and the consequent purposes and uses of these
data. Here, it was suggested, there needs to be a “fit” or an alignment of these purposes in
order to ensure integrity of practice and the effective use of data.
Many of the descriptive accounts from the systemic and administrative levels focused on
literacies and numeracy. Stakeholders linked the uses and purposes of data to improving
12 | P a g e
students’ literacy and numeracy performances. Improvement was frequently gauged
through NAPLAN testing; however, diagnostic literacy and numeracy assessments were also
used.
While much of the data related to literacies and numeracy, the stakeholders generally did
not speak of such data in relation to the actual curriculum, for example, the Australian
Curriculum and literacy and numeracy as general capabilities, or to the use of literacies and
numeracy in the learning areas of the Curriculum. While this omission could be related to
the context of the data gathering process associated with this study, it is thought to more
likely be associated with the very, very strong emphasis on measurement and performance
data from high-stakes testing (NAPLAN) and the known consequences of such an emphasis
which includes a narrowing of the curriculum (Darling-Hammond, 2010; Klenowski, 2011;
Sahlberg, 2010; Stobart, 2008; Thompson, 2012). Additionally, this omission could relate to
the need for a greater focus on aligning curriculum with pedagogy and assessment (Hayes
et al., 2006).
7. The danger of failing to use data to differentiate learning to meet students’ individual
needs
As indicated in reporting of the findings earlier, the notion of differentiation was not
significantly addressed in the stakeholder accounts. As such, there is a danger that while
there is an abundance of data, these data are not focused on providing differentiated
learning opportunities for those who are most marginalised in our society.
Classroom data can be used to help align classroom instruction with learning goals and
simultaneously be used to refocus pedagogy on content and skills that are lacking (Moon,
2005). The danger associated with misalignment or a lack of differentiation is that students
become disengaged with the learning process and there is an associated risk of the
emergence of negative behaviours and negative attitudes that contribute further to an
achievement gap (Moon, 2005).
13 | P a g e
8. Developing a broader understanding of what counts as data
Many of the descriptive accounts provided a limited understanding of what counts as data.
Data was almost exclusively limited to student performance data, often derived from tests,
and, as indicated earlier, the data often focused only on literacies and numeracy.
With respect to the data described and exemplars offered, there were none that recorded
students’ abilities to engage in analysis and evaluation, to apply knowledge and skills to
real-life contexts and problem-solving, or to use critical thinking – the so-called 21st
century skills. In addition, there were few references to students’ skills in communication or
related to their affect and social-emotional well-being, although one school referred to
tracking students with respect to their career aspirations and consequent achievements on
their future career paths. Finally, in discussing data, there were few references of the need
to take into account or to “read” and interpret data in the contexts of students’ access and
engagement with learning, their opportunities to learn, and the teaching practices
employed. Thus it is suggested that developing a broader understanding of what counts as
data and ensuring that attention is paid to the broader contexts of data gathering so that
richer and more nuanced understandings and uses of data can be developed are essential.
An effective professional learning program that helps teachers learn to use classroom data
well would focus on:
The AITSL standards are written with respect to the competencies of individual teachers,
but classroom assessment occurs within collegial professional practices, involves shared
responsibility for students’ learning across time, and collective accountability to parents, to
the schooling systems that employ them, and to the local and broader community. This
report has highlighted the importance of teachers collaborating in planning and
implementing quality assessment, interpreting and moderating assessment, and reporting
to parents. This shared, joint, and collective set of practices needs to be highlighted more
in the standards per se. To meet the requirements of Standard 5 teachers need skills in
conducting professional conversations about assessment, they need confidence in sharing
their practices with colleagues, and in participating in moderation meetings. Across time as
students progress up the grades, teachers need to discuss the cumulative effects of their
teaching and be able to plan whole school approaches to teaching and assessment.
14 | P a g e
1.2 Recognising the distinctive assessment requirements of different curricular domains
The AISTL standards are neutral (silent) with regard to the distinctive assessment
requirements of different curricular domains. This report has highlighted the constraining
effects of accountability pressures on curriculum coverage and emphasis. The current
context of heightened assessment pressures has led teachers, principals and systems to be
preoccupied with literacy and numeracy outcomes. The broad scope and importance of
learning across curricular domains needs to be re-emphasised and professional
development needs to be focussed on quality assessment practices across domains.
Paradoxically, it is in the broader and less scrutinised curriculum areas such the Arts and
Social Studies for example, that some exemplary assessment practices were reported and
where students were more challenged with complex tasks and invited to be more agentic
in self-assessment.
Principals and school leaders can help teachers learn to use classroom data well by:
2.1 Turning external accountability demands into opportunities for improving student
learning
The leadership of principals is crucial in establishing the culture of assessment within the
school. The principal’s role is situated strategically between the demands and expectations
of the schooling system and policy-makers on the one hand, and the actual circumstances
of classroom teachers and their students on the other. While tests such as NAPLAN will be
salient for principals in responding to the external demands of vertical accountability, the
effective principal has the capacity to interpret such test results realistically in the context
of their school, and to orchestrate whole school assessment and teaching practices that
consider the holistic education of students taking into account the full range of curricular
offerings. In short, the effective principal can tell the whole story of learning at the school
and identify where improvements need to be made.
Principals can direct material and personnel resources to improve the quality of assessment
within the school. For example, professional development programs can be planned to
increase teachers’ assessment literacy, and particular staff with assessment expertise can
be redeployed to coach groups of teachers in improving their assessment practices. The
principal can find ways to facilitate professional dialogues amongst teachers, create
opportunities for groups of teachers to moderate their assessment judgements, and
identify the learning needs of students across and within grades. A key task here is to
15 | P a g e
create a climate of trust where teachers are willing to share their “warts and all” stories
and seek better outcomes for their students.
2.3 Advocating for differentiation practices that raise expectations of students while
supporting their learning needs
Principals also are responsible for selecting assessment instruments that are used to assess
students’ prior learning when they arrive at the school and for the kinds of instructional
grouping practices that arise from such assessment instruments. Inflexible grouping based
on notions of inherent ability has a detrimental effect on the learning of students most in
need. But equally, a lack of understanding of students’ prior learning and their current
needs has negative consequences for students. The effective principal can advise teachers
on how to differentiate their teaching methods for the needs of different learners without
creating low expectations for some students.
3. For teachers:
Teachers can develop their capacity to use classroom data well by:
Teachers have the intensity of daily contact with their students and interact myriad times
with students across a school year. Through this process they come to understand students
and their learning needs in relation to the demands of the curriculum and external
accountability tests such as NAPLAN. Therefore assessment is not separate from classroom
teaching but embedded in everyday interactions. Taking time to reflect on this rich
information and to consider each student in the class on a regular basis is the foundation of
good assessment and teaching. Reflection on the class as a whole and on individuals within
the class is necessary if teachers are to provide effective learning experiences for students.
3.2 Planning learning opportunities for their students that are informed by interpretation
of assessment data
The challenge for teachers is to assess students in a manner that leads directly back into
more effective teaching and scaffolding of their students’ learning. Such “assessment for
learning” is difficult to sustain when external and vertical accountability pressures are high.
The effective teacher has the capacity to manage these pressures whilst maintaining a
focus on the breadth and depth of the overall learning journey of their students.
16 | P a g e
3.3 Designing assessment that provides students with useful feedback on their learning
Hattie and Timperley (2007) distinguished four levels of teacher feedback: (i) focus on task
per se; (ii) focus on how to process the task; (iii) focus on how to develop self-regulation
and self-assessment; (iv) focus on students and their “selves”. It is clear from this report
that too often students experience school assessment and feedback as powerful messages
about them and their capacities, rather than as useful information and support for
improving their capacities and developing self-regulation and self-assessment. The
development of an assessment culture for learning would banish feedback that limited
students’ expectations and decreased their sense of confidence in themselves as learners.
Good assessment can demonstrate to students that they are capable of more than they
expected – effective teachers know how to design such assessment tasks.
4.1 The QCT should disseminate the exemplars of good practice provided in this report as
interactive web resources that provide a focus for teacher professional learning
The report includes five exemplars of good practice that capture teachers’ impressions,
implementation strategies and outcomes, and evidence of success. The exemplars feature
the authentic voices of teachers, principals, and education officers, together with pictures
from real classrooms and a set of discussion questions that could be used to stimulate
teachers’ professional dialogue or individual reflection.
17 | P a g e
Introduction
Terms of reference
The Queensland College of Teachers (QCT) has indicated that from 2013 the Australian
Professional Standards for Teachers (the Standards) (AITSL, 2011), endorsed by all
Australian State and Territory Ministers for Education, will be used to underpin the
functions of the QCT. Table 1: Australian Professional Standards for Teachers outlines the
key areas of the Standards.
Domains Standards
of teaching
The Standards are a public statement that describes the professional knowledge,
professional practice, and professional engagement required of teachers. Standard 5, the
focus on assessing, providing feedback and reporting on student learning is the focus of this
report. This incorporates teachers’ understandings of how to interpret and use classroom
(and systemic) data to support and improve students’ learning, which is of particular
importance to teachers in an age of increasing accountabilities for student outcomes.
Schools and teachers now have access to a myriad of formal and informal data from both
external and internal sources. To maximise the potential to inform and generate improved
student outcomes teachers need to be inquiry-minded and data-literate.
18 | P a g e
The scope of this report, outlined by the QCT, includes:
Report structure
The report is structured into three sections: the literature, the descriptive accounts, and
the key features for improving teachers’ classroom practices related to data (see Figure 1:
Report structure).
Section 1:
Literature
Teachers
using
classroom
data
Section 2:
Descriptive
accounts
Section 3: Cases
outlining
improvement
strategies and
practices
The literature section is divided into four sub-parts that are comprised of themes related to
both systemic and classroom practices surrounding the use of data (see Figure 2: Themes of
the literature). The literature-derived themes relate to accountability, assessment literacy,
numerate teachers, and data literacy and practices, and reflect the emphases that were
identified through the literature review.
19 | P a g e
FIGURE 2: THEMES OF THE LITERATURE
DATA
These four themes as well as the analytic framework for this project are loosely related to
Standard 5. Table 2: Standards and analytic framework for this project outlines the sections
of Standard 5 with the elements of the analytic framework, within a matrix structure.
5.1 Assessing
student
learning
5.2 Providing
feedback to
students on
their learning
5.3 Making
consistent and
comparable
judgments
5.4
Interpreting
student data
5.5 Reporting
on student
achievement
20 | P a g e
Referring to the matrix columns, Accountabilities are clearly important and foregrounded
within the terms of reference, the literature and the school practices related to data use in
schools. The P-12 structure is taken into account in presenting different strategies that
teachers might deploy in assessment of and for learning, and in considering the different
professional contexts of early childhood, primary, and secondary school teaching. With
regard to Alignment and differentiation, the examples of assessment address the uses of
assessment for different purposes such as to group students for instruction, individualise
assistance, or create other types of adaptations to students’ different strengths and needs.
Students’ self-assessment is implicated in the strategies that teachers deploy and the goals
that they have for their teaching such as developing metacognitive skills and meta-learning
skills (learning how to learn). Finally, assessment using Challenging tasks is referred to in
the section on strategies under hybrid forms of teaching and assessment.
Data collection
The first data set in this report was collected through consultation interviews with senior
staff in various key organisational stakeholder groups, including educational authorities
from government, Catholic and Independent sectors in Queensland, to identify examples of
effective practice. Consultations also addressed issues arising from the literature review,
the current national and state educational policy context regarding student assessment and
the interpretation of assessment data, and expectations of how teachers might exemplify
the Australian Professional Standards for Teachers, with particular reference to Standard 5.
Another data set was collected through interviews with key informants, such as principals,
heads of departments, and teachers, during school visits across the three sectors. This data
set included the collection of artefacts and exemplars that demonstrate effective practice.
Additional data was gathered from public documents such as annual reports, and education
and school websites.
General titles will be used when referring to participants. For example, ‘education officer’
refers to individuals employed at a district or regional level within an education authority.
Some of these individuals are in roles other than those of education officers, but are
identified using the collective title. The sector (government, Catholic or Independent) has
not been identified for reasons of confidentiality. “Principal”, “head of department” and
“teacher” will be used to identify the participants’ role, but again, the sector will not be
mentioned. In total, there were five individuals identified with the title “education officer”,
two as “principal”, one as “head of department”, and eight as “teacher”.
21 | P a g e
Section 1: Literature
Literature review
The literature review is organised in three parts. Part A sets the context for the review by
introducing the notion of vertical and horizontal accountabilities and discussing conditions
that hinder or enable effective use of data to improve teaching practice. Part B shifts the
focus to teachers, and in particular how to develop teachers’ assessment literacy and data
numeracy. Part C identifies strategies for using data effectively and includes an illustrative
example of a mathematics education research and development project. Additionally,
Appendix A provides an annotated bibliography of a range of literature on data in schools.
Part A: Context
Politicians and policy makers frequently link school accountability with school
performativity (Newmann, King, & Rigdon, 1997). Often, such perceptions are linked to
business-derived models that favour strong external accountability, with incentives and
sanctions as a means of compelling schools to improve students’ learning outcomes.
Research by Newmann et al. (1997) found that:
22 | P a g e
strong internal accountability tended to reinforce a school's organizational capacity.
(Newmann et al., 1997, p. 41)
This competitive discourse regarding school and teacher accountability has gained further
traction in Australian schools since 2008 with the introduction of annual NAPLAN tests in
Years 3, 5, 7 and 9. NAPLAN is a whole population test rather than a sample test for
specific purposes, and as a high-stakes population test for schools and teachers, it has had
predictable negative effects on pedagogy, curriculum, assessment, and the way time is
allocated to teaching tasks across the year. Australia did not learn from international
experience. For example, reflecting on the USA assessment and testing context, Marsh,
Payne and Hamilton (2006) wrote in a review paper for RAND:
RAND’s research studies and others raise concerns about the consequences of
high-stakes state testing and excessive reliance on test data (e.g., Hamilton,
2003). While some responses to testing and test results … have the potential
to improve educational outcomes, others may be less productive, such as
increased time spent on test-taking strategies, increased focus on problem
styles and formats that appear on state tests, or targeting instruction on
“bubble kids”1 … Other concerns about emphasis on test results revolve around
the potential narrowing of instruction to the subject areas and content covered
1
See Text box 1: Bubble kids
23 | P a g e
on state tests. Finally, there is a risk of excessive testing, due to the addition of
progress tests and other assessments intended to prepare students for state
tests. (Marsh et al., 2006, p. 10, emphasis added)
The website for the National Assessment Program in Australia claims that “NAPLAN test
results provide an objective view of students’ performance”. This goal is currently
confounded in actual practice because NAPLAN tests have ceased being an objective
“litmus test” of how schools, classes and students are performing in some aspects of
literacy and numeracy, to being the objective per se. That is, parents, teachers, and
students are devoting substantial time and resources to simply performing well on the
kinds of items included in the tests. The tests have become the end and so distort the
objective function claimed. Extensive in-
class coaching by teachers on NAPLAN items
as well as directly teaching test-taking Bubble kids are those
heuristics and skills is being supplemented just below a certain level of
at home by an increasing number of parents proficiency. They are often the
coaching their children specifically on focus of intense teaching because
NAPLAN items. One of the top selling raising them above that proficiency
publications in Australia currently is a level has a very positive effect on
NAPLAN preparation booklet for parents. perceived effectiveness of the
These kinds of unintended consequences teacher. Other students in the
are raising levels of stress regarding testing class may be neglected somewhat.
taking and distorting classroom practices.
Datnow, Park, and Kennedy (2008)
Research by Hardy and Grootenboer (in suggest that such practices
press) into the implementation of NAPLAN “inadvertently compromise the
in three low socioeconomic status (SES) overarching purpose of data-
school communities in south-east driven practice” (p. 95).
Queensland: a rural school (approx. 350
students); a large urban school (almost 1000
TEXT BOX 1: BUBBLE KIDS
students); and a smaller urban school (340
students) revealed that teachers expended
considerable energy on NAPLAN, as they sought to improve test scores and achieve
genuinely educative purposes. These purposes, however, remained in tension; that is,
considerable effort and energy were expended by teachers as they grappled with the
political demands for increased test results, and educative concerns about how to provide
productive academic and social learning opportunities for students.
The Australian experience is part of a global trend. Vertical accountability procedures are
deployed almost everywhere. Professor Pasi Salhberg has critiqued this current orthodoxy,
termed GERM (global education reform movement). GERM attempts to promote reforms in
24 | P a g e
teaching and schooling through public pressure on teachers and principals, comparison of
education systems within and across nations, and through rewards and sanctions
distributed on the basis of test performance. As early as 2002, Earl and Katz wrote:
Large-scale assessment and testing has moved from being an instrument for
decision-making about students to being the lever for holding schools
accountable for results (Firestone et al., 1998). Leaders in states, districts and
schools are required to demonstrate their successes and many are creating
organisational report cards as a way of communicating with the public. (Earl &
Katz, 2002, p. 1004)
In their paper, Leading Schools in a Data Rich World, Earl and Katz (2002) examined a
number of school districts in England and Canada and summarised the dilemmas for
teachers and schools arising from high-stakes testing regimes in the following manner:
25 | P a g e
Finally, Earl and Katz (2002)
propose a forward looking Learning Commissions
accountability where data are
used not as a surveillance activity
A Learning Commission is a group of eight
but in the service of
community members who have a stake in
improvement. Overt
schooling. This group might include
accountability controls can create
students, parents, employers, educators,
a sense of urgency but, as Earl
school volunteers, community service
and Katz argue, accountability is
workers, and any other concerned
much more than providing a
members of the community.
ledger sheet or identifying the
The Commission provides an opportunity
“culprits”. Rather, real
for these people to share their knowledge
accountability is looking forward,
about schools and to gather knowledge
using data to inform judgements
and views from others.
about current performance, and
This evidence provides a basis for making
formulate plans for reasonable
recommendations for changes that could
actions.
help communities to understand and
support schools better and schools to
Horizontal support their students and communities in
accountabilities: Lessons more productive ways.
for classroom assessment The Commission meets once a month with
each meeting lasting a few hours. There
Assessment practices influenced by are two ways to participate in the Learning
horizontal accountabilities are Commission. First, you can submit an
designed to enhance learning per se Expression of Interest to sit on the
(assessment for learning), and to Commission. These will be considered by
address the locally situated needs the Chair, who may ask to hear more about
and aspirations of students, their why you would like to be involved. Second,
families, and communities. Horizontal you can submit an Expression of Interest to
accountability is seen as a fairer share your views with the Commission.
method of auditing the performance This would involve attending one meeting
of teachers and schools because it to talk with the Commissioners, providing
foregrounds the local conditions and you with an opportunity to contribute to
challenges in the community and the the findings of the Commission.
school. Additionally, it broadens the SOURCE: Media release, Dr Sam Sellar,
scope of what is considered PETRA project (Pursuing Equity Through
worthwhile learning from a narrow Rich Accountabilities)
focus on basic skills in a few key
curriculum areas to a range of
outcomes that include measures of TEXT BOX 2: LEARNING COMMISSIONS
student well-being, learning beyond
the classroom and the school, and a concern to build networks and social capital with local
26 | P a g e
community members and groups so that school is not divorced from community life.
Consequently, as Sahlberg (2007) suggests, horizontal accountability “preserves and
enhances trust among teachers, students, school leaders and education authorities in the
accountability processes and involves them in the process, offering them a strong sense of
professional responsibility and initiative” (p. 155).
Within this growing global trend that focuses on accountability, the challenge for educators
and educational institutions is to understand the purposes of accountability testing.
Stiggins (2002) suggests:
Politicians routinely ask, How can we use assessment as the basis for doling out
rewards and punishments to increase teacher and student effort? They want
to know how we can intensify the intimidation associated with annual testing
so as to force greater achievement. (p. 758)
National testing, such as NAPLAN, provides a measure for educators that can be viewed as
a reference point, and should be used to improve student learning (Klenowski, 2011). This
can become difficult in the face of national comparison testing that is publically available in
the media, such as league tables, as there is the potential to pervert the system in order to
maintain a particular position on these ranks (Stobart, 2008). A recent review of the effects
of NAPLAN by Greg Thompson (2012) suggest there is the potential for such high-stakes
tests to narrow the curriculum, cause a fixation on basic skills rather than higher-order
thinking, focus on content rather than the investigative processes, and increase time
pressures on teachers and schools. This, and other research, indicates that such testing
becomes high-stakes as it can potentially modify teachers’ curriculum and pedagogy
practices (Darling-Hammond, 2010; Klenowski, 2011; Sahlberg, 2010; Stobart, 2008) and
generate a shift to teacher-centred classrooms (Wildy, 2004). Additionally, Stiggins (2002)
makes the following observation about resources and standardised tests in US schools:
27 | P a g e
Student achievement suffers because these once-a-year tests are incapable of
providing teachers with the moment-to-moment and day-to-day information
about student achievement that they need to make crucial instructional
decisions. Teachers must rely on classroom assessment to do this. The problem
is that teachers are unable to gather or effectively use dependable information
on student achievement each day because of the drain of resources for
excessive standardized testing. (p. 759)
Wildy (2004) refers to the work of Frederiksen and White who suggest there are two
considerations in relation to student assessment and accountability: student outcomes and
the processes used to obtain those outcomes. The research identifies the key constructs of
directness and transparency to elaborate on these considerations (Wildy, 2004):
Williams and Ryan (2000) shift the debate regarding improving student learning outcomes
to beyond the notion of accountability, suggesting that student assessment data from
national tests can help improve student learning outcomes. For example, they suggest that
students’ errors and misconceptions, in particular in mathematics testing, can help teachers
28 | P a g e
effectively design diagnostic mathematics teaching strategies (Williams & Ryan, 2000).
However, they also indicate that many teachers do not use diagnostic methods for
improving classroom practice (Williams & Ryan, 2000, p. 50). Additionally, they suggest
teachers predominantly associate assessment with summative testing (Williams & Ryan,
2000, p. 50), thereby missing opportunities in diagnostic testing. They also review previous
research that indicates that “genuinely valid summative assessment should fully reflect the
curriculum it assesses and so be formative as well” (Williams & Ryan, 2000, p. 52). When
quoting Gipps (1994), it is suggested education systems should move from a “testing
culture” to an “assessment culture”, where the tests become an opportunity for teachers
to improve the pedagogy (Williams & Ryan, 2000) (also see Theme 2).
Research by Ingram et al. (2004) has identified seven barriers to the effective use of data in
schools. These are divided into cultural challenges, technical challenges, and political
challenges:
Cultural challenges: culture is considered a strong determinant in how teachers use data to
make judgements, and in determining the type of data that teachers think is needed.
Barrier 1: Many teachers have developed their own personal metric for judging
the effectiveness of their teaching and often this metric differs from the
metrics of external parties (e.g., state accountability systems and school
boards).
Technical challenges: technical factors that affect the use of data include:
Barrier 5: Data that teachers want are rarely available and refer to
characteristics of student learning that are hard to measure.
Barrier 6: Schools rarely provide the time needed to collect and analyse data.
(Ingram et al., 2004, p. 1281)
29 | P a g e
Political challenges: the inherently political nature of educational systems results in
difficulties in terms of using data for decision-making.
Barrier 7: Data have often been used politically, leading to mistrust of data and
data avoidance. (Ingram et al., 2004, p. 1282)
Additionally, this last barrier, related to mistrust of data, was explained by teachers in this
study through the sense that the data could be misused or “used as a tool to force a
decision that has already been made rather than as information to shape a decision”
(Ingram et al., 2004, p. 1276). This highlights the concern that often school norms are not
consistent with the intent of accountability policies (Ingram et al., 2004).
Wayman, Cho, and Johnston (2007) have contextualised aspects of data collection,
interpretation, and analysis in schools:
For years, educational entities have collected data on school process and
student learning. Recent accountability policies have brought public attention
to these data, increased the amount of data collected, and tied funding to
certain characteristics of these data. Consequently, educators respond to
reporting requirements while simultaneously struggling with better ways to
understand these data internally to improve practice. (p. 2)
Young (2006) adds that government accountability policies place “tremendous faith in the
power of data – especially standardised test data – to effect school improvement” (p. 521).
Based on their district-wide evaluation of a US county, Wayman et al. (2007) made the
following recommendations regarding data in education systems:
While appearing “commonsensical”, these roles or practices within an organisation are able
to help support teachers with using classroom data for improved teaching and developing
assessment literacy.
31 | P a g e
Part B: Focus on teachers
Two themes emerged from consideration of the literature on teacher’s roles in assessment.
The first theme centres on what is commonly referred to as teachers’ “assessment
literacy”, including their ability to select appropriate assessment techniques, to assess
students’ learning for both summative and formative purposes, to provide feedback to
students on their learning, and to use effective strategies for making consistent and
comparable assessment judgments. The second theme concerns teachers’ “data numeracy”
that is their ability to interpret student data in order to modify their teaching so as to
improve student learning. These two themes are thus closely associated with most of the
elements of Standard 5 in the Australian Professional Standards for Teachers (Assess,
provide feedback and report on student learning; see Tables 1 and 2).
Assessment literacy amongst teachers provides the basis for productive examination of
data, where teachers make critical sense of it, develop action plans based on the data, take
action, and monitor progress along the way (Fullan, 2001). Recent local research in
Queensland has highlighted the importance of the history of externally moderated school-
based assessment in the senior secondary years of schooling. It seems to have promoted
substantial assessment literacy at least amongst teachers in the senior secondary school.
Assessment literacy is associated with an assessment culture in the school that encourages
professional dialogue and a common language for discussing and analysing the
characteristics of good assessment practice. However, Phase 1 evaluation of the
Queensland Curriculum, Assessment and Reporting Framework (QCAR)(Goos et al., 2008)
found lower levels of familiarity with and confidence in implementing good assessment
amongst primary and lower secondary school teachers.
What assessment tasks do teachers use? A sample of Year 4, 6, 8, and 9 teachers who
responded to a survey on assessment practice claimed that their most commonly used
assessment techniques include:
The first three of these are more formal assessment tasks that may contribute to a
summative judgement of the quality of students’ work, while the last two are informal
techniques that teachers use to monitor students’ developing understanding and adjust
their teaching strategies in order to improve learning.
32 | P a g e
These five assessment techniques were also judged by more than half the teachers to
provide quality evidence of students’ learning. Other techniques that met this requirement
were:
Student reflective writing and student self-assessment were techniques that were judged
to provide quality evidence but were rarely used by teachers. That these additional
techniques were not used more frequently may reflect a perception by teachers that they
are too time-consuming to implement as a part of day-to-day classroom activities, despite
the quality evidence that they may provide and the role of these techniques in helping
students gauge their own progress. This temporal factor of feeling “time-poor” is part of
the current audit culture and needs to be reflected upon by school leaders and addressed if
quality pedagogy and assessment is to be enacted in classrooms.
Teachers who responded to the survey claimed they most commonly used an assessment
rubric, criteria sheet, or standards scheme to judge the quality of students’ work.
Interestingly, this response was not borne out by data collected in the schools visited by the
research team. Teachers in these schools seldom provided criteria sheets to accompany the
assessment tasks that were collected as exemplars. Surveyed teachers reported that
common assessment tasks and assessment rubrics were the most frequently used
strategies for achieving consistency and comparability of judgements. Moderation
meetings were seldom used, even though teachers indicated they would like to use this
strategy.
Three years after this baseline evaluation, a follow up evaluation found evidence that QCAR
had improved the consistency of assessment across classrooms (Mills et al., 2012). This was
largely due to the increased use of criteria sheets in primary schools and the use of
consistent terminology for describing assessment criteria and standards in both primary
and secondary schools. Interviews with principals and teachers indicated that QCAR had
worked to improve the comparability of assessment judgments within and across schools.
Moderation appears to have been the most significant factor in contributing to this view,
with most schools now reporting that moderation meetings were commonplace.
Traditionally, there have been two distinct practices: the activity of instruction and the
activity of assessment, with the two practices rarely coinciding (Even, 2005, p. 46). Shepard
(2000) refers to the work of Beth Graue who suggested "assessment and instruction are
often conceived as curiously separate in both time and purpose" (p. 4). However, research
suggests there is benefit in combining the two practices, in that formative assessment,
especially the feedback associated with such assessments, can improve student learning
outcomes (Black, Harrison, Lee, Marshall, & Wiliam, 2004; Black & Wiliam, 1998; Hattie &
Timperley, 2007; Hill, 2011; Kirkup, 2006; Quint, Sepanik, & Smith, 2008). Black and Wiliam
(1998) identify formative assessment as that where the evidence from the assessment is
used by teachers to “adapt their teaching to meet student needs” (p. 140). An OECD (2005)
report, Formative assessment: Improving learning in secondary classrooms, indicated the
following practices were successful in formative assessment:
However, the same report identified barriers to the practices of formative assessment.
These included the following:
Black and Wiliam (1998) also identified a number of barriers, suggesting there is a “poverty
of practice” in relation to assessment practices:
While marking is usually conscientious, it often fails to offer guidance on how work
can be improved.
Often teachers only pay lip service to [formative assessment] but consider the
practice as unrealistic. (Black & Wiliam, 1998, p. 141)
Additionally, their research suggests that assessment practices are often linked with three
difficulties (Black & Wiliam, 1998):
Effective learning:
The tests used by teachers encourage rote and superficial learning even when
teachers say they want to develop understanding; many teachers seem unaware of
the inconsistency.
The questions and other methods teachers use are not shared with other teachers
in the same school, and they are not critically reviewed in relation to what they
actually assess.
For primary teachers particularly, there is a tendency to emphasize quantity and
presentation of work and to neglect its quality in relation to learning.
Negative impact:
The giving of marks and the grading function are overemphasized, while the giving
of useful advice and the learning function are underemphasized.
Approaches are used in which pupils are compared with one another, the prime
purpose of which seems to them to be competition rather than personal
improvement; in consequence, assessment feedback teaches low-achieving pupils
that they lack "ability," causing them to come to believe that they are not able to
learn.
Managerial role of assessments:
Teachers' feedback to pupils seems to serve social and managerial functions, often
at the expense of the learning function.
Teachers are often able to predict pupils' results on external tests because their
own tests imitate them, but at the same time teachers know too little about their
pupils' learning needs.
The collection of marks to fill in records is given higher priority than the analysis of
pupils' work to discern learning needs; furthermore, some teachers pay no attention
35 | P a g e
to the assessment records of their pupils' previous teachers. (Black & Wiliam, 1998,
pp. 141-142)
Assessment feedback
Other research suggests the format of this feedback is very important (Black et al., 2004;
Kirkup, 2006). Black et al. (2004) note that, “while student learning can be advanced by
feedback through comments, the giving of numerical scores or grades has a negative effect,
in that students ignore comments when marks are also given” (p. 13). Here, the suggestion
is that individualised feedback is more effective than numerical data. Overall, whether
feedback is given via grades or commentary, the suggestion is that it should cause
“thinking” to take place (Black et al., 2004, p. 14). Peer and self-assessment is one way the
authors suggest this can occur, as such strategies enable student to become familiar with
their learning goals. Additionally, there is the understanding that summative testing should
become a positive part of the learning process and one way of accomplishing this is to use
summative tests for formative purposes. Black et al. (2004) suggest:
Students can be engaged in a reflective review of the work they have done to
enable them to plan their revision effectively.
Students can be encouraged to set questions and mark answers so as to gain an
understanding of the assessment process and further refine their efforts for
improvement.
Students should be encouraged through peer assessment and self-assessment to
apply criteria to help them understand how their work might be improved. This
may include providing opportunities for students to rework examination answers
in class. (Black et al., 2004, p. 16)
They argue that FS is the least effective level of feedback, while FR and FP are powerful as
tools of “deep processing and mastery of tasks”, while FT is “powerful when the task
information subsequently is useful for improving strategy processing or enhancing self-
regulation” (Hattie & Timperley, 2007, pp. 90-91).
36 | P a g e
Kirkup (2006) reviewed previous research that suggests formative assessment practices
have been devalued as a consequence of the dominance of external, summative, high-
stakes testing programs and that these programs have a negative impact on students’
motivation for learning (see Theme 1). Hattie and Jaeger (1998) suggest that the value of
testing resides with the degree of feedback it can provide teachers and students so that
they can review their practices. However, Hattie and Jaeger (1998) also observe that
testing is often not used as a mechanism for feedback, but rather a measurement of
change.
FEEDBACK SYSTEMS
Some research suggests there is a need for formative feedback systems that identify the
“networks of structures, people, and practices that help teachers and administrators
translate testing data into practical information for everyday use” (Halverson, 2010, p.
131). Halverson (2010) indicates, “data-driven instructional improvement relies on
developing coherent systems that allow school staff to generate, interpret, and act upon
quality formative information on students and school programs” (p. 130). The model is
based on a system of feedback where loops of instruction, assessment and actuation allow
practices to be documented which in turn, allow for further understanding on how to
organise feedback systems within schools (Halverson, 2010, p. 145).
Authentic assessment
Darling-Hammond, Ancess, and Falk (1995) suggest that schools should aim to develop
‘authentic’ assessments where intellectual and practical skills are “transferable” to “real-
life” social settings and work environments. While Hargreaves, Earl, and Schmidt (2002)
question the notion of “authentic experiences”, they do acknowledge that many of the
“newer” assessment techniques provide powerful and productive learning experiences for
students (p. 70). Using the Canadian context, they identify the new curriculum policies as
being associated with the three closely related components of outcomes, integrated
curriculum, and assessment. This has similarities to the Queensland-based Productive
Pedagogies that use the three message systems of pedagogy, curriculum, and assessment
alignment (Hayes et al., 2006).
The “new” curriculum is intended to motivate students to take greater responsibility for
their learning, and to make assessment an integral part of the learning experience by
encouraging students to create and apply rather than memorise and focus on basic skills
(Hargreaves et al., 2002, p. 70). For example, a “new” assessment technique was identified
as the “portfolio-based assessment” (Hargreaves et al., 2002). It was reported that many
teachers liked the portfolios as they assisted students in developing independence by
setting up their own learning plan (Hargreaves et al., 2002, p. 78). However, Hargreaves et
al. (2002), using a postmodern analytic perspective, warned that “authentic” assessments
“simulate reality as much as they create it, producing beautiful ‘fakes’" (p. 89). They add:
37 | P a g e
Perhaps few things are more contrived and less authentic than authentic
assessment, where there is a constant sorting, sifting, and reflecting on one's
achievements in a portfolio, assessing one's peers using complex grids of
criteria, or engaging in stage-managed three-way interviews with parents and
students. Little could be more artificial or manufactured than this. (Hargreaves
et al., 2002, pp. 89-90)
A New Zealand study has highlighted the importance of student engagement in self and
peer assessment as “authentic ways in which students can develop evaluative and
productive knowledge and expertise, necessary prerequisites if they are to become
autonomous learners” (Dixon, Hawe, & Parr, 2011, p. 365). These student-centred practices
are grounded in principles of assessment for learning that foreground student autonomy
and agency (Dixon et al., 2011) and develop students’ self-monitoring of their learning.
However, research by Dixon et al. (2011) suggests there is significant variability and
inconsistency in both the uptake and the implementation of assessment reforms, as
teachers’ beliefs are mediating factors in this context (p. 365).
Without a crystal clear vision of the meaning of academic success and without
the ability to translate that vision into high-quality assessments … we would
remain unable to assist students in attaining higher levels of academic
achievement. (p. 238)
He adds that in order to develop teachers with high levels of assessment literacy, there
needs to be time for professional development (see Theme 3) and time to integrate the
ideas learned into instruction (Stiggins, 1995, p. 243).
Within the theme of assessment literacy, as well as notions of formative assessment and
authentic assessment, assessment for learning is a related concept. However, Swaffield
(2011) is quick to point out that assessment for learning is not synonymous with formative
assessment, highlighting points of difference:
Assessment for learning is a learning and teaching process, while formative assessment
is a purpose and some argue a function of certain assessments;
Assessment for learning is concerned with the immediate and near future, while
formative assessment can have a very long time span;
The protagonists and beneficiaries of assessment for learning are the particular pupils
and teacher in the specific classroom (or learning environment), while formative
38 | P a g e
assessment can involve and be of use to other teachers, pupils and other people in
different settings;
In assessment for learning pupils exercise agency and autonomy, while in formative
assessment they can be passive recipients of teachers’ decisions and actions;
Assessment for learning is a learning process in itself, while formative assessment
provides information to guide future learning; and
Assessment for learning is concerned with learning how to learn as well as specific
learning intentions, while formative assessment concentrates on curriculum objectives.
(Swaffield, 2011, p. 443)
Additionally, he uses the term “authentic assessment for learning”, but is not using the
word authentic in the same way as authors in the previous section, that is, he is not
implying “real life” experiences, but rather is focused on the student and teacher
participants in the assessment process. Swaffield (2011) identifies the characteristics of
assessment for learning as focusing on learning; conceptualising the learning objectives;
and a focus on roles and relationships (Swaffield, 2011, pp. 437-441).
Stiggins (2002) suggests that there is an assessment crisis in the US, as a result of a flawed
vision that focuses on informing decisions of policy makers, giving weight to meeting their
information needs, rather than those of teachers and students (p. 760). Instead,
assessment should be for learning, where the flow of information about student
achievement is used by teachers not only to check on student learning, but also to improve
it by:
39 | P a g e
actively involving students in communicating with their teacher and their families about
their achievement status and improvement. (Stiggins, 2002, pp. 761-762)
Hill (2011) indicates that secondary teachers are more deeply situated in a performativity
context than primary teachers and the micropolitics of the secondary school subject
departments is more resistant to change (p. 359). Additionally, Black and Wiliam (1998)
suggest, “There is no quick fix that can alter existing practice by promising rapid rewards”
(p. 146). Hill (2011) reviews research on strategies that use cycles of ‘planning, action and
review’, suggesting that teachers need to understand how assessment for learning is able
to improve learning (p. 350) (also see Theme 3). To achieve this, Hill (2011) suggested,
The principal and senior management team’s role was to work with the
assessment-literate Assessment for Learning facilitators to decide and plan
how best to engage the teachers in this learning … [and] school systematically
built in ways for teachers to spend time on professional learning. These
included literature study groups, action research teams who tried out new
strategies and reported back on their success or otherwise, and other inquiry
activities within and across schools. (p. 360)
Shepard (2000) conceives such processes, including assessment, as part of the learning
process and as developing a ‘learning culture’. Shepard (2000) also argues that assessment
should:
be dynamic and on-going, so that assessment is moved to the middle of the teaching
and learning process, rather than being postponed to the end;
establish students’ prior knowledge through checklists or pre-tests;
provide effective feedback;
reveal whether students are able to transfer and use new knowledge in new situations;
make use of explicit criteria where students can learn to evaluate their own work;
include self-assessment and increased student responsibility for learning; and
support evaluation of teaching practices. (pp. 10-12)
40 | P a g e
Theme 3: Data and numerate teachers
Matters (2006) suggests, “Data shape the landscape of our [teachers’] professional lives. Or
if they don’t, they should, given that education is a profession” (p. 7). Despite this
pronouncement, there is a tendency amongst teachers to distrust data, in particular
statistical data (Ingram et al., 2004; Matters, 2006). As well as a distrust of data, Wu (2010)
suggests that too often the “statistical complexity [of the data] prevents the non-technical
stakeholders from fully appreciating the caveats in the results, leading to misinterpretation,
over-interpretation and even worse, making inappropriate policy decisions” (p. 24).
Additionally, a recent US government report indicates little change as these data are having
little effect on teachers’ daily instructional decision making (Means, Padilla, DeBarger, &
Bakia, 2009, p. viii). Therefore, it is important that teachers develop their data knowledge
and develop as numerate teachers.
Numerate teachers as a theme is also relevant to the current context where data-driven
decision making within schools and classrooms is more and more prevalent. There is an
extensive literature on this approach that will be further reviewed.
Data-driven systems
Supovitz and Klein (2003) suggest that while students produce a large volume of work every
school year, “only a fraction of those data are mined for instructional guidance” (p. 13). The
purpose of classroom data should therefore be understood as a means of diagnosing
problems and analysing solutions rather than simply being about the evaluation of students
(Bedwell, 2004, p. 19). As part of daily classroom practice, Bedwell (2004) however, notes,
41 | P a g e
“What is needed is for teachers to become more systematic in the collection, analysis, and
interpretation of data in order to facilitate high-quality decisions more consistently” (p. 9).
Datnow et al. (2008) agrees, suggesting the “’data-driven’ teacher uses formative
assessment data on a regular basis to make adjustments to his or her instructional plan”
(Datnow et al., 2008, p. 5). Additionally, research suggests that “high-performing schools
and school systems use student data in all facets of their work to continuously inform and
improve their instruction” (Datnow et al., 2008, p. 5).
A US study by Supovitz and Klein (2003) indicated that while school leaders were of the
opinion that both external and internal data were useful, the internal data were of greater
value in relation to the provision of instructional guidance (pp. 12-13). Table 3: Sources and
uses of student performance data, outlines examples on external and internal data.
Running Records
Providing cross-grade/subject guidance
throughout the school year
Theme Tests
School-wide Refining professional development
Refining assistance plans for low-performing
Uniform Writing students
Assessments
Examples
Reinforcing culture of inquiry
Grades
Portfolios
Providing quick and flexible feedback throughout
the year
Individual Writing Folders
Teacher Allowing for opportunistic adjustments in
instruction and targeted assistance
Conference Logs Individualized to particular style and needs of
Assessments classroom teacher
Reading & Writing
Journals
42 | P a g e
Additionally, the findings in the study by Supovitz and Klein (2003) indicated that in
effective school data systems, the interplay between these three data sources creates an
interwoven evidence base. The external, school-wide, and individual teacher data sources
are equally significant and mutually reinforced. Figure 3: Interplay of the three data sources
in effective school data systems, demonstrates the nature of such systems, showing each
data source as the same sized circle, and each circle overlapping.
FIGURE 3: INTERPLAY OF THE THREE DATA SOURCES IN EFFECTIVE SCHOOL DATA SYSTEMS
External School-
wide
Individual
teacher
43 | P a g e
2. other student data relevant to the students such as student mobility, attendance
data, behavioural incident data and homework completion; and
3. contextual data that are not under the direct control of the teacher (such as
students’ linguistic background, gender or community socio-economic factors) but
are important to consider when planning for improved student achievement (van
Barneveld, 2008).
Evidence-based decision making (Hattie, 2012; Tozer & Holmes, 2005), evidence-informed
teaching decisions (Timperley & Parr, 2004), data-driven instruction (Bedwell, 2004), data-
based decision making (Feldman & Tung, 2001; Schildkamp, Lai, & Earl, 2013), and data-
informed decision making (Means et al., 2009; Pettit, 2010; Wayman et al., 2007) are all
concepts identified in the research literature to describe the systematic use of evidence by
teachers, schools, and districts to improve instruction, student learning, and other school
practices. Timperley and Parr (2004) identify five key principles for evidence-informed
teaching (see Table 4: Five principles for Evidence-Informed Teaching).
Principles
Principle 1: The process of making teaching decisions involves a consideration of both the
curriculum and the evidence of student achievement with regard to specific learning
intentions.
Principle 2: The focus should be on the effectiveness of day-to-day teaching activities, not
additional programmes.
Principle 3: The collection of evidence needs to be ongoing and should be used both to
identify student needs and to monitor the effectiveness of the intervention.
Principle 4: Best practice is established by examining the data by class and finding the
most successful teacher or teachers who can then assist others.
44 | P a g e
As the goal is to improve student learning, Kennedy and Datnow (2011) suggest that
student voice needs to be included in the process. They identify a three-tiered typology of
student engagement in data-driven decision making (see Figure 4: Three-tiered typology of
student engagement and data-driven decision making).
Tier 3:
Engaging
students in
data use
Tier 2: Using
data to assess
student
engagement in
learning
Tier 1:
Students’
active, dialogic
involvement in
DDDM reform
efforts
Tier 3 was the most evident typology of student engagement while Tier 1 had the most
potential for effective interactions. Kennedy and Datnow (2011) add:
If students are active and the endeavor seeks to build the school community,
the interactions are dialogic. Students and teachers speak and listen to each
other; student narratives and needs become integrated into the organization.
As school personnel listen to students, they better understand how students
learn, what students need, and how the organization can better respond. (p.
1251)
In terms of improving student learning, there is strong support within the literature for
school systems to have a cycle-based focus where “performance data is constantly
gathered, shared, analysed, and used to inform what is taught and how it is taught”
(Barnes, 2004; Datnow, Park, & Wohlstetter, 2007, p. 18). The Annenberg Institute for
School Reform (Barnes, 2004) developed a continuous, non-linear inquiry process
comprised of six essential activities (see Figure 5: The Inquiry Cycle – A philosophy of
continuous improvement).
45 | P a g e
FIGURE 5: THE INQUIRY CYCLE – A PHILOSOPHY OF CONTINUOUS IMPROVEMENT
Identify desired
student
achievement
outcomes
Evaluate
Develop
impact on
essential
practice and
questions
outcomes
THE
INQUIRY
CYCLE
Identify,
Choose and
collect, and
implement
organise
actions
relevant data
Analyse data
Another example of a three stage inquiry cycle was developed by Data Wise project from
the Harvard Graduate School of Education (Boudett et al., 2006). This cycle of inquiry is
summarised in Figure 6: Data Wise Improvement Process and includes teachers to prepare,
inquire, and act.
46 | P a g e
FIGURE 6: DATA WISE IMPROVEMENT PROCESS
4. Dig into
3. Create data
student
overview
data
7. Plan
8. Act and assess to assess
progress
Other examples of inquiry cycles are referred to in the literature, for example, Timperley et
al. (2007) refer to a cycle of teacher inquiry and knowledge-building to promote valued
student outcomes (p. xliii). What is interesting about this cycle is the inclusion of the
teachers’ own learning needs, as part of the process (also see professional learning
section).
Datnow et al. (2007) identify the following key strategies as useful to sit alongside the cycle
of inquiry and enable educators with their use of data. These four strategies are:
47 | P a g e
The benefits associated with data-
informed decision making are Data teams
expressed by Datnow et al. (2008)
as increasing a school’s ability to This is an example of a strategy used by data-
become a learning organisation by: driven schools to develop teachers’ confidence
and competency with the use of data.
1. informing instructional
strategies; The data teams were comprised of volunteer
2. helping to set and refine goals; “teachers and administrators who engage in
3. fostering a culture of inquiry, site-based data collection and research for the
reinforcing school priorities, purpose of supporting standards-based
and aiding communication instruction in the classroom and data-driven
between stakeholders; and decision making” (Datnow et al., 2008, p. 24).
4. improving teachers’
expectations of their The responsibilities of this team included:
students.(Datnow et al., 2008)
Create a mind-set that decisions are made
In particular, the researchers on data, not instinct;
suggest that while school goals Offer professional development for
could include aspects of national principals and teachers on the effective use
targets and standards, schools of data;
should also develop their own Ensure that regular formative and
goals. Additionally, there is a focus summative assessments are given to
on supporting teachers within monitor student progress and to facilitate
schools to become confident and the adjustment of instruction and curricular
numerate in their use of data. As an programs as necessary;
example, “data teams” are a Ensure that an individual student’s data will
strategy used in schools to support remain confidential and only be used for
teachers (see Text box 3: Data purposes of planning instruction and
teams). Hattie (2012) communication with the child’s parents or
conceptualises such data teams guardians; and
within a four-step model of a Fostering a culture of inquiry that supports
repeating cycle: the use of data at all levels leading to a
culture of continuous improvement.
1. Collecting and charting the
data, with the aim of making SOURCE: Datnow et al. (2008, pp. 24-25)
the data visible;
2. Use the evidence to
prioritise and set, review, TEXT BOX 3: DATA TEAMS
and revise goals;
3. Reviews instructional strategies and how they impact on students, what needs to
change, and using the data as ‘indicators’ to make mid-course corrections; and
48 | P a g e
4. Monitor the impact of the strategies on students and student learning. (p. 61)
Leadership
The role of the leader is to model data use, but also to establish the conditions that support
and encourage teachers’ use of data and data systems (Wayman, 2005), in particular,
guiding the process of data investigation that results in improved student outcomes (van
Barneveld, 2008). Stiggins and Duke (2008) outline 10 leadership competencies related to
assessment (see Table 5: 10 Leadership competencies in assessment).
49 | P a g e
TABLE 5: 10 LEADERSHIP COMPETENCIES IN ASSESSMENT
Understands the principles of assessment for (that is, used in support of) learning
and works with staff to integrate them into classroom instruction
Knows and can evaluate the teacher’s classroom assessment competencies and
help teachers learn to assess accurately and use the results productively
Creates the conditions necessary for the appropriate use and reporting of student
achievement information, and can communicate effectively with all members of
the school community about student assessment results and their relationship to
improving curriculum and instruction
Understands the standards of quality for student assessments and how to verify
their use in their school/district assessments
Understands the issues related to the unethical and inappropriate use of student
assessment and protects students and staff from such misuse
50 | P a g e
Professional learning
51 | P a g e
the data, are most effective.
Professional judgement
52 | P a g e
Part C: Strategies
This final section of the literature review deals with strategies for sourcing and interpreting
data that can inform teaching.
Kirkup, Sizmur, Sturman, and Lewis (2005) identify the following as examples of how the
analysis of data can be used to:
Based on their study, they add that there are a number of perceived negative outcomes of
the use of data:
Data can easily translate into numerical targets that are, in themselves,
meaningless. Numerical data only becomes meaningful if it serves to pose
questions about the actual learning that is (or isn’t) taking place and how it can
be developed further. (Kirkup et al., 2005, p. 49)
Love (2004) approaches the topic of using data from a macro level suggesting there should
be less emphasis on some uses and more on others, as summarised in Table 6: Emphasis
placed on data use.
Data sources
There are endless ways in which data sources can be categorised. For example, Armstrong
and Anthes (2001) divide data sources into:
53 | P a g e
Demographic data: background information,
Achievement data: including students’ results, and
Instructional process data: related to the curriculum. (p. 1)
54 | P a g e
lessons
teachers discussing work in progress with students
teachers listening to student oral presentations
teachers observing and judging practical work
C. Data from the hybrid process of teaching and assessment might be generated by:
rich tasks
QCATS
challenging tasks with scaffolded support
use of the negotiated curriculum model – What do you know? What do you want to
know? How can you find out? How can you show others your new knowledge?
However these examples are categorised, such data can assist schools with developing
school improvement plans to monitor progress and set targets (see Text box 6: School
improvement plans).
Various reports have focused on the use of data from numeracy and literacy testing
(Northern Territory Government, 2012; Queensland Studies Authority, 2012). These reports
are aimed at teachers to help them understand and use the NAPLAN data. The Queensland
Studies Authority (2012) suggests schools should use NAPLAN data within the context of
existing school assessment data, indicating that formal and informal school testing results
should be consistent with NAPLAN results (p. 2). Additionally, the Queensland Studies
Authority (QSA) encourages the use of analysis software, SunLANDA, to analyse student
performance of the NAPLAN tests (see Figure 7: SunLANDA data analysis).
55 | P a g e
FIGURE 7: SUNLANDA DATA ANALYSIS
SOURCE: http://www.qsa.qld.edu.au/downloads/p_10/sunlanda_manual.pdf
The report suggests that NAPLAN data can be analysed in the following ways: using the
marking keys and item descriptions in the school NAPLAN report; using test results for
individual students; using test results for groups of students; and using test results for
school planning (Queensland Studies Authority, 2012, p. 14). The SunLANDA handbook also
provides information regarding the analysis of NAPLAN data using this software.
The Northern Territory government report also promotes the use of analysis software, RAAD
(Reporting and Analysing Achievement Data), to analyse NAPLAN data. Figure 8: Reporting and
Analysing Achievement Data sample demonstrates a graphical analysis of the Year 5 reading data.
56 | P a g e
FIGURE 8: REPORTING AND ANALYSING ACHIEVEMENT DATA SAMPLE
Additionally, the Northern Territory government suggests teachers use a “data sources and
question matrix” (see Table 7: Data sources and questions matrix) to identify responses to
three driving questions that enables a significant analysis of the data.
As well as literature that demonstrates how NAPLAN data can be analysed, Klenowski and
Funnell (2013) indicate that the “analyses of NAPLAN data alone are limited and insufficient
to understand how to address issues of equity and more specifically, how to increase
Indigenous students' participation in learning” (p. 35).
57 | P a g e
TABLE 7: DATA SOURCES AND QUESTIONS MATRIX
Australian
Curriculum -
Teacher
observations, tests,
projects,
performances, tasks
etc.
Commercial test
results
Students’ learning
logs, reflections and
self-assessment
rubrics
58 | P a g e
A mathematics-based example
This two-year project involved teachers meeting together eight times for professional
development workshops. The professional development workshops were framed around
research imperatives in the field in relation to teacher knowledge for teaching proportional
reasoning: a deep understanding of multiplicative concepts; the capacity to use multiple
representations and experiences to develop associated concepts; the ability to distinguish
between and characterise additive and multiplicative reasoning; provision of a range of
contexts in which students may reason multiplicatively; and understanding that
59 | P a g e
proportional reasoning is developmental but is enhanced through deliberate and
appropriate, conceptual-based teaching and learning approaches. While measurement of
student performance and consideration of students’ reasoning have been suggested as
important aspects of planning professional development, there is a lack of documented
evidence on the effect of teacher professional development on student learning outcomes
(Osborne, Simon, Christodoulou, Howell-Richardson, & Richardson, 2013; Sowder, 2007;
Watson & Beswick, 2011).The design of the professional learning adhered to principles of
effective teacher professional development that includes measures of student
performance, providing teachers with information about their students’ reasoning; allowing
teachers time to reflect on their students’ learning needs; consulting teachers about their
perceived needs; collaboration between teachers and mentors or experts; acknowledging
teachers’ local context in the design of professional development; presenting teachers with
a range of information, including theoretical aspects; and providing teachers with
opportunities to reflect on their own practice (Sowder, 2007).
At the second teacher workshop in the first year of the project, data from the assessment
instrument were presented and discussed. Data showed that many students were often
able to employ algorithmic approaches to solving proportional situations particularly in
Grades 8-9, however, they appeared to have difficulty with the underlying concepts. The
two-tier nature of the assessment instrument provided teachers with ready snap-shots of
students’ conceptions and misconceptions associated with proportional reasoning. The
data served to draw teachers’ attention to the learning needs of their students, and
provided the researchers with an evidence base to frame workshop tasks and activities that
would hence support teachers to focus on practices that might assist them in addressing
their students’ learning needs in their classrooms.
At the end of the first year of the project, the diagnostic assessment task was administered
again. Results indicated large increases in scores, to the extent that they were beyond the
first assessment of students at least two years older. Students showed a greater application
of appropriate multiplicative thinking as required for proportional situations, rather than
inappropriate additive thinking that had been the case for the pre-assessment. These data
suggested that involvement in the professional development and the resulting changes in
teachers’ knowledge and classroom practices enhanced the students’ proportional
reasoning skills. Teacher data showed a greater teacher awareness of proportional
reasoning demands and opportunities, not only in mathematics but in all curriculum
learning areas. This project demonstrates the power of using student assessment data to
support the development of teacher knowledge and resulting change in teaching practice.
However, teacher knowledge development and pedagogical change was not merely a result
of student data. This was mediated by researchers who could draw upon the research
background of the topic to identify appropriate teaching strategies and then design the
professional development that introduced these strategies to teachers, as well as the initial
diagnostic instrument. The complexity and intermingled nature of factors affecting teacher
60 | P a g e
professional learning, particularly in relation to effective professional development
programs as outlined by Sowder (2007), are highlighted in this study. This is important
because data alone cannot make a difference to learning or teaching. Teachers need to
work out (perhaps with help from researchers, as in this example) what the data reveal
about student learning and how to modify their teaching approaches so as to address
students’ learning difficulties.
61 | P a g e
Section 2: Descriptive accounts
The descriptive accounts have been developed following consultations with a range of
stakeholders across Queensland school sectors. These are framed around Standard 5:
These accounts focus on systemic and school practices relating to the use of data within
government, Catholic and Independent education and school authorities. As such, the
accounts in this section were constructed as “mandated” or “required” practices related to
the use of data within some of the school contexts. These are by no means indicative of
practices across all education and school authorities and sectors.
This section of the report is structured using the five key elements of Standard 5. These are
layered with the elements of the analytic framework (see Table 2: Standards and analytic
framework for this project) to form a matrix of possible understandings of the uses of data,
within classrooms, as well as in school and systemic contexts. Table 8: Summary of
descriptive accounts of using classroom data outlines these practices in relation to the
analytic framework used in this study (see Table 7: Data sources and questions matrix).
62 | P a g e
education authorities and schools, suggesting, as does the literature, that the availability
and diversity of these data is unprecedented (Wayman, 2005). A principal indicated:
We've never had a time when we've had access to as much data as we do now
but the emphasis I think has got to remain within schools around what are the
actions that it's leading to? What's the data telling us that then leads to the
strategy that we're then implementing?
Additionally, an education officer noted the importance of acknowledging that data is more
than just “numbers”, stating:
We were confronted with this real thing about putting faces on the data. So it
was not just about the numbers, but these numbers represented little people.
So what are we doing for those little people?
While there may be access to diverse and plentiful data, this account suggests that the uses
of data in schools must acknowledge that data is not “faceless”, nor are its purposes or
applications solely systemic.
This is not to say that district-wide decision making does not benefit from the use of data to
chart success as well as identify challenges. For example, an education officer indicated,
Instead, the perception was that data, as well as being easily accessed, needed to be useful
and meaningful for individual teachers rather than only having systemic value. An
Education Officer stated:
So instead of the data always just being something that you collected for the
system, the data actually became this useful tool for them [teachers], for
planning at all levels … It also became the opportunity to celebrate … where
they could see the big changes.
When data are used during cycles of planning, teaching, assessment, and reflection (see
Theme 3), opportunities arise to celebrate the achievements of the staff and students.
63 | P a g e
TABLE 8: SUMMARY OF DESCRIPTIVE ACCOUNTS OF USING CLASSROOM DATA
64 | P a g e
Assessing student learning
The stakeholders discussed the Standard, Assessing student learning, in different ways,
however, generally in relation to alignment of pedagogy and assessment, with a lesser
focus on curriculum. The purposes of assessment tools were generally framed within a
whole-school approach to assessing student learning, outlining alignment and some
differentiation opportunities in relation to the use of data in schools. When asked about
differentiation with regard to student learning and assessment, one principal quite
honestly and frankly responded, “I would say it’s probably done very poorly at this school”.
One sector also indicated they made diagnostic assessment such as the Developmental
Reading Assessment (DRA) available to their schools, should they express an interest in
utilising such formative assessment in relation to student learning and for alignment and
aspects of differentiation. That is, they deployed specific assessment instruments for use in
schools2 in order to identify groups of students for particular instruction, or to individualise
assistance given to students, or create other types of adaptations to meet students’
different strengths and needs.
Alignment
Using frameworks
Establishing systemic and school goals was established in the literature as a strategy for
effective decision making (see Theme 3). One stakeholder account indicated the systemic
use of the “Breakthrough” framework, outlined by Fullan, Hill, and Crevola (2006), to
reform schools practices. Central to the Breakthrough framework are the Triple P
Components (see Figure 9: The Triple P Components).
2
This was a school-based decision as to whether the DRA was used within particular schools. The stakeholder
indicated that they made this available to schools and supported those schools that indicated they wanted to
implement the DRA.
65 | P a g e
FIGURE 9: THE TRIPLE P COMPONENTS
Personalisation places each learner at the centre of their education. Precision is linked to
personalisation in that precision requires that schools cater to the learning needs of the
individual, for example, through assessment for learning (see Theme 2) using data as a tool
for improving teaching and learning (Fullan et al., 2006). Similarly, professional learning is
linked to both personalisation and precision, as the authors suggest you cannot have either
personalisation or precision without individual and collective daily learning on the part of
the teacher (Fullan et al., 2006). Such frameworks are used by education systems to guide
student assessment practices.
An education officer indicated that through their sector’s use of this framework,
It became really evident that the weakest link we had was the assessment to
instruction piece ... This tool can help with understanding the students' needs,
understanding what you need to do for instruction. But it's not the only thing
you can use … There has been the use of assessment for reporting, as opposed
to the use of assessment for understanding.
A principal agreed with this position, suggesting that data gathered at schools were also
primarily for reporting rather than learning. The principal stated:
There's a lot of data gathered around reporting ... I think probably over the last
two years, the data has been used as a reporting tool, not as a learning tool. So
I think that's the journey that we're on - to make sure that we're using the data
as a learning tool, rather than a reporting tool.
With regard to assessment for reporting, assessment practices become more articulated
and coordinated across the school, accountability reporting horizontally to parents and
vertically to the system also becomes more detailed. Having a more individualised and
66 | P a g e
precise sense of children’s progress increased the confidence of teachers in reporting to
parents and their supervisors.
Such frameworks are able to provide a structure for school reform, outline and clarify the
multiple purposes and audiences of assessment information, and identify the importance
of feedback.
Classroom data
67 | P a g e
This comment and the one below suggests that teachers need to integrate data from more
“objective” sources such as NAPLAN, with their own assessment of students’ progress that
arises from engagement in the classroom:
All of that classroom information and classroom data is, I think, just as valuable
as the whole school stuff that we collect at the end of the term. Ideally we'd
like to think that teachers are not surprised at the end of the term when they
see data, that they've already got an understanding that Fred struggles with
his reading because they've had structures and processes in the class that
that's become apparent.
Now that might be a little bit utopian in the expectation that the classroom
teacher knows and understands the 28 kids in their class and that the data is a
reaffirmation of that information. In some cases it might draw out some more
specifics, and particularly with some targeted collection of information that
would certainly be useful, but it is then about the process to implement a
different approach to increase that understanding.
Again the emphasis from this respondent is on affirming teacher knowledge of student
achievements based on classroom assessment data with other assessment data and
implementing pedagogical change on the basis of that diverse assessment data.
Diagnostic tools
Evidence of reading progress from the tool was often displayed in spreadsheets (see
Section 2: Collating and displaying data, and Section 3: Data walls) that became the focus of
68 | P a g e
whole staff discussions about reading development of students across grades. While
individual teachers were initially defensive abut displaying their class data, the education
officer reported that honest appraisal of the whole school approach helped establish a
culture of shared responsibility for learning and a sharing of knowledge and resources.
Conversations
It's the starting point for their conversations. What happened was where they
may have been sitting back, going “Yeah, yeah, we're okay”. Suddenly, they
were hit in the face that they didn't have one kid in Year 6 reading at Year 6
level, according to this standard [DRA], which is a fairly high standard actually.
It's not a NAPLAN kind of standard, that's fairly low in terms of national
minimum standard. So it was quite confronting, and challenging. So there were
times when there were tears and there were statements, “This can't be right”.
Consequently, the use of the DRA testing identified a problem and teachers were
challenged by the results. Conversations regarding classroom practices ensued and
pedagogy was altered to improve students’ reading outcomes.
In another situation, an education officer gave an example of how formative and diagnostic
assessments such as the DRA were able to generate conversations amongst teachers to
improve reading outcomes. In this example, the data from the DRA was also mapped on a
data wall (see Section 3). The education officer indicated:
In one of the schools, one of the Year 9 boys … was reading at Year 4. This is
amazing too. You've got a group of 10 teachers standing, looking at this wall,
having a conversation. That doesn't happen very often, does it, about data?
Even though - sometimes - they were challenging conversations. Because it
suddenly became, “Holy hell, this kid is in Year 9 and his card [on the data wall]
is down there at Year 4. How did that happen?” There were some really
uncomfortable conversations. To the point then, the Learning Support Teacher
was actually saying, “That can't be right. That's wrong”.
This kid had beautifully masked his difficulty. Everybody just had kept going,
“Oh yeah, he's okay”. But they didn't have mechanisms in place to properly
69 | P a g e
track and monitor. Each year, this child had gone through. So it's been a really
powerful mechanism as well for them.
This comment highlights the importance of cross-grade tracking of student learning and
ensuring that specific learning difficulties of individual children are identified and
addressed. In this example, assessment is used to identify those who are having difficulty,
in this case in reading. When used as a formative tool, that is when such a tool or other
similar tools are used in an ongoing manner during learning to monitor progress, provide
feedback, and differentiate instruction and assessment, they provide teachers with precise
and timely information so that instruction can be modified – differentiated – to suit
individual students’ strengths and needs. Such formative assessment tools also allow both
the teachers and students to set new learning goals or targets and allow teachers to select
new and appropriate pedagogical practices or interventions.
When such a tool, or similar tools are used diagnostically, that is, usually prior to
instruction, to set learning goals and plan instruction and assessments that are
differentiated and personalised, then they assist teachers by providing information about
students’ existing abilities and skills, and allow teachers (and other professionals, for
example, support teachers) to examine in close detail where students’ difficulties are
occurring in order to provide a starting point for intervention or new learning.
At a systemic level, stakeholders generally discussed this element of the Standard in a way
that was embedded in discussion regarding other elements of the Standard, and identified
the “formal” and systemic practices for providing feedback to students, for example, within
understandings of Interpreting student data (tracking progress) and Reporting on student
achievement (academic reporting). However, teachers discussed feedback more specifically
in terms of providing frequent feedback to their students, both formally and informally.
This included discussing marks and grades and relating them to standards on criteria
sheets; providing annotations on their assignments or other assessment tasks; and
providing oral feedback during lessons and as part of a process of returning assignments
and assessment tasks to students.
The stakeholders suggested that at the core of Standard, Making consistent and
comparable judgements, is not just about consistency and comparability, but also about
equity (Maxwell, 2002). Maxwell (2002) explains,
70 | P a g e
Characteristics of the task and the context are therefore of critical importance
in interpreting the student’s performance and judging the standard reached.
Moderation buttresses equity by checking that these characteristics have been
properly considered in interpreting the evidence and that the student’s
performance has been appropriately compared with the standard. (p. 17)
While the notion of making consistent and comparable judgments is grounded within
practices of equity, these practices can also be framed around understandings of vertical
and horizontal accountability.
Accountability
Professional judgements
A principal gave the following account of teachers’ assessing student learning suggesting:
Therefore, while making comparable and consistent judgments is about doing the best by
the students and their work, that is, comparable and consistent equity, it is also about
teachers’ depth of skills and their development of an understanding of the standards of the
criteria representing those standards and exemplars, making judgements, and comparing
judgements. The discussions amongst teachers that lead to a convergence of assessment
judgements associated with students’ examples during the moderation process can be
viewed as authentic professional learning (Maxwell, 2002). The Principal added:
The “big chunk” that is important to this principal is the provision of feedback to teachers
from their peers on the quality of student learning at a particular school. However, as one
principal indicated:
71 | P a g e
With any human process like that [assessment], it's never going to be 100 per
cent consistent. I love rugby league and the referees within rugby league are
the perfect example of that. They're getting paid lots of money; they have all
the technology in the world and there's no consistency there. In all of life,
whenever there's human decision-making involved, you're not going to get
100 per cent consistency. So whilst I think we've got some really good
processes in place to make it as good as we can, if we're driving for or
expecting 100 per cent consistency in that process, we're going to be
disappointed. I think sometimes where maybe people are punching holes in
their processes because that's where - we're never going to get that.
Here, the principal is discussing the assessment of student learning in relation to teacher
professional judgements. This also relates to another aspect of the Standard, making
consistent and comparable judgements, where examples of practices such as moderation
will be discussed later in this section. There is a perceived concern in relation to
accountability and the public perception of teachers’ consistency in relation to assessing
student learning.
Various sectors identified different practices for providing in-school support for teachers,
specifically in relation to making consistent and comparable judgements. One sector
identified a strategy where teachers were trained as mentors at their “Coaching Academy”.
These staff members were then embedded in their school as support personnel. Other
sectors indicated the availability of “education officers” who were external to the school,
and available for consultation and advice when required.
We got to the point where we realised our weakness was that the schools
would always be waiting for the outside person to come in and give them
some guidance. So we realised that we had to have that person as somebody
within their school. From there, our literacy and numeracy Coaching Academy
was developed.
72 | P a g e
This sector now has 150 schools that have signed up at the Coaching Academy with 186
coaches embedded in these schools. The education officer identified the school-based
position as significant in terms of the success of the mentoring program. The education
officer stated:
Unlike other coaching models, ours was very much job-embedded. Then
contextualised and chosen from someone within the school. So we now work
with the coaches, who now do all of that training. We make everything
available to the coaches within the school.
Their role is helping the teachers through that process. So they would actually
help - well ideally, they work with the teachers - to help them. Like if they've
got this scenario here, how do I plan a critical learning instructional pathway
for these students? Or it could be that then the teacher is saying, “Well I've got
no idea how to do that.” So the coach might actually give them advice on how
to implement that; they are a resource person.
EDUCATION OFFICERS
The education officers were another source of in-school support for teachers, however,
their positions were not school-based. As such, their availability may vary and there may be
a time delay between when teachers need support and when they are able to obtain it.
While the various roles of the education officers can be seen as accountability-focused, the
education officers in this study identified their role as one of mentoring the teachers,
helping them review student data and make decisions regarding pedagogical practices.
Take for example this scenario identified by an education officer:
The other day I was working in a school, we were looking in our planned
results and we were comparing the NAPLAN results - we looked at reading,
writing, and spelling, and numeracy - and the numeracy results were lower
than some of the other areas. So we were having conversations around,
what's the pedagogy in your other learning areas? What's happening there
that's not happening in numeracy? They were the questions that we were
having. What's the difference between those classes and these classes, or
what's happening, what do you think is not happening? Those sorts of things.
They were going away to reflect on that, think about what is going on there
that we haven't brought in to here. What's something good that we can take
and trial.
Here, the role of the education officers was also to help teachers reflect on the alignment
of pedagogy and the curriculum, comparing practices across reading, writing, spelling and
numeracy.
73 | P a g e
Moderation
When authentic pedagogy is practised, the teachers do not teach and then
hand over the assessment that ‘counts’ to external experts to assess what the
students have learnt. Authentic pedagogy occurs when the act of ‘teaching’
involves placing high-stakes judgments in the hands of teachers. (QSA, 2010, p.
5)
The specific roles within the moderation process associated with teachers, district review
panels and state review panels are outlined in Text box 11: Moderation roles.
A principal supports the role of the moderation process and teachers making consistent
and comparable judgements about student assessment. The principal stated:
74 | P a g e
Additionally, the principal indicated that
their school has assimilated many of the
moderation practices into their school-
based practices, and broadened the
Moderation roles
range of year levels involved. The
principal indicated: Teachers assess student progress
throughout Years 11 and 12. At the
I think we, like many other end of Year 12, teachers decide the
schools, have been working over exit levels of achievement to be
the last, probably four or five awarded to their students. Teachers’
years, to bring some of those judgments about the standards
processes into the junior achieved by their students are
secondary part of the school. moderated by the QSA, using trained
expert panels of teachers from
Consequently, many of the external
schools.
moderation practices become adopted
as internal, school-based practices.
District review panels are appointed
and trained by the QSA for each
INTERNAL MODERATION
subject in each of the QSA’s 13
Another example of an internal
districts. Each district panel comprises
moderation process undertaken by one
practising teachers and a chair, with
sector focuses on an annual day where
one member for every two schools
teachers come together to discuss their
offering the subject in the district.
assessment judgements, curriculum and
pedagogy. An education officer
State review panels are established
explained:
for each subject. Each state panel
comprises practising teachers and a
It's a chance for schools to come
chair, with membership based on the
together to discuss assessment
number of districts with schools
and making judgements. But
consistency of teacher offering the subject. (QSA, 2010, p. 7)
judgement is actually a
[multiple] pronged approach.
TEXT BOX 11: MODERATION ROLES
It's about understanding the
curriculum and curriculum
alignment, it's ensuring - and one of the four is moderating practice.
This sector provides publicly documented information about the process, suggesting the:
75 | P a g e
enhances the climate of collegiality and professionalism within and between
schools. (Pentti, 2004, p. 18)
The day is once a year, but schools work within their own school first before
the come to meet with other schools. So in effect they're doing that last day
twice because they look at the student work from within their own schools, so
then P to 7 will get together and look at what they've been collecting and what
they've been working on, together before they go and meet with other people
in other schools. So that particular part of the process happens a couple of
times.
However, the process is unlike the external moderation system in that the group of
teachers do not operate as an “authority” that regulates the process and generates
changes in students’ assessment results. An education officer explained:
They'll actually send in some work samples to say well this is the standard our
students are at and then we - that's going to inform our work in the
organisation as well in terms of well, is the alignment there, is there
consistency throughout clusters. So it's a bit like - we've seen years across
Queensland where you have moderation, but we don't have any arbitrator. I
mean, we're not going to go to schools and say, look you just said that child's
well above standard and we don't agree with you. It's none of that because
we're not playing for our sheep stations here. So it's just a way of seeing
whether people across the organisation are interpreting the curriculum in a
consistent way.
Additionally, the CTJ process is more than a moderating process that focuses on teachers’
judgements. It also acts as a professional learning activity for teachers. An education officer
suggests, “Some people find it's just a burden and it's something they've got to do. Other
people see it as an opportunity for professional development”. As well as an opportunity
for professional learning and development, another education officer suggested the CTJ
process was an opportunity for teachers to focus on aligning pedagogy, curriculum and
assessment (Hayes et al., 2006), suggesting:
It's more than moderating because then before we can make a decision on
what it is, we need to go back and understand the intent of the curriculum and
a shared understanding of what things are and on it goes … Because I think we
are one of the few organisations around Australia actually, that has something
on a systemic level where we're comparing work across schools.
76 | P a g e
The CTJ process is also about encouraging reflexivity on pedagogy, curriculum and
assessment, based on the data available, and as part of an ongoing cycle of continuous
improvement (see Theme 3). An education officer explained:
So you'd hope they'd [teachers] use that data and if they use all the
information, so there's your data again in evidence, so whatever they were
doing next year with this, they'd think about it all and they'd think, okay if
some of my year 6s, if this was a multi age class, some of my year 6s got and
my year 7s didn't, well maybe it's not the task. Maybe again now I've got to
think about my approach to teaching this and getting students to understand it
so they're all successful at it.
While practices such as moderation are tools that promote vertical accountability and
public confidence, they are also effective practices that encourage reflexivity and teacher
professional learning opportunities.
Professional learning
A lot of our work that I would probably argue is work around data is actually
work around staff with the skills to be able to implement various strategies
that they can use once they know more about their kids.
I said [to a teacher] the improvement in the children, the program you put in,
to me, that's the side salad for the moment. The main dish was what you
learnt. I said, so let's talk about your learning, rather than what the kids learn.
That took him by surprise!
77 | P a g e
professional learning”. This is how this sector personalised their teacher professional
learning, thereby ensuring both vertical and horizontal accountability. The education
officer elaborated on this process, stating:
What we have really done is we've walked the talk. So if we were expecting
them to personalise what they were doing, and use their data and all the rest
of it, we had to do the same. Which was okay, it's not just this one size fits all,
that you're all going to get a dose of this PD. What do you need? So based on
what you're seeing with your students, what do you need?
Additionally, some sectors indicated their preference for making professional learning
activities for teachers available online. An education officer indicated:
A lot of our PD, we've actually started calling it “Point of Need”, is delivered
online or virtually. We do them from 7:00 in the morning - 7:00 til 8:00, and
then 3:30 til 4:30 and we double up. We record them, so that they can access
them online afterwards. As they are recorded and added online, they
[teachers] can just get on there and play that again. Any of the PowerPoints
that we've used, any of the videos we use, everything goes up there. So they
can do all of that back in their own schools. Then we've been encouraging the
coaches3 to add their own stuff and share. This whole site is just full of all the
resources and everything that they need.
This online resource is a repository of professional learning resources that teachers can
access as required, or as determined by the individual teacher. Professional learning
activities were not generic activities mandated for all staff to attend or participate, instead,
they were small-scale and individualised making such activities more meaningful to
teachers (Wayman, 2005).
This part of the Standard, interpreting student data, focuses on the use of both internal and
external student assessment data to improve pedagogical practices. The practices
identified across the sectors are grouped into accountability practices, and alignment and
differentiation practices within the P-12 structure form this analysis.
3
This is referring to the Coaching Academy coaches identified by one of the sectors. See the next section on
In-school support for teachers.
78 | P a g e
Accountability
As identified in Theme 3, schools often operate within inquiry cycles, guided by models or
frameworks, for the continuous improvement of teaching and learning. One sector
identified a specific framework for incorporating the use of data within the inquiry process.
The Collaborative Data Inquiry Model is a publicly-documented model for the effective
analysis of student data (Education Queensland, 2013). The model cannot be reproduced
within this report, due to explicit copyright instructions on this sector’s website. The model
is displayed as a figure-eight, with goals and targets at the centre of the process. This is
followed by a sequence of steps – collect, interrogate, infer, and verify forming one side of
the figure-eight, followed by – plan, implement, assess and reflect forming the other side of
the shape. This model is available as a professional learning resource for teachers and
schools within the sector (for other examples of cycles of inquiry see Theme 3).
In one sector, attendance, suspension, and exclusion data is often included in the schools’
publically-available annual reports (see Figure 10: Attendance data available through
school annual reports as an example).
79 | P a g e
FIGURE 10: ATTENDANCE DATA AVAILABLE THROUGH SCHOOL ANNUAL REPORTS
Additionally, other school-based analyses were conducted through shorter and more
frequent cycles of data collection. One principal indicated that they had a school-based
position called a “data coordinator” who would review attendance data from the roll-
marking software called ID Attend. The principal stated:
The data coordinator that has no extra financial incentives but a weekly
allocation of two additional free lessons, which are about 70 minutes long, so
140 minutes a week for the year, to coordinate our data collection. So that
role incorporates our behaviour data and referrals that are recorded on [our
Warehousing tool] for state schools. It includes ID Attend data which is the
program we use for attendance and I guess the regular data elements within
the school as well.
Students, and their parents, were made accountable for their attendance through the
analysis of these data. Additionally, some schools opt to purchase additional systems that
enable text messages to be sent to the students’ parent or guardian to inform them of their
child’s absences.
80 | P a g e
These data can also be used for gap analysis, focusing on equity groups (see Table 9:
Attendance outcomes - All Indigenous students) to ensure that gaps in attendance and
learning achievement can be routinely monitored and used to inform school-wide policies
and practises. For example, in Table 9: Attendance outcomes - All Indigenous students, one
school tracked the effectiveness of a program to enforce the attendance of Indigenous
students. While the data indicated significant reduction in the gap, the principal
committed to further improvement and complete elimination of the “gap”.
2011 78% 7%
2012 79% 7%
2013 84% 5%
Within the Standard, Interpreting student data, various strategies and practices were
associated with alignment and differentiation within the P-12 structure. These practices
included tracking student progress, utilising standardised test results, and differentiated
pedagogy.
Tracking progress
Various sectors identified the practice of tracking student progress as a strategy operating
in terms of vertical accountability, but also as a means of differentiating teaching
instruction. The ways of tracking student progress vary within the P-12 structure of
schooling. While many of the ways of tracking focus on documenting students’ academic
progress, there are some practices that focus on tracking students’ career aspirations and
consequent career pathways.
81 | P a g e
OP Analyser, to help young people understand their current predicted position in relation
to their OP4. The principal indicated:
While this is a descriptive account of how the school works towards interpreting student
data, it is also an account of providing feedback to students. Wiliam (2011) suggests that
such feedback practices provide information to the learner. As the Principal suggested,
“explaining” to students where they are positioned and helping them increase the “own
understanding”, both clearly and explicitly provided feedback to the students that enabled
them to move forward in their learning as well as encouraging the students to take
responsibility for their own learning (Wiliam, 2011).
Once a term, those students have a meeting with a mentor that's assigned to
them. This is either a member of admin, head of department or one of our
coordinators. We've got a range of subject area coordinators, student
coordinators that are involved in that. So that's every Year 11 and 12 student
will sit down once a term. If they're an OP student, the information will include
each of their subjects and the OP Analyser sheet that not only has their level of
achievement, but the data that goes into that to try and boost their
understanding of the process around OPs.
This second descriptive account identifies mentoring as another feedback practice. These
individualised mentoring sessions with the learners enable them to take a more active role
in the learning and career goals (Wiliam, 2011).
4
An OP is required by students seeking tertiary entrance. An OP 1 in Queensland was equivalent to an
Australian Tertiary Admissions Rank (ATAR) of 99 (QTAC, 2013).
5
The QCS is the Queensland Core Skills test, a common state-wide test designed for Year 12 students that
contributes to a young person’s OP.
82 | P a g e
TOWARDS IMPROVED ACADEMIC OUTCOMES - FEEDBACK TO STUDENTS
Some schools also use academic reporting data for tracking Year 8 to 10 students. A
Principal explained, “Staff interrogating data, at the end of each reporting period, we target
students that have failed three or more subjects. We have a process for our Year 8, 9, and
10 students”. The principal elaborated on this process:
Our head of department for these year levels collates the information that
these are the students in Year 8 who've received a D or an E in three or more
subjects. They then coordinate the process where information goes to
teachers to inform them that they're teaching a student who requires an
Academic Improvement Plan. The Academic Improvement Plan is about trying
to target support for students who are in that position.
However, the principal provided the following warning, suggesting that such tracking
practices needed to be less about compliance or accountability and more about helping the
individual student. The principal indicated:
[In the past] I think we've dipped a little bit too much into the compliance
process of, let's complete the form to say that we've got something here for
little Johnny because he's failed three or more subjects, as opposed to having
… an intervention strategy for someone who's in need and not passing the
subjects.
The principal suggested that currently, teachers were asked to discuss the academic
progress identified in the tracking data with the students and their parents. Again, this
aspect of interpreting student data is also a practice for providing feedback to be students
and their parents. The principal indicated:
The process that we've got at the moment includes an element where the
teacher is required to discuss the problems of why little Johnny may not have
passed with the student and with their parents, and then to develop a plan for
action for the teacher to be able to implement - for the student to be able to
implement and the parent to be able to support.
Consequently, this process, rather than one that is compliance-based, has operated more
effectively within this school environment. The principal suggested, “We do have some
really pleasing situations and data around students that have improved considerably
through that process”.
83 | P a g e
We also have their QCE information, access to their report cards and all of that
information as to whether or not they're on track, whether it's on track to get
into the university course they want to go to or on track to get their QCE or if
they've got a particular interest or they want to head down a school-based
traineeship pathway or employment pathway. Are they doing the things at
school that’s required to help them achieve their goals?
Additionally the principal indicated that schools use the academic reporting data and the
tracking data to support students and provide guidance regarding career opportunities and
pathways. The principal suggested:
Within the Standard, Interpreting student data, stakeholders indicated standardised test
data were frequently used for alignment purposes within the P-10 structure, and
sometimes for differentiation. A principal indicated:
We have taken our staff right through the data. Going from the global view of
how the school is performing around - the global data, like NAPLAN data, and
we collect PAT-R and PAT-M6 data in reading and mathematics. So looking at
that globally and then bringing that back down to a classroom level.
We've previously used the data that we get from primary schools and from
NAPLAN for our future Year 8 students, to help inform classes and groupings
for Year 8. As I said, we are moving away from that a little bit at the moment.
6
PAT-R is the Progressive Achievement Tests in Reading and PAT-M is the Progressive Achievement Tests in
Mathematics.
84 | P a g e
This year for instance, we've got approximately 30 kids in our 8A class. Now
some of those kids are performing really well and working really well in that
area. Some of the kids aren't necessarily doing badly but they're not in the top
30 of the students in the cohort. We've got other students outside of that
cohort that are performing very well in their other classes that aren't getting
the same benefits that we provide to the 8A students.
Here, the Year 7 NAPLAN results were not a useful method for differentiating class
groupings. The principal reflected on this, stating:
I think the data and the information that we get from primary schools, whilst
it's useful and worthwhile, isn't an exact indication as to how the kids are
going to perform and I think we make some very serious and almost life-long
decisions about class groupings based on data that's not 100 per cent reliable.
I think we consequently make some decisions around - or have made some
decisions around how we provide opportunities and run our classes in Year 8
based on that data.
Consequently, while still looking at using the NAPLAN data, class differentiation in the form
of ‘streaming’ would not be practiced in the future at this school. The Principal stated:
So next year we're altering that process a little bit. We'll still look at and use
some of that information but we're looking at - instead of having our top 30
students isolated in an A class, that they'll be distributed down, as all the
students will, across all of our Year 8
classes in that respect.
The timing of the release of the NAPLAN data posed problems for teachers seeking to use
the data to improve teaching and learning. Regardless, this does not alter the situation
that standardised test data, such and NAPLAN, is high-stakes (Berliner, 2011; Darling-
Hammond, 2010; Stobart, 2008). An education officer noted this, stating, “Meanwhile, the
pressure is on because NAPLAN - no matter what anybody says, NAPLAN has become high
stakes. It became high stakes the minute they put it on My School”.
Yes there are certainly trends we see through NAPLAN - is the only thing that,
because again, it's the consistency of it. I think there we've got to be very
careful of this. At a system level, we can probably only make decisions on
reliable data and probably the most reliable - I'm not saying it's the best
reliable - but the most reliable data is NAPLAN data because of what it is.
Here, the notion of reliability is identified as having degrees associated with it, that is,
“reliable” data, “most reliable” data, and “best reliable” data. The notion of “reliability” of
such testing and their “capacity to achieve their own objectives of impartial, reliable and
unbiased reporting designed to facilitate student, school and system improvement” is
questioned within the international literature (Polesel, Dulfer, & Turnbull, 2012, p. 4).
While the “reliability” of high-stakes testing programs is questioned, a recent report also
found that such testing regimes impact on the well-being of the children, with potential
impacts including students’ self-esteem, stress, anxiety, pressure and fear (Polesel et al.,
2012). Additionally, such data is often used for benchmarking purposes (see Text box 12:
Benchmarking).
Another sector added the following commentary about the systemic use of NAPLAN data,
suggesting:
But I think we're cautious about that in our messages about the NAPLAN
anyway, it's only one thing. But that's what's driving so much of where our
schools perceive themselves and all of that, to be honest.
86 | P a g e
Such calls for caution are commendable as the international literature suggests that high-
stakes testing can impact on the quality of students’ learning experiences (Polesel et al.,
2012).
Pedagogy
Within the Standard, Interpreting student data, some stakeholders indicated that the
notion of the daily interaction between teacher and students, as well as teachers’
pedagogical understandings of their students, provided useful situations for data gathering
and interpretation. In a sense, the students’ daily progress became the data source. This
was in contrast to NAPLAN data driving teaching and learning. One of the sectors indicated
data obtained from the students’ “daily learning journey” was valuable and in most cases
preferable to NAPLAN data in relation to improving teaching and learning. An education
officer gave the following account:
A principal added:
I think there's some good practitioners … that will look at that data and then
use that for planning. You can certainly see that in terms of the results for their
children. You can see the positive impact that has. Now that would be
different from each year level, depending on the individual teachers.
While this principal has suggested that good teacher practitioners use day-to-day data,
there is also the suggestion that students’ results are linked to good teacher practitioners.
While it is commonly understood that teachers make “a” difference to student learning, it
is generally not understood that teachers make “the” difference to student learning (Gale,
2006; Hayes et al., 2006). Additionally, international research in the area of teacher quality
and student achievement (Amrein & Berliner, 2002; Darling-Hammond, 2000) suggests
87 | P a g e
there is very little evidence to correlate a relationship between teacher quality and
students’ academic performance.
There were accounts of data being used in relation to pedagogy and the provision of
assessment feedback to students, and pedagogy and the curriculum.
I think one of the other things, we talk about - it's like when students have
completed their piece of assessment, particularly in the secondary, what then?
Often it's just, “Well then we're going to teach the next topic now”.
This commentary perhaps suggests that the element of feedback in pedagogical practices is
overlooked or not handled particularly well, that is, in relation to supporting the students’
current learning and future improvement. The education officer suggested a “more cyclical
and integrated approach” to providing feedback seemed more appropriate. Additionally,
students’ self and peer-assessment and reflection is also considered an effective strategy
(Black et al., 2004), but not noted in the descriptive accounts provided by the stakeholders.
Data were able to assist these schools with aligning pedagogy, assessment and the
curriculum. An education officer gave the following account of an experience in a school:
The other day when I was having a meeting with some staff and we looked at
their results, and their results were good, however, they were happier with
reading and writing. So even though it obviously was good, they were saying
we can be better, we can get our numeracy to the same results as our reading.
88 | P a g e
Reviewing classroom and NAPLAN data enabled the teachers at this school to begin
conversations to improve pedagogy and student outcomes. The education officer
elaborated:
So we started with the conversation, well what's going on, what's happening in
reading that's not happening in numeracy? How can we go away and think
about this? They were starting to say things like, well in reading we really
broke it down into these areas and we'd go away and we'd do these activities,
and there'd be feedback, and we'd have times when we talk about what we're
doing.
The education officer outlined a specific conversation that was had by the teachers at the
school to help identify aspects of the situation and possible solutions:
Maybe we're not doing that in numeracy, we're just doing these tasks, it's a bit
more closed and they were recognising the different strategies. So they were
going to go away and make a list of all the things that were happening in those
other areas; what was happening in numeracy, and then they were going to
see, what would work, what would be a strategy that would work from one
area to the next. What was happening in numeracy that might not be the
most effective practice that they think, well let's work on that. But they'd also
had the education officer for numeracy working with them.
In this account, the classroom and NAPLAN data facilitated conversations between
teachers that identified areas for improvement and possible solutions.
This part of the Standard, Reporting on student achievement, focuses on data obtained
through communications with parents, data from academic reporting, and the collating,
analysing and displaying of this and other data using warehousing and productivity
systems. The practices identified across the sectors are grouped into accountability
practices as part of this analysis as they are generally mandated practices within school
systems. However, it should not be understood as the only reason for reporting. Reporting
on student achievement will be discussed as part of a home-school partnership that
encourages support, collaboration, and builds trust.
89 | P a g e
Accountability
Communication with parents can take multiple forms, including online and social media
formats. A principal gives the following account:
The more communication we can have between our teachers and the parents,
the better. It's wonderful if they're listening to things that I'm saying, and it's
wonderful if they're checking our website or Facebook page or the school
newsletter to see information from me but really what I would love more
times over is that they're able to listen and hear from the teachers that are
teaching their kids, where there's the direct impact.
The principal elaborated on the types of communication mediums that have been set up
through the school:
The principal suggested parents and a teacher communicating with each other was more
important than parents contacting the principal. Additionally, he acknowledged that the
teachers needed to feel supported in their communications with parents. The principal
stated:
I guess that's the challenge for us, and part of that challenge is having staff feel
comfortable, having those conversations. Part of that challenge is having staff
have that time in their day to be able to make a phone call, and the parents
having the time in their day to be able to receive the phone call and vice versa.
Academic reporting
We report to parents four times a year at the end of each term, so we do a fair
bit of work around our whole school results, our faculty results, our class
results and individual student results with academic data. The other element
90 | P a g e
that we report at the same time as that is effort and behaviour data and that's
across all schools [in this sector] … We do emphasise that.
I think we do a good job by adding in the fourth time, a lot of schools only
report twice a year, and some three times. It's good to have the end of term
reporting process to give information to parents and also to have that
information within the school.
While four reporting cycles within a year may be an effective strategy for generating
parental interest and support, it does raise the question of the intensification of teachers’
work (Hargreaves, 1994) and the pressure this places on teachers’ time. Additionally, this is
an example of one school moving beyond the requirements of a mandated vertical
accountability system in terms of reporting cycles. As the principal indicated, other schools
operate under shorter reporting cycles, prescribed by their education authority.
COMMENT-FREE REPORTING
One sector principal indicated that they used “comment-free reporting” as a strategy at
their school. The principal stated:
We don't actually put comments on the report card. Partly because I think a lot
of the comments, and certainly the generic comments that are available within
the system - there's lots of words without saying a lot … So we remove those
comments.
91 | P a g e
The principal indicated that this strategy had been effective for their school:
Our numbers or our data around those parent teacher interviews are very
good as well. Each year over the last four years the number of parents coming
in and the number of interviews and face-to-face interactions we've had have
increased. Whether they're solely related to removing the comments, I'm not
sure but we're certainly increasing that face-to-face time.
Wayman (2005) suggests that while schools have been significantly “data-rich”
environments, they have been “information poor” due to the ineffective, inflexible and
often frustrating ways in which data is stored. Computer systems are often used to collate
and display data in schools. In particular, stakeholders discussed the use of productivity
software such as spreadsheets and warehousing tools available through online portals,
specific to the different sectors, which collate, analyse and display classroom and systemic
data.
SPREADSHEETS
This example given by one sector on the use of spreadsheets was linked to the
Developmental Reading Assessment (DRA) discussed earlier in this section. Such
spreadsheets provide “running totals” of students’ progress. Figure 11: Spreadsheet
collating, analysing and displaying DRA classroom data, is an example of one such
spreadsheet.
FIGURE 11: SPREADSHEET COLLATING, ANALYSING AND DISPLAYING DRA CLASSROOM DATA
This is one of the spreadsheets that they get from the class profile with the
DRA … It's in an Excel document, and we then taught them [teachers] how to
do conditional coding - conditional formatting … They colour code red for their
“screaming needs”, and 4s are greens obviously. This then becomes not just
the target for their instruction. It also became the target for professional
development.
This starts to just show you trends. So what they can do is - they get to see
graphs … about where they're sitting. Because it's just in Excel, they can apply
filters and everything. They [the children] can do all kinds of analysis and
tricks. We can look at doing pivot tables and what's this showing up for
spelling? What are your patterns? But more at a school level than probably an
individual class level.
WAREHOUSING TOOLS
Wayman (2005) suggests, “Recent technological gains have resulted in tools and models
that efficiently warehouse data for the examination of relationships commonly explored in
the education arena”(p. 299). This warehousing of data within online portals was a practice
identified by two sectors in this study. Often, the data were made available through “user-
friendly data presentation interfaces” that “connect the user to the database and are the
intermediaries through which users may examine relationships within the data” (Wayman,
2005, p. 299). Many of these warehouses contain “preformatted reports that are
previously compiled summaries of data that are available for viewing or printing with one
click and require no specifications, alterations, or input from the user” (Wayman, 2005, p.
299).
93 | P a g e
While there is significant publically available information about these warehousing systems
that were identified by the two sectors in this study, these systems will not be named
within this report and the data collected from the stakeholders will be presented as one so
as not to affect confidentiality. Generally, these portals were identified as supporting
teachers by collating and displaying analysed data on NAPLAN testing, academic reporting,
student outcomes, student behaviour, career planning, student records, and details of any
contact made with parents. A principal indicated the online portal “provided a wealth of
information for classroom teachers and increasingly the broad strategic or central office
data that's available to me as principal”.
Access
An education officer identified the range of access associated with warehousing tools,
explaining:
Schools can access it [the tool] through our portal, but outside people can't
access it. It is for people [within our organisation] and each school can only
see their own set of data, they can't see other schools, whereas people
working in the office, education officers and so forth, to help the schools that
they work with, they can see a range of schools to see what the needs are. So
the site is divided up into different areas, such as student profiles, NAPLAN,
our SRS [School Reporting System] data, which is the report cards, and we've
also got things that can come through from the QSA [Queensland Studies
Authority].
Student history
The warehousing systems can also catalogue students’ behaviour and academic histories at
their schools, however, only within the single organisation. An education officer explained:
Students who come into the system from other schools, their data doesn't
come with them, we have to start with them from scratch. So if a student in
Grade 8 happened to come into the organisation at Grade 8, we don't have
access to any of their prior information. But if a student started in one of our
schools in Prep, as long as they stayed within our system, as long as they were
going from school to school, all of that information would travel with them, so
we've got a picture for that.
Uses
While warehousing tools have been identified within the framework of accountability, one
sector specifically indicated that their data tool was used to encourage conversations
between teachers to improve student learning rather than “punish” perceived poor
performance of either students or teachers. An education officer indicated:
94 | P a g e
So, as a system, we're talking about how's it [warehousing tool] is being used
in schools, what are the conversations we're having, what can we do to help
teachers look at their NAPLAN results and ask questions of it. Not necessarily
as a pointing fingers or anything like that, it's what does this data bring up for
us, what are the questions that we're - what are the conversations we can
have about what's going on and it's used for that business to start dialogues
within schools and say what's going on … we've looked at some of the NAPLAN
data for example, just recently, looking at our trends and looking at where
we're seeing areas that obviously need some work, so that will have some
influence on what we do.
Significantly, the tools were highlighted as being effective catalysts for reflecting on
pedagogical practices. An education officer gave this hypothetical account:
So they now have access to the data and it's just them, I suppose, trying to
look at why things are the way they are. It's getting them thinking about
pedagogical practices and as a result of that - that's probably where we as
education officers that are often being called in now. I'm starting to get phone
calls to say, “Hey, in the area of mathematics we've noticed that our students
have been going backwards” - I'm making this up for the record – “Our
students have been going backwards, significantly backwards for the last five
years since I've been the principal. Come and help me because maybe it's
something I'm doing wrong”. Made up story. But they're the sorts of things
that schools now have the ability to start questioning.
The warehousing tools are able to provide trending data for schools to interrogate their
practices.
Additionally, the tools are able to display cross-referenced data that is able to provide
preformatted displays on individuals, classes, and schools. An education officer indicated:
So the tool can be used in lots of ways in terms of individual students, the
whole school trends, class trends, so you can break it down into different areas
… Seeing trends and also inconsistencies with some of their school-based
assessment and national standardised assessments, so they're raising
questions about that.
Scaffolded support
One of the sectors indicated their warehousing tool provided built-in support for teachers
that scaffolded teachers’ experiences in terms of understanding and interpreting the data.
An education officer explained:
I think one of the things we'd agree with is that teachers generally need a lot
of assistance, how to interpret the data. It's not necessarily putting - well,
95 | P a g e
even putting the data together - this tool puts the data graphically for them …
Just in terms of how we use the data then, the info button on the right hand
side is there to assist teachers. It actually gives descriptions of what the graphs
are and what you might use. So one of the things probably we've recognised,
that there are teachers in our schools who don't know how to interpret data.
So as a way of addressing that need, as part of this package being put
together, there's an info button about that, which we found was a valuable
tool.
Aside from the technical assistance provided through the tool’s built-in knowledge base
regarding the simple statistical and graphical data visualisations, the education officer also
indicated that teachers need support and assistance with interrogating the data and
determining courses of action. The education officer stated:
96 | P a g e
example)
97 | P a g e
Section 2: Summary of key ideas
ONE: The main focus of the descriptive accounts from stakeholders focused on
accountabilities and alignment. There were some accounts related to differentiation
and differences within the P-12 structure, but no accounts addressed the notion of
student self-assessment or challenging tasks. Additionally, there was a greater focus on
the primary (P-7).
TWO: Descriptive accounts indicated that education systems and school are inundated
with measurement data. Additionally, stakeholders suggested there is a greater focus
on measurement data as a tool for improving vertical accountability, so much so that
Lingard and Sellar (2013) suggest some of this measurement data, in particular the
NAPLAN data, act as “catalyst data” and are pivotal to school and system accountability
practices.
THREE: Visual data displays were used to collate and display many of these data via
online portals that were accessible by computers. Access to data was related to the
position of the individual within the education system. While this provides an ease of
access, it is also a mechanism of control. Increasingly, within societies of control,
schools rely more and more on numerical codes and data, and the utilisation of socio-
technological mechanisms such as passwords that create gatekeepers and users
(Deleuze, 1995). Consequently, institutions such as schools are becoming corporate
systems that are increasingly required to maintain copious records of measurement
and performance data in order to enable auditing (Power, 1999).
The incidence of online warehousing tools and the utilisation of dashboards was
evident across two of the sectors. These data displays provided teachers, schools, and
education systems with tools for comparison of individual students, classes, and
schools across a state, nationally, and internationally, as well as comparisons of
teachers based on extrapolated student data. These warehousing tools also scaffolded
the data interpretation process for the teachers by providing pre-determined formats
and visual displays. However, Australian and international studies (Hayes et al., 2006;
Sahlberg, 2007) have indicated the limited usefulness of these data and cautioned
against the use of simplistic analyses and international comparisons derived from these
data as they fail to take into account the many underlying characteristics such as socio-
economic status or family background, that may explain the comparative performance
of schools.
FOUR: The stakeholders identified a tendency towards the use of other visual and
public displays of data within school communities, such as the use of spreadsheets
and data walls. Such displays become the sites of professional conversations between
98 | P a g e
teachers. Conversations surrounding these data walls were generally focused on data
obtained from diagnostic assessments rather than high-stakes NAPLAN data, or
conversations about assessment practices and pedagogy. The displays, while public,
were generally reserved for the teacher cohort, rather than being made available to
students and their parents. However, conversations between teachers that were based
on data from the visual displays also led to conversations with students and their
parents.
FIVE: The descriptive accounts focused on the multiple purposes of data. Effective use
of data was dependent upon alignment between the purposes for which data were
collected and the consequent purposes and uses of these data. Here, it was suggested,
there needs to be a “fit” or an alignment of these purposes in order to ensure integrity
of practice and an effective use of data.
SIX: Many of the descriptive accounts from the systemic and administrative levels
focused on literacy and numeracy. Stakeholders linked the uses and purposes of data
to improving the students’ literacy and numeracy performances. Improvement was
frequently gauged through NAPLAN testing, however, diagnostic literacy and numeracy
assessments were also used. While focusing on literacies and numeracies, the
stakeholders generally did not address notions of the actual curriculum, for example,
the Australian curriculum or the key learning areas within the curriculum structure.
This omission, while it may relate to the context of the data gathering process
associated with this study, could relate to the understanding that such a focus on
measurement and performance data from high-stakes testing narrows the curriculum
(Darling-Hammond, 2010; Klenowski, 2011; Sahlberg, 2010; Stobart, 2008; Thompson,
2012). Additionally, this omission could also relate to the need for a greater focus on
aligning curriculum with pedagogy and assessment (Hayes et al., 2006).
SEVEN: The notion of differentiation was not significantly addressed in the stakeholder
accounts. As such, there is a danger that while there is an abundance of data, these
data are not focused on providing differentiated learning opportunities for those most
marginalised in our society. Consequently, if these students or groups of students are
not identified as “under-performing” and do not get the opportunities to obtain
instruction that is aligned with their educational needs, there is a risk that such
students “fall through the cracks”. Classroom data is able to help with aligning
classroom instruction with learning goals and these data can therefore be used to
refocus pedagogy on content and skills where performance is down (Moon, 2005). The
danger associated with misalignment or a lack of differentiation is that students
become disengaged with the learning process and risk displaying negative behavioural
and attitudinal behaviours that propagate the achievement gap (Moon, 2005).
99 | P a g e
EIGTH: Many of the descriptive accounts provided a limited understanding of what
counts as data. Data was almost exclusively limited to student performance data, often
derived from tests, and as indicated earlier the data often focused only on literacy and
numeracy.
With respect to the data described and exemplars offered there were none that
recorded students’ abilities to engage in analysis and evaluation, to apply knowledge
and skills to real-life contexts and problem-solving, or to use critical thinking – the so-
called 21st century skills. In addition there were few references to students’ skills in
communication or related to their affect and social-emotional well-being, although one
school referred to tracking students with respect to their career aspirations and
consequent achievements on their future career paths. Finally, in discussing data, there
were few references of the need to take into account or to “read” and interpret data in
the contexts of students’ access and engagement with learning, their opportunities to
learn, and the teaching practices employed. Thus it is suggested that developing a
broader understanding of what counts as data and ensuring that attention is paid to
the broader contexts of data gathering so that richer and more nuanced
understandings and uses of data can be developed are essential.
100 | P a g e
Section 3: Cases of strategies and
practices for using classroom data
The cases are drawn from an analysis of interview data, classroom observations, and school
artefacts. They identify useful strategies and practices for using classroom data to improve
instruction and school programs. Table 10: Summary of cases of strategies and practices for
using classroom data outlines the elements of the matrix relevant to each case and the
alignment of each case with the Standards and the study’s analytic framework.
The cases represent a consolidation of the data gathered from school visits, the previously
analysed descriptive accounts from the stakeholders from each of the sectors, and the
research literature. The case format7 is divided into five main sections: context; focus
areas; implementation and outcomes; evidence of success; and questions for discussion.
The case is contextualised in terms of the area of the practice, the setting, and the goals
and purposes of the strategy or practice or program that use classroom data. The section
on focus areas outlines how the cases relate to the previous sections of the report,
including the Standards, the themes of the research literature, and the descriptive accounts
from the stakeholders. The third section outlines implementation and outcomes of the
strategy or practice or program, focusing on teachers’ impressions. The next section
identifies anecdotal and empirical examples pertaining to the success of the program. The
final section includes a series of questions that can be used to stimulate discussion related
to the cases, but specifically to the Standard.
7
The general style of the case and some of the headings in some of the sections were drawn from a model
provided by Timperley et al. (2007).
101 | P a g e
Professional Practice | Standard 5 | Focus areas 5.1, 5.4 |
102 | P a g e
TABLE 10: SUMMARY OF CASES OF STRATEGIES AND PRACTICES FOR USING CLASSROOM DATA
103 | P a g e
comparable collaborative improve consistency of
judgments professional learning teacher judgments
situations
Interpreting Case 4 – tracking and Case 1 – collating and Case 4 – using tracking
student data profiling student data displaying reading data data to make informed
to identify progress for analysis choices
and aspirations Case 2 – interpreting
student data using
visual data displays
Case 3 – collaborative
interpreting student
data within Learning
Circles
Case 5 – identifying
and interpreting
relevant assessment
data for improved
learning outcomes
Reporting on Case 4 – using tracking
student systems to monitor
achievement student progress and
differentiate learning
104 | P a g e
Case 1: Collecting and using diagnostic data focusing on reading
1. CONTEXT
Area of practice Reading: The staff at this school implemented a whole-school program
that focused on the Developmental Reading Assessment (DRA) to
improve reading within the school.
105 | P a g e
purposes performances using formative and diagnostic tools such as the
DRA.
The practice is linked to the notion of early intervention where
teachers use ongoing assessment data to determine intervention
strategies such as reteaching, varying instructional approaches,
using different materials, or providing students with more time
(Sharratt & Fullan, 2012, p. 124).
The practice is a strategy that identifies the low-achieving Year 1
and prep students and thereby can potentially narrow the
achievement gap in later years (Sharratt & Fullan, 2012, p. 124).
2. FOCUS AREAS
Related 5.1: Assessing student learning
sections of 5.4: Interpreting student data
Standard 5
However, the principal at this school did indicate that there was a level
of anxiety associated with implementing new practices.
Principal: “When we first took it on, there was quite a bit of anxiety in
the staffroom and it was significantly challenging initially because to
collect the data initially when they hadn't done it before takes a lot of
time.”
107 | P a g e
teacher, to facilitate the focus learning that each child needs. Going
through school myself I didn't do very well. We all learnt together
regardless of where we were on our journey. That didn't work for me and
subsequently I failed everything. Coming into a school as a teacher, this
process has now allowed me to focus in on a concept, understand that
concept well and then be able to facilitate it in several different ways to
a focus student. So for me it's great for the kids but personally it's
helping my journey in leaps and bounds. So the data is a two-way thing
… It's a great system … it does help the teacher focus and understand
why and what they're teaching and who they're teaching it to.”
Principal: “It takes a lot of time. So in that first 12-month period we got
in quite a few relief teachers to actually relieve staff while they were
doing the one-on-one testing with their children because each child
initially is tested for around about 20 to 30 minutes now. Now we know
how to do it. Back then it was more like 40 to 45 minutes per child. So we
probably put about 40 days of relief into our budget that year to get us
through that first 12 months. We found that the next six months, it
dropped down to about 20 days and now we get very few. When new
staff come on board in the school we obviously try and support the
growth in this area but the seasoned staff have done it for a while now
and know how to fit it into their class schedule.”
108 | P a g e
PICTURE 1: SPREADSHEET DISPLAY ON THE LEFT AND AUSTRALIAN
CURRICULUM ON THE RIGHT
Teacher: “We have now got to a point where we can organise our
schedule now based around how long it's going to take and they do it in
the back of the classroom while students are still working on other things
and so we've got it to that point. But it needed a lot of support initially to
get teachers in the head space of … this is the way we collect the data
here.”
Teacher: “We used to do a lot of testing … [of] spelling and all those
things but we just did them and they just got put away. They didn't drive
what we did in the classroom. Yeah. Whereas this sort of data [the DRA]
is actually driving what we actually do in the classroom day to day, so
that was the big change for us.”
Another teacher added how they are vigilant and have a greater
awareness of the students’ progress.
Teacher: “But now the focus is like - its alarm bells - Who needs help?
What do we need to do? It's not about … What are the effects on you as
a teacher? But it’s whether the kid's moving [forward] … We need to
make sure we're moving everyone.”
109 | P a g e
DIFFERENTIATION OF ACITIVITIES AND ASSESSMENTS:
Additionally, the data collected from the diagnostic tools enabled
teachers to provide differentiated activities for small groups. Sharratt
and Fullan (2012) identify such approaches as differentiated instruction
or intentional teaching. They suggest there are five questions that
teachers should ask:
What am I teaching?
Why am I teaching it?
How will I teach it?
How will I know when all students have learned it?
What then?
Flexible groupings are often associated with this practice as they are
able to meet individual students’ needs. “Teachers scaffold learning for
each student through modelling, questioning, clarifying, chunking,
sharing, rehearsing, guiding, and making their thinking visible through
words, pictures, and symbols” (Sharratt & Fullan, 2012, p. 114).
110 | P a g e
IMPROVING
COLLABORATION: The
teachers suggested one of the
outcomes of the reading
program was improved
teacher collaboration and the
development of a common
language. Sharratt and Fullan
(2012) suggest that co-
teaching, co-planning, co-
debriefing, and co-reflecting
are effective strategies for
building effective data use for
improved teaching and
learning (p. 117). Comments
from teachers include.
Teacher: “Now, people plan together because - even though yes, you've
got Year 1 and Year 2, we're still using the same language, we've got the
same focus and all that. So it's certainly - now the professional
conversations in the staffroom have certainly changed”.
111 | P a g e
professional learning is delivered by credible colleagues who understand
the school context (Sharratt & Fullan, 2012, p. 117). In-house
instructional coaches such as the Year 1 teacher were an effective
practice used at this school.
Teacher: “When I test kids now, the stress level is way down [for the
children] because the language I use to them, I always say to them this is
not about you, this is so I know what I need to do as a teacher, where I
need to teach you better. So it's taken a whole lot of stress and anxiety
off them. Whereas before they were like, ‘Oh I'm doing a test and it's for
my report’.”
8
See Section 2: Making consistent and comparable judgements – section on Mentoring teachers – Coaching
Academy.
112 | P a g e
4. EVIDENCE OF SUCCESS
Anecdotal INCREASED PEER AND SELF-ASSESSMENTS: The teachers
indicated the children in their classes became more aware of the
learning process. They developed a deeper understanding of the
assessment tasks and were more reflexive of their own performances.
Students were also given increased responsibility and autonomy and the
opportunity to develop their skills in making judgements. Additionally,
students developed skill in delivering more relevant feedback to their
peers. Peer and self-assessments also develop independent learners
who can apply what is being learned to new situations (Sharratt &
Fullan, 2012, p. 73). Additionally, peers can influence learning by
helping, tutoring, developing friendships and giving feedback (Sharratt &
Fullan, 2012, p. 73). One teacher gave an example.
Teacher: “The learning is all supplied on our website so the kids are able
to access all the rubrics, all the information, all the links, everything that
they need it's all there. It's a self-pace thing and what it's doing is it's
focusing on the kids' learning to their multiple intelligences. They're able
to present in different mobile forms according to their strengths. It's
quite exciting, we've only been doing it for six months. There's been a lot
of hiccups but I see a great advantage and it's teaching thematically and
you've got three different year levels working together to coach and
work with each other as well. For me, a child working with a child is one
of the most powerful forms of teaching if they're on the right track. I find
a great benefit and it's quite a buzz to sit in there and watch these little
pockets of kids busy working away. There's no front of the classroom,
there's no teacher walking around asking, What are you doing?”
113 | P a g e
Empirical USING NAPLAN DATA: NAPLAN data were identified as a means of
verifying school improvement. Improvements were attributed to the
programs and pedagogy at the school. An explanation was provided by
the principal (also see data table:
Reading).
Reading
Principal: “The first cohort that went
through, where we started to collect 467
data - which is now a cohort that's 434-500
Above Substantially above
just finishing Year 5, what we
actually have seen this year with our SIMILAR ALL
NAPLAN data is that the state and 439 420
the national average are quite close 430-448
on the graphs but our school is
between 40 and 60 points beyond in every area.”
The example of empirical evidence from the NAPLAN data has been
generated using hypothetical data representing one year of testing, for
example 2012, and one year level, for example, Year 3.
Principal: “So it's showing to us that that data collection and using the
data to drive the learning is actually making a significant difference in
the way children learn. And hence our journey down this path. It wasn't
driven by NAPLAN data from our perspective. It was driven by a need to
prepare children for high school in a better way.”
114 | P a g e
Case 2: Using data in visual displays
Data walls
1. CONTEXT
Area of Visual displays: Students’ progress is represented and displayed through
practice student’ assessment data in a visual manner.
115 | P a g e
Goals and Encourage ownership of data analysis, problem-solving, and
purposes decision-making.
Identify the students who require additional assistance (or
intervention).
Promote choices through discussions regarding instructional
strategies.
Allow for whole-school and classroom goal setting, based on visual
data displays and analysis.
Develop productive communication tools
2. FOCUS AREAS
Related 5.1 Assessing student learning
sections of 5.3: Making consistent and comparable judgements
Standard 5 5.4: Interpreting student data
116 | P a g e
3. IMPLEMENTATION AND OUTCOMES
Data walls VISUAL DISPLAYS OF DATA: The data on data walls is displayed
visually, in this example, using colour-coded cards to denote year levels and
progress (see Picture 3: Colour-coded data wall).
Education officer: “All these little cards are colour coded for the year levels.
So it's an at a glance opportunity to see where each and every student is in
the school. They're the individual student cards. There's other stuff that sits
there on that card. Like even some of their [students’] previous scores. So
they move these along according to how their kids are progressing.”
Education officer: “But you can see Year 6, the colour is red. So you see very
quickly we've got a few kids in Year 6 still reading at Year 4 level. So they
become, what are we all doing to help these kids move along?"
117 | P a g e
place. So they used a blue, vinyl sheet, and put all of the cards there. Then
they can just roll it up and bring it out when they need to. So then they have
conversations as a whole staff around this data.”
USING SHARED DATA: The data moved from being classroom data that
was accessed and analysed by the classroom teacher to whole-school data
where the responsibility for students’ progress was shared between
teachers.
Education officer: “So it's not just then the classroom teacher having that
data in their book. But it actually becomes a conversation across the school
as well, where they talk about areas of need.”
4. VARIATIONS
Learning
charts
Other visual displays such as learning charts (see above picture) become
part of classroom instruction. These charts are clearly display in the
classroom and list learning goals as well as teacher and student co-
constructed success criteria (Sharratt & Fullan, 2012, p. 67).
118 | P a g e
Anchor
charts
Data charts
119 | P a g e
5. Questions for discussion:
1. What are (a) the benefits and drawbacks of visual display of data, and (b) the
usefulness of such practices in your own school context?
2. Identify additional variations to those listed in Section 4 (Variations), and explain how
you have used these strategies and practices in your own school.
3. Conversations amongst staff are useful and effective planning tools. How would you
anticipate students be given feedback regarding their progress if they were performing
(a) below benchmark standards, (b) above benchmark standards, or (c) at benchmark
standards? Would the approaches vary in each situation? Why or why not?
4. What types of feedback strategies could be used to ensure students understand what
is required to make effective improvements in their progress?
120 | P a g e
Case 3: Data-informed planning cycles
1. CONTEXT
Area of practice Supporting teachers: Learning Circles were identified as
fortnightly meeting for teachers to focus on collaborative
planning for the improvement of student outcomes.
Data-informed planning cycles: Developing continuous cycles of
improvement based on learning data.
121 | P a g e
really well, so that judgement process is there. But not just making
judgements around where students are at, but using those judgements
to plan the intervention process for students.”
2. FOCUS AREAS
Related 5.3: Making consistent and comparable judgements
sections of 5.4: Interpreting student data
Standard 5
Related themes Data and numerate teachers: Teachers use data to inform
from the decision making related to practice, pedagogy, and assessment
literature to improve student learning outcomes. The practice of utilising
Learning Circles continuously informs this cycle of improvement
and professional learning.
Using data: Data can be drawn from a variety of sources,
including demographic data, achievement data, and data related
to the instructional process. This example uses a range of data
sources to obtain an overall picture of the students and their
progress.
122 | P a g e
able to inform decision making processes.
Principal: “We then linked that back into … the class dashboard in [the
warehousing tool], so we linked that back to teachers accessing the class
dashboard.”
123 | P a g e
Principal: “So that's the first step a teacher needs to do, bring that back
into a Learning Circle. Then they need to go away and work on a five
week program around that.”
Principal: “Then if they come back and say this kid hasn't made any
progress, then that's a Tier - what we call a Tier 2 structure … So it's a
Tier 2, and that would then come to the Student Support Committee.
Then we'd look at what intervention we would go to, and it could be go
to the guidance officer. It could be that we go we need more data, so
we're actually going to do a range of tests, and - yeah.”
124 | P a g e
Learning maps are associated with the notion of visible learning (Hattie,
2012). Here, the learning and assessment intentions were made explicit,
along with the success criteria.
4. EVIDENCE OF SUCCESS
Anecdotal USING CYCLES OF IMPROVEMENT: The Learning Circles are a
continual and cyclical process that promotes improvement. Learning
Circles use classroom data to inform their decision making.
Once one cycle is completed, there is usually a review and then planning
commences for the next cycle of improvement.
125 | P a g e
5. Questions for discussion:
1. What kinds of student data might the Learning Circles use to identify modified
planning, learning or intervention plans, as part of a data-informed planning cycle?
2. In your opinion, are Learning Circles effective strategies for developing (a) teachers’
professional learning, (b) modified or differentiated pedagogy, (c) improving teachers’
practices around using classroom data? Can you identify additional strategies or
practices, other than Learning Circles, that would encourage the above practices?
3. Reflect and discuss how the experiences within Learning Circles may be different for
teachers at various stages of their careers, that is, graduate, proficient, highly
accomplished, and lead.
4. What are the benefits of being involved with Learning Circles or other similar networks
of educators for (a) the individual teacher, (b) other colleagues, and (c) the school?
126 | P a g e
Case 4: Data for tracking progress and mentoring students
1. CONTEXT
Area of Providing feedback to students: This collection of strategies focus
practice on tracking students’ progress in secondary school and providing
them with feedback regarding their progress towards identified
career pathways.
Mentoring students: Support is provided for students in the form of
mentors, that is, personnel in schools other than their classroom
teachers who are able to advise students on the basis of their
identified career pathways.
127 | P a g e
HoD: “It's a complex business. To all of life's complex problems there are a
number of simple answers and unfortunately they're all wrong … The take
home message is know your stuff. If you are going to be a player in the
system and you are responsible for the outcomes of students in schools,
then I think it is incumbent on you to know the system, to know your
students, to understand your subject very well, and to understand what
you're assessing and why you're assessing it and how you can help students
to get there.”
2. FOCUS AREAS
Related 5.2: Providing feedback to students on their learning
sections of 5.4: Interpreting student data
Standard 5 5.5: Reporting on student achievement
128 | P a g e
pertaining to students’ academic progress.
HoD: “Well, I look after all the VET9 and all that side of it, all that
management; I look after all the tracking; I manage all the QCE 10 ; all of the
traineeships and apprenticeships; plus I teach senior classes. Oh, and the
SDCS11, I also manage all that; and I look after the OP Analyser database.”
HoD: “So for example I started at work, 3:00 am Monday morning, I started
work at 4:30 [am] on Tuesday morning. So there'll just be times during the
year when instead of working 50-hour weeks I'll be working 75, 80 hour
weeks.”
HoD: “So at the end of their first term in Year 11 we then have reporting
data. So our tech man who looks after [the warehousing tool], he creates
spreadsheets for me … and I will get from him, that, which you can see is
extremely useless. It's got all the information on it but it's extremely
useless. So what I then do, my first task is to organise that into something
useable.”
HoD: “So I then take that rough spreadsheet or that raw data, and from
that data I then organise it with the main information of their names, their
gender, whether they're Aboriginal and Torres Strait Islander, the class
code, the staff code, the subject, their effort, and their achievement.”
9
VET refers to Vocational Education and Training.
10
QCE refers to the Queensland Certificate of Education, Queensland's senior school qualification.
11
SDCS refers to Student Data Capture System, the data collection system for schools to report student
details to the Queensland Studies Authority.
129 | P a g e
HoD: “So, yellow means that's an outside cert [certificate] that that
student's doing. When we find out that student has completed the cert it
becomes green and the number of QCE credits will go in there … Purple
means they’ve left a subject … Everywhere you see a red under the QCE, the
student is not on track to get the credits for that. Over here in the last
column you see their OP and the OP of course, I generate from OP Analyser,
so I also have to get all of the class lists and the levels and rungs for each
student.”
HoD: “Then from there I can then use this database to get an OP indicator
using our previous data because I'm also look after the Core Skills test and
all that sort of stuff as well and run the Core Skills test practice and take
that program with the seniors and the 11s as well, along with the Head of
English. So from all that information, that gets me that one bit of
information there.”
HoD: “As well as that we don't do what some schools do to get their OP
data looking good, which is give the flick to the slower ones, because with a
pass in English and even an OP of 20 there are many university courses you
can get into. For many of these students they'll be the first one in their
family. So we don't discourage them even if they're struggling.”
HoD: “What we do is try and inform them, ‘Okay, you're struggling, you
might get a 20, however get this pass in English, get this pass, you can still
take these courses and go these ways’. We're always looking for
scholarships for them, we're always looking for ways into the TAFE and Cert
III course for them, however, with the money that's been cut out of TAFE …
that will impact on students in places like this. Last year I think we had eight
students took up a scholarship for Cert IIIs at TAFE for this year but I don't
think there's any scholarships for next year. So sure, we differentiate and
that butts in with the mentoring system.”
130 | P a g e
MENTORING AS FEEDBACK: Mentoring was provided through
individual meetings between students and a member of the
leadership/management team. The first stage involved a management
team meeting. This is a type of case management that focuses on learning
performances and career pathways rather than student behaviour
(Sharratt & Fullan, 2012). Additionally, this is a type of descriptive feedback
that provides students with “practical, direct, and useful insights” that
outline how they can use the information in order to successfully attain
their learning and career goals (Sharratt & Fullan, 2012, p. 71).
HoD: “The students will be divvied out by the management team … We talk
about every kid, any situations we might not know about. So we all get to
know oh, that kid, I know that mum's just left or whatever it might be, so
that we can intervene appropriately. If it's a guidance officer issue or a
cancellation issue, whatever it might be.”
The next stage involved the mentors meeting their allocated students.
12
QTAC refers to the Queensland Tertiary Admissions Centre, the organisation that processes applications
for the majority of undergraduate courses at Queensland universities.
131 | P a g e
DEVELOPING RESPONSIBILITY AND OWNERSHIP: The students
were given some ownership over the process. They were asked to take
responsibility for checking the displayed data about their progress;
ensuring they understood the process and took ownership for
understanding the key aspects of the learning and career data.
HoD: “I've put them on a display board, the rank orders, they're updated all
the time … right outside my office. Kids are there all the time. Absolutely
there all the time. I tell the kids … you must check and if you think it's
wrong, you chase it up and if you don't get an answer you like, come and
see me and I'll chase it up because the whole process has got to be totally
accountable. Students should know exactly how it works, why it works and
where they are and why they're there. Teachers should be able to tell them,
‘Well, you're a SA513 and you're not a SA10 because…’ and that's something
that I have pushed and pushed and pushed. Well, wherever I've worked,
with varying degrees of success, sometimes no success at all but here,
finally, great success.”
4. EVIDENCE OF SUCCESS
Empirical There was empirical evidence to suggest that the strategies that were
implemented as part of tracking students’ academic progress and career
pathways through a system of data tracking and mentoring was effective.
13
SA5 and SA10 refer to the rank order, that is, Sound Achievement 5 is a lower grade than a Sound
Achievement 10 on the 5-point scale (Very High Achievement (VHA); High Achievement (HA); Sound
Achievement (SA); Low Achievement (LA); and Very Low Achievement (VLA)). Each of the five levels has 10
rungs.
132 | P a g e
IMPROVED QCE AND VET RESULTS:
HoD: “Our QCE data, the first year I think we were under 30 per cent. This
year it'll be 90. So you can see there, that's just the last two years. VET
completion rates were under 20 per cent or not much. We get them up to
64 this year. The number of students who got no qualifications whatsoever I
think was 35 per cent in the first year. We're now down to three students or
just under 3 per cent.”
5. VARIATIONS
Variation: A variation to this practice of tracking students’ academic data and career
Tracking pathways involves tracking students’ attendance and behaviour. This was
behaviours managed by the Data Coordinator at the school.
1. Are there benefits to profiling and tracking students’ progress and career aspirations
using compilations of data identified in this case? Are there alternative ways of tracking
students’ progress?
2. How can a school facilitate a school-wide approach to providing effective and
meaningful feedback to students regarding their progress?
3. How are teachers able to use such tracking data to better (a) modify their teaching
practice in relation to the analysis of assessment data, and (b) for supporting
colleagues in this process?
4. How effective is the practice of mentoring in providing feedback to students?
5. How useful is the practice of encouraging students to take ownership and responsibility
for their own progress and career choices?
133 | P a g e
Case 5: Data and differentiated learning
1. CONTEXT
Area of Catering to students’ specific needs: Differentiated learning and
practice assessment was a way teachers at this school were able to cater to
students’ specific learning needs and goals.
134 | P a g e
Goals and Students’ success: Teachers were focused on enabling all students
purposes to attain successful and equitable educational outcomes.
2. FOCUS AREAS
Related 5.1: Assessing student learning
sections of 5.2: Providing feedback to students on their learning
Standard 5 5.4: Interpreting student data
135 | P a g e
Teacher: “We have ascertained students and all those sorts of things, so
that they are accessing the same sort of curriculum as our high flying
students. So they are sort of introduced into looking at what an essay would
be, even though we wouldn't expect them to go off and write an individual -
you know, you wouldn't expect them to able to write an essay on their own.
With a lot of guidance and a lot of structure put in place, then they can
achieve certain things in that area. So there is also a process of
deconstruction and reconstruction, and an individual rewrites and things
like that, to teach the genres that the students need to know and all those
sorts of things.”
Teacher: “That begins basically when they arrive if they are a student with
disabilities and it's reviewed every year. The curriculum is modified to
actually cater for all their needs and basically what the goals are, what the
targets are and in each area, what is going to happen and who is going to
be responsible.”
136 | P a g e
The teacher then explained about the IEP for this student:
Teacher: “His IEP is more specific and more curriculum orientated. Some
personal and social capabilities and everything else so here we set some
goals to focus on expected learnings, to strengthen the partnership
between home, school and personal and just to support the verification
process. That's what the IEP is for.”
Practical USING GRAPHIC ORGANISERS: These are thinking tools that enable
examples - concepts to be broken down in to simpler ideas. These often involve visual
Outcomes or diagrammatic representations of the concepts and use key words rather
than detailed paragraphs. The teachers at this school indicated they used
graphic organisers in their teaching.
Teacher: “When they’ve handed their work in, they generally get feedback
from how they’ve gone in that – on their task sheets and things like that …
A lot of verbal feedback too, on their progress as well. Because they’re
always keen to sort of see how they’re going and ask you how they're
going. Being a small school, you tend to know the students.”
Teacher: “As far as I know, most of the teachers will give feedback to their
students verbally, initially, and then when they've submitted their task, then
they'll give them the written feedback as well, on their progress.”
Teacher: “I give verbal feedback. Then we would write a little note on their
charts and put them in their folders so that they were able to take them
137 | P a g e
home to their parents as well.”
Teacher: “I would reconstruct the whole thing and write out steps for them.
Step one, what is conflict? You may find your answers in the handout that I
gave you or you may like to use your laptop. So it’s all self-explanatory right
from the word go all the way through. So scaffolding, yeah, which I do all
the time with these kids. They need the scaffolding. It’s really important.”
Teacher: “So they had to perform a play. They were able to come up with a
conflict all right. Some of them had battles on the floor and stuff, but the
empathy and then resolving it was not possible. But the penny didn't drop.
Then we realised after a while that it wasn't working, because these kids
[ASD] have no empathy, they don't have emotion. They don't know how to
do it, they didn't understand the resolution. So we're actually changing
this.”
138 | P a g e
USING ONE-ON-ONE EXPLANATIONS: As part of the scaffolding
strategy, teachers used individualised explanations to differentiate learning
processes for the students.
Teacher: “So I'll have the child in and I'll say, ‘What's the problem? You've
got maths, English and science. It was due last week. What's the problem?’
Half the time they'll burst into tears, so okay, ‘When was it due?’ Whatever.
‘What is the task?’ [and they say] ‘I don't know’. I would say, ‘But your
teacher just told me that they explained it to you.’ ‘I still didn't understand.’
I ask, ‘So did you go back to the teacher and say you still didn't
understand?’ ‘No.’ ‘Okay, if I got the task out for you and I ask the teacher
what it is, will you do with me?’ They usually say, ‘Yes’. They're in the
classroom and they're embarrassed to say, ‘I don't understand’. This is a big
problem every term, a huge problem.”
Teacher: “Because the school is so small, we're in contact with the parents
on a regular basis. So we try and encourage staff as well that if a kid is
struggling or something's - don't leave it till the last minute, until the
reports go home. Just keep informing the parents along the way. So we're
in constant contact with parents.”
139 | P a g e
References
ACARA. (2011). Australian Curriculum, Assessment and Reporting Authority. Retrieved 12
August, 2013, from http://www.acara.edu.au/assessment/assessment.html
AITSL. (2011). National Professional Standards for Teachers. Retrieved September, 2013,
from
http://www.aitsl.edu.au/verve/_resources/AITSL_National_Professional_Standards_for
_Teachers.pdf
Allal, L. (2013). Teachers’ professional judgement in assessment: A cognitive act and a
socially situated practice. Assessment in Education: Principles, Policy & Practice, 20(1),
20-34.
Amrein, A. L., & Berliner, D. C. (2002). High-stakes testing, uncertainty, and student learning.
Education Policy Analysis Archives, 10(18).
Armstrong, J., & Anthes, K. (2001). How data can help: Putting information to work to raise
student achievement. American School Board Journal, 188(11), 38-41.
Barnes, F. D. (2004). Inquiry and action: Making school improvement part of daily practice.
Providence, RI: Annenberg Institute for School Reform, Brown University.
Bedwell, L. E. (2004). Data-driven instruction. Phi Delta Kappa, 516, 3, 7-33.
Bennett, R. (2006). Foreword. In G. Matters (Ed.), Using data to support learning in schools:
Students, teachers, systems. Camberwell, AU: Australian Council for Educational
Research.
Berliner, D. C. (2011). Rational responses to high stakes testing: The case of curriculum
narrowing and the harm that follows. Cambridge Journal of Education, 41(3), 287-302.
Bernhardt, V. L. (2013). Translating data into information to improve teaching and learning.
Moorabbin, Vic: Hawker Brownlow.
Black, P., Harrison, C., Lee, C., Marshall, B., & Wiliam, D. (2004). Working inside the black
box: Assessment for learning in the classroom. Phi Delta Kappan, 86(1), 18-21.
Black, P., & Wiliam, D. (1998). Inside the Black Box: Raising Standards through Classroom
Assessment. The Phi Delta Kappan, 80(2), 139-144, 146-148.
Boudett, K. P., City, E. A., & Murnane, R. J. (2006). The "Data Wise" Improvement Process.
Principal Leadership, 7(2), 53-56.
140 | P a g e
Brunner, C., Fasca, C., Heinze, J., Honey, M., Light, D., & Mardinach, E. W., Dara. (2005).
Linking data and learning: The Grow Network Study. Journal of Education for Students
Placed at Risk (JESPAR), 10(3), 241-267.
Collay, M. (1998). Learning Circles: Creating conditions for professional development.
Thousand Oaks, CA: Corwin Press.
Darling-Hammond, L. (2000). Teacher quality and student achievement: A review of state
policy evidence. Education Policy Analysis Archives, 8(1).
Darling-Hammond, L. (2010). The flat world and education: How America's commitment to
equity will determine our future. New York: Teachers College Press.
Darling-Hammond, L., Ancess, J., & Falk, B. (1995). Authentic assessment in action: Studies of
schools and students at work. New York: Teachers College Press.
Datnow, A., Park, V., & Kennedy, B. L. (2008). Acting on data: How urban high schools use
data to improve instruction. Los Angeles: Center on Educational Governance, University
of Southern California.
Datnow, A., Park, V., & Wohlstetter, P. (2007). Achieving with Data: How high-performing
school systems use data to improve instruction for elementary students. California:
Center on Educational Governance, University of Southern California.
Deleuze, G. (Ed.). (1995). Negotiations 1972-1990. New York: Columbia University Press.
Dixon, H. R., Hawe, E., & Parr, J. (2011). Enacting assessment for learning: The beliefs
practice nexus. Assessment in Education: Principles, Policy & Practice, 18(4), 365-379.
Dole, S., Hilton, A., Hilton, G., & Goos, M. (2013, 17-20 March). Essential curriculum:
Proportional reasoning for numeracy. Paper presented at the Oxford Educational
Research Symposium, Oxford, UK.
Earl, L., & Katz, S. (2002). Leading schools in a data-rich world. In K. Leithwood & P. Hallinger
(Eds.), Second international handbook of educational leadership and administration (pp.
1003-1024). Dordrecht, Netherlands: Kluwer.
Education Commission of the States. (n.d.). No Child Left Behind issue brief: Data-driven
decisionmaking. from http://www.ecs.org/clearinghouse/35/52/3552.pdf
Education Queensland. (2013). Using student data. Retrieved 2 December, 2013, from
http://www.learningplace.com.au/default_community.asp?orgid=126&suborgid=869
Education Services Australia. (n.d.). Assessment for learning. Retrieved 1 August, 2013,
from http://www.assessmentforlearning.edu.au/default.asp
141 | P a g e
Even, R. (2005). Using Assessment to Inform Instructional Decisions: How Hard Can It Be?
Mathematics Education Research Journal, 17(3), 45-61.
Feldman, J., & Tung, R. (2001). Whole school reform: How schools use the data-based inquiry
and decision making process. Paper presented at the American Educational Research
Association, Seattle.
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.78.1885&rep=rep1&type=p
df
Fullan, M. (2001). The new meaning of educational change (3 ed.). New York: Teachers
College Press.
Fullan, M., Hill, P., & Crevola, C. (2006). Breakthrough. Thousand Oaks, CA: Corwin.
Gale, T. (2006). How did we ever arrive at the conclusion that teachers are the problem?: A
critical reading in the discourses of Australian schooling. English in Australia, 41(2), 12-
26.
Gipps, C. V. (1994). Beyond testing: Towards a theory of educational assessment. London:
FalmerPress.
Goos, M., Mills, M., Gilbert, R., Gowlett, C., Tony, W., Renshaw, P., . . . Honan, E. (2008).
Queensland Curriculum, Assessment and Reporting (QCAR) Framework Evaluation
Project (Phase 1). Brisbane: Department of Education, Training and the Arts.
Halsam, F., & Treagust, D. (1987). Diagnosing secondary students’ misconceptions of
photosynthesis and respiration in plants using a two-tier multiple choice instrument.
Journal of Biological Education, 21, 203-211.
Halverson, R. (2010). School formative feedback systems. Peabody Journal of Education,
85(2), 130-146.
Hamilton, L., Halverson, R., Jackson, S., Mandinach, E., Supovitz, J., & Wayman, Jeffrey C.
(2009). Using student achievement data to support instructional decision making (NCEE
2009-4067). Washington, DC: National Center for Education Evaluation and Regional
Assistance, Institute of Education Sciences, U.S. Department of Education.
Hardy, I., & Grootenboer, P. (in press). Schools, teachers and community: Cultivating the
conditions for engaged student learning. Journal of Curriculum Studies.
Hargreaves, A. (1994). Changing teachers, changing times: Teachers' work and culture in the
postmodern age. London: Continuum.
142 | P a g e
Hargreaves, A., Earl, L., & Schmidt, M. (2002). Perspectives on alternative assessment
reform. American Educational Research Journal, 39(1), 69-95.
Hattie, J. (2009). Visible Learning: A synthesis of over 800 meta-analyses relating to
achievement. London: Routledge.
Hattie, J. (2012). Visible learning for teachers: Maximizing impact on learning. London:
Routledge.
Hattie, J., & Jaeger, R. (1998). Assessment and classroom learning: a deductive approach.
Assessment in Education, 5(1), 111-122.
Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research,
77(1), 81-112.
Hayes, D., Mills, M., Christie, P., & Lingard, B. (2006). Teachers and schooling: Making a
difference. Crows Nest, AU: Allen & Unwin.
Hill, M. F. (2011). ‘Getting traction’: Enablers and barriers to implementing Assessment for
Learning in secondary schools. Assessment in Education: Principles, Policy & Practice,
18(4), 347-364.
Hilton, A., Hilton, G., Dole, S., & Goos, M. (in press). Development and application of a two-
tier diagnostic instrument to assess middle years students’ proportional reasoning.
Mathematics Education Research Journal, accepted February 13, 2013.
Huffman, D., & Kalnin, J. (2003). Collaborative inquiry to make data-based decisions in
schools. Teaching and Teacher Education, 19, 569-580.
Ingram, D., Seashore Louis, K., & Schroeder, R. G. (2004). Accountability policies and teacher
decision making: Barriers to the use of data to improve practice. Teachers College
Record, 106(6), 1258–1287.
Johnston, P. (2003). Assessment conversations. The Reading Teacher, 57(1), 90-92.
Kennedy, B. L., & Datnow, A. (2011). Student involvement and data-driven decision making:
Developing a new typology. Youth Society, 43(4), 1246-1271.
Kerr, K. A., Marsh, J. A., Schuyler Ikemoto, G., Darilek, H., & Barney, H. (2006). Strategies to
promote data use for instructional improvement: Actions, outcomes, and lessons from
three urban districts. American Journal of Education, 112(4), 496-520.
Kirkup, C. (2006). Using assessment information to inform teaching and learning. Education
3-13: International Journal of Primary, Elementary and Early Years Education, 34(2),
153-162.
143 | P a g e
Kirkup, C., Sizmur, J., Sturman, L., & Lewis, K. (2005). Schools’ use of data in teaching and
learning. England: National Foundation for Educational Research.
Klenowski, V. (2011). Assessment for learning in the accountability era: Queensland,
Australia. Studies in Educational Evaluation, 37, 78-83.
Klenowski, V., & Funnell, B. (2013). Exploring the conditions to support assessment for more
equitable learning outcomes. Curriculum Perspectives, 33(3), 33-45.
Lachat, M. A., & Smith, S. (2005). Practices that support data use in urban high schools.
Journal of Education for Students Placed at Risk, 10(3), 333-349.
Lingard, B. (2011a, 22-23 August). Making multiple differences: Schooling for equitable
futures. Paper presented at the Biennial Equity Conference 2011Schooling for Futures,
Sydney.
Lingard, B. (2011b, 14-15 July). Structuring, governing and managing the system for equity.
Paper presented at the Australian College of Educators 2011 National Conference,
Sydney.
Lingard, B., & Sellar, S. (2013). 'Catalyst data’: Perverse systemic effects of audit and
accountability in Australian schooling. Journal of Education Policy, 28(5), 634-656.
Little, J. W., Gearhart, M., Curry, M., & Kafka, J. (2003). Looking at student work for teacher
learning, teacher community, and school reform. The Phi Delta Kappan, 85(3), 184-192.
Love, N. (2004). Taking data to new depths. Journal of Staff Development, 25(4), 22-26.
Marsh, J. A. (2012). Interventions promoting educators’ use of data: Research insights and
gaps. Teachers College Record, 114(11), 1-48.
Marsh, J. A., Pane, J. F., & Hamilton, L. S. (2006). Making sense of data-driven decision
making in education. Santa Monica, CA: RAND Corporation.
Matters, G. (2006). Using data to support learning in schools: Students, teachers, systems.
Camberwell, Vic: Australian Council for Educational Research.
Maxwell, G. S. (2002). Moderation of teacher judgments in student assessment. Brisbane:
Queensland School Curriculum Council.
Means, B., Padilla, C., DeBarger, A., & Bakia, M. (2009). Implementing data-informed
decision making in schools—Teacher access, supports and use. California: U.S.
Department of Education Office of Planning, Evaluation and Policy Development.
Mills, M., Goos, M., Monk, S., Muspratt, S., Renshaw, P., Gilbert, R., . . . K, S. (2012). A
curriculum in transition: Final report for the Queensland Curriculum, Assessment and
144 | P a g e
Reporting Framework (QCAR) Evaluation. Brisbane: Queensland Department of
Education and Training.
Moon, T. R. (2005). The role of assessment in differentiation. Theory into Practice, 44(3),
226-233.
New Zealand Ministry of Education. (2013). Assessment online. Retrieved 12 August, 2013,
from http://assessment.tki.org.nz/
Newmann, F. M., King, M. B., & Rigdon, M. (1997). Accountability and school performance:
Implications from restructuring schools. HARVARD EDUCATIONAL REVIEW, 67(1), 41-74.
Northern Territory Government. (2012). 2012 NAPLAN: Data analysis guide. Retrieved 12
August, 2013, from www.det.nt.gov.au/teachers-educators/assessment-reporting
OECD. (2005). Formative assessment: Improving learning in secondary classrooms Paris:
OECD.
Osborne, J., Simon, S., Christodoulou, A., Howell-Richardson, C., & Richardson, K. (2013).
Learning to argue: A case study of four schools and their attempt to develop the use of
argumentation as a common instructional practice and its impact on students. Journal
of Research in Science Teaching, 50(3), 315-347.
Pedagogical reflections. (2013). Pedagogical reflections blog. Retrieved 6 December, 2013,
from http://jperk30.edublogs.org/category/new-school/
Pentti, A. (2004). Assessment in the context of moderation. Curriculum Matters:
Assessment, monitoring and reporting, 3(2), 18-19.
Pettit, P. (2010). From Data-informed to Data-led?: School Leadership within the Context of
External Testing. Leading and Managing, 16(2), 90-107.
Pierce, R., Chick, H., & Gordon, I. (2013). Teachers' perceptions of the factors influencing
their engagement with statistical reports on student achievement data. Australian
Journal of Education, 57(3), 237-255.
Polesel, J., Dulfer, N., & Turnbull, M. (2012). The experience of education: The impacts of
high stakes testing on school students and their families. Sydney, AU: The Whitlam
Institute.
Power, M. (1999). The audit society: Rituals of verification. Oxford, UK: Oxford University
Press.
QSA. (2010). School-based assessment: The Queensland system. Brisbane: Queensland
Studies Authority.
145 | P a g e
QSA. (2013). Senior moderation hub. Retrieved 2 December, 2013, from
http://www.qsa.qld.edu.au/586.html
QTAC. (2013). 2012 Australian Year 12 Conversion Table. Retrieved 2 December, 2013,
from http://www.qtac.edu.au/Applying-CurrentYr12/InterstateAdmissions.html
Queensland Studies Authority. (2012). 2012 NAPLAN: Test reporting handbook. Brisbane:
Queensland Studies Authority.
Quint, J. C., Sepanik, S., & Smith, J. K. (2008). Using student data to improve teaching and
learning: Findings from an evaluation of the Formative Assessments of Students
Thinking in Reading (FAST-R) Program in Boston elementary schools., from
www.mdrc.org
Rathvon, N. (2013). Developmental Reading Assessment Review. Retrieved 6 December,
2013, from http://www.natalierathvon.com/images/DRA_Review-08-25-2006.pdf
Sadler, D. R. (2013). Assuring academic achievement standards: From moderation to
calibration. Assessment in Education: Principles, Policy & Practice, 20(1), 5-19.
Sahlberg, P. (2007). Education policies for raising student learning: The Finnish approach.
Journal of Education Policy, 22(2), 147-171.
Sahlberg, P. (2010). Rethinking accountability in a knowledge society. Journal of Educational
Change, 11(1), 45-61.
Schildkamp, K., Lai, M. K., & Earl, L. (Eds.). (2013). Data-based decision making in education:
Challenges and opportunities. London: Springer.
Sharratt, L., & Fullan, M. (2012). Putting faces on the data. Thousand Oaks, CA: Corwin.
Shepard, L. A. (2000). The Role of Assessment in a Learning Culture. Educational Researcher,
29(7), 4-14.
Sowder, J. (2007). The mathematics education and development of teachers. In F. K. Lester
Jnr. (Ed.), Second handbook of research on mathematics teaching and learning (pp. 157-
223). Reston, VA: National Council of Teachers of Mathematics.
Stiggins, R. (1995). Assessment literacy for the 21st century. Phi Delta Kappan, 77(3), 238-
245.
Stiggins, R. (2002). Assessment crisis: The absence of Assessment FOR Learning. Phi Delta
Kappan, 83(10), 758-765.
Stiggins, R., & Duke, D. (2008). Effective Instructional Leadership Requires Assessment
Leadership. The Phi Delta Kappan, 90(4), 285-291.
146 | P a g e
Stobart, G. (2008). Testing times: The uses and abuses of assessment. London: Routledge.
Supovitz, J. A., & Klein, V. (2003). Mapping a Course for Improved Student Learning: How
Innovative Schools Systematically Use Student Performance Data to Guide
Improvement. Pennsylvania: Consortium for Policy Research in Education.
Swaffield, S. (2011). Getting to the heart of authentic Assessment for Learning. Assessment
in Education: Principles, Policy & Practice, 18(4), 433-449.
Thompson, G. (2012). Effects of NAPLAN (National Assessment Program - Literacy and
Numeracy): Executive summary. Retrieved 25 October, 2013, from
http://effectsofnaplan.edu.au/wp-content/uploads/2012/09/Executive-
Summary.pdf?lid=14
Timperley, H. (2005). Instructional Leadership Challenges: The Case of Using Student
Achievement Information for Instructional Improvement. Leadership and Policy in
Schools, 4(1), 3-22.
Timperley, H., & Parr, J. (2004). Using evidence in teaching practice: Implications for
professional learning. Auckland, NZ: Hodder Moa Beckett.
Timperley, H., Wilson, A., Barrar, H., Fung, I., & University of Auckland. (2007). Teacher
Professional Learning and Development: Best Evidence Synthesis Iteration [BES].
Auckland: New Zealand Ministry of Education.
Tozer, L., & Holmes, M. (2005). Moving on from Count Me In Too: Evidence-based teaching
and learning in numeracy in the early and middle years of schooling. Paper presented at
the Using data to support learning, Melbourne.
http://www.acer.edu.au/documents/RC2005_LTozerandMHolmes.pdf
Tuysuz, C. (2009). Development of a two-tier diagnostic instrument to assess students’
understanding of chemistry. Scientific Research and Essay, 4(6), 626-631.
van Barneveld, C. (2008). Using data to improve student achievement. Retrieved 7 August,
2013, from
http://www.edu.gov.on.ca/eng/literacynumeracy/inspire/research/using_data.pdf
Watson, J., & Beswick, K. (2011). School pupil change associated with a continuing
professional development programme for teachers. Journal of Education for Teaching,
37(1), 63-75.
147 | P a g e
Wayman, Jeffrey C. (2005). Involving Teachers in Data-Driven Decision Making: Using
Computer Data Systems to Support Teacher Inquiry and Reflection. Journal of Education
for Students Placed at Risk (JESPAR), 10(3), 295-308.
Wayman, Jeffrey C., Cho, V., & Johnston, M. T. (2007). The data-informed district: A district-
wide evaluation of data use in the Natrona County School District. Austin, TX: The
University of Texas.
Wayman, Jeffrey C., & Stringfield, S. (2006). Technology‐supported involvement of entire
faculties in examination of student data for instructional improvement. American
Journal of Education, 112(4), 549-571. doi: 10.1086/505059
Wildy, H. (2004). Designing assessments for instruction and accountability: An application of
validity theory to assessing scientific inquiry. Yearbook of the National Society for the
Study of Education, 103(2), 155–168.
Wiliam, D. (2011). Embedded formative assessment. Bloomington, US: Solution Tree.
Williams, J., & Ryan, J. (2000). National testing and the improvement of classroom teaching:
Can they coexist? British Educational Research Journal, 26(1), 49-73.
Wu, M. (2010). Measurement, sampling, and equating errors in large-scale assessments.
Educational Measurement: Issues and Practice, 29(4), 15-27.
Wyatt-Smith, C., & Klenowski, V. (2013). Explicit, latent and meta-criteria: Types of criteria at
play in professional judgement practice. Assessment in Education: Principles, Policy &
Practice, 20(1), 35-52.
Young, V. M. (2006). Teachers’ Use of Data: Loose Coupling, Agenda Setting, and Team
Norms. American Journal of Education, 112(4), 521-548.
148 | P a g e
Appendix A: Annotated bibliography
ACARA. (2011). Australian Curriculum, Assessment and Reporting Authority. Retrieved 12
August, 2013, from http://www.acara.edu.au/assessment/assessment.html
ACARA is the independent authority responsible for the development of a national
curriculum, a national assessment program, and a national data collection and reporting
program that supports 21st century learning for Australian students.
Allal, L. (2013). Teachers’ professional judgement in assessment: A cognitive act and a socially
situated practice. Assessment in Education: Principles, Policy & Practice, 20(1), 20-34.
This paper presents a study of teachers’ professional judgement in the area of summative
assessment. It adopts a situated perspective on assessment practices in classroom and
school settings. The study is based on interviews with 10 sixth-grade teachers and on the
assessment documents they used when determining end-of-term grades in students’
report cards. The main findings from qualitative data analysis highlight both the individual
cognitive and the socially situated aspects of teachers’ judgements. The findings are
discussed with respect to three levels of teacher judgement and the implications for
activities of social moderation.
Allen, R. (2005). Using the evidence of student achievement for improvements at individual,
class and school level. Paper presented at the Using data to support learning, Melbourne.
Retrieved September, 2013 from http://www.acer.edu.au/documents/RC2005_RegAllen.pdf
Techniques using student work as direct and visible evidence of achievement, of the
repertoires of practice of students and teachers, provide a powerful opportunity for
teachers and schools seeking to improve the learning of the students they have. This is a
purpose different from that of the analyst modelling patterns in large data sets of test
scores or the concerns with complex causality found in small-n studies and the methods
consequently differ. Critical elements of techniques for using student work include the
value of seeking a student, rather than subject or teacher, perspective, open to both the
official – what is recognised as part of school – and the unofficial – unrecognised factors
that underpin students’ practices. This paper describes the nature, use and importance of
some powerful techniques through which teachers can use data to improve student
learning.
American Association of School Administrators. (2002). Using data to improve schools: What’s
working. Retrieved 7 August, 2013, from
http://www.aasa.org/uploadedFiles/Policy_and_Advocacy/files/UsingDataToImproveSchools.
pdf
School system leaders are discovering the power of data for promoting school
improvement. With recent advances in technology and the increased demand for assessing
149 | P a g e
student learning, an unprecedented amount of data are available to educators. School
districts across America are beginning to use the tools necessary to make effective use of
the data. In addition to test scores, many educators are collecting data about citizenship,
character, healthy lifestyles, school climate and parental and community involvement.
Using Data to Improve Schools: What’s Working is an easy-to-read guide to using data to
drive school improvement. School system leaders and their staffs can learn from this book
how to build a district-wide culture of inquiry that values the use of data for sound
decision-making. School board members, parents and community members interested in
helping improve schools will find tools for their work as well in this guide. It describes the
challenges and the successes of educators from districts both large and small committed to
using data.
Armstrong, J., & Anthes, K. (2001). How data can help: Putting information to work to raise
student achievement. American School Board Journal, 188(11), 38-41.
What do you do with the reams of information your state sends every year, showing how
your students performed on the state’s student assessment program? Some school
districts take this information—and add more of their own—to improve their curriculum,
their teaching strategies, and their overall student achievement. As researchers at the
Education Commission of the States (ECS), we wanted to understand how districts can use
data most effectively. So this past spring we set out to conduct interviews in six school
districts in five different states (California, Colorado, Iowa, Maryland, and Texas) that had
reputations as exemplary data users.
Axworthy, D. (2005). Turning data into information that improves learning: The WA
experience. Paper presented at the Using data to support learning, Melbourne. Retrieved
September 2013, from www.acer.edu.au/documents/RC2005_DavidAxworthy.pdf
This paper will look at some examples of the way in which the Western Australian
Department of Education and Training is presenting student performance data and
transforming it into information to assist teachers to modify their teaching practices and
improve the learning of their students.
Barnes, F. D. (2004). Inquiry and action: Making school improvement part of daily practice.
Providence, RI: Annenberg Institute for School Reform, Brown University.
The School-Improvement Guide describes a school self-study cycle of inquiry and action,
designed to help a school community develop the habits of collaboration, discussion,
inquiry, and data-informed decision making that fuel ongoing improvement.
150 | P a g e
convince the public that all is well in the schools. It can be argued that schools do not
actually need drastic reform but merely steady improvement at the instructional level.
Bernhardt, V. L. (2009). Data, data everywhere: Bringing all the data together for continuous
school improvement. Larchmont, NY: Eye on Education.
This book describes what it takes to increase student achievement at every grade level,
subject area, and student group. Readers will learn how to use data to drive their
continuous improvement process as they develop an appreciation of the various types of
data, uses for data, and how data are involved with the school improvement process.
Bernhardt, V. L. (2009). From questions to actions: Using questionnaire data for continuous
school improvement. Larchmont, NY: Eye On Education.
Victoria L. Bernhardt and Bradley J. Geise explain how to collect and analyse data with an
eye toward positive change. In addition to gaining an overview of the questionnaire
process, you'll learn to start with your survey's purpose, create questionnaires that get
valuable answers, analyse data, share results in a dynamic way, use meaningful data to
understand the needs of teachers and students, and then implement targeted
improvement plans.
Bernhardt, V. L. (2013). Translating data into information to improve teaching and learning.
Moorabbin, Vic: Hawker Brownlow.
This book helps educators think through the selection of the data elements and data tools
needed to support quality decisions for improving teaching and learning. It shows you how
to use data to help make decisions about strategies to improve student achievement.
Black, P., Harrison, C., Lee, C., Marshall, B., & Wiliam, D. (2003). Assessment for learning:
Putting it into practice. Maidenhead: Open University Press.
This book gives teachers, school heads, and other leaders and trainers ideas and advice
about improving formative assessment in the classroom, based on two years of work in a
project that involved the team of authors at King's College working with 36 teachers in
schools.
Black, P., Harrison, C., Lee, C., Marshall, B., & Wiliam, D. (2004). Working inside the black box:
Assessment for learning in the classroom. Phi Delta Kappan, 86(1), 18-21.
In their widely read article “Inside the Black Box,” Mr. Black and Mr. Wiliam demonstrated
that improving formative assessment raises student achievement. Now they and their
colleagues report on a follow-up project that has helped teachers change their practice and
students change their behavior so that everyone shares responsibility for the students’
learning.
151 | P a g e
Black, P., & Wiliam, D. (1998). Inside the Black Box: Raising Standards through Classroom
Assessment. Phi Delta Kappan, 80(2), 139-144, 146-148.
Firm evidence shows that formative assessment is an essential component of classroom
work and that its development can raise standards of achievement, Mr. Black and Mr.
Wiliam point out. Indeed, they know of no other way of raising standards for which such a
strong prima facie case can be made.
Black, P., & Wiliam, D. (2005). Lessons from around the world: How policies, politics and
cultures constrain and afford assessment practices. Curriculum Journal, 16(2), 249-261.
This article outlines the main assessment traditions in four countries – England, France,
Germany and the United States – in order to explore the prospects for the integration of
summative and formative functions of assessment during compulsory schooling. In
England, teachers’ judgments do feed into national assessments, at 7, 11, 14 and 16, but
concerns for reliability and accountability mean that such judgments are made in a way
that has little impact on learning. In France, teachers have no involvement in the formal
assessment of students, and, possibly as a result, have been free to concentrate on the use
of assessment to serve learning. In Germany, faith in the education system has been
considerably undermined by recent unfavourable international comparisons, although faith
in the ability of tests both to measure learning accurately and to allocate students to
different educational pathways appears to be unshaken. In the United States, multiple
demands for accountability at different levels of the system have resulted in multiple
assessment systems, but these tend to be focused on measuring the amount of learning
that has taken place, providing little insight into how it might be improved. It is concluded
that the effective integration of formative and summative functions of assessment will
need to take different forms in different countries, and is likely to be extremely difficult.
Bobis, J. (2009). Count me in too: The learning framework in number and its impact on teacher
knowledge and pedagogy. Sydney: NSW Department of Education & Training.
This study is concerned with teacher professional learning and the impact of this learning
on teaching practices. Its focus is on teacher knowledge of the Learning Framework In
Number [LFIN] from the Count Me In Too [CMIT] numeracy project operating in
Department of Education & Training (DET) schools across New South Wales (NSWDET,
2007). In particular, the study addresses the following research questions: 1. What are
teachers’ perceptions about the degree to which CMIT is being implemented at the school
and classroom levels? 2. What are teachers’ perceptions about the extent of their
knowledge of the Learning Framework In Number? 3. Do teachers perceive that the
Learning Framework In Number has impacted on teaching practices at the school,
classroom and student levels? If so, how? If not, why? 4. How confident do teachers feel
about identifying children’s levels of mathematical development on the LFIN? 5. To what
extent is the CMIT planning matrix a useful tool for identifying the level of reported
implementation of the program at the school and classroom levels?
152 | P a g e
Bobis, J., Clarke, B., Clarke, D., Gould, P., Thomas, G., Wright, B., & Young-Leveridge, J. (2005).
Supporting teachers in the development of young children’s mathematical thinking: Three
large scale cases. Mathematics Education Research Journal, 16(3), 27–57.
Recognition of the importance of the early childhood years in the development of
numeracy is a significant characteristic of the New Zealand Numeracy Development
Project, the Victorian Early Numeracy Research Project and the Count Me In Too program
in New South Wales, Australia. This article outlines the background, key components and
major impacts of these three innovative and successful professional development and
research initiatives. Juxtaposing the three projects highlights important commonalities—
research-based frameworks, diagnostic interviews, and whole-school approaches to
professional development. Each program has been significant in rethinking what
mathematics and how mathematics is taught to young children.
Boomer, G., Lester, N., Onore, C., & Cook, J. (Eds.). (1992). Negotiating the curriculum:
Educating for the 21st century. London: Falmer.
This work presents an ongoing international dialogue about the theory and practice of
curriculum negotiating in the classroom at elementary, primary, secondary and university
levels.
Boudett, K. P., City, E. A., & Murnane, R. J. (Eds.). (2005). Data Wise: A step-by-step guide to
using assessment results to improve teaching and learning. Cambridge, MA: Harvard
Education Press.
In the wake of the accountability movement, school administrators are inundated with
data about their students. How can they use this information to support student
achievement? This book presents a clear and carefully tested blueprint for school leaders.
It shows how examining test scores and other classroom data can become a catalyst for
important schoolwide conversations that will enhance schools' ability to capture teachers'
knowledge, foster collaboration, identify obstacles to change, and enhance school culture
and climate.
Boudett, K. P., City, E. A., & Murnane, R. J. (2006). The "Data Wise" Improvement Process.
Principal Leadership, 7(2), 53-56.
The barriers to constructive, regular use of student assessment data to improve instruction
can seem insurmountable. There is just so much data. Where do you start? How do you
make time for the work? How do you build your faculty’s a culture that focuses on
improvement, not blame? How do you maintain momentum in the face of all the other
demands at your school?
153 | P a g e
Boudett, K. P., & Steele, J. L. (Eds.). (2007). Data Wise in action: Stories of schools using data
to improve teaching and learning. Cambridge, MA: Harvard Education Press.
What does it look like when a school uses data wisely? "Data Wise in Action", a new
companion and sequel to the bestselling "Data Wise", tells the stories of eight very
different schools following the Data Wise process of using assessment results to improve
teaching and learning. "Data Wise in Action" highlights the leadership challenges schools
face in each phase of the eight-step Data Wise cycle and illustrates how staff members use
creativity and collaboration to overcome those challenges. "Data Wise in Action" builds on
the work of leading faculty and graduate students at the Harvard Graduate School of
Education, who joined with exemplary practitioners in 2005 to produce "Data Wise: A Step-
by-Step Guide to Using Assessment Results to Improve Teaching and Learning". Since its
publication, "Data Wise" has been read by thousands of school leaders, many of whom
have shared the book with colleagues and staff. The success of the original book has
generated a new demand among school leaders: to hear real stories from schools that are
implementing the Data Wise process. "Data Wise in Action" answers that need. It offers
both inspiration and practical guidance for school leaders.
Brunner, C., Fasca, C., Heinze, J., Honey, M., Light, D., & Mardinach, E. W., Dara. (2005).
Linking data and learning: The Grow Network Study. Journal of Education for Students Placed
at Risk (JESPAR), 10(3), 241-267.
During the last decade, standards, assessments, and accountability have emerged as three
prongs of a national education reform movement that has asked district and school
administrators to think very differently about educational decision making and the use of
data. However, research about data-driven decision making is limited, especially
concerning teachers. This article describes findings from a 2-year exploratory study that
examined how educators within the New York City public school system are using data—
made available to teachers through the print- and Web-based reporting systems of the
Grow Network—to inform decisions about teaching and learning. In this article, we
summarize what we learned about the specific ways in which teachers and administrators
make use of the Grow Reports® to inform educational practices.
Coburn, C. E., & Talbert, J. E. (2006). Conceptions of Evidence Use in School Districts: Mapping
the Terrain. American Journal of Education, 112(4), 469-495.
Current policies place unprecedented demands on districts to use evidence to guide their
educational improvement efforts. How districts respond is likely to be influenced by how
individuals in the district conceptualize what it means to use evidence in their ongoing
work. This study draws on sense-making and institutional theory to investigate how
individuals in one urban school district conceive of evidence-based practice. The study
develops grounded typologies that describe the ways that individuals conceptualize high-
quality evidence, appropriate evidence use, and high-quality research. It then explains
variation in conceptions, pointing to the ways organizational responsibilities and reform
154 | P a g e
history shape how individuals come to understand evidence-based practice. The article
closes by suggesting implications for district response to federal policy demands for
evidence-based practice.
Darling-Hammond, L., Ancess, J., & Falk, B. (1995). Authentic assessment in action: Studies of
schools and students at work. New York: Teachers College Press.
This book examines, through case studies of elementary and secondary school classrooms,
how five schools have developed "authentic," performance-based assessments of students'
learning, and how this work has influenced the teaching and learning experiences students
encounter in school. This important and timely book reveals the changing dynamics of
classroom life as it moves from more traditional pedagogy to one which asks students to
master intellectual and practical skills that are eminently "transferable" to "real-life" social
settings and workplaces.
Datnow, A., Park, V., & Kennedy, B. L. (2008). Acting on data: How urban high schools use
data to improve instruction. Los Angeles: Center on Educational Governance, University of
Southern California.
The above statement is from a principal in a high-performing, urban charter high school
where educators are committed to using data to inform their instruction. As she suggests, a
“data-driven” teacher uses formative assessment data on a regular basis to make
adjustments to his or her instructional plan. Moreover, a teacher truly committed to
improving student achievement is not willing to settle, but expects all of his or her students
to reach high standards. In recent years, an increasingly clear and persuasive body of
research is demonstrating what common sense tells us: that high-performing schools and
school systems use student data in all facets of their work to continuously inform and
improve their instruction. Successful practices that involve the instructional uses of data
have been demonstrated and documented in elementary schools. At the high school level,
these practices appear to be more difficult to design and to take longer to implement in
replicable, consistent, and successful ways. Despite this, several pioneering secondary
schools—and the school systems of which they are part—are making significant inroads in
using data to improve instruction and hence to improve student outcomes. While still
imperfect and occasionally idiosyncratic, these exemplars offer valuable lessons for all
secondary schools relentlessly focused on improving their students’ achievement. This
study of data-driven instructional decision making at the secondary level examined four
urban high schools and districts across the U.S. where instructional data practices are
taking hold; each was identified as a leader in this area. Our study included two high
schools belonging to traditional school districts and two that were part of nonprofit charter
school management organizations (CMOs). All of these schools have records of improving
student achievement over time.
155 | P a g e
Datnow, A., Park, V., & Wohlstetter, P. (2007). Achieving with Data: How high-performing
school systems use data to improve instruction for elementary students. California: Center on
Educational Governance, University of Southern California.
Using data to improve decision making is a promising systemic reform strategy. However,
there is a dearth of rigorous research conducted thus far on this practice. Recently, New
Schools Venture Fund in San Francisco set an agenda to help fill this research gap. As part
of a study of data-driven decision making, we were fortunate to visit schools and districts
where practices, such as the one depicted in the above quote, are indeed becoming
commonplace. In this report, we capture the work of four school systems that were
identified as leaders in data-driven decision making. Our study included two mid-size urban
school districts and two non-profit charter management organizations (CMOs). All of these
school systems have records of improving student achievement over time.
Dixon, H. R., Hawe, E., & Parr, J. (2011). Enacting assessment for learning: The beliefs practice
nexus. Assessment in Education: Principles, Policy & Practice, 18(4), 365-379.
Engagement in self and peer assessment are authentic ways in which students can develop
evaluative and productive knowledge and expertise, necessary prerequisites if they are to
become autonomous learners. Teachers in the current study who had articulated similar
beliefs in regard to the importance of developing student autonomy and who had
described similar practices to develop self-monitoring behaviour were observed teaching a
written language unit. However, the ways these practices ‘played out’ in the classroom was
a matter of considerable variation in regard to the nature of the judgements made, the
degree of student involvement in evaluative and productive activities, and the amount of
control maintained by the teacher. Two particular cases are used to draw attention to
teachers’ espoused beliefs and their congruence with practice. Given the powerful role that
beliefs play in the enactment of specific assessment for learning practices,
recommendations for teacher professional development are made.
Earl, L. (2005). From accounting to accountability: Harnessing data for school improvement.
Paper presented at the Using data to support learning, Melbourne. Retrieved September
2013, from http://www.acer.edu.au/documents/RC2005_Earl.pdf
There was a time in education when decisions were based on the best judgements of the
people in authority. It was assumed that school leaders, as professionals in the field, had
both the responsibility and the right to make decisions about students, schools and even
about education more broadly. They did so using a combination of intimate and privileged
knowledge of the context, political savvy, professional training and logical analysis. Data
played almost no part in decisions. In fact, there was not much data available about
schools. Instead, leaders relied on their tacit knowledge to formulate and execute plans.
156 | P a g e
Earl, L., & Fullan, M. (2003). Using data in leadership for learning. Cambridge Journal of
Education, 33(3), 383-394. doi: 10.1080/0305764032000122023
School leaders are faced with the daunting task of anticipating the future and making
conscious adaptations to their practices, in order to keep up and to be responsive to the
environment. To succeed in a rapidly changing and increasingly complex world, it is vital
that schools grow, develop, adapt and take charge of change so that they can control their
own futures. This paper will examine the tension that exists for school leaders in relation to
data about their schools and their students, arguing that the explicit connections between
data and large-scale reforms make it impossible to avoid a critical approach to data,
drawing on research in Ontario and Manitoba in Canada, and examining parallels with
evidence from research in England, to highlight the challenges involved in using data
effectively in different political contexts and mandated policies on the uses of data.
Earl, L., & Katz, S. (2002). Leading schools in a data-rich world. In K. Leithwood & P. Hallinger
(Eds.), Second international handbook of educational leadership and administration (pp.
1003-1024). Dordrecht, Netherlands: Kluwer.
Each week we receive the Times Educational Supplement from England and Education
Week from the US. Between us, we also subscribe to several Canadian newspapers and a
number of educational journals and popular magazines from different countries. A quick
glance at any of these publications makes it very clear that there is no escaping the
presence of data in education. As researchers whose stock in trade is "data", we have
become increasingly interested in the role that data have to play in educational change,
particularly in how school and district leaders feel about, understand, and use the
mountains of data that are being generated about schools.
Earl, L., & Torrance, N. (2000). Embedding accountability and improvement into large-scale
assessment: What difference does it make? Peabody Journal of Education, 75(4), 114-141.
Assessment-led reform has become one of the most widely favored strategies to promote
more credible forms of public accountability (Black, 1998). Unlike the assessment agenda
of the 1960s and 1970s, assessment programs in the 1990s and beyond are part of a
broader scheme for changing education. Large-scale assessment has become the vehicle of
choice for accountability purposes around the world, and testing has become the lever for
holding schools accountable for results (Firestone, Mayrowetz, & Fairman, 1998).
Assessment reform, like other educational initiatives, is not singular, nor is it static. Like
other reform efforts, it is caught in a maelstrom of rapid change and uncertainty. The
knowledge base on assessment is being developed as the assessment procedures are being
implemented, and many states, provinces, and countries are on a quest for the "best"
approach.
157 | P a g e
Education Commission of the States. (n.d.). No Child Left Behind issue brief: Data-driven
decision-making, Retrieved September 2013, from
http://www.ecs.org/clearinghouse/35/52/3552.pdf
Nearly every state reports annually to districts on how well their schools and students are
meeting state standards. With schools being held accountable for helping all children
achieve state standards, and assessment data measuring how well schools and students
are meeting those standards, the question is: How can districts support schools’ use of
data, and what types of data can be used to make decisions that improve student and
school performance?
Education Services Australia. (n.d.). Assessment for learning. Retrieved 1 August, 2013, from
http://www.assessmentforlearning.edu.au/default.asp
The Assessment for Learning website has been developed by Curriculum Corporation on
behalf of the education departments of the States, Territories and Commonwealth of
Australia.
Eltis, K. J. (2003). Time to teach, time to learn: Report on the evaluation of outcomes
assessment and reporting in NSW government schools. State of NSW: Department of
Education and Training.
The Evaluation being reported on had its origins late in 2002 when the NSW Teachers
Federation approached the then Minister for Education and Training, the Hon John Watkins
MP, with the request that a study be undertaken of demands created for teachers as a
result of the introduction of outcomes assessment and reporting. The Minister agreed that
an Evaluation should take place and asked if I would conduct the study, having completed a
similar exercise in 1995. Agreement was reached on Terms of Reference and Associate
Professor Stephen Crump from the Faculty of Education and Social Work at the University
of Sydney agreed to work with me. We began our work in February 2003.
Even, R. (2005). Using assessment to inform instructional decisions: How hard can it be?
Mathematics Education Research Journal, 17(3), 45-61.
In this article, two problems associated with the expectation that teachers use
contemporary assessment techniques are examined. The first problem relates to teachers'
sense-making of assessment data. Illustrative cases revealed that teachers' processes of
interpretation of students' understanding, knowledge and learning of mathematics draws
on a rich knowledge base of understandings, beliefs, and attitudes. Consequently, the
process of sense-making of students' mathematical understandings involves ambiguity and
difficulty. The second problem relates to ways of helping teachers adopt contemporary
assessment approaches. A professional development activity served as the example
examined. Three aspects of what the course instructor promoted with respect to
contemporary assessment were analysed: (1) the assessment methods and tools advocated
in the course, (2) the degree to which the integration of assessment with instruction was
158 | P a g e
promoted, and (3) the purposes for assessment highlighted in the course. It appeared that
attention was paid to the use of contemporary assessment tools, but this was associated
with traditional assessment purposes. Learning to use the new assessment tool did,
however, influence instruction and fostered greater integration of assessment and
instruction than before--a characteristic of contemporary assessment. The article
concludes with a discussion of the current expectation that teachers use assessment data
to improve instruction.
Feldman, J., & Tung, R. (2001). Whole school reform: How schools use the data-based inquiry
and decision making process. Paper presented at the American Educational Research
Association, Seattle. Retrieved September 2013, from
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.78.1885&rep=rep1&type=pdf
In the current culture of high-stakes tests, school accountability, and standards, schools
are under increasing external pressure. Schools are inundated with a wide variety of data
and are looking for ways to understand how to interpret the data that is provided to them,
as well as how to use the process of inquiry to improve the quality of instruction offered by
their school. Many schools are currently working with data in limited ways, often as a
reactionary response to external pressure, with little thought given to what the process can
do for the school. Few schools use a process of data-based inquiry and decision making
(DBDM) which includes the whole faculty in looking at and thinking about data, with a goal
of creating an inquiry-minded school. “Inquiry-minded schools recognize that improving
teaching and learning is an intentional and ongoing process” (Rallis & MacMullen, 2000).
Frederiksen, J. R., & White, B. Y. (2004). Designing assessments for instruction and
accountability: An application of validity theory to assessing scientific inquiry. Yearbook of the
National Society for the Study of Education, 103(2), 74-104.
This chapter is concerned with how assessments of students’ work in classrooms, although
primarily intended to promote learning, can also become an important source of
information for evaluating a school’s effectiveness within an accountability system
(Shepard, 2000). On the face of it, formative assessment practices used in the classroom to
support learning and summative assessments used for accountability purposes seem to be
incompatible. In their classroom activities, students know ahead of time the tasks on which
they will be assessed, and they can prepare for them and get help in doing them. In
addition, teachers’ judgments of their students’ work could be influenced if their classroom
assessments were to be used for accountability purposes. For reasons such as these,
assessments used for accountability typically are external assessments based on tasks that
are not known to the students or the teacher; they often use items that are objectively
scored; and they are scored externally (Baker, Linn, Herman, & Koretz, 2002; Linn, 2000).
159 | P a g e
Fullan, M., Hill, P., & Crevola, C. (2006). Breakthrough. Thousand Oaks, CA: Corwin.
Breakthrough presents a revolutionary new approach to educational reform, breaking
away from the conventional paradigm to help educators create focused instruction,
transform the classroom experience, and dramatically raise, and sustain, performance
levels for students and teachers alike. This book provides the breakthrough concepts
needed for developing precise, validated, data-driven instruction personalised to each and
every student.
Gallagher, L., Means, B., & Padilla, C. (2008). Teachers’ use of student data systems to
improve instruction: 2005 to 2007. Washington, D.C.: US Department of Education, Office of
Planning, Evaluation and Policy Development.
The following findings are based on analyses of national survey data from district
technology coordinators and teachers from 2005 and 2007:
Gardner, J. (Ed.). (2013). Assessment and learning (2 ed.). Los Angeles: SAGE.
Key areas of assessment for learning covered in this volume include the practice of
assessment for learning in the classroom, developing motivation for learning, professional
learning and assessment for learning, and assessment and theories of learning.
160 | P a g e
Gorard, S. (2001). The role of secondary data in combining methodological approaches.
Educational Review, 54(3), 231-237.
This paper contains a plea for the greater use of numeric secondary data as a routine part
of all studies, whatever their primary method. It starts with a rehearsal of the current poor
public image of UK educational research, and some of the possible reasons for that. This
rehearsal includes consideration of the limitations in some examples of influential work by
established researchers. The paper continues with a summary of the reasons for using
secondary data, and one example of a project based solely on secondary data. It concludes
that the purportedly poor quality of some UK educational research, allied to the potential
of secondary data, might actually empower novice researchers, enabling them to critique
established work and to conduct powerful and informative analyses of their own.
Hall, K., Conway, P. F., Rath, A., Murphy, R., & McKeon, J. (2008). Reporting to parents in
primary school: Communication, meaning and learning. Dublin, Ireland: National Council for
Curriculum and Assessment (NCCA).
How schools report to parents about the learning of their children is becoming increasingly
important and challenging in the light of a) new developments and understanding about
learning and assessment, b) Ireland’s relatively recent cultural diversity, and c) recent
legislation and official policy highlighting how schools are accountable to students, parents
and the State. The NCCA’s Reporting Children’s Progress in Primary Schools endorses the
role of parents, as partners with schools, in extending children’s learning. School reporting
practices are central to this role. The nature of these practices is the theme of this NCCA-
commissioned study. In terms of assessment policy and practice, we note that reporting is
more closely linked with summative than formative assessment (as indicated in the shaded
column in Table 1). As such, in terms of formal reporting at both parent-teacher meetings
and in relation to written report cards the emphasis is on ‘what has been learned by
students to date’, that is, ‘assessment of learning’ (AoL).
161 | P a g e
Hamilton, L., Halverson, R., Jackson, S., Mandinach, E., Supovitz, J., & Wayman, J. (2009).
Using student achievement data to support instructional decision making (NCEE 2009-4067).
Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute
of Education Sciences, U.S. Department of Education.
As educators face increasing pressure from federal, state, and local accountability policies
to improve student achievement, the use of data has become more central to how many
educators evaluate their practices and monitor students’ academic progress. Despite this
trend, questions about how educators should use data to make instructional decisions
remain mostly unanswered. In response, this guide provides a framework for using student
achievement data to support instructional decision making. These decisions include, but
are not limited to, how to adapt lessons or assignments in response to students’ needs,
alter classroom goals or objectives, or modify student-grouping arrangements. The guide
also provides recommendations for creating the organizational and technological
conditions that foster effective data use. Each recommendation describes action steps for
implementation, as well as suggestions for addressing obstacles that may impede progress.
In adopting this framework, educators will be best served by implementing the
recommendations in this guide together rather than individually.
Hargreaves, A., Earl, L., & Schmidt, M. (2002). Perspectives on alternative assessment reform.
American Educational Research Journal, 39(1), 69-95.
This article examines classroom assessment reform from four perspectives: technological,
cultural, political, and postmodern. Each perspective highlights different issues and
problems in the phenomenon of classroom assessment. The technological perspective
focuses on issues of organization, structure, strategy, and skill in developing new
assessment techniques. The cultural perspective examines how alternative assessments are
interpreted and integrated into the social and cultural context of schools. The political
perspective views assessment issues as being embedded in and resulting from the
dynamics of power and control in human interaction. Here assessment problems are
caused by inappropriate use, political and bureaucratic interference, or institutional
priorities and requirements. Last, the postmodern perspective is based on the view that in
today's complex and uncertain world, human beings are not completely knowable and that
"authentic" experiences and assessments are fundamentally questionable. Using a semi-
structured interview protocol, teachers were asked about their personal understanding of
alternative forms of assessment; about how they had acquired this understanding; how
they integrated changes into their practices; what these practices looked like; what
successes and obstacles they encountered during implementation; and what support
systems had been provided for them.
162 | P a g e
Hattie, J. (2012). Visible learning for teachers: Maximizing impact on learning. London:
Routledge.
In November 2008, John Hattie's ground-breaking book Visible Learning synthesised the
results of more than fifteen years research involving millions of students and represented
the biggest ever collection of evidence-based research into what actually works in schools
to improve learning. Visible Learning for Teachers takes the next step and brings those
ground breaking concepts to a completely new audience. Written for students, pre-service
and in-service teachers, it explains how to apply the principles of Visible Learning to any
classroom anywhere in the world. The author offers concise and user-friendly summaries of
the most successful interventions and offers practical step-by-step guidance to the
successful implementation of visible learning and visible teaching in the classroom.
Hattie, J., & Anderman, E. M. (Eds.). (2013). International guide to student achievement. New
York: Routledge.
The International Guide to Student Achievement brings together and critically examines
the major influences shaping student achievement today. There are many, often
competing, claims about how to enhance student achievement, raising the questions of
"What works?" and "What works best?" World-renowned bestselling authors, John Hattie
and Eric M. Anderman have invited an international group of scholars to write brief,
empirically-supported articles that examine predictors of academic achievement across a
variety of topics and domains.
Rather than telling people what to do in their schools and classrooms, this guide simply
provides the first-ever compendium of research that summarizes what is known about the
major influences shaping students’ academic achievement around the world. Readers can
apply this knowledge base to their own school and classroom settings. The 150+ entries
serve as intellectual building blocks to creatively mix into new or existing educational
arrangements and aim for quick, easy reference. Chapter authors follow a common format
that allows readers to more seamlessly compare and contrast information across entries,
guiding readers to apply this knowledge to their own classrooms, their curriculums and
teaching strategies, and their teacher training programs.
Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research,
77(1), 81-112.
Feedback is one of the most powerful influences on learning and achievement, but this
impact can be either positive or negative. Its power is frequently mentioned in articles
about learning and teaching, but surprisingly few recent studies have systematically
investigated its meaning. This article provides a conceptual analysis of feedback and
reviews the evidence related to its impact on learning and achievement. This evidence
shows that although feedback is among the major influences, the type of feedback and the
way it is given can be differentially effective. A model of feedback is then proposed that
163 | P a g e
identifies the particular properties and circumstances that make it effective, and some
typically thorny issues are discussed, including the timing of feedback and the effects of
positive and negative feedback. Finally, this analysis is used to suggest ways in which
feedback can be used to enhance its effectiveness in classrooms.
Hill, M. F. (2009). Ways of seeing: using ethnography and Foucault's `toolkit' to view
assessment practices differently. Qualitative Research, 9(3), 309-330.
Tamboukou and Ball ask, what value can genealogy and/or ethnography add to the other?
This article illustrates, through an educational exemplar study, how being genealogically
driven can produce new ways of seeing and thinking about practices, within the field of
educational assessment. To date, neither the qualitative nor the quantitative methods
customarily applied to the assessment field have been able to illuminate why, since the late
1980s, accountability demands have caused New Zealand primary school teachers to
prioritize the use of summative classroom assessment practices when research indicates
that formative practices are clearly more productive of learning. Using ethnographic data
gathering techniques and grounded theory in combination with Foucauldian tools and
notions of genealogy, discourses, surveillance, and ‘the history of the present’, it is argued,
enabled new ways to think about why teachers have normalized particular assessment
practices in New Zealand’s self-managing schools. In short, this article argues that it is
extremely helpful to mix modernist ethnographic methods that focus on the micro-
practices of teaching with post-modernist theoretical tools in order to provide different
ways of seeing.
Hill, M. F. (2011). ‘Getting traction’: Enablers and barriers to implementing Assessment for
Learning in secondary schools. Assessment in Education: Principles, Policy & Practice, 18(4),
347-364.
While there is clear evidence that using classroom assessment for learning can improve
learning significantly, research has demonstrated that it is particularly challenging for
secondary teachers to change their practices in an assessment for learning direction. Many
factors have been suggested as inhibiting change in these schools. Some studies do link
improved student outcomes to changes in teaching brought about through professional
development. They suggest that the same factors enhance professional learning in primary
and secondary schools. However, the way large secondary schools are organised by subject
departments exerts enormous influence on how professional learning in the sector can and
should be structured. The results of three success case studies demonstrated the critical
impact of school-level factors on changing assessment for learning practices in secondary
schools. These factors enabled these three schools to address the departmental constraints
and supported cross-curricular professional learning in assessment for learning.
164 | P a g e
Huffman, D., & Kalnin, J. (2003). Collaborative inquiry to make data-based decisions in
schools. Teaching and Teacher Education, 19, 569-580.
The purpose of this study was to investigate the impact of a long-term collaborative
inquiry project for diverse teams of teachers, administrators, school board members, and
parents. The teams engaged in collaborative inquiry to collect and analyse local data to
make data based decisions about how to improve teaching and learning. The results
suggest the collaborative inquiry not only positively influenced the teachers, but also
helped them engage in a continuous improvement process that allowed them to take more
ownership over local data and expand their role in their schools’ decision-making
processes.
Ingram, D., Seashore Louis, K., & Schroeder, R. G. (2004). Accountability policies and teacher
decision making: Barriers to the use of data to improve practice. Teachers College Record,
106(6), 1258–1287.
One assumption underlying accountability policies is that results from standardized tests
and other sources will be used to make decisions about school and classroom practice. We
explore this assumption using data from a longitudinal study of nine high schools
nominated as leading practitioners of Continuous Improvement (CI) practices. We use the
key beliefs underlying continuous improvement derived from educational applications of
Deming’s TQM models and organizational learning to analyse teachers’ responses to
district expectations that they would use data to assess their own, their colleagues’, and
their schools’ effectiveness and to make improvements. The findings suggest that most
teachers are willing, but they have significant concerns about the kind of information that
is available and how it is used to judge their own and colleagues’ performance. Our analysis
reveals some cultural assumptions that are inconsistent with accountability policies and
with theories of continuous improvement and organizational learning. We also identify
barriers to use of testing and other data that help to account for the limited impacts.
Kelly, A., & Downey, C. (2010). Using effectiveness data for school improvement: Developing
and utilising metrics (eBook ed.). London: Taylor and Francis.
Data metrics in schools are becoming increasingly complex, but despite their best efforts,
teachers and academics generally find them something of a ‘black-box’. This book lifts the
lid on that box, exploring the provenance and problematization of existing techniques and
developing new algorithms for measuring the more oblique aspects of in-school
performance. Using contextual value-added measures in England as a foundation - they
have become the template of choice for policy-makers around the world and a basis for
some excellent school effectiveness research - the book explores the potential of
performance and progress data to guide student and teacher self-evaluation, to set targets
and allocate resources, to evaluate initiatives and identify good practice, to assess and
reward staff responsibility, and to inform policy in relation to emerging issues like school
choice, equality of opportunity and post-compulsory progression. Using Effectiveness Data
165 | P a g e
for School Improvement brings together for the first time in one place the various metrics
and models, and their basis in research. A full technical specification is included so that
both ‘data experts’ and ‘data novices’, academics and practitioners, can use the book to
understand and maximize what is potentially a hugely transforming, but under-utilized,
resource and an increasingly important aspect to school and curriculum management.
Kennedy, B. L., & Datnow, A. (2011). Student involvement and data-driven decision making:
Developing a new typology. Youth Society, 43(4), 1246-1271.
Existing literature supports the inclusion of students in education reform, documenting
benefits for both students and educators. When student voice is not included in reform
efforts, these efforts are more likely to flounder. The emerging educational reform of data-
driven decision making (DDDM) offers promise for increasing student achievement.
However, scant research documents the involvement of students in DDDM reforms. Using
a theoretical framework that advocates for democratically involving students in education
reform, this cross-case analysis examines the role of students in DDDM reforms in
elementary and high schools known to be exemplars of data-driven decision making. Based
on findings of efforts made by exemplar districts as well as actions they did not take to
involve students, the authors conclude that a new typology is necessary for assessing
student involvement in DDDM. Consequently, the authors propose a new three-tiered
typology for conceptualizing this phenomenon.
Kerr, K. A., Marsh, J. A., Schuyler Ikemoto, G., Darilek, H., & Barney, H. (2006). Strategies to
promote data use for instructional improvement: Actions, outcomes, and lessons from three
urban districts. American Journal of Education, 112(4), 496-520.
The current high-stakes accountability environment has created strong incentives for
educators to systematically collect and use data to inform instructional decisions. This
article examines the strategies employed by three urban school districts to promote data
use for instructional improvement and their effect on administrator, principal, and teacher
practice. Several factors are found to affect data use, including accessibility and timeliness
of data, perceptions of data validity, training, and support for teachers with regard to data
analysis and interpretation, and the alignment of data strategies with other instructional
initiatives.
Kirkup, C. (2006). Using assessment information to inform teaching and learning. Education 3-
13: International Journal of Primary, Elementary and Early Years Education, 34(2), 153-162.
The ways in which teachers and head teachers integrate external summative testing and
formative assessment practices were explored by means of a questionnaire survey and a
small number of case studies. A clear distinction was found between end of key stage tests
and optional tests in the extent to which they could be used directly to support teaching
and learning. Within the current context of high-stakes external testing, it was found that,
166 | P a g e
although qualitative information from formal tests was proving useful at school and class
level, potential benefits for individual pupils were not being fully realised.
Kirkup, C., Sizmur, J., Sturman, L., & Lewis, K. (2005). Schools’ use of data in teaching and
learning. England: National Foundation for Educational Research.
The National Foundation for Educational Research (NFER) was commissioned by the
Department for Education and Skills (DfES) to conduct a study of primary, secondary and
special maintained schools in England to assess the use of data in teaching and learning.
Klenowski, V., & Funnell, B. (2013). Exploring the conditions to support assessment for more
equitable learning outcomes. Curriculum Perspectives, 33(3), 33-45.
NAPLAN results have gained socio-political prominence and have been used as indicators of
educational outcomes for all students, including Indigenous students. Despite the promise
of open and in-depth access to NAPLAN data as a vehicle for intervention, we argue that
the use of NAPLAN data as a basis for teachers and schools to reduce variance in learning
outcomes is insufficient. NAPLAN tests are designed to show statistical variance at the level
of the school and the individual, yet do not factor in the sociocultural and cognitive
conditions Indigenous students' experience when taking the tests. We contend that further
understanding of these influences may help teachers understand how to develop their
classroom practices to secure better numeracy and literacy outcomes for all students.
167 | P a g e
Empirical research findings demonstrate how teachers can develop their classroom
practices from an understanding of the extraneous cognitive load imposed by test taking.
We have analysed Indigenous students' experience of solving mathematical test problems
to discover evidence of extraneous cognitive load. We have also explored conditions that
are more supportive of learning derived from a classroom intervention which provides an
alternative way to both assess and build learning for Indigenous students. We conclude
that conditions to support assessment for more equitable learning outcomes require a
reduction in cognitive load for Indigenous students while maintaining a high level of
expectation and participation in problem solving.
Lachat, M. A., & Smith, S. (2005). Practices that support data use in urban high schools.
Journal of Education for Students Placed at Risk, 10(3), 333-349.
This article presents initial findings of a case study focusing on data use in five low-
performing urban high schools undergoing comprehensive school-wide reform. The case
study investigates: (a) the ways in which disaggregated data can be used to examine
progress and guide improvement in the process of restructuring urban, low-performing
high schools; (b) factors and conditions that either promote or act as barriers to data use;
and (c) the policy and practice implications of achieving effective data use in a high school
reform process. Study findings point to several key factors that have an impact on data use
in the study sites: the quality and accuracy of available data, staff access to timely data, the
capacity for data disaggregation, and the collaborative use of data organized around a clear
set of questions, and leadership structures that support school-wide use of data. The
findings build on current literature and also contribute new knowledge of the key roles
played by a data team and a data coach in fostering effective data use in high school
reform.
Lesaux, N. K., & Marietta, S. H. (2012). Making assessment matter: using test results to
differentiate reading instruction. New York: Guilford Publications, Inc.
All too often, literacy assessments are given only for accountability purposes and fail to be
seen as valuable resources for planning and differentiating instruction. This clear, concise
book shows K-5 educators how to implement a comprehensive, balanced assessment
battery that integrates accountability concerns with data-driven instruction. Teachers learn
to use different types of test scores to understand and address students' specific learning
needs. The book features an in-depth case example of a diverse elementary school that
serves many struggling readers and English language learners.
168 | P a g e
the 1980s, and the standards-based accountability systems of the 1990s. Questions
regarding the impact, validity, and generalizability of reported gains, and the credibility of
results in high-stakes accountability uses are discussed. Emphasis is given to three issues
regarding currently popular accountability systems. These are (a) the role of content
standards, (b) the dual goals of high performance standards and common standards for all
students, and (c) the validity of accountability models. Some suggestions for dealing with
the most severe limitations of accountability are provided
Little, J. W., Gearhart, M., Curry, M., & Kafka, J. (2003). Looking at student work for teacher
learning, teacher community, and school reform. Phi Delta Kappan, 85(3), 184-192.
Teachers are usually alone when they examine student work and think about student
performance. The authors describe several projects that have enabled teachers to leave
the isolation of their own classrooms and think together about student work in the broader
contexts of school improvement and professional development
Love, N. (2004). Taking data to new depths. Journal of Staff Development, 25(4), 22-26.
There's a ton of data being collected. The trick is to know how to use it effectively.
Luke, A., Freebody, P., Shun, L., & Gopinathan, S. (2005). Towards research-based innovation
and reform: Singapore schooling in transition. Asia Pacific Journal of Education, 25(1), 5-28.
The challenges facing the Singapore education system in the new millennium are unique
and unprecedented in Asia. Demands for new skills, knowledges, and flexible competencies
for globalised economies and cosmopolitan cultures will require system-wide innovation
and reform. But there is a dearth of international benchmarks and prototypes for such
reforms. This paper describes the current Core Research Program underway at the National
Institute of Education in Singapore, a multilevel analysis of Singaporean schooling,
pedagogy, youth and educational outcomes. It describes student background,
performance, classroom practices, student artefacts and outcomes, and student
longitudinal life pathways. The case is made that a systematic focus on teachers' and
students' work in everyday classroom contexts is the necessary starting point for
pedagogical innovation and change. This, it is argued, can constitute a rich multidisciplinary
evidence base for educational policy.
Mansell, W., James, M., & Assessment Reform Group. (2009). Assessment in schools Fit for
purpose? A commentary by the Teaching and Learning Research Programme. London:
Economic and Social Research Council, Teaching and Learning Research Programme.
Perhaps no area of education policy is as contentious – or as consistently newsworthy – as
assessment. Recent headlines show how emotive and controversial it can be: “Tests
blamed for blighting children’s lives”; “New fears over dumbing down of key exams”;
“Science exam standards ‘eroded’”1. The public, formal, face of assessment – typically
“high-stakes” examinations such as GCSEs, A-levels, Scottish Highers, the Welsh
169 | P a g e
Baccalaureate or national curriculum tests in England – often dominates debate. But all
good teachers also use assessment informally in the classroom to judge what progress
pupils have made with their understanding, and to provide information on how they can be
helped to move forward.
Marsh, J. A. (2012). Interventions promoting educators’ use of data: Research insights and
gaps. Teachers College Record, 114(11), 1-48.
Background/Context: In recent years, states, districts, schools, and external partners have
recognized the need to proactively foster the use of data to guide educational decision-
making and practice. Understanding that data alone will not guarantee use, individuals at
all levels have invested in interventions to support better access to, interpretation of, and
responses to data of all kinds. Despite the emergence of these efforts, there has been little
systematic examination of research on such efforts.
Conclusions/Recommendations: The review finds that the current research base is limited
in quantity and quality. It suggests the need for more methodologically rigorous research
and greater attention to the organizational and student-level outcomes of interventions,
comparative analyses, interventions that help educators move from knowledge to action,
and specific ways in which the quality of data and leadership practices shape the
effectiveness of interventions.
170 | P a g e
Marsh, J. A., Pane, J. F., & Hamilton, L. S. (2006). Making sense of data-driven decision making
in education. Santa Monica, CA: RAND Corporation.
Data-driven decision making (DDDM), applied to student achievement testing data, is a
central focus of many school and district reform efforts, in part because of federal and
state test-based accountability policies. This paper uses RAND research to show how
schools and districts are analyzing achievement test results and other types of data to
make decisions to improve student success. It examines DDDM policies and suggests future
research in the field. A conceptual framework, adapted from the literature and used to
organize the discussion, recognizes that multiple data types (input, outcome, process, and
satisfaction data) can inform decisions, and that the presence of raw data does not ensure
its effective use. Research questions addressed are: what types of data are administrators
and teachers using, and how are they using them; what support is available to help with
the use of the data; and what factors influence the use of data for decision making? RAND
research suggests that most educators find data useful for informing aspects of their work
and that they use data to improve teaching and learning. The first implication of this work
is that DDDM does not guarantee effective decision making: having data does not mean
that it will be used appropriately or lead to improvements. Second, practitioners and
policymakers should promote the use of various data types collected at multiple points in
time. Third, equal attention needs to be paid to analysing data and taking action based on
data. Capacity-building efforts may be needed to achieve this goal. Fourth, RAND research
raises concerns about the consequences of high-stakes testing and excessive reliance on
test data. Fifth, attaching stakes to data such as local progress tests can lead to the same
negative practices that appear in high-stakes testing systems. Finally, policymakers seeking
to promote educators’ data use should consider giving teachers flexibility to alter
instruction based on data analyses. More research is needed on the effects of DDDM on
instruction, student achievement, and other outcomes; how the focus on state test results
affects the validity of those tests; and the quality of data being examined, the analyses
educators are undertaking, and the decisions they are making.
171 | P a g e
Matters, G. (2006). Using data to support learning in schools: Students, teachers, systems.
Camberwell, Vic: Australian Council for Educational Research.
In ‘Using data to support learning’, Gabrielle Matters envisions an educational system built
around ‘evidenced-based practice’, the idea that decisions at all levels should be grounded
in data. From this perspective, we are not only concerned with giving teachers the data
they need to make more informed decisions about their students, but with all of the
decision makers that constitute the educational system and all of the decisions they need
to make to facilitate achievement.
Matthews, J., Trimble, S., & Gay, A. (2007). But what do you do with the data? Education
Digest: Essential Readings Condensed for Quick Review, 73(3), 53-56.
Using data to redesign instruction is a means of increasing student achievement. Educators
in Camden County (Georgia) Schools have used data from benchmark testing since 1999.
They hired a commercial vendor to design a benchmark test that is administered four times
a year and use the data to generate subject-area reports that can be further disaggregated
by grade, team, teacher, and student. To use data, teachers must accept the data, know
what the numbers indicate, and be ready to change their instruction. Therefore, teacher
leaders in each of the 12 schools organize the test data and help teachers through the
stages of growth that are inherent with data usage (Trimble, Gay, & Matthews, 2005). This
article discusses three steps that Camden uses to get the most out of its test data: (1)
Schedule intensive data sessions; (2) Prepare data for teachers to examine; and (3) Lead
teachers in data analysis. Data from benchmark tests are only useful if teachers and
principals know how to use them to modify instruction. These steps can help school leaders
make the most of benchmark data.
Means, B., Padilla, C., DeBarger, A., & Bakia, M. (2009). Implementing data-informed decision
making in schools—Teacher access, supports and use. California: U.S. Department of
Education Office of Planning, Evaluation and Policy Development.
The collection, analysis and use of educational data are central to the improvement of
student outcomes envisioned by No Child Left Behind (NCLB). The use of data in
educational decision making is expected to span all layers of the education system—from
the federal to the state, district, school and classroom levels. The implementation of the
NCLB legislation has been accompanied by a demand for data systems capable of providing
a longitudinal record of each student’s educational experiences and performance over
time. The conceptual framework developed for the study identifies six prerequisites and
supports for data-informed decision making: (a) state, district and school data systems; (b)
leadership for educational improvement and the use of data; (c) tools for generating
actionable data; (d) social structures and time set aside for analysing and interpreting data;
(e) professional development and technical support for data interpretation; and (f) tools for
acting on data.
172 | P a g e
Meiers, M. (2008). Using data to improve student learning. The Digest. Retrieved 1 August,
2013, from http://www.nswteachers.nsw.edu.au
This Digest is focused on studies that have investigated how data can be used in schools to
examine teaching practices in order to improve student learning. A selection of relevant
websites is listed, and a full reference list is provided. Links to those references for which
full-text online access is freely available are also included. Topics include Data in schools,
Understanding and interpreting data, Purposes for using data, Does performance feedback
lead to improvement?, On a large scale, Comment: Using data in classrooms. Produced by
ACER for the NSW Institute of Teachers
Mokhtari, K., Thoma, J., & Edwards, P. (2009). How one elementary school uses data to help
raise students' reading achievement. The Reading Teacher, 63(4), 334-337.
In this column, we share the collective reflections of a group of teachers and a school
administrator in one Midwestern elementary school, which highlight the value of using
data collaboratively to bring about instructional change and to improve student reading
achievement. If you are a classroom teacher or a school administrator, chances are you are
inundated with all sorts of data, including student demographic information, reading and
writing test scores, and an array of formative assessment data used for documenting and
promoting student reading and learning. The key question here is, What are the factors
that contribute to effective uses of data to help raise students' reading achievement? In
this column, we share collective reflections from two literacy specialists and one school
administrator in one Midwestern U.S. elementary school, which highlight the value of using
data collaboratively to bring about instructional change and to improve student reading
achievement.
Moon, T. R. (2005). The role of assessment in differentiation. Theory into Practice, 44(3), 226-
233.
With the increasing diversity in classrooms, teachers are faced with a broad range of
students representing a wide variety of educational needs. To effectively address students'
diverse education needs, teachers must engage in good decision making. This article
explores the bidirectional relationship between differentiation and assessment through the
lens of decision-making. Particularly, the article investigates the 3 phases of assessment -
planning instruction, guiding instruction, and evaluating instruction. It also asks 4
questions: Why does assessment matter? What happens if it is misaligned with learning
goals? How does the teacher use the assessment data? What does it look like? The article
concludes with a summary of the 3 principle building blocks of differentiation-active
learning, high expectations for students, social context of learning-and their implications
for assessment.
173 | P a g e
New Zealand Ministry of Education. (2013). Assessment online. Retrieved 12 August, 2013,
from http://assessment.tki.org.nz/
Information and ideas relating to this process and the principles expressed in the Ministry
of Education Position Paper: Assessment (2011) are presented on this site.
Newmann, F. M., King, M. B., & Rigdon, M. (1997). Accountability and school performance:
Implications from restructuring schools. Harvard Educational Review, 67(1), 41-74.
Many politicians and policymakers today link school accountability and school
performance. Drawing on evidence from the corporate world, they assume that strong
external accountability will impel schools to improve student achievement. In this article,
however, Fred Neumann, M. Bruce King; and Mark Rigdon argue that three issues keep this
popular theory from working in practice. 1) implementation controversies around
standards, incentives, and constituencies; b) insufficient efforts to organize the human,
technical, and social resources of a school into an effective collective enterprise - what the
authors term ''organizational capacity'' - and c) failure to recognize the importance of
internal school accountability. In a study of twenty-four restructuring schools, the authors
found that strong accountability was rare; that organizational capacity was not related to
accountability; that schools with strong external accountability tended to have low
organizational capacity; and that strong internal accountability tended to reinforce a
school's organizational capacity. Although the implications' of this study for both
accountability policy and, more broadly, school restructuring efforts may appear
disconcerting, the authors conclude with several practical guidelines to stimulate the Kind
of internal accountability that they found to be related to enhanced school performance.
Northern Territory Government. (2012). 2012 NAPLAN: Data analysis guide. Retrieved 12
August, 2013, from www.det.nt.gov.au/teachers-educators/assessment-reporting
A shift is occurring in the way educators view data and it’s potential to inform professional
learning needs, intervention requirements and resource allocation. Data can be used to
focus discussion on teaching, learning, assessment, teacher pedagogy, and monitoring of
progress. School improvement requires more than just presenting the data and assuming it
will automatically transform teachers’ thinking. Rather, teachers need sensitive coaching
and facilitation to study the data and make connections between data and instructional
decision making. School leaders, as data coaches, need to ask purposeful questions to
promote thoughtful discussion which should be followed by targeted action. Effective data
use has to become part of a schools’ culture. A culture where:
174 | P a g e
OECD. (2005). Formative assessment: Improving learning in secondary classrooms. Paris:
OECD.
This study features a collection of eight case studies of exemplary cases from secondary
schools as well as international literature reviews and policy analysis related to formative
assessment – the frequent assessments of student progress to identify learning needs and
shape teaching. It examines such issues as benefits and barriers for using formative
assessment, policy frameworks and implications, and formative assessment in practice.
Achievement gains attributed to formative assessment are reported as being quite high,
but it is not yet practiced systematically. This book makes the case for use of formative
assessment and shows how it can be put into practice.
Pettit, P. (2010). From Data-informed to Data-led?: School Leadership within the Context of
External Testing. Leading and Managing, 16(2), 90-107.
There is an expectation at system and national policy levels that data on student
achievement are collected for the purposes of improving student learning, program
accountability and public reporting. This article reports on a recent study that explored
how the experience of external literacy and numeracy testing and data utilisation affects
attitudes to the tests, teaching practice and school leadership. The research employed a
'mixed methods' approach to obtain both qualitative and quantitative data from
participants in 55 systemic Catholic primary, central and secondary schools in one
Australian diocese. The study found differences in the way that leadership in using data
from external testing of literacy and numeracy is perceived within the school. This was
particularly in relation to how data is analysed and used, the degree of staff involvement in
the process, and associated issues surrounding whole-school planning using the testing
results. There was evidence that schools were not effectively using such data and that
accountability for testing results was viewed according to their perceived purpose. The
findings from the study demonstrate the importance of the perceived value of such data in
informing decisions about student outcomes, and the central role of evidence-based
leadership at the school level in utilising such evidence of learning.
175 | P a g e
Phelps, R. P. (Ed.). (2005). Defending standardized testing. Mahwah, New Jersey: Lawrence
Erlbaum Associates.
The education reform movement of the past two decades has focused on raising academic
standards. Some standards advocates attach a testing mechanism to gauge the extent to
which high standards are actually accomplished, whereas some critics accuse the push for
standards and testing of impeding reform and perpetuating inequality. At the same time,
the testing profession has produced advances in the format, accuracy, dependability, and
utility of tests. Never before has obtaining such an abundance of accurate and useful
information about student learning been possible. Meanwhile, the American public
remains steadfast in support of testing to measure student performance and monitor the
performance of educational systems.
Many educational testing experts who acknowledge the benefits of testing also believe
that those benefits have been insufficiently articulated. Although much has been written
on standardized testing policy, most of the material has been written by opponents. The
contributing authors of this volume are both accomplished researchers and practitioners
who are respected and admired worldwide. They bring to the project an abundance of
experience working with standardized tests.
Pierce, R., & Chick, H. (2011). Teachers’ intentions to use national literacy and numeracy
assessment data: a pilot study. Australian Educational Researcher, 38(4), 433-447.
In recent years the educational policy environment has emphasised data-driven change.
This has increased the expectation for school personnel to use statistical information to
inform their programs and to improve teaching practices. Such data include system reports
of student achievement tests and socio-economic profiles provided to schools by various
state education departments' data services. This paper reports on a pilot study that
explored factors affecting Mathematics and English teachers' intentions to engage with the
statistical data their schools receive and to consider these data when making decisions
about their teaching practices. It was found that most teachers perceived that such data
identify weak students and some teachers (mostly mathematics teachers) thought that
they can help to identify curriculum topics that need attention. Most teachers felt that the
reports were not easy to understand. Confidence in dealing with statistical data was a
problem for many teachers, but especially for English teachers.
Pierce, R., Chick, H., & Gordon, I. (2013). Teachers' perceptions of the factors influencing their
engagement with statistical reports on student achievement data. Australian Journal of
Education, 57(3), 237-255.
In Australia, as in other countries, school students participate in national literacy and
numeracy testing with the resulting reports being sent to teachers and school
administrators. In this study, the Theory of Planned Behaviour provides a framework for
examining teachers’ perceptions of factors influencing their intention to engage with these
176 | P a g e
data. Most teachers perceived the data to be useful, but there were some negatively held
views. For both primary and secondary teachers, males were more positive and had weaker
perceptions of barriers to their use of data from system reports compared to females.
Teachers who had studied statistics at the post-secondary level and/or attended relevant
professional learning generally felt more capable of using the data, and senior teachers and
principals were more favourably disposed to using these kinds of statistical reports. Many
teachers had concerns about the timeliness of the data’s release and the effort required to
interpret them.
Queensland Studies Authority. (2012). 2012 NAPLAN: Test reporting handbook. Brisbane:
Queensland Studies Authority.
The NAPLAN tests were developed using the nationally agreed Statements of Learning for
English and Statements of Learning for Mathematics, referred to as the Statements of
Learning (SoLs). These statements describe essential skills, knowledge, understandings and
capabilities that all young Australians should have the opportunity to learn by the end of
Years 3, 5, 7 and 9. In SunLANDA we have provided the links between the questions and
the SoLs. It is important that principals and teachers note the scope of the tests and how
they were scored. Students were assessed in four areas: Language conventions, Writing,
Reading and Numeracy.
Quint, J. C., Sepanik, S., & Smith, J. K. (2008). Using student data to improve teaching and
learning: Findings from an evaluation of the Formative Assessments of Students. Thinking in
Reading (FAST-R) Program in Boston elementary schools, Retrieved September 2013, from
www.mdrc.org
Formative assessments - assessments that measure what students do and do not know, so
that teachers can modify their instruction accordingly--have been widely hailed as a
potential vehicle for improving student achievement. Yet little solid research evidence
exists about their effectiveness, especially in reform-rich school districts. This study
examines the effects of the Formative Assessments of Student Thinking in Reading (FAST-R)
initiative in the Boston Public Schools system (BPS), where the use of data to improve
instruction is a general priority of the school district. The study looks at changes in reading
scores over time at 21 BPS schools that operated FAST-R during the 2005-2006 and 2006-
2007 school years and changes at a group of comparison schools serving demographically
similar students during the same period. The MDRC evaluation includes process and impact
analyses. The process analysis found that teachers at the FAST-R schools who took a survey
administered as part of the study reported that the professional development they
received from the BPE FAST-R coaches was helpful and contributed to their understanding
of data and their ability to work with students. At the same time, while the intervention
was implemented as intended (it was meant to be flexible and to provide as much or as
little coaching to individual schools as administrators and teachers sought), it was not very
intensive; the majority of survey respondents spent only one to five hours with the FAST-R
177 | P a g e
data coach during the 2006-2007 school year. Moreover, comparison school teachers who
took the survey reported receiving at least as much professional development as their
FAST-R counterparts, were as likely to find it useful, and spent as much or more time
analysing data, including data from other (non-FAST-R) formative assessments. The impact
analysis examines the effects of FAST-R on the reading test scores of third- and fourth-
graders. FAST-R's impacts on student achievement--that is, the difference that FAST-R
made over and above what was going on in the comparison schools--are generally positive
but not statistically significant, as measured by MCAS reading scores. In other words, these
differences could have arisen by chance. Effects on another measure of student reading,
the Stanford Achievement Test, are more mixed but are also not statistically significant.
While FAST-R schools put in place a particular model of data utilization, other BPS schools
were pursuing similar goals, and this fact, along with the intervention's lack of intensity,
may have undercut the likelihood that it would generate substantial and statistically
significant impacts in this evaluation. Thus, this single study in a single district is not the last
word on the potential of FAST-R. Much remains to be discovered about how teachers can
best learn to use data to improve their instruction and boost the achievement of their
students. Following an Overview, Preface, and an Executive Summary, this report is
organized into four chapters. Chapter 3 discusses the professional development activities in
FAST-R and non-FAST-R schools highlighted by the findings of the principal and teacher
surveys. The chapter also considers how teachers perceived the utility of the FAST-R
intervention for their instructional practices. Chapter 4 describes the findings from the
impact analysis of FAST-R with regard to student achievement, exploring the range of
student outcomes on the MCAS and the SAT-9 reading assessments. In addition, the
chapter reports on an analysis to measure the impact of FAST-R on students' ability to
make inferences and find evidence while reading. Lastly, subgroup analyses to compare the
effect of FAST-R on various groups of students (by, for example, gender and socioeconomic
status) are discussed. Chapter 5 presents the overall conclusions that may be drawn from
the study's analyses and their implications for the use of formative assessments and data-
driven instruction to improve reading skills. Appended are: (1) The Analytic Model Used in
the FAST-R Impact Analysis; (2) List of FAST-R and Non-FAST-R Schools; (3) Subgroup
Analyses of the Effects of the FAST-R Program; and (4) Sample of FAST-R Assessment
Student and Teacher Materials. (Contains 28 tables, 5 figures, and 3 boxes.)
Rowley, G., & Congdon, P. (2005). Data-driven school improvement through the VCE Data
Service. Paper presented at the Using data to support learning, Melbourne.
http://www.acer.edu.au/documents/RC2005_GRowleyandPCongdon.pdf
As the holder of student achievement data spanning three sectors and four levels, the
Victorian Curriculum and Assessment Authority (VCAA) has a responsibility to provide these
data to schools in ways that enable school staff to use them effectively and easily. With the
discontinuation of the publication of school achievement indices, the VCAA was forced to
confront a range of issues surrounding the question of which data belonged to the student,
178 | P a g e
which was the property of the school, and which belonged to the general public. In 2002, a
new balance was struck. A key component in this balance was the introduction of the VCE
Data Service. The VCE Data Service is an online service that connects schools to the entire
VCE data set going back to 1998, and provides them with the capacity to generate a range
of analyses related to their own school, and how its results compare to those of other
schools in the State, schools in the same sector (government, Catholic and Independent),
and to schools in its Like School Group.
Salpeter, J. (2004). Data: Mining with a mission. Technology and Learning, 24(8), 30-37.
Data-driven decision making is the buzz phrase of choice for the new decade, but once
we've got the information, how do we use it to yield results? Here 20 school administrators
share the expertise.
Schildkamp, K., Lai, M. K., & Earl, L. (Eds.). (2013). Data-based decision making in education:
Challenges and opportunities. London: Springer.
This resource provides an overview of data-based decision making, how classroom
achievement data can be used to raise student achievement, and some case studies.
Schwartz, W. (2002). Data-driven equity in urban schools. ERIC Digest. Retrieved 7 August,
2013, from http://files.eric.ed.gov/fulltext/ED467688.pdf
The 2002 reauthorization of the Elementary and Secondary Education Act mandates that
schools receiving federal funding must desegregate their student performance data by
race, gender, and socioeconomic status in order to provide progress information to the
community and state. Data-driven decision making is particularly important in urban
schools whose populations are disproportionately poor, minority, and in need of special
services. This digest discusses the types of data that schools should collect and the ways to
179 | P a g e
use the information effectively in decision making to enhance equity. It begins by
explaining how to use data to enhance quality, then it describes data types (student
learning data, student demographics, perceptions data, and school process data). It goes
on to explain the disaggregation of data, which allows schools to determine more
accurately the effects of programs and strategies on segments of its student body. Finally,
this digest explains data-driven decision making and describes how to choose a technology
tool to support data-driven decision making (functionality, data storage capacity, training,
and format). A sidebar presents data disaggregation tools. Data support resources are
listed.
Sharratt, L., & Fullan, M. (2012). Putting faces on the data. Thousand Oaks, CA: Corwin.
Students are people – not data. Assessment data can bury you or give you focused
information on how to reach every student. Putting faces on the data shows how to
develop a common language for sharing all students’ progress with all teachers and leaders
and how to use ongoing assessment to inform instruction.
Slavin, R. E., Cheung, A., Holmes, G., Madden, N. A., & Chamberlain, A. (2013). Effects of a
data-driven district reform model on state assessment outcomes. American Educational
Research Journal, 50(2), 371-396.
A district-level reform model created by the Center for Data-Driven Reform in Education
(CDDRE) provided consultation with district leaders on strategic use of data and selection
of proven programs. Fifty-nine districts in seven states were randomly assigned to CDDRE
or control conditions. A total of 397 elementary and 225 middle schools were followed
over a period of up to 4 years. In a district-level hierarchical linear modeling (HLM) analysis
controlling for pretests, few important differences on state tests were found 1 and 2 years
180 | P a g e
after CDDRE services began. Positive effects were found on reading outcomes in
elementary schools by Year 4. An exploratory analysis found that reading effects were
larger for schools that selected reading programs with good evidence of effectiveness than
for those that did not.
Smith, M. (2005). Data for schools in NSW: What is provided and can it help? Paper presented
at the Using data to support learning, Melbourne. Retrieved September 2013, from
http://research.acer.edu.au/research_conference_2005/11
Lifting the performance of New South Wales (NSW) students in literacy, numeracy and
other key outcome areas to world-class standards is a central priority of this Government.
The crucial responsibilities, shared between schools and the system, for effective
educational provision are articulated in the most recent Framework for School
Development and Accountability for NSW government schools. The role of 108 very senior
officers, School Education Directors, recently appointed to regions across the state, is to
ensure the effective implementation of this framework. The aim is to consolidate and focus
existing accountability, improvement and reporting policies to improve and enrich student
outcomes.
Stiggins, R. (2005). From formative assessment to assessment for learning: A path to success
in standards-based schools. Phi Delta Kappan, 87(4), 324-328.
As the mission of schools changes from ranking students to ensuring that all learn to
specified standards, Mr. Stiggins argues that the purpose and form of assessments must
change as well. Society has seen fit to redefine the role of its schools. No longer are they to
be places that merely sort and rank students according to their achievement. Now, they
are to be places where all students become competent, where all students meet pre-
specified standards and so aren’t left behind. With increasing intensity, policy makers are
turning to assessment as the power tool that will compel schools to fulfil this new role. If
we look closely at the union of this redefined mission and the growing reliance on
assessment, we can find a surprising and immensely powerful way to use assessment in the
development of effective schools.
Stiggins, R., & Duke, D. (2008). Effective instructional leadership requires assessment
leadership. Phi Delta Kappan, 90(4), 285-291.
Principals can play a pivotal role in the improvement of student learning by helping
teachers develop and use sound classroom assessments that will strengthen instruction
and student learning. The typical teacher will spend a quarter to a third of her or his
available professional time involved in assessment-related activities. If they do it well, both
teachers and students gain access to evidence that can be used in making sound
instructional decisions; if they do it poorly, learning will suffer. In spite of this, little of
principals' preparation time is spent learning about assessments.
181 | P a g e
Stiggins, R. J. (1995). Assessment literacy for the 21st century. Phi Delta Kappan, 77(3), 238-
245.
Without a clear vision of the meaning of academic success and without the ability to
effectively assess student attainment of those achievement targets at the classroom,
building, and district levels, we will remain unable to help students attain higher levels of
academic achievement, regardless of the instructional methods we use or how we organize
our schools Mr Stiggins warns.
Stiggins, R. J. (2002). Assessment crisis: The absence of Assessment FOR Learning. Phi Delta
Kappan, 83(10), 758-765.
If we wish to maximize student achievement in the U.S., we must pay far greater attention
to the improvement of classroom assessment, Mr. Stiggins warns. Both assessment of
learning and assessment for learning are essential. But one is currently in place, and the
other is not.
Supovitz, J. A., & Klein, V. (2003). Mapping a course for improved student learning: How
innovative schools systematically use student performance data to guide improvement.
Pennsylvania: Consortium for Policy Research in Education.
To be useful to teachers and school leaders, test data must provide more than just a
destination. Student performance results must also provide guidance that informs
educators that they are moving in the right direction, while providing interactive and
recursive feedback for mid-course adjustments. In order for student performance data to
be useful to teachers and school leaders, and to make it worthwhile for them to make the
extensive efforts necessary to learn to interpret and act upon what they learn, data
feedback systems must rely on multiple sources of data collected and analysed at regular
intervals.
This report is about building better roadmaps for teachers and school leaders in order to
guide their instructional decision-making. The data required for more precise decision-
making come from systematically exploiting a variety of student performance data at both
the individual classroom and school levels. Rather than just relying on one individual test to
provide guidance, innovative school leaders are building more comprehensive systems of
assessments that provide better interim information from multiple perspectives. Through
more sophisticated data systems, teachers and school leaders can foster a more inquiry
oriented approach that involves ongoing and sustained investigations into the kinds of
teaching that produce more powerful student learning. In this report, we show how
innovative teachers and school leaders are creatively using their data to help guide their
strategic decisions. Through their examples, we develop and describe a theory of what a
system of school data use might look like.
182 | P a g e
Sutherland, S. (2004). Creating a culture of data use for continuous improvement: A case
study of an Edison Project school. American Journal of Evaluation, 25(3), 277-293.
In this article, I outline how, with the appropriate mix of external requirements and
internal motivation, structure and capacity, a school can promote and maintain a culture of
evaluative practices, specifically data use, for continuous improvement. I draw upon
qualitative data conducted in a study of an Edison Project school. The findings from this
paper are the result of a larger 4-year study examining Comprehensive School Reform in 12
Title 1 schools in three states. Interviews were conducted with principals, teachers, other
staff, parents and students, as well as with state officials, the superintendent, assistant
superintendent and other high level administrators in the district. Observations were
undertaken at the classroom level. Key findings include the importance of obtaining a
multidimensional view of educational change. That is, we need to obtain a picture of the
school reform environment (i.e., interconnections between the state, district, and school
levels), and simultaneously investigate the interplay among critical school levers that are
required to promote and maintain a culture of data use for school improvement. From this
multidimensional view, we are better positioned to investigate and evaluate school change
efforts.
Swaffield, S. (2011). Getting to the heart of authentic Assessment for Learning. Assessment in
Education: Principles, Policy & Practice, 18(4), 433-449.
Assessment for Learning (AfL) has gained increasing international prominence in both
policy and practice but some of its proliferation, notably the national strategy in England,
has been accompanied by distortion of essential features. This paper presents an
understanding of authentic (in the sense of genuine) AfL informed by literature and
particularly by two major research projects. Assessment for learning is characterised by
information being used to inform learning and teaching, its focus on learning conceived
broadly, and actively engage progressively more autonomous students. It is distinctive in its
timescale, protagonists, beneficiaries, the role of students, the relationship between
student and teacher, and the centrality of learning to the process – all of which can but
may not necessarily be features of formative assessment. An examination of the document
setting out the National Assessment for Learning Strategy in England reveals the ways that
it is at odds with authentic assessment for learning.
Symonds, K. W. (2003). After the test: How schools are using data to close the achievement
gap. San Francisco, CA: Bay Area School Reform Collaborative.
In recent years, closing the achievement gaps between higher-and lower-achieving groups
of students has become the focus of state and federal policy. Yet, while there are decades
of research about classroom-level practices associated with increased student
performance, few studies have examined the school-level policies and strategies that help
close the achievement gaps. In order to identify effective school-level policies and
strategies, the Bay Area School Reform Collaborative (BASRC) surveyed 32 K–8 schools in
183 | P a g e
the San Francisco Bay Area and compared responses from schools narrowing the gaps with
schools maintaining or widening the gaps. Schools’ gaps were measured using California’s
Academic Performance Index (API) ranking system over the four-year time period between
1998–99 and 2001–02. We defined gap-closing schools as those schools in which all
students made improvement but low-performing students made more rapid progress.
Conversely, we defined non-gap-closing schools as those schools in which high-performing
students made more improvement than low-performing students. We also conducted case
studies of three schools making outstanding progress in narrowing the achievement gaps.
Thomas, G., & Tagg, A. (2005). Numeracy Development Project Longitudinal Study: Patterns
of achievement. Retrieved 6 August, 2013, from
http://www2.nzmaths.co.nz/numeracy/references/comp05/comp05_thomas_tagg.pdf
The central aim of the Numeracy Development Projects (NDP) is to improve student
performance in mathematics through improving the professional capability of teachers.
The aim of the NDP Longitudinal Study is to investigate the longer term impact of the NDP
on student achievement in number and in mathematics more generally. This paper reports
on the mathematics achievement of students in 26 schools that participated in the NDP
prior to 2004. Not surprisingly, the findings suggest that the project has had the strongest
impact on the students’ performance on number items that are directly related to the NDP.
The achievement of students as measured against the Number Framework indicates that,
at most year levels, strategy level attainment has increased over time.
Thomas, G., Tagg, A., & Ward, J. (2003). An evaluation of the Early Numeracy Project, 2002:
Exploring issues in mathematics education. Retrieved 6 August, 2013, from
http://www2.nzmaths.co.nz/numeracy/References/eval_enp2002.pdf
The Numeracy Development Project sits within the context of the Ministry of Education’s
Literacy and Numeracy strategy. Three themes underpin that strategy: clarifying
expectations, improving professional capability and involving the community.
Thorn, C. A. (2002). Data use in the classroom: The challenges of implementing data-based
decision-making at the school level. Madison: Wisconsin Center for Education Research,
University of Wisconsin.
This paper examines problems school-level staff encounter when attempting to implement
data-based decision-making reform efforts, specifically those focused on teaching and
learning in the classroom. The paper also offers recommendations for professional
development that address gaps in traditional principal and teacher training. Many schools
and districts are exploring data-driven decision making as a solution for improving resource
allocation and instructional program decisions. One of the most challenging problems
policy makers and educators face in attempting to implement curriculum reforms is that
intervention decisions are made at least one organizational level above that of the
teachers—the persons actually engaged in instruction. Any systemic effort to implement a
184 | P a g e
focus on data-based decision making at the school and classroom levels faces several
challenges. First, most data available within district information systems are limited to
what has been deemed important for the operational needs of schools and the district and
are only available on systems supported by centralized computing services. These data
include attendance, discipline, and basic demographic data. District systems also contain
detailed information about human resources, budgets, and other business processes.
Typically, the only available outcome data are grades and the results from centrally (and
often annually) administered tests. These data, although useful to help frame annual
analysis of school-, classroom-, or student-level outcomes, are inadequate for making
midcourse or interim instructional decisions within a single grade or marking period.
185 | P a g e
together with conclusions about the realities of instructional leadership and the support
that might be needed to undertake it effectively.
Timperley, H., & Parr, J. (2004). Using evidence in teaching practice: Implications for
professional learning. Auckland, NZ: Hodder Moa Beckett.
For teachers to use evidence to improve teaching and learning in their classrooms they
need information about what their students know and can do, evidence about their own
practice and its impact on students, and knowledge of the research evidence and that from
other established sources to give direction for improvements to practice. Teachers,
however, cannot be expected to know and do all this on their own, but need the support of
well-informed leaders who have sufficient knowledge both to lead teachers’ evidence-
informed inquiry and to engage in their own inquiry into the effectiveness of their
leadership practice in promoting teacher and student learning.
Timperley, H., Wilson, A., Barrar, H., Fung, I., & University of Auckland. (2007). Teacher
professional learning and development: Best Evidence Synthesis [BES]iteration. Auckland: New
Zealand Ministry of Education.
Over the past several decades the focus on educational change has been pervasive and
unrelenting as education systems everywhere have struggled to meet the needs of the
times. For those of us who have a long history of involvement in education, it is sometimes
hard to imagine that there could be anything new under the educational reform sun, as old
ideas are recycled and the pace of change often seems painfully slow. But periodically,
something surfaces that has the power to fundamentally reshape how we work. The
Iterative Best Evidence Synthesis Programme, of which this BES is part, has this potential.
Tovani, C. (2011). So what do they really know?: Assessment that informs teaching and
learning (eBook ed.). Portland, Maine: Stenhouse Publishers.
In So What Do They Really Know? Cris Tovani explores the complex issue of monitoring,
assessing, and grading students' thinking and performance with fairness and fidelity. Like
all teachers, Cris struggles to balance her student-centred instruction with school system
mandates. Her recommendations are realistic and practical; she understands that what
isn't manageable isn't sustainable. Cris describes the systems and structure she uses in her
own classroom and shows teachers how to use assessments to monitor student growth
and provide targeted feedback that enables students to master content goals. She also
shares ways to bring students into the assessment cycle so they can monitor their own
learning, maximizing motivation and engagement.
186 | P a g e
Tozer, L., & Holmes, M. (2005). Moving on from Count Me In Too: Evidence-based teaching
and learning in numeracy in the early and middle years of schooling. Paper presented at the
Using data to support learning, Melbourne. Retrieved September 2013, from
http://www.acer.edu.au/documents/RC2005_LTozerandMHolmes.pdf
New Zealand developed the Early Numeracy Project for Years 1–3 in 2000–2001, based on
the New South Wales’ Count Me In Too, and much has happened in mathematics
education since. Change is inevitable and numeracy has moved on. Today the New Zealand
Number Framework, the Diagnostic Interview and Teaching Model now underpin
numeracy teaching practice in over 14,000 classrooms from Year 1–9. Important
developments to date have included a flexible national database and web site, well-
developed supporting materials and data-rich annual evaluation reports which inform
future direction and expectation of achievement. Because the Numeracy Project is
evolving, further development and consolidation will continue. This paper gives a brief
background to the Numeracy Project and outlines, through a story, how evidence-based
teaching is an integral part of classroom practice.
van Barneveld, C. (2008). Using data to improve student achievement. Retrieved 7 August,
2013, from
http://www.edu.gov.on.ca/eng/literacynumeracy/inspire/research/using_data.pdf
In the context of education, data is a synonym for information. Data can be words,
numbers, or observations that are collected systematically, usually for a specific purpose.
Educational data include (but are not limited to):
Waddell, G., & Lee, G. (2008). Crunching numbers, changing practices: A close look at student
data turns the tide in efforts to close the achievement gap. JSD: The Journal of the National
Staff Development Council, 29(3), 18-21.
A close look at student data turns the tide in efforts to close the achievement gap.
187 | P a g e
Wayman, J. C. (2005). Involving teachers in data-driven decision making: Using computer data
systems to support teacher inquiry and reflection. Journal of Education for Students Placed at
Risk (JESPAR), 10(3), 295-308.
Accountability mandates such as No Child Left Behind (NCLB) have drawn attention to the
practical use of student data for school improvement. Nevertheless, schools may struggle
with these mandates because student data are often stored in forms that are difficult to
access, manipulate, and interpret. Such access barriers additionally preclude the use of
data at the classroom level to inform and impact instruction. Fortunately, there are newly
available computer technologies that allow efficient organization and access to student
data. In addition to allowing easier accountability reporting, these tools allow user-friendly
data access at all educational levels, meaning that teachers can use these tools to engage
in the informed reflection necessary to improve classroom practice. In this article, I discuss
teacher use of these systems, providing insight into the function of these tools and
discussing conditions that make these tools of the most service to teachers.
Wayman, J. C., Cho, V., & Johnston, M. T. (2007). The data-informed district: A district-wide
evaluation of data use in the Natrona County School District. Austin, TX: The University of
Texas.
For years, educational entities have collected data on school process and student learning.
Recent accountability policies have brought public attention to these data, increased the
amount of data collected, and tied funding to certain characteristics of these data.
Consequently, educators respond to reporting requirements while simultaneously
struggling with better ways to understand these data internally to improve practice. To
understand and improve district data use, individuals from the Natrona County School
District (NCSD) commissioned a district-wide evaluation of data uses and procedures for
data-based decision-making. In this report, we present findings from this evaluation.
Results provided an in-depth description of data use at every level, showing the hardships
of using data but also highlighting many positive structures upon which to build an
effective initiative. As a result of this evaluation, the authors recommended the following:
(a) a framework to guide NCSD in establishing itself as a data-informed district where data
and practice are integrated throughout; (b) a plan for acquiring an efficient data system
that can integrate data district-wide; (c) a blueprint for NCSD to use in establishing a
healthy, district-wide data initiative; and (d) specific issues for NCSD to address in getting
up to speed on data use, such as school dropouts, school differences, public perception,
and areas for further study.
188 | P a g e
the Australian Council for Educational Research (ACER) reported by Forster and Masters. In
their chapter they argue for a conceptual bridge between a program for system-wide
monitoring of student achievement (NSELS) and a classroom assessment resource (DART).
The Data Club provides a practical bridge between statewide, system-level data used for
national accountability purposes and school-level use of the same data.
Williams, J., & Ryan, J. (2000). National testing and the improvement of classroom teaching:
Can they coexist? British Educational Research Journal, 26(1), 49-73.
Can children's responses to tests be used to provide useful diagnostic information for
classroom teachers? This article describes an analysis of children's performance in the 1997
UK mathematics tests by 7 and 14 year-olds. The children's responses and errors were
scaled against their ability using Rasch methodology. These were then interpreted in terms
of the literature on the psychology of mathematics education, especially that related to
misconceptions, and an attempt was made to describe children's progression in thinking as
it relates to their test performance. This work has been reported to all primary and
secondary schools in England and Wales, and is intended to provide a description of what
the children believe and know. As such it provides a resource to help raise teachers'
awareness of their children's thinking and present openings for diagnostic assessment and
teaching. Opportunities for and obstacles to developing this approach in future are
discussed.
Woods, A., & Amorsen, A. (2011). Evaluation of the Year 1 literacy and numeracy checkpoints
assessments trial – 2010. Brisbane: School of Early Childhood, Faculty of Education,
Queensland University of Technology.
The draft Year 1 Literacy and Numeracy Checkpoints Assessments were in open and
supported trial during Semester 2, 2010. The purpose of these trials was to evaluate the
Year 1 Literacy and Numeracy Checkpoints Assessments (hereafter the Year 1 Checkpoints)
that were designed in 2009 as a way to incorporate the use of the Year 1 Literacy and
Numeracy Indicators as formative assessment in Year 1 in Queensland Schools. In these
trials there were no mandated reporting requirements. The processes of assessment were
related to future teaching decisions. As such the trials were trials of materials and the
processes of using those materials to assess students, plan and teach in year 1 classrooms.
189 | P a g e
Woods, K., & Griffin, P. (2013). Judgement-based performance measures of literacy for
students with additional needs: Seeing students through the eyes of experienced special
education teachers. Assessment in Education: Principles, Policy & Practice, 20(3), 325-348.
This article describes the development of judgement-based performance measures to
support the instruction of students with additional learning needs. The focus of the
research was the design of assessment materials and protocols to help teachers recognise
and respond to students’ proficiency in foundational literacy skills. It drew on the expertise
of special education teachers to provide all teachers with an evidence framework against
which to observe their students’ learning. The assessment materials were trialled in 53
schools and used to monitor literacy learning for 547 students, who ranged in age from 3 to
18 years and represented children and young people with a wide diversity and severity of
disabilities. The article reports a new approach to judgement-based performance
measurement which directs teachers’ observations to meaningful shifts and
transformations in foundational literacy skills for students with additional needs.
Wyatt-Smith, C., & Klenowski, V. (2013). Explicit, latent and meta-criteria: Types of criteria at
play in professional judgement practice. Assessment in Education: Principles, Policy &
Practice, 20(1), 35-52.
This paper engages with debates about whether comprehensive prior specification of
criteria and standards is sufficient for informed professional judgement. A preoccupation
has emerged with the specificity and explication of criteria intended to regulate judgement.
This has resulted in criteria compliance in the use of defined standards to validate
judgements and improve reliability and consistency. Compliance has become a priority, the
consequence being the prominence of explicit criteria, to the lack of acknowledgement of
the operation of latent and meta-criteria within judgement practice. This paper examines
judgement as a process involving three categories of assessment criteria in the context of
standards-referenced systems: explicit, latent and meta-criteria. These are understood to
be wholly interrelated and interdependent. A conceptualisation of judgement involving the
interplay of the three criteria types is presented with an exploration of how they function
to focus or alter assessments of quality in judgements of achievement in English and
Mathematics.
Young, V. M. (2006). Teachers’ Use of Data: Loose Coupling, Agenda Setting, and Team
Norms. American Journal of Education, 112(4), 521-548.
This article explores the influence of grade-level team norms and district and school
leadership on teachers’ data use. Using an embedded-systems perspective to consider
teachers’ data use in four schools located in two different districts, the research takes the
practitioners’ perspective on what constitutes data. Findings indicate that establishing
rationale for teachers to use particular data, modeling such use, and structuring time for
teachers to learn about using data are deliberate agenda-setting activities. Varying degrees
190 | P a g e
of loose coupling between the case study districts underscore how grade-level norms and
agenda setting mediate teachers’ collaborative use of data.
191 | P a g e
Appendix B: Alphabetical list of
strategies and practices for using
classroom data
A
Academic reporting
Accountability - horizontal
Accountability - vertical
Active reflection
Anchor charts
Assessment feedback
Assessment focused websites
Assessment for learning
Authentic assessment
B
Barriers to effective use of data
Benchmarking
Bubble kids
C
Check-in cards
Classroom organisation
Class sizes and students’ results
Collaboration
Collaborative planning
Collective and collaborative review of student work
Conversations
Confidentiality
Contact with parents
Cycles of improvement
Cycles of continuous improvement
D
Dashboards
Data charts
192 | P a g e
Data-driven systems
Data-informed decision making
Data sources
Data teams
Data walls
Diagnostic tools - reading
Differentiation of activities and assessments
Differentiating assessment
Differentiating the curriculum and pedagogy
E-F
Educational support plans and individual education plans
Facilitating learning
Feedback systems
Formative assessment
Frameworks
G-I
Graphic organisers
Informed choice and career options
Internal school data
Inquiry models
J-L
Leadership
Learning charts
Learning Commission
Learning maps
Learning programs
M-N
Mentoring as feedback
Mentoring teachers
Moderation
NAPLAN data
Numeracy and literacy data
193 | P a g e
O-P
One-on-one explanations
Peer and student self-assessment
Professional judgements
Professional learning
Q-R
QCE results
Reflection to refocus classroom practices
Responsibility and ownership
Reviewing classroom data
S
Scaffolding
Scaffolding interpretation of data
School improvement plans
Self-paced learning
Shared data
Spreadsheets
Standardised test data
Summative assessment
Supporting teachers
T
Tracking attendance
Tracking behaviour
Tracking career pathways
Tracking progress
Tiered support
Two-tiered questioning
Types of data systems
Types of data
U-Z
Verbal feedback
Warehousing tools
194 | P a g e