Goos, Geiger, Bennison, Dole, & Forgasz Research Engagement and Impact in Mathematics Education Merrilyn Goos (Chair) Vince Geiger The University of Queensland <m.goos@uq.edu.au> Australian Catholic University <vincent.geiger@acu.edu.au> Anne Bennison Shelley Dole The University of Queensland <a.bennison@uq.edu.au> Sunshine Coast University <sdole@usc.edu.au> Helen Forgasz Monash University <helen.forgasz@monash.edu> While measures of research quality are widely accepted in the education research community, there may be less agreement on what constitutes evidence of impact and on where to look for it. The aims of this symposium are to consider some key issues in undertaking the Australian government’s national assessment of research engagement and impact, and to propose some approaches to evidencing engagement and impact in the context of mathematics education research. Each of the four symposium papers draws on our Numeracy Across the Curriculum (NAC) research program in order to ground our discussion in specific cases of research that have been reported at previous MERGA conferences. In the first paper, Evidencing research engagement and impact, Merrilyn Goos establishes the theoretical and policy context for the symposium in terms of the apparent lack of connection between educational research and practice. She analyses aspects of the NAC research program to trace rich connections between her own teaching, research and service roles that led to beneficial knowledge exchanges (engagement), and intricate links between research activities, outputs and outcomes across multiple projects (impact). Such an analysis can suggest “where to look” for evidence of engagement and impact. In the second paper, The convoluted nature of a research impact pathway, Vince Geiger develops a case study of an aspect of his own research within the NAC program to illustrate the complexity of the journey from research origin through to potential impact. The documentation of this research progress allows for reflection on how future impact can be “read” while research is taking place. In the third paper, Engagement and impact through research participation and resource development, Anne Bennison and Shelley Dole illustrate how knowledge exchange and uptake of resources developed through research can provide evidence of research engagement and impact, respectively. The analysis suggests ways in which collaborative research (an ARC Linkage Project on proportional reasoning and numeracy) and contract research (funded by the Queensland College of Teachers) can be translated for economic and social benefits. In the fourth paper, “Numeracy for learners and teachers”: Impact on MTeach students, Helen Forgasz evaluates the impact of a compulsory unit taken by all primary and secondary pre-service teachers in the Monash University Master of Teaching. The unit design incorporates elements of the Numeracy Across the Curriculum model to address AITSL standards for knowledge and understanding of literacy and numeracy teaching strategies, and interpreting student data. The evaluation reveals substantial impact on students’ understanding of numeracy and confidence in incorporating numeracy in their teaching, thus highlighting the contribution of research to improving teacher education. (2017). In A. Downton, S. Livy, & J. Hall (Eds.), 40 years on: We are still learning! Proceedings of the 40th Annual Conference of the Mathematics Education Research Group of Australasia (p. 629). Melbourne: MERGA. Goos Evidencing Research Engagement and Impact Merrilyn Goos The University of Queensland <m.goos@uq.edu.au> This paper outlines current developments in the Australian government’s plan to introduce a national assessment of research engagement and impact and considers implications for mathematics education. A well-established research program seeking to embed numeracy across the school curriculum is used to illustrate forms of research engagement and impact. The analysis of this program demonstrates rich connections between research, policy and practice, and suggests “where to look” for evidence of engagement and impact. In December 2015, as part of its National Innovation and Science Agenda, the Australian government announced the development of a national assessment of research engagement and impact. It is envisaged that the assessment will be implemented in parallel with the national evaluation of research quality – Excellence in Research for Australia (ERA). The Australian Research Council (ARC) and the Department of Education and Training released an Engagement and Impact Assessment Consultation Paper in May 2016 to seek feedback from stakeholders on how this assessment should be undertaken (ARC and DET, 2016). While measures of research quality are widely accepted in the education research community, there may be less agreement on what constitutes evidence of impact and on where to look for this evidence. The aims of this paper are to consider some of the key issues in undertaking a national assessment of research engagement and impact that were raised in the Consultation Paper, and to propose some approaches to evidencing engagement and impact in the context of mathematics education research. Theoretical Background: The Gap between Research, Policy, and Practice Education research is often criticised for its lack of impact on classroom practice. Explanations for the apparent research-practice gap sometimes highlight the different processes used by researchers and teachers to improve educational practice. For example, Richardson (1994) suggested that, whereas formal research aims to contribute to an established and general knowledge base, the practical inquiry of teachers is focused on solving immediate day-to-day problems. Writing from an educational leadership and policy perspective, Levin (2010) invoked the idea of knowledge mobilisation to examine connections between the production, communication, and use of research. He argued that not only do researchers have a responsibility communicate their findings beyond academia, but policy-makers and practitioners also need to be willing to find, share, and use good research in their work. The apparent lack of connection between education research, policy and practice seems to be particularly relevant to mathematics education. For example, national and international assessments of mathematics achievement, such as the National Assessment Program – Literacy and Numeracy (NAPLAN), the OECD’s Programme for International Student Assessment (PISA), and the IEA’s Trends in International Mathematics and Science Study (TIMSS) allow governments to compare performances within and between countries and can create pressure to change mathematics curricula and teaching practices. Nevertheless, it is well documented that classroom practice remains resistant to the reform approaches promoted by mathematics education researchers (e.g., Gill & Boote, 2012). In (2017). In A. Downton, S. Livy, & J. Hall (Eds.), 40 years on: We are still learning! Proceedings of the 40th Annual Conference of the Mathematics Education Research Group of Australasia (pp. 630-633). Melbourne: MERGA. light of discussions in the literature and the move within Australia towards a national assessment of research engagement and impact, it is especially timely for mathematics educators to consider how to evidence the uptake and benefit of their research. Defining and Evidencing Research Engagement and Impact For the purpose of illustration, I refer to the Numeracy Across the Curriculum (NAC) research program to which the presenters of this symposium have contributed in different combinations and in different ways. The program builds on sixteen years of productive engagement with teachers, teacher educators, policy-makers, school systems, and the Australian and international research community. The research was motivated by a desire to challenge narrow “basic skills” interpretations of numeracy that prepare low-achieving students to do no more than “survive” in the world beyond school. As a result, the research team developed a rich interpretation of numeracy that connects the mathematics learned at school with out-of-school situations that additionally require problem solving, critical judgment, and making sense of the non-mathematical context. This approach necessarily positions numeracy as an across-the-curriculum commitment that extends beyond the mathematics classroom. The most significant outcome of the program is a model of numeracy for the 21st century that recognises the intellectual, affective, and contextual demands of becoming a numerate person. According to this model, numeracy development requires attention to real-life contexts, the application of mathematical knowledge, the use of representational, physical and digital tools, and positive dispositions towards the use of mathematics. A further important and overarching element of the model is a critical orientation to the use of mathematics (see Goos, Geiger, & Dole, 2014). Research Engagement The ARC Consultation Paper draws on the definition used by the Academy of Technological Sciences and Engineering (ATSE) to develop metrics for Australian universities’ research engagement. Engagement was defined as: the interaction between researchers and research organisations and their larger communities/ industries for the mutually beneficial exchange of knowledge, understanding and resources in a context of partnership and reciprocity. (ATSE, 2015) However, the Consultation Paper notes that metrics, which are largely based on research commercialisation income and patents, may not capture the complexity of some forms of research engagement. As a qualitative alternative, Figure 1 maps the interactions between my own academic teaching, research, and service roles that led to beneficial knowledge exchanges in teacher education, professional development, and consultancy settings, involving practitioners, school leaders, education systems, professional associations, and teacher registration authorities as part of my contribution to the NAC research program. Also noticeable in this diagram is the absence of a neat linear progression from research contexts towards contexts of application. Instead, knowledge exchange has built reciprocal relationships across the boundaries between research, policy, and practice. Research Impact The ARC (2012) defines research impact as “the demonstrable contribution that research makes to the economy, society, culture, national security, public policy or services, health, the environment, or quality of life, beyond contributions to academia”. Teaching Service 2000-2002 Curriculum integration project (initial teacher education) Research 2002-2003 DEST project: Home, school, community partnerships to support numeracy 2005 Qld Board of Teacher Registration: develop numeracy standards for graduate teachers 2007 SA Dept. Education & Children’s Services (DECS) consultancy on Numeracy in the FutureSACE Rich model of numeracy (1) 2009 SA DECS contract research (2) 2010-11 Brisbane Catholic Education (BCE) contract research (3) 2012 BCE contract research (4) 2012-14 ARC Discovery Project (5) 2010-14 ARC Linkage Project (6) 2012-16 PhD project 2015 Masters course on literacy & numeracy across the curriculum 2017 ITE course on numeracy across the curriculum (7) 2014-15 Qld College of Teachers web-based resource development (8) 2010-12 AAMT Make It Count numeracy evaluation Figure 1. Personal engagement map: Numeracy across the curriculum research program While noting that there are no clearly defined indicators for research impact, the Consultation Paper refers to peer reviewed case studies – conducted as part of the recent UK REF exercise – as being an appropriate means of assessment. Nevertheless, case studies are expensive to produce. The ARC has also developed a Research Impact Pathway table (http://www.arc.gov.au/research-impact-principles-and-framework#table) to assist grant applicants to identify the potential benefits of their proposed research. The table’s column headings are listed below, together with education-relevant examples: 1. 2. 3. 4. Inputs: budget, research assistants, infrastructure; Activities: research project, undergraduate teaching, professional development; Outputs: publications, PhD graduates, resources developed; Outcomes: uptake of resources, research incorporated into teacher education and support materials, changes in policy based on research evidence; 5. Benefits: improved outcomes for learners, improved teaching practice. Figure 2 provides a mapping of some of the outputs and outcomes of the NAC research program, using the same eight research activities identified in Figure 1. Evidencing direct benefits to students and teachers remains a pressing challenge for much education research. Research Activities Sample Research Outputs and Outcomes #1 #2 Curriculum audit methodology * Principles for task design and curriculum planning * * Professional development approach * * Methods for monitoring teacher learning trajectories * #3 #5 #6 #7 #8 * Whole school approaches to numeracy leadership Resources for teachers and teacher educators #4 * * * * * * * * * * * * * * * * * * * Figure 2. Partial impact map: Numeracy across the curriculum research program. Issues for Consideration Two of the key issues identified in the Consultation Paper are worth considering here. The first involves accounting for the variable time lags between undertaking research and achieving identifiable benefits for end-users. As Figure 1 shows, it can take more than ten years for education research to make a demonstrable contribution to society. The second issue refers to the difficulties in determining the attribution of research engagement and impact, for example, if an impact can be traced back to more than one project, as is the case in the NAC research program (see Figure 2). Not only was impact derived across multiple research activities, but these activities also spanned the multiple universities in which the research team members worked. It remains to be seen how a national assessment of impact could be undertaken if the unit of analysis is the individual university. Beyond the immediacy of an impending national assessment of research engagement and impact, there is surely value for mathematics educators in retrospectively analysing our own research to illuminate the opportunities taken, decisions made, and relationships built in pursuing research that makes a difference. Such an analysis might help us not only to learn “where to look” for evidence of past impact, but also to plan future research projects with an eye to demonstrating potential benefits for educational policy and practice. References Academy of Technological Sciences and Engineering (2015). Research engagement for Australia: Measuring research engagement between universities and end users. Retrieved from http://atse.uberflip.com/i/499806-research-engagement-for-australia-measuring-research-andengagement-between-universities-and-end-users-proposal Australian Research Council. (2012). Research impact principles and framework. Retrieved from http://www.arc.gov.au/research-impact-principles-and-framework#Definition Australian Research Council and Department of Education and Training. (2016). Engagement and impact assessment consultation paper. Retrieved from http://www.arc.gov.au/sites/default /files/filedepot/Public/ARC/consultation_papers/ARC_Engagement_and_Impact_Consultation_Paper .pdf Gill, M., & Boote, D. (2012). Classroom culture, mathematics culture, and the failures of reform: The need for a collective view of culture. Teachers College Record, 114(12), 1-45. Goos, M., Geiger, V. & Dole, S. (2014). Transforming professional practice in numeracy teaching. In Y. Li, E. Silver, & S. Li (Eds.), Transforming mathematics instruction: Multiple approaches and practices (pp. 81-102). New York, NY: Springer. Levin, B. (2010). Leadership for evidence-informed education. School Leadership and Management, 30(4), 303-315. Richardson, V. (1994). Conducting research on practice. Educational Researcher, 23(5), 5-10. Geiger The Convoluted Nature of a Research Impact Pathway Vince Geiger Australian Catholic University <vincent.geiger@acu.edu.au> How research inputs and activities translate into outputs, outcomes and benefits is an increasingly important question within Australian mathematics education research. The pathway from the development of new ideas that drive educational projects through to innovations that have broad influence at local, national and international levels, however, is often convoluted and notoriously difficult to strategise. In this paper, I develop a case study of an aspect of my own research to illustrate the complexity of the journey from research origin through to impact. The documentation of this research progress allows for reflection on how future impact can be “read” while research is taking place. The generation of innovative ideas and procurement of funds for testing new approaches in the field is at the heart of research in education. However, the “impact” of such research, educational outcomes and benefits to society, is no longer seen as a matter of potential but rather an expectation by increasingly numbers of funding bodies. While measures of research quality have been a topic of discussion for more than a decade within the Australian research context, the Engagement and Impact pilot currently being conducted by the Australian Research Council (ARC) provides evidence that a sharper focus will be drawn on this issue in future funding rounds. The proposed Engagement and Impact Assessment, as part of the National Innovation and Science Agenda (Australian Government, 2017) will consider: research interactions with a broad range of stakeholders including: industry; Government, non-governmental organisations; and research contributions to the economy, society and environment. This assessment will be conducted as a companion exercise to the Excellence in Research for Australia and is anticipated to be a significant consideration in future ARC application assessments. Such an exercise poses the challenge of how researchers will identify and then document the impact of their work? And, perhaps more importantly, dares researchers to think if it is possible to consider or strategise impact – to “read” impact – as a component of the research enterprise before and/or during the conduct of an investigation and not just in hindsight. In this paper, I present a case study based on my contribution to the Numeracy Across the Curriculum (NAC) research program to illustrate the complexity of the journey from research origin through to impact. The journey began with a small research project and was sustained via a series of both self-generated and serendipitous opportunities. Analysis of this journey will focus on how perceptions of impact can be constructed through hindsight. A brief reflection on implications for “reading” impact in order to inform decisions about future individual, team or institution research behaviour will conclude the paper. Research Impact Pathway As a feature of the advice provided by the ARC on the nature of research impact a Research Impact Pathway (RIP) table (Australian Research Council, 2017) was developed. The Pathway depicts impact as a progression through five junctures – Inputs, Activities, Outputs, Outcomes and Benefits. Forms of evidence for each juncture are exemplified within the table, for example, possible Inputs include research income, staff, background IP, infrastructure and collections, while the exemplars of Outcomes are listed as (2017). In A. Downton, S. Livy, & J. Hall (Eds.), 40 years on: We are still learning! Proceedings of the 40th Annual Conference of the Mathematics Education Research Group of Australasia (pp. 634-637). Melbourne: MERGA. commercial products, licences and revenue, new companies – spin offs, start ups or joint ventures, job creation, implementation of programs and policy, citations and integration into policy. Progression towards impact is presented in a sequential, linear fashion as if one step leads naturally to the next. What happens in practice however is likely to be more convoluted. The convoluted nature of my impact progression within the NAC program is presented in the remainder of this paper. Method Figure 1, complemented by Tables 1-3, is a representation of evidence within impact junctures, defined by the RIP table, against time. The evidence included here relates to aspects in which I have been directly involved within the NAC research program, for example, authorship or co-authorship of publications, as named investigator or coinvestigator of a project. Inputs are in the form of research income associated with projects (Table 1) supported via a number of funding sources. Only Activities (Table 2) that have direct connection to the NAC research program have been included. Outputs consist of publications and teaching resources. Outcomes have been identified as results of research that have receive attention across education systems or internationally. To date, I don’t believe the program can have serious claim to Benefits as exemplified in the RIP table. Thus, Benefits is represented as a blank rectangle in Figure 1. Figure 1. Representation of Geiger's RIP through the NAC research program. Solid black arrows have been used to indicate publications or resources that were a direct Output from research projects (Inputs). The number of Outputs connected to an Input is recorded next to an arrow when this exceeds one. Table 1 Inputs: Research Income Project Time A B C D E F G H I Project name 2009 2010-2011 2010-2012 2012 2012 2012-2014 2014-2015 Numeracy in the learning areas (middle years) Leading numeracy learning Make it count: Numeracy, mathematics and Indigenous learners Sustaining numeracy curriculum leadership: A whole school approach Models of leading curriculum reform in numeracy Enhancing numeracy learning and teaching across the curriculum Numeracy teaching across the curriculum in Queensland: Resources for teachers 2015-2017 Designing and implementing cross-curricular numeracy tasks for effective teaching and learning 2016 Review of the PIAAC numeracy assessment framework Table 2 Activities Activity Time J K Activity name 2016 ICME Topic Study Group Plenary 2014-2015 Guest Editor ZDM (Special Issue – Numeracy) Table 3 Outcomes Activity Time Activity name L M N O P Q R 2009 Organising structure for Numeracy in the Middle Years Curriculum – Department of Education and Children’s Services, South Australia 2011- An instructional planning tool adopted by Brisbane Catholic Education 2015 Numeracy skills framework – Department of Education NSW 2011- Numeracy teaching resource package on Education Queensland’s website 2015- Numeracy across the curriculum resource package on QCT Website 2016 One of Springer’s most downloaded chapters in the last two years and was made freely available as a part of their World Teacher’s Day promotion. 2011- Citations (177) Dashed arrows have been used to indicate outcomes from any of the preceding junctures – Inputs, Activities and Outputs. For example, the Outcome, Numeracy skills framework, developed by the Department of Education NSW draws directly on the NAC team’s research to provide system wide advice to teachers about the integration of numeracy across the curriculum. In contrast, the dotted arrows flow in the opposite direction of the assumed RIP illustrating how research Outputs can also feedback into Activities that eventually new projects (Inputs). The specific example identified in Figure 1 relates to how publication Outputs helped build a case for a special issue of ZDM – Mathematics Education on Numeracy (2015) (Activities). This issue, resulted in a personal invitation for me to be part of the organising committee for the Topic Study Group (TSG) on Mathematical Literacy at ICME 2016 (Activity) and an additional invitation to deliver a plenary with two NAC team members (Goos and Forgasz) within the TSG (Activities/Outputs). An invitation from the OECD for me to contribute to Review of the PIAAC numeracy assessment framework managed by ACER (Input) followed from this series of events. This new Input will be the foundation of additional outputs and activities. Conclusions The general, direction of flow from Inputs to Outputs to Outcomes in Figure 1 is consistent with that of the RIP table, shown with solid and dashed arrows. The role of Activities, however, seems to be absent in the actualised pathway of my contribution to the NAC program – giving the appearance that Activities play no role in moving from Inputs to Outcomes. Perhaps this is because the exemplars provided in the RIP table are too limited (research work and training, workshop/conference organising, facility use, membership of learned societies and academies, and community and stakeholder engagement), restricting my selection of Activities and so further consideration is needed for what happenings can be considered Activities. Journal articles, for example, are not a direct output from research income (Inputs) as first the research itself must be conducted, data gathered and analysed, the article written (and usually revised); all of which are more akin to Activities than Inputs. Additionally, some influences may be too subtle to be captured in a representation such as Figure 1, such as promoting the need for research on a particular issue through a Learned Society – which leads to funds from concerned parties becoming available for research; a circuitous but still productive pathway. The pathway indicated by the dotted arrows, however, flows in the opposite direction of that of the RIP table and demonstrates that RIP junctures can be bi-directionally influential; in this case moving from Outputs back to Inputs. Activities, in this pathway (Table 2), were significantly influential in highlighting the quality of the work in the NAC research program, leading to the inclusion of a NAC team member in a new project of international standing – a Review of the PIAAC numeracy assessment framework (Inputs). The preceding analysis shows that my contribution to the NAC research program can be mapped from inputs to outcomes via the RIP sequence but also demonstrates that Outputs, at least, can be backward mapped to new research endeavours (Inputs). The trends identified here leads to questions about individual and institutional behaviours related to impact. How would an analysis of impact differ when considering an individual, research team or institution and what strategic decisions would result at each level? Is it possible to utilise trends identified via RIP sequences using hindsight to make strategic decisions about the type Activities, Outputs or Outcomes an individual, research team or institution should pursue in order to make greatest impact and/or lead to further research? What measures can be employed in order to shorten the timeframe of the RIP sequence – adding substance to claims of successful research investment? These are questions that will need to be addressed as we move into the new era of Engagement and Impact. References Australian Government. (2017). National innovation and science agenda. Retrieved from http://innovation.gov.au/. Australian Research Council. (2017). Research impact pathway table. Retrieved from http://www.arc.gov.au/sites/default/files/filedepot/Public/ARC/Research%20Impact/Research_Impact_P athway_Table.pdf Bennison & Dole Engagement and Impact through Research Participation and Resource Development Anne Bennison Shelley Dole The University of Queensland <a.bennison@uq.edu.au> University of the Sunshine Coast <sdole@usc.edu.au> This paper utilises two projects that are part of a well-established research program seeking to embed numeracy across the curriculum to illustrate how knowledge exchange and uptake of resources can be used to provide evidence of research engagement and impact. The aim of the first project was to build teachers’ pedagogy around the promotion of proportional reasoning as a cross-curricular concept and a key component of numeracy; whilst that of the second was to develop resources to assist teachers to embed numeracy across the curriculum. Participation of stakeholders and the resources produced provide evidence of engagement and impact of the two projects. A national assessment of research engagement and impact that requires universities to provide evidence of how research is translated into economic, social, and other benefits is part of the Australian Government’s National Innovation and Science Agenda. An Engagement and Impact Assessment Consultation Paper, released in May 2016, sought feedback from stakeholders on how this assessment should be undertaken (ARC and DET, 2016). Several key issues associated with measuring engagement and assessing impact were raised in the Consultation Paper. The aim of this paper is to illustrate how knowledge exchange and the uptake of resources by teachers and education systems could be used to evidence research engagement and impact. Research Engagement and Impact Research engagement has been defined as the exchange of knowledge, understanding and resources that result from interactions between researchers and their wider communities (ATSE, 2015). The emphasis is on research benefit. In a recent review of trends and strategies for commercialising public research, the OECD (2013) noted that there are multiple ways in which research can be translated for economic and social benefits: Knowledge transfer and commercialisation of public research refer in a broader sense to the multiple ways in which knowledge from universities and public research institutions (PRIs) can be exploited by firms and researchers themselves so as to generate economic and social value and industrial development. (p. 18) Among the various forms of research engagement considered of high significance for industry, and seen therefore as more directly transferable (and hence more impactful) are collaborative research and contract research (ARC and DET, 2016). Both these types of research were undertaken as part of the Numeracy Across the Curriculum (NAC) research program undertaken by the presenters of this symposium. Impact is more difficult to define and assess. The ARC’s Research Impact Pathway table (http://www.arc.gov.au/research-impact-principles-and-framework#table) provides one way of identifying potential benefits of proposed research. The table includes examples of impact at five stages over the life of a research project and beyond its formal conclusion under the headings of Inputs, Activities, Outputs, Outcomes, and Benefits. (2017). In A. Downton, S. Livy, & J. Hall (Eds.), 40 years on: We are still learning! Proceedings of the 40th Annual Conference of the Mathematics Education Research Group of Australasia (pp. 638-641). Melbourne: MERGA. In this paper, we draw upon two specific projects from the NAC research program in an attempt to illustrate an approach to evidencing engagement and impact. The projects (an ARC Linkage and an Industry funded project) fit the categories of collaborative and contract research respectively, thus by their nature evidence research engagement. We analyse the impacts of these two projects in relation to inputs, activities, outputs, outcomes, and potential benefits to further evidence engagement and highlight the impact of each. Both these projects drew on the model of numeracy for the 21st century developed earlier in the NAC research program. According to this model, numeracy development encompasses five dimensions: mathematical knowledge, context, tools (representational, physical and digital), and positive dispositions toward the use of mathematics, which are embedded in a critical orientation (Goos, Geiger, & Dole, 2014). ARC Linkage Project The Enhancing Proportional Reasoning project was an ARC Linkage project conducted in 2010-2014 and included collaborative partner funding of 30%. The study found that improving teachers’ understanding of the elements of the numeracy model broadened their teaching focus beyond the teaching of mathematical knowledge. Teachers became more aware of the need to incorporate tools, including digital tools to enhance students’ numeracy ability. They more directly included a focus on student dispositions (e.g., confidence, resilience and risk taking) as a key element of promoting personal numeracy. Teachers also reported a greater awareness of and ability to identify “numeracy moments” in cross-curricular circumstances, thus broadening students’ numeracy development opportunities and making this acquisition more “real life” (see Dole, Hilton, G., & Hilton, A., 2015). Teachers indicated that regular identification of proportional reasoning teaching and learning opportunities in cross-curricular contexts led to students’ improved ability to identify and work with proportional situations as well as improving their meta-language, allowing them to communicate their ideas about proportional situations more precisely and concisely (see Hilton, A., Hilton, G., Dole, & Goos, 2016). This project has received international recognition for its research/practitioner focus (Hilton, A., Hilton, G., Dole, & Goos, 2013). Industry Project Numeracy teaching across the curriculum in Queensland: Resources for teachers was an Industry project conducted in 2014-2015 in response to a call from the Queensland College of Teachers (QCT). The project addressed a particular need of the QCT: to enhance the teaching of numeracy across the curriculum through web-based resources that could be made readily available to teachers via the QCT website. The project included a literature review of national and international good practice, an audit of existing material and consultation with stakeholders (e.g., employing authorities and teacher professional associations) to identify gaps and areas where teachers would benefit from new resources, developing video vignettes of examples of good practice in Queensland schools, and providing a brief report to the QCT (Goos, Geiger, Bennison, & Roberts, 2015). The theoretical framework that informed resource development encompassed the Board of Teacher Registration, Queensland (2005) Numeracy Standards and model of numeracy developed by the NAC research program. The audit of existing materials and interviews with stakeholders revealed that there are very few resources available to support teachers’ understanding and enactment of numeracy across the curriculum. The findings highlighted important gaps including that almost none of the existing materials addressed the need for teachers to develop the capacity to recognise and take advantage of the numeracy learning demands and opportunities within the subjects they teach. In response to these findings, six video vignettes were produced: an interview with a numeracy expert that explains some of the evidence base for the examples of good practice, a set of four classroom vignettes illustrating good practice in teaching numeracy across the curriculum at different year levels and in different subjects, and an interview with a school numeracy team that provides an example of how a whole school approach to numeracy can be developed. Discussion and Concluding Remarks It is possible to identify evidence of engagement and impact in the two projects presented here. There was knowledge exchange between researchers and stakeholders in the consultation process, publications, and availability of resources produced. The impact of the projects, using the column headings in the ARC’s Research Impact Pathway table, are summarised in Figure 1. ARC Research Impact Pathway Inputs Activities Outputs Outcomes Benefits ARC Linkage Industry Project Co-funding from ARC and industry partners (Education Authorities in two states) 5 regional school clusters in two states; 60 classroom teachers; 10 school leaders; PD package for teachers (10 modules); Resource development; State conference on Proportional Reasoning in both states (105 and 130 attendees respectively); 6 refereed journal articles; 5 refereed conference papers; 6 conference presentations; 1 international research award; Book proposal 11 invited keynote addresses, national and international; 20 teacher workshop presentations; Citations; Integration into school policy Potential for improved teaching practice and improved outcomes for learners Funding from the QCT Collaborative stakeholder engagement (teacher registration and employing authorities, teacher professional associations); Resource development Brief report to QCT; 1 refereed conference paper; 6 video vignettes Video vignettes made available on the QCT website (https://www.filmpond.com/#/ponds/qctthe-university-of-queensland) Potential for improved teaching practice and improved outcomes for learners. Figure 1. Mapping of project impact against ARC’s Research Impact Pathway. Knowledge transfer between researchers and stakeholders, along with resources that have been taken up by stakeholders, provide evidence of engagement and impact of the ARC Linkage Project and the Industry Project. The outcomes and outputs of the two projects are summarised in Figure 2 and illustrate how each project contributes to the engagement and impact of the NAC research program which has been conducted over a 16-year period by researchers in multiple universities, with outputs and outcomes being built upon in successive projects. Research Activities Sample Research Outputs and Outcomes 2010-2014 ARC Linkage Project 2014-2015 Industry Project Principles for task design and curriculum planning * * Professional development approach * * Whole school approaches to numeracy leadership * * Development of resources for teachers * * Assessment of numeracy capability * Figure 2. Impact map for ARC Linkage Project and Industry Project. This analysis also illustrates two issues identified in the Consultation Paper; that is, the time lag between research and benefits for end-users and the difficulty in attributing impact to a single project or university. Acknowledgements Funding from the Australian Research Council (Linkage Project LP100200222) and Queensland College of Teachers supported the projects discussed in this paper. References Academy of Technological Sciences and Engineering. (2015). Research engagement for Australia. Measuring research engagement between universities and end users. Retrieved from http://atse.uberflip.com/i/499806-research-engagement-for-australia-measuring-research-andengagement-between-universities-and-end-users-proposal Australian Research Council and Department of Education and Training. (2016). Engagement and impact assessment consultation paper. Retrieved from http://www.arc.gov.au/sites/default/files/filedepot /Public/ARC/consultation_papers/ARC_Engagement_and_Impact_Consultation_Paper.pdf Board of Teacher Registration, Queensland. (2005). Numeracy in teacher education: The way forward in the 21st century. Retrieved from http://qct.edu.au/pdf/Archive/BTR_NumeracyReport2005.pdf Dole, S., Hilton, A., Hilton, G. (2015). Proportional reasoning as essential numeracy. In M. Marshman, V. Geiger, & A. Bennison (Eds.), Mathematics education in the margins: Proceedings of the 38th Annual Conference of the Mathematics Education Research Group of Australasia (pp. 189-196). Adelaide: MERGA. Goos, M., Geiger, V., Bennison, A., & Roberts, J. (2015). Numeracy teaching across the curriculum in Queensland: Resources for teachers. Final report. Brisbane: The University of Queensland. Retrieved from http://qct.edu.au/pdf/Numeracy_Teaching_Across_Curriculum_QLD.pdf Goos, M., Geiger, V. & Dole, S. (2014). Transforming professional practice in numeracy teaching. In Y. Li, E. Silver, & S. Li (Eds.), Transforming mathematics instruction: Multiple approaches and practices (pp. 81-102). New York, NY: Springer. Hilton, A., Hilton, G., Dole, S., & Goos, M. (2016). Promoting students’ proportional reasoning skills through an ongoing professional development program for teachers. Educational Studies in Mathematics, 92(2), 193-219. Hilton, A., Hilton, G., Dole, S., & Goos, M. (2013). Proportional reasoning as a key to numeracy across the curriculum: Collaboration between practitioners and researchers. Paper presented at the European Association for Practitioner Research on Improving Learning in Education and Professional Practice Conference, Biel/Bienne, Switzerland. Organisation for Economic Co-Operation and Development. (2013). Commercialising public research: New trends and strategies. Paris, France: Author. Forgasz “Numeracy for Learners and Teachers”: Impact on MTeach Students1 Helen Forgasz Monash University <helen.forgasz@monash.edu> “Numeracy for learners and teachers” is a compulsory unit for all primary and secondary pre-service teachers in the Monash University Master of Teaching course. In the first two years that the unit was taught (2015 and 2016), research was conducted to evaluate the unit’s impact on students’ understanding of the construct, numeracy, and on their confidence to incorporate numeracy in their teaching across the curriculum. In both years, surveys were administered before commencement and on completion of the unit; a small number of students were also interviewed. The major findings from the two-year study are presented in this paper. Introduction There were two main drivers for the development of the unit, Numeracy for Learners and Teachers (EDF5017), as a compulsory study in the Monash University Master of Teaching (MTeach) course. All MTeach students, except those focusing only on becoming teachers in the early years, must complete this unit. The two drivers were: 1. Numeracy as one of seven general capabilities in the Australian Curriculum (AC; Australian Curriculum, Assessment and Reporting Authority [ACARA], 2016), the basis of the curriculum in each state/territory of Australia. “Teachers are expected to teach and assess general capabilities to the extent that they are incorporated within learning area content” (ACARA, n.d.-a) In the AC, numeracy is defined as encompassing “the knowledge, skills, behaviours and dispositions that students need to use mathematics in a wide range of situations. It involves students recognising and understanding the role of mathematics in the world and having the dispositions and capacities to use mathematical knowledge and skills purposefully” (ACARA, n.d.-b). 2. The graduate standards developed by the Australian Institute for Teaching and School Leadership (AITSL). These standards must also be met as part of the accreditation process for providers of teacher education. The specific AITSL (2014) graduate standards underpinning the development of EDF5017 were: • Standard 2.5 Literacy and numeracy strategies: “Know and understand literacy and numeracy teaching strategies and their application in teaching areas”. • Standard 5.4 Interpret student data: “Demonstrate the capacity to interpret student assessment data to evaluate student learning and modify teaching practice”. The definition of numeracy embraced in EDF5017 is that adopted in the AC and described above. The elements of the 21st Century Model of Numeracy (Goos, Geiger, & Dole, 2014) scaffolded the curriculum design for the unit. Focusing on the identification of numeracy demands and opportunities across all AC curricular domains at all grade levels, as well as developing the personal numeracy skills needed by practicing teachers were among the outcome goals of the unit. 1 I acknowledge the contributions of my colleague, Jennifer Hall, who conducted this research with me. (2017). In A. Downton, S. Livy, & J. Hall (Eds.), 40 years on: We are still learning! Proceedings of the 40th Annual Conference of the Mathematics Education Research Group of Australasia (pp. 642-645). Melbourne: MERGA. EDF5017 was first offered in 2015. In that year, the cohorts enrolled were MTeach (Secondary) and MTeach (Primary/Secondary) students; in 2016, the cohorts were MTeach (Primary) and MTeach (Early Years/Primary) students. Due to a revision of the timing of some MTeach offerings, from 2017, the four cohorts of students will be enrolled in the unit simultaneously. In this paper, I present some of the findings from a two-year study conducted with the EDF5017 students that was aimed at evaluating the unit’s overall impact. The research focus reported here is on students’ understanding of the construct, numeracy, and on the students’ confidence to incorporate numeracy in their teaching across the curriculum. The MTeach cohort split in 2015 and 2016 enabled the data to be examined for any differences among secondary pre-service teachers (2015) and primary pre-service teachers (2016). Methods and Analyses A mixed methods approach was adopted to gather data to evaluate the success of the unit. Pre- and post-surveys were developed so that changes in views in response to studying EDF5017 could be gauged. The items focused on in this paper include those designed to identify changes in the pre-service teachers’ understandings of the construct numeracy, and in their confidence to incorporate numeracy in teaching. Interviews were conducted with volunteers a short time after the post-survey had been completed, the main aim being to gather views on the content and structure of the unit. Results and Discussion The Samples The pre- and post-survey samples in 2015 and 2016 are shown in Table 1. Table 1 2015 and 2016 Pre- and Post-Survey Samples Pre-survey Participants Gender Age MTeach stream 2015 Post-survey 53 began; 40 finished 81% female 77% aged 25-34 Secondary only (74%) 35 began; 20 finished 74% female 74% aged 25-34 Secondary only (80%) Pre-survey 2016 Post-survey 46 began; 22 finished 90% female 80% aged 25-34 Primary only (79%) 21 began; 13 finished 81% female 86% aged 25-34 Primary only (90%) As can be seen in Table 1, most respondents were female, and most were aged 25-34. Of the 2015 cohort, more were enrolled in MTeach (Sec) than in MTeach (Prim/Sec); for the 2016 cohort, more were enrolled in MTeach (Prim) than in MTeach (EY/Prim). Findings Differences between numeracy and mathematics. Responses to the item “Are there differences between mathematics and numeracy?” (Yes/No/Unsure) to the 2015 and 2016 pre- and post-surveys are shown in Table 2. Table 2 Are there Differences between Mathematics and Numeracy? 2015 Yes No Unsure 2016 Pre-survey (n = 45) Post-survey (n = 21) Pre-survey (n = 29) Post-survey (n = 13) 76% 4% 20% 95% 0% 5% 90% 0% 10% 92% 8% 0% As can be seen in Table 2, for the 2015 cohort, there was a noteworthy increase in the proportion of secondary pre-service teachers who answered “Yes” after completing studies in EDF5017. While starting from a very high base (90% of students), among the 2016 primary pre-service teacher cohort, there is no noticeable difference in the proportions saying “Yes” in the pre- and post-surveys. One possible explanation for the 2016 cohort being aware that there is a difference is that this cohort had already completed units in the teaching of primary mathematics and the issue had already been discussed in those units. Participants were also asked to explain their answers to the question. Typical answers are shown below: I'd never really given it much thought before now. Both scare me!!! One is the subject, the other is the application of the subject in real life situations. Mathematics is to numeracy what language is to literacy - only part of the whole. I think that numeracy is a broader concept than mathematics, because otherwise we wouldn’t have pure maths. Confidence incorporating numeracy into the teaching of their subject area(s). On both the pre- and post-surveys, respondents were asked to indicate on a 5-point response format (very lacking in confidence to very confident) how confident they felt about incorporating numeracy into the teaching in their subject area/s. The pre- and post-survey responses for the 2015 and 2016 samples are shown in Figures 1 and 2 respectively. It can be seen in both Figures 1 and 2, that there were noteworthy changes in confidence from pre- to post-survey. That is, in both samples of pre-service teachers, more were somewhat or very confident after studying EDF5017 than before. Figure 1. Pre- and post-survey responses from 2015 participants. Figure 2. Pre- and post-survey responses from 2016 participants. The following comment from one of the 2015 (secondary) post-survey respondents encapsulates the sentiments of many of the students: I have a clearer understanding of what numeracy entails, have been provided examples with how it would work in my method curriculum areas, and feel confident that I have adequate mathematical reasoning and numeracy skills to be able to handle this in my teaching. In the post-survey only, students were asked if the unit had impacted their views of numeracy. The majority responded, “Yes” (86% in 2015, 85% in 2016). Some typical explanations for their positive responses included: I did not know the word before this unit. I understand it is my responsibility to teach this [numeracy] – AITSL and curriculum require it. I now know the difference between mathematics and numeracy. Final Words Clearly, completing EDF5017 resulted in a substantial and important impact on students’ confidence in incorporating numeracy in their teaching, and in having a better appreciation of what numeracy is and how it differs from mathematics. Units such as EDF5017 are now expected for accreditation of teacher education programs. Based on the findings reported here, it is anticipated that the benefits to the school population and the future citizenry of Australia are likely. References Australian Curriculum, Assessment and Reporting Authority. (2016). General capabilities. Retrieved from http://www.acara.edu.au/curriculum/general-capabilities Australian Curriculum, Assessment and Reporting Authority. (n.d.-a). General capabilities. Overview. Introduction. Retrieved from http://www.australiancurriculum.edu.au/generalcapabilities/overview /introduction Australian Curriculum, Assessment and Reporting Authority. (n.d.-b). Numeracy: Introduction. Retrieved from http://www.australiancurriculum.edu.au/generalcapabilities/numeracy/introduction/introduction Australian Institute for Teaching and School Leadership. (2014). Australian Professional Standards for Teachers. Retrieved from http://www.aitsl.edu.au/australian-professional-standards-forteachers/standards/list Goos, M., Geiger, V., & Dole, S. (2014). Transforming professional practice in numeracy teaching. In Y. Li, E. Silver, & S. Li (Eds.), Transforming mathematics instruction: Multiple approaches and practices (pp. 81-102). New York, NY: Springer.