A GROUNDED APPROACH TO THE INVESTIGATION OF ASSESSMENT PRACTICES IN BUILT ENVIRONMENT UNDERGRADUATE PROGRAMMES Lloyd Scott1 and Christopher Fortune2 1 2 Faculty of Built Environment, Dublin Institute of Technology, Dublin 1, Ireland School of Built Environment, University of Salford, Salford, UK Assessment practices in Higher Education (HE) have been undergoing wide-ranging changes over the last number of years and this has been particularly evident in a number disciplines. These changes are in response to a variety of stimuli including a move towards greater accountability, new developments in the use of learning technology and concerns about what graduates need to know, to understand and to be able to do following graduation. The discipline of the Built Environment has been receiving attention in this regard and the validity and effectiveness of traditional modes of assessment have begun to receive consideration. Formative assessment and the use of feedback mechanisms have begun to be recognised as a driving force for enhancing student learning. The above situation is examined in the context of Built Environment undergraduate programmes and discusses the need for a research project in the context of the changing HE educational environment. The aim of such a research project is to help improve the quality of student learning in Built Environment undergraduate programmes. Seminal literature is explored in order to identify, inform and shape the assessment practices of academics. A design is articulated for the research which uses a grounded theory-like approach to conduct the preliminary study. The results of the initial research, when analysed, set out the views and preferences of senior academics and help inform the next stages of this work in progress. The ongoing work anticipates developing a grounded model for the formative assessment of Built Environment undergraduates for the enhancement of student learning. Keywords: built environment, education, formative assessment, grounded theory. INTRODUCTION “What the student does is actually more important in determining what is learned than what the teacher does” (Shuell, 1986) This paper focuses on some of the wide-ranging changes that have taken place in assessment practices employed in Higher Education. It looks at assessment in general and is followed by setting out its context within the Built Environment, defining it as a theoretical entity. The research design for the research enquiry is considered and explored and the chosen methodology explained and defended. The scope of the research project is offered along with the results of the initial phase. Some early considerations are presented and the next stage of the process identified and how the initial research might impact on the research project considered. 1 lloyd.scott@dit.ie Scott, L. and Fortune, C.J. (2009) A grounded approach to the investigation of assessment practices in built environment undergraduate programmes. In: Dainty, A. (Ed) Procs 25th Annual ARCOM Conference, 7-9 September 2009, Nottingham, UK, Association of Researchers in Construction Management, 475-84. 475 Scott and Fortune THE CHANGING LANDSCAPE OF HIGHER EDUCATION The drivers of change in HE are numerous and the pressures for that change are occurring globally. Higher education in Ireland has not been ignored on this front. Changes have been brought about in quality assurance arrangements; a National Framework of Qualifications has been introduced and at institutional level many HE establishments have a stated objective of enhancing the student learning experience. An example of this, the Dublin Institute of Technology, a multi-disciplinary provider of Higher Education in Ireland, has put in place a strategic imperative to develop a multi-level learner-centered learning environment through the roll out of a modular structure. A new learning environment was supported by the National Qualifications Authority of Ireland (NQAI) requirement that all awards should be defined in terms of learning outcomes, the achievement of which would be confirmed through the use of appropriate assessment strategies. The traditional approaches to assessment in HE typically place heavy reliance on tacit understanding of standards and in the current environment of rapidly changing contexts this can be a point of strain. Examples of the rapidly changing contexts which have encouraged practitioners to look at more innovative approaches to assessment include massification, new kinds of teaching and learning, computer-aided assessment, new approaches to intended learning outcomes and declining resources. THE IMPACT OF ASSESSMENT ON STUDENT LEARNING Assessment practices in HE have been undergoing wide-ranging changes over the last number of years and this has been particularly evident in a number of disciplines. These changes are in response to stimuli including a move towards greater accountability, new developments in the use of learning technology and concerns about what graduates need to know, to understand and to be able to do following graduation. The discipline of the Built Environment has been receiving attention in this regard and the validity and effectiveness of traditional modes of assessment have begun to receive consideration. Assessment in HE is a very complex business and as assessment is something that is experienced by almost all involved in HE it is important that an assessment system is recognisable and understood by all. There are many reasons to assess students and Brown et al. (1996) discuss ten. Five of these refer to traditional summative assessment and the need for evidence and the classification of learning. The other five focus on formative assessment through guidance for improvement; providing opportunity for students to rectify mistakes to diagnose faults; motivation; providing variety in assessment method and providing direction to the learning process. This might imply that equal importance is placed on both formative and summative, but this is not the case. An investigation of the assessment practices in undergraduate programmes in Built Environment indicates that while the „tide is starting to turn‟ there is still an over reliance on the traditional summative examination at the end of a module or unit of learning. The seminal research material on formative assessment and the use of feedback mechanisms indicates that these methodologies have begun to be recognised as a driving force for enhancing student learning. This has yet to have a complete impact at programme or module level in many undergraduate BE programmes. Research literature informs us that assessment is most effective when it is closely aligned to the learning outcomes. Cross (1996) refers to assessment and feedback as providing one of three conditions for learner success. It is generally acknowledged that a student‟s 476 Undergraduate assessment practices approach to learning and the quality of learning achieved will be influenced by the way in which this learning is to be assessed (e.g. Gibbs, 1999; Entwistle and Ramsden, 1983). In addition, adopting a holistic approach to curriculum design that seeks to constructively align assessments with the learning outcomes, and teaching and learning methods that create a seamlessly inter-related curriculum (Biggs1999) are important if a diversity of desired learning outcomes is to be achieved (e.g. Gibbs, 1999). Boud (1995) also identifies a need to move from seeing particular assessments in isolation towards recognising them as linked to the other kinds of assessment used, the proximity, frequency and also the context of their usage. Furthermore, bunching of similar types of assessment at certain key points, perhaps at the middle and end of programmes, is likely to result in students‟ adoption of a surface approach and the attainment of a limited number of lower-level learning outcomes (Scouller, 1996). In other words, cross programme strategic planning of appropriate assessments is fundamental if the intention is for students to attain higher-level learning outcomes such as problem solving and critical thinking (Biggs, 1999; Gibbs, 1999).The critical importance of formative assessment (assessment that contributes to the student‟s learning through the provision of feedback about performance; Yorke,2003) should not be underestimated by lecturers and is confirmed by the review work of Black and Wiliam (1998). Assessment for learning, more commonly understood as formative assessment, is defined by Black and Wiliam (1998, p.22) as “all those activities undertaken by teachers and/or by their students, which provide information to be used as feedback to modify the teaching and learning activities in which they are engaged”. In very simple terms, assessment may be defined as such activities that measure student learning. Boud (1990) posited that assessment has two purposes, firstly that of improving the quality of learning where learners engage in activities and are given feedback that will direct them to effectiveness in their learning (commonly referred to as formative feedback). The second concerns that of the accreditation of knowledge or performance, which occurs generally for the award of a degree or diploma (commonly referred to as summative assessment). Today, students are more focused and they approach assessment with a better understanding of what is involved. Bloxham and Boyd refer to students as “being cue conscious concentrating on passing an assessment” (2007, p.19). We now hear academics speak in terms of formative and summative assessment, however we have a long way to come before assessment and feedback become central to learning and, in turn, to the student experience. With the importance of life-long learning beginning to permeate thorough HE, along with the impact of the National Frameworks of Qualifications in Ireland, a greater, more explicit emphasis and attention is being paid to learning outcomes and competencies. A student-centred learning framework puts the learner at the centre of the learning process, in which assessment plays an important part. It is widely accepted that assessment has a direct impact on students‟ learning (Askham, 1997; Black and Wiliam, 1998; Stiggins, 2002). We are all familiar with the term that assessment drives learning; this is true in many instances, where the learner looks at what has to be learned in terms of what he or she needs to do to pass the assessment and get a good grade. Research indicates that what students focus on during the course of their studies is hugely influenced by the assessment methods employed to measure the learning experienced (Ramsden, 1992). Therefore, the importance of taking cognisance of assessment for learning and assessment of learning has relevance for lecturers in the design of their assessment 477 Scott and Fortune strategies. Assessment of learning is where assessment for accountability purposes is paramount; its function is to determine a student's level of performance on a specific task or at the conclusion of a unit of teaching and learning. The information gained from this kind of assessment is often used in reporting and is purely of a summative nature. However, assessment for learning, on the other hand, acknowledges that assessment should occur as a regular part of teaching and learning and that the information gained from assessment activities can be used to shape the teaching and learning process. It can, most importantly, also be used by the learner to enhance learning and achievement. Gibbs and Simpson (2004) have developed a model that promotes eleven conditions under which assessment supports learning, as outlined in table 1 below. Seven of the eleven conditions refer to feedback. Table 1: Gibbs and Simpson (2004) promoting 11 conditions under which assessment supports learning 1. Sufficient assessed tasks are provided for students to capture study time 2. These tasks are engaged with by students, orienting them to allocate appropriate amounts of time and effort to the most important aspects of the course 3. Tackling the assessed task engages the students in productive learning activity of an appropriate kind 4. Assessment communicates clear and high expectations 5. Sufficient feedback is provided, both often and in enough detail 6. The feedback focuses on students‟ performance, on their learning and on actions under the students‟ control, rather than on the students themselves and on their characteristics 7. The feedback is timely in that it is received by students while it still matters to them and in time for them to pay attention to further learning or receive further assistance 8. Feedback is appropriate to the purpose of the assignment and to its criteria for success 9. Feedback is appropriate, in relation to students‟ understanding of what they are supposed to be doing 10. Feedback is received and attended to 11. Feedback is acted upon by the student. THE BUILT ENVIRONMENT While not the main focus of this paper it is necessary to consider and conceptualise the field of Built Environment (BE). Human society has found it necessary to categorise the different forms of knowledge since well back to the times of Aristotle and Plato in an attempt to make the world more intelligible. Those associated with the BE are no different in this regard. It has begun to emerge as a distinct discipline in the more recent past; however in that discourse it has been identified as problematic. Boyd (2007) refers to the general conception of the BE as one of a „development process‟ and he argues that it does not exist theoretically. Ratcliffe (2007), on the other hand, proffers that while the BE is both vague and elusive it is a generic phrase of distinction and pertinence and is best portrayed and understood „as a set of processes‟ rather than one single entity. This set of processes includes the planning process, design process, construction process, regulatory process, financial process, transportation process and information process. Griffiths (2004) describes it as a range of practice-orientated subjects concerned with the design, development and management of buildings, spaces and places‟. In HE the field of BE has begun to make significant headway as a recognised discipline where schools of Built Environment have been set up and begun to flourish. The UK Research Assessment Exercise sub-panel makes reference to the field as 478 Undergraduate assessment practices encompassing „architecture, building science and engineering, construction and landscape urbanism‟ (HEFCE, 2005). While school and department configuration is often a matter of the culture of a Higher Education institution, reference to BE by the RAE is acknowledgement of the existence of this discipline. In the Irish HE context, while considered very much at a developmental stage, the field of BE has begun to be recognised and embedded as a distinct discipline. Again, schools and faculty have emerged in the organisation structure of Higher Education institutions across the country. For the purpose of this research the BE refers to the disciplines of architecture, architectural technology, construction management and construction economics. These disciplines will be the focus of the research as they are the most representative group in terms of BE programmes offered in HE on the island of Ireland. In all the main providers of BE education at undergraduate level, the above areas are offered. RATIONALE FOR RESEARCH DESIGN Human beings have always shown an interest and concern to come to terms with their environment and to try to make sense and understand the nature of the phenomena to their senses (Cohen et al., 2001).At the commencement of any research project many question occupy the thought of the researcher. What does this journey entail? Where to start? What philosophical stance should one take? What research methods should be employed to effectively achieve the goal(s) of the research? All research needs to be subjected to careful methodological assessment and reflection while theory provides the discourse and a vocabulary to describe what we think. In this regard, the principal aim of the research is to help to improve the quality of student learning in Built Environment undergraduate education. The central research question therefore can be summarised as: Are assessment practices currently in use in BE education maximising their potential to improve the quality of students‟ learning? In attempting to address the aim of the research several research questions are posed:  How are academics in BE education currently assessing learning?  To what extent do academics align their assessment practices to educational theory?  Are the institutional procedures around assessment in conflict with the embedding of a student-focused assessment strategy?  What are students‟ experiences and perceptions of assessment?  What are students‟ experiences of formative assessment and feedback?  To what extent do the existing assessment methods encourage a deep approach to learning?  Do students get an opportunity to reflect on their learning?  What model can be developed that will enhance the experiences of students with respect to assessment?  How will the improvements brought about by this new model be measured? A research framework gives the theoretical background to a research project and most researchers take time to „struggle‟ and come to terms with the theoretical aspects of 479 Scott and Fortune the task. Saunders et al. Research Onion model (2003:87) provides an appropriate form within which to frame this research inquiry. Traditional research design strategies usually rely on a literature review leading on to the formation of a hypothesis which can be put to test by experimentation in the real world. The use of ethnographical and case study approaches can, however, limit the researcher. Grounded Theory (GT), on the other hand, investigates perceived actualities in the real world and analyses that data with no preconceived hypothesis (Glaser and Strauss, 1967). Creswell (2008) offers indicates types of GT, the systematic procedure associated with Strauss and Corbin (1998), the emergent design aligned with the Glaser (1992) and the constructivist approach espoused by Charmaz (2006).The constructivist GT approach which is positioned between the more positivistic stances of Glaser & Strauss and Corbin and the post-modern researchers Lyotard, Foucault and Derrida, and in the camp of Charmaz and Bryant, who question the importance of method, is favoured. The focus on gaining an understanding of the meaning the participants have is an important factor in this research and hence a constructivist bias. Their views, values, assumptions and ideologies with respect to assessment in the BE education are what are sought. The research process considered and developed to address the methodological requirements of a GT approach (Bryant, 2002). The basic GT guidelines are adopted in line with twenty-first century methodological approaches and assumptions. The analysis of data from each of the interview phases, along with data gathered from the survey of academics will influence the emergent theoretical model. The first phase of the research process Table 2: Concepts and codes arising from the first phase of the research enquiry Concepts Purpose of assessment Learning and teaching Academic Summative assessment Formative assessment Open Codes Examination, coursework, regulations, assessment criteria, policies and procedures, summative assessment, formative assessment, holistic assessment, compliance, Teaching methods, improve student learning, innovative practice, scaffolding, reflective learners, modularisation, semesterisation, constructive alignment, student centred learning, independent learning, over assessment, modules, active learning Changing practice, learning outcomes approach, traditionalists, coursework, staff development, innovation, course board, Examination, coursework, portfolio, measurement, variety, practical tests, peer assessment Importance of formative assessment, student involvement, peer assessment, feedback, continuous assessment, portfolio, flexibility, In the first phase of the research semi–structured interviews were conducted with five senior academics in management positions between September and November 2008 from Schools in the University/Institutes of Technology sector around the island of Ireland. The interviews lasted up to one hour and were taped following agreement with the interviewees; transcription followed each interview. In a GT approach analysis involves the assignment of concepts and themes to the data gathered, a process recognised as coding. This process was adopted in the case of the data from the interviews. From the analysis the emerging themes and concepts are identified in table 2 from this first phase of the research. 480 Undergraduate assessment practices One emerging concept that is very much identifiable among the Heads interviewed is the difference in philosophical position with respect to assessment and how they view the assessment of student learning. The analysis of the data reflects differing positions as evidenced by the quotes below: „The academic staff of the School working with the students are not looking at just the final product as presented but are looking at the process by which the final product was arrived at‟ (Interviewee A) „From a management perspective …. I see it being engaged a lot with the compliance with National Framework of Qualifications and adopting changes in relation to learning outcomes process‟ (Interviewee B) „This is because we have always proportioned our assessment into end of year exams and coursework‟ (Interviewee C) Tradition and academic discipline influence the attitude towards the approach to assessment, while the type of educational organisation too has a distinct impact. The importance of assessment in the educational process was alluded to by all and that formative assessment has an important part to play in this. However the mechanism on how this is achieved differed between each Head. There is a disparity of understanding in the purposes of assessment, particularly as we move towards a more student-centred learning environment. This is evidence by the approach taken in the different institutions with respect to the design of the assessment strategies at module and programme level. There are elements of re-formulating position based on the learning outcomes paradigm in which we find ourselves. For example the interviewee B stated: „what we haven‟t done is link assessment methodologies to module learning outcomes‟ This further emphasises the traditional approach adopted in many programmes and the reliance on the measurement assessment strategy as opposed to a more holistic strategy. There is evidence of student engagement in active learning tasks as referred to by interviewees 3, 4, and 5; however those tasks are not linked to the overall assessment strategy. Students are required to take a summative exam at the end of a module where they may have demonstrated the achievement of the learning outcomes during the active learning tasks. A clear example of „over assessment‟ and a reliance on the traditional summative examination. This position reflect the polarised position across academic institutions in their advancement to the more „constructively aligned model‟ advocated by Biggs (1999). This is a common position not just in the BE but across many other disciplines as academic engage in reflecting on and introducing a learning outcomes based approach. One interviewee (4) indicated that academic lecturing staff are unaware that they are „empowered‟ to make the appropriate changes to effect learning and hence the more traditional approaches are the preserve. There is still an over reliance on the „formal summative assessments‟ or controlled examination as a means of verifying student attainment. There is clear tension between the summative assessment and the formative assessment processes and using this knowledge/ information to help teaching and learning. Again, the diverse position of each school along the continuum is very much in evidence. In some instances there has been full engagement in the alignment of 481 Scott and Fortune programme and module learning outcomes while other schools have only just begun to grapple with this. This one would feel has a direct relationship with their approach and configuration of the assessment strategies employed. This is allied to a complete agreement of the need to strengthen the processes of assessment and in particular the formative assessment elements. The down side is there is no real sense or vision of how this might be achieved. The notion of developing reflective practice through assessment and its contribution to enhancement of student learning and motivation is referred to. Student involvement in assessment where the academics can benefit from the use of peer assessment on various levels was identified as problematic. The analysis suggests that it happens in a very limited amount of cases. Interviewee 5 indicated that students „do not perhaps participate as much as they should and that there should be more opportunities to engage the learner more‟. The often laborious process of marking student work can be potentially reduced if some of the assessment is carried out by the students. More fundamentally it can be used to open meaningful dialogue about the work and enhance feedback opportunities. Time constraints and the difficulties associated with peer assessment are cited as the issues associated with engaging students in the assessment. The risks of involving students in their own or colleagues‟ assessment should not be underestimated. There is intense pressure on the higher education sector to maintain standards. Any change to assessment practice must be able to withstand scrutiny and above all be rigorous and transparent (Race 2001). There are fears that putting assessment in the hands of the students will make the assessment less reliable. To ensure consistency, measures can be built in, including multiple assessment of the same piece of work by a number of students. Clear definition of marking criteria is another essential element of successful peer assessment. Criteria may be developed with students, but if this is not possible, at the very least they must be made clear prior to students attempting the exercise. Another emergent theme was the need for inter and intra collegial discussion/ discourse opportunities to discuss not only assessment practices in the Built Environment but also other pertinent pedagogic matters. Ways should be explored of how we might share best practice and how this might begin to effect change in the discipline. This emerged where interviewees made comment on the need to for staff development and training. CONCLUDING THOUGHTS This paper has provided a summary overview of the author‟s research to date with the scrutiny of senior academics regarding their views and experiences in the context of assessment practices in undergraduate Built Environment education. As this is workin-progress, the paper focused on the methodology employed and a number of key issues emerging from the segment of data analysed thus far. There is a strong history of assessment in the programmes offered in Built Environment undergraduate programmes, particularly the more formal summative assessment. One of the questions to be addressed in the next phase of the process is the extent to which academics are engaging with the most up to date and effective assessment process that will enhance student learning. Interviews with the programme managers and a survey of the built environment academic community will endeavour to address this. The analysis of the results of the initial research set out the views and preferences of senior academics and will help inform the next stages of this work in progress. The ongoing 482 Undergraduate assessment practices work anticipates developing a grounded model for the formative assessment of Built Environment undergraduates for the enhancement of student learning. REFERENCES Askham, P. (1997) An instrumental response to the instrumental student: Assessment for learning, Studies in Educational Evaluation 23(4), 299-317. Biggs, J. (1999) Teaching for Quality Learning in University, Open University Press. Black , P. and Wiliam, D. (1998) Assessment and classroom learning in Assessment in Education, 5:1, pp. 7-74. Bloxham, S., Boyd, P. (2007) Developing Effective Assessment in Higher Education, Open University Press Boud, D. (1990) Assessment and the promotion of academic values, Studies in Higher Education, 15, 1, 101-111. Boud, D. (1995) Assessment and Learning: contradictory or complimentary, in Knight, P. (Ed), Assessment for Learning in Higher Education, London: Kogan Page. Boyd, D. (2007) Searching for a unified theory of property and construction. In Kosela, L and Roberts, P. (Eds), „Towards the Foundation of Theory for the Built Environment Symposium‟ 18-19 June 2007, University of Salford, Research Institute for the Built Environment Brown S., Race P., & Smith B., (1996) 500 Tips on Assessment London: Kogan Page Bryant, A. (2002)Re-grounding grounded theory Journal of Information Technology Theory and Application, 4, 1, 25-42 Charmaz, K. (2006) Constructing Grounded Theory A Practical Guide Through Qualitative Analysis Los Angeles Sage Cohen, L., Manion, L., and Morrison, K. (2001) Research Methods in Education (5th Ed.)Routledge /Falmer : London & New York Creswell, J. W (2008) Educational Research: 3rd Edition New Jersey: Pearson Prentice Hall. Cross, K.P. (1996) Improving teaching and learning through classroom assessment and classroom research, in Gibbs,G. (Ed), Improving student learning using research to improve student learning, , K., Stanley,N., Davison,N. (Eds), Teaching in the Disciplines/ Learning in Context, pp108-113. Dainty, A R J. (2008) Methodological pluralism in construction management research in Knight, A. and Ruddock, L. Advanced Research Methods in the Built Environment United Kingdom Wiley-Blackwell Entwistle, N.J., Ramsden,P. (1983) Understanding Student Learning, London: room Helm. Gibbs, G. (1999) Using assessment strategically to change the way students learn in Brown, S., Glasner, A. (Ed) Assessment Matters in Higher Education: choosing and using diverse approaches, Buckingham: Open University Higher, pp 41-53. Gibbs,G. and Simpson,C. (2004) Conditions under which assessment supports student learning. Learning and Teaching in Higher Education 1, pp3-31 Glaser, B. G. (1992). Basics of grounded theory analysis: Emergence vs. forcing. Mill Valley, CA: Sociology Press Glaser, B. and Strauss, A. (1967) The Discovery of Grounded Theory Chicago, Aldine Griffiths, R. (2004) knowledge production and the research- teaching nexus: the case of the built environment disciplines, Studies in Higher Education, 29, 6, 709-726 483 Scott and Fortune HEFCE (2005) RAE 2008 Consultation on Assessment Panels‟ Draft Criteria and Working Methods - UOA30, Architecture and the Built Environment London: higher Education Funding Council of England Race, P. (2001) A briefing on self peer and group assessment. LTSN (Learning and Teaching Subject Network) Generic Centre, Assessment Series 9 URL: http://tinurl.comyttsswqs (accessed: 31 January 2008) Ramsden, P. (1992) Learning to Teach, London:Routledge. Ratcliffe, J (2007) Built Environment Futures: adopting the foresight principle in formulating and applying a theoretical approach towards the creation of a sustainable built environment. In Kosela, L and Roberts, P. (Eds), „Towards the Foundation of Theory for the Built Environment Symposium‟ 18-19 June 2007, University of Salford, Research Institute for the Built Environment Sadler, D.R. (1989) Formative assessment and the design of instructional systems, Instructional Science, 18, pp145-165. Saunders, M., Lewis,P., Thornhill, A. (2003) Research Methods for business Students, Prentice Hall Scouller, K. (1996) Influence in assessment methods on students‟ learning approaches, perceptions and preferences: assignment essay versus short answer questions, Research and Development in Higher Education, 19, 3, 776-781. Shuell, T. (1986) Cognitive conceptions of learning, Review of Research in Education, 19, 405-450. Stiggins, S. (2002) Assessment crisis: the absence of assessment for learning Phi Delta Kappan, 83, 10, 758-765. Strauss, A. and Corbin, J. (1990) Basic of Qualitative Research: Grounded Theory Procedures and Techniques Newbury Park CA Sage Publications Yorke, M., (2003) Formative assessment in higher education: moves towards theory and the enhancement of pedagogic practice, Higher Education, 45, 477-501. 484