Development of professional competencies in engineering education

Session S1C Development of Professional Competencies in Engineering Education Åsa Cajander, Mats Daniels, and Brian R. von Konsky Asa.Cajander@it.uu.se, Mats.Daniels@it.uu.se, and B.vonKonsky@curtin.edu.au This paper reports on a study conducted at a Swedish university in which engineering students reflect on their attainment of professional competencies using generic graduate attributes as a guiding framework. In this study, the nine graduate attributes from Curtin University in Perth, Australia were used as this framework. The Swedish students were asked to reflect on their understanding of the competencies, and to select three on which to focus over the course of the semester. Students predominantly selected professional skills, thinking skills, information skills, communication skills, and intercultural understanding. In a subsequent reflection, students were asked to self-assess improvement of their chosen attributes. Results showed that while the teaching staff judged that the students had further developed the competencies, students lacked a framework to see this for themselves and reported difficulty in reflecting on progress in professional competency attainment. This suggests that a more holistic approach should be taken earlier in the degree to focus student attention on professional competencies, and provide ongoing experience reflecting on progressive achievement. Index Terms: Engineering education, Professional competencies, Graduate attributes, Reflections, Open-Ended Group Projects. INTRODUCTION Development of professional competence is typically stated in the goals of engineering education programs but is often not communicated in the description of the individual courses within the same programs. There are several reasons for this, but the perhaps most prominent is the nontrivial challenge of agreeing on professional competency definitions. This paper suggests ways to define and assess these graduate attributes. Experiences creating learning environments that contribute to the development of competencies in engineering education at two universities will be presented and analyzed. Both of these case studies are based on the graduate attributes defined by Curtin University in Perth, Australia. The first case study describes Curtin experience with an electronic portfolio, called the iPortfolio, used by students to self assess attainment of Curtin’s graduate attributes. The other case study describes the use of Curtin’s graduate attributes in a course at Uppsala University in Sweden. In the latter example, students and staff agreed on personal learning goals for individual students, following one-on-one interviews regarding student perceptions of the these attributes and their meaning. The development of the Swedish students’ selected competencies was supported through the use of reflection at various points during the course, partly based on inspiration from the reflective practitioner work by Schön (1987). The activity showed promise and we will report on the outcome and use an action research manner to speculate on how the approach can be improved in the next instance of the course. We will conclude with a discussion about how a learning technology such as the iPortfolio can aid in the development of students’ professional competencies. BACKGROUND Previous research concluded that the nomenclature, definitions and perceived significance of generic competencies, also called graduate attributes, capabilities, and outcomes, are not universally accepted or agreed upon by the various stakeholders [Dörge, 2010; Oliver 2010]. There is an enormous quantity of literature on this topic, but for brevity three sources provide some insight in relation to engineering education: the Definition and Selection of Competencies (DeSeCo) project from the Organization for Economic Co-operation and Development (OECD), the Australian Council for Educational Research (ACER), and ABET (formally known as the Accreditation Board for Engineering and Technology) from USA. A brief description of these large international approaches to defining competencies follows. OECD The OECD countries started the Program for International Student Assessment (PISA) in 1997 [OECD 2005], leading to the Definition and Selection of Competencies (DeSeCo) Project. The goal was to ensure that students nearing the end of their compulsory schooling and young adults have the knowledge and skills necessary to function in modern society. This effort includes periodic assessment and comparison of skills in reading, mathematics, science and problem solving. At the same time, there is an understanding that success in life depends on a far wider set of competencies that fall into three broad categories: using tools interactively, including language and technology; 978-1-61284-469-5/11/$26.00 ©2011 IEEE October 12 - 15, 2011, Rapid City, SD 41st ASEE/IEEE Frontiers in Education Conference S1C-1 Session S1C acting autonomously, and interacting inn heterogeneous groups. The DeSeCo view is that competenciees are more than just a set of knowledge and skills. Compettencies must also capture the ability to deal with compleex situations in particular contexts. The key competencies needed to deal with such situations must: contribute to valuued outcomes for societies and individuals; help individualss meet important demands in a wide variety of contexts; and be important not just for specialists but for all individuals. Moreover, key competencies defined by the DeSeCo project recognize that technology is changging rapidly and continuously, that societies are becoming m more diverse and compartmentalized, and that globalization is creating new forms of interdependence. ACER The Australian Council of Education R Research (ACER) has developed and evaluated a Graduate Skills Assessment (GSA) test [ACER 2002]. Skills evaluatedd in the GSA test include critical thinking, problem solvinng, interpersonal understandings, and written communicatiion. These are generic in nature, and are intended to bee applicable in a variety of workplaces. ACER was commissioned to generatee assessment for transferable competencies that have broad relevance to academic work and graduate employmentt. Competencies were defined at a meta-level, and includeed the ability to identify, select, and apply an appropriate reepertoire of more specific knowledge and skills to deal with ddiscipline specific tasks. This was based on the premise that ccompetencies are generic and transferable across disciplines. ACER built their work partly on the Mayer competencies [Mayer Committee 1992] prresented a list of employability skills and competencies thatt they considered suitable to be addressed by formal educaation. However, ACER saw these as limited in that they oomitted personal traits and were not based on any ttheory of skill development. This can be contrasted with a rather different statement from the Association of Graduatted Recruiters in UK [Association of Graduate Recruiters 19995], where selfreliance skills are seen as particularly impoortant. Examples of such skills are self-awareness, self-prom motion, exploring and creating opportunities, action planniing, networking, matching and decision making, negottiation, political awareness, coping with uncertainty, devvelopment focus, transfer skills, and self-confidence. ABET ABET accredits degree programs, primarilly in USA. It is an influential source for defining whaat is meant by professional competencies for computing, engineering, and related disciplines. There is a long list of competenciees that must be examined in an ABET accreditation [ABE ET 2010]. More concrete interpretations of these compeetencies can be a process uses obtained from looking at how the accreditation these lists. Competencies in this Research Competencies as described in n this paper should be viewed from both communicativ ve and socio-historical perspectives. The authors’ view iss that learning occurs all the time and everywhere. This has led to the goal of creating learning environments th hat extend as much as possible into students’ everyday lives and to provide a framework for integrating learn ning from the formal curriculum with that arising from other lifewide experiences. CURTIN UNIVE ERSITY Curtin University is Western Austtralia’s largest university with approximately 45,000 studentss, nearly 17,000 of whom are offshore and onshore internation nal students. During a recent curriculum renewal project, Curtin reaffirmed its commitment to th he set of nine graduate attributes listed in Table 1 (Curtin University, U 2011). TABLE 1. A CURTIN’S GRADUATE ATTRIBUTES Graduate Descriptor Attribute 1. Discipline Apply disciplline knowledge, understand its theoretical un nderpinnings, and ways of knowledge thinking; Exteend the boundaries of knowledge th hrough research. 2. Thinking skills Apply logicall and rational processes to analyze the co omponents of an issue; Think creatively to generate g innovative solutions. 3. Information Decide what information i is needed and where it migh ht be found using appropriate skills technologies; Make valid judgments and synthesize infformation from a range of sources. 4. Communication Communicatee in ways appropriate to the skills discipline, audience and purpose. 5. Technology Use appropriaate technologies recognizing their advantag ges and limitations. skills 6. Learning how to Use a range of o learning strategies; Take learn responsibility y for one's own learning and development;; Sustain intellectual curiosity; know how to continue to learn as a graduate. 7. International Think globallly and consider issues from a perspective variety of perspectives; Apply international standards and d practices within a discipline or professionaal area. 8. Cultural Respect indiv vidual human rights; Recognize the importancce of cultural diversity understanding particularly th he perspective of Indigenous Australians; Value V diversity of language. 9. Professional Work indepen ndently and in teams; Demonstrate leadership, professional skills behavior and ethical practices. These attributes have beeen contextualized and embedded in all degree programs att the University, with the intention that every student is ab ble to demonstrate them upon graduation (Oliver et al., 20 009). How the graduate attributes are embedded and asssessed is defined in a 978-1-61284-469-5/11/$26.00 ©2011 IEEE E October 12 - 15, 1 2011, Rapid City, SD 41st ASE EE/IEEE Frontiers in Education Conference S1C-2 Session S1C curriculum map for each degree program. Additionally, an institution-wide electronic portfolio, called the iPortfolio, provides an opportunity for students to demonstrate their attainment (von Konsky et al., 2010). The iPortfolio is on-line space where members of the Curtin community can reflect on their learning and professional development. Based on Curtin’s philosophy of teaching and learning, the iPortfolio can be used to selfassess attainment of the graduate attributes and discipline specific professional competencies. Entries in the iPortfolio can be private, shared only with critical friends, mentors, or learning facilitators for the purpose of personal reflection on learning and professional development, and in some cases in conjunction with designated learning experiences or formal assessment. Alternatively, iPortfolio entries can be shared with prospective employers, professional body representatives, and the broader community at the discretion of the iPortfolio holder. FIGURE 1 THE ABOUT ME PAGE FOR AN EXAMPLE IPORTFOLIO FOR A HYPOTHETICAL ENGINEERING STUDENT. The iPortfolio offers a number of tabs, most of which can contain entries that embed artifacts demonstrating professional competencies, employability skills and graduate attributes, together with critical reflections that place these into context. Available tabs are shown across the top of an example iPortfolio in Figure 1. The Evidence Manager, used to upload, tag, and access evidence is shown on the right hand side of the figure. Critical friends, mentors, and members of the teaching staff invited to offer feedback are shown across the bottom of the figure. The tabs available in the iPortfolio are: About Me – A profile picture, personal details, biographical information, and a statement of goals. The About Me page for a hypothetical student is shown in Figure 1. My Ratings – Self-assessment of graduate attribute attainment, with the aggregated assessment of invited critical friends, mentors, and learning facilitators. My Course – Entries related to formal learning, which can include self-assessment of discipline specific professional competencies. My Journals – Reflective journals, which can include journals dedicated to the development of skills and graduate attributes and professional competencies arising from paid employment, fieldwork, and site visits. My Networks - Reflection arising from participation in professional networks and organizations, plus links between portfolio holders requesting feedback. My Showcases – Finished work that highlights skills and abilities to others, including examples that demonstrate attainment of professional competencies and graduate attributes. Use of video and audio to demonstrate skills and attributes that are difficult to convey in print-based formats are particularly encouraged. The principal tab used to self-assess graduate attribute attainment in the iPortfolio is the My Ratings tab. Additionally, an expandable set of templates is available in other tabs to guide reflection on attainment of discipline specific professional competencies, formal learning experiences, and for assessment associated with specific subjects. The My Ratings tab contains space for a reflection and evidence demonstrating attainment of each of the nine graduate attributes. Additionally, iPortrfolio owners self – assess their level of attainment for each attribute, based on the Dreyfus five-stage model [Dreyfus, 2004; von Konsky et al., 2010]. This ranges between one to five stars. The iPortfolio holder awards one star if they believe that they are like a novice for that attribute. Five stars are awarded if they believe that the evidence and reflection demonstrates them to be an expert. As with all iPortfolio entries, the My Ratings tab is private until the iPortoflio holder takes action to share it with others. For each graduate attributee, those invited to view the My Ratings tab can see the evidence and reflection provided by the iPortfolio holder. However, those invited to view the My Ratings tab do not see the number of stars that the iPortfolio holder has assigned to each attribute. Instead, those invited to view an iPortfolio My Ratings tab make their own assessment of the level of achievement based on the reflection and evidence provided by the iPortfolio holder. The iPortfolio holder can then visualize their self-assessed star rating for each attribute against the average value awarded by their invited referees, provided that three or more individuals have been invited to view the page. An example is shown in Figure 2. An iPortfolio holder can use the automatically generated spider diagram, similar to the one shown in Figure 2, to establish if the evidence and reflections they have provided adequately conveys to others their personal view regarding their competence with each attribute. It may be that the iPortfolio holder is more competent with a given attribute than assessed by the invited referees. In this case the evidence or reflection may be inadequate to demonstrate this. Alternatively, the iPortfolio holder may have an inflated view regarding his or her own competence. This represents an opportunity for further reflection and refinement. 978-1-61284-469-5/11/$26.00 ©2011 IEEE October 12 - 15, 2011, Rapid City, SD 41st ASEE/IEEE Frontiers in Education Conference S1C-3 Session S1C Discipline Knowledge 5 4 3 2 1 0 Professional Skills Intercultural Understandings Thinking Skills Information Skills International Perspectives Comn Skills Lifelong Learning Technology Skills My Rating Avg of those invited to offer feedback FIGURE 2 SELF-ASSESSED AND AVERAGE REFEREE ASSIGNED COMPETENCE LEVELS FOR EACH ATTRIBUTE The iPortfolio My Ratings tab has been used as a formative learning exercise for a course in which students produce a video resume as a summative assessment. The video resume is intended to demonstrate a student’s skills and abilities using a narrative approach. In the narrative, students recount examples that describe how they have demonstrated the graduate attributes in the past. These examples of “a time when stories” are based on evidence and reflections in their My Ratings tab. The video resume is submitted for formal assessment in the subject, and should professionally communicate skills and abilities to prospective employers [von Konsky et al., 2010]. Additionally, it has been suggested that students should consider sharing their iPortfolio My Ratings tab with the referees they nominate when competing for employment. Doing so highlights a candidate’s view regarding their skills and strengths. This has the potential to prepare a referee to be a strong advocate on the student’s behalf when communicating to the student’s prospective employers, and to do so based on the graduate attributes. UPPSALA UNIVERSITY spend half their study time on the course. The course results in a pass or fail grade, and is taught in English. Seven years ago the Uppsala course started to include students from the Department of Computer Science and Software Engineering at Rose-Hulman Institute of Technology in the United States. These students are enrolled in a course titled “Computing in a Global Society.” In this course, students explore the importance and relevance of globalization in relation to software engineering and computer science and. Like the course at Uppsala University, there is a focus on professional competencies, team dynamics and collaboration in a wide context. The IT in Society course is based on the Open-Ended Group Project (OEGP) concept and addresses the development of professional competencies [Daniels, 2011]. The nine Curtin University graduate attributes were used during the fall semester of 2010 to place a student focus on attainment of professional competencies. This included both naming some of these competencies and discussing their meaning. This was done in the form of a written reflection and a follow-up individual meeting. Most students had clear difficulties describing the meaning of the graduate attributes and placing them in a personal context. They also had difficulty rating their own abilities with respect to their level of attribute attainment. This was partly due to being unclear regarding with whom they should compare their ability, e.g. professionals or fellow students. It was our impression that they had no prior experience at “estimating” levels of professional competency. The students had to state, in order, three of the attributes that they wanted to focus on during the course at the end of the initial individual meeting. The professional skills attribute was clearly the most common attribute selected by students. Additionally, thinking skills, information skills, communication skills, and cultural understanding were more commonly selected than other graduate attributes. Surprisingly, the technology skills attribute was not chosen by any of the students as one of their three top choices. This was followed up in the final reflection and an individual meeting. Most students had difficulty showing how they had gone about improving in their chosen competencies. This impression was strengthened during an interview series with the American students three months after the course. The disappointing findings indicate that more effort is needed to reinforce the development of professional competencies as a learning objective of the course. A formal “letter of agreement” will be used in the upcoming instance of the course so that students better appreciate the significance of professional competency assessment as a key course component. We also believe that the students need additional learning experiences and reflection exercises, Uppsala University was founded in 1477 and is one of the oldest universities in northern Europe. Uppsala is a complete university with 45 0000 enrolled students. The IT in Society course has run since 1998 at the department of Information Technology at Uppsala University. During the course, students address a real problem of significant complexity. A learning outcome is to evaluate, criticize and validate solutions to IT-related problems from perspectives including ethics, sustainable development, work environment, economy and usefulness. Writing a report and delivering oral presentations to real clients are parts of the course. The course runs for one semester and the students 978-1-61284-469-5/11/$26.00 ©2011 IEEE October 12 - 15, 2011, Rapid City, SD 41st ASEE/IEEE Frontiers in Education Conference S1C-4 Session S1C with an early focus on setting goals to improve chosen competencies. Moreover, the focus on development of professional competences will be reinforced in future instances of the course through an explicit description of this learning outcome in information provided to students at the beginning of the semester. The revised course goal includes: “Evaluate and analyze one's abilities and competencies regarding working in a multi cultural and distributed project, as well as develop strategies that lead to lifelong learning.” DISCUSSION Our experience has demonstrated that students advanced the development of their chosen graduate attributes by reflecting on them in conjunction with formal assessments in this subject. However, it was also seen that students lacked experience and confidence in doing so. It is likely that this is due to limited significant prior experience with reflection in general, with the significant emphasis of prior learning focusing on the assessment of technical outcomes. This suggests that students are not likely to reflect on the development of professional competencies unless required to do so, and that students are not likely to incorporate reflection as part of their on-going professional practice. This further suggests that students should be introduced to reflecting on the attainment of professional competencies earlier in their degree program, and that the adoption of a lifelong and lifewide electronic portfolio like Curtin’s iPortfolio should be considered. ACKNOWLEDGMENT We would like to thank the following organizational units at Uppsala University for funding this research: Faculty of Natural Sciences and Technology, the Department of Information Technology and the Faculty of Educational Sciences. REFERENCES ABET “Criteria for accrediting applied science programs, ABET, inc. Baltimore, 2010. Dissertations from the Faculty of Science and Technology, ISSN 1651-6214; 808, 2011. Dreyfus, S.E., The five-stage model of adult skill acquisition. Bulletin of Science and Technology 24(3): 2004, pp. 177-181. Dörge, C. “Competencies and skills: Filling old skins with new wine”, KCKS 2010, IFIP AICT 324, 2010, pp. 78-89. Kinsella, M.L.A., Lee, K., Oliver, B., vonKonsky, B.R., and Parsons, R. Electronic portfolio use as an ePortfolio – Key Competencies – Identity, European Institute for E-Learning (EIfEL), 5-7 July 2010, London. Mayer Committee (1992) Putting general education to work: The key competencies report, AEC/MOVEET, Melbourne. OECD (2005) The Definition and Selection of Key Competencies: Executive Summary, accessed 31 May 2011 http://www.pisa.oecd.org/dataoecd/47/61/35070367.pdf Oliver, B. “Teaching Fellowship: Benchmarking Partnerships for Graduate Employability”. Sydney: Australian Learning and Teaching Council. Available at: http://www.altc.edu.au/resource-benchmarkingpartnerships-oliver-curtin-2010. Oliver, B., von Konsky, B.R., Jones, S., Ferns, S., Tucker, B Curtin’s iPortfolio: facilitating student learning within and beyond the formal curriculum, Learning Communities: International Journal of Learning in Social Contexts, ISSN 1329-1440, December e-Portfolio Edition, http://www.cdu.edu.au/centres/spil/publications_ijlsc.html, 2009. Schön, D: (1987): Educating the Reflective Practitioner, Jossey Bass, San Francisco von Konsky, B.R., Oliver, B., Nikoletatos, P, and Wilkinson, H. Showcase U on iTunes U: The iPortfolio enables student self-assessment of key capabilities and the public showcase of achievements, Learning Forum London 2010: ePortfolio – Key Competencies – Identity, European Institute for E--Learning (EIfEL), 5-7 July 2010, London. AUTHOR INFORMATION Åsa Cajander, Assistant Professor, Department Information Technology, asa.cajander@it.uu.se of Mats Daniels, Senior lecturer, Department of Information Technology, mats.daniels@it.uu.se Brian R. von Konsky, Senior Lecturer and Senior Online Education Developer, Curtin University, B.vonKonsky@curtin.edu.au ACER (2002) Graduate skills assessment, Australian Council for Educational Research 03/02. The Association of Graduate Recruiters (1995) Skills for graduates in the 21th century, The Association for Graduate Recruiters, Cambridge. Cajander, Å., Daniels, M., McDermott, R., and von Konsky, B.R., “Assessing professional skills in higher education”, Thirteenth Australasian Computing Education Conference (ACE2011), Perth, Australia, 2011. Curtin University, Curtin Graduate Attributes, http://otl.curtin.edu.au/teaching_learning/attributes.cfm, accessed on 7 April 2011. Daniels, M. Developing and Assessing Professional Competencies: a Pipe Dream?: Experiences from an Open-Ended Group Project Learning Environment, Digital Comprehensive Summaries of Uppsala 978-1-61284-469-5/11/$26.00 ©2011 IEEE October 12 - 15, 2011, Rapid City, SD 41st ASEE/IEEE Frontiers in Education Conference S1C-5