How learning works in design education

How Learning Works in Design Education: Educating for Creative Awareness Through Formative Reflexivity Kathryn Rivard Human-Computer Interaction Institute Carnegie Mellon University 5000 Forbes Avenue Pittsburgh, PA 15213 USA krivard@cs.cmu.edu ABSTRACT This paper reviews and extends educational principles from recent learning sciences literature to address the nuanced needs of creative design education. We have performed a variety of ethnographic and qualitative research activities, including interviews with design students and learning experts, as well as reflecting on our experiences as design educators and practitioners. Our findings identify opportunities in the areas of the classroom environment, learning objectives, formative strategies for student achievement, iterative learning, and suggest the value of an adaptive interface between objectives and learning strategies. We therefore propose a new model of reflexive learning to both improve design education and support creativity and self-leadership in studio design practice. Author Keywords Design education, creativity, transfer, feedback, motivation, practice, knowledge, fluency, integration, formative failure. ACM Classification Keywords K.3.2. Computer and Information Science Education: Curriculum; Literacy; Self-assessment General Terms Design INTRODUCTION Education is challenging, and graduate education in a creative field is even more challenging. In many ways, students are set up to fail by the education they have been subject to all of their lives. The American K-12 school system has chosen to cope with such issues as not enough good teachers and inadequate funding by leaning ever more heavily on quantitative evaluation of students [36]. Laws intended to increase the quality of education such as the No Child Left Behind Act of 2001 rely exclusively on standardized testing to measure learning, despite criticism and resistance from education experts including the International Reading Association (IRA) [15]1, Measures of Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. 1 DIS 2012, June 2012, Newcastle, UK. The IRA is 11-15, against high-stakes assessments of any kind. Copyright 2012 ACM 978-1-4503-1210-3/12/06...$10.00. Haakon Faste Human-Computer Interaction Institute Carnegie Mellon University 5000 Forbes Avenue Pittsburgh, PA 15213 USA hfaste@cs.cmu.edu Effective Teachers [8]2, and the National Education Association (NEA) [39]3. As we might expect, the students this education system finds to be “excellent” are generally particularly good at following instructions, at working within the system and not making waves [43]. A creative field, by contrast, needs people who excel at making waves: at questioning whether instructions are worth following, and at seeing what’s missing, and asking why [16]. These behaviors are often disruptive in a K12 classroom, bog down lecture-style undergraduate coursework, and are antisocial in cultures that value group cohesion over individuality [23, 26, 28]. Nevertheless, research suggests that successful people must be able to integrate the analytical thinking they were taught in school with two other thinking styles: creative thinking, which they may have left behind in childhood, and practical thinking, which will help them operate in an often messy and illogical world [43]. University programs seeking to graduate excellent designers who can be successful in the world are thus especially charged. Their student bodies are already limited to those individuals who would have enough interest in a certification program to apply, and who must then score well enough on standardized measures (and the associated dysfunctional learning that comes with them) to be selected for admission. If these students are to succeed, universities must cultivate an environment that gives students unprecedented access to their creative, wave-making, practical, street-smart, and empathetic selves. Among the numerous issues unique to design education at the university level are the following: (1) Many of the best university faculty are highly creative people, since it takes courage, conviction, and a certain directed whimsy to push the envelope and extend human knowledge. Unfortunately, in 2 This project of the Bill and Melinda Gates Foundation determines that multi-axial assessments are effective and useful in evaluating teachers, but reliance solely on standardized test scores is inappropriate and unnecessary. 3 Senior Policy Analyst Patti Ralabate argues that the requirements for which and how many students should be allowed to take what test is demoralizing for disabled students and doesn’t allow students to accurately represent their abilities. Creative Thinking Analytical Thinking Despite the numerous challenges to creative education, we believe there is a clear and insistent opportunity to examine education in creative fields, and particularly in fields that require the integration of both creative and technical proficiency. Indeed, we hold that the design of interactive systems would be greatly improved by an increased quality and diversity of creative styles in the graduates of programs in design education. In this paper we discuss the findings of qualitative research we have performed to collect primary information about what students and teachers experience now, and leverage existing knowledge about the learning process and goals for thinking skills in order to contextualize these primary findings. We then draw conclusions about what the shape of design education should grow to be for the betterment of interactive system design. Critique Evaluate 3UREOHP,GHQWLÀFDWLRQ 5HÁHFW Strategy Determine What's Important 5HGHÀQHWKH3UREOHP 3LWFK Fearless Idea Generation .QRZOHGJH,QVSLUDWLRQDOYV5HVWULFWLYH 'HI\5HVLVWDQFH Take Risks Tolerate Ambiguity )DLWK 6HOI(IÀFDF\ 4XHVWLRQ$FFHSW$VVXPSWLRQV Build Self-Knowledge 'HOD\*UDWLÀFDWLRQ ,QVWUXFWLRQDO&UHDWLYLW\ TEACHING CREATIVE DESIGN IN THE CLASSROOM Creativity is generally defined any behavior that leads to ideas or problem solutions that are original in a given culture or cultural context [1, 33]. For our purposes we will consider creativity to be a cognitive mechanism performed by an individual learner, as described by Dewey [14], although networks of collaborating individuals are also capable of manifesting creative behavior [31]. The skills that support creativity are broad, deep, and highly interconnected. They vary from purely internal skills that help people think and develop their own ideas, to external skills that may allow ideas to be expressed, information to come in, or both simultaneously. Many authors have emphasized the need for teaching to understand and address the unique creative learning styles of every student [19, 23, 28, 34, 41]. Sternberg, for example, has found through extensive research and live learning studies that success in the classroom, the workplace, and in life require the balanced integration of analytical, creative, and practical thinking, while conventional education focuses almost exclusively on analytical thinking alone. In Teaching for Successful Intelligence [43], he lays out a series of component skills for each thinking style as a guide for instructors, an overview of which is provided in figure 1. The following section builds on Sternberg’s theory through a reinterpretation of some of these skills to better support the needs of creative education as evidenced in recent literature on design. Extending Learning Principles to Design Education In 2010 researchers at Carnegie Mellon University published a book combining current research on education with practical experience and guiding principles. This resource, titled How Learning Works [2], provides a useful survey of contemporary learning sciences research, a clearly defined vocabulary of educational terms, and best practices for creating and teaching university classes grounded in the state of the art. Given the comprehensive nature of this work, we will briefly summarize their framework below. However, How Learning Works was Practical Thinking addition to the selection biases described above, being creative can make it very difficult to teach creativity and remember what it was like to be a beginner, since creative styles are diverse and learning is highly contextual. (2) Creativity is very difficult to evaluate and measure using traditional quantitative means [10]. (3) The creative process is open-ended and resistant to regularly scheduled class time, assignment deadlines, midterms, finals, the whole semester system, etc. [29, 16]. (4) In most university environments, research is emphasized (or at least informally valued) over teaching [23, 18]. (5) Authority struggles can lead instructors to “keep secrets” from students (whether on purpose or not) to maintain their oracle status. (6) Leading any creative process requires teachers to be adaptive and resist prescriptive methods. This tactic requires facility with a vast arsenal of techniques, many of which may appear irrelevant to the instructor’s discipline. (7) Most universities do not provide resources for instructors to develop their teaching skills, and those that do often stretch these resources to maximize their relevance to all disciplines. We have learned in our work that instructors in any field are notorious for selfselecting against advice which has been framed in general terms and not tailored to their discipline; this behavior is twice as detrimental for instructors in the creative fields, where counterintuitive techniques can be especially valuable. Control Impulses %DODQFH6NLOOV Build Motivation Set Priorities and Keep Them in View Commit to Starting Something / Commit by Starting Something Conquer Fear of Failure 2YHUFRPH3URFUDVWLQDWLRQ $FFHSW$SSURSULDWH5HVSRQVLELOLW\ Manage Self-Pity Complete Tasks 6SUHDG<RXUVHOIDWWKH5LJKW7KLFNQHVV %H3URGXFW2ULHQWHG :RUN6R<RX&DQ&RQFHQWUDWH 7UDQVODWH7KRXJKW,QWR$FWLRQ 'HYHORS6HOI&RQÀGHQFH Play to Your Strengths / Person-Environment Fit +DQGOH3HUVRQDO'LIÀFXOWLHV Persevere, Don't Perseverate Be Independent Figure 1. Thinking styles and component skills (from Sternberg [43]). written for a broad educational audience, and we feel that many of the assumptions that hold true for traditional calland-response or memorization-based methods are irrelevant or even harmful in a design education context. For example, the book places a great deal of emphasis on instructors taking the time to dig into expert blind spots and uncover hidden structures of knowledge organization, with the assumption that the most valuable contribution of the instructor is to reveal and convey these structures to students. One of the recurring themes from our research and recent design literature, however, is the resistance of design to conform to such absolute organizational structures. If structures are to be employed, they are most useful on much smaller scales: each designer might find a unique structure that works for them, drawing on their own pattern of background experiences; it’s even likely that each project will respond best to a customized hierarchy drawing on the available information, user needs, and the particular personality of the design team. The instructor cannot in good conscience dictate to students which organizational structure is best, and must instead lead and inspire students to find these structures for themselves. We find a similar pattern in many of the principles covered in the book: since design by nature must build on convention, it is not only the instructor’s job to identify and communicate conventions to students, but also to establish the convention of breaking with convention when appropriate. With this somewhat more abstract outlook in mind, we can proceed to review the structures put forth by How Learning Works. Figure 2 serves as a brief guide to the most important concepts and how they relate to the goal of student learning. These concepts are: (1) transfer; (2) knowledge organization; (3) motivation and value; (4) feedback; (5) students’ prior knowledge; and (6) practice, fluency, and integration. Our analysis provides a brief description of each concept as presented by the book, and discusses how the concept engages with creativity and design education specifically. 1. Transfer Transfer is what allows a student to take an isolated fact and apply it to a new context, internalizing the fact as a general principle. It is rarely obvious to the beginner whether a skill learned in one context can be applied to another, and it is the role of the instructor to help students make these connections appropriately and avoid misguided connections. For example, the principle of the addition of integers applies to the addition of fractions, but only to the numerator, and only when the denominators are equal. A lack of transfer would leave a student at a loss at how to approach the addition of fractions, and inappropriate transfer might lead a student to add the denominators as well as the numerators. When we speak of creativity, transfer takes on a whole new meaning, and inappropriate transfer nearly ceases to exist. The application of concepts in contexts where they don’t belong is one of the most powerful tools in the design toolbox, because it allows us to ask the question, “What if...?” The use of analogy, ambiguity, conflict, and paradox have been wellstudied in creative fields (e.g. [20, 24, 25, 33]), and would seem to contradict the book’s recommendations of how transfer should be guided by instructors. We could consider a decision table of the choice to apply a concept or not in appropriate or inappropriate contexts, as in figure 3. A firstlevel understanding of transfer in conventional environments values the application of concepts in appropriate contexts and not in inappropriate ones. We have already seen how the “What if...?” application of a concept in an inappropriate context results in creativity. Contrariwise, the ability to make observations with a fresh mind—to consciously omit concepts in their appropriate contexts—is extremely valuable in a creative environment, while in conventional environments this behavior is considered to indicate incomplete understanding. In this way, creativity and design education is positively focused on the blocks in the decision table which conventional education eschews. 2. Knowledge Organization 3 Motivation and Value fuels 4 Feedback informs Progress 6SHFLÀF)DFWV increases 5 Prior Knowledge initializes Student’s Understanding represents Practice develops Fluency Integration broadens Transfer illuminates, leverages General Principle 1 Beginners form sparse relationships between facts, often because they are not yet familiar enough with the material to do more than rote memorization. Experts form dense structures of their knowledge that allow them to navigate complex relationships with ease. Experts might use hierarchical structures or webs, depending on the situation, and this decision is rarely conscious—most experts use multiple structures at will. Context Environment makes explicit Knowledge Organization 6 Figure 2. A structural map of the learning framework proposed by How Learning Works [2]. Apply Appropriate Not Appropriate Conventional Creative Conventional Creative Correct Boring Incorrect Creativity Incomplete Investigative Correct N/A 2 Don!t Apply Figure 3. Decision table for the application of transfer in conventional and creative environments. In design, knowledge organization is most relevant to building an awareness of design processes [1, 30]. How Learning Works proposes several activities for instructors to use to expose students’ knowledge organizational structures, highlight “correct” relationships and hierarchies, and encourage students to use multiple structures to organize what they know, but we feel that the creative classroom would best employ such techniques on the fly as real-time responses to student needs. Cross has observed that often “designers deviate from a structured plan or methodical process into the ‘opportunistic’ pursuit of issues or partial solutions that catch the designer’s attention.” [11] Such deviations from top-down approaches have been shown to be cognitively advantageous [27], especially in the context of real-world design problems such as creative education [4, 17]. 3. Motivation and Value To hold students’ attention, instructors must be aware of what motivates them. How Learning Works presents motivation as a combination of two key elements, value and expectancy. A task could hold intrinsic value (pure enjoyment), instrumental value (a means to an end) or attainment value (pride of completion) [3]. By modifying the presentation of material, an instructor can highlight its value to students to help motivate them. If the value of a task is not evident, students may not see any reason to do it. Expectancy is about whether or not a student expects to succeed at the task [46, 47] Objective expectancy is whether the student understands that by fulfilling some set of requirements, success is physically possible. If it’s clearly not possible for anyone to succeed, a student is likely to not bother. Efficacy expectancy is more personal, and is whether the student believes that he or she specifically has the skills and resources necessary to fulfill those requirements [6]. If a student doubts his or her own abilities, they may assume failure is unavoidable and not bother. A common tenet in design is “form over content,” and the design classroom tends to operate on the same principle. The trouble with objective expectancy is that a design challenge need not be physically possible to be a useful exercise. A recent final project of an innovative architecture class at the University of Innsbruck, for example, was to plan the robbery of any nearby bank [40]. While execution of the plan was not required, students did need to present something plausible, motivating their engagement with research, ideation, analysis, documentation, and developing time and cost plans. Students were encouraged to examine and exploit the weak points of their chosen bank, with the objective of “stealing” assets like time, space, image, future clients, electric power, etc. One student planned the theft of a ballpoint pen chained to the counter in his target bank; another designed a mechanism to waste the bank’s time, effectively thieving significant sums of money. As this example demonstrates, the goal of the creativity educator is not to show students that tasks are possible, but to frame assignments so that whether the tasks are possible no longer matters. The remaining topics in motivation and value relate to flow [12], personal confidence, and pride of quality, which are all directly relevant to the practice of design. 4. Feedback Feedback gives students an opportunity to verify their knowledge externally. Formative feedback helps students in the present, and evaluates what a student is doing now or plans to do next. Summative feedback helps students retrospectively, and can only be applied by the student if they encounter another situation they can identify as similar. Distinguishing formative from summative feedback experiences highlights one of the essential challenges of creativity education. To truly encourage the necessary design culture of forced fast failure, iteration, and broad endeavors, instructors must be involved with students’ progress at very early stages. The epitome of summative feedback, the final grade, does nothing to help the student direct his or her own learning. Only formative experiences can lead students to lead themselves. Because the intensity of involvement required of the instructor in formative feedback is so high, there is an opportunity to distribute some of the load to the community. A classroom of students can generate far more feedback than a single instructor, and with appropriate guidance or moderation the instructor can leverage this work to further both the learning of the student being critiqued and the confidence and independence of the students offering feedback. 5. Students’ Prior Knowledge Prior knowledge can be helpful or harmful. How Learning Works highlights that no course and no assignment is taught in a vacuum: it is essential for educators to be aware of what relevant knowledge students come in with, and make use of it if possible to create better, denser knowledge structures, facilitate student motivation by lending value to the material, and identify and repair instances of inappropriate, insufficient, or inaccurate prior knowledge. The book describes several methods for revealing the prior knowledge that students come in with, from talking to instructors of prerequisite courses to having students brainstorm in class before instruction is provided. It is also critical to separate declarative knowledge from procedural knowledge: frequently, students will be familiar with terminology but not with the processes those terms represent or when they are relevant [7, 44]. Finally, even if students are known to have prior knowledge, they may not utilize their existing knowledge in new contexts. To create transfer in this area, instructors must activate prior knowledge by explicitly drawing connections and reminding students of the relevance of what they already know. In design, prior knowledge and knowledge organization are closely related [1]. Where knowledge organization is about design process, prior knowledge is about building an awareness of design content. Engaging students in an information-purging activity early in the course can be an excellent way to both identify students’ prior knowledge of the topic and build students’ familiarity with exhaustive brainstorming activities. Students are often unaware that the experiences of their entire lives are relevant to the choices they make as designers, and may need repeated prompting to use this knowledge with confidence. Getting students in the habit of being aware of what they know is also an important component of the self-awareness necessary for self-leadership. 6. Practice, Fluency, and Integration Much of the proficiency we hope to give students depends on their ability to do more than one thing at the same time. Multitasking and integration require a certain degree of fluency in one or more skills, that is, to be able to do them effectively without having to concentrate. Fluency is developed using targeted practice. Good qualities of practice opportunities include having a specific goal, limiting the scope of the task appropriately to allow focus on the goal, and having sufficient time and repetitions to build familiarity, routine, and reflex. Each of these factors should be sensitive to the ability level of the student, with the objective of targeting the student’s flow zone. Many of the activities associated with creativity assume a high degree of fluency, particularly in self-awareness and communication. Speed practice in drawing, brainstorming, and concept generation can help get students to a place where greater abstractions and connections can be manipulated. However, instructors should be careful to target practice in the areas students will need for their assignments: developing students’ fluency in skills they don’t use, while leaving them adrift in skills they desperately need, will surely devastate whatever trust has been established in the instructor. METHODOLOGY A variety of quantitative and ethnographic techniques were used to gather and synthesize data that would allow us to define and contextualize the space of design education, and identify recommendations for its implementation in the classroom. These included structured brainstorms, traditional interviews, card-sorting activities, and public brainstorms, as described in the following sections. Curriculum Review and Synthesis of Design Skills Our first research activity involved a review of the design curriculum in our university’s Human-Computer Interaction department. Two core classes comprise this program, which is required for all HCI Masters students and the majority of PhD candidates. The two classes are Communication Design Fundamentals (CDF), which covers the essentials of visual design for both print and the screen (including typography, color, information hierarchy, composition, etc.), and Basic Interaction Design (BID), a studio-based and collaborative project-oriented course in which student teams design mobile and UbiComp systems addressing real-world needs. One of this paper’s authors is a graduate student in this program who had recently taken both classes, and the other is a professor who has taught them both. We analyzed the assignments and activities from BID and CDF, labeling each with the set of “core skills” it was intended to help students build. This analysis resulted in a deck of 44 skill-related verbs, which were then grouped based on affinity (Figure 4). To validate these groupings, we worked with two additional students who had recently completed both courses and allowed them to come up with alternate possible organizations of the skill deck. critique question divide destroy sketch create prototype visualize extract abstract synthesize connect integrate reflect needfind triage delegate scope frame constrain lead delineate initiate look explore follow immerse engage read observe empathize ideate brainstorm combine collaborate permit expand articulate justify pitch compel realize communicate describe Figure 4. Pool of skill words from the BID and CDF courses, used in skillsort activities, in their initial affinity grouping. While both students came from design backgrounds, their arrangement of the skill cards varied dramatically. One student focused on the differences between skills of craftsmanship and skills of thinking or feeling, and the differences between individual work versus collaboration. The other student focused on the differences between atomic skills (e.g. rendering, synthesis, analysis) and integrated skills (e.g. leadership, advocacy, subversive design that result from deep experience with combinations of lower-level skills. We used this data to inform our understanding of knowledge organization in the design domain, suggesting that designers need not subscribe to a single or fixed set of knowledge hierarchies, and likely benefit from developing their own knowledge structures. Interviews and Discussions with Education Experts In-context interviews were performed with two individuals—a former art student at our university who had struggled in her undergraduate program, and a learning sciences researcher and expert on teaching and classroom education—with the aim of synthesizing areas of design opportunity in creative education. The intent of these interviews was not to perform a rigorous cultural study of design education, but rather to inspire our design research process, ground some of our hunches empirically, and explore some of the topics in the literature we had surveyed by examining their effect in a “real-world” qualitative context in-depth. An open-ended interview protocol was used, and our sessions were recorded using digital audio. We focused our inquiry on understanding the motivations and concerns involved with design education. Interviews were then transcribed and then coded using an iterative, grounded theory approach [22], and synthesized, along with the findings of our other activities, to determine areas of potential design opportunity. Interview 1: Disillusioned with Art School “Going to school for art was one of my worst ideas ever.” The first participant we spoke to is a video artist now struggling to feel like she has marketable skills. She described always having had diverse interests—in addition to art, she is passionate about math, biology, and computing—but felt like she didn’t have much chance to explore that diversity in the college art curriculum from which she recently graduated. Without a network of prerequisites to draw upon, she found herself stuck in introductory-level non-major classes where she felt that questions and discussion were culturally stigmatized and “implied you were stupid.” She found her art major community, while more empowering in its perception that such participation indicated passion in a topic, as nevertheless somehow more insular and elitist. She’d hoped to go to art school to be able to explore diverse possibilities, but most of the ideas she saw from students and faculty were art about art: far more “meta” than she had hoped. She was frustrated by the amount of time spent on required courses that she didn’t feel were doing her much good, and struggled with the learning environment. To see if we could better articulate the conflict she felt about her coursework, we made an affinity diagram together during the interview that contained the elements of her curricular experiences and life as an artist that she felt were important, both positive and negative. After several iterations together a model was developed in which art and art education has three main components: Skill, including practical experience with tools, techniques, and software; Voice, including activities that extend a student’s artistic horizons and develop intent; and Context, which makes a student aware of the connections between her work and larger communities of practice, including history, grantwriting, reputation, and collaboration. In our participant’s view, these components all feed into one another in different combinations to help inform, explain, and justify successful projects. After developing this model, we also had our participant perform a card generation and sorting activity to catalogue her “likes” and “wishes” about her university experience. This “I like, I wish” exercise is a way of highlighting both positive and negative aspects of user experience to foster further effort and creativity, and was inspired by George Prince’s creativity research in the 1970s [38]. An affinity sort of the cards identified the following opportunities, which she felt would have greatly enhanced her undergraduate education: • Improve and focus on worldly participation and interpersonal interactions. • Diversify feedback and evaluation. • Help resolve or address conflicting needs hierarchies. • Redirect negative energy into positive pursuits. • Highlight the joy of learning and deep thinking. These findings helped us identify how students could be invited to cooperate in motivating their own learning to create a positive and productive educational experience. This information was incorporated into our evolving model of design education. Interview 2: Eberly Center for Teaching Excellence One of our main resources in pursuing this work was the Eberly Center for Teaching Excellence at Carnegie Mellon, which aims to “distill the research on learning for faculty and graduate students and collaborate with them to design and implement meaningful educational experiences.” [45] The center offers workshops, private consultations, and resources to instructors to help improve teaching at the University. We spoke with Dr. Marie Norman, Associate Director. She frequently offers consultations to instructors looking to improve their teaching skills, and as an educator of educators we were particularly interested in speaking with her as a primary source. In many ways we view instructors as the primary “user group”/audience for this project, since we will depend on them to implement our findings in practice. Dr. Norman indicated that the workshops offered by the Center are attended by three types of people: brand-new teachers, teachers who are frustrated, and teachers who are already passionate about their teaching and seek enrichment rather than assistance. The primary challenge with workshops is with those frustrated teachers looking for a “quick fix” to solve a classroom problem they might wish to see as simple. Dr. Norman stressed that poorly understood quick fixes often only make classroom problems worse. Dr. Norman described her Eberly Center teaching work as much harder than her work teaching undergraduate and graduate students. She said that most teachers are listening only for content that seems to pertain most closely to their problems, to the point that anything not specifically tailored to their department or topic of instruction is largely ignored. Teachers presented with new techniques often won’t generate or work their own examples to try them out, and well-executed examples must be provided if teachers are to see their value. She said there were two ways to best reach teachers: motivate the value of the material by linking to teachers’ own classroom frustrations, and recognize that many excellent researchers expect to be equally good teachers of their topic, and are surprised and dismayed to find that their teaching is failing. It is convenient to highlight here the apparently pervasive phenomenon of an instructor being a strong researcher but a poor teacher. It is a common perception to assume that experts can teach without additional training. Without such training, however, we believe that universities end up with many poor teachers, teaching standards drop, and there is little perceived institutional need for adequate training, especially given the research-driven financial structures of many modern universities. Research is more lucrative to a university than teaching, to be sure, but with better teachers they would certainly educate better researchers with deeper and more nuanced understandings of their fields. Dr. Norman had many recommendations relevant for our research, but the ones that seemed most compelling were those that had helped her personally when teaching her classes. Like many instructors, she recognizes the value of group work but struggles with how to evaluate students individually. The state-of-the-art in process evaluations for group work is poor. Peer evaluations are commonly used, but are risky given unknown social factors such as jealousy, etc. More interesting, Dr. Norman felt, was the prospect of having students do regular reflections as a sort of selfassessment. Having students list explicitly what they learned and what they would do differently can help an instructor place them in the best situation for personalized learning within the class context. Leading prompts can be used to subvert some of the personality types that are more difficult to assess, such as the excessively shy and the excessively bossy. 10-Minute Reflections The reflection-based evaluations suggested by Dr. Norman inspired us to use personal reflections as a general mechanism for self-awareness and learning. We quickly prototyped this idea by having two graduate students enrolled in our HCI Masters program write, draw, or build such “reflections” for 10 minutes a day, for one week, based on prompts randomly drawn from a deck of 30 we constructed from several sources of introspective activities [13, 32, 42, 43]. Reflection prompts varied from concrete questions like, “What kind of person is a good leader? Describe the qualities he or she should have,” to brainstorming warm-ups like, “Write about all the uses for the safety pin that you can think of,” to the more whimsical, “Imagine that you and your classmates are rabbits. Talk about a rabbit’s typical day.” Our testers were self-selected volunteers from our design research laboratory, and enjoyed the activity, but wished the prompts could have been targeted to develop a particular skill or awareness area as a sort of preparation for the rest of their work that day. This feedback suggests an opportunity to study the use of short reflection activities more closely to determine their effectiveness as a more focused learning tool. an attempt to align well-understood models of general classroom education with the targeted case of creativity education in design. We learned a great deal about what guidelines transfer directly between these domains, what guidelines transfer with some modification, and what guidelines must be completely rewritten. Our final synthesis, summarizing all of the data described in the studies above, resulted in the extraction of seven key opportunities for creativity educators. They are: • Structure core content in clear and easily digestible ways. • Challenge students to wrestle deeply with problems of appropriate levels of difficulty • Tune instructional activities in response to transparent learning objectives. • Help students commit to leading themselves efficiently in a scaffolded environment. • Structure activities that force critical reflection. • Cultivate a passion for rigorous design excellence. • Build a respectful and supportive (trusting) community by celebrating differences in a reduced risk environment. These opportunities span the entire classroom dynamic, from curriculum development and assignment selection to the structure of evaluations to the interactions of the people in the classroom, students and instructors alike. Each class of students is different, and each student in that class has varying prior knowledge and awareness of that knowledge. A class full of students who all have extensive backgrounds in the visual arts may not appreciate a detailed tutorial on basic perspective drawing. A class full of students who have been computer-bound for years, on the other hand, may be desperate for someone to show them how to draw a straight line with a pen. The instructor has the advantage of experience, and can and should leverage that experience to look at the students and determine what is needed. As a culminating activity we created a large deck of personal “likes” and “wishes” about the creativity coursework environment at our university, including our perspectives on the CDF and BID courses described above. An initial deck of 110 likes and wishes was generated from our own reflections, and an additional 84 were provided by a pool of graduate students interested in the design research activities of our laboratory (drawn from a more comprehensive set of data described in [18]). We iteratively synthesized the deck using affinity diagrams, and leveraged our findings from the literature and ethnographic studies described above to inform the resulting groupings. Identified opportunities and their implications for creativity education are discussed below. In all educational environments, the best instructors are able to respond to their students and tune each classroom experience to the needs of the particular group. In creativity education this trait is essential. Where many disciplines can be considered to be about correct answers, creativity is about process and confidence exploring alternatives. Where other disciplines can help students develop skills by having them follow instructions, creativity instruction must find a way to awaken students’ self-leadership and conviction early on. By way of conclusion, we will discuss these and other learning objectives that are particularly important for creative fields. We will also identify how they shape the types of course activities that are most appropriate, and the types of feedback and attention instructors must be prepared to give. DISCUSSION Reflexivity in Teaching and Learning I Like, I Wish, Brainstorming, and Community Synthesis In this project, starting from a small body of published knowledge and guidelines for successful education, we synthesized our own experiences in creativity and design education, as well as those of students in our discipline, in Achieving a responsive teaching environment for creative education requires the combination of two central factors: (1) instructional reflexivity, that is, the ability of the teacher to listen and react to the needs of the individual aspects of learning. Students will need to learn to prepare for critiques, forcing them to format their ideas for outside input and encouraging deeper thinking about how concepts and presented artifacts fit together. This requires drawing on basic communication skills, as well as the deeper thinking required to consider how their work fits together as part of an integrative whole. In this regard, the development of a compelling story can lead students toward passion for design excellence, and requiring a variety of oral/visual presentation formats can drive the development of basic speaking and visual expression skills. Receiving criticism on their designs will force students to remove ego from their ideas, and identify holes in their process or presentation method. Instructors may guard against defensiveness by using an open didactic strategy of sharing ideas among an entire class [17]. Generating criticism in real-time requires students to rapidly identify how the specific events or facts in the project line up with their existing understanding of broad principles and learning objectives. This facilitates transfer [2], helps cross-link concepts, and increases fluency and flexibility with the presented material. Finally, studentstudent feedback interactions can also be used to draw on a supportive community, and if done with the right sort of sensitivity, can help create it. Because critiques involve risktaking by both authors and audience, building a supportive environment is essential for creative learning. This is especially true since risk-taking behavior is unconventional in most “serious” educational environments, and most university students—and particularly those in the most students and class as a whole, and (2) strategies for “forcing” critical reflection on the part of the students, or what we refer to as learning reflexivity. Instructional reflexivity, the first goal, requires awareness of students’ skills and mindsets, tapping into what we already know about diagnosing prior knowledge, formative and summative assessments both formal and informal, and the notion of students’ values and motivations [2]. It also requires achieving a deep awareness of learning objectives at several levels of detail, where “depth” of knowledge is generally synonymous with how differing facets of knowledge interconnect (i.e., knowledge organization) and why they’re important. With these two fundamentals in place, assignments and activities can be selected to support learning objectives and function in tandem with students’ values and motivations. Expressing and clarifying assignment requirements productively is also essential, in order to direct student focus to learning objectives which may be significantly more abstract than completing a checklist. Finally, teachers must be willing and able to take risks, i.e., slash the syllabus if it isn’t working, and drop planned didactic activities in favor of spontaneous inspirational ones as they strike. This in turn requires an informal awareness of students’ mindsets, which may need to be assessed distinctly from other aspects of student performance. Learning reflexivity, the second goal, is not entirely in the hands of the educator alone. Critiques are an excellent example of how learning reflexivity can support multiple 1. LEARNING ENVIRONMENT ctional Reflexivi 4. Instru ty Structure core content in clear and easily digestible ways. 3. FORMATIVE STRATEGY Challenge students to wrestle deeply with appropriate challenges. Tune instructional activities in response to transparent learning objectives. Help students commit to leading themselves efficiently in a scaffolded environment. Cultivate a passion for rigorous design excellence. Structure activities that force critical reflection. 5. Learning Reflexivity Build a respectful and supportive (trusting) community by celebrating differences in a reduced risk environment. 2. LEARNING OBJECTIVES 6. ITERATIVE LEARNING THROUGH FORMATIVE FAILURE Figure 5. The Reflexive Learning model for design education: (1) establish a community of teachers and learners who support, respect, and trust one another, and who celebrate their differences; (2) define learning objectives; (3) develop formative strategies that will provide foundational skills and engage the community to their end; (4) employ instructional reflexivity to examine learning objectives and, in the context of the community and its progress and nature, tune formative strategies transparently in real time; (5) provide opportunities for students to experience learning reflexivity and determine whether their performance is accomplishing learning activities; (6) iterate to educate through formative failure. . competitive schools—are likely to have been subjected to large doses of conventional education in the past. Proposed Model for Design Education Combined with the opportunity findings from our research identified above, the notion of reflexivity in teaching and learning forms a framework for supporting creativity and design education that models the classroom and the people and behaviors it contains. Figure 5 contains a visualization of this model, which we refer to as the Reflexive Learning model, described in greater detail below. We first establish the learning environment (1) as a community of teachers and learners who support, respect, and trust one another; who celebrate their differences. Together the community fosters a space where the perception of risk is reduced, and all members can extend their limits with the goal of developing self-leadership. Given this learning environment, we place two spheres within it: learning objectives (2), and the formative strategies (3) that will provide foundational skills and engage the community in the pursuit of their objectives. A more detailed study of formative strategies that are both suitable for students and accessible to instructors is a topic of interest for future work. Two different reflexive processes connect the two spheres, modulating the diffusion of information from one to the other. Instructional Reflexivity (4) is employed by instructors to examine learning objectives and, in the context of the community and its progress and nature, tune formative strategies in real time, maintaining sufficient transparency to invite the participation of the community at large. Learning Reflexivity (5) is employed by the student to determine whether his or her performance on learning activities is still pointing them in the direction of their learning objectives. Learning reflexivity is guided and scaffolded by the instructor at first, but it is understood that students will acquire self-monitoring habits that allow them to increasingly do this for themselves over the course of each semester. Together these reflexive activities by the instructor and student represent a unifying concept we call Reflexive Learning, giving a name to the barrier between learning objectives and formative strategies. In traditional courses, various grade-focused assessments stand in for reflexive learning, but are not conducive to building self-learners because such grade-focused systems nearly always depend on an oracle. In the presence of an oracle, students become motivated by following instructions or otherwise guessing what the oracle wants, rather than by reflecting on their own learning. It is ludicrous to expect students to develop independence, personal conviction, and ethics in an environment where evaluation is so closely tied to criteria that are not of the student’s making. It is for this reason that when we speak of assessment, we premeditate a heavy student role. The ability to perform assessment, evaluation, and critique are all essential skills for each student to develop both individually and as a member of a group. In conclusion, reflexive activities are the engine of design learning. Building personal habits of reflection as a sustained activity combining doing and thinking should be the ultimate goal of educators in design [21]. These skills require “right mode” synthetic and “left mode” analytical deftness in equal measure: to first generate concepts that are broad enough to encompass the boundaries of the solution space, and then select the optimal strategy for addressing the task and the context at hand [17]. A side effect of this “generate-then-prune” procedure is the necessity of producing ill-fitting ideas—failed ideas—in order to identify the excellent ones. If students are to learn to fully explore solution spaces, they must become comfortable with failure as a necessary and desirable part of design process [37]. Facilitating this comfort means placing students in an environment that actively supports failed ideas rather than punishing them. In this way reflexive learning as the core of our model shapes the learning environment in turn: the classroom must be a place in which it is safe to be risky and risky to be safe. We find these formative failures to be so important to design education that we call them out specifically as a product of our model; indeed, they are the only things that make successful design education possible. REFERENCES 1. Amabile, Creativity in Context, Westview Press, 1996 2. Ambrose, S. A., Bridges, M. W., DiPietro, M., Lovett, M. C. and Norman, M. K. How Learning Works, JosseyBass, San Francisco, CA, 2010 3. Atkinson, J. An Introduction to Motivation, Van Nostrand, Princeton N.J., 1964 4. Ball, L. and Ormerod, T. Structured and Opportunistic Processing in Design: a critical discussion, Journal of Human-Computer Studies, 43, 1995, pp. 131-151 5. Balzer, W. K., Doherty, M. E., and O’Conner, R. Effects of Cognitive Feedback on Performance, Psychological Bulletin, vol. 106, 1989, pp. 410-433 6. Bandura, A. Self-efficacy: The exercise of control, Freeman, New York, 1997 7. Berry, D. C. and Broadbent, D. E. Interactive Tasks and the Implicit-Explicit Distinction, British Journal of Psychology, vol. 79, 1988, pp. 251-272 8. Bill & Melinda Gates Foundation, Learning About Teaching: Initial Findings from the Measures of Effective Teaching Project, 2010, Available at http://s3. documentcloud.org/documents/18327/met-researchpaper.pdf. 9. Black, P. and William, D. Assessment and Classroom Learning, Assessment in Education, vol. 5, 1998, pp. 7-74. 10. Carroll, E. A., Latulipe, C., Fung, R. and Terry, M. Creativity factor evaluation: Towards a standardized survey metric for creativity support. In Proceedings of ACM Creativity & Cognition, 2009 11. Cross, N. Expertise in Design: An Overview, Design Studies, 25(5), September 2004 12. Csíkszentmihályi, Mihály, Flow: The Psychology of Optimal Experience, HarperCollins, 2001 13. Dahlstrom, L. M. Doing the Days: A Year’s Worth of Creative Journaling, Drawing, Listening, Reading, Thinking, Arts & Crafts Activities for Children Ages 812, Free Spirit Publishing, Minneapolis, MN, 1994 14. Dewey, J. Democracy and Education, Free Press, 1966 15. International Reading Association, High-Stakes Assessments in Reading: A Position Statement of the International Reading Association, 1999. Available at http://www.reading.org/Libraries/Position_Statements_a nd_Resolutions/ps1035_high_stakes.sflb.ashx 16. Faste, R. A. Unteaching Uncreativity, American Society for Engineering Education, online at http://www.faste foundation.org/publications/unteaching_uncreativity.pdf 17. Faste, R. A. Ambidextrous Thinking, Innovations in Mechanical Engineering Curricula for the 1990s, American Society of Mechanical Engineers, 1994 18. Faste, H. Designing an Improved HCI Laboratory: A Massive Synthesis of Likes & Wishes, CHI 2012 Extended Abstracts, 2012 19. Gardner, H. Frames of Mind: The Theory of Multiple Intelligences, Basic Books, 1985 20. Gaver, W.W., Beaver, J. and Benford, S. Ambiguity as a Resource for Design,” Proc. CHI, 2003 21. Gibbs, G. Learning by doing: a guide to teaching and learning methods. London: Further Education Unit, 1988 22. Glaser, B.G., & Strauss, A. Discovery of grounded theory: Strategies for qualitative research. Sociology Press, 1967 23. Goodman, P. Compulsory Mis-education and the Community of Scholars, Vintage Books, NY, 1964 24. Gordon, W. J. J., Synectics, Collier-MacMillan, 1961 25. Gordon, W. J. J. and Poze, T. Conscious/Subconscious Interaction in a Creative Act, The Journal of Creative Behavior, vol. 15, no. 1, first quarter, 1981 26. Greenhalgh, P. Emotional Growth and Learning, Routledge, New York, NY, 1994 27. Guindon, R. Designing the Design Process: exploiting opportunistic thoughts, Human-Computer Interaction, Vol. 5, 1990, pp. 305–344 28. Holt, J. How Children Learn, Da Capo Press, Revised edition, 1995 29. Holt, J. What Do I Do Monday?, Boynton/Cook, 1995 30. Hubka, V. Design Science: Introduction to needs, scope and organization of engineering design knowledge, Springer, 1996 31. Hutchins, E. Cognition in the Wild, MIT Press, 1996 32. Koberg, D. and Bagnall, J. Values Tech: Leading You to a Higher Degree of Self Awareness, William Kaufmann, Inc., Los Altos, CA, 1976 33. Koestler, Arthur, The Act of Creation, Dell Publishing, New York, 1964 34. Kolb, D.A. Experiential learning: experience as the source of learning and development, Englewood Cliffs, New Jersey: Prentice-Hall, 1984 35. McKim, R., Experiences in Visual Thinking, Brooks/Cole, 1972 36. Pellegrino, J. Knowing What Students Know, Issues in Science and Technology, XIX(2), 48-52. Winter 2002-03 37. Petroski, H. To Engineer Is Human: The Role of Failure in Successful Design, Vintage, 1992 38. Prince, G. M. The Practice of Creativity, Collier, 1970 39. Ralabate, P. Statement Submitted by NEA to the Aspen Institute’s Commission on No Child Left Behind, 2006, Available http: //www.nea.org/home/Statement-of-PattiRalabate.html 40. Regine. 21 bank branches vs. 21 architecture students, We make money not art. 13 June 2011. Retrieved August 2011 from http://we-make-money-notart.com/archives/2011/06/blasbichlers-twentyone.php 41. Robinson, K. Out of Our Minds: Learning to be creative, Capstone, 2001 42. Ryan Madson, P. Improv Wisdom: Don’t Prepare, Just Show Up 2005 43. Sternberg, R. J. and Grigorenko, E. L. Teaching for Successful Intelligence: To Increase Student Learning and Achievement, 2nd ed., Corwin Press, 2007 44. Sun, R., Merrill, E. and Peterson, T. From Implicit Skills to Explicit Knowledge: A bottom-up model of skill learning, Cognitive Science, 25, 2001, pp. 203-244 45. The Eberly Center for Teaching Excellence – Enhancing Education – Carnegie Mellon University. Accessed July 2011. Available at http://www.cmu.edu/teaching/ eberly/index.html 46. Wigfield, A. and Eccles, J. The Development of Achievement Task Values: A theoretical analysis, Developmental Review, vol. 12, 1992, pp. 265-310 47. Wigfield, A. and Eccles, J. Expectancy-value theory of achievement motivation, Contemporary Educational Psychology, vol. 25, 2000, pp. 68-81