Software for Managing Complex Learning: Examples from an Educational Psychology Course

Author(s): Daniel L. Schwartz, Sean Brophy, Xiaodong Lin, John D. Bransford

Source: Educational Technology Research and Development, Vol. 47, No. 2 (1999), pp. 39-59
Published by: Springer
Stable URL: http://www.jstor.org/stable/30221079
Accessed: 12/11/2010 11:22

Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at
http://www.jstor.org/page/info/about/policies/terms.jsp. JSTOR's Terms and Conditions of Use provides, in part, that unless
you have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and you
may use content in the JSTOR archive only for your personal, non-commercial use.

Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained at
http://www.jstor.org/action/showPublisher?publisherCode=springer.

Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printed
page of such transmission.

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of
content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms
of scholarship. For more information about JSTOR, please contact support@jstor.org.

Springer is collaborating with JSTOR to digitize, preserve and extend access to Educational Technology
Research and Development.

http://www.jstor.org
Software for Managing
Complex Learning: Examples from an
Educational Psychology Course

DaonelL Schwartz
Sear Bropny
Xicodong Lin
John D Bransford

Inquiry-based instructionincludingproblem-, Z New developments in learning theory sug-

and
project-, case-based methodsoftenincorpo- gest that many teachers-the present authors
ratecomplexsetsof learningactivities.The included--can improve student learning by
numerousactivititesruntheriskof becoming changing their teaching practices (e.g., Cogni-
disconnected in themindsof learnersand tion and Technology Group at Vanderbilt
teachers.STAR.Legacy is a softwareshellthat [CTGV],1996).As college teachers,we often find
canhelpdesignersorganizelearningactivities that our predominantmethod of teaching is to
intoan inquirycyclethatis easyto under- assign chapterreadingsand then to give lectures
standandpedagogically sound.Toensurethat and demonstrations of points we think are
classroomteacherscanadapttheinquiryactivi- important (see also, Nunn, 1996). We assess
tiesaccordingto theirlocalresources and learningby asking students to answer multiple-
needs,STAR.Legacy was built uponfour types choice questions, give presentations, or write
ofdesignprinciples:learnercentered,knowl- essays that paraphraseand elaborate on what
edgecentered,assessmentcentered,andcom- they have learned. These methods of teaching
munitycentered.Wedescribehowa and assessment "work"in the sense that most
STAR.Legacy constructed for an educational studentscan demonstratethatthey have learned
psychologycourse helpedpreservice teachers something. Nevertheless, the quality of their
designand learn abouteffectiveinquiry-based learning is often less than satisfying. Reading
instruction. assignmentsand follow-uplecturescan produce
evidence of learningthatlooks successfulat first
glancebut misses many elementsof understand-
ing when analyzed in more detail (Bransford&
Schwartz,in press;Schwartz& Bransford,1998).
Students,forexample,often fail to use spontane-
ously what they have learned in a new setting
despite the fact that it is highly relevant.White-
head (1929)referredto the failureto apply learn-
ing as the "inert knowledge" problem. A
number of studies show that traditional
approaches to instruction often produce inert
knowledge (e.g., Bereiter& Scardamalia,1985;
Bransford,Franks,Vye, & Sherwood,1989;Gick
& Holyoak, 1983;Perfetto,Bransford,& Franks,
1983).

Vol.47. No.2, 1999,pp. 39-59 ISSN1042-1629

ETR&D, 39
40 ETR&D.
Vol.47 No. 2

Instructionalinnovations such as problem- ture at a deep level. However, if students are

based, case-based and project-basedlearning given opportunities to develop well-differenti-
have been designed to combat the inert knowl- ated knowledge, lecturescan be a powerful way
edge problem (for precise distinctions among to help students organize their knowledge and
these approaches see Barronet al., 1998; Wil- experiences(Schwartz& Bransford,1998).There
liams, 1992). Instead of simply assigning fact- are times for lectures and readings, but they
based readings or providing lectures, students need to occur when students are prepared to
begin their inquiry with challenging problems, appreciatethe insights that they contain.
and they learn information relevant to those In our experiences,case-, problem-and proj-
challenges as the need arises. Instructional ect-based learning are most effective when
approaches that are organized around cases, teachers, students and other interested parties
problems and projects have been used for a form learningcommunities,where there is indi-
number of years in professional schools for vidual accountabilityyet people collaborate in
training in medicine (e.g., Barrows,1985),busi- order to achieve important objectives, and
ness (e.g., Gragg, 1940), law (e.g., Williams, where there is access to expertise that often lies
1992), and educational administration (e.g., outside the classroom community (e.g.,
Bridges& Hallinger,1995).These approachesto Bransfordet al., in press). Frequentopportuni-
instructionare also being used with increasing ties for formatively assessing individual and
frequency in K-12 education (e.g., Barronet al., group progress are also important for helping
1998; CTGV, 1992; 1997; Krajcik,Blumenfeld, students achieve (e.g., Barronet al., 1998;Vye et
Marx,Bass,& Fredricks,1998;Penner,Lehrer,& al., 1998).
Schauble, 1998). Williams (1992) provides an In this article we describe a software shell,
excellent review of problem-based and case-
STAR.Legacy,thatis designed to guide attempts
based learning. Data on the effectiveness of to help students learn fromcase-, problem-,and
these approaches for student learning are dis- project-basedlearning. STAR.Legacysupports
cussed in Barronet al. (1998; see also, CTGV, the integrationof four types of learningenviron-
1997;Hmelo, 1998;Michael,Klee, Bransford,& ments that we believe are especially important
Warren,1993;Vye et al., 1998). for enhancinglearning(CTGV,in press):
There are risks associated with the use of 1. Learner-centeredenvironmentsthat focus on
case-based, problem-based and project-based knowledge, skills and attitudes that students
learning.A majorrisk is thatengagementcan be bring to the learningsituation
mistaken for learning.Forexample, when com- 2. Knowledge-centered environments that
pleting a hands-on activity such as building a focus on knowledge that is organizedaround
model rocket, students may be active and coreconceptsor big ideas thatsupport subse-
enthused, yet assessments of the systematic quent learning in the disciplines (e.g., see
understandingmay yield disappointing results Brown& Campione,1994;CTGV,in press)
(examples are discussed in Barronet al., 1998). 3. Assessment-centeredenvironmentsthathelp
Another risk comes from the assumption that
students' thinking to become visible so that
these are constructivist activities that require both they and their teachers can assess and
teachers to eliminate traditionalactivities such
revise theirunderstanding
as assigning fact-based readings or providing
4. Community-centeredenvironmentsthatcap-
lectures. Assumptions such as these fail to dif-
italize on local settings to create a sense of
ferentiateconstructivismas a theoryof knowing
collaboration-both among students and
from theories of pedagogy. Constructivisttheo-
with other membersof the community
ries assume that people always use their prior
knowledge to constructnew knowledge even if Integratingthese four types of environments
they are sitting through a lecture (e.g., Cobb, requires what we call flexibly adaptiveinstruc-
1994).Lecturesareoften not the best way to help tional design (Schwartz, Lin, Brophy, &
novices learn because their knowledge is not Bransford,in press). As teachersuse problem-,
sufficiently differentiated to understand a lec- project-, or case-based materials, they need to
 FORCOMPLEXLEARNING
SOFTWARE 41

tailor learning activities to the unique qualities including a model of inquiry that draws atten-
of their settings while maintaining a focus on tion to each of the learningenvironmentswithin
coredisciplinaryknowledge.To adapta curricu- a single software shell and that provides a
lum (knowledge-centered) to their setting, framework for making pedagogically sound
teachersneed to work with the priorknowledge, modifications.It supportsflexibilityby including
skills, and cultural resources that a specific a suite of software tools that are simple to learn
group of students brings to a situation. To and use, and thatmake it easy to modify a given
accomplish this, students need opportunitiesto STAR.Legacy.
bring their knowledge and beliefs to bear on
school subjects(learner-centered),and teachers Figure 1 provides an overview of the
need frequent opportunities for assessing stu- STAR.Legacyinquiry model. This figure is a
screen shot of the software'sprimary interface.
dent progress toward knowledge standards
The interfaceorganizesstudent activityinto typ-
(assessment-centered).And, because commu- ical phases of inquiry and helps make their
nity is a powerful and variable property of
learningprocessesvisible to themselves and the
learning settings, teachers should be given teacher.By clickingon the differenticons of the
opportunities to maximize community interface, the software branches to the corre-
resources to motivate and enable collaboration
and achievement(community-centered). sponding "pages" of resources and activities.
These pages provide opportunitiesfor students
STAR.Legacy promotes flexibly adaptive to complete progressively complex challenges
instructional design in two primary ways. It (the Challengemountainsin Figure1), to gener-
helps teachers adapt complex curricula by ate their own ideas on how to meet the

Figure1 D STAR.Legacy"HomePage"

Look Ahead"
&Reflect
Back 1ps
Reflect Back 'dL

Help

The Challenges
Generate
Ideas
Go Public
a?aorn;nZb

Test Your
Mettle Multiple
Perspectives

Research
& Revise
42 ETR&D,
Vol.47, No. 2

challenges (Generate Ideas), to compare their instructionto the needs of theirstudents and the
ideas to others and reflect on the differences resources of their community. One measure of
(Multiple Perspectives),as well as to develop, whether we achieved these objectives comes
assess, and revise theirunderstanding(Research from the legacies the students created to teach
& Revise, Test Your Mettle)before they present next year's students. Later in the article, we
their final solutions to each challenge (Go Pub- describehow students' legacies naturallyincor-
lic). To convey these multiple opportunitiesfor porated many of the important pedagogical
learning, assessing, and making modifications, principles of STAR.Legacyas well as resources
we use the name STAR,an acronymforsoftware from the local community. Unfortunately,
technologyfor actionand reflection. because LBD.Legacywas new to us and the stu-
In addition to the explicit inquiry cycle, dents, we were not prepared to experimentally
which guides decisions about adaptation, compare student learning and behavior in this
context with learning in more traditional con-
STAR.Legacyincludes simple software tools for texts. Nevertheless,our experiencesand the leg-
modifying the resourcesavailablein each phase acies the students created provided informal
of inquiry. Typically, teachers work with and
data thatarequite valuableand suggest research
modify a STAR.Legacythat has been filled with issues that are worthy of subsequent investiga-
activities and resources designed for inquiry
tion. In the remainderof the article,we describe
into a specific topic such as ecosystems or elec-
these observations in the process of detailing
trical circuits (see Border Blues.Legacy in
four aspectsof STAR.Legacy:
Schwartz,Lin, et al., in press; and DC.Legacyin
Schwartz,Biswas,et al., in press).STAR.Legacy 1. The explicitnessof the inquiry model
makes it easy for teachersto add new resources, 2. The componentsof a single learningcycle
including texts; patches to other programs or 3. The multiple learningcycles that help people
websites; and video clips of themselves, their progressivelydeepen theirunderstanding
colleagues and their previous students. For 4. The importance of reflecting on the overall
example, video clips of colleagues commenting
on various challenges might occur in the Multi- learning process and creating legacies for
other people to use
ple Perspectivespart of the inquiry cycle. This
allows teachers to introducetheir students to a We conclude with a discussion of the soft-
larger learning community; for example, to ware tools that make STAR.Legacyflexible and
other professors on campus (or teachers in a the prospectsfor furtherdevelopmentof flexibly
school) who have expertise that is relevantto a adaptive instructionaldesign.
particularcourse.STAR.Legacyalso enablesstu-
dents to contributeto the learningenvironment.
After developing sufficientunderstanding,stu- MAKING
THEINQUIRY
MODEL
EXPLICIT
dents can leave a legacy for next year's students
by adding their own insights or lessons (hence
the name Legacy).This can be very motivating Every STAR.Legacy, including LBD.Legacy,
uses the basic interfaceshown in Figure 1. The
forstudents,and it helps curriculaevolve so that
interface is meant to provide an explicit model
they best fit local needs. forstructuringinquiryin problem-,project-,and
In the following sections we describe case-basedlearning.The model includes a num-
STAR.Legacyin more detail. We describeit in a ber of components that we describe more fully
context where it was used it in an attempt to below. For example, there is the Look-Ahead&
improveour own practicesas college professors. Reflect Backcomponent that students complete
We developed a STAR.Legacycalled Learning at the start and finish of any STAR.Legacy.
By Doing Legacy (LBD.Legacy).LBD.Legacy Thereare the multipleChallengemountainsthat
was designed to informeducationalpsychology students are encouraged to "climb" one after
students about problem- and project-based anotherto progressivelydeepen their expertise.
learning. Our objective was to help preservice And, there is the basic learning cycle that stu-
teachers learn to design and adapt this type of dents complete as they engage in inquiry for
 FORCOMPLEXLEARNING
SOFTWARE 43

each one of the multiple challenges of a rationale for a particular component of the
STAR.Legacy. inquirymodel, along with suggestions for using
the specificactivitiesincluded with that compo-
These components offer one possible formal-
nent. Figure 2, for example, shows the explana-
ization of insights gathered through collabora-
tion that appears when an individual clicks on
tions with teachers, trainers, curriculum
the Tips apple on the Look-Ahead & Reflect
designers, and researchers.The specificcompo- Back page of LBD.Legacy. Hopefully, such
nents of STAR.Legacywere chosen becausethey
have repeatedly appeared as important, yet explanationshelp people understand the point
of an instructional design feature, and this
often implicit, components of learning and
understanding will, in turn, help them adapt
instruction---componentssuch as continually instructionto theirown ends.
making formative assessmentsof student prog-
ress, and continuallyilluminatingways for stu- The explicit inquiry model of STAR.Legacy
dents to situate and think about a topic. has been in response to our observation that
Although STAR.Legacytries to formalizethese learningis enhancedwhen teachersand learners
components,it is not intended to replaceon-the- can "see where they are"in a complex sequence
spot, expertdecisions. Rather,it is meantto aug- of inquiry. This became apparent during the
ment local expertise by helping teachers (and implementation of an integrated model of
students) develop an understandingof learning instruction and assessment called SMART,an
events. Teachersand students, forexample,may acronym for "Scientificand MathematicalAre-
choose to jump around the learning cycle nas for RefiningThinking"(Barronet al., 1998;
depending on their assessmentsof theircurrent CTGV, 1997). Using the SMARTmodel, class-
learningneeds. To help people understandthese rooms progress from problem-based learning
options, context-sensitive "Tips" provide the that develops a solid knowledge foundation to

Figure2 O A SampleTipforthe LookAhead-ReflectBackPage

Look Ahead Begin Here

&Reflec
Reflect i. ps
Look Ahead &
Reflect Back
IRs
The Look Ahead component includes a task that helps students IP
develop a concrete model of the sorts of things they will be
learning about. The Look Ahead serves a number of purposes:
* Learners preview where they are going in their learning and ook
the sorts of things they will come to understand.
* Students and teachers can pre-assess the things they
already know about the domain and what they need to learn
more about.
* It can be motivating for students as it develops both their
curiosity and their aspirations about what they will know.
In this Look Ahead, students are asked to write what they
think is aood and bad about an actual hands-on pDrolect that

Conmas Hand*OnProjet
44 Vol.47, No. 2
ETR&D,

more open-ended project-based learning. with team partnerswho teach other disciplines
Withinthe model, thereare many opportunities because they could see that their instructional
for students to generatetheirown ideas, consult techniquesactually shared a common structure
knowledge resources, share thoughts, and of inquiry.
assess and revise their understanding.By mov-
ing through cycles of learning and revision in
the context of related problem and project THEBASIC
TRAVERSING
challenges, students progressivelydeepen their STAR.LEGACY
CYCLE
understanding (Barronet al., 1998; Vye et al.,
1998). In this section we traverse the elements of the
We knew from prior implementations of STAR.Legacycycle. We focus primarily on the
SMARTthat students (and teachers)often felt LBD.Legacy that was designed to help pre-
lost in the detailsof the specificactivities(Barron service teachers understand the strengths,
et al., 1995).They did not know how particular weaknesses, and differenthow-tos of problem-,
activitiesfit togetheror how the activitieswould project, and case-based approaches to instruc-
contributeto their overall understandingof the tion. The LBD.Legacybegins with a preview in
problem or their ability to complete the project. Look Ahead (& Reflect Back).Afterwards, stu-
Students and teachers were at risk of viewing dents meet their firstinquiry challenge by click-
the curriculumas a "humongouscompilation" ing on the smallest mountain. To complete this
of learning activities that were not particularly challenge, they complete each of the compo-
connected or dedicated to any larger goal. To nents of the learningcycle. When done with the
avoid this, we started to post "SMARTMaps" first challenge, students click on the second
thatstudents and teacherscould consultto mon- mountain to meet their second inquiry
itor learning(Vye et al., 1998).These precursors challenge. They again complete the learning
of the STAR.Legacyvisualization showed how cycle, using a new set of resourcestailoredto the
different learning activities joined to make a second challenge. After completing the same
sequence that supported the ultimate goal of process for the third challenge, students con-
completing the final project. The explicit clude LBD.Legacyby revisitingthe Look Ahead
SMART Maps helped students. For example, & Reflect Back. In the following sections, we
because the activity allotted a time for revision, tracethis sequenceand describethose classroom
students learned that revision is a naturalcom- observationsthat seem most promising for fur-
therresearch.
ponent of learning,ratherthan a punishmentfor
not learning(Schwartz,Lin,et al., in press).
One of the goals of presenting an explicit
inquiry model is to help teachers and students LookAhead
transfer inquiry practices from one topic to
anotherbecause they can see similaritiesacross The Look Ahead & Reflect Back is represented
the activities.Forexample, during the past year by the binocularsin the upper-leftcornerof the
we have had the opportunity to introduce the main interface(Figure1). Studentscomplete the
STAR.Legacyframework to a number of K-12 Look Ahead before they meet any of the
inservice teachers who had been using various challenges.Clickingon the binocularicon brings
non-Legacycurriculumunits, such as our prob- students to a screen that is designed to help
lem-based Jasper Adventure and Scientists in them preview the knowledge domain and
Action series (for descriptions,see CTGV,1997). develop learninggoals.
We showed them how these units looked when Nearly all models of learningand instruction
placed in the STAR.Legacy framework. The emphasize the importance of goal setting (e.g.,
response was extremely positive. Teachersfelt Newell & Simon, 1972).Often,goals for learning
thatthey could instantly"see"the implicitlearn- arepresentedas specificobjectives,primarilyfor
ing cycles common to much of their instruction. the teacher's eyes. For educational psychology
Moreover, the teachers found it easier to talk these might include, "studentswill list three rea-
 FORCOMPLEXLEARNING
SOFTWARE 45

sons that problem-based learning improves prior knowledge, interests,and communitiesof

transfer,""students will compare and contrast practice.(b) It provides learnersan opportunity
summative and formativeassessment,"and so to see their destination in a domain of knowl-
forth. Many attempts to list specific objectives edge and to see the sortsof things they will come
fallshortof the ideal becausethey are frequently to understand. (c) It helps develop a shared
perceived as a discrete list of statements that domain model that facilitates discourse and
often seem vague and unrelated.Ourpreference community within the classroom.(d) It offers a
is to help teachersand students develop a more learnerand teacherpre-assessment.As learners,
coherent and concrete model of the topic that students can identify what they need to learn
they will learnabout and the sorts of things they more about. Forteachers,this activity can occur
will be able to accomplish by completing their at a classroomlevel so thatteacherscan appraise
Legacyjourney. their class's initial domain knowledge. This can
The LookAhead & ReflectBackfor LBD.Leg- help teachersanticipatelearningneeds, as well
as help them design and tailor their use of
acy begins with an audio narratorwho explains
that one of the benefits of expertise is the STAR.Legacyand other classroom resources.
Forexample,with the LBD.LegacyLookAhead,
increased ability to notice significantevents in
we were able to see that few of the education
the environment. As an example, the students
students saw anything wrong with an activity
are shown an eight-second,digitized video clip
thatprovidedno contextfor learningaboutcom-
of a single play in a footballgame. Afterwards,
passes, and no informationthat explained why
they watch protocols of two women who were
asked to recountwhat they had seen in the play. compasses work, or why they are useful. This
indicatedimportantgaps in the students'under-
The firstwoman, a novice, states,"I'mso embar-
rassed. I think ... I thinkthe quarterbackran the standingof how to transformlearningby doing
into doing with understanding.(e) Finally, the
football."The second, a relative expert, states,
LookAheadcan serve as a benchmarkfor reflec-
"OK,the defensive backsjumpedoffside during tion and self-assessment. In particular, after
a blitz.The quarterbackhandedthe ball to one of
the running backs who broke two tackles and completing the learningcycles, students return
to ReflectBackon what they have learnedcom-
was on his way to a touchdown."The students
pared to their firsttry at the LookAhead. As we
then get a chance to watch the football play document later, this can help students see how
again to assess their own expertise. After this much they have learned.
example, the narratorexplains that LBD.Legacy
will increase their ability to notice and under-
stand importantaspects of a type of instruction
called learningby doing,in which students com- TheChallenges
plete problems, projects, and hands-on activi-
ties. To get a sense of theirown expertisein this
After completing the Look Ahead, students are
domain, the narratorsuggests that the students
evaluate the provided exampleof a prototypical prepared to work on the STAR.Legacy
challenges. To access the first challenge, stu-
hands-on project that involves making a com- dents click on the smallest mountain. In
pass from magnets. The students do not spend LBD.Legacy,Challenge1 asks students to gener-
their time studying this particularproject in ate and explain reasons why they might want
LBD.Legacy.The compass projectsimply offers people to learn by completing projects. To
a brief and relatively familiar example of an anchortheir inquiryinto this challenge, the stu-
activity that falls under the rubricof learningby dents watch a clip of a movie that shows a high-
doing. The students' task is to notice what is school historyteacherwho is giving a stultifying
good about the compass projectand what could lectureon the chronologyof the Depression era
be improved and how. (fora discussion of anchors,see Bransford,Sher-
Currently, we see five potential benefits to wood, Hasselbring, Kinzer, & Williams, 1990;
the Look Ahead. (a) It helps students to situate CTGV, 1994). Afterwards, a "narrator" in
the upcoming lessons and learninggoals in their LBD.Legacysuggests that the students use this
46 Vol. 47, No. 2
ETR&D,

example of instruction as a foil to help them tronic notebook where they could be subse-
thinkabout why projectscan be a useful alterna- quently revised and improved upon.
tive. The remainder of the learning cycle pre- In a computer-richclassroom,one way to use
pares the students to complete this challenge. the notebookfeatureis to have each student gen-
Aftercompletingthe learningcycle and present- erate ideas into a separate notebook. Alterna-
ing their final answer to Challenge 1, the stu- tively, for the LBD.Legacy,the challenge was
dents repeatthe process with Challenges2 and 3 watched as a whole class, and students gener-
(describedbelow). ated theirinitialideas on paper.Afterwards,stu-
dents offered their ideas in the whole-class
GenerateIdeas context, and the teacher combined those ideas
into the collectivenotebookshown in the figure.
After students understand a challenge, they The notebook served as a focal point for further
discussion of the generatedideas.
progressto GenerateIdeaswhere they explicitly
make theirfirstattemptat generatingissues and Thereareseveral reasonsthatGenerateIdeas
answers relevantto the challenge.Typically,the is an explicit component of the Legacy cycle.
GenerateIdeaspage includes a text thatreminds One is thatit encouragesstudents to share ideas;
students of the challenge (e.g., justify project- everyone has an opportunity to hear what oth-
based learning) and that instructs students to ers think. Forthe teacher,this complements the
record their initial ideas. Figure 3 shows the Look Ahead by providing a more specific
ideas that one group of education students gen- assessment of what the students understand
erated for Challenge 1 after watching the video about the topic. For example, Figure 3 reveals
clip of the excessively passive classroom. The that students primarily think about a project's
students' thoughts were entered into an elec- benefitsin termsof motivation.Forthe students,

Figure3 0 TheClass'sNotebookEntries
forthe GenerateIdea Phaseof Challenge1

Generate
Ideas

Help

Projectsinsurekidsworkat own level. Thisis Notebook

good becausetheycan understandcompared
to a lectureover theirhead.

Professorwon't have to lecture. Kids'llpay

attention.

Can accomplishmorein 45minutesof project

than listeningto a lecture.
SWhyPeojct
 FORCOMPLEXLEARNING
SOFTWARE 47

it provides an opportunity to learn what other between what the students generatedand what
students think.Too frequently,studentshave no the experts generated, students are helped to
idea about their peers' knowledge. As a conse- grasp the significance of new informationand
quence, it becomes difficult forstudents to build understandits relevancefor helping them think
community by taking advantage of knowledge differently(Bransford& Schwartz,in press).
distributedthroughouta classroom.
Recall,for example, thatthe students primar-
A second reason to have students generate ily generatedideas about the motivationalvalue
ideas is to make thinking explicit rather than of projects(see Figure3). Contrastthis with the
allowing it to remaintacitand vague. The act of observationsgeneratedby the four experts who
specifically recordingtheir views about a topic comprise the Multiple Perspectives for
helps students discover what they think and Challenge1. One expertpoints out that he origi-
know. This awareness is facilitated when stu- nally thought motivation was the only signifi-
dents contrast their ideas with other ideas. cant issue, but then as he investigatedthe issue
Appropriate contrasts, much like setting two of learning he found out that the way students
wines side by side or seeing two ideas juxta- are expected to learn in this format often has
posed, can help students notice importantdis- profoundeffects on their subsequentabilitiesto
tinctions they might otherwise overlook or use thatinformation.Anotherexpertnotices that
dismiss (Bransford et al., 1989; Bransford& the students are all individually seated in the
Nitsch, 1978;Gibson & Gibson, 1957;Schwartz video segment and that there is none of the nat-
& Bransford,1998). For example, most instruc- uralsocial interactionthatoften facilitateslearn-
tors in the behavioralsciences have experienced ing. A thirdexpertpoints out that it is important
the frustrationof students'statingthata particu- to rememberthat thereis a time for lecturesand
lar point is just "commonsense," when in fact, that they can be very effective-the trick is to
the students would never have applied the com- prepare people to be told. The fourth expert
mon sense on their own. When students are explains that the tests that usually accompany
asked first to generateideas, they become more lecture-styleinstructionoften find no parallelin
appreciativeof the contrastbetween theirinitial the real world. Consequentlystudents spend an
observationsand ensuing observationsthatthey inordinateamount of time learninga skill (e.g.,
had originallyoverlooked.Thisappreciationcan preparing for multiple-choice tests) that
help them grasp what is new about ideas rather becomes obsolete immediately upon leaving
than merely assimilatethem into old ideas and school.
gloss over what is new and important. Afterthe educationstudents listened to these
four perspectives, the class was silent. The
instructor parenthetically commented, "It is
MultiplePerspectives amazing to find out what you never even
thought about,"The whole class nodded vigor-
Following the initial generationof ideas comes ously in agreement. A useful line of research
Multiple Perspectives. There are many situa- might formally document whether producing
tions where multiple perspectives are a natural ideas about a topic beforehearingdifferentper-
component of learning(e.g., a conferencepanel, spectives on that topic increaseslearners'intel-
lectualengagementwith the ideas at hand.
cooperative jigsaw groups, differentvoices in a
novel). In STAR.Legacy,this importantfeature In addition to complementing the Generate
of learning is made explicit. For Challenge 1 of Ideas phase, the Multiple Perspectives phase
LBD.Legacy,students hear the ideas generated serves a numberof purposes. First,the perspec-
by four educational experts who also saw the tives provide guidance into the topics that the
video of the Depression-era lecture. These students should explore to learn about the
experts introduce students to vocabulary and domain. The perspectives do not give away
perspectives that are quite different from their solutions;instead they directthe students to rel-
own and that characterizeexpert approachesto evant domains of inquiry. In the LBD.Legacy,
the topic. By focusing attention on contrasts these domains are cognition, social interaction,
48 ETR&D.Vol. 47. No 2

and assessment. Second, the Multiple Perspec- to build the local intellectualcommunity. When
tives featureindicatesthata given situationusu- students saw one professor, for example, they
ally has multiple vantage points and that this is stated, "Hey, we've read stuff by him. So, that's
acceptable.This is differentfrom much instruc- who he is. I've seen him around."Severalteams
tion that provides only one model for how to of students took advantageof learningabout the
think about the material. Learning multiple scholarly resources in their neighborhood. As
entry points into a given topic increasesthe flex- we describe later, they decided to interview
ibility of futureproblemsolving (Spiro& Jehng, these faculty as a component of the legacy they
1990).Third, there is great added value in hav- left for next year's students. This approach to
ing expertscommenton the types of demonstra- Multiple Perspectives also helps to build com-
tion tapes that are currently used in many munity among the experts themselves. The
classrooms. Experts, for example, often notice experts were curious to see what the other
what is missingin the videotape and why that experts, their colleagues, had to say. Although
absenceis theoreticallysignificant. they knew one another through college func-
One of the excitingpotentialsof MultiplePer- tions, they rarely had a chance to hear one
another'sexpertise.Likethe students, afterhear-
spectives is that it can combine and make avail-
able distributed expertise among people who ing their colleagues' insights, they often com-
cannot easily be brought together. This has a mented, "I hadn't even thought of that." One
worthwhile question for future research is
profound impact on the instructor'sability to
whether this asynchronousgatheringof people
adapt and tailor instruction by capitalizingon
local resources.And, it serves as a way to create has a meaningful impact on the local commu-
an intellectual community among people who nity. A second question is whether teachersfind
it sufficientlycompelling thatthey will add their
are too busy to meet formally.A good instance
of these benefits may be had by jumping ahead own experts to adapt a Legacy to their local cir-
for a moment to Challenge 2 of the second cumstances.
inquirycycle. InChallenge2 of LBD.Legacy,stu-
dents are asked to develop a set of design princi-
ples for project-basedlearning and to organize Research&Revise
those principlesinto a helpful visualization.
To help students make headway on this In Research& Revise, students may take part in
challenge, we recruitedlocal facultyand master many different activities including collabora-
teachers for the Multiple Perspectives. Each
tions, consulting resources, listening to just-in-
expert received a video cassette containing a tinr. lectures, completing skill-buildinglessons,
short segment that served as the anchor for
working with legacies left by students from pre-
Challenge 2. The tape shows a local TV news vious years, and conducting simulations and
report in which children launch model rockets hands-on experiments. This component of
as part of a school project.The teachersand fac-
STAR.Legacyis very inclusive and supports
ulty were asked to watch the videotape at their most instructional designs. For example, for
leisure.A week or so later,they were videotaped
Challenge 1 of LBD.Legacy,the resources are
as they gave short commentarieson what they
primarilytextual.The students readarticlessug-
had noticed relevant to design principles for
gested by experts,and they use their textbookas
learning. Complementing their brief commen- a resourcefor completingsimple tasks that help
taries,they were asked to suggest a readingthat them understand the potential benefits of proj-
would help students who wanted to learnmore. ect-based learning. For Challenge 2, however,
Students also watched the videotape as part of the students analyze video cases, practice
their design challenge, and they generatedtheir instructionaltechniques with one another, and
ideas in the notebook prior to hearing the evaluate Websites that offer project-basedles-
experts'perspectives. sons. In either case, the key criterionfor includ-
Recruitinglocal experts to make up the Mul- ing a resourceis that the instructionalmaterials
tiple Perspectiveson the news broadcasthelped should help students reach the goals of explor-
 FORCOMPLEXLEARNING
SOFTWARE 49

ing a challenge and of revising the ideas that the challenge. The "test"can take a number of
they originallygenerated. different forms, including multiple-choicetests
We have found it useful to organize the with feedback, rubricsfor evaluating products
resources according to the Multiple Perspec- they plan to make public, and "near transfer"
tives. Figure4, for example, shows the resource problems.Test YourMettleis meant as a forma-
topics available for Challenge 2. Each of the tive instructionalevent, not a final exam. It is a
images is keyed to one of the six experts who chance for students to bump against the world
commented on Challenge 2. By clicking on an to see if their knowledge is up to the task.If it is
image, students move to a page with resources not, they should returnto Research& Revise to
specificallyassociated with the ideas generated improve theirunderstanding.
by the relevant expert. This organizationhelps For the first cycle of LBD.Legacy,the Test
students and teachersfollow those perspectives Your Mettle asked the students to evaluate the
that seem particularlysuited to their needs; we reasons they had generated for using project-
do not assume that students and teacherswill based learning. Eachstudent preparedfive rea-
necessarily use all the resourcesavailable in a sons. The students met in small groups and
STAR.Legacy. selected their best reasons from among their
group. They were told that the reasons from
TestYourMettle each group would be compiled and the class
would vote on the top ten reasons. These rea-
When students think that they have developed sons would then be posted on the WorldWide
their understanding of the original challenge, Web through Go Public, and they would be
they are asked to complete Test Your Mettle evaluatedby studentsat StanfordUniversity.As
before they can Go Publicwith theirsolution to the students met in their groups, they began to

Figure4 F Research & RevisePage for Challenge 2

Research
Tips
Social Nornms

Help

MeetingStandards
Culture
Preparednefor
ColIaamating Notebook

Coguition
Legacies
Assessment
Task
Reminder
Doingwith
UnderNanding
Home-School
Conmnecions

Learningin Context
50 Vol. 47, No. 2
ETR&D,

question what made a good reason, and they Go Public

wanted examples. Test Your Mettle provided
this information. It stated, for example, that a Aftercompleting Test YourMettle,students are
good reason might explain the conditions prepared to Go Public with their thinking and
needed to achieve a benefit from project-based make available to others their best solutions to
the original challenge. Go Public for Challenge
learning. For example, they received a sample
frame that stated, "Projectscan provide excel- 1, for example, required students to post their
lent opportunities for assessing student under- solutions on the Web for evaluation by an out-
side audience.Thereare several reasons that we
standing BECAUSE.... But,this can only occur
IF ... ." Eventually,the differentgroups sponta- ask students to Go Publicwith their knowledge.
One is that public presentations of knowledge
neously began to ask or demand that they get a
chance to rewrite their reasons. The students add a high stakescomponentthat motivates stu-
returned to Research & Revise to help them dents to do well. For example, we suspect that
develop an improvedset of five reasons. asking VanderbiltUniversity students to post
their "reasonsfor project-basedlearning"on the
Test YourMettleshould make thinkingsuffi-
Web for evaluationby Stanfordstudents had an
ciently visible so that a teacher,knowledgeable
appreciableeffect on their willingness to revise
peer, or even the student, can identify the need for Challenge1. Anotherreason for going public
for further learning. The preceding example is to make student thinkingvisible so other stu-
reliedon the college students to assess theirown dents and teachers can appreciate high quality
work relative to general standards and to one elements of understanding.Consider the case of
another. This had substantial effects on their the design-principlevisualizationsthatstudents
evaluations of the quality of their own learning
produced for Challenge 2. Many students were
and performance.An importantresearchques-
annoyed by the unusual nature of the task. Yet,
tion is how to support less sophisticated stu- when students publicly presented their solu-
dents so they can effectively learn to self-assess tions in Go Public,the students in the class were
(for an approachthat uses contrastingcases, see
uniformlyimpressedby the creativityand depth
Lin & Bielaczyc,1998).This is an importantskill revealed in the visualizations-visualizations
to develop. Ideally, people should be able to that ranged from the jigsaw puzzle shown in
make estimations of their own understanding
Figure5 to a three-dimensionalmobile to a Jeo-
and make needed course correctionsbeforethey
pardyTM-type game with columns of index cards
take a big exam or turn in a big projectat the that had principles on one side and reasons on
office. the other. The opportunity to see each other's
Test Your Mettle should serve as a powerful ideas is important if for no other reason than
formative assessment event that guides and that students often do not have a chance to learn
motivates students to revise and improve their or appreciate the more complex ideas of their
work. In both math and science, we have found peers.
thatformativeassessmentcoupled with revision There are many different ways that one
opportunities significantly increases achieve- might encourage students to Go Public. In our
ment among secondary school students (e.g., work, we have explored asking students to pub-
Barronet al., 1995;CTGV,1997;Vye et al., 1998). lish to the Internet,to presentwithin their class-
Although it seems obvious that formative rooms, to meet with a panel of outside experts,
assessmentcoupled with revisionshould lead to or simply to write in a collective notebook.One
better performance,it is strikinghow few class- approachthat we find especially exciting can be
rooms actuallyencourageformativeassessment done in the context of open-ended challenges
and revision, especially formative self-assess- where there is little concern about copying of
ments undertakenby students. Most classroom answers. Students publish to the STAR.Legacy
science projects,for example,are only evaluated itself and help adapt it for future generations.
by the teacher at the end (Towler & Broadfoot, An example of this approach may be found in
1992).Students never have the chance to "alpha Challenge3 of LBD.Legacy.This challenge asks
test"and improve theirprojects(Gardner,1991). students to write a college-style essay that pres-
 FORCOMPLEXLEARNING
SOFTWARE 51

Figure5 O- A Design-PrincipleVisualizationMade Publicin Response to Challenge 2

low

ilk

" 0,46 ,Jd3\ te

~:e;
q~t~h0(tYwA

ents a good and bad versionof a projectand uses challenges. Elsewhere,we have found that it is
the class readings to explain the differences. often advisable for students to begin with more
These essays have been added to LBD.Legacy, circumscribedproblemsbefore they take on the
and thereforehave been made public. Students complexity and open-endedness of projects
from the class can take home a CD-ROMthat (Barronet al., 1998).Forexample, in one study,
includesthese essays, and studentswho take the 6th-grade students in one of two conditions
class in subsequent years will be able to read designed business plans for a booth at their
these legacies to help guide theirown work. We school's fun fair. For this project, students in
say more about leaving legaciesbelow. each condition received the same directive to
show how theirplans satisfied various financial
and logistic constraints.In the Problem-to-Proj-
MULTIPLE
LEARNING FOR
CYCLES ect condition, students first completed a prob-
PROGRESSIVE
DEEPENING lem-basedactivityby working on a video-based
adventure called "TheBig Splash"from the Jas-
As illustratedin Figure 1, STAR.Legacyencour- per Adventureseries. In the Project-Onlycondi-
ages multiplechallenges,representedas increas- tion, students did not complete any preparatory
ingly tall mountains. (STAR.Legacycurrently activity prior to designing their fun fair booth.
supportsup to five challenges.)Thesechallenges The differenceswere definitive. Studentsin the
provide opportunities for students progres- Problem-to-Projectcondition developed plans
sively to deepen their knowledge of the topic that mathematicallyevaluated the feasibility of
being explored. The early challenges can pre- differentbooth plans in terms of cost and likely
pare students for more ambitious later income.In contrast,students in the Project-Only
52 ETR&D,Vol. 47, No. 2

condition spent most of their time debating the To demonstrate that Reflect Back helps stu-
frills that would make the fun booth appealing. dents appreciate their own learning, we ran a
For LBD.Legacy,we used three challenges small experiment with an LBD.Legacyclass.
that were intended to help the students first Half (11) of the college sophomores in one class
notice valuablepsychologicalaspectsof project- were randomly assigned to the experimental,
based learning,then consider how to design for self-comparisoncondition. At the beginning of
those aspects, and finally actually develop les- instructionwith LBD.Legacy,they completed a
sons according to principles of psychology and Look-Aheadthat required them to evaluate the
instructionaldesign. Challenge1 asked students compass-buildingprojectdescribedearlier.The
to create reasons why project-basedlearning other half of the students completed a filler
may be useful. Challenge 2 asked students to activity.Two months later,after completing the
create and organize design principles for three challenges of the LBD.Legacy,all 22 stu-
implementing problem- and project-basedcur- dents evaluated the compass-buildingprojectas
ricula. Challenge 3 asked students to design part of Reflect Back.At this time, the self-com-
good and bad projectsand to explain why they parison students were given their original
were good and bad in light of their previous responses. An hour later, masked as part of a
readings and activities. For each challenge, the differentactivity,the studentswere asked to rate
students moved through the usual sequence of how much they felt they had learned in the class
generatingideas, hearingdifferentperspectives, on a 7-point scale with 7 being the most. The
consulting resources that included scholarly self-comparisonstudents rated their learning at
articlesas well as activities,assessing the quality 5.3 whereas the control students rated their
of theirwork, and then going public.
learning at 4.1, a reliabledifference;t(20) = 1.9,
This may seem like a lot of time to spend on SE = .63,p < .05,one-tailed.Of course, we cannot
one idea, namely, learning by doing. However,
prove that the self-assessmentstudents' evalua-
along the way, the students progressivelydeep- tions were more accurate.Even so, besides the
ened their understandingof importantthemes obvious implication for how to increase course
in educational psychology. Unlike traditional
ratings,the results indicate that the opportunity
classes, the themes of cognition, social interac- to contrast present and past knowledge helps
tion, and assessmentwere not taughtas self-con- students appreciate how much they have
tained topics, presented in modular chapters. learned.
Instead, LBD.Legacy introduced important
fields and scholarsin the service of helping stu- Students rarely have a chance to appreciate
dents understand an important method of how much they have learned.Even though stu-
instruction. dents usually receive grades, grades are indirect
measures of knowledge growth and do not
REFLECTING
BACKON LEARNING
AND depend on the student's recognizing-specific
LEAVING
LEGACIES knowledge gains. The lack of opportunities to
reflect on knowledge growth may be problem-
Reflect Back
atic,especiallyfor lower-achievingstudents. It is
important for students to recognize when they
We noted earlierthat STARstands for software
have been successful learners. Reflecting on
technology for action and reflection.One way
growth is especiallyimportantafterlearningsit-
that we encouragereflectioncomes throughthe
uations that seemed confusing and perhaps
ReflectBackthat occurs aftercompleting all the
challenges. Students revisit the original Look frustrating.We want students to develop a "tol-
Ahead activity and compare their new, erance for ambiguity" (Kuhn, 1962) and
informed responses to their original work. This "healthy courage spans" (Wertime, 1979). See-
gives the students a chance to see how much ing their perseverance rewarded with knowl-
they have learned and to further extend their edge gains is important to this end (Dweck,
knowledge about the domain and their own 1989). The Reflect Back feature encourages
learning. observationsabout one's learning.
 FORCOMPLEXLEARNING
SOFTWARE 53

LeavingLegacies instructorsof educationalpsychology had asked

students to teach the class for a given topic. The
STAR.Legacycapitalizeson technologicaldevel- number of "stand-up lectures"was surprising.
The students who worked with LBD.Legacy
opments foreasily pressingCD-ROMs.Students
can receive a CD of a STAR.Legacyafter it has behaved quite differently.
been adapted and modified by the teacherand ForChallenge2 of LBD.Legacy,students had
by the class. This CD provides an excellent to develop a visual representation of design
review of the course content. It can also include principles. The Multiple Perspectives for this
the students' solutions to the challengesas well challenge included six experts who each recom-
as the legacies they and the teachershave left for mended relevant readings. Consequently,
the next cohortof students.The studentscan see Research& Revise included six sets of readings
that they have createda useful product for oth- on six topics (see Figure4). Requiringall the stu-
ers. We believe this can be very motivating for dents to read all the articleswould have taken
the students as they realizethattheirinsightsare too long. Therefore,small groups of students
valuable. were asked to become experts in a topic. Their
Not only do the legacies serve the students task was to develop a firm understanding of
who create them, they also benefit subsequent their readings and then to create 30 min of
students who use the modified Legacy. For instructionthat would help the rest of the class
learnwhat they felt were the importantpoints of
example, we are studying the benefitsof having
middle-school students leave legacies that the material. They were told they could teach
describelearningstories.The students narratea any way they wanted, but that their instruction
personal event or realizationthat they believe eventually had to be "programmed"into their
may help the next cohort of students do a good respectiveResearch& Revisesubsection.
job of learning. We predict that hearing a stu- Two aspects of the students' work suggest
dent from a prior class explain something like, that furtherdevelopmentand researchon flexi-
"thechallengewas initiallyfrustratingbut stick- bly adaptive instructionaldesign is warranted.
ing to it was worth it,"will have positive effects. The first is the students' capitalizationof local
Another way the legacies can help subse- resources. The second is their development of
instruction that tacitly included importantele-
quent students is that they can provide exam-
ments of the STAR.Legacycycle.
ples of what a good challenge response looks
like. For the LBD.Legacy, we have already
described how the students left examples of
on LocalResources
Capitalizing
their essays for future generationsof users. As
anotherexample, we videotaped a few students The students took advantage of local resources
who described the design-principle visualiza- in two ways. (a) They spontaneously tracked
tions they createdfor Challenge2. By including down the local experts shown in the Multiple
these as legacies, next generationstudents can
Perspectives.Some students simply talked to an
get an idea of the creativity that is possible expert to deepen their own understandingand
within Challenge2 (e.g., Figure5).
gather more readings. Other students inter-
Legacies can also add new domain content. viewed and videotaped the expertsfor inclusion
This, of course, is part of the idea of flexibly in their instruction.In one case, they even bor-
adaptive instructional design-instructional rowed the raw footage discussed in one of the
resourcescan be supplemented with new mate- expert'sarticles,and used it to createa noticing
rials that come from teachers,students,and the activitywhere otherstudents tried to find exam-
local community. The first students who used ples of the theoreticalpoints made in the article.
LBD.Legacyleft materials that now constitute In each of these cases, one can see how the Mul-
importantresources.We describethese legacies tiple Perspectivesaspectof STAR.Legacyhelped
next because they bear on the question of to createintellectualcommunity.(b)They found
whetherLBD.Legacycan facilitateteachers'abil- local experts who did not appear in Multiple
ities to design instruction.In previous years, the Perspectives.In one case, for example, the stu-
54 Vol.47, No. 2
ETR&D,

dents created a lesson that compared the case- go backwards in the cycle. This worried us
based and clinical methods of legal instruction. because we do not want to imply that learning
To do this, they interviewed law professorsand events can only occur in a given sequence. We
students, as well as lawyers who reflectedback want people to think about these important
on the relationshipbetween their legal educa- learningevents and to use them flexibly in their
tion and their practice.In tandem, the two uses own instruction.Generatingideas, for example,
of local resourcesprovide a nice exampleof how is fundamentalto all learningand it should cer-
STAR.Legacy supports the development of tainly "beallowed" in Research& Revise.Fortu-
instructionthat adapts accordingto opportuni- nately, after having seen the students' legacies,
ties within the local community. we believe that some of people's confusion
stems from their short exposure to the
STAR.Legacy framework and the novelty of
theInquiryModel
Incorporating flexiblyadaptive instructionaldesign.
The students'legacies naturallytended to incor- The students who used LBD.Legacyadapted
porate elements of the learning cycle in flexible its inquiry model for their own legacies. The
ways. Previously, we found that many people majority of the students adapted the inquiry
who have seen short demonstrations of model in a similar way. First,they began with a
STAR.Legacyhave tended to worry about pro- challenge. For example, Figure 6 shows the leg-
ceduraldetails. They asked how Test YourMet- acy that one group of students developed to
tle differs from a Challenge,wondered whether teach about home-school communication. In
students could Generate Ideas in Research & their legacy, they included four family profiles
Revise, and questioned whether students could in written form (one is shown in the figure).

Figure6 O The Legacy One Groupof Students Leftas a Resource for LearningAbout
Home-SchoolConnections

Instructions

Research
ThomasFamily
Lupton Family
Lione Family
0 Tips
Chavez Family
What you should do.

Help
Youaregrandmotherand-
atherione YouW
grandf hae
hadcustody. two,.,
ofyour
their
grandchilidren.sine
i o yearsag .
parentsd:ied Notebook
the
Youhavevowedtetlake
bestcarepossible
andotfyour
grandchildren the
prpoyde
bet .educetiohaa'e6ibleble
Yo
ur.
Ms. Jones Mrs. Smith sources-ofincomea-dre
securitybenefits :sociaal
a aide
etirement.
and'occiaalndij
pay,
ijobs
Iniordercm.ak~ure the:
chil&deni eveytling'hey
need. Youareactivelyinvolved
urnicular
extrac You,-
ectigitUies.
eadhnight
spendIsailytihme
Nithhomeworkand
andhelp
You
projects. Go Back
wo.uld[enioyt
withthe
regularconitact
.keep
teacherandbeasinolvedas
possible.

Mr.Rodriguez Mr.Chang
 FORCOMPLEXLEARNING
SOFTWARE 55

They also included four videotaped interviews tinue to adapt or create instructiononce they
with teachers. The challenge was to decide leave the luxury of the universitysetting.
which family should go with which teacherand
to explain why. A second common component
of the students' legacies was the instructionto SOFTWARE
TOOLSTHATPROMOTE
the class members that they explicitly generate FLEXIBLY
ADAPTIVE
theirsolutions to the challenge.So, in the case of INSTRUCTIONAL
DESIGN
the family-teacherconnection, the class gener-
ated its solution to the challenge of matching Our surveys of professional designers indicate
parents and teachers. Third, the students thata commoncriterionfor a good design model
included the view of an expertor severalexperts is whether it helps designers make design deci-
who explained their answers and reasoning.In sions as efficiently and smoothly as possible.
the family-teachercase, the expert viewpoint STAR.Legacy,however, requiresmany instruc-
was simply a written statement that had been tional decisions that depend on an understand-
crafted by the students. In the case of a legacy ing of learning theory (and on having
featuring legal education, the students video- knowledge of the domain under instruction).
We try to scaffold the application of learning
taped an expert who wrote an article on legal
education. In either case, the students used the theory by including the explicit inquiry cycle
and the Tips described earlier. We are also
experts as a combination of Multiple Perspec-
tives and Research& Revise.This is becausethe exploring a more inductive scaffold where
expertspointed out what the students had over- designersreceive a templateLegacythat houses
several examples of priordesigns for each com-
looked, but they also delivered the primarycon-
tent about what is importantto know about the ponent. The designers can choose one of the
domain. Finally, as the fourth component of examples as a model to guide their design for
thatcomponent,and they can slowly adapt it by
their legacies, the students included a Test Your
Mettle that allowed the class to extend the swapping in their own content (Bell, 1998). At
this point, however, we are primarilytrying to
knowledge to a new context and to self-assess maximize creativity in design ratherthan pre-
whether they had learned the intended con-
maturely constraining people's designs. Our
cepts. Forexample, one group of studentsasked
the class to apply its knowledge to a previously experiences with the LBD.Legacysuggest that
over a more protractedperiod of time people
unseen video of a classroom.Theclass had to see
become comfortable with the STAR.Legacy
if it could notice the theoreticallyimportantcon-
framework.Additionally, in work with profes-
cepts at play in this classroomsetting. sional instructionaldesigners (Schwartz,Lin, et
All told, the students naturallyincorporated al., in press), we have noticed how different
the ideas of a challenge, a period of generation, components,such as Test Your Mettle,increas-
the delivery of expert resources from the com- ingly help designers think explicitly about the
munity, and the opportunityto test understand- contentthey expect students to know. Similarly,
ing. Some of the students seemed to be aware we have found that when designers begin to
that they were explicitly adapting the Legacy sequence multiple challenges ratherthan use a
cycle, whereas others simply thought they had single one, they begin to consider what ideas
come up with a neat way to teach.In eithercase, they expect the learnersto generalize.
the results are encouraging because they show We assume that an understandingof learn-
that the STAR.Legacy framework provided ing goals and events is an excellent way to help
enough of a design theory that the students designers to adapt to and capitalize on local
could create original instruction and could needs and resources.To this end, we have devel-
adapt STAR.Legacyto their own instructional oped the explicit inquirycycle. We also assume
goals. A worthwhile line of research,in addition that the simplicityof authoringin STAR.Legacy
to formalizing the preceding observations, encourages flexibility. STAR.Legacydoes not
might be to follow teachersinto theirclassrooms include many authoring tools. Those tools that
to determinewhether,when, and how they con- are included are primarilyfor helping designers
56 Vol.47, No. 2
ETR&D,

access content that is developed and stored out- according to types of media. For example, in
side of the STAR.Legacyshell. For example, Research& Revise, they organized the learning
LBD.Legacylists articles that students should resourcesaccordingto useful Internetlocations,
consult; it launches video and audio as in the suggestions for hands-on activities, textual
case of Multiple Perspectives;it launches such resources, video segments, simulations, and so
applicationsas a Web Browserand a simulation; forth.This is a fine organizationalscheme for a
and it suggests activities that can be completed merchandise catalog, but ideally, like instruc-
in the classroom.In this regard,STAR.Legacyis tional materialsshould be organized at the ped-
more of an organizational"launchpad" than a agogical level (Murray,1998)or by the concept
place to programlarge amounts of originalcon- they serve, rather than by the media they use.
tent. Consequently,we modified the tools to encour-
Ourgoal is to provide simple ways forpeople age designers to "justify"the learning function
to author and adapt STAR.Legacyin pedagogi- for a particularinstructionalactivity. Figure 7
cally sound fashion. Consequently, we have shows an example of the main programming
kept the authoring capabilities quite simple. tool that encouragesthis justification.
There is a single tool pallet that people use to Users add a learning event to STAR.Legacy
include content for any of the components of by clicking on a design tool from the pallet
STAR.Legacy.We have also designed the tools shown at the rightof Figure7. The tool opens the
to encouragereflectionon learninggoals. This is dialog box shown at the upper left of the figure.
important because we found that when we This box asks the designer to define various fea-
introduced prototypes of STAR.Legacy,people tures of the learning event, including physical
tended to organize different learning materials appearance,a descriptiveaccountof its learning

Figure7 Z The ProgrammingToolsforAdding a LearningEventto STAR.Legacy

4 File Edit Tools Uiew Objects TeHt Tips 00 00 00Q.

Action Name: Parent-TeacherInteraction

Re Action:
O Movie @ Go to card
O Sound 0 Open URL
a Launch O Launch application
( Clear ] info Help
Details: Card Name -
SShow Parent-Teacher Interaction
Name
(Description CardName FSelect Card EditCardJ Notebook

SDmageone

* Youwill try to connectfamiliesandteachersonthe basis

of importantpropertiesthat younoticein each.
* Youwill learn thatthereare severaldistinctteacher
styles whenit comesto makingconnectionswith the home. Paret-Tea lnter.ctiori
* Youwill also learnthatfor parentsthereare several
distincttypesof expectationsfor beinginvolvedin their GO Back
child'sschooling.

.................
 FORCOMPLEXLEARNING
SOFTWARE 57

function,and its action.The actiondefines what To support improved pedagogy as we

type of interactiveevent occurs when a student researchlearningtheory,we have createda mul-
or teacher clicks on the interface.These events timedia, instructional design environment
include playing a movie or sound, opening a called STAR.Legacy.STAR.Legacyoffers one
browserto a specificlocation,showing text,sim- example of how to organizeand facilitatelearn-
ulations, and so forth.In the figure, the selected ing in "challenge-based"environmentssuch as
action opens a new page (shown in Figure6). case-, problem- and project-based learning.
To encourageattentionto instructionalgoals, STAR.Legacysupports the design of environ-
ments that are simultaneously learner,knowl-
designers are expected to provide each learning
event with a caption and a learningdescription. edge, assessment and community centered
The caption is meant to tell students what topic (elaborationof this point is availablein CTGV,in
or issue is covered by that particularlearning press).
event. During instruction,the descriptionof the An especially important feature of
learning event appears when the user rolls the STAR.Legacyis that it is designed to be flexibly
mouse over the caption. The description is adaptive.Instructionaldesign would be easierif
expected to preview how the activity relates to we could assume that we are aiming at a fixed
accomplishing the challenge. This provides the target. Assuming student and classroomhomo-
dual functionsof encouragingdesignersto think geneity, however, is analogous to the common
about meeting instructionalgoals and helping assumptionthat all the people from a particular
learners recognize the intent and content of a foreigncultureare the same. It is easy, forexam-
learning event as it relates to the largergoal of ple, to read about the Japaneseeducationalsys-
meeting a challenge.The descriptivebox at the tem and assume that they have only one way of
lower left of Figure7 begins to meet these goals, doing things. Anthropologists, however, note
although it does not directly map the learning that culture does not impose homogeneity;
activity back to the challenge of creatingdesign instead, it provides ways for organizing the
principlesfor project-basedlearning. diversity inherentwithin its population (Sato&
McLaughlin,1992). Ratherthan assuming that
diversityis the exception,STAR.Legacyis based
CONCLUSIONS on the assumption that settings, teachers,and
learners have importantvariabilityin terms of
Our discussion was founded on a basic assump- local practices and standards, learning
tion: to optimize the effectivenessof an instruc- resources,styles, and prior knowledge. Conse-
tionaldesign, it is importantto make the design
quently, we want teachers to be able to adapt
conformto importantprinciplesof learningand instructional materials flexibly to fit their cir-
assessment, and to fit the requirements,skills, cumstances.
and resources of teachers, learners, and their
Our discussion of STAR.Legacyfocused pri-
community. We, among others, are working
toward a learning theory for instructional marilyon our uses of it to improve our practices
as college teachers(whichoften include teaching
design that can help teachers adapt or design
instructionfor theirclassrooms.In a coursesuch preservice teachers). The data gathered are
as educational psychology, the theories of informal for the most part, but informative
nonetheless. In particular,people have consis-
Piaget, Bandura,and Vygotsky provide import-
ant psychological insights into learning. How- tently reactedextremelypositively to the ability
of STAR.Legacyto help them visualize the
ever, those theories do not easily translateinto
instructionalpractices.We believe a more "user inquiry process. We also found that students
(many of whom were preservice teachers)and
friendly"theory can help people know enough
about the process of learning to make sound designerswere able to reflecton learningand to
instructional decisions and adaptations of breakfree fromprevious ways of doing things.
instructional materials. We also believe that a Our ultimategoal is to use STAR.Legacyas a
technologicalenvironmentthat makes it easy to platform for furtherresearch-not so much to
implementthis theory is useful in this regard. study STAR.Legacybut to inquire into general
58 Vol. 47, No. 2
ETR&D,

questions relevant to the design of complex tool design. Journalof theLearningSciences,7, 65-107.
instruction. For example, what are the advan- Bereiter,C., & Scardamalia,M. (1985).Cognitive cop-
ing strategiesand the problemof "inert"knowledge.
tages of beginning a unit with a clear overview In S.F. Chipman, J.W. Segal, & R. Glaser (Eds.),
(includinga pretestthathelps students visualize Thinkingandlearningskills:Currentresearchandopen
the kinds of environments in which they must questions(Vol.2, pp. 65-80). Hillsdale,NJ:Erlbaum.
eventually use their knowledge) versus simply Bransford,J.D., Franks,J.J.,Vye, N.J., & Sherwood,
jumping into a challenge? To what extent do R.D.(1989).New approachesto instruction:Because
wisdom can't be told. In S. Vosniadou& A. Ortony
opportunitiesfor students to firstgeneratetheir (Eds.),Similarityandanalogicalreasoning.New York:
own ideas help them better appreciate the
CambridgeUniversityPress.
insights offeredby the expertsfeaturedin Multi- Bransford,J.D., & Nitsch, K.E. (1978). Coming to
ple Perspectives?What is the effect of seeing understandthings we could not previously under-
one's own teacher as a videotaped expert?To stand. In J.F.Kavanagh& W. Strange(Eds.),Speech
what extent might case-, problem-,and project- and languagein the laboratory,schoolandclinic.Cam-
based lessons benefit from a summarizing lec- bridge, MA:MITPress.
Bransford,J.D.,&Schwartz,D.L.(in press). Rethinking
ture designed to help students organize the transfer: A simple proposal with multiple
diverse experiences that were part of their implications. To appear in A. Iran-Nejad& P.D.
inquiry (see Schwartz & Bransford,1998)?Our Pearson (Eds.), Reviewof Researchin Education, 24,
simple study on how to increase students' xxx-xxx. Washington DC: American Educational
ResearchAssociation.
appreciation of their learning shows how
Bransford,J.D., Sherwood, R.D., Hasselbring, T.S.,
STAR.Legacy readily supports research into Kinzer, C.K., & Williams, S.M. (1990). Anchored
these questions and more. Our plan is to con- instruction:Whywe need it and how technologycan
tinue to collaboratewith teachersand designers, help. In D. Nix & R.Spiro(Eds.),Cognition,education,
and to submit our questions to empiricaltest. OW and multi-media:Exploringideasin high technology.
Hillsdale,NJ:Erlbaum.
Bransford,J.D.,Zech, L., Schwartz,D.L., Barron,B.J.,
SeanBrophy,XiaodongLin,and
DanielL.Schwartz, Vye, N., & Cognition and Technology Group at
JohnD. Bransford
arewiththeLearningTechnology Vanderbilt. (in press). Design environments that
CenteratVanderbilt DanielSchwartz
University. inviteand sustainmathematicalthinking.To appear
maybereachedatdan.schwartz@vanderbilt.edu. in P. Cobb (Ed.), Symbolizing,communicating, and
This work was supportedby grant# R305F60090 mathematizing:Perspectiveson discourse,tools, and
from the Departmentof Education.The authors instructionaldesign.Mahwah,NJ:Erlbaum.
thankthe educationalpsychologystudents fortheir Bridges, E.M., & Hallinger, P. (1995). Implementing
contributionsto this paperand Amy Rycefor her problembasedlearningin leadership development. Uni-
editorialtalents. versity of Oregon: ERICClearinghouseof Educa-
tionalManagement.
Brown,A.L.,& Campione,J.C.(1994).Guided discov-
ery in a communityof learners.In K. McGilly(Ed.),
REFERENCES Classroomlessons: Integratingcognitive theoryand
classroompractice(pp. 229-272). Cambridge, MA:
Barron, B.J., Schwartz, D.L., Vye, N.J., Moore, A., MITPress
Petrosino,A., Zech, L., Bransford,J.D.,& Cognition Cobb,P.(1994).Whereis themind?Constructivist
and
and TechnologyGroupat Vanderbilt.(1998).Doing sociocultural perspectives on mathematicaldevel-
with understanding:Lessonsfromresearchon prob- opment. Educational Researcher, 23, 13-20.
lem- and project-basedlearning.Journalof theLearn- Cognition and Technology Group at Vanderbilt.
ing Sciences,7, 271-311 (1992).The Jasperseries as an example of anchored
Barron,B.J., Vye, N.J., Zech, L., Schwartz,D., instruction: Theory, program description, and
Bransford,J.D.,Goldman,S.R.,Pellegrino,J.,Morris, assessment data. EducationalPsychologist,27, 291-
J., Garrison,S., & Kantor,R. (1995).Creatingcon- 315.
texts for community-basedproblem solving: The Cognition and Technology Group at Vanderbilt.
Jasperchallengeseries.In C.N. Hedley, P. Atonacci, (1994). Generative learning and anchored instruc-
& M. Rabinowitz (Eds.), Thinkingand literacy:The tion:Design,researchand implementationissues. In
mindat work.Hillsdale,NJ:Erlbaum. B.P.M.Creemersand G.J.Reezigt (Eds.),New direc-
Barrows,H.S. (1985).Howto designa problem-based cur- tions in educationalresearch:Contributions from an
riculumfor thepreclinical years.New York:Springer. international Groningen:ICO.
perspective.
Bell, B. (1998). Investigateand decide learningenvi- Cognition and Technology Group at Vanderbilt.
ronments:Specializing task models for authoring (1996). Looking at technology in context: A frame-
 FORCOMPLEXLEARNING
SOFTWARE 59

work for understandingtechnologyand education Nunn, C.E. (1996). Discussion in college classrooms:
research.In D.C. Berliner& R.C.Calfee (Eds.),The Triangulatingobservationand survey results.Jour-
handbook of educationalpsychology(pp. 807-840).NY: nalof HigherEducation, 67,'23-26.
Simon& SchusterMacMillan. Penner,D.E., Lehrer,R., & Schauble,L. (1998).From
Cognition and Technology Group at Vanderbilt. physical models to biomechanics:A design-based
(1997). The Jasperproject:Lessonsin curriculum, modeling approach.Journalof the LearningSciences,
instruction,assessment,and professional development. 7, 429-449.
Mahwah,NJ:Erlbaum. Perfetto,G.A., Bransford,J.D., & Franks,J.J.(1983).
Cognition and Technology Group at Vanderbilt.(in Constraintson access in a problemsolving context.
press).Adventuresin anchoredinstruction:Lessons MemoryandCognition,11,24-31.
frombeyond the ivory tower.To appearin R.Glaser
Sato, N.E., & McLaughlin,M W. (1992).Contextmat-
(Ed.), Advancesin instructionalpsychology(Vol. 5),
ters:Teachingin Japanand in the UnitedStates.Phi
Mahwah,NJ:Erlbaum.
DeltaKappan,73,359-366.
Dweck, C.S. (1989). Motivation.In A. Lesgold & R.
Glaser (Eds.), Foundations for a psychologyof educa- Schwartz,D.L.,Biswas,G., Bransford,J.D.,Bhuva,B.,
tion.Hillsdale,NJ:Erlbaum. Balac,T., & Brophy,S. (in press). Computertools
Gardner,H. (1991).Assessmentin context:The alter- that link assessment and instruction:Investigating
native to standardized testing. In B. Gifford& C. what makeselectricityhardto learn.To appearin S.
O'Connor(Eds.),Futureassessments: Lajoie(Ed.),Computers ascognitivetools:Thenextgen-
Changingviews
and instruction(pp. 77-120). eration.Mahwah,NJ:Erlbaum.
of aptitude,achievement,
Boston:Kluwer. Schwartz,D.L.,& Bransford,J.D.(1998).A time fortell-
Gibson,J.J.,& Gibson,E.J.(1957).Perceptuallearning: ing. Cognition& Instruction, 16,475-522.
Differentiationor enrichment.Psychological Review, Schwartz,D.L.,Lin,X.D.,Brophy,S., & Bransford,J.D.
62, 32-51. (in press). Towards the development of flexibly
Gick, M.L.,& Holyoak, K.J.(1983).Schemainduction adaptive instructional design. To appear in C.
and analogicaltransfer.CognitivePsychology,15, 1- Reigeluth(Ed.),Instructional designtheoriesandmod-
38. els, VolumeII.Mahwah,NJ:Erlbaum.
Gragg,C.L. (1940,October19). Wisdomcan'tbe told. Spiro, R.J.,& Jehng, J.C. (1990).Cognitive flexibility
HarvardAlumniBulletin. and hypertext:Theoryand technologyfor the non-
Hmelo,C.E.(1998).Problem-basedlearning:Effectson linear and multidimensionaltraversalof complex
the early acquisitionof cognitive skills in medicine. subjectmatter.In D. Nix and R.J.Spiro(Eds.),Cogni-
Journalof theLearningSciences,7, 173-208. tion,education, andmultimedia:Exploringideasin high
Krajcik,J.,Blumenfeld,P.C.,Marx,R.W.,Bass,K.M.,& technology. Hillsdale,NJ:Erlbaum.
Fredricks,J. (1998).Inquiryin project-basedscience Towler, L., & Broadfoot,P. (1992).Self-assessmentin
classrooms:Initial attempts by middle school stu- the primaryschool. Educational Review,44, 137-151.
dents. Journalof theLearningSciences,7, 313-350.
Vye, N.J., Schwartz, D.L., Bransford,J.D.,Barron,B.,
Kuhn,T.S. (1962).Thestructureof scientficrevolutions.
Zech, L., and Cognition and TechnologyGroup at
Chicago:Universityof ChicagoPress. Vanderbilt.(1998).SMARTenvironmentsthat sup-
Lin,X.D.,& Bielaczyc,K. (1998).Supporting metacogni-
tiveactivitiesin learningaboutcomplexsubjectdomains. port monitoring,reflection,and revision.In Hacker,
Dunlosky,& Graesser(Eds.),Metacognition in educa-
Manuscriptsubmittedforpublication. tionaltheoryandpractice.Mahwah,NJ:Erlbaum.
Michael, A.L., Klee, T., Bransford,J.D., & Warren,S.
(1993).The transitionfromtheoryto therapy:Testof Wertime,R. (1979).Students'problemsand "courage
two instructionalmethods. AppliedCognitivePsy- spans."InJ.Lockheadand J.Clements(Eds.),Cogni-
tive processinstruction.Philadelphia:The Franklin
chology,7, 139-154.
InstitutePress.
Murray,T. (1998).Authoringknowledge-basedtutors:
Tools for content, instructionalstrategy, student Whitehead,A.N. (1929). Theaims of education.New
model, and interfacedesign. Journalof the Learning York:MacMillan.
Sciences,7, 5-64. Williams, S.E. (1992). Putting case-based instruction
Newell, A., &Simon,H. (1972).Humanproblem solving. into context:Examplesfromlegal and medicaledu-
EnglewoodCliffs,NJ:Prentice-Hall. cation.Journalof theLearningSciences,2, 367-427.