International Review of Education

Title Page

Authors: Sharon Nelson-Barber and Zanette Johnson

Title: Raising the standard for testing research-based interventions in Indigenous learning
communities

Corresponding author: Sharon Nelson-Barber

snelson@wested.org
650.452.8267

Author/s mailing address:

Sharon Nelson-Barber Zanette Johnson
WestEd Intrinsic Impact Consulting
17605 Brookhurst Drive P.O. Box 20421
Lake Oswego, OR 97034 Stanford, CA 93409

Abstract

The post-colonial US educational paradigm has a flattening effect on the contours of diverse
languages and community traditions, yet preserving the vibrance of cultural communities is
integral to the rich weave of our collective human heritage. For distinctive populations, like
Indigenous groups, research-based interventions can actively inhibit learner achievement and
suppress cultural vitality. This article presents three cases from Diné (Navajo) public schools that
illustrate how, in a specific cultural milieu, research-based “best practices” sometimes are not for
the best. Each case highlights some of the ways in which “understanding context” is an essential
ingredient paving the way for student success. If we are to optimize the potency of educational
innovations developed for Indigenous learners, interventions must adhere to a higher standard of
assessment practice, that employs local testing and incorporating stakeholder opinions as part of
its strategic design that responds to the particular needs and dispositions of the community’s
unique learners.

Keywords: Indigenous learning communities, research-based educational interventions, self-

determination, comprehensive school reform, assessment strategies

Acknowledgements

Preparation of this paper was supported in part by a grant from the US Department of Education
(S283B120006) to WestEd. The findings and opinions expressed here are those of the authors and do not
necessarily reflect the views of the funding agency. It recounts and elaborates on the earlier article:
Nelson-Barber, S. & Johnson. Z. (2016). Acknowledging the perils of “best practices” in an Indigenous
community, Contemporary Educational Psychology, Special Issue on Indigenous Issues in Education and
Research: Looking forward, 47, 44-50. All references to individuals are anonymous and any similarities
to known individuals are coincidental.
Introduction

The post-colonial US educational paradigm has a flattening effect on the vibrance of

diverse languages and community traditions, yet preserving the unique contours of cultural

communities is integral to the rich weave of our collective human heritage. The lateral

thinking embedded within those diverse cultures is a key asset as we pursue diverse

solution strategies to help Homo sapiens sapiens adapt, and survive during a critical period

in the Anthropocene. Forced assimilation, removal from homes and ancestral lands,

language bans, colonial and military schooling have created a legacy of cultural trauma that

continues in the present day with cultural disrespect (e.g., mining at Oak Flats, and Apache

sacred site), the deliberate transgression of Native lands (e.g., Dakota Access Pipeline oil

spill), and criminal neglect (e.g., the epidemic of rape and murder of Indigenous women

that goes uninvestigated).

The unrelieved, ongoing intensity of this cultural wounding has required a great

deal of energy simply for survival, and shifted focus away from proactive community

efforts. However, it has also motivated many Indigenous communities to organize and

drive toward self-determination (see for instance., Idle No More and Kū Kiaʻī Mauna).

Emotional, mental, cultural and spiritual health indicators improve among peoples

engaging in efforts toward revitalization and cultural perpetuation (Trimble, King,

LaFromboise, BigFoot, & Norman, 2014; LaFromboise, Hoyt, Oliver, & Whitbeck, 2006).

We as researchers need to follow the lead of how these diverse communities feel it is best

to proceed. Our experience with research in Indigenous communities tells us that

Indigenous methodology, which prioritizes Indigenous voices, is the the most suitable

2
approach to take, since the debate and direction is for the commuities to decide. It is a

process that must foreground their wisdom, expertise and perspectives.

We reclaim self-determination every time we speak our languages and practice our

ceremonies; each time we consider Indigenous ways of knowing in contrast to colonial or

dominant-paradigm models and make an intentional choice about which one makes more sense

to apply for the context at hand. This kind of deliberate decision-making from a place of

alignment within an Indigenous knowledge system is an essential element of educational

processes, if we are to create room for biocultural diversity to flourish. Continuing the tradition

of the dominant post-colonial paradigm, as Ravitch (2016) describes, the pressure to create a

national curriculum in the Common Core State Standards Initiative1 suppresses that diversity as

it promotes uniform ideas of excellence and discourages true equity in favor of the appearance of

“equalness.”

Although their freedom was always constrained, there was some level of protection for

remote Indigenous communities in rural, island or arctic locations, where it could be seen that

heritage traditions and language formed a key aspect of community integrity. Local school

administrators exercised their own decision-making to some degree, in some cases informed by

commuity stakeholders. Under the federal reach of No Child Left Behind2 and the Common Core

State Standards there is little room for even that small measure of local control. As Indigenous

educators persist and resist, we are nurturing the treasured human wisdom that is richly

1
The Common Core State Standards Initiative sets forth uniform educational standards across the US intended to
support student preparation for college and careers.

2
Public Law 107-110 enacted the 2001 reauthorization of the Elementary and Secondary Education Act of 1965,
popularly known as the No Child Left Behind Act (NCLB).

3
represented among the Indigenous communities of the world–true wealth that forms a

foundational resource for our survival in the current era of rapid change and climate instability.

Background

The shifting sands created by the Common Core State Standards Initiative in the United

States have been so intense and long lasting that now it is not uncommon for school districts and

teachers to throw up their hands and wonder, “Where can we begin to make a difference?”

New tests, new technologies, and new standards are fueling the ever-changing landscape

for schools, leaving little room for pedagogical growth, and limiting time for reflection and

action research on the part of teachers. Many learning communities are looking to educational

researchers for evidence on promising innovations to direct them, and have rushed to integrate

new research-supported alternatives like computer-adaptive assessments that pledge to improve

student performance on high stakes tests. However, many of the scientifically research-based

innovations available fail to account for a fundamental variable in their effectiveness: context.

Governmental emphasis on the scientific study of educational methods intensified with

the 2001 No Child Left Behind Act, a reauthorization of the Elementary and Secondary

Education Act of 1965 making randomized controlled trials in educational research an urgent

priority. However, this new focus had an adverse impact on Indigenous students (Winstead,

Lawrence, Brantmeier, and Frey 2008). As Medin and Bang (2014) point out, the norms and

values of non-dominant communities do not drive the development of scientific educational

interventions. Instead, tests of interventions are based on established norms belonging to the

White middle-class, thus hindering students who have to accommodate for cultural differences.

Despite colonization and the assimilative practices of the past, which were targeted to extinguish

4
heritage languages, practices and identity (e.g., forced removal of children from their homes to

boarding schools), Indigenous communities maintain their own established norms, values and

lifeways (Brave Heart & DeBruyn, 1998; Nelson-Barber & Trumbull, 2016; McCarty & Lee,

2014).

As noted by Nelson-Barber & Johnson (2016) when mainstream assumptions are

encoded into educational norms, it poses challenges for Indigenous students who must bridge the

gap in areas like: “community values, worldview and epistemology, modes of discourse

(rhetorical style, questioning style, interactional style including nonverbal communication),

participation in cultural practices, environmental knowledge, sense of place, low/high classroom

status, socioeconomic status, identity strength, and teacher/student roles” (p. 45). Indigenous

students should not bear the weight of always having to adapt; educational leaders and

researchers need to take on greater responsibility for creating a wider range of learning processes

and resources that can be shown to be accessible to all students. The newest reauthorization of

the Elementary and Secondary Education Act in 2015, known as the “Every Student Succeeds

Act,” has promised a heightened decision-making focus at the local level, but little has changed

in assessment practice, or for Indigenous learners. Similarly, strategies for guidance and

technical assistance around culturally-inclusive practices have been developed in some states;3

however, with the exception of the state of Montana’s 1999 Indian Education for All legislation,

only a few states have formal expectations for teachers.4 In other words, teachers are expected to

use “recommended” strategies in a context where major state-level policies, like “English Only,”

3
See Arizona’s 2015 Culturally Inclusive Practices: Implementation Guidance for Local Education Agencies and
Alaska’s 2012 Guide to Implementing the Alaska Cultural Standards for Educators.

4
See the 2015 Culture in the Classroom: Indicators and Evidence for Evaluating Culturally Responsive Teaching
Using the Alaska Cultural Standards for Educators.

5
run in direct opposition to those recommendations, as well as to the aims of Indigenous

educators.

The use of scientific methods in educational research is widely accepted and

governmentally-encouraged; however, scholars continue to debate whether its use is even

appropriate (Shavelson and Towne 2002), given the diverse variables within learning contexts,

which cannot be mapped effectively in controlled experiments. The National Center for

Education Research (http://www2.ed.gov/programs/edresearch/index.html) recognizes this and

promotes a more nuanced view in pursuit of its mission to gather “scientific evidence of what

works, for whom, and under what conditions.” We must unmask the convenient illusion that a

single set of “solutions” will ever be found. We take the position that emerging practices and

technologies must be specifically examined as they operate within specific communities across

diverse contexts. This article seeks to illuminate how large-scale educational research can be

misused and misapplied when generalized without local testing in communities with unique

features, needs and contours. The three case examples that follow communicate some of the

effects of misapplied educational research findings, and explain how detrimental outcomes for

learners can result from using innovations that are scientifically proven to be “best practices.”

Methods

This article recounts and extends our recent research (Nelson-Barber and Johnson 2016)

designed to learn what teachers believed to be particularly beneficial for student growth in their

local community contexts. We sought initially to identify the kinds of pedagogical practices that

teachers found effective for Diné (Navajo)5 learners, and our study expanded to include what the

5
The heritage language of the Navajo Nation is known as Diné bizaad. Many members of the nation refer to
themselves and their people as Diné (also written Dineh), meaning “The People.” The word Navajo is commonly

6
teachers emphasized: student achievement increases when teachers employ cultural integration

strategies in lieu of so-called “best practices.”

Over a three-year period we held a series of meetings with practicing Diné educators

from across the Navajo Nation who were earning their Masters’ degrees and Native language

certifications as members of a dual language teacher preparation program. Teachers taught at a

diverse set of community-controlled, contract, charter, public and BIE schools. Throughout the

research period we conducted individual and group interviews and focus groups with 30

educators, visited 18 Diné schools, and observed classroom teaching across communities. The

personal stories we collected included information about teaching practices, successes, past

experiences as students, current professional challenges, and dreams. In these interviews,

teachers characterized “culture teaching” as indistinguishable from and fundamentally integrated

with content area knowledge. Due to the prevalence of this pattern noted early on, we inquired

more broadly about what they felt was important for Navajo learners’ success, incorporating

Indigenous research methodology (Kovach 2010; Tuhiwai Smith 2013), and privileging the

voices of Indigenous participants.

We also applied qualitative analytical methods (Spradley, 1979; Glaser and Strauss 1967;

Bogdan and Biklen 1998) that yielded teacher accounts of the many ways in which schools’

implementation of district and state guidelines through compulsory research-based solutions did

not result in Indigenous learner success. We used a grounded theory approach to develop

the following conceptual framework that models the many aspects that influence the cultural

community where they teach (DeGroat, Edgewater, Hale, Johnson, Lockard & Nelson-Barber

2015). (See Figure 1)

used to describe the Diné in the English language, as it was common during the period of Spanish colonization. The
terms are used interchangeably in this paper, though speakers of Diné bizaad often have strong preferences
depending on context.

7
 Figure 1

The conceptions of teaching and learning that emerged from the perspectives of these

teachers differ substantially from current “conventional” or “dominant” models. The strategies

Diné teachers were using successfully to boost indigenous student achievement were very

specific and community-culture oriented; yet, many expressed that there were eminent

constraints that kept them from enacting (and developing) the strategies they knew to be

beneficial for student learning.

Teachers identified myriad ways to shape the relationships, communication pathways,

institutional structures, and curricular conditions inside the system so that they could fully

implement promising practices that make use of local wisdom in content areas to promote

8
content excellence. In the diagram, these key relationships affecting teaching and learning are

represented in yellow. The dark blue entries denote areas of opportunity where needs can be met

by providing better access to information, research and technical assistance for teachers and

communities. The green circles represent desired features that would characterize a system that

supports optimal implementation of culture- and language-based teaching. It is also important to

draw attention to the structure of the diagram as a whole, that represents the eight-sided hogan, a

traditional Navajo dwelling.

Findings

The Indigenous teacher interviews revealed several areas where concerns converged; the

ineffectiveness of “best practices” in Navajo communities was a key issue. These three cases

identify salient patterns in ways that research-based strategies and “best practices” fail to

promote Indigenous learner achievement at the individual, teacher and system levels.

Case 1 — System Level: Family Involvement as a “Best Practice”

As reported in our previous work (Nelson-Barber and Johnson 2016), the No Child Left

Behind Act of 2001 emphasized the need for schools to involve families as partners in support of

children’s education. We reiterate research that attests to the importance of family involvement

as a practice proven to further learner outcomes (Nelson-Barber and Johnson 2016; Weiss,

Lopez, and Rosenberg 2010). However, Navajo teachers find that it is difficult to carry out

family involvement activities in their communities when issues such as colonial literacy and

assimilative schooling, differing cultural values regarding success, diminishing intergenerational

transmission of knowledge and fluency in the heritage language continuously intrude.

9
 One widely-accepted approach to promote family involvement asks siblings and parents

to engage in shared reading activities each day. Unfortunately, such exercises are an impediment

to some families because of differences in language use patterns between Diné children, parents

and their grandparents; patterns of time at work affect reading at home, and books may also be in

short supply. Also cultural methods of child-rearing focus on behaviors, interactions and

conversations that may differ significantly from approaches to literacy that are, at times in direct

conflict with those found in conventional schooling. For example, community members may be

unaccustomed to “active reading” approaches such as making predictions or questioning. Due to

the experience-based nature of Diné communication, it is often uncomfortable for older adults to

discuss content with which they don’t directly engage.

Although many teachers are fluent and want to use heritage language and specialized

interactive protocols in the classroom, finding ways to authentically integrate such local

knowledge can be deterred when schools ask them to model the stories, structures and skills of

the dominant culture (Ladson-Billings 2014). As teachers noted in our 2016 study (Nelson-

Barber & Johnson 2016), community members use discourse styles in both Diné and English that

are specific to their locale and do not necessarily align with the patterns presumed by “best

practices.” Adverse consequences can occur when assumptions like these go unchallenged.

One teacher described an element of her Navajo family’s spiritual tradition that has been

changing— waking up in the dawn hours and running Eastward, toward the sun rising, to pray.

She explains that although this was a regular event for her family each morning, today’s parents

and children rarely engage in this joint activity. As more and more elders pass on to the next

world, fewer fully acculturated community members are around to model many of the cultural

protocols. The needed focus on heritage activities that are known to build Diné language and

10
cultural identity is overtaken by pressure to use research-based practices. This teacher is very

concerned that defining facets of Navajo life are becoming less visible in her community and

children are learning fewer cultural essentials. Given these arresting circumstances it becomes

increasingly difficult to bring community knowledge into the classroom.

Historically, children were engaged early in life through practical heritage practices like

gathering plants for medicine, livestock-herding or carrying water. Over time these activities

provided a series of prominent spaces for youth to acquire the Navajo worldview, including

sense of place and relationships that are fundamental to Diné identity. Heritage language learning

was an inseparable, foundational aspect of cultural and spiritual growth, a process led by

grandparents through a process of intergenerational knowledge transmission. However, largely

due to the cultural and historical trauma resulting from boarding school experience, many in the

older generations have had to rediscover identity along with the enduring contributions of their

people (Brave Heart and DeBruyn 1998). With the goal of easing ongoing trauma and paving the

way for greater success in school and in life, many opted to speak only English to their children.

This particular effort has impeded the cycle of intergenerational learning so central to Navajo

life. Today it continues to disrupt the normal socialization of many children as competent

participants in Navajo society and curbs their abilities to communicate with elder family

members who speak only Diné bizaad (House 2005).

One teacher suggests that ‘family involvement’ models might accommodate student

learning in and out of school. Use of different types of local, community-determined activities

can take advantage of the locally- and globally-valued knowledges that exist in Diné

communities. Teachers interviewed in this study call for malleable formats, schedules and

learning destinations to improve learner performance and sustain quality Navajo cultural life.

11
Case 2: Teacher Level: Fidelity of Implementation as a “Best Practice”

In ongoing efforts to improve students’ academic performance, many school districts

across the US are seeking curricular and programmatic innovations that are proven to boost

learning. To fully execute these intentions and raise test scores in ways similar to those achieved

in research trials, teachers are being asked to follow particular fidelity of implementation

procedures. Though this is a sound area of focus for many learning communities, Navajo

educators have particular ideas about conditions that can inhibit learner success. They reference

contextual issues and concerns that must be addressed alongside any curricular or program

reforms.

A prevailing area of need is finding ways to lessen the cultural trauma that carries over

into all aspects of a child’s life, including the classroom (Brave Heart, Chase, Elkins and

Altschul 2011; Benham & Heck 1998). Significant historical loss—of culture, language, identity,

traditions—continuously impacts daily relationships, a child’s knowledge base, understanding of

one’s place, filial relations, to name a few. These factors in turn influence how any program or

curriculum or test is received and whether or not students will benefit. Do recommended

programmes account for cultural differences? Community-specific needs? Dialectal protocols?

Social hierarchies?

The enduring, painful resonance of historically-rooted cultural trauma is something that

all educators in this study recognized; their primary recommendation was to explicitly

acknowledge cultural trauma as a barrier to learning, and address it accordingly. Rather than

maintaining fidelity to externally-devised curricula or programmes as best practice, cultural

12
integration was the critical action teachers identified as the best mechanism to curtail the impact

of cultural trauma on learning.

According to these Diné teachers, maintaining fidelity to programs in Navajo

communities creates a loss for learners who aren’t able to benefit from teachers’ cultural

expertise. Informed teachers are able to meld local traditions of Indigenous knowledge, ecology

and heritage language with school-based “best practices” in ways that are context-sensitive—

considering factors like the tribal heterogeneity or homogeneity of the group, the cultural identity

of the authors of texts taught, and the students’ interpretive frame for the content etc. to enhance

student experience (Kana‘iaupuni & Kawai‘ae‘a 2008).

Hawaiʻi teachers in Kanaʻiaupuni and Ledward’s (2013) study of teacher practices in

Native Hawaiian culture-based schools did just this. They combined conventional strategies of

best practice with context-adaptive, culture-based approaches, providing higher quality

instruction that better resonated with student understandings. Unlike these educators the Diné

teachers, whose opinions appear here, had little to no license to make individual instructional

decisions related to culture or for infusing heritage language in their lessons.

By contrast, after making a strong case for bringing place-based learning to his students,

middle school-level Diné educator, Brendan, was able to undertake his brand of culture teaching.

One of his goals was to bring alive the scientific knowledge that is implicit in many of the

heritage activities his students regularly engage in, ranging from sheep herding to weather

prediction. He also wanted to help them make connections between important rites and rituals

and the cultural responsibilities that accompany seasonal changes. Essential to this approach

were trips to cultural sites in the community. Brendan understood that academic content becomes

clearer to students when it can be linked to real-life experiences and cultural understandings.

13
This kind of skill development contributes to the growth of knowledgeable contributors to a

harmonious Diné society.

Such an approach is one of many strategies that can be used and adapted at the system

level to promote conceptual development and content learning in the formal education of youth,

and it is particularly effective for Diné learners in spite of the fact that it does not resemble

content delivery in other communities.

A different set of issues arise for another Diné culture teacher. On the surface,

elementary-level educator, Burke’s, classroom seems ideal. He understands the importance of

exposing his grade 4 -10 students to their heritage language and culture and, with a hooghan on

campus his classroom, he delivers culture-focused material. However, culture-teaching is so

much more than sharing words, artifacts and history. Burke also needs the pedagogical and

structural latitude to organize his teaching in culturally-aligned ways. For example, conventional

structures like class period length do not allow the time needed to accommodate various

culturally responsive practices--opening protocols, the iterative patterns of Navajo language or

the distinctive pedagogy that is culturally-sustaining. Burke understands the irony. Although the

hooghan is characterized as a healing place, he is backed into a corner, following the rules of

others and unable to fully execute what he knows can best support learning outcomes for his

students.

This case illustrates reasons why fidelity of implementation cannot stand as an

unexamined goal for Indigenous learning communities—it must be interrogated in relationship to

the community contours that address the needs of learners and the assets that Indigenous teachers

bring with them as professionals.

14
Case 3: Individual Learner Level: Adoption of Comprehensive School Reform Programs as a
“Best Practice”

Although many US school districts favor comprehensive school reform programs (CSR),

research has yet to establish the impacts of these methods on student achievement (Borman,

Hewes, Overman and Brown 2002; Vernez, Karam, Mariano and DeMartini 2006). Still, some

educators presume that the tightly-regulated instructional approaches associated with these

programs are appropriate for diverse populations (Aladjem and Borman 2006; Ede 2006). As an

example Ladson-Billings (2014) points to teacher preparation for high poverty schools that often

centers on structure rather than on more innovative instructional and curricular processes.

Datnow, Borman, Stringfield, Overman and Castellano (2003) note that “in some cases,

educators’ beliefs about student ability, race, and language served as constraints to reform,”

adding stereotyping as an additional factor alongside fidelity of implementation that contributes

to the variable results of these programs for diverse learners.

Case 3 contrasts two episodes of Diné teaching in high-performing upper elementary

classrooms—one that illustrates instruction that is culturally-situated and learner-directed and

the other that uses conventional CSR methodology.

Structure is abundant in Briana’s CSR classroom. Entering the room one feels a strategic

sense of “order”—from the conventional arrangement of student desks in rows to the level of

control exhibited by the teacher. As youth sit quietly at their desks, talk is in English and strictly

regulated with students kept on task via teacher directives and queries. Briana uses verbal, vocal

and gestural cues to alert students to key lesson content. There appears to be little need for

behavior management as students seem to know and abide by classroom rules. For the most part,

the noise level in the room is relatively low; there are very few extraneous student utterances—

15
something noticeable for seven- and eight-year olds. Bulletin boards and walls around the room

are predictably filled with programme materials associated with the content being taught.

The atmosphere of close management, relative quiet, and passive learning in this

classroom is striking. When exposed to the style of instruction modeled here students learn that

the teacher will distinguish for them the most meaningful information to take away from each

lesson. Testing reveals that Briana’s students are high performers.

Not all teachers take the inflexible approach that Briana’s school does. In the second

classroom the activity level is far more dynamic. Rather than positioned for participation

individually at their desks, Ashley’s fourth grade students collaborate in small groups, then

complete work collectively or individually given their particular needs. When Ashley addresses

the group as a whole, she provides visual stimuli, interjects questions and commentary, and uses

non-verbal communication and visual cues as directives. Students also have license to use their

heritage language at all times and across all lessons. Through this complex orchestration of

meaning making, students are learning to think critically, to self-regulate, to communicate

effectively along with other 21st century learning skills. In this classroom, students are coming to

understand that their learning goes well beyond locating information that the teacher identifies as

important. They are learning to discriminate essential knowledge for themselves. Ashley’s

students’ test results also reveal high academic proficiency.

Although students in both classrooms show high levels of growth over the year, Briana

reports several observations. She notes that her students score well on tests administered soon

after she has presented her material. Her scripted, iterative teaching methods boost her students’

understandings in the short-term; however, these youth do not perform well when tests ask them

to apply what they have learned long after the material has been experienced. Students in both

16
Briana’s and Ashley’s classrooms score well on tests and thus appear to perform comparably;

but, the skills being developed in each case are quite different. Ashley’s students are learning

skills that extend far beyond the particular content of any lesson—skills that will last over time

and in multiple contexts. In fact, the judgement and self-regulation learned in Ashley’s class are

essential understandings needed for populations who continue to endure consequences of

colonization and historical cultural trauma described earlier.

There are more reasons why Ashley’s approach supports her students’ academic success.

She brings to her classroom instructional participation structures that are widely-used in

Indigenous homes and communities (Lee, 2016; Nelson-Barber 2017). These repertoires of

practice utilize an expert-apprentice strategy in which adults serve as models and facilitators,

guiding children to learn by observing and doing with an emphasis on collective action,

collaboration and cooperation. Concepts are taught in meaninful contexts and serve authentic

purposes. Students are encouraged to take responsibility for their own learning and are allowed

to make choices about when and how to display their learning. Lessons can be student-initiated

but the expert teacher will always be there to direct.

Although CSR has been widely adopted as a “best practice,” Briana’s methods push her

Indigenous students farther way from the culturally-valued practices of their community.

Students learn that the personal agency and discernment so important at home are instead a

distraction or liability in their teacher-regulated class. Test scores may improve, but the

distintinctive norms and values of Indigenous communities are not coherent with the processes

of CSR.

17
Discussion and Conclusion

The Indigenous educators who participated in this study detailed their ongoing efforts to

improve educational outcomes for Native students. The processes they use to accomplish this

necessarily center on ensuring adherence to local values that respect and sustain Indigenous

ways of knowing. Despite the historical impacts of colonial schooling that continue to

undermine strong use of heritage language and cultural practices, tribes and Nations, like the

Navajo, are recovering and reinstating their own unique learning processes, leading not only to

improved student performance, but also developing young people who are confident in their

identity and much healthier emotionally (See e.g., Kanaʻiaupuni, Ledward, & Jensen 2010;

LaFromboise, 2006; Lipka, Hogan,Webster,Yanez, Adams, Clark, & Lacy 2005).

It is imperative that school districts conduct pilot testing using diverse assessment

strategies, and utilize heuristics based on local values and needs, to determine whether a novel

research-based intervention is appropriate for their particular communities of learners before

formally adopting new practices at scale. In an earlier account (Nelson-Barber & Johnson 2016),

we recommended a number of areas that districts and school leaders should consider (p.48),

which we reiterate here:

 Build capacity in action research and the recursive use of data

 Use data to map relevant relationships and model variables salient in past

initiatives

 Anticipate impacts of the proposed intervention by thinking through detailed

culture-focused questions

 Envision and test adaptations

 Gather baseline snapshots of performance (for later comparison)

  Develop a heuristic set of questions around community priorities that can guide

and broaden reflection when considering adoption of novel or external practices

The above list is far from comprehensive; there are many angles of approach to the challenge, all

of which begin with reflection and focus on the key features of the context in which an

intervention is being considered for use.

As seen in this study, even well-documented effective educational interventions can have

negative effects when applied across diverse communities in a monolithic fashion. The leaders of

educational programs in Indigenous communities must acknowledge the reality that “the specific

propositions derived from general theories of learning can be viewed only as hypotheses. They

may be true, but it is quite possible that they are false. Until they have been subjected to

empirical test they must be viewed as unproved” (Averch, Carroll, Donaldson, Kiesling and

Pincus 1972, p. 21).

Indigenous communities are influenced by historical, colonial and modern-day factors in

a way that makes them dissimilar from one another, and from communities within the dominant

culture of the United States. Given the understandings set forth in this paper, we can no longer

consider it “reasonable” to propose that the same best practices will function similarly in settings

that vary so much. Action steps like the following will be needed if LEAs, SEAs and educational

systems are to meet a higher standard for assessing the appropriateness of practices for specific

communities:

 Name the assumptions that the best practice or intervention holds as invisible
truths; making those assumptions visible is a first step in generating adaptations
and/or viable alternatives that are based on what is real within the community
context

 Focus on the data rather than on expected results; be open to surprises

19
  Engage community members at the early stages of research planning, to better
understand key variables, local interpretations, and desired performances

 Identify key indicators that matter within the context, as indicated in context-
sensitive approaches like developmental evaluation and situation awareness
(Endsley, 1995, 2000; Endsley, Bolté and Jones 2003; Patton 1994, 2011)

 Employ member checking as an ongoing strategy to find out how community

members feel about the intervention’s impact—rather than waiting for results at
the end of the pilot

 Share the results of your pilot in ways that can be heard and understood by the
community (e.g., a visual metaphor from the culture; radio campaign; posters; a
family night with food, games and feedback opportunities)

 Send clear, simple messages about what you tried, what worked/did not work and
why you think so; share your experience plainly, so that others (including
community members and other researchers) can learn, reframe your interpretation
if needed, and/or test for themselves

If we make a professional habit of testing out interventions with local adaptations and

data-gathering on effectiveness, we may find that some of the most functional solutions are

utterly obvious if the context is well understood. We may also find that other best practices that

work extremely well are actually counterintuitive by common logic.

The experience of local testing of interventions may also present us with challenges to

accepted research methodology, if, for example, it becomes clear that certain solutions have a

timeline that goes beyond the school year and the typical window for assessment (in a case like

that, decisions affecting research design and/or changes to the intervention would need to be

made based on preliminary analysis of a trend, rather than waiting for negative outcomes to play

out in full).

When researchers become aware of the genuine differences between diverse,

Indigenous communities, they are able to generate research designs that work around

20
some of the background assumptions of large-scale scientifically-based research, and test

the “hypotheses” that those offer. We must enhance the practice of testing and assessing

the function of research-based “best practice” interventions in Indigenous communities in

order to minimize negative outcomes like those described in this study, and to determine

the most effective practices for a given locality. This research-based approach to

customizing and/or replacing best practices requires open-mindedness, resource

allocation, and time to identify appropriate adaptations and alternatives to the research-

based practices.

The outcomes we can justly anticipate include the tangible improvements in

pedagogy, assessment and achievement for diverse learners of all types that become

possible when researchers use responsive, context-sensitive methods for measuring what

students know and are able to do. Policy changes that are broad enough to embrace a range

of worldviews, lived experiences, and locally-valued knowledge can support this type of

transformative result born from local experimentation. Such an investment in the

perpetuation of Indigenous knowledge, which is part of the collective human heritage, can

have broad scale impact on the overall viability of our species, and form a far more resilient

and diverse foundation of knowledge for surviving and thriving throughout the biosphere

in our era of rapid change.

21
References

Aladjem, D. & Borman, K. (Eds.). (2006) Comprehensive school reform. Washington, DC:
Urban Institute Press.

Averch, H. A., Carroll, S. J., Donaldson, T. S., Kiesling, H. J., & Pincus, J. A. (1972). How
effective is schooling? A critical review of research. RAND Corporation.
http://www.rand.org/pubs/reports/R0956.html

Benham, M. K., & Heck, R. (1998). Culture and educational policy in Hawaiʻi: The silencing
of native voices. Mahwah, NJ: Lawrence Erlbaum Associates.

Bogdan, R. & Biklen, S. (1998). Qualitative research for education: An introduction to theory
and methods (3rd ed.). Needham Heights, MA: Allyn & Bacon.

Borman, G. D., Hewes, G. M., Overman, L. T., & Brown, S. (2003). Comprehensive school
reform and achievement: A meta-analysis. Review of educational research, 73(2), 125-
230.

Brave Heart, M., Chase, J., Elkins, J., & Altschul, D. B. (2011). Historical trauma among
indigenous peoples of the Americas: Concepts, research, and clinical considerations.
Journal of Psychoactive Drugs, 43(4), 282–290. doi:10 .1080/02791072.2011.628913

Brave Heart, M. & DeBruyn, L. (1998). The American Indian holocaust: Healing historical
unresolved grief, American Indian and Alaska Native Mental Health Research, (8)2, 60-
82.

Datnow, A., Borman, G. D., Stringfield, S., Overman, L. T., & Castellano, M. (2003).
Comprehensive school reform in culturally and linguistically diverse contexts:
Implementation and outcomes from a four-year study. Educational Evaluation and
Policy Analysis, 25(2), 143-170.

DeGroat, J., Edgewater, V., Hale, V., Johnson, Z., Lockard, L., & Nelson-Barber, S. (2015).
Creating an ecology of success for American Indian learner success. Symposium
presented at the annual meeting of the National Indian Education Association, Portland,
OR.

Ede, A. (2006). Scripted curriculum: Is it a prescription for success? Childhood Education,

83(1), 29-32.

22
Endsley, M. R. (1995). Toward a theory of situation awareness in dynamic systems. Human
Factors: The Journal of the Human Factors and Ergonomics Society, 37(1), 32-64.

Endsley, M. R. (2000). Theoretical underpinnings of situation awareness: A critical review. In

M. R. Endsley and D. J. Garland (Eds.) Situation awareness analysis and measurement
(pp. 3-32). Mahwah, NJ: Lawrence Erlbaum Associates.

Endsley, M. R., Bolté, B. & Jones, D.G. (2003). Designing for situation awareness: An
approach to user-centered design. New York, NY: Taylor & Francis US.

Glaser, B., & Strauss, A. (1967). The discovery of grounded theory: Strategies for qualitative
research. New York: Routledge.

House, D. (2005). Language shift among the Navajos: Identity politics and cultural continuity.
Tucson: U AZ Press.

Kana’iaupuni, S. M, & Ledward, B. C. (2013). Hoʻopilina: The call for cultural relevance in
education. Hūlili, 9, 154-204.

Kanaʻiaupuni, S. M., Ledward, B. & Jensen, U. (2010). Culture-Based Education and its
relationship to student outcomes. Honolulu, HI: Kamehameha Schools, Research &
Evaluation Division.

Kana‘iaupuni, S. M., & Kawai‘ae‘a, K. K. (2008). E Lauhoe Mai Na Wa‘a: Toward a Hawaiian
Indigenous Education Teaching Framework. Hūlili: Multidisciplinary Research on
Hawaiian Well-Being, 5, 67-90.

Kovach, M. (2010). Indigenous methodologies: Characteristics, conversations and contexts.

Toronto: University of Toronto Press.

Ladson-Billings, G. (2014). The pedagogy of poverty: The big lies about poor children. In P.
Gorski & K. Zenkov (Eds.). The big lies of school reform: Finding better solutions for
the future of public education. (pp. 7-16). New York: Routledge.

LaFromboise, T. (September, 2006). Teen suicide prevention., The Prevention Researcher,

13(3), 16-18.

LaFromboise, T. D., Hoyt, D. R., Oliver, L., and Whitbeck, L. B. (2006). Family, community,
and school influences on resilience among American Indian adolescents in the upper
midwest. Sociology Department, Faculty Publications. 26.
http://digitalcommons.unl.edu/sociologyfacpub/26

Lee, T. (2016). The home-school-community interface in language revitalization in the USA and
Canada, in S.M. Coronel-Molina & T.L. McCarty (Eds.), Indigenous language
revitalization in the Americas (pp. 99-115). New York, NY: Routledge.

23
Lipka, J., Hogan, M., Webster, J, Yanez, E., Adams, B., Clark, S., & Lacy, D. (2005). Math in a
cultural context: Two case studies of a successful culturally based math project,
Anthropology & Education Quarterly, 36(4), 367-385.

McCarty, T., & Lee, T. (2014). Cricical culturally sustaining/revitalizing pedagogy and
Indigeous education sovereignty, Harvard Educational Review, 84(1), 101-124.

Medin, D., & Bang, M. (2014). Who's asking? Native science, western science, and science
education. Cambridge, MA: MIT Press.

Nelson-Barber, S. (2017). The “perfect storm” in Indigenous education: Stories about context,
culture and community knowledge. Honoring all Teachers. Proceedings of the 7th
American Indian/Indigenous Teacher Education Conference, June, Flagstaff, AZ.

Nelson-Barber, S., & Johnson, Z. (2016). Acknowledging the perils of “best practices” in an
Indigenous community, Contemporary Educational Psychology, Special Issue on
Indigenous Issues in Education and Research: Looking forward, 47, 44-50.

Nelson-Barber, S., & Trumbull, E. (2016). The common core initiative, educational outcomes
and American Indian/Alaska Native students: Observations and recommendations.
Knowledge Brief. San Francisco, CA: WestEd.

Patton, M. Q. (2011). Developmental evaluation: Applying complexity concepts to enhance

innovation and use. New York, NY: Guilford Press.

Patton, M. Q. (1994). Developmental evaluation. Evaluation Practice. 15(3), 311-319.

Ravitch, D. (2016). The death and life of the great American school system: How testing and
choice are undermining education. New York, NY: Basic Books.

Shavelson, R. & Towne, L. (Eds.). (2002). Scientific research in education. Washington, DC:
National Academy Press.

Spradley, J. S. (1979). The ethnographic interview. New York, NY: Holt, Rinehart, and

Winston.

Trimble, J., King, J., LaFromboise, T., BigFoot, D., & Norman, D. (2014). American Indian and
Alaska Native mental health perspectives. In R. Parekh (Ed.), The Massachusetts General
Hospital Textbook on Diversity and Cultural Sensitivity in Mental Health (pp.119-139).
New York, NY: Springer.

Tuhiwai Smith, L. (2013). Decolonizing methodologies: Research and Indigenous peoples. New
York: Zed Books Ltd.

24
Vernez G, Karam R, Mariano LT, DeMartini C (2006) Evaluating comprehensive school reform
models at scale: Focus on implementation. RAND Corporation. Abstract retrieved from
http://www.rand.org/pubs/monographs/MG546.html (not included in the publication

Weiss, H. B., Lopez, M. E., & Rosenberg, H. (2010). Beyond random acts: Family, school, and
community engagement as an integral part of education reform. National Policy Forum
for Family, School, & Community Engagement. Cambridge, MA: Harvard Family
Research Project.

Winstead, T., Lawrence, A., Brantmeier, E. J., & Frey, C. J. (2008). Language, sovereignty,
cultural contestation, and American Indian schools: No Child Left Behind and a Navajo
test case. Journal of American Indian Education, 47, 46-64.

Montana's accreditation standards provide guidelines for the implementation of Indian

Education for All. www.opi.mt.gov/IndianEd/EdforAll.html 

25