Theory of Change

Global Mental Health has developed interventions that strive to work across great difference—variously conceptualized as cultural, socio-economic, geographic, or pertaining to the characteristics of health systems. This article discusses how the evaluation framework Theory of Change (ToC) facilitates the production of ‘global’ knowledge across such differences. Drawing on 14 months of multi-sited fieldwork among Global Mental Health actors in Europe, North America and South Africa, it traces the differential use of ToC in GMH interventions. While much critical scholarship of Global Health metrics holds that techniques of quantification rely on universals that necessarily betray the “real world”, ToC unsettles these critiques. It comes into view as an epistemic and relational device that produces ‘contingent universals’—concepts that are true and measurable until they stop working in the field, or until the parameters of ‘what works’ shift to a new iteration. As such, Theory of Change produces actionable—rather than true—knowledge attuned to open-ended change, both desirable (impact) and unforeseen (adaptation). Its effects, however, are ambiguous. ToC presents us with a horizoning technique that enables what I call “virtuous failure” within the evidence-based paradigm. It may equally harbor the potential to disrupt distinctions such as bricolage (tinkering) and design (planning) and their respective politics, as it may tie neatly into audit cultures, depending on its use. The article analyzes the novel stakes of reflexive evaluation techniques and calls on anthropology and critical Global Health for renewed empirical engagement.

Evidence-based medicine (EBM), with its evidence hierarchies and emphasis on RCTs, meta-analyses and systematic reviews, sets the model for evidence-based policy almost everywhere, policing no exception. But how closely should policing follow this model? We argue that RCTs can tell you little about what you need to know for real-world practice: will this policy work where and when you implement it? Defending that it will do so takes good theory. For RCTs to play a role in theory development, they must be set into a larger body of knowledge, including local knowledge about the sites of implementation. Unfortunately the standard EBM model generally ignores the other kinds of knowledge needed. An alternative model for evidence-based policing, similar to that of the new movement for 'EBM+' and immanent in the practice of realist synthesis, focuses on the arguments that proposed policing policies will work where and when they are implemented and looks for the evidence needed to support those arguments.

Background: Cardiovascular disease (CVD) is the leading cause of morbidity and mortality globally. Digital behaviour change interventions (dBCIs) may optimise prevention of CVDs due to their ability to overcome typical barriers to cardiac rehabilitation (CR). Despite their potential, most dBCIs report limited adherence and small effect sizes. Research has suggested dBCIs may be more effective if grounded in appropriate theory. Objective: To review evidence supporting the current use of behaviour change theories and models guiding dBCI development in CVD groups. Methods: A realist review method was chosen as it is a theory-driven approach that seeks to explore complex mechanisms underlying program effectiveness. A scoping literature search in PubMed and Scopus within ten years (2006 to 2016) identified 5 reviews of dBCI strategies for people living with CVD. Narrative content analysis methods were applied to extract reported correlations between behaviour change models and health behaviour changes related to diet, exercise, alcohol and drug use, medication adherence, tracking physiological symptoms, and improved psychosocial management of stress. Results: Findings showed limited use of behaviour change theory in dBCIs for CVD. In the few apps that drew on theory, most used Bandura's Social Cognitive Theory (SCT) and associated concept of self-efficacy (i.e., one's self-confidence to execute requisite actions to satisfy situational demands). However, scant reporting of dose-dependent links between theories and health outcomes prevented generalisations to be made. Further, ubiquitous use of SCT-based models was problematic, due to growing evidence that behaviour change is part of a nonlinear complex adaptive system that defies prediction or control.

Theories of change (ToCs) are development tools that describe how a certain set up of conditions and actions leads to producing specific results. They are used to design, monitor, evaluate, and scale interventions and are increasingly required by both donor and research agencies. In this paper, we explore the links between theories of change and actual social scientific theories about change, which have not been explored in academic literature. We make three essential contributions. First, we explain how the rising interest in evidence-based policy fuelled the evolution of development management tools, including Logit Models, Log Frames and theories of change. Second, we argue that the narrow understanding of ‘research evidence’ resulted in scientific contributions being limited to conducting experimental evaluations (RCTs) meant to test and validate the projects’ theories of change. This we criticise as both methodologically unsound and epistemologically limiting. Third, we introduce alternative sources of ‘evidence’ that credit the social science theories and methods and engage meaningfully with local stakeholders (target communities and implementing staff).

Evaluation is essential to understand whether and how policies and other interventions work, why they sometimes fail, and whether they represent a good use of resources. Evaluability assessment (EA) is a means of collaboratively planning and designing evaluations, seeking to ensure they generate relevant and robust evidence that supports decision-making and contributes to the wider evidence base. This article reports on the context, the process undertaken and evidence from participants in an EA facilitated with public service workers involved in implementing a complex, area-based community improvement initiative. This is a novel context in which to conduct an EA. We show how the process allows practitioners at all levels to identify activities for evaluation and co-produce the theory of change developed through the EA. This enables evaluation recommendations to be developed that are relevant to the implementation of the programme, and which take account of available data and resourc...

Background: An Aboriginal-developed empowerment and social and emotional wellbeing program, known as Family Wellbeing (FWB), has been found to strengthen the protective factors that help Indigenous Australians to deal with the legacy of colonisation and intergenerational trauma. This article reviews the research that has accompanied the implementation of the program, over a 23 year period. The aim is to assess the long-term impact of FWB research and identify the key enablers of research impact and the limitations of the impact assessment exercise. This will inform more comprehensive monitoring of research impact into the future. Methods: To assess impact, the study took an implementation science approach, incorporating theory of change and service utilisation frameworks, to create a logic model underpinned by Indigenous research principles. A research impact narrative was developed based on mixed methods analysis of publicly available data on: 1) FWB program participation; 2) resea...

Mother-tongue or first language (L1)-based multilingual education programs are necessarily complex and may require a more nonlinear approach to program design. These programs operate within and act upon a range of psycholinguistic, sociolinguistic, and sociopolitical issues that include language structure and literacy assessment, language policy and politics, and cultural and social behavior change linked to literacy expansion. The broad use of one-size-fits-all outcomes-based design approaches for L1-based multilingual education programs often result in designs that are retrofitted to new country settings and ill-suited to the context in which they are implemented. This paper looks at some of the many features that can be used to inform the development of L1-based multilingual education in the context of early literacy programming. Specifically, it examines the use of alternative approaches in the development of flexible theory of change design that integrate early literacy and L1-...

Mother-tongue or first language (L1)-based multilingual education programs are necessarily complex and may require a more nonlinear approach to program design. These programs operate within and act upon a range of psycholinguistic, sociolinguistic, and sociopolitical issues that include language structure and literacy assessment, language policy and politics, and cultural and social behavior change linked to literacy expansion. The broad use of one-size-fits-all outcomes-based design approaches for L1-based multilingual education programs often result in designs that are retrofitted to new country settings and ill-suited to the context in which they are implemented. This paper looks at some of the many features that can be used to inform the development of L1-based multilingual education in the context of early literacy programming. Specifically, it examines the use of alternative approaches in the development of flexible theory of change design that integrate early literacy and L1-based multilingual education program design frameworks to more suitably address the sociolinguistic, sociopolitical, and psycholinguistic assumptions underpinning multilingual education approaches.

A theory of change is a purposeful model of how an initiative—such as a policy, a strategy, a program, or a project—contributes through a chain of early and intermediate outcomes to the intended result. Theories of change help navigate the complexity of social change.

Social programs are based on explicit or implicit assumptions of how and why a certain intervention will work. Recently, these assumptions have captured the attention of academics and practitioners alike, sparking sudden interest in the so called theories of change (ToCh). By illustrating how each intermediate outcome links to another in a logical and/or chronological flow, ToCh are explanations of how certain groups of stakeholders expect to reach a commonly understood long-term goal. In this research, we take the comprehensive case study of a recent ecosystem services project in Huamantanga, Peru, as a strategic site to illustrate the diversity of perceptions and assumptions around the project’s theory (ies) of change held by a heterogeneous group of stakeholders. Founded by the British research consortium, Ecosystem Services for Poverty Alleviation (ESPA), the intervention built upon launching a community-led environmental virtual observatory (EVO): an open technology platform fo...

L’Organizzazione delle Nazioni Unite per l’Alimentazione e l’Agricoltura (FAO) è un’organizzazione intergovernativa costituita nel 1945 che conta 194 Stati membri, oltre all’Unione Europea come “member organization" e le Isole Faroe e Tokelau in qualità di “membri associati”. Opera a livello globale principalmente per ridurre la fame, l’insicurezza alimentare e la malnutrizione.
Il budget totale della FAO per il 2016-2017 è di $2.6 miliardi, di cui quasi il 40% proviene da contributi dei Paesi membri. Per massimizzare l’efficacia nell’uso delle risorse e garantire trasparenza e accountability, nel 2004 la FAO ha lanciato un processo di riforma che l’ha portata a diventare un’organizzazione results-based.
Anche se la ToC non è mai stata sistematicamente associata a organizzazioni internazionali, è evidente il link che la collega ai recenti processi di riforma di queste istituzioni.

This work takes a critical look at the concept of anti-generic learning for rebellious lawyering propounded by Professor Mizanur Rahman. The author analyses the concept and explores why the theory, albeit viable and potential, has not been able to bring about any visible impact on the legal system and legal education of Bangladesh. The author starts with the humble premise that anti-generic learning for rebellious lawyering cannot be termed as an effective method in Bangladesh because it has never had a proper opportunity to be tested. The foundation of this premise is built in two parts. In part 1, the concepts of anti-generic learning and rebellious lawyering are discussed, and in part 2, the author focused on the limitations of the concept.

Information systems (IS) is a discipline that draws upon many other disciplines to bridge theory and practice and address the information and knowledge needs of individuals, organisations and society. We propose that an ideal education in IS would be delivered via cross-faculty programs of study that are not combinations of units from different faculties and disciplines, but programs which include a coherent and cohesive set of units co-designed and co-delivered by teaching staff from more than one faculty. This allows students, and teachers, to appreciate the different content and perspectives within the same context, as they will experience in the workplace, and allow them to develop deeper understandings of the complexity that can arise in their roles as mediators and communicators in finding appropriate IT solutions. Such a model poses a radical change, and thus the framework we offer uses a ‘theory of change’ agenda.

The Disease Control Priorities program (DCP) has pioneered the use of economic evidence in health. The theory of change (ToC) put forward by Norheim is a further welcome and necessary step towards translating DCP evidence into better priority setting in low- and middle-income countries (LMICs). We also agree that institutionalising evidence for informed priority-setting processes is crucial. Unfortunately, there have been missed opportunities for the DCP program to challenge ill-judged global norms about opportunity costs and too little respect has been shown for the wider set of local circumstances that may enable, or disable, the productive application of the DCP evidence base. We suggest that the best way forward for the global health community is a new platform that integrates the many existing development initiatives and that is driven by countries’ asks.

This paper provides a review of the literature about family centres and other examples of centre-based practice. The literature reflects a period of some 25 years in which practice has sought to integrate protection, support for families and local development in local centres. ...

Quantitative evaluations might be insufficient for measuring the impact of interventions promoting the right to health, particularly in their ability to contribute to a greater understanding of processes at the individual,
community, and larger population level through which certain results are obtained. This paper discusses the
application of a qualitative approach, the “most significant change” (MSC) methodology, in the Philippines, Palestine, the Democratic Republic of the Congo, and El Salvador between 2010 and 2013 by Third World Health Aid and its partner organizations. MSC is based on storytelling through which the central question—
what changes occurred?—is developed in terms of, “who did what, when, why, and why was it important?”
The approach focuses on personal stories that reflect on experiences of change for individuals over time. MSC
implementation over several years allowed the organizations to observe significant change, as well as evolving types of change. Participants shifted their stories from “how the programs helped them” and “what they could do to help others benefit from the programs” to “what they could do to help their organizations.” The
MSC technique is useful as a complement to quantitative methods, as it is a slow, participatory, and intensive endeavor that builds capacity while being applied. This makes MSC a useful monitoring tool for programs with participatory and empowering objectives.

In political systems that we identify as 'violent political marketplaces', policy priorities for democracy activists and external actors include ending armed conflict, building governance institutions in a post-conflict setting, reforming the security sector, and promoting democracy and/or justice. How can policymakers and activists formulate evidence-based theories of change to help guide them in the pursuit of these goals? This question is the focus of this memorandum.

The obvious point is that these goals are extremely difficult to achieve in the context of violent political marketplaces. These are countries that have been known as 'fragile states' but could also (and more accurately) be described as open political systems on the margins of global capitalism. Not all fragile states are political marketplaces; we use the term to refer specifically to countries/contexts where politics is transactional and structured according to the laws of supply and demand rather than regulated by formal institutions (more on this below). Moreover, these political systems are often very violent. Examples include Somalia, South Sudan, Sudan, Yemen, the Democratic Republic of the Congo (DRC) and arguably Syria and Iraq.

We argue that theories of change in these contexts need to focus on the logic of transactional politics rather than on formal institutions. Further, they need to account for constant turbulence and its counterpart unpredictability, while remaining aware of how external interventions are an integral part of the dynamics of these systems. Following from this-we conclude that three broad types of interventions are possible: (a) tactically engineering short term outcomes to reduce violence; (b) a top-down reconfiguration of the political system; and (c) preparing the ground so that domestic actors can take advantage of eventual opportunities for democratic transformation. These interventions each come with their own risks, operate across variable timescales , and may be combined; in all cases, however, they need to be tailored to the particularities of the political system in question.

This paper is prompted by our own reflections on Theories of Change (TOC). We have been working with IRC International Water and Sanitation Centre, a number of Research Programme Consortia and the Mobilising Knowledge for Development (MK4D) programme at the Institute of Development Studies (IDS), to develop theories of change. Having explored our own ideas, and as a result of a ‘learning lab’ – our mechanism for reflection – we are sharing these early thoughts and practice, to enable others within the knowledge intermediary sector and beyond to discuss some of the practicalities of developing a Theory of Change.

The full service extended schools initiative in England sought to build the capacity of schools to deploy multiple interventions across a range of child, family, community contexts in order to improve the lives of children and families, and to enhance the sustainability of the areas where they live. As such, it was part of wider international developments to broaden the role of schools and integrate their work with that of other child and family services. The complexity and indeterminacy of such initiatives presents particular challenges to evaluators, and mean that simple input-output approaches to evaluation are inappropriate. This paper reports the use of a theory of change approach to the full service extended schools evaluation. It argues that an approach of this kind is better able to deal with complexity. However, in practice, this approach did not simply replicate standard outcomes evaluation processes in a more context-sensitive way. Rather, it involved fundamental shifts in what counted as valued outcomes, in a focus on potential rather than effectiveness, and in the roles of actors and evaluators. The paper concludes that, as the role of schools widens, educational evaluators will also have to rethink their roles.

This paper is prompted by our own reflections on Theories of Change (TOC). We have been working with IRC International Water and Sanitation Centre, a number of Research Programme Consortia and the Mobilising Knowledge for Development (MK4D) programme at the Institute of Development Studies (IDS), to develop theories of change. Having explored our own ideas, and as a result of a ‘learning lab’ – our mechanism for reflection – we are sharing these early thoughts and practice, to enable others within the knowledge intermediary sector and beyond to discuss some of the practicalities of developing a Theory of Change.

The "Red de Cuidados Antirracistas" is preparing under the guidance of the "Espacio del Inmigrante del Raval," both local-based organizations in Barcelona, a Change Analysis Report of the optimization to improve the pedagogical and resistance community services for migrant individuals in Barcelona. The report aims to create in-depth scenario planning due to the emergency that has affected families and individuals that identify as part of the migrant community, acting as an essential source of information for future antiracism education and community development programs. A Theory of Change planning is suggested, and based on the information found, contributions to the organizational knowledge report is presented in this paper. The contributions, a framework of questions, a strategy to understand the context, are based on understandings from the community based non-formal education and resistance in the global crisis context.

Background An Aboriginal-developed empowerment and social and emotional wellbeing program, known as Family Wellbeing (FWB), has been found to strengthen the protective factors that help Indigenous Australians to deal with the legacy of colonisation and intergenerational trauma. This article reviews the research that has accompanied the implementation of the program, over a 23 year period. The aim is to assess the long-term impact of FWB research and identify the key enablers of research impact and the limitations of the impact assessment exercise. This will inform more comprehensive monitoring of research impact into the future. Methods To assess impact, the study took an implementation science approach, incorporating theory of change and service utilisation frameworks, to create a logic model underpinned by Indigenous research principles. A research impact narrative was developed based on mixed methods analysis of publicly available data on: 1) FWB program participation; 2) researc...

Making value judgements on one's academic work is a fundamental skill, particularly for those within the higher education sector. Evaluative practices such as reflection in and on action (Schön, 1987; Cowan, 2006) and surveys on teaching effectiveness (Jones, Gaffney-Rhys and Jones, 1987; Praslova, 2010) are often proposed as a means of gathering evidence. The aim of this paper is to ask what theories of change (ToC) (Connell and Kubisch, 1999), self-evaluative practices, and barriers to effective implementation could be identified and what level of freedom and autonomy lecturers experience in evaluating their own work. This small study used a RUFDATA evaluative tool (Saunders, 2000) with sixteen full-time lecturers and took place in an Institute of Technology (IoT) in Ireland. Five theories of change were identified (1) transformative focused, (2) profession focused, (3) discipline focused, (4) workplace focused and (5) module focused, along with six self-evaluation practice themes and nine barriers to effective practice. Utilising Bamber's (2011b) discretion framework, participants were found to have autonomy in choosing what and how to evaluate their practice. The practical knowledge (Habermas, 1971) presented here offers an alternative approach to how academic work is 'valued', judged and understood from the lecturer's perspective. This paper proposes the view of self-evaluation, not as a set of standalone practices, but as part of a sequential process arising from a lecturer's purposeful academic intentions. The theories of change are used to judge the value of their work within the setting. Therefore, the proposed categories can offer a discourse to justify specific self-evaluative practices for quality assurance and enhancement.