Scaling-up bibliography

The ExpandNet bibliography includes publications, websites, grey literature, and conference reports that either directly address scaling up or provide valuable insights on scaling up. Included are materials from a range of global health and development technical areas as well as the various sciences relevant to scale up. The references below are useful, but not exhaustive, and not the result of a systematic search process. If you would like us to consider including an additional reference, please contact us.

Search and Filters

762 results

All references in the bibliography are grouped by topic below. To jump to down to view references related primarily to a specific topic, use the navigation links below. To create a personalized search, use the search and filtering box at left. Please note that using the search function will provide more robust results than the below navigation links below since some references span across multiple topic areas (i.e. Scaling-up theories, frameworks and concepts AND Family planning).

Scaling up theories, frameworks and concepts

ADBG. 2022. "African Development Bank Group: News." Website.
The overarching objective of the African Development Bank (AfDB) Group is to spur sustainable economic development and social progress in its regional member countries (RMCs), thus contributing to poverty reduction.

The Bank Group achieves this objective by: mobilizing and allocating resources for investment in RMCs; and
providing policy advice and technical assistance to support development efforts.

In 2015, all multilateral development institutions have agreed on a same set of objectives, called the Sustainable Development Goals.
Advance Africa. 2002. "Ten dimensions of scaling up reproductive health programs: An introduction."
This is the introduction to a series of issue papers for FP/RH program managers that consider the following questions on the subject of scaling up: A question of change: How do we know when we have achieved scale? A question of capacity: What management, technological, and human competencies are necessary to bring programs to scale? A question of strategy: What strategies most effectively produce the desired leap? A question of impact: How should the desired impact be measured? A question of sustainability: How do we maintain the gains of an expanded and comprehensive program? A question of access: What kind of coverage is enough to qualify as “scaled up”? A question of supply and demand: What is being scaled up? A question of cost: How much will it cost to scale up? A question of resources: What resources are needed and how can they be mobilized? A question of timing: When is the right time to scale up? Scaling Up Reproductive Health Programs: What’s New? “Scaling up” has entered the thinking of program managers as one of the important contemporary challenges for reproductive health programs. On the surface, achieving scale has always been a concern, whether the challenge was defined as increasing the number of users of modern methods, overcoming periods of stagnation, meeting unmet need, or improving program performance in areas of demand, access, and quality. The complexity of reproductive health programs with their multiple client groups, priorities, and linkages to the HIV/AIDS epidemic further complicates the strategies and technologies which need to be brought together to expand programs and increase impact. Sociopolitical changes, health sector reforms, and the shifting sands of resource availability—from money to contraceptives—create additional hurdles for program managers. What is new? It can be argued that the scaling up of reproductive health programs has been going on since they became a focus of health sector and social development. In some situations, programs scale up almost by default as they grow to accommodate population increase, even though indicators remain the same. However, the current term “scaling up” suggests a combination of strategies and technologies that are designed to be faster and of greater magnitude than the normal process of program expansion. It is no longer a question of waiting to see what happens and being surprised by changes, but to create purposeful change. When leaders act to scale up programs, they must make the process of expansion more predictable, eliminate the cycles of growth and stagnation, and most important, transform embryonic programs (often comprised of scattered pilot projects with very limited capacity) into large systems offering a variety of services to entire populations. These are the new challenges that differentiate the challenges of today’s scaling up from the challenges of past program.
Agapitova N, Linn JF. 2016. "Scaling Up Social Enterprise Innovations: Approaches and Lessons." Brookings, Global Economy and Development Working Paper 95.
in 2015 the international community agreed on a set of ambitious sustainable development goals (SDGs) for the global society, to be achieved by 2030. one of the lessons that the implementation of the millennium development goals (mdgs) has highlighted is the importance of a systematic approach to identify and sequence development interventions—policies, programs, and projects—to achieve such goals at a meaningful scale.1 The Chinese approach to development, which consists of identifying a problem and long-term goal, testing alternative solutions, and then implementing those that are promising in a sustained manner, learning and adapting as one proceeds—Deng Xiaoping’s “crossing the river by feeling the stones”—is an approach that holds promise for successful achievement of the Sdgs. Having observed the Chinese way, then World Bank group president james wolfensohn in 2004, together with the Chinese government, convened a major international conference in Shanghai on scaling up successful development interventions, and in 2005 the world bank group (wbg) published the results of the conference, including an assessment of the Chinese approach. (Moreno-Dodson 2005). Some ten years later, the WBG once again is addressing the question of how to support scaling up of successful development interventions, at a time when the challenge and opportunity of scaling up have become a widely recognized issue for many development institutions and experts. (Cooley and linn 2014)
Anandajayasekeram P. 2018. "Assessing the Scalability of a Research and Development Project: Concepts, Framework, and Assessment." Research in Agriculture and Applied Economics. 10.22004/ag.econ.276970
A good understanding of the scaling up process and a framework for analysing scalability is critical for informed decision making. In this paper a six step process is proposed to assess the scalability of an intervention/project. The approach was used to assess the scalability of the electronic voucher systems of Zambia and the Super Seeds Project in Zimbabwe. Estimated scalability indexes for these two projects were 77 and 85 respectively indicating the high potential for scaling up. The numerical score should not be viewed as carrying mathematical precision, because the scoring is based on subjective assessments. Through a validation process it was established that the approach is logically consistent and technically sound .The methodology also allows for a careful and methodological diagnosis of constraints to scaling-up. Key requirements to perform this analysis are a good understanding of the scaling up process in the local setting: effective participation and engagement of the key stakeholders, and external facilitator with no vested interest in the outcome. However mechanical application of the approach or superficial comparison of scalability indexes of different projects is likely to result in misleading conclusions. The model should be tested more broadly to assess its robustness and wider applicability. Acknowledgement : This paper is based on a research project originally commissioned by Vuna, a regional Climate Smart Agriculture Programme (operating from 2015 - 2018), and funded by the British Government s Department for International Development (DFID) as part of the United Kingdom s aid programme. However, the views and recommendations contained in this report are those of the author, and DFID is not responsible for, or bound by the recommendations made.
Anderson I. 2012. Scaling up development results: A literature review and implications for Australia's aid program. Australian Agency for International Development (AusAID), Canberra.
Australia and the UK are the only two OECD bilateral aid programs that currently involve significant scaling up of expenditure. Yet “scaling up” involves much more than increased expenditure, important as that is. Scaling up also involves new ways of doing business that scale up, and then sustain, development impact. Scaling up therefore does not simply mean doing more of the same, only bigger. An Effective Aid Program for Australia Making a real difference – Delivering Real Results (Australian Government 2011), the Government’s response to the Independent Review of Aid Effectiveness, recognises the importance of greater selectivity and larger average size programs focused on where Australia can make a difference. An increasing aid budget gives Australia the opportunity, and the financial means, to scale up development impact but this will require new approaches. Taking as its starting point the “Taking Activities to Scale” approach developed by the Brookings Institution (Chandy and Linn 2011) this paper captures some key lessons and experiences of other approaches to scaling up currently being discussed in the international literature. It complements the findings of the ODE-led case study of field level experience of scaling: AusAID’s CAVAC program1 in Cambodia. The paper is structured as follows. Chapter two provides definitions of scaling up from the literature, emphasising that scaling up is more to do with scaling up development impact than it is about scaling up expenditure, although the two are clearly linked. Chapter three summarises seven reasons why scaling up is central to any broader discussion about development effectiveness. It includes examples from the literature where scaling up has been successful – and where it has not. Chapter four summarises some of the main analytical frameworks for scaling up, starting with the Brookings model of taking activities to scale. Chapter five summarises some key practical lessons in scaling up from the literature. Chapter six then discusses possible implications for the Australian aid program in the light of that literature review, and the building blocks we have to work with. Chapter seven provides a short conclusion. This is followed by the References section of literature reviewed.
Anneke Slob and Alf Morten Jerve. 2008. "Managing Aid Exit and Transformation. Lessons from Botswana, Eritrea, India, Malawi and South Africa. Synthesis Report." Stockholm: Sida/Norad/ Danida/ Netherland's Ministry of Foreign Affairs 222 p.
What are the consequences in the recipient countries, when donor countries close down their bilateral aid programmes? Are exit practices consistent with established principles of partnership and mutuality in development co-operation? These are the two main questions under scrutiny in this evaluation initiated in 2005 by four donor countries - Denmark, the Netherlands, Norway and Sweden. The evaluation is based on country studies in Botswana, Eritrea, India, Malawi and South Africa including 14 exit cases involving any one of the four donors. It is a joint donor evaluation, and representatives of the partner countries were consulted in various ways. The evaluation was carried out in 2007/08 by a consortium of ECORYS (the Netherlands) and Chr. Michelsen Institute (Norway). It was guided by a Steering Group with representatives of the four commissioning donors, and Sida acted as lead agency in the management of the evaluation.
Aston T & Guerzovich F. 2022. "Contrasting alternative pathways to scale." Medium.
This is the third post in a post series about how to shape and navigate pathways to scale in social accountability. In post 1, we explained that scale is a complex change process. We questioned and reorganized how practitioners and the literature view scale — a to-do recommended in a recently published synthesis paper from the Action for Empowerment and Accountability (A4EA) programme. In post 2, we introduced a “resonance” pathway to scale, which has been largely overlooked in theory, but resonates with many practitioners we talk to. In this post, we’ll discuss the theoretical underpinnings of two pathways that are more familiar in the literature: best practice and resistance. We’ll also provide examples to illustrate them.
Aston T & Guerzovich F. 2022. "Introducing a resonance pathway to scale." Medium.
In our previous post we previewed our forthcoming paper “How do we shape and navigate pathways to social accountability scale? Introducing a middle-level Theory of Change.” We discussed the dilemma of scaling up in the Transparency, Participation, and Accountability (TPA) sector, and particularly, in social accountability. We argued that there is not one but several pathways to scale and that there is no best pathway, only the pathway that best fits with the available opportunities and constraints in any given context. We defined the pathways as: (1) the replication of best practice; (2) leveraging the countervailing power of resistance; and (3) seeking resonance with existing public sector efforts. In the next three posts we’ll illustrate what these pathways look like. This post will focus on the resonance pathway. We’ll briefly discuss some theory and illustrate it with examples from three contexts (Nigeria, Bangladesh, and Moldova). For a discussion of the other two pathways to scale, you’ll have to wait for the next post.
Baker E. 2010. "Taking programs to scale: a phased approach to expanding proven interventions." J Public Health Management Practice, 2010, 16(3), 264–269.
This column identifies the critical success factors at various stages of program replication and provides insights that may be useful to those seeking to take programs to scale. Any entrepreneur with a successful program thinks of enlarging it or “taking it to scale.” This is just as true of public health as of private business. Many public health programs are developed at a community level and effectiveness research is conducted to evaluate them. Once research indicates that a program works in a local setting, interest often develops to expand it. In some cases, the expansion succeeds, but at other times, barriers are encountered and the process of taking the program to scale fails. Research literature on taking a project to scale has been developed, which draws on a range of program experiences, thus leading to the development of various models for the process. This monograph identifies the critical success factors at various stages of program replication and provides insights that may be useful to those seeking to take programs to scale. Previous research and experience on taking programs to scale has identified five phases that should occur sequentially. At each phase of the process, the implementation team must make a conscious decision about the advisability of moving on to the next stage based on the outcome of the previous stage. The five phases, which are discussed in detail below, are preexploration, exploration, installation, initial program implementation, and ongoing program operations. Throughout this phased approach, care must be taken to ensure that replication activities are well thought out and the conditions for successful replication are carefully examined. As a result of this careful examination, a “go-no-go” decision should be made following the exploration phase to avoid proceeding ahead on the basis of “wishful thinking.” Further, this phased approach provides for greater clarity of roles and responsibilities as the process unfolds. Finally, this approach allows for progressive development of relationships that are central to effective partnerships in taking programs to scale. A few examples may help to illustrate this approach.
Bangser M. 2014. "A Funders Guide to Using Evidence of Program Effectiveness in Scale-up Decisions." Social Impact Exchange at Growth Philanthropy Network. MDRC, New York.
Foundations and other private funders increasingly seek opportunities to “scale impact” — that is, to extend the benefits of cost-effective interventions to more people, either by expanding these efforts in their current locations or by replicating them in new locations. However, the effect that funders seek to have on vexing social problems, such as entrenched poverty, the educational achievement gap, and health disparities, will not materialize unless they can identify interventions that truly work and then support sustained, high-quality implementation of these interventions as they scale up. There should be a high bar of reliable evidence to justify substantial scale-up because the stakes are high: Many lives will be affected, substantial funding is typically involved, and valuable resources can be wasted if funders back the wrong interventions. Nevertheless, funders often act without sufficient evidence to guide decisions on whether to invest in particular scale-up opportunities, as well as to confirm that the scale-ups that they do support have been successful. This Guide focuses on eight key questions that funders should generally ask during the stages of scaleup to help them direct resources to the right places. It includes references to the extensive, thoughtful work done by others while providing context and recommendations in a format that facilitates funders’ use of this material. The Guide draws primarily on lessons and principles from evaluations of programs to improve educational, employment, health, and other outcomes for individuals, while recognizing that many funders are also interested in scaling other approaches, including, for example, supporting institutions, disseminating best practices that are embedded in programs with multiple elements, and advocating for changes in public policies and systems.2 (The lessons in this Guide are relevant to this broad range of interventions, but the terms “program” or “services” are used throughout for the sake of simplicity.) building the evidence to support scale-up decisions is typically not a one-time event. It is a continuous — and typically multi-step — journey involving a variety of partners: those who help produce the evidence (for example, grantees, third-party evaluators,4 and the agencies that provide relevant data); key public and private decision-makers to whom the evidence must be communicated; and the service providers that scale up evidence-based approaches in a policy environment that is influenced by a range of public officials, advocacy groups, and other constituencies.5 Ideally, an effective evidence-building process will: Tap the potential for collaborative funding among philanthropic organizations, government, and businesses to support ongoing innovation, evidence-building, and phased scale-up of interventions. To achieve this, funders need to be willing to invest in data collection, analysis, and communication of findings.
Barker P, Reid, A &Schall MW. 2016. "A framework for scaling up health interventions: lessons from large-scale improvement initiatives in Africa." Implementation Science, 11:12.
Background Scaling up complex health interventions to large populations is not a straightforward task. Without intentional, guided efforts to scale up, it can take many years for a new evidence-based intervention to be broadly implemented. For the past decade, researchers and implementers have developed models of scale-up that move beyond earlier paradigms that assumed ideas and practices would successfully spread through a combination of publication, policy, training, and example. Drawing from the previously reported frameworks for scaling up health interventions and our experience in the USA and abroad, we describe a framework for taking health interventions to full scale, and we use two large-scale improvement initiatives in Africa to illustrate the framework in action. We first identified other scale-up approaches for comparison and analysis of common constructs by searching for systematic reviews of scale-up in health care, reviewing those bibliographies, speaking with experts, and reviewing common research databases (PubMed, Google Scholar) for papers in English from peer-reviewed and “gray” sources that discussed models, frameworks, or theories for scale-up from 2000 to 2014. We then analyzed the results of this external review in the context of the models and frameworks developed over the past 20 years by Associates in Process Improvement (API) and the Institute for Healthcare improvement (IHI). Finally, we reflected on two national-scale improvement initiatives that IHI had undertaken in Ghana and South Africa that were testing grounds for early iterations of the framework presented in this paper. Results The framework describes three core components: a sequence of activities that are required to get a program of work to full scale, the mechanisms that are required to facilitate the adoption of interventions, and the underlying factors and support systems required for successful scale-up. The four steps in the sequence include (1) Set-up, which prepares the ground for introduction and testing of the intervention that will be taken to full scale; (2) Develop the Scalable Unit, which is an early testing phase; (3) Test of Scale-up, which then tests the intervention in a variety of settings that are likely to represent different contexts that will be encountered at full scale; and (4) Go to Full Scale, which unfolds rapidly to enable a larger number of sites or divisions to adopt and/or replicate the intervention. Conclusions Our framework echoes, amplifies, and systematizes the three dominant themes that occur to varying extents in a number of existing scale-up frameworks. We call out the crucial importance of defining a scalable unit of organization. If a scalable unit can be defined, and successful results achieved by implementing an intervention in this unit without major addition of resources, it is more likely that the intervention can be fully and rapidly scaled.
Bates MA, & Glennerster R. 2017. "The Generalizability Puzzle." Stanford Social Innovation Review.
In 2013, the president of Rwanda asked us for evaluation results from across the continent that could provide lessons for his country’s policy decisions. One program tested in Kenya jumped out, and the Rwandan government wanted to know whether it would likely work in Rwanda as well. “Sugar Daddies Risk Awareness,” an HIV-prevention program, was remarkably effective in reducing a key means of HIV transmission: sexual relationships between teenage girls and older men. A randomized controlled trial (RCT) found that showing eighth-grade girls and boys a 10-minute video and statistics on the higher rates of HIV among older men dramatically changed behavior: The number of teen girls who became pregnant with an older man within the following 12 months fell by more than 60 percent.1 This study was compelling partly because of its methodology: Random assignment determined which girls received the risk awareness program and which girls continued to receive the standard curriculum. Our government partners could thereby have confidence that the reduction in risky behavior was actually caused by the program. But if they replicated this approach in a new context, could they expect the impact to be similar? Policy makers repeatedly face this generalizability puzzle—whether the results of a specific program generalize to other contexts—and there has been a long-standing debate among policy makers about the appropriate response. But the discussion is often framed by confusing and unhelpful questions, such as: Should policy makers rely on less rigorous evidence from a local context or more rigorous evidence from elsewhere? And must a new experiment always be done locally before a program is scaled up? These questions present false choices. Rigorous impact evaluations are designed not to replace the need for local data but to enhance their value. This complementarity between detailed knowledge of local institutions and global knowledge of common behavioral relationships is fundamental to the philosophy and practice of our work at the Abdul Latif Jameel Poverty Action Lab (J-PAL), a center at the Massachusetts Institute of Technology (founded in 2003) with a network of affiliated professors and professional staff around the world.
Beets M, von Klinggraeff L, Weaver RG, Armstrong B, Burkart S. 2021. "Small studies, big decisions: the role of pilot/feasibility studies in incremental science and premature scale-up of behavioral interventions" Pilot and Feasibility Studies. 7:173.
Careful consideration and planning are required to establish “sufficient” evidence to ensure an investment in a larger, more well-powered behavioral intervention trial is worthwhile. In the behavioral sciences, this process typically occurs where smaller-scale studies inform larger-scale trials. Believing that one can do the same things and expect the same outcomes in a larger-scale trial that were done in a smaller-scale preliminary study (i.e., pilot/feasibility) is wishful thinking, yet common practice. Starting small makes sense, but small studies come with big decisions that can influence the usefulness of the evidence designed to inform decisions about moving forward with a larger-scale trial. The purpose of this commentary is to discuss what may constitute sufficient evidence for moving forward to a definitive trial. The discussion focuses on challenges often encountered when conducting pilot/feasibility studies, referred to as common (mis)steps, that can lead to inflated estimates of both feasibility and efficacy, and how the intentional design and execution of one or more, often small, pilot/ feasibility studies can play a central role in developing an intervention that scales beyond a highly localized context.

Main body: Establishing sufficient evidence to support larger-scale, definitive trials, from smaller studies, is complicated. For any given behavioral intervention, the type and amount of evidence necessary to be deemed sufficient is inherently variable and can range anywhere from qualitative interviews of individuals representative of the target population to a small-scale randomized trial that mimics the anticipated larger-scale trial. Major challenges and common (mis)steps in the execution of pilot/feasibility studies discussed are those focused on selecting the right sample size, issues with scaling, adaptations and their influence on the preliminary feasibility and efficacy estimates observed, as well as the growing pains of progressing from small to large samples. Finally, funding and resource constraints for conducting informative pilot/feasibility study(ies) are discussed.

Conclusion: Sufficient evidence to scale will always remain in the eye of the beholder. An understanding of how to design informative small pilot/feasibility studies can assist in speeding up incremental science (where everything needs to be piloted) while slowing down premature scale-up (where any evidence is sufficient for scaling).

Keywords: Scaling, Translation, Intervention, Pilot, Feasibility, Early-stage
Begovic M, Linn JF, Vrbensky R. 2017. "Scaling Up the Impact of Development Interventions." Global Economy & Development Working Paper 101.
This paper reports on a review of whether and how the programs and projects supported by the United Nations Development Program (UNDP) in four countries (Bosnia and Herzegovina, Egypt, Moldova, and Tajikistan) apply a systematic approach to scaling up in pursuit of the Sustainable Development Goals (SDGs). The paper applies an operational framework consisting of six basic questions: (i) Is there a pathway to scale? (ii) What is the problem to be solved, the vision and target of scale? (iii) What ideas, innovations or models are to be scaled up? (iv) How can the enabling conditions (drivers and spaces) be put in place? (v) How about the sequencing of key steps? (vi) Does monitoring and evaluation support learning for scaling up? The paper concludes that many of UNDP’s programs and projects pursue pathways to scale, but that overall a more systematic operational approach along the lines suggested in this paper would be desirable.
Binswanger HP and Aiyar SS. 2003. "Scaling Up Community Driven Development Theoretical Underpinnings and Program Design Implications." Mimeo. The World Bank.
Community-driven development boasts many islands of production of outputs by different actors on the basis of success, but these have not scaled up to cover entire subsidiarity, lack of adaptation to the local context using countries. Binswanger and Aiyar examine the possible field-tested manuals, and lack of scaling-up logistics. The obstacles to scaling up, and possible solutions. They authors consider ways of reducing economic and fiscal consider the theoretical case for community-driven costs, overcoming hostile institutional barriers, development and case studies of success in both sectoral overcoming problems of co-production, adapting to the and multisectoral programs. Obstacles to scaling up local context with field testing, and providing scaling-up include high economic and fiscal costs, adverse logistics. Detailed annexes and checklists provide a guide institutional barriers, problems associated with the co- to program design, diagnostics, and tools.
Boa-Alvarado M, Woltering L, Sanjuán M. 2021. "Below the Tip of the Iceberg: Why Systems Change is the Key to Scaling Innovations and Solving Development Challenges." Next Billion.
Two years ago, we wrote a NextBillion article on why so many promising innovations are so hard to scale to a level where they have a significant impact on the Sustainable Development Goals. We called for a massive break with the linear and technology-driven way of providing solutions for global problems. We proposed some strategies to develop a more systemic and problem-driven approach to scaling successful initiatives, but we also recognized that the widespread application of such approaches was an exception rather than a rule. Since then, we have observed a surge in the use of words like “systems thinking” and “transformation” in the development sector. For example, in our line of work at the International Maize and Wheat Improvement Center (CIMMYT), we speak less of “agriculture” and more of “agri-food systems,” in which production and consumption are connected and limited to our planetary boundaries. In exploring the implications of this new way of thinking, we’ve asked ourselves what “agri-food system change” really involves, and what that means for scaling innovations in a systems context. To help guide this work, we applied an emblematic system thinking tool, the iceberg model, to the case of scaling land restoration practices in Central America. These are practices that curb erosion and improve soil structure and fertility to allow increased farm productivity and improve water and food security. The iceberg model helped us to recognize the systemic root causes of land degradation, and to identify what it takes to restore lands at a large scale. Below, we’ll explore this model and discuss how it has impacted our efforts to support land restoration and improve agri-food systems.
Bourns L, McClure D, Obrecht A. 2019. "Humanitarian Innovation: Untangling the Many Paths to Scale." Global Alliance for Humanitarian Innovation (GAHI).
First published in February 2019, this paper responds to a persistent humanitarian challenge: why do good ideas, demonstrated through pilots, fail to reach a scale at which they can maximise value for people affected by crises? The Untangling the Many Paths to Scale paper offers a new scale framework designed with humanitarian innovation in mind, shaped by four key factors: solution value, difficulty, contextual variation, and operational sustainability. Each combination of factors may have its own methodology and scaling journey, offering innovators a broader, more realistic range of options for determining how to take innovations to scale. Recognizing the diversity of pathways to scale allows for a more realistic consideration of resources, skills, and steps involved in scaling. You can find out more information about this resource on the GAHI website and through watching the video below.
Bradach J. 2004. "Going to Scale: The Challenge of Replicating Social Programs." Stanford Social Innovation Review, Stanford University, Palo Alto, CA.
HOMELESSNESS, illiteracy, chronic unemployment: nonprofits struggle to address society’s most intractable problems. And yet, as Bill Clinton noted, in reviewing schoolreform initiatives during his presidency, “Nearly every problem has been solved by someone, somewhere.” The frustration is that “we can’t seem to replicate [those solutions] anywhere else.”1 With a few exceptions, the nonprofit sector in the United States is comprised of cottage enterprises – thousands upon thousands of programs, each operating in a single neighborhood, in a single city or town. Often, this may be the most appropriate form of organization, but in some – perhaps many – cases, it represents a substantial loss to society overall. Time, funds, and imagination are poured into new programs that at best reinvent the wheel, while the potential of programs that have already proven their effectiveness remains sadly underdeveloped. One impediment to replication is the prevailing bias among funders to support innovative, “breakthrough” ideas.2 Another is the fact that, for many people, the concept conjures up images of bureaucracy and centralized control. Such images are uninviting in any sphere, but they are especially problematic in the nonprofit sector, where local “ownership” by donors and volunteers plays such an important part in organizational success. Add in the fact that for many social entrepreneurs, autonomy is an important form of psychic income, and it becomes easy to understand why implementing someone else’s dream tends not to be nearly as satisfying as building one’s own. In practice, however, replication is anything but a cookie-cutter process. The objective is to reproduce a successful program’s results, not to slavishly recreate every one of its features. At the heart of replication is the movement of an organization’s theory of change to a new location. In some cases, this might entail transferring a handful of practices from one site to another; in others, the wholesale cloning of the organization’s culture. Whatever the specifics, the right choice – including whether to replicate at all – will be strongly influenced by the complexity of the organization’s theory of change and the degree to which it can be articulated and standardized.3 Before turning to replication in the social sector, however, it is worth spending a moment on its for-profit sector analogue, franchising. Born in the 1920s, the franchise has become one of the dominant organization forms of our time, accounting today for roughly 50 percent of all U.S. retail sales. Franchise organizations align the energy and investment of local entrepreneurs with the strength of a network that may encompass hundreds or even thousands of units operating under the same trademark in different locations. While there are sharp differences between the forprofit and nonprofit sectors, which limit the analogy, franchising offers some thought-provoking lessons for social enterprises seeking to grow.
Bradach J. 2016. "Recommended Reads for Transformative Scale: September 2016." Bridgespan Group.
1. Diving In: Nonprofits, NGOs, and Design: This discussion between Jocelyn Wyatt (@jocelynw) of and Jeff Wishnie (@jwishnie) of the UN Foundation’s Digital Impact Alliance highlights the opportunities and challenges of using human-centered design in the social sector. These tools—“designing with and for the human/user/beneficiary”—have huge promise for helping to create more effective and more desirable (and therefore scalable) interventions. This piece offers interesting thoughts on some different approaches for embedding them in a social sector organization. 2. How do you scale up an effective education intervention? Iteratively, that’s how: World Bank economist David Evans (@tukopamoja) writes about some important research on implementing and scaling interventions through government systems. The research paper he discusses, which is largely focused on the efforts of the fantastic education NGO Pratham (@prathamusa) and Indian government schools, shows the power of using impact assessments to iterate toward an improved “2.0” intervention. More of this type of implementation-focused research is needed! 3. From Gutenberg to Zuckerberg—The Transformation of Business: Mark Bonchek (@MarkBonchek) writes about how leaders reaching scale through a platform model must learn where and how to cede control of the platform to its users. He describes a “‘one-to-many” dynamic for distributing information that emerged with the printing press. In today’s platform models—which thrive by connecting “many-to-many”— hierarchy gives way to networks, and processes to principles. As the business world shifts to these more dynamic, relational models, I can’t help but wonder what the potential of these models might be for scaling in the social sector. (Yet it feels like we have just begun to scratch the surface.) 4. In the Fight Against Hunger, Technology Brings Power to the People: Beth Simone Noveck (@bethnoveck) writes about potent efforts to end hunger by using data to address food shortage and pricing problems. The systems changes necessary—from data transparency policies to mechanisms for crowdsourcing stakeholder data—to generate and exchange this kind of information on a large scale are significant, but there is also tremendous potential for using data to address system problems of this type at scale. 5. Can This Data-Driven Organization Help Those Most Desperate Escape Life on the Streets?: Finally, this profile of the work of Community Solutions (@cmtysolutions) offers insight into a compelling example of an ongoing transformative scale effort. Community Solutions has brought a data-driven approach to attempting to end homelessness, blending local action with a platform for comparing and sharing data, and embedding learning and improvement in their systems. Intriguingly, “because of this emphasis on data, Community Solutions increasingly thinks of itself as a tech company.” An exciting read with implications for many.
Bradley EH, Curry L, Pérez-Escamilla R, Berg D, Bledsoe S, Ciccone DK, Fox A, Minhas D, Pallas S, Talbert-Slagle K, Taylor L, Yuan C. 2011. "Dissemination, diffusion and scale up of family health innovations in low-income countries." Yale Global Health Leadership Institute, New Haven.
In this report, we present the AIDED model for guiding dissemination, diffusion, and scale up of family health innovations in low-income countries. The model was developed using in-depth interviews with experts and practitioners, a systematic review of peer-reviewed and gray literature, and pressure testing with multiple audiences. The AIDED model posits five interrelated components to the complex process of scale up: 1) assess, 2) innovate, 3) develop, 4) engage, and 5) devolve. We identify key activities in the five components that have been linked to successful scale-up efforts of selected family health innovations: Depo-Provera, exclusive breastfeeding, community health worker approaches, and social marketing. The model represents scale up as a complex adaptive system in which the several interlocking parts interact in diverse and sometimes unpredictable ways. Nonetheless, the indepth interviews and literature synthesis suggests important patterns that are prominent in successful scale-up efforts and less apparent in failed efforts. These include explicit, early investment in assessment of community receptivity to the innovation and of the key environmental forces that may promote or limit scale up; tailoring of the innovation to fit target user groups; development of political, regulatory, socio-cultural, and economic support for the use of the innovation in target user groups; deep engagement with target user groups to ensure that the innovation is translated, integrated, and replicated effectively; and devolving of efforts to spread the innovation from the index user groups to additional sets of user groups often through social and professional networks and relationships. We found only limited evidence for differences in effective scale-up approaches across the different innovation types.
Bradley EH, Curry LA, Taylor LA, Pallas SW, Talbert-Slagle K, Yuan C, Fox A, Minhas D, Ciccone DK, Berg D, Pérez-Escamilla R. 2012. "A model for scale up of family health innovations in low-income and middle-income settings: A mixed methods study." BMJ Open, Aug 24;2(4).
Many family health innovations that have been shown to be both efficacious and cost-effective fail to scale up for widespread use particularly in low-income and middle-income countries (LMIC). Although individual cases of successful scale-up, in which widespread take up occurs, have been described, we lack an integrated and practical model of scale-up that may be applicable to a wide range of public health innovations in LMIC.

To develop an integrated and practical model of scale-up that synthesises experiences of family health programmes in LMICs.

Data sources: We conducted a mixed methods study that included in-depth interviews with 33 key informants and a systematic review of peer-reviewed and grey literature from 11 electronic databases and 20 global health agency web sites. Study eligibility criteria, participants and interventions: We included key informants and studies that reported on the scale up of several family health innovations including Depo-Provera as an example of a product innovation, exclusive breastfeeding as an example of a health behaviour innovation, community health workers (CHWs) as an example of an organisational innovation and social marketing as an example of a business model innovation. Key informants were drawn from non-governmental, government and international organisations using snowball sampling. An article was excluded if the article: did not meet the study's definition of the innovation; did not address dissemination, diffusion, scale up or sustainability of the innovation; did not address low-income or middle-income countries; was superficial in its discussion and/or did not provide empirical evidence about scale-up of the innovation; was not available online in full text; or was not available in English, French, Spanish or Portuguese, resulting in a final sample of 41 peer-reviewed articles and 30 grey literature sources.
Brantley C. 2019. "New publications: Shifting the mindset from “reaching many” to sustainable change." CIMMYT website.
Over the last few years, the research and development communities have deemed “scaling” a priority in order to help contribute to and achieve the Sustainable Development Goals (SDGs). On smaller scales, there has been great success in reducing hunger and poverty, but it has rarely expanded to regional or national levels. The International Maize and Wheat Improvement Center (CIMMYT) scaling head Lennart Woltering, in collaboration with colleagues Kate Fehlenberg and Bruno Gerard, as well as with international development experts Jan Ubels of SNV and Larry Cooley of Management Systems International, have been studying the process of scaling to understand why successful pilot projects are no guarantee for success at scale. In a new paper published in Agricultural Systems, they argue that pilot projects are usually set up and managed in heavily controlled environments that do not reflect the reality at scale. Furthermore, confusion of what scaling is and how it can be executed often results in a narrow focus on solely reaching numbers. “Counting household adoption of a practice at the end of a project is a poor metric of whether these people can and will sustain adoption after the project ends, let alone if adoption will reach others and actually contributes to improved livelihoods,” Woltering states. According to Woltering, “This paper is a call for a new scaling narrative, from one that is short-term and piecemeal, to one that recognizes the systemic nature of problems and solutions to achieve sustainable change at scale.” This requires a change in mindset, skills and ways of collaborating than what we currently consider normal. “Meaningful impact at scale hardly occurs within a project context, but when new ways of working are becoming ‘the new normal’ by a critical mass of actors ‘in the real world’,” Woltering explained. The authors present a number of frameworks that help to assess the scalability of innovations and the design of scaling strategies from the onset of projects and how to systematically think through key elements needed for scaling success. This includes CIMMYT’s very own Scaling Scan. Reaching the SDGs requires scaling interventions to be seen as building blocks within a system of other initiatives with the same goals.
Bulthuis S, Kok M, Raven J, Dieleman M. 2020. "Factors influencing the scale-up of public health interventions in low- and middle-income countries: A qualitative systematic literature review." Health Policy and Planning, 35(2), 219-234.
To achieve universal health coverage, the scale-up of high impact public health interventions is essential. However, scale-up is challenging and often not successful. Therefore, a systematic review was conducted to provide insights into the factors influencing the scale-up of public health interventions in low- and middle-income countries (LMICs). Two databases were searched for studies with a qualitative research component. The GRADE-CERQual approach was applied to assess the confidence in the evidence for each key review finding. A multi-level perspective on transition was applied to ensure a focus on vertical scale-up for sustainability. According to this theory, changes in the way of organizing (structure), doing (practice) and thinking (culture) need to take place to ensure the scale-up of an intervention. Among the most prominent factors influencing scale-up through changes in structure was the availability of financial, human and material resources. Inadequate supply chains were often barriers to scale-up. Advocacy activities positively influenced scale-up, and changes in the policy environment hindered or facilitated scale-up. The most outstanding factors influencing scale-up through changes in practice were the availability of a strategic plan for scale-up and the way in which training and supervision was conducted. Furthermore, collaborations such as community participation and partnerships facilitated scale-up, as well as the availability of research and monitoring and evaluation data. Factors influencing scale-up through a change in culture were less prominent in the literature. While some studies articulated the acceptability of the intervention in a given sociocultural environment, more emphasis was placed on the importance of stakeholders feeling a need for a specific intervention to facilitate its scale-up. All identified factors should be taken into account when scaling up public health interventions in LMICs. The different factors are strongly interlinked, and most of them are related to one crucial first step: the development of a scale-up strategy before scaling up.
Bulthuis S, Kok M, Raven J, Dieleman M. 2019. "Factors influencing the scale-up of public health interventions in low- and middle-income countries: a qualitative systematic literature review." Health Policy and Planning.
To achieve universal health coverage, the scale-up of high impact public health interventions is essential. However, scale-up is challenging and often not successful. Therefore, a systematic review was conducted to provide insights into the factors influencing the scale-up of public health interventions in low- and middle-income countries (LMICs). Two databases were searched for studies with a qualitative research component. The GRADE-CERQual approach was applied to assess the confidence in the evidence for each key review finding. A multi-level perspective on transition was applied to ensure a focus on vertical scale-up for sustainability. According to this theory, changes in the way of organizing (structure), doing (practice) and thinking (culture) need to take place to ensure the scale-up of an intervention. Among the most prominent factors influencing scale-up through changes in structure was the availability of financial, human and material resources. Inadequate supply chains were often barriers to scale-up. Advocacy activities positively influenced scale-up, and changes in the policy environment hindered or facilitated scale-up. The most outstanding factors influencing scale-up through changes in practice were the availability of a strategic plan for scale-up and the way in which training and supervision was conducted. Furthermore, collaborations such as community participation and partnerships facilitated scale-up, as well as the availability of research and monitoring and evaluation data. Factors influencing scale-up through a change in culture were less prominent in the literature. While some studies articulated the acceptability of the intervention in a given sociocultural environment, more emphasis was placed on the importance of stakeholders feeling a need for a specific intervention to facilitate its scale-up. All identified factors should be taken into account when scaling up public health interventions in LMICs. The different factors are strongly interlinked, and most of them are related to one crucial first step: the development of a scale-up strategy before scaling up. Scale-up, public health interventions, barriers and facilitators, LMICs, systematic review
CARE. 2020. "CARE Guidance Note: CARE’s approach to Impact at Scale."
How does an INGO like CARE contribute to sustainable, systemic change at scale, far beyond the direct work we and our partners carry out with communities? That is the challenge we’ve put at the heart of our new 10-year strategy. Drawing on learning from within CARE’s programs and within the wider scaling up community, we have also developed a new guidance note, outlining six pathways to Impact at Scale. These pathways are framed around Riddell and Moore’s three approaches of scaling up, scaling out and scaling deep. The guidance note outlines a number of examples from CARE’s programs at global, regional or national levels, that apply different combinations of these pathways, as well as highlighting learning from other organizations working on scaling up.
CARE. 2019. "Scale X Design Accelerator." Website.
The global humanitarian aid and development industry has a problem: innovation is everywhere, but examples of successfully scaled solutions are far less common. Even when we achieve impact at scale, the process can take decades. CARE’s Scale X Design Accelerator is bridging the gap between innovation and impact. We envision a future where we design for scale from the outset. Where pilot models are tested for scale and sustainability along with impact and proof of concept. Where development practitioners are armed with the science and skills needed for scaling. Where we distill our most promising solutions from the noise and give them the practical means to succeed.
Carter B, Joshi A, & Remme M. 2018. "Scaling Up Inclusive Approaches for Marginalised and Vulnerable People." K4D Emerging Issues Report. Brighton, UK: Institute of Development Studies.
This rapid review summarises the evidence on how to scale up inclusive approaches to complex social change. It looks at how to design scalable inclusive change interventions, as well as how to plan and manage the scale-up process. Focusing on interventions with the aim of reaching the most marginalised and transform social norms, it covers programmes aiming to deliver inclusive outcomes for women and girls (with a particular focus on preventing violence against women and girls) and persons with disabilities. To date, many interventions seeking to change harmful gender and disability norms have been implemented as small-scale projects. There are limited experiences of scale-up and fewer evaluations of these experiences. However, there are some documented case studies as well as emerging analysis that draw out lessons learned. From this evidence base, this rapid desk review identifies eight critical issues commonly highlighted as important considerations when scaling up inclusive change interventions: 1. Opportunities for systemic approach, including integrating political and community-level scale-up, and coordinating across multiple sectors and stakeholders 2. Political support for scale-up 3. Strategic choices: balancing reach, speed, cost, quality, equity, and sustainability 4. Catalysing change: tipping points, diffusion effects, and local champions 5. Locally grounded, participatory, and adaptive approaches 6. Long-term approaches with funding models to match 7. Cost-effective and financially feasible scale-up strategies 8. Measuring impact and sustainability.
CASE at Duke. 2021. "CASE Scaling Readiness Diagnostic: The oft-neglected side of scaling." Published in Scaling Pathways.
What is it? At CASE at Duke, we’ve created a free tool to help you identify areas of weakness and clearly articulate areas of strength along seven key — but oft-neglected — foundations for impact at scale. What do you get? In minutes, you’ll answer questions and receive a custom report with areas for growth, what that growth should look like, and suggested resources to help get you there. How do I start? Assess your organization on all seven key elements together (38 questions), or explore them one-by-one.
Centre for Epidemiology and Evidence. Milat AJ, Newson R, and King L. 2014. "Increasing the scale of population health interventions: A guide." Evidence and Evaluation Guidance Series, Population and Public Health Division. Sydney: NSW Ministry of Health.
In order to achieve population-wide health improvements, population health interventions found to be effective in a research setting need to be implemented as widely as possible. This involves a change of scale or a scaling up of the intervention. Scaling up such interventions is necessary to ensure the target population has access to the most effective services and programs available. However, not all interventions shown to be effective in a research setting are suitable for scaling up. The scalability of an intervention is not only determined by its effectiveness but other key issues such as the likely reach and adoption of the intervention, the costs of operating at scale, and the acceptability and fit of the intervention with the local context. It is important that attention is paid to the scalability of an intervention, so resources are allocated to interventions that are more likely to be successfully scaled up, and therefore more likely to have an impact on the health of the population as a whole. Scaling up is also more likely to be successful if a systematic approach to scaling up is adopted from the outset. Such an approach is required to help policy makers and practitioners address the substantial challenges faced when interventions are scaled up. For example, the same human, technical and financial resources available in the research setting in which the original intervention was tested may not be available when the intervention is scaled up. The intervention will need to be implemented in the ‘real world’ where few existing support systems may be in place and other pressing priorities and competing interests need to be considered. In addition, the context in which the intervention is scaled up is likely to be highly political, rapidly changing, and influenced by a variety of factors, inputs and relationships. Under such circumstances, successful scaling up calls for careful balancing between achieving desired outcomes and implementation constraints. It also requires an implementation process that uses and engages existing health system capacities, wherever possible, rather than imposing additional requirements and burdens on the system. Finally, the process of scaling up requires ongoing monitoring and the flexibility to adjust to changes in the political, social or organisational context. Introduction Scaling up refers to deliberate efforts to increase the impact of successfully tested health interventions to benefit more people and foster policy and program development on a lasting basis. However, at the current time, few policy makers and practitioners have skills in and knowledge of scaling up methods. Further, there are relatively few examples in the published literature where the steps and considerations involved in scaling up an intervention are described. As a consequence, population health interventions found to be effective in a research setting remain under-utilised by the field of population health
CGIAR, Wageningen University. 2022. "Innovation and Scaling: An introductory course on Innovation and Scaling." E-Course.
This course offers an accessible and robust introduction to the concepts of innovation and scaling.
Chambers R. 1992. "Spreading and self-improving: A strategy for scaling-up." In: Making a difference: NGOs and Development in a Changing World. Eds. Michael Edwards and David Hulme. London, England: Save the Children/Earthscan.
As Western aid budgets are slashed and government involvement with aid programmes reduced, NGOs in the voluntary sector are finding themselves taking an ever-increasing share of development work overseas. As they do so, they are forced to grow and to assume new responsibilities, taking more important and wide-ranging decisions - in many cases, without having had the chance to step back and review the options before them and the best ways of maximizing the impact they make. This collection of essays explores the strategies available to NGOs to enhance their development work, reviewing the ways that options can be understood, appropriate programmes and likely problems.
Clark C, Langsam K, Martin E, Worsham E. 2018. "Financing for Scaled Impact." Scaling Pathways, Innovation Investment Alliance, Skoll Foundation, CASE at Duke.
As social enterprises drive toward scaled impact, the journey can be accelerated—or significantly slowed—by the strategies and tactics used to manage resource needs. Accessing the right financing at the right points in the journey is critical to scaling success. As an analogy, consider the needs of a car with various options at the driver’s disposal to help navigate the road ahead. In this analogy, if you are a social entrepreneur, the car is your enterprise and you are the driver. With your car, you travel over a bumpy road, moving closer and closer to your end game or your target equilibrium change.4 Your car can travel on many pathways, and, as outlined in Pivoting to Impact, 5 you will face inevitable roadblocks and will need to make strategic pivots along the way. To be successful in financing this journey, you must set your course (understanding the rules of the road), determine which types of fuel you will use and when, and explore how you will use your car’s internal gears to help you navigate and accelerate.
Clark C, Langsam K, Martin E, Worsham E. 2018. "Financing for Scaled Impact." Scaling Pathways, Innovation Investment Alliance, Skoll Foundation, CASE at Duke.
As social enterprises drive toward scaled impact, the journey can be accelerated—or significantly slowed—by the strategies and tactics used to manage resource needs. Accessing the right financing at the right points in the journey is critical to scaling success. As an analogy, consider the needs of a car with various options at the driver’s disposal to help navigate the road ahead. In this analogy, if you are a social entrepreneur, the car is your enterprise and you are the driver. With your car, you travel over a bumpy road, moving closer and closer to your end game or your target equilibrium change.4 Your car can travel on many pathways, and, as outlined in Pivoting to Impact, 5 you will face inevitable roadblocks and will need to make strategic pivots along the way. To be successful in financing this journey, you must set your course (understanding the rules of the road), determine which types of fuel you will use and when, and explore how you will use your car’s internal gears to help you navigate and accelerate.
Cooley L and Linn JF. 2014. "Taking Innovations to Scale: Methods, Applications and Lessons." Results for Development Institute.
The international development community increasingly recognizes the need to go beyond fragmented, one-of projects. In response, there is now much talk and some action on scaling up successful innovations and pilot projects with an explicit goal of achieving sustainable impact at scale. However, many questions remain about the practical implications of pursuing a systematic scaling up approach and about how the approaches being pursued by diferent institutions and practitioners relate to each other. This paper considers two of the most widely used approaches to scaling up, developed in parallel during the mid-2000s. The first approach was devised by Management Systems International (MSI), a management consulting firm focused on designing and applying policy and management solutions to common development problems, mostly in developing countries. True to its mandate, MSI focused on designing a management framework for practitioners. MSI published the first version of a handbook in March 2006 under the title “Scaling Up – From Vision to Large-Scale Change: A Management Framework for Practitioners,” authored by Larry Cooley and Richard Kohl. Based on extensive experience applying this framework in diferent country and sectoral contexts, MSI issued a second, and substantially revised, edition of the handbook in 2012, under the same title and under the principal authorship of Larry Cooley and Rajani R. Ved. The second edition was accompanied by a scaling up toolkit publication, which provides details and examples of application for fifteen specific management tools referred to in the handbook.1 This approach is here referred to as the “MSI framework.” The second approach was initially developed in the Wolfensohn Center for Development at Brookings and published in 2008 in a Brookings working paper under the title “Scaling Up: A Framework and Lessons for Development Efectiveness from Literature and Practice,” by Arntraud Hartmann and Johannes Linn.2 This approach was then applied and further developed in the context of an institutional scaling up review of – and in collaboration with – the International Fund for Agricultural Development (IFAD), and in advisory and research undertakings with various aid agencies. In keeping with the objective of developing an institutional-level framework for IFAD, the approach aimed to provide high-level policy and operational guidance on the scaling up challenge. This approach is here referred to as the “IFAD framework.”3
Cooley L and Papoulidis J. 2017. "Scalable solutions in fragile states." Future Development Blog, The Brookings Institution.
Developing Economies Economic Development Global Development Global Poverty By 2030, an estimated 80 percent of the world’s extreme poor will live in “fragile states” where violent conflict is the most concentrated and vulnerability to natural disaster and climate change is on the rise. The current paradigm of promoting economic growth and poverty reduction will not work in the hardest places. Instead, responding to the challenges of fragility will require a new paradigm focused on sustainably meeting basic needs for hundreds of millions of the most vulnerable persons, managing shocks and stress that disrupt development, and addressing the root causes of fragility. None of these challenges can be properly addressed without working at scale. This is not simply a matter of expanding successful programs to reach more people. Instead, we argue that the most promising interventions for tackling fragility and building resilience only “kick-in” at scale because of the higher degree of functioning social capital they require. While not a panacea, scaling-up approaches provide a valuable organizing framework for strengthening and expanding social capital and overcoming the piecemeal, one-off, and non-strategic character of aid projects in fragile states. SCALING AS AN ORGANIZING FRAMEWORK Fragile states by definition face higher risks and insufficient capacities to deal with them at the community, state, or system level. Traditional pathways for service delivery—governments and markets—are often low capacity or otherwise compromised. In addition, international assistance is often piecemeal, short-term, and uncoordinated. Scaling frameworks offer a way to organize disparate capacities and resources among partners for the widespread and sustainable provision of basic services like health, water, and education. Scaling is arguably less risky than one-off projects. Several frameworks seek to unpack and improve the process of scaling valuable interventions. Among the most widely disseminated is Management Systems International’s framework, which (see the figure below) has three objectives—designing interventions with scale in mind, assessing scalability, and enhancing the scaling process—and is organized into three steps. Step 1 focuses on establishing and testing scalable interventions and viable pathways. Step 2, establishing the preconditions for scaling, focuses on the “political” tasks needed to support scaling. And, Step 3 includes the organizational, coordination, and accountability tasks essential for large-scale delivery and quality control.
Cooley L and Papoulidis J. 2017. "Tipping the Scales: Shifting from Projects to Scalable Solutions in Fragile States." Development 60, 190–196 (2017).
This article proposes a new paradigm and management framework for programming in fragile states. Confronted by the root causes of fragility and the pressing needs of hundreds of millions of the world’s most vulnerable persons, it argues for focusing on both problems simultaneously. The article singles out four features that distinguish scaling strategies in fragile states and suggests that the most promising interventions for tackling fragility and building resilience only ‘kick-in’ at scale because of the higher degree of functioning social capital they require. The article cites a number of examples to support the view that scaling-up approaches provide a valuable organizing framework for integrating a focus on social capital into programming and overcoming the piecemeal, one-off and non-strategic character of aid programmes in fragile states.
Cooley L, Guerrero I. 2018. "The Broken Part of the Business Model in Taking Innovation to Scale." MSI, Imago.
It’s as if someone suddenly flipped a switch. Discussions about scale and scaling, previously considered esoteric topics, now appear in almost every conversation about ending poverty, saving lives, or protecting the planet. Whether focused on social enterprise or governmental action, social investment or fortunes at the base of the pyramid, these conversations acknowledge the need to confront explicitly the obstacles that stand between successful pilot projects and the solution to population-level problems. As two people who have worked on the issues of scale and scaling for more than a decade , we are pleased at the attention the issue of scale is receiving. But we believe that there is a central element missing from many of these discussions. That element relates to overcoming predictable obstacles that innovations face when moving from the margin to the mainstream. At one end of the “supply chain”, support for innovation enjoys robust funding, strong institutions, and widespread success witnessed in the proliferation of innovation hubs and grand challenges. At the other end of the chain, markets and governments have structures and funding models that allow them to deliver goods and services sustainably at scale. But these two parts of the chain are separated by a broken link. We refer to this broken link as “intermediation” and it is the subject of this paper.
Cooley L, Kohl R, Ved R. 2016. "Scaling Up - From Vision to Large-Scale Change A Management Framework for Practitioners." Management Systems International.
The concept of “scaling up” has become increasingly popular as concerned donors note the relatively poor record of innovative pilot projects in extending their reach to large populations. The Scaling Up Management (SUM) FRAMEWORK and the guidelines presented in this document seek to improve this track record through practical advice on a three‐step, ten task process for effective scaling up. Scaling up is drawing the attention of a widening circle of donors, philanthropists, governments, NGOs, activists, and researchers as growing concerns emerge regarding the challenge of reaching large numbers of those in need. Despite this growing interest and an expanding array of documented cases, relatively little evidence‐based guidance exists about how to maximize prospects for new and innovative service delivery models to achieve scale. Written primarily for officials charged with making funding decisions and implementing programs, this paper seeks to provide concrete advice derived from theory and practice. It is intended to inform decisions regarding: Selecting projects with the potential to go to scale Designing projects to maximize their scalability Managing the scaling up process The SUM FRAMEWORK presented in this publication is organized as a series of steps and tasks based on the conviction that scaling up can be successfully managed. It is our hope that the development and humanitarian communities will in future years be filled with a growing number of entrepreneurial idealists who are able to bring ever‐increasing professionalism to the scaling up process.
Cooley L, Kohl R, Ved R. 2016. "Scaling Up - From Vision to Large-Scale Change, A Management Framework for Practitioners, Third Edition, 2016." Management Systems International.
The concept of “scaling up” has become increasingly popular as concerned donors and service providers find themselves under pressure to reduce costs, improve social outcomes, and explain why it has proven so difficult to accelerate the spread of best practices. To support this effort, the SUM Framework was developed to serve three related objectives, namely: To provide an easily understood and straightforward way for donors and investors to assess the scalability of proposed interventions; To provide guidelines for designing pilot projects and other innovations “with scale in mind”; and To provide tools and approaches to help practitioners manage the scaling up process. Scaling up has drawn the attention of a widening circle of donors, philanthropists, governments, NGOs, activists, social investors and researchers galvanized by the challenge of solving big problems and reaching large numbers of those in need. Despite this growing interest and an expanding array of documented cases, relatively little evidence-based guidance exists about how to maximize prospects for new and innovative products and service delivery models achieving and sustaining outcomes at scale. Written primarily for officials charged with making funding decisions and implementing programs, this paper seeks to provide concrete advice derived from theory and practice. It is intended to inform decisions regarding: Selecting projects with the potential to go to scale, Designing projects to maximize their scalability; and Managing the scaling up process The SUM Framework is organized as a series of steps and tasks based on the conviction that scaling up can be successfully planned and managed. It is our hope that future years will witness a growing number of entrepreneurial idealists able to bring ever-increasing professionalism to the scaling up process.
Cooley L, Kohl R. 2005. "Scaling Up—From Vision to Large-scale Change: A Management Framework for Practitioners." Management Systems International (MSI), Washington DC.
The concept of “scaling up” has become increasingly popular as donors have acknowledged with concern the relatively poor record of innovative pilot projects in extending their reach to large populations. Recognizing this, in October 2003, the John D. and Catherine T. MacArthur Foundation awarded a grant to Management Systems International (MSI) to develop a field-tested framework and set of guidelines for improved management of the scaling-up process. This framework was intended to be of direct and immediate use to those planning, implementing, and funding pilot projects and to those hoping to take the results of such projects to scale. An earlier draft of the Scaling Up Management (SUM) Framework was used in field tests with reproductive health non-governmental organizations (NGOs) in Nigeria and Mexico and as a basis for initial dissemination efforts. This revised version of the framework reflects that experience and incorporates the feedback from initial dissemination. One significant finding emerged from this research: Few so-called “pilot projects” take the steps needed to maximize their prospects for scaling up. The framework and guidelines presented in this document seek to improve this track record by offering practical advice on a three-step process to carry out each of ten key tasks needed for effective scaling up. Scaling up is drawing the attention of an ever-increasing circle of donors, philanthropists, governments, NGOs, activists, and researchers. As interest grows, so do their concerns regarding the replicability of successful innovations and the challenges of reaching large numbers of those in need. Despite this growing interest and an expanding array of documented cases, relatively little evidence-based advice exists about maximizing the prospects for new and innovative service-delivery models to achieve scale. Written primarily for officials charged with making funding decisions and implementing programs, this paper seeks to provide concrete advice derived from theory and practice. It is intended to inform decisions about: Y Selecting projects with the potential to go to scale; Y Designing projects to maximize their scalability; and Y Managing the scaling-up process. The SUM Framework presented in this publication is organized as a series of Steps and Tasks. This approach is based on the conviction that scaling up can be successfully managed and that this process can be carried out most effectively by breaking it down into concrete strategies and actions. It is our hope that the development and humanitarian communities will be filled in future years with a growing number of entrepreneurial idealists bringing everincreasing professionalism to the scalingup process.
Cooley L, Reilly T, Tolani N, Ngo J, Zodrow G. 2021. "Scaling up – from vision to large-scale change: a management framework for practitioners, second edition." Management Systems International (MSI), Washington DC.
The concept of “scaling up” has become increasingly popular as concerned donors note the relatively poor record of innovative pilot projects in extending their reach to large populations. The Scaling Up Management (SUM) FRAMEWORK and the guidelines presented in this document seek to improve this track record through practical advice on a three‐step, ten task process for effective scaling up. These steps and tasks include: Step 1: Develop a Scaling Up Plan Task 1: Create a Vision 1A. The Model: What Is Being Scaled Up? 1B. The Methods: How Will Scaling Up Be Accomplished? 1C. Organizational Roles: Who Performs the Key Functions? 1D. Dimensions of Scaling Up: Where and For Whom Does Scaling Up Occur? Task 2: Assess Scalability 2A. Determining the Viability of the Model for Scaling Up 2B. Analyzing the Organizational and Social Context Task 3: Fill Information Gaps Task 4: Prepare a Scaling Up Plan Step 2: Establish the Pre‐Conditions for Scaling Up Task 5: Legitimize Change Task 6: Build a Constituency Task 7: Realign and Mobilize Resources Step 3: Implement the Scaling Up Process Task 8: Modify Organizational Structures Task 9: Coordinate Action Task 10: Track Performance and Maintain Momentum The theory and practice underlying the FRAMEWORK come from the discipline of “strategic management.” More specifically, Step 1 brings to bear best practices related to strategic planning in complex settings; Step 2 focuses on change management functions associated with consensus building, policy change, and resource allocation; and Step 3 emphasizes the operational aspects of multi‐actor program implementation. Scaling up is drawing the attention of a widening circle of donors, philanthropists, governments, NGOs, activists, and researchers as growing concerns emerge regarding the challenge of reaching large numbers of those in need. Despite this growing interest and an expanding array of documented cases, relatively little evidence‐based guidance exists about how to maximize prospects for new and innovative service delivery models to achieve scale. Written primarily for officials charged with making funding decisions and implementing programs, this paper seeks to provide concrete advice derived from theory and practice. It is intended to inform decisions regarding: Selecting projects with the potential to go to scale Designing projects to maximize their scalability Managing the scaling up process The SUM FRAMEWORK presented in this publication is organized as a series of steps and tasks based on the conviction that scaling up can be successfully managed. It is our hope that the development and humanitarian communities will in future years be filled with a growing number of entrepreneurial idealists who are able to bring ever‐increasing professionalism to the scaling up process.
CORE. 2005. "Scale and Scaling-Up." A CORE Group Background Paper on Scaling-Up Maternal, Newborn and Child Health Services, July 11, 2005.
This paper briefly summarizes definitions, approaches, and challenges to achieving “scale” in community-focused health programs as discussed at the 2005 CORE spring meeting and the USAID child survival and health grants program mini-university. This paper is meant to harmonize a vocabulary for use by NGOs and their partners as they further discuss, debate, and analyze how NGOs and their partners can reach more people with high quality maternal, child and neonatal health interventions. Case studies and further documentation of discussions on scale can be found in the proceedings from the CORE Spring 2005 Meeting available on the CORE website
Dan Berelowitz. 2022. "Mission to Scale (Podcast)." Spring Impact, Hueman Group Media.
Extreme poverty, climate change, racial injustice. As the world’s problems accelerate, so should our solutions. Mission to Scale reveals the tools, mindsets and strategies that organizations and funders need — to make the most impact. On the podcast, Spring Impact Founder Dan Berelowitz talks to brilliant leaders from today’s most dynamic nonprofits, social enterprises, and purpose-driven brands about pursuing change, at scale. From creating lean strategy to team building, each episode helps you take the guesswork out of changing more lives. This podcast is a production of Spring Impact and Hueman Group Media.
Dearing JW, & Cox JG. 2018. "Diffusion Of Innovations Theory, Principles, And Practice." Health Affairs. 37:2.
Aspects of the research and practice paradigm known as the diffusion of innovations are applicable to the complex context of health care, for both explanatory and interventionist purposes. This article answers the question, “What is diffusion?” by identifying the parameters of diffusion processes: what they are, how they operate, and why worthy innovations in health care do not spread more rapidly. We clarify how the diffusion of innovations is related to processes of dissemination and implementation, sustainability, improvement activity, and scale-up, and we suggest the diffusion principles that can be readily used in the design of interventions. TOPICS IMPROVING CARE PERFORMANCE MEASURES QUALITY IMPROVEMENT INITIATIVES SYSTEMS OF CARE PUBLIC HEALTH
Diaz J, Simmons R, Diaz M, Cabral F, Chinaglia M. 2007. "Scaling up family planning service innovations in Brazil: the influence of politics and decentralization." In: Simmons R, Fajans P, Ghiron L, eds. Scaling up health service delivery: from pilot innovations to policies and programmes. Geneva, World Health Organization, 2007:135–156.
The principles of strategic management suggest that a major step in ensuring effective scaling up is to understand the diverse environments in which health service innovations are expanded. When service innovations are expanded in the public sector, the political and administrative institutions, as well as the health sector setting constitute major environmental influences. This chapter analyses these factors in Brazil, using the experience of a project which sought to enhance equitable access and improve the quality of care in public sector family planning services. Nongovernmental organizations acted as the resource team that facilitated the testing of the original service innovations in one municipality and then assisted with their expansion to others. The chapter shows that scaling up is influenced by an ongoing process of decentralization and by the politics of family planning. Scaling up family planning innovations faces special challenges, which would not be encountered in other areas of reproductive health in Brazil.
Diaz M, Cabral F. 2007. "An innovative educational approach to capacity building and scaling up reproductive health services in Latin America." In: Simmons R, Fajans P, Ghiron L, eds. Scaling up health service delivery: from pilot innovations to policies and programmes. Geneva, World Health Organization, 2007:157–177.
As governments seek to meet the global health agendas of the past decade, new approaches to the training of health professionals are needed. Training must move away from an exclusive focus on technical skills and begin to incorporate educational strategies that empower providers, programme managers and community leaders to become agents of change. This chapter describes a methodology for in-service training that builds on Paulo Freire’s educational philosophy and explains how the capacity to provide innovative training was scaled up in public sector reproductive health services in Brazil, Bolivia and Chile. Statistics on the training sessions demonstrate the reach of this training initiative, and testimonials show its profound impact on newly trained trainers.
Dickson KE, Simen-Kapeu A, Kinney MV, Huicho L, Vesel L, Lackritz E, Johnson JDG, Xylander SV, Rafique N, Sylla M, Mwansambo C, Daelmans B, Lawn JE. 2014. "Every Newborn: health-systems bottlenecks and strategies to accelerate scale-up in countries." Lancet, 384(9941) 438-454.
Universal coverage of essential interventions would reduce neonatal deaths by an estimated 71%, benefi t women and children after the fi rst month, and reduce stillbirths. However, the packages with the greatest eff ect (care around birth, care of small and ill newborn babies), have low and inequitable coverage and are the most sensitive markers of health system function. In eight of the 13 countries with the most neonatal deaths (55% worldwide), we undertook a systematic assessment of bottlenecks to essential maternal and newborn health care, involving more than 600 experts. Of 2465 bottlenecks identifi ed, common constraints were found in all high-burden countries, notably regarding the health workforce, fi nancing, and service delivery. However, bottlenecks for specifi c interventions might diff er across similar health systems. For example, the implementation of kangaroo mother care was noted as challenging in the four Asian country workshops, but was regarded as a feasible aspect of preterm care by respondents in the four African countries. If all high-burden countries achieved the neonatal mortality rates of their region’s fastest progressing countries, then the mortality goal of ten per 1000 livebirths by 2035 recommended in this Series and the Every Newborn Action Plan would be exceeded. We therefore examined fast progressing countries to identify strategies to reduce neonatal mortality. We identifi ed several key factors: (1) workforce planning to increase numbers and upgrade specifi c skills for care at birth and of small and ill newborn babies, task sharing, incentives for rural health workers; (2) fi nancial protection measures, such as expansion of health insurance, conditional cash transfers, and performance-based fi nancing; and (3) dynamic leadership including innovation and community empowerment. Adapting from the 2005 Lancet Series on neonatal survival and drawing on this Every Newborn Series, we propose a country-led, data-driven process to sharpen national health plans, seize opportunities to address the quality gap for care at birth and care of small and ill newborn babies, and systematically scale up care to reach every mother and newborn baby, particularly the poorest.
Dror I, Wu N. 2020. "Scaling better together: The International Livestock Research Institute’s framework for scaling." Nairobi, Kenya: ILRI.
The International Livestock Research Institute (ILRI) works to improve food and nutritional security and reduce poverty in developing countries through research for efficient, safe and sustainable use of livestock. ILRI’s core business is to undertake livestock research for development. However, translating research outputs into outcomes, and ultimately to impact at scale, has been an enduring challenge for ILRI, other CGIAR centres and research institutions. To address this challenge, ILRI needs to adopt evidence-based development approaches and methods of achieving impact at scale, and maximize the probability of success by working in partnership with development partners such as the private sector, governments, NGOs, development banks, etc. In 2017, ILRI created the Impact at Scale Program (I@S) to ensure the organization has the requisite expertise to manage projects that are rooted in development, effectively partner with other organizations to deliver at scale and ensure it can demonstrate the impact of its livestock research for development. One of the key product lines of I@S is designed to provide the institute and its key partners with a framework for scaling proven technologies and solutions. This includes coming up with a systematic and pragmatic approach to scaling by scanning ILRI’s research portfolio to identify readiness for scaling different solutions. This is meant to provide a clear and objective picture of current and long-term trends with regards to the scaling potential of different priority commodities, geographies, themes in ILRI programs, the institute overall and the wider livestock sector. Lack of appropriate systems to apply existing scaling tools can be a challenge for ILRI and CGIAR when trying to design and execute research projects with the end in mind. To make scaling concepts and tools more accessible to ILRI and CGIAR researchers and their partners, I@S has reviewed the landscape of scaling in the context of agricultural research for development with the aim of summarizing relevant approaches and tools that livestock projects can embed along with a detailed process on how they can be supported and implemented systematically. This document provides an overview of the steps in a scaling process as envisaged for ILRI and provides short summaries of nine tools related to scalability assessment developed by research institutes, development agencies, nonprofit organizations and private companies. The document also provides a summary of our assessment of these tools. By applying these tools and principles, we hope research projects will achieve a better “scaling mindset” from the early stages of project design. The outlined process hinges on the close involvement and expertise of project teams across ILRI and CGIAR to ensure ownership and a shared understanding of scaling objectives. It also depends on the commitment of scaling coordinators who help facilitate smooth implementation of scaling assessments and subsequent tracking and implementation of scaling plans. By design, the approach is iterative and agile and focuses on practical steps and facilitating skills and approaches that allow projects to identify and adapt to changes quickly. When reviewing the tools, our aim was to curate a process that would be “fit for purpose” for the ILRI/ CGIAR operational environment. Therefore, we focused on a set of evaluation criteria that we felt were most relevant to this goal. A summary table is provided below, and a more detailed description of the methodology used and the rationale for our selection can be found in Table 2, Part II: Synthesis on scaling tools and guides.
Duflo E. 2004. "Scaling Up and Evaluation." The International Bank for Reconstruction and Development / The World Bank.
This paper discusses the role that impact evaluations should play in scaling up. Credible impact evaluations are needed to ensure that the most effective programs are scaled up at the national or international levels. Scaling up is possible only if a case can be made that programs that have been successful on a small scale would work in other contexts. Therefore the very objective of scaling up implies that learning from experience is possible. Because programs that have been shown to be successful can be replicated in other countries while unsuccessful programs can be abandoned, impact evaluations are international public goods, thus the international agencies should have a key role in promoting and financing them. In doing this, they would achieve three important objectives: improve the rates of return on the programs they support, improve the rates of return on the programs other policymakers support by providing evidence on the basis of which programs can be selected, and build long-term support for international aid and development by making it possible to credibly signal what programs work and what programs do not work. The paper argues that considerable scope exists for expanding the use of randomized evaluations. For a broad class of development programs, randomized evaluation can be used to overcome the problems often encountered when using current evaluation practices.
Edwards M and Hulme D. 1992. "Scaling-up the developmental impact of NGOs: Concepts and experiences." In Making a difference: NGOs and Development in a Changing World. Eds. Michael Edwards and David Hulme. London, England: Save the Children/Earthscan.
As Western aid budgets are slashed and government involvement with aid programmes reduced, NGOs in the voluntary sector are finding themselves taking an ever-increasing share of development work overseas. As they do so, they are forced to grow and to assume new responsibilities, taking more important and wide-ranging decisions - in many cases, without having had the chance to step back and review the options before them and the best ways of maximizing the impact they make. This collection of essays explores the strategies available to NGOs to enhance their development work, reviewing the ways that options can be understood, appropriate programmes and likely problems.
Evans D. 2022. "Scaling Programs Effectively: Two New Books on Potential Pitfalls and the Tools to Avoid Them." World Bank Blogs.
You’ve designed a program to help more children learn to read, or to reduce the number of women who die in childbirth, or to increase how much wheat farmers grow. You pilot the program. You even invest in a careful evaluation. It works. Fantastic news! You realize that this could benefit so many more people than just the participants in your little pilot, so you convince a government agency to scale it up, or maybe you get funding to scale it up yourself. But at scale, the promised results fail to materialize. So what happened? Interventions that are effective at scale are the golden nuggets of public policy: valuable, rare, and even apparent winners are often revealed to be fool’s gold. They can be so challenging to find that you could be tempted to throw up your hands and say that “Nothing scales!” While that’s an overstatement (many interventions have had positive impacts at scale), there are also many, many failures. What factors drive the drop (or, in some cases, disappearance) of impacts as programs go from pilot to scale, and how can we avoid them?
Evans SH, & Clark P. 2011. "Disseminating orphan innovations." Stanford Social Innovation Review. Winter 2011. Leland Stanford Junior University.
The social sector invests a great deal of time and money trying to create social innovations, but pays scant attention to the challenges of spreading successful ones to other locations. Disseminating innovations takes a distinct, sophisticated skill set, one that often requires customizing the program to new circumstances, not replicating. The uneven distribution of social benefits may be partly due to inevitable lag times in spreading good ideas. We believe, however, that clumsy or weakhearted attempts at dissemination also have kept best practices from spreading. The principal at Chicago’s Providence-St. Mel said that the school’s accomplishments “are not rocket science,” but there is indeed a science behind transplanting innovations, one that can be learned partly from successful examples. Such learning will grow more likely where enthusiasts for social benefits recognize that creating social innovations and disseminating them call upon vastly different talents, and where the people who want to launch an innovation in a new location are fully prepared to customize it for the new locale—even, perhaps especially, when that means giving up the glory and the credit to achieve the greater goal. Customization requires slogging labor and is more nuanced than replication, but it is often the only way that a successful but orphaned innovation can take root elsewhere. For us, putting in that extra effort has made all the difference.
ExpandNet 2017. "Bibliography: Systematic Approaches for Scaling Up Best Practices Version 2."
Evidence to Action CoP curated this selection of peer-reviewed articles, reports, briefs, and other grey literature that address systematic approaches to scaling up. It constitutes literature of relevance to the discussions of the Community of Practice on Systematic Approaches to Scaling Up. Topics include: Definitions of scaling up, Special issues or components of scaling up, Low- and middle-income countries, Family planning/reproductive health
ExpandNet. 2011. "A field-based and participatory approach to supporting the development of scaling-up strategies."
As governments seek to meet the global health agendas of the past decade, new approaches to the training of health professionals are needed. Training must move away from an exclusive focus on technical skills and begin to incorporate educational strategies that empower providers, programme managers and community leaders to become agents of change. This chapter describes a methodology for in-service training that builds on Paulo Freire’s educational philosophy and explains how the capacity to provide innovative training was scaled up in public sector reproductive health services in Brazil, Bolivia and Chile. Statistics on the training sessions demonstrate the reach of this training initiative, and testimonials show its profound impact on newly trained trainers."
Fajans P, Simmons R, and Ghiron L. 2006. "Helping public sector health systems innovate: the strategic approach to strengthening reproductive health policies and programs." American Journal of Public Health, 96:435-440.
Abstract Public sector health systems that provide services to poor and marginalized populations in developing countries face great challenges. Change associated with health sector reform and structural adjustment often leaves these already-strained institutions with fewer resources and insufficient capacity to relieve health burdens. The Strategic Approach to Strengthening Reproductive Health Policies and Programs is a methodological innovation developed by the World Health Organization and its partners to help countries identify and prioritize their reproductive health service needs, test appropriate interventions, and scale up successful innovations to a subnational or national level. The participatory, interdisciplinary, and country-owned process can set in motion much-needed change. We describe key features of this approach, provide illustrations from country experiences, and use insights from the diffusion of innovation literature to explain the approach\'s dissemination and sustainability.
Fajans P, Thom N T, Whittaker M, Satia J, Phuong Mai TT. 2007. "Strategic choices in scaling up: introducing injectable contraception and improving quality of care in Viet Nam." In: Simmons R, Fajans P, Ghiron L, eds. Scaling up health service delivery: from pilot innovations to policies and programmes. Geneva, World Health Organization, 2007:31–51.
This chapter analyses the process of scaling up introduction of the injectable contraceptive depot-medroxyprogesterone acetate (DMPA) as part of a package of interventions to improve quality of care in the provision of all contraceptives in the Vietnamese family planning programme. After a strategic assessment of the need for contraceptive introduction and pilot testing of the interventions in three provinces, these interventions were scaled up to 21 of Viet Nam’s 64 provinces. Although DMPA was widely introduced, going to scale did not fully achieve the gains in quality of care for all methods found in the pilot phase. Three interrelated variables affected this outcome: the degree of change required in the service delivery system, the pace of expansion, and available resources to support expansion. In this case, scaling up proceeded faster than was desirable, given the extensive changes entailed by the interventions and the limitations in resources. Before embarking on rapid expansion involving complex programmatic changes, planners of scaling-up strategies should carefully assess the balance between these three variables.
Fixsen A. 2013. "Monitoring and Evaluating Scaling up of health system Interventions: Theory and Practice." Draft manuscript.
Hundreds of pilot studies of health innovations have been conducted around the world. Evaluations of many of them have revealed positive results on intended health outcomes. Yet few of these successfully-piloted innovations have been taken to scale, with fewer still scaled up sustainably. This is at least partially due to our lack of understanding of the scale-up process and to gaps in our knowledge of how to monitor and evaluate this process. Whether the innovation involves introducing new methods of contraception, decreasing HIV transmission from mother to child, or improving post-abortion care, taking ‘what works’ to scale is neither linear nor quick (Patton, 2011). The experience of the Institute for Reproductive Health, Georgetown University (IRH) scaling up Standard Days Method® (SDM) in five countries suggests that monitoring and evaluation (M&E) practices geared specifically toward the scale-up phase can increase the probability of achieving sustainable, large-scale implementation by providing real-time feedback designed to meet stakeholder needs. The research-to-practice continuum can be conceptualized in three phases: pilot, scaling up, and large scale implementation. The particular M&E approaches and tools most appropriate and effective for each stage vary considerably. During the pilot phase, the M&E priority is to accurately measure the effect of an innovation, the complete package of interventions to be scaled up. During the scale-up period, the emphasis shifts to measuring processes to ensure the innovation is implemented with fidelity, at an acceptable pace, and achieves desired coverage, while maintaining the fidelity of the innovation. Once an innovation is operating at scale, that is, has become a routine part of services, efforts continue to measure fidelity, but may also include determination of population-level impact. M&E processes, indicators, benchmarks, and data collection methods need to reflect these evolving priorities. The M&E process, as well as the information it generates, can support the scale-up process by helping stakeholders clearly define the innovation, maintain fidelity as the reach of the program expands, identify the need for adaptations, and ensure that the adapted innovation continues to produce the desired effect. A collaborative process of benchmarking, process documentation, and continuous feedback will engage stakeholders and involve new partners as services expand. This process can also help these stakeholders remain attentive and make midcourse adjustments as needed, to ensure that scale-up remains on track in a changing environment. M&E data can also verify that the evolving innovation maintains its effectiveness, and it can provide opportunities to advocate for investment and partnerships, while fostering discussions which reinforce the core values of the innovation.
Franco LM, Marquez L. 2011. "Effectiveness of collaborative improvement: evidence from 27 applications in 12 less developed and middle income countries." British Medical Journal Quality Safety. 20-658-665.
The improvement collaborative approach has been widely promoted in developed countries as an effective method to spread clinical practices, but little has been published on its effectiveness in developing country settings. Between 1998 and 2008, the United States Agency for International Development funded 54 collaboratives in 14 low- and middle-income countries, adapting the approach to resource-constrained environments.

The authors analysed data on provider compliance with standards and outcomes from 27 collaboratives in 12 countries that met study inclusion criteria (at least 12 months of data available for analysis and indicators measured as percentages). The dataset, representing 1338 facility-based teams, consisted of 135 time-series charts related to maternal, newborn and child health, HIV/AIDS, family planning, malaria and tuberculosis. An average of 28 months of data was available for each chart.

Eighty-seven per cent of these charts achieved performance levels of 80% or higher, and 76% reached at least 90% performance, even though two-thirds had a baseline performance below 50%. Teams achieved average increases of 51.9 percentage points (SE¼28.0) per chart, with baseline value being the main determinant of absolute increase. Teams consistently maintained this level of performance for an average of 13 months (69% of months of observation). The average time to reach 80% performance was 9.2 months (SE 8.5), and to reach 90% performance, 14.4 months (SE¼12.0).

Collaborative improvement can produce significant, sustained gains in compliance with standards and outcomes in less-developed settings and merits wider application as a strategy for health systems strengthening.
Fraser SW. 2007. "Undressing the Elephant: Why the spread of good practice isn’t working in healthcare; presenting symptoms and suggested treatment" Lulu Press.
The premise of this book is that the concept of "spread" and the business of "spreading good practice" as we continue to apply it in healthcare, may be a large part of why "spread" doesn't happen. If you're looking for an alternative and occasionally irreverent view, then read on. This is a personal account reflecting on many years worth of consulting practice, mistakes included. aims behaviour better ideas blood letting change management change process chapter communication context Crossing the Canyon customisation developed diabetes difficult Diffusion of Innovations DoI engagement ensure example expect facilitation feedback focus FURTHER READING goal happens idea bias identified important improvement projects influence initiatives instant messaging IPOD issues learning look management rollout mean measurement plan medical errors models niche nini’s norm opinion leaders opinion leadership organisation Pareto Pareto analysis Pareto chart patients PDSA cycles pilot sites population of impact potential adopters pragmatists PRESENTING SYMPTOMS PREVENTING OCCURRENCES primary care practices problem process measures PROGNOSIS project management project team quality improvement reinvention Seth Godin social networking solution specific spread and adoption spread phase spread plan spread programs spreading good practice strategy successful change SUGGESTED TREATMENT target population techniques theory topic virtual virtual teams
Frost LJ, Reich M. 2008. "Access: How do good health technologies get to poor people in poor countries?" Harvard University Press, Cambridge, MA, FutureGenerations.
Many people in developing countries lack access to health technologies even basic ones. Why do these problems in access persist? What can be done to improve access to good health technologies especially for poor people in poor countries? This book answers those questions by developing a comprehensive analytical framework for access and examining six case studies to explain why some health technologies achieved more access than others. The technologies include praziquantel (for the treatment of schistosomiasis) hepatitis B vaccine malaria rapid diagnostic tests vaccine vial monitors for temperature exposure the Norplant implant contraceptive and female condoms. Based on research studies commissioned by the Bill & Melinda Gates Foundation to better understand the development adoption and uptake of health technologies in poor countries the book concludes with specific lessons on strategies to improve access. These lessons will be of keen interest to students of health and development public health professionals and health technology developers -- all who seek to improve access to health technologies in poor countries.
Ganju E, Heyman C. 2018. "Scaling Global Change: A Social Entrepreneur’s Guide to Surviving the Start-up Phase and Driving Impact." Wiley.
Grow your start-up into a global influence with real-world impact Scaling Global Change provides social entrepreneurs with the strong organizational foundation they need to change the world. Through the story of Room to Read, one of the fastest-growing nonprofits in the last 18 years, this book features clear, real-world lessons for growing a non-profit or social enterprise, with special insight into girls’ education and literacy programming in lower-income countries. By outlining theories of program, operational, and system-level change, the discussion delves into the meat of the entrepreneurial spirit and applies it directly to everyday strategic decisions. The book begins with an overview of essential communication, vision, and execution fundamentals, and then dives into a discussion of metrics, monitoring, planning, leadership, and more. Clear guidance on internal operations, fundraising, team building, management, and other central topics provides a roadmap for new and experienced leaders, while further exploration of influence, strategy, and government funding relates the wisdom of experience from the perspective of a successful organization. Cross the chasm from start-up to mature organization with worldwide impact Gain insight into the theoretical and practical underpinnings of nonprofit success Adopt new perspectives on effectiveness, excellence, and influence Translate ideas into action in a way that will change the world Social entrepreneurship has taken off more than ever, and the market is crowded with optimistic leaders wanting to change the world. How do you differentiate your organization from the pack? How can you stand out, stand up, and make a real impact? These lessons are gained through experience and building a strong organizational culture; Room to Read has treaded this path and found itself at the heights of success. With Scaling Global Change, you reap the benefit of experiential lessons while applying them to the success of your own organization.
Garcia JR. 2019. "Are We Bigger Yet? A Behavioral Approach to Assessing Scaling-Up." Earth-Eval.
As program designers and implementers, how do we make sure our intervention will scale up? As program evaluators, how do we know if something is going to scale up? In 2018, we at the GEF IEO decided to find out by looking at completed projects that reported both successful and less successful scaling outcomes.

We identified factors mentioned in 18 interviews with GEF Secretariat teams and GEF Agencies at the corporate level. We then validated these factors using 20 cases built primarily from document reviews, supported by evidence from an additional 40 cases. Six of the cases involved field visits and interviews specifically to assess whether or not--and how--scaling activities were sustained post-GEF support. Each case consisted of one or more linked projects.
Garcia JR. 2021. "How does the GEF help Scale Up impact? A Case Study-Based Evaluation." GEF Independent Evaluation Office.
Key findings of evaluation

The Global Environment Facility’s (GEF’s) focus on scaling is more explicit than in many other international development institutions. But like other institutions, the GEF’s vision for scaling-up is not consistently clear in operational guidance across its portfolio.

GEF support to scaling-up activities has varied widely in terms of grant amount, implementation period, and project modality. Overall, GEF-supported initiatives typically last longer than five years and leverage higher cofinancing ratios at the scaling-up stage.

The GEF has contributed to postproject continuation of scaled-up activities by catalyzing sustainable financing sources and strengthening institutional capacities. Political and economic changes pose risks to long-term sustainability of scaling-up activities.

The GEF’s comparative advantage lies in supporting pilots that demonstrate positive benefits and establishing enabling conditions for scale-up. These strengths attract support from other actors that then provide funding for full scale-up.

Cases generally reported a higher magnitude of environmental outcomes per dollar per year during scale-up versus piloting. The extent of scaling could not be measured in one case because common environmental indicators were lacking between pilot and scaling projects.

The GEF partnership benefits from its Agencies having different comparative advantages in different modes of scaling.
Garcia JR. 2019. "Want to scale up? Change-proof your program! Lessons from the GEF (PART 1)." Earth-Eval.
Change is the only constant, we all learn sooner or later. But what to do when political and economic changes threaten the success of your carefully-implemented program?

In 2018, we at the GEF IEO examined how impact is scaled up across different types of environmental interventions funded by the GEF. We looked at 20 cases in-depth, supported by evidence from an additional 40 cases, to assess which factors mattered the most in whether something was scaled up or not. It turns out that what matters for scaling up impact is very similar to what makes an intervention successful or not to begin with.

In 12 out of 20 cases that experienced some political or economic shift during project implementation or scaling-up, two things made a difference: high political priority given to the intervention by supporting institutions, and ownership by long-term staff in these institutions.
Garcia JR. 2019. "Want to scale up? Change-proof your program! Lessons from the GEF (PART 2)." Earth-Eval.
When we at the GEF IEO looked at how impact was scaled up in GEF-supported interventions, many of the interviewees used terms such as “magic moment”, “luck” or “perfect storm” when referring to how scaling-up happened “spontaneously” through serendipitous circumstances, even when the project itself had no concrete plans for scaling-up. On the other hand, we also know how seemingly successful programs can quickly fall apart under political and economic changes.

How do we reduce the negative effects of such changes – and maybe even turn them into “lucky” positive outcomes? In the previous post, we talked about how contextual factors such as high political priority and a sense of ownership among key stakeholders can be developed through program activities, thus catalyzing support that can weather these changes.

In this post, we discuss how sometimes the solution is not in more program activities, but in using the existing context to your advantage: first, by choosing the right institutions and individuals to partner with, and second, by leveraging current priorities and trends to align with your program’s target outcomes.
Gargani J & McLean R. 2017. "Scaling Science." Features. Stanford Social Innovation Review.
In early 2014, the Ebola virus began its devastation of West Africa, moving through countries, communities, and families with grim efficiency. Over the next two years, 60 percent of those infected with the virus died—more than 11,000 people. A brutal killer, Ebola renders its victims delirious and unable to cope on their own. One of the hardest-hit countries was Sierra Leone, which had just 136 doctors for more than 6 million inhabitants. Almost immediately, it fell to family and friends to act as caregivers. Ebola killed them, too. In the worst-hit areas, the virus eliminated entire families. Those who fell ill started running off to die alone rather than risk infecting loved ones. Eventually, social gatherings were banned, schools were closed, and households were separated. Society and the economy ground to a halt. The crisis was unprecedented. Since Ebola was first detected in 1976, each of the subsequent 27 outbreaks was stopped in less than three months—until 2014. Why did this outbreak last for two years and kill more than all previous outbreaks combined? A complete answer has yet to emerge, but two factors were critical. First, we lacked know-how. There were no preexisting, evidence-based solutions to combat an outbreak of this magnitude. Second, the context was pernicious. A variety of circumstances, including unprepared health systems at the national level and social disintegration, compounded the problem and destabilized even the most holistic solutions. In these types of circumstances, the way we usually scale solutions is ineffective. The traditional approach to delivering interventions at scale starts with the assumption that we have reliable solutions and favorable contexts. When this is the case, as it sometimes is, we are urged to scale what works by efficiently allocating resources to organizations with evidence-based solutions. But as the Ebola crisis shows, this is not always the case. Many of our most pressing problems are the ones we have been unable to solve, perhaps for years, for decades, or longer. Most are not crises on par with an Ebola outbreak, but fixtures of the status quo. Issues that in the development sphere are often called “wicked problems.” So, how do we scale when we don’t know what works?
Gaye PA, Nelson D. 2009. "Effective scale-up: Avoiding the same old traps." Human Resources for Health, 7:2.
Despite progress in developing more effective training methodologies, training initiatives for health workers continue to experience common pitfalls that have beset the overall success and cost-effectiveness of these programs for decades. These include lack of country-level coordination of health training, inequitable access to training, interrupted services, and failure to reinforce skills and knowledge training by addressing other performance factors. These pitfalls are now seen as aggravating the current crisis in human resources for health and impeding the effective scale-up of training and the potential impact of promising strategies such as task shifting to address health worker shortages. Drawing on IntraHealth International's lessons learned in designing reproductive health and HIV/AIDS training and performance improvement programmes, this commentary discusses promising practices for strengthening human resources for health through more efficient and effective training and learning programmes that avoid the same old traps. These promising practices include the following: Assessing performance gaps and opportunities before designing a training initiative; addressing performance factors other than skills and knowledge that health workers need to perform well; applying a "learning for performance" approach; standardizing curricula throughout a country; linking pre-service education, in-service training and professional associations; enhancing traditional education; strengthening human resources information systems to improve workforce planning, policies and management; applying technology to meet training needs.
George A, Menotti EP, Rivera D, Marsh DR. 2011. "Community case management in Nicaragua: Lessons in fostering adoption and expanding implementation." Health Policy and Planning, 26: 327-337.
Community case management (CCM) as applied to child survival is a strategy that enables trained community health workers or volunteers to assess, classify, treat and refer sick children who reside beyond the reach of fixed health facilities. The Nicaraguan Ministry of Health (MOH) and Save the Children trained and supported brigadistas (community health volunteers) in CCM to improve equitable access to treatment for pneumonia, diarrhoea and dysentery for children in remote areas. In this article, we examine the policy landscape and processes that influenced the adoption and implementation of CCM in Nicaragua. Contextual factors in the policy landscape that facilitated CCM included an international technical consensus supporting the strategy; the role of government in health care provision and commitment to reaching the poor; a history of community participation; the existence of community-based child survival strategies; the decentralization of implementation authority; internal MOH champions; and a credible catalyst organization. Challenges included scepticism about community-level cadres; resistance from health personnel; operational gaps in treatment norms and materials to support the strategy; resource constraints affecting service delivery; tensions around decentralization; and changes in administration. In order to capitalize on the opportunities and overcome the challenges that characterized the policy landscape, stakeholders pursued various efforts to support CCM including sparking interest, framing issues, monitoring and communicating results, ensuring support and cohesion among health personnel, supporting local adaptation, assuring credibility and ownership, joint problem solving, addressing sustainability and fostering learning. While delineated as separate efforts, these policy and implementation processes were dynamic and interactive in nature, balancing various tensions. Our qualitative analysis highlights the importance of supporting routine monitoring and documentation of these strategic operational policy and management issues vital for CCM success. We also demonstrate that while challenges to CCM adoption and implementation exist, they are not insurmountable. Community case management, child health, policy analysis, scaling up implementation, Nicaragua
Gerhager B, Klien S, Stahl J. 2018. "Guidelines on scaling-up for programme managers and planning officers." GIZ.
Scaling-up is a managed process designed to mainstream an innovation through a project and generate broad impact. In the development cooperation context, the term ‘innovation’ does not refer to the invention of new technologies or products in the traditional sense. Instead, it involves changes within organisations and cooperation systems and in the way they deliver their services. The term ‘innovations’ might refer to strategies, approaches, methods or lessons learned that are new in a specific context. Significant changes need to be made to established routines within an area of social concern if these innovations are to be mainstreamed.
Gericke CA, Kurowski C, Ranson MK, and Mills A. 2005. "Intervention Complexity: A Conceptual Framework to Inform Priority-setting in Health." Bulletin of the World Health Organization, 83, 285-293.
Health interventions vary substantially in the degree of effort required to implement them. To some extent this is apparent in their financial cost, but the nature and availability of non-financial resources is often of similar importance. In particular, human resource requirements are frequently a major constraint. We propose a conceptual framework for the analysis of interventions according to their degree of technical complexity; this complements the notion of institutional capacity in considering the feasibility of implementing an intervention. Interventions are categorized into four dimensions: characteristics of the basic intervention; characteristics of delivery; requirements on government capacity; and usage characteristics. The analysis of intervention complexity should lead to a better understanding of supply- and demand-side constraints to scaling up, indicate priorities for further research and development, and can point to potential areas for improvement of specific aspects of each intervention to close the gap between the complexity of an intervention and the capacity to implement it. The framework is illustrated using the examples of scaling up condom social marketing programmes, and the DOTS strategy for tuberculosis control in highly resource-constrained countries. The framework could be used as a tool for policy-makers, planners and programme managers when considering the expansion of existing projects or the introduction of new interventions. Intervention complexity thus complements the considerations of burden of disease, cost-effectiveness, affordability and political feasibility in health policy decision-making. Reducing the technical complexity of interventions will be crucial to meeting the health-related Millennium Development Goals.

Keywords: Health priorities/organization and administration; Delivery of health care/organization and administration; Condoms; Social marketing; Directly observed therapy; Tuberculosis, HIV infections/prevention and control; Multidrug-resistant/prevention and control; Models, Theoretical; Developing countries
Gericke CA, Kurowski C, Ranson MK, Mills A. 2003. "Feasibility of scaling-up interventions: The role of intervention design" Berlin University of Technology, Germany, and London School of Hygiene and Tropical Medicine, U.K.
Intervention complexity is a useful way to think about feasibility –It complements burden of disease, cost-effectiveness, and affordability considerations –It can help to identify R&D priorities to simplify interventions –It can guide decisions on how to implement interventions in specific settings Intervention complexity is a useful additional criterion for decision making on scaling-up health interventions
Ghiron L, Ramirez-Ferrero E, Badiani R, Benevides E, Ntabona A, Fajans P, Simmons R. 2021. "Promoting Scale‑Up Across a Global Project Platform: Lessons from the Evidence to Action Project." Global Implementation Research and Applications (2021) 1:69–76.
The USAID-funded flagship family planning service delivery project named Evidence to Action (E2A) worked from 2011 to 2021 to improve family planning and reproductive health for women and girls across seventeen nations in sub-Saharan Africa using a “scaling-up mindset.” The paper discusses three key lessons emerging from the project’s experience with applying ExpandNet’s systematic approach to scale up. The methodology uses ExpandNet/WHO’s scaling-up framework and guidance tools to design and implement pilot or demonstration projects in ways that look ahead to their future scale up; develop a scaling-up strategy with local stakeholders; and then strategically manage the scaling-up process. The paper describes how a scaling-up mindset was engendered, first within the project’s technical team in Washington and then how they subsequently sought to build capacity at the country level to support scale-up work throughout E2A’s portfolio of activities. The project worked with local multi-stakeholder resource teams, often led by government officials, to equip them to lead the scale-up of family planning and health system strengthening interventions. Examples from project experience in the Democratic Republic of the Congo, Kenya, Nigeria, and Uganda illustrating key concepts are discussed. E2A also established a community of practice on systematic approaches to scale up as a platform for sharing learning across a variety of technical agencies engaged in scale-up work and to create learning opportunities for interacting with thought leaders around critical scale-up issues.

Keywords Scaling up framework
· Implementation
· ExpandNet
· E2A
· Family planning
· Capacity building
· Country ownership
· Mindset
Ghiron L, Shilingi L, and Kabiswa C. 2014. "Beginning with sustainable scale up in mind: initial results from a population, health and environment project in East Africa." RH Health Matters, 2(432)8492.
Small-scale pilot projects have demonstrated that integrated population, health and environment approaches can address the needs and rights of vulnerable communities. However, these and other types of health and development projects have rarely gone on to influence larger policy and programme development. ExpandNet, a network of health professionals working on scaling up, argues this is because projects are often not designed with future sustainability and scaling up in mind. Developing and implementing sustainable interventions that can be applied on a larger scale requires a different mindset and new approaches to small-scale/pilot testing. This paper shows how this new approach is being applied and the initial lessons from its use in the Health of People and Environment in the Lake Victoria Basin Project currently underway in Uganda and Kenya. Specific lessons that are emerging are: 1) ongoing, meaningful stakeholder engagement has significantly shaped the design and implementation, 2) multi-sectoral projects are complex and striving for simplicity in the interventins is challenging, and 3) projects that address a sharply felt need experience substantial pressure for scale up, even before their effectiveness is established. Implicit in this paper is the recommendation that other projects would also benefit from applying a scale-up perspective from the outset. © 2014 Reproductive Health Matters

Keywords: scaling up of services, pilot projects, health policy and programmes, population, health and environment, scale, development, WHO Strategic Approach, Kenya, Uganda
Gillespie D, Karklins S, Creanga A, Khan S, Cho NH. 2007. "Scaling Up Health Technologies: Report and Bibliography." The Bill and Melinda Gates Institute for Population and Reproductive Health, Johns Hopkins Bloomberg School of Public Health.
Since the visionary Alma Alta declaration of Health for All by the Year 2000 three decades ago, there has been much progress in improving the health of the world. There have also been many disappointments. The poor and much of sub-Saharan Africa and South Asia are still characterized by their high mortality rates. The failure to achieve past goals and likely failure to achieve future health goals like the Millennium Development Goals (MDGs) is frustrating because with each declaration, the international health community insists that those goals can be met. Unlike Winston Churchill, who in the darkest days of World War II, said, “Give us the tools and we will finish the job,” international health experts largely feel we have most of the tools needed to finish the job. If we have the tools to “finish the job,” then why haven’t we? And, what steps can be taken to “finish the job?” These are the two principle questions this report will address. For this report, we analyzed both successful and unsuccessful maternal and child health efforts to increase the accessibility of effective, mortality-reducing interventions. Although we drew upon the experiences of countries in Southeast Asia and Central and Latin America, our focus throughout the paper was on sub-Saharan Africa and South Asia, the regions with the most severe and challenging maternal and child health problems. The review of scaling up1 experiences in the last several decades has many lessons relevant for the future. Interventions that have been brought to scale have many things in common. The intervention is an easy to use, inexpensive, and effective product for a specific disease or condition that lends itself to a vertical, campaign-type of delivery system. We found examples of successful scaled-up efforts, but outside of the small pox and polio campaigns, few that were universally successful. Some countries achieved excellent coverage, others failed. More disturbing were instances in which successful scaling up of an intervention was achieved, but not sustained. The fact that health technologies are effectively scaled up and sustained in some countries but not in others suggests that factors external to the health technology determine the success of an intervention. A culling of less promising health technologies being championed for reducing maternal and child mortality is advisable since the requisite wherewithal to bring them all to scale is lacking now and in the foreseeable future, even assuming a much larger global commitment. It is also recommended that large scale research be conducted to determine the scaling up feasibility of promising, efficacious interventions which are unavailable to large populations.
Gillespie S. 2004. "Scaling Up Community-Driven Development: A Synthesis of Experience." Washington, D.C.: International Food Policy Research Institute.
While many Community-Driven Development (CDD) initiatives may be successful, their impact is often limited by their small scale. Building on past and ongoing work on CDD, this study addresses the fundamental question: how can CDD initiatives motivate and empower the greatest number of communities to take control of their own development? What are the key contextual factors, institutional arrangements, capacity elements and processes related to successful scaling up of CDD, and, conversely, what are the main constraints or limiting factors, in different contexts? Drawing upon recent literature and the findings from five case studies, key lessons on how best to stimulate, facilitate and support the scaling up of CDD in different situations, along with some major challenges, are highlighted. Lessons include the need for donors and supporters of CDD including governments to think of the process beyond the project, and of transformation or transition rather than exit. Donor push and community pull factors need to be balanced to prevent \"supply-driven demand-driven development\". Overall, capacity is pivotal to successful CDD and its successful scaling up over time. Capacity is more than simply resources -- it includes motivation and commitment which in turn requires appropriate incentives at all levels. Capacity development takes time and resources but is an essential upfront and ongoing investment, with the capacity and commitment of facilitators and local leaders being particularly important. A \"learning by doing\" culture - one that values adaptation, flexibility and openness to change -- needs to be fostered at all levels, with time horizons adjusted accordingly. The building of a library of well-documented context-specific experiences through good monitoring, evaluation and operational research will be useful in advocating for improvements in the contextual environment.
Gilson l, Schneider H. 2010. "Commentary: Managing scaling up: what are the key issues?" Health Policy and Planning 25(2):97-98.
We want to pick up and expand on two sets of inter-related issues raised in the paper by Mangham and Hanson (2010): the importance of political commitment and of strategic management to scaling up. As the paper makes clear, these issues are integral to successful scaling up. Evidence on progress towards the Millennium Development Goals, for example, demonstrates the need for strong government leadership, whilst policy and strategic management is one of the five barriers recognized to constrain expansion of health interventions’ coverage. But, as the paper also shows, within scaling up debates little attention has so far been paid either to how to generate political commitment to scale up public health interventions, or to the nature of the strategic management also required to support this process. Yet there is (some) relevant health evidence, as well as relevant theory, around policy implementation (e.g. Hill and Hupe 2002) and innovation (Greenhalgh et al.2004), from which to draw useful insights. First, we look at political commitment. For low- and middle-income settings there is a relatively rich body of empirical work examining how policy agendas are set, including the transfer of policy ideas between international and national jurisdictions (Gilson and Raphaely 2008). This work critiques the idea that political will is the central determinant of successful policy change (Reich 1995). Instead, it shows how political commitment to a new policy or programme has to be actively created, essentially through advocacy. Examining maternal health policy experience, Shiffman (2007) highlights the importance of alliance building to consolidate influence, working with political entrepreneurs to get public health issues onto policy agendas, using credible measures to demonstrate the severity of the problem addressed through a new programme and avoid denial of its importance, organizing focusing events to publicize the issue and presenting policy makers with policy alternatives of proven effectiveness which show them the problem can be addressed. These sorts of advocacy efforts may also be important once an issue or programme is on the policy agenda, to secure sustained domestic budgetary commitments (Crichton 2008). When international actors seek to influence national policy agendas, they are, therefore, likely to be more effective when they work as advocates of change rather than adopting more coercive approaches (Walt et al.2004).
Glaser EM, Abelson HH, Garrison KN. 1983. "Putting Knowledge to Use: Facilitating the Diffusion of Knowledge and the Implementation of Planned Change." San Francisco, California: Jossey-Bass Inc.
In their book, Putting Knowledge to Use, Glaser, Abelson, and Garrison undertake a large and important task. The intent of the book is (1) to review the extensive literature on knowledge use and the design of social change over the last decade, (2) to organize the research findings from multiple disciplines on these problems in a coherent, theoretical framework, and in so doing, (3) to create a new basis for understanding the complex, dynamic process of social learning and organizational change. The need to understand the process of using systematic social inquiry to solve actual social problems has been recognized for more than a decade. Argyris (1970, 1982), Rivlin (1971), Lindblom and Cohen (1979), Weiss (1977), Tornatzky (1980), and others have all identified the problems involved in translating new and relevant knowledge into existing social practice in order to revitalize the institutions, effectiveness, and creativity of our social order. The urgency of this need is underscored by the rapid proliferation of information and new knowledge made possible by social science research and technical advances in informa-tion processing and telecommunications (Bell, 1973, 1976). Yet, the potential for achieving effective utilization of this knowledge is limited by our ability to assess and apply the relevant knowledge to appropriate social tasks in timely, accurate fashion. Increasing our ability to do so requires a reformulation of the use of knowledge in social interaction. It is precisely this conceptual reformulation of the use of knowledge in contemporary social practice that the authors undertake in this volume. The design of the book represents a rational effort to decompose this large, complex problem into understandable components. In a brief, thoughtful introduction, the authors outline their conception of the task and scope of knowledge utilization and its generation of social change. Knowledge is broadly defined as facts, information,
Global Community of Practice on Scaling Development Outcomes. 2021. "Resource Directory."
Webinars on crosscutting issues. Agriculture and Rural Development (ARD) Working Group Climate Change Working Group Education Working Group Fragile States Working Group Health Working Group Monitoring and Evaluation (M&E) Working Group Nutrition Working Group Social Enterprise Scaling Up Working Group Youth Employment Working Group The Scaling Up Community of Practice (CoP) was founded six years ago by Larry Cooley and Johannes Linn. The purpose of the CoP is to provide a platform for knowledge exchange among experts and practitioners on approaches to scaling up development interventions, for developing partnerships, and for championing the idea that scaling up development impact is critical for achieving global development aspirations, such as the Sustainable Development Goals and climate change aspirations. Interest in scaling has greatly increased over these six years. The CoP has now more than 700 members from over 200 institutions (including bilateral and multilateral development organizations, operating NGOs, grant making foundations, universities and think tanks) and from many different sectoral and thematic areas of professional expertise. The CoP has organized four annual two-day in-person workshops in the Washington Metropolitan Area and currently operates nine sectoral and thematic Working Groups (education, health, agriculture, nutrition, social enterprises, youth employment, climate change, fragile states, and monitoring and evaluation) each of which meets virtually three times a year, as well as in-person during past CoP workshops. The CoP has issued eighteen Newsletters to date, sharing news about scaling up research and practice from the Working Groups, from CoP members, and from the broader development community. Three years ago, an Executive Committee was formed from among the membership. Management Systems International (MSI) has generously provided administrative and logistical support since the establishment of the CoP. There is strong support among the membership for the continuation and strengthening of the CoP. The CoP receives financial and in-kind support from a number of organizations and individuals, which has allowed it to host its annual workshop, invite and support southern participants, and develop and support a website.
Gogovor A, Zomahoun HTV, Ben Charif A, McLean RKD, Moher D, Milat A, Wolfenden L, Prévost K, Aubin E, Rochon P, Ekanmian G, Sawadogo J, Rheault N, Légaré F. 2020. "Essential items for reporting of scaling studies of health interventions (SUCCEED): protocol for a systematic review and Delphi process."
The lack of a reporting guideline for scaling of evidence-based practices (EBPs) studies has prompted the registration of the Standards for reporting studies assessing the impact of scaling strategies of EBPs (SUCCEED) with EQUATOR Network. The development of SUCCEED will be guided by the following main steps recommended for developing health research reporting guidelines.

Executive Committee. We established a committee composed of members of the core research team and of an advisory group. Systematic review. The protocol was registered with the Open Science Framework on 29 November 2019 ( We will include reporting guidelines or other reports that may include items relevant to studies assessing the impact of scaling strategies. We will search the following electronic databases: EMBASE, PsycINFO, Cochrane Library, CINAHL, Web of Science, from inception. In addition, we will systematically search websites of EQUATOR and other relevant organizations. Experts in the field of reporting guidelines will also be contacted. Study selection and data extraction will be conducted independently by two reviewers. A narrative analysis will be conducted to compile a list of items for the Delphi exercise. Consensus process. We will invite panelists with expertise in: development of relevant reporting guidelines, methodologists, content experts, patient/member of the public, implementers, journal editors, and funders. We anticipated that three rounds of web-based Delphi consensus will be needed for an acceptable degree of agreement. We will use a 9-point scale (1 = extremely irrelevant to 9 = extremely relevant). Participants’ response will be categorized as irrelevant (1–3), equivocal (4–6) and relevant (7–9). For each item, the consensus is reached if at least 80% of the participants’ votes fall within the same category. The list of items from the final round will be discussed at face-to-face consensus meeting. Guideline validation. Participants will be authors of scaling studies. We will collect quantitative (questionnaire) and qualitative (semi-structured interview) data. Descriptive analyses will be conducted on quantitative data and constant comparative techniques on qualitative data.

Essential items for reporting scaling studies will contribute to better reporting of scaling studies and facilitate the transparency and scaling of evidence-based health interventions.
Gonsalves J. 2000. "Going to scale: can we bring more benefits to more people more quickly?" Workshop highlights presented by the CGIAR-NGO Committee and The Global Forum for Agricultural Research with BMZ (German Ministry of Development Corporation), MISEREOR (German Catholic Church Development Agency), Rockefeller Foundation, IRRI (International Rice Research Institute), and IIRR (International Institute of Rural Reconstruction). 10-14 April, IIRR , Philippines.
Participants at the workshops made it clear that appropriate technologies and approaches are available that can lead to the production of more food by poor farmer groups in marginal environments without creating dependence on external inputs and without damaging the natural resource base.
The recommendations and conclusions give guidelines on how to proceed with scaling up processes.
Graham T. 2018. "How to design for scale: lessons for ambitious new interventions." Apolitical.
In many respects, the so-called Graduation Program has been a roaring success: it’s the only intervention proven to lift people out of extreme poverty. With food parcels, life skills classes and two years of asset transfers, it’s a complete package. Almost 100 countries have experimented with it, providing a rock-solid evidence base. So why isn’t it national policy across the world? The biggest determinant of whether an intervention will succeed at a huge scale, changing the lives of millions rather than dozens or hundreds or even thousands, is not whether it works, but whether it was designed in a way that made it suitable for scale in the first place. The path from developing a program that works to helping huge numbers of people is a long one, often taking 10 to 15 years. Organisations that don’t anticipate the difficulties will develop programs with features that make them inherently difficult or even impossible to scale up. The Graduation Program, for example, is expensive. And now governments that want to implement it are trying to cut costs — but they don’t know what to cut and what to keep without losing the effects. For reasons like this, success on a small scale is no indication, let alone a guarantee, of success on the big stage. Only interventions that are relatively simple, clearly better than the alternatives and not reliant on unique conditions are likely to scale well. But it can be done. The secret is to plan for that long journey right at the beginning and to design something that will not just survive, but thrive as it gets bigger. So how do you do that?
Graham T. 2018. "The world is scattered with pilot projects trying to work holistically." Apolitical.
Q&A with Karen Levy of Evidence Action, the NGO that delivers solutions to hundreds of millions of people.
Graham T. 2018. "You can’t worry about scaling second — you have to worry right at the start." Apolitical website blog.
In wealthy and developing countries alike, service providers are under pressure to reduce costs, improve social outcomes and explain why it has proven so difficult to accelerate the spread of best practices. In 2003, the MacArthur Foundation gave one of its famous grants to an unusual recipient: a private company called Management Systems International (MSI). Their mission? To develop and test a framework to improve the scaling up process. That grant turned out to be the first of five. MSI was founded by Larry Cooley, who has worked in more than 50 countries in strategic management, public sector reform and international development. That includes serving as an advisor to cabinet officials in more than a dozen countries, and overseeing a nine-year effort to rebuild Iraqi public administration. Cooley currently curates a global community of practice on scaling up development outcomes and serves as the scaling advisor for the MacArthur Foundation’s 100&Change grant, which recently awarded $100 million to The Sesame Workshop to bring their early childhood intervention to Syrian refugees. He spoke to Apolitical about what he’s learned over nearly three decades grappling with the question of scale. You’ve been applying your framework for over a decade. What’s the most important thing you’ve learned? You can’t worry about scaling second — you have to worry about it right at the beginning. There’s a model that I used for a long time that talked about three steps: effectiveness, efficiency, expansion. First build something that works, then figure out how to produce it efficiently, and then come up with a plan to spread it. And that made perfectly good sense to me then — but I now think it’s wrong. \"If you don’t worry about how it will spread at the very beginning, you will develop something inherently difficult to scale\" If you don’t worry about how it will spread at the very beginning, you will develop something with features that make it inherently either impossible or difficult to scale. This is a very difficult row to hoe, psychologically, because it means you have to plan for scale and nevertheless remain sceptical about whether the thing you’re working on should be scaled. Because there’s another problem: people fall in love with their pilots. So as soon as they start down the road, they’re not only planning for scale but that’s their success standard. Then they’re too quick on the draw: they’re anxious to scale prior to the time that they should. \"People fall in love with their pilots\" And, if you look at the amount of time it takes for things to go from a good idea, even a great idea, to a scaled up solution, the mode is about 15 years. That’s much longer than the time horizon of most interveners. Figuring out strategies that keep things in motion and commitments in place for as long as it takes to see an innovation all the way through the system is still developing as an art form.
Gray J. 2010. "Global health experts seek to transform programs through implementation science." Fogarty International Center. Global Health Matters Newsletter, Volume 9, Issue 2.
Implementation science, or research to translate evidence-based findings into common practice, is quickly becoming an integral component of many global health programs. This novel approach to health research addresses knowledge gaps between innovations such as vaccines, drugs and care strategies and their delivery to patients and communities. Studies continue to show, for example, that the risk of HIV transmission can be reduced through condom use, yet infection rates continue to rise in some regions of the world. Research on implementation identifies barriers to proven interventions and facilitates the creation of local strategies to overcome them. The concept of translating research into practice and policy was discussed in-depth at a recent two-day implementation and dissemination conference sponsored by NIH and a subsequent Fogarty satellite meeting focused on implementation science in the global health context. "It's an area ripe for exploration," said NIH Director Dr. Francis Collins. "Implementation science is part of our mission." Collins says it's important to ensure that research reaches communities in need, as well as conducting research to determine what works and what doesn't in the real world. The NIH conference, now in its third year, drew approximately 630 attendees and focused on methods and measures. Over 50 presenters showcased research and health care delivery projects addressing a wide range of issues, including war-related trauma, stroke care, community mental health, homelessness, STD prevention, family planning and school-based interventions. "Research must combine relevance to decision-making," said Dr. Julio Frenk, Dean of the Harvard School of Public Health, drawing from his experiences establishing highly successful, large-scale health programs in his native Mexico. "Excellence and relevance can have a fruitful relationship through knowledge translation." Implementation science is of particular importance to global health as the concept addresses hard questions such as how best to translate new findings into practice in different cultural settings, how to reduce the fragmenting of health systems into programs centered on a single disease - particularly in low-resource settings - and why health interventions lose efficacy over time or sometimes display unintended effects. Lynn Freedman, professor of clinical population and family health at Columbia University, warned of an additional challenge, namely, the "dangerous fallacy" that technical input, political will and money will amount to functioning services. She used the analogy (see Figure 1: Chart of Problems with Varying Levels of Complexity) of following a recipe, launching a rocket to the moon and raising a child as three problems with different levels of complexity. Implementation science falls into the most challenging category.
Greenhalgh T, Papoutsi C. 2019. "Spreading and scaling up innovation and improvement." BMJ 2019; 365 :l2068.
Key messages
Spread (replicating an intervention) and scale-up (building infrastructure to support full scale implementation) are difficult Implementation science takes a structured and phased approach to developing, replicating, and evaluating an intervention in multiple sites Complexity science encourages a flexible and adaptive approach to change in a dynamic, self organising system Social science approaches consider why people act in the way they do, especially the organisational and wider social forces that shape and constrain people’s actions These approaches may be used in combination to tackle the challenges of spread and scale-up
Growth Philanthropy Network, Duke University. 2012. "Scaling Social Impact: A Literature Toolkit for Funders." Growth Philanthropy Network, Duke University.
The literature around scaling strategies for nonprofits relevant to grantmakers is vast. Our literature review included reading and reviewing every piece of work in the Social Impact Exchange’s 175 item online database, plus selected reviews of other published content from many of the Exchange’s Knowledge Working Group’s member organizations’ websites and data repositories. (The Knowledge Working Group and its members are detailed in Appendix B. More detail on our methodology is in Appendix D.) This executive summary offers our overall observations about the state of knowledge from these sources and some of the themes that emerged. The Majority of the Scaling Literature is Written for Practitioners, Not Funders. A majority of the existing literature in our database review is written for a broad nonprofit audience. As a result, the knowledge base on scaling is mostly focused on providing nonprofit leaders, practitioners, program directors and managers with the information they need to undertake a scaling initiative, not on discussing the funder’s role in supporting scaling initiatives. Still, There are Significant Resources Relevant to Funders. Approximately one third of all the literature in the database addresses roles of the funder. Of that third, the resources that were written specifically for grantmakers or funders are generally written by foundations, or were commissioned by foundations and written by consulting firms, evaluation firms, and intermediaries. Our editorial filter for this review was to include the materials we judged to be most useful to funders and grantmakers interested in supporting scaling initiatives. Therefore, this review is not a comprehensive listing of all content on scaling, but an editorialized selection, based on pieces we believe reveal fundamental, generalizable frameworks, theories and lessons for grantmakers that are communicated in effective, readable ways. For this reason, this review does not include many purely academic studies. Evolution of Literature from Scaling Organizations, to Scaling Impact: The earlier literature on scaling focuses on scaling organizations – for example, the literature in the Exchange’s database dates to 1994, with a report published by Public/Private Ventures on the strategy of replicating social programs. However, in the last decade or so, the literature has shifted to include a broader definition of scale – moving away from the concept of scaling as organizational growth and towards the concept of scaling impact, or the outcomes the organization has generated beyond just the organization itself.
Guerzovich F & Aston T. 2022. "How context shapes pathways to scale in social accountability." Medium.
In this fourth post in our series on pathways to scale social accountability, we discuss context. We argue that context influences which pathways to scale are more likely to be effective at a particular point in time. We believe that, for selecting the preferred pathway, there are four especially important aspects/dimensions of context that are worth exploring further. We highlight these four factors in the table and discuss them at greater length below.
Guerzovich F & Aston T. 2021. "Pathways to scale in social accountability." Medium.
Scale is a complex change process that is often misunderstood. We need to better understand what it looks like in practice, whether, how, and under what conditions (by whom, where, when) social accountability might be scaled up. Better understanding pathways to scale is the theme of this 5-part blog series.
Guerzovich F & Aston T. 2022. "Scale up in Time: Revisiting how we Evidence Process & Context." Medium.
This is the last post of a 5-part series about scale in social accountability, and more broadly the transparency, participation, and accountability (TPA) sector. We started by discussing what scale looks like in the real world. We explained how, in our view, there is no single pathway to scale, or quoting Byrne (2013: 1) “different mechanisms [or paths can] produce different outcomes and… [the] same outcome… the search for the — that is to say universal, always and everywhere, nomothetic — model that fits the data” is in vain. Ultimately, no single model fits the data. We then presented multiple paths through which scale up may happen (resonance, resistance, and best practice). Lastly, we focused on a handful of contextual factors that may help practitioners to understand under which conditions each path might be a better bet. In this post, we argue that politics in the transparency, participation, and accountability field happen in time. There is more to politics and change in time than black and white debates about short and long term support. Temporal contexts are about stability and change, timing time horizons, sequences, feedback loops, gradualism vs. shock therapies, etc. They provide a concrete way to to grapple with how agents contribute to scale through different pathways in complex, uncertain systems that make up a world of greys and mixed results — i.e. the world of much TPA work.
Hanson K, Ranson MK, Oliviera-Cruz V, Mills A. 2003. "Expanding access to priority health interventions: a framework for understanding the constraints to scaling-up." Journal of International Development, 15:1–14.
The Commission on Macroeconomics and Health recommended a significant expansion in funding for health interventions in poor countries. However, there are a range of constraints to expanding access to health services: As well as an absolute lack of resources, access to health interventions is hindered by problems of demand, weak service delivery systems, policies at the health and cross-sectoral levels, and constraints related to governance, corruption and geography. This special issue is devoted to analysis of the nature and intensity of these constraints, and how they can best be overcome.
Hardee K, Ashford L, Rottach E, Jolivet R, Kiesel R. 2012. "The policy dimensions of scaling up health initiatives." Health Policy Project, Futures Group, Washington, DC.
Adopting new practices in health on a large scale requires systematic approaches to planning, implementation, and follow-up; and often calls for profound and lasting changes in health systems. Any systematic approach must include addressing the policy dimensions of scaling up. Without attention to the policies that underlie health systems and health services, the scale-up of promising pilot projects is not likely to succeed and be sustained. Interest in scale-up has grown in recent years because of an increased urgency to rapidly expand effective interventions to improve the health of mothers, children, and families, particularly the poor and underserved. This paper focuses on efforts to scale up interventions in family planning, reproductive health, and maternal, neonatal, and child health in developing countries. It defines “scale-up” and describes some of the frameworks and approaches to scaling up found in recent health literature and how they address policy. The paper also reviews the experience of selected organizations in scaling up best practices and how they have addressed policy issues. Often, frameworks for scaling up mention policies only in passing, as if addressing policy were a single step. Few scale-up frameworks and methodologies offer systematic guidance on identifying and addressing policy issues at each phase of scale-up, from planning through implementation, and on monitoring and evaluation for sustainability. Similarly, many programs tend to focus more on expansion than on institutionalization of new practices. As a result, program planners may fail to pay attention to policy throughout the health system, which is essential for programs to be successfully established and sustained. “Policy” should be understood as more than a national law or health policy that supports a program or intervention. Operational policies are the rules, regulations, guidelines, and administrative norms that governments use to translate national laws and policies into programs and services. The policy process encompasses decisions made at a national or decentralized level (including funding decisions) that affect whether and how services are delivered. Thus, attention must be paid to policies at multiple levels of the health system and over time to ensure sustainable scale-up. A supportive policy environment will facilitate the scale-up of health interventions. This paper does not replace the valuable guides that are available for scaling up health innovations. Rather, it focuses on lessons learned related to policy implementation associated with scaling up and outlines key actions to ensure supportive policies, regardless of the scale-up model or approach used.
Hardee K. 2013. "Approach for addressing and measuring policy development and implementation in the scale-up of family planning and maternal, neonatal, and child health programs." Futures Group, Health Policy Project, Washington, DC.
To achieve the greatest possible improvement in family planning (FP) and maternal, neonatal, and child health (MNCH) outcomes, successful interventions, practices, and approaches must be \"scaled up,\" that is, implemented on a larger scale and incorporated into the laws, policies, and structures that govern health systems. In recent years, growing recognition of the importance of scale-up has led to intensified efforts to identify and scale up best practices in FP/MNCH and improve scale-up processes. Policy development and implementation are central to successful and sustainable scale-up. This package of materials presents a programming approach designed to help countries advance the integration and measurement of policy development and implementation into the scale-up of FP/MNCH interventions and best practices. The approach provides planners and implementers with initial guidance and suggestions on how to systematically address policy development and implementation as they scale up FP/MNCH programs
Hartmann A, and Linn J. 2008. "Scaling up through aid – The real challenge." Wolfensohn Center for Development Policy Brief. The Brookings Institution, Washington.
At the Gleneagles Summit in 2005, leaders of the G8 group of nations committed to increase aid to poor nations by $50 billion per year. During the same year, in a meeting in Paris, donors promised to coordinate their interventions for more eff ective delivery. Th ese commitments are now often referred to as the promise of donors to “scale up aid.” Increasing aid fl ows and improving coordination are indeed important goals and, in fact, goals that donors seem to have trouble meeting. Th e international donor community met this fall in Accra and will meet in Doha in November 2008 to review progress with this aspect of scaling up aid, and it is hoped that they will recommit to meet the ambitious targets set three years ago. Scaling up aid is only one of the challenges that donors face. A more important challenge is to “scale up through aid,” meaning that aid fl ows should not merely support short-lived, one-time and partial development interventions—pilot projects, short-term technical assistance, programs that only address part of the problem, but leave major bottlenecks unaddressed—but should support projects, programs and policies that scale up successful interventions in a country, region or globally to reach the entire target population. Scaling up means that programs are long-term and sustained and that external support is aligned with country needs and deals comprehensively with the development challenges—often by working in partnership with other donors and pooling resources. Th is is the scaling up challenge that donors should address head-on, but so far have not. Th is policy brief reports on the fi ndings of an in-depth review of the literature and practice of scaling up development interventions and focuses on the role that aid donors can play in supporting scaling up for eff ective development. It stresses that successful scaling up with external assistance means that donor agencies need to: work with a vision and leadership; help create the political constituencies for large-scale implementation; create linkages among project, program and policy interventions; strengthen the institutional capacity of the implementing entities; provide for eff ective incentives and accountabilities of their own staff and management; work together with each other; monitor and evaluate the progress of programs with special attention to the scaling up dimension; and fi nally make sure they focus on eff ective preparation and fl exible implementation of the scaling up process. While this is a long-term agenda, donors can take a few practical steps right away that will provide a basis for a more ambitious eff ort over time.
Hartmann A, and Linn J. 2008. "Scaling up: A framework and lessons for development effectiveness from literature and practice." Wolfensohn Center for Development, Working Paper 5. The Brookings Institution, Washington.
Scaling up of development interventions is much debated today as a way to improve their impact and effectiveness. Based on a review of scaling up literature and practice, this paper develops a framework for the key dynamics that allow the scaling up process to happen. The authors explore the possible approaches and paths to scaling up, the drivers of expansion and of replication, the space that has to be created for interventions to grow, and the role of evaluation and of careful planning and implementation. They draw a number of lessons for the development analyst and practitioner. More than anything else, scaling up is about political and organizational leadership, about vision, values and mindset, and about incentives and accountability—all oriented to make scaling up a central element of individual, institutional, national and international development efforts. The paper concludes by highlighting some implications for aid and aid donors.
Havelock RG. 1971. "Planning for Innovation through Dissemination and Utilization of Knowledge." Ann Arbor, MI: University of Michigan, Center for Research on Utilization of Scientific Knowledge, Institute for Social Research.
This report provides a framework for understanding the processes of innovation, dissemination, and knowledge utilization, and it reviews the relevant literature in education and other fields of practice within this framework. Dissemination and utilization (D&U) is viewed as a transfer of messages by various media between resource systems and users. Major sections analyze characteristics of individuals and organizations which inhibit or facilitate this transfer. The process is interpreted at four levels; the individual, the interpersonal, the organization , and the social system. Additional chapters deal specifically with specialized "linking" roles between resource and user, types of messages, types of media, and phase models of the process. Major conclusions from the review are as follows. The principle models of D & U employed by most authors can be grouped under three perspectives identified as ( 1 ) "Research, Development, and Diffusion", ( 2 ) "Social Interaction", and ( 3 ) "Problem Solving". Each of these three viewpoints contributes significantly to our understanding of the total D&U process. They can be brought together in a "linkage model" which incorporates important features of all three. Linkage is seen as a series of two - way interaction processes which connect user systems with various resource systems including basic and applied research, development, and practice. Senders and receivers can achieve successful linkage only if they exchange messages in two - way interaction and continuously make the effort to simulate each other's problem solving behavior. Hence, the resource systems must appreciate the user's internal needs and problem solving patterns, and the user, in turn, must be able to appreciate the invention, solution formulation and evaluation processes of the resource systems. This type of collaborative interaction will not only make solutions more relevant and effective but will build relationships of trust, mutual perceptions by user and resource persons that the other is truly concerned, will listen, and will be able to provide useful information. These trust relations over time can become channels for the rapid, effective, and efficient transfer of information. Effective knowledge utilization also requires a degree of division of labor, coordination and collaboration throughout the social system. The role of government should be to monitor the "natural" knowledge flow system and develop means to support, facilitate, and coordinate linkage activities so that the total system can function more effectively.
Heller N, de Ferranti D, Feeny T. 2018. "Chasing Ghosts: How the focus on innovation in development has gone down the wrong path." Results for Development (R4D).
Soon after the ink dried on world leaders’ ambitious commitments in 2015 to meet the Sustainable Development Goals, thoughtful observers pointed out that the goals could not be achieved by simply ramping up the best approaches known then. New solutions — better, faster or more affordable — would have to be found and applied widely across Africa, Asia, Latin America, as well as in high-income countries. The importance of innovation in international development, already a familiar refrain for decades, has received even more attention since the adoption of the SDGs. But while global dialogue about innovation has proselytized in terms of the power of doing things differently, in most sectors, we remain seemingly stuck in an endless cycle of pilots, hype and showcases. And we often have little to show in terms of sustained gains and improved access to services for the poorest and most vulnerable. Why the continued cycle of discovery, overhype and disappointment? Current thinking suggests either a market failure centered on access to financing (“we need more money to scale”) or a positioning breakdown (“social entrepreneurs can only take things so far, and government is dropping the ball”). Based on new research, we suggest a more fundamental disconnect in the current work to leverage innovation to drive development outcomes: the continued fetishization of the breakthrough “innovation” as the most important unit of analysis.
Hodgins S and Quissel K. 2016. "Scale-up as if Impact Mattered: Learning and Adaptation as the Essential (often Missing) Ingredient." SNL Working Paper.
Over the past generation, globally, there have been enormous gains in health and well-being. In large part this has been a consequence of improvements in economic conditions. And there have been synergistic effects across social sectors; for example, improvements in women’s level of education and literacy have had further benefits with regard to child health, nutritional status and well-being. But specific program efforts in health have also made an important contribution. Primary healthcare programs and services have been developed and in many important instances are reaching a large proportion of those who can benefit from them and have made an important contribution to improved population health status. Immunization is a case in point. Nevertheless there are also instances of initiatives or program efforts that develop momentum and continue, drawing considerable resources, despite not making the impact they were intended to. There is a failure to acknowledge and remediate. This phenomenon of dysfunctional persistence of efforts that in fact are not delivering is seen across human endeavors. Most people in the United States are aware of the DARE anti-drugs program which has been running for over 30 years and, at its peak, was active in 3/4s of all school districts in the country. Billions have been spent on the program and it continues to chug along despite rigorous evaluations that have systematically demonstrated that it’s not been working (Lynam 1999, Birkeland 2005). There is now a well-developed literature on the phenomenon of escalation, the process of increasing commitment to a failing course of action (Ross and Staw 1987, 1997; Brockner 1992; Arkes and Blumer 1985; and others). But beyond factors identified in this literature from other sectors, there are also dynamics more specific to global health and development assistance. This discussion paper presents results of an initial effort to investigate this set of issues. It is intended as part of a broader ongoing effort to better characterize the problem, identify solutions and galvanize efforts for improved effectiveness. This paper presents findings of a series of interviews with global health leaders with substantial on the-ground experience. With input from an advisory group, a list was drawn up consisting of 30 potential interviewees. The general criteria for selection included: Anticipated depth of insight into the questions of interest, Breadth and depth of experience, and Diversity across the sample with regard to: geography of program experience (focusing mainly on low-income countries), global health content areas, and institutional role (ministry of health, donor agency, technical assistance agency, evaluator).
Howard-Grabman L, Snetro G. 2006. "How to Mobilize Communities for Health and Social Change – Chapter Seven – Prepare to Scale Up." Washington, D.C. Save the Children.Washington, DC: IFPRI.
Scaling-up community mobilization means expanding the impact of a successful mobilization effort beyond a single or limited number of communities to the regional, national, or even multinational level. While the appeal of scaling-up is obvious, the challenge is to do so without diminishing the quality of the original effort. Experience over the last decade is beginning to show that community mobilization approaches can be scaled up. This chapter will look at some of these experiences and will lay out steps to help you scale up successful community mobilization approaches. Programs achieve scale either by starting out at scale (or very quickly going to scale) or through incremental efforts to expand coverage.1 Programs typically scale up in one of five major ways: Planned Expansion: a steady process of expanding the number of sites for a particular program model once it has been pilot-tested and refined. 2 Explosion: sudden implementation of a large-scale program or intervention, without any cultivation of policy support or gradual organizational development prior to implementation. Association: expanding program size and coverage through common efforts and alliances among a network of organizations. Grafting: adding a new young adult reproductive health program, for example, to an already existing program. Diffusion: other organizations learning about approaches through access to materials and case studies and replicating the approach. Without significant uptake—the degree to which other significant development actors (e.g., NGOs, community-based groups, bilateral and multilateral agencies, host governments) adopt and adapt methodologies— scale cannot be reached. Uptake is significantly different from replication in that the former involves adaptation of strategies or methodologies to fit varying program contexts. In order to achieve substantial uptake, an organization needs to:
Humanitarian Innovation Fund. 2022. "Pilot Develop and test your solution in humanitarian settings." HIF, Elrha, Gray Dot Catalyst.
The Humanitarian Innovation Guide is a growing online resource to help individuals and organisations find their starting point and navigate the humanitarian innovation journey.
Hurlburt M, Aarons GA, Fettes D. 2014. "Interagency Collaborative Team Model for Capacity Building to Scale-Up Evidence-Based Practice." Child Youth Serv Rev. 2014;39:160-168. doi:10.1016/j.childyouth.2013.10.005
System-wide scale up of evidence-based practice (EBP) is a complex process. Yet, few strategic approaches exist to support EBP implementation and sustainment across a service system. Building on the Exploration, Preparation, Implementation, and Sustainment (EPIS) implementation framework, we developed and are testing the Interagency Collaborative Team (ICT) process model to implement an evidence-based child neglect intervention (i.e., SafeCare®) within a large children\'s service system. The ICT model emphasizes the role of local agency collaborations in creating structural supports for successful implementation.

We describe the ICT model and present preliminary qualitative results from use of the implementation model in one large scale EBP implementation. Qualitative interviews were conducted to assess challenges in building system, organization, and home visitor collaboration and capacity to implement the EBP. Data collection and analysis centered on EBP implementation issues, as well as the experiences of home visitors under the ICT model.

Six notable issues relating to implementation process emerged from participant interviews, including: (a) initial commitment and collaboration among stakeholders, (b) leadership, (c) communication, (d) practice fit with local context, (e) ongoing negotiation and problem solving, and (f) early successes. These issues highlight strengths and areas for development in the ICT model.

Use of the ICT model led to sustained and widespread use of SafeCare in one large county. Although some aspects of the implementation model may benefit from enhancement, qualitative findings suggest that the ICT process generates strong structural supports for implementation and creates conditions in which tensions between EBP structure and local contextual variations can be resolved in ways that support the expansion and maintenance of an EBP while preserving potential for public health benefit.

Keywords: Evidence-based practice; Implementation; Process Model; Sustainment; Teams.
IDRC, scalingXchange. 2022. "A Call to Action from the Global South." Website.
This call is a result of the scalingXchange—a virtual gathering of researchers and innovators from across the Global South to learn from scaling experiences and how funders and development agencies can better support scaling efforts.
IDRC. 2020. "Evaluation of IDRC’s strategy to scale research results." IDRC, Scaling Science, OTT Consulting.
This evaluation/ strategic review employs a mixed-methods design which supports the interconnected components: an organisational review, studies of grantee perceptions and IDRC’s external position, and a series of thematic case studies. The evaluation developed a ‘scaling pathway’ conceptual framework. It looked for three types of policy outcomes, as defined by IDRC’s Policy and Evaluation division: (1) Expanded policy capacities of external actors, including for scaling, (2) Informed policy dialogues and decision-making processes, and (3) Contributions to policy implementation or change. IDRC’s Scaling Science approach focuses on scaling impact rather than actions.
IFAD. 2015. "IFAD's operational framework for scaling up results." IFAD Programme Management Department.
Given the large-scale problem of reducing rural poverty that it is mandated to address and the limited resources available from official development assistance (ODA), IFAD needs to increase the impact of every dollar it invests in agriculture and rural development. For this reason, scaling up the results of successful development initiatives is an overarching priority that directly supports the achievement of IFAD’s mandate.
IFAD. 2019. "Scaling up e-learning" E-Learning.
What is this module about? Learn about scaling up, its basic concepts and why it is important for organisations like IFAD working in agriculture and rural development. Why do I need to know this? Understanding scaling up will help you to ensure that your country programmes have a wide reach and long-lasting impact.
IMAGO. 2022. "IMAGO." Website.
IMAGO was founded in 2014 by Isabel Guerrero and Zachary Green to close the missing middle in development. Our goal was to help grassroots projects scale the impact of their effective development solutions through participatory, and human-centered approaches. In biology IMAGO is the last stage of the growth process in an insect, when it attains maturity and finds its wings. IMAGO Global Grassroots helps its clients accelerate the process of finding their wings so they can develop their inborn potential and scale their impact.
Implementing Best Practices Consortium. 2013. "Guide to fostering change to scale up effective health services: A K4H Toolkit." Geneva, Washington DC.
This guide is based on the recognition that change is inevitable for survival and that directed and planned change has a much greater chance of success than ad hoc attempts to introduce new practices. Everyone working to improve health, including donors, researchers, technical experts, service providers and advocates at the global, national, district, community and family level is in the business of fostering, leading or implementing change. But not everyone involved in this work has a clear pathway that links proven change practices with evidence-based clinical and programmatic innovations. Purpose of the Guide Successful change is not an end in itself. Rather, it is a means of improving the availability and quality of services, expanding utilization and, ultimately, improving health outcomes. This guide links effective change practices with proven clinical and programmatic practices to achieve results by: Describing principles fundamental to effective change. Increasing awareness of proven approaches to effective change. Providing “how-to” steps for successful change including scale-up. Describing key challenges of scaling up and recommending strategies, tools or approaches for meeting those challenges. Offering cases that show how the steps have been implemented in real-life situations. Audience for the Guide This guide is for policy-makers, programme managers, operations researchers or other health professionals who want to bring about change in a health practice or set of practices. Changes in health practices can originate at different levels. Sometimes an evidence-based practice is introduced nationally or regionally to resolve a widespread public health concern. For example, a Minister of Health or donor might initiate the provision of insecticide-treated bed nets to reduce infant deaths from malaria or the provision of MgSO4 to prevent deaths from eclampsia. But sometimes the change “bubbles up” from the staff of a health facility seeking to address a service delivery challenge. There are many examples of hospital directors who introduced proven practices to reduce infection rates or clinic nurses who mobilized their staff to improve counselling to increase family planning acceptors. A range of groups can act together as a Change Coordination Team to support change, including international, regional and country members of the IBP Consortium; representatives from Ministries of Health and other government ministries; non-governmental organizations (NGOs) and faith-based organizations (FBOs); regional and country WHO, UNFPA, and IPPF offices; USAID missions; and regional and country offices of USAID cooperating agencies and other donors and decision makers. Active players in a successful change process often include representatives of national, regional or institutional organizations who are in a position to support the change as part of the coordination team, as well as those who are implementing the change on the ground at a programme level, service delivery site or in the community
Indig D, Lee K, Grunseit A, Milat A, & Bauman A. 2017. "Pathways for scaling up public health interventions." BMC Public Health, 18, 68.
To achieve population-wide health improvement, public health interventions found effective in selected samples need to be ‘scaled up’ and implemented more widely. The pathways through which interventions are scaled up are not well characterised. The aim of this paper is to identify examples of public health interventions which have been scaled up and to develop a conceptual framework which quantifies and describes this process. Methods A multi-stage international literature search was undertaken to identify examples of public health interventions in high income countries that have been scaled up or implemented at scale. Initial abstract review identified articles which met all the criteria of being a: 1) public health intervention; 2) chronic disease prevention focus; 3) program delivered at a wide geographical scale (state, national or international). Interventions were reviewed and coded into a conceptual framework pathway to document their scaling up process. For each program, an in-depth review of the identified articles was undertaken along with a broad internet based search to determine the outcomes of the dissemination process. A conceptual framework of scaling up pathways was developed that involved four stages (development, efficacy testing, real world trial and dissemination) to which the 40 programs were mapped. Results The search identified 40 public health interventions that showed evidence of being scaled up. Four pathways were identified to capture the different scaling up trajectories taken which included: ‘Type I – Comprehensive’ (55%) which passed through all four stages, ‘Type II – Efficacy omitters’ (5%) which did not conduct efficacy testing, ‘Type III – Trial omitters’ (25%) which did not conduct a real world trial, and ‘Type IV – At scale dissemination’ (15%) which skipped both efficacy testing and a real world trial. Conclusions This is the first study to classify and quantify the potential pathways through which public health interventions in high income countries are scaled up to reach the broader population. Mapping these pathways not only demonstrates the different trajectories that occur in scaling up public health interventions, but also allows the variation across scaling up pathways to be classified. The policy and practice determinants leading to each pathway remain for future study, especially to identify the conditions under which efficacy and replication stages are missing.
Innovation Investment Alliance 2021. "Scaling Pathways." USAID, Skoll Foundation, Mercy Corps, CASE at Duke.
How do social entrepreneurs and the funders who support them scale solutions to widespread problems, such as poverty and climate change? The Innovation Investment Alliance and CASE at Duke University have partnered to create Scaling Pathways, an in-depth look at best practices and case studies for scaling impact. The series includes:
• Pivoting to Impact: Diving into the critical global lessons learned and information vital to enterprises and funders trying to scale.
• Case Studies: Telling the stories of three organizations, their scaling strategies, pivots, successes, and failures on the road to scale.
• Thematic Studies: Detailing some best practices in financing, government partnerships, talent acquisition, pathways to scale, and data.
• Scaling Snapshots: Brief narratives on the scaling journies and strategies of leading social enterprises.

The Innovation Investment Alliance (IIA) is a funding and learning partnership between the Skoll Foundation and USAID’s Global Development Lab, with support from Mercy Corps, that has invested almost $50 million in eight proven, transformative social enterprises to scale their impact. Together, we aim to create systems-level change across sectors and geographies and draw out lessons on scaling that are applicable to the social enterprise community and inform the ongoing conversation on how to create sustainable impact at scale.
INSPIRE Working Group. 2021. "INSPIRE Guide to Adaptation and Scale Up." New York: INSPIRE Working Group; 2021.
This Guide was created as a resource for the adaptation and scale up of a country's unique action plan to address violence against children. It can be used as activities to end violence against children are selected, implemented, adapted, and scaled. National commitments of countries eager to end violence against children have grown rapidly since the UN Convention on the Rights of the Child in 1989. The pandemic of violence against children is one that the world continues to confront in a focused and coordinated way. In 2016, the multi-agency, multi-sectoral technical package ‘INSPIRE’ was published. INSPIRE represents a select group of strategies based on the best available evidence to help countries and communities intensify their focus on the prevention programs and services with the greatest potential to reduce violence against children. The INSPIRE strategies encompass: Implementation and enforcement of laws Norms and values Safe environments Parent and caregiver support Income and economic strengthening Response and support services Education and life skills. Also in that year, the Global Partnership to End Violence Against Children was established with the country “Pathfinder” model at the center of its strategy. Pathfinding countries are those whose government leaders commit to comprehensive action to end all forms of violence against children. The Pathfinder model has the potential to serve as an ideal platform for coordinated national action to scale implementation of the INSPIRE strategies. This Guide’s subject matter is on the adaptation and scale up of the INSPIRE approach. As each country’s approach to violence against children is unique to its needs, priorities, and capacities, the Guide is designed to highlight the decisions that need to be made in the adaptation and scaling up of the multi-sectoral INSPIRE approach. It then provides simple tools for collecting and analyzing the information needed to take action. This Guide is a practical and flexible tool to help policy makers and practitioners make decisions as they select, adapt and implement activities to prevent and respond to violence against children. It is our hope that this will be a significant next step towards INSPIRE’s goal of improving the lives of children and their families. This Guide is a reference that can be used as priority activities to end violence against children are selected and implemented. While neither adaptation or scale up are new ideas to most readers, we often approach them without adequate planning. We may make adaptation decisions without anticipating shifts in resources, capacities, mandates, policies, or timelines. Often, we decide to expand activities to new sites assuming they will take root without considering long-term sustainability. We can avoid this by developing a structured adaptation and scale-up strategy. This will support our country’s action plan to end violence against children by establishing goals appropriate to local conditions; setting benchmarks to know if we are meeting our objectives; and creating conditions that help us broaden and sustain success.
International Development Research Centre. 2020. "What is scaling science? - YouTube [Video file]."
More and more organisations are talking about scaling. But what is it, and what does it mean for our work? This animation explains a new approach that focuses on scaling our impact, rather than our actions, and how you may recognise it in your work.
International Institute of Rural Reconstruction. 2000. "Going to scale: can we bring more benefits to more people more quickly?" YC James Yen Center.
Participants at the workshops made it clear that appropriate technologies and approaches are available that can lead to the production of more food by poor farmer groups in marginal environments without creating dependence on external inputs and without damaging the natural resource base. The recommendations and conclusions give guidelines on how to proceed with scaling up processes. agriculture, capacity building, community participation, institutional aspects, monitoring, partnerships, replicability, research, rural development, sdicap, sdiman, small-scale activities
Jonasova M and Cooke S. 2012. "Thinking systematically about scaling up: developing guidance for scaling up World Bank-supported agriculture and rural development operations."
The objective of this report is to assess the usefulness of providing guidance for scaling up good practices in core ARD business lines, and to test the prospects for doing so. The output of the document is a guide for a systematic discussion on scaling up of Competitive Grant Schemes (CGSs) for agricultural research and extension at key decision points during the life of an ARD project. This report addresses the other end of the state-of-practice spectrum - good practices and beyond. The preparation of this report entailed five main activities: An overview of scaling up concepts and approaches; the selection of a particular sub-area within one of ARD's core business lines - scaling up CGS for agricultural research and extension; application of the IFAD/Brookings framing questions to five World Bank projects that were identified as addressing that business line - using information provided by the project's task team leaders (TTLs) or other member of the project team; the development of sub-area specific guidance for a systematic discussion on scaling up based on the findings from a series of five case studies; and validation of the scaling up guidance for CGSs for agricultural research and extension by World Bank practitioners and other internal consultations.
Jowell R. 2003. "Trying it out – the role of pilots in policy-making: Report of a review of government pilots." Edinburgh, Scotland: National Centre for Social Research.
In Chapter 2, we listed some 27 recommendations arising out of the multipronged survey we conducted, the literature search and our deliberations. They add up to a strong endorsement of the case for piloting new policy initiatives wherever practicable. And they provide enthusiastic support for the fact that the practice is now being embraced so widely across government. There is no doubt that, costly and time consuming as some pilots are, the overall benefits they provide to good governance far outweigh their disadvantages. Naturally, they fulfil an important defensive role in guarding against the inclusion of embarrassing, often expensive and preventable mistakes into new policy initiatives. But they play a highly constructive role in promoting innovation (via explicit, small-scale experiments and trials), and in helping to fine-tune policies and their delivery mechanisms in advance of their national roll-out. In short, policy pilots have become an indispensable tool of modern government. A large part of this report deals with the sorts of practical considerations that either enhance or diminish the optimal use of policy piloting in Britain. In sum they suggest that, excellent though some practice already is, there is still a long way to go before this will be uniformly true across all administrations, departments or, for that matter, across all pilots within any department. A great deal of practice still falls far short of its potential, and by no means all the obstacles to good practice will be simple to surmount. Some, such as the deep-seated suspicion in some quarters of RCTs, even in circumstances when they would seem to be an ideal mechanism, will take time to overcome, but surely will be. Others, such as the routine assumption that any new policy initiative must necessarily be introduced at the earliest possible moment, even when a small delay will help to ensure it is wellhoned, will probably take more of a culture change to rectify. On the other hand, British policy pilots have been gaining in sophistication in recent years, both in their methodology and in their analysis, and many debilitating notions of what used to be considered possible or desirable have demonstrably been dispersed. We were particularly taken with the enthusiasm we encountered both among Ministers and senior civil servants who had experienced recent pilots in action. They had generally been convinced not only of the immense value of piloting in general, but – perhaps more importantly – of the desirability of more experimentation within policy pilots, designed explicitly to try out different models to achieve particular ends. Britain still has lessons to learn from abroad, particularly about the methodology of piloting and its role in overall evaluation strategies. While our political and legislative frameworks remain less conducive to an optimal use of policy piloting than in, say, the US, great strides have been made in the past few years in both these respects.
Kaufman J, Erli Z, Zhenming X. 2007. "Quality of care in China: from pilot project to national programme." In: Simmons R, Fajans P, Ghiron L, eds. Scaling up health service delivery: from pilot innovations to policies and programmes. Geneva, World Health Organization, 2007:53-70.
China’s family planning programme ranks as history’s most intensive effort to control national population growth. While some have lauded China’s effort to limit births as a fundamental part of its sustainable development goals, the population policy has also generated much international criticism. A long-overdue reform has begun to focus the family planning programme on client needs, informed choice of contraceptives, and better quality services. Partly inspired by the International Conference on Population and Development in 1994, the reform began as a pilot project in six counties and is now a blueprint for reorienting the national family planning programme. This chapter reviews the process by which a small innovative pilot project was scaled up into a national reform effort and the lessons learned about scaling up sensitive but needed innovation in a difficult political environment. These lessons relate to the importance of local ownership, adapting concepts to make them locally meaningful, careful choice of pilot sites to ensure success, mobilizing political networks, cultivating and educating allies in senior leadership positions, strategic use of donor funding and technical assistance, and the willingness to transfer project management to the next generation of leaders.
Koh H. 2017. "Scaling Out." Stanford Social Innovation Review.
For solutions to get to scale, we need strong entrepreneurs who can build on existing breakthrough ideas, rather than creating entirely new ones. The shortage of high-quality impact investment opportunities with the potential to generate both financial and social returns remains a critical challenge for the industry. There is a growing sense that the profile of enterprises investors would like to see—particularly in terms of quality leadership teams and business models—is not reflected in the pipeline of enterprises actually emerging. Meanwhile, philanthropic and aid donors supporting this sector are increasingly frustrated, as examples of large-scale impact remain few and far between.
Kohl R. 2021. "Crosscutting Issues Affecting Scaling: A Review and Appraisal of Scaling in International Development." Global Community of Practice on Scaling Development Outcomes Working Paper.
This paper has identified and described ten crosscutting topics about scaling in international development that merit further exploration. They were selected drawing on the discussions that took place at the CoP Annual Meetings; in some cases by explicit mention, and in other cases as underlying issues. These issues are based on the author’s subjective assessment of their importance, interest, or where further exploration might be most beneficial to CoP members and scaling practitioners writ large. No claim is being made that these topics are the ten most important to scaling at this moment, or that other topics are not important. Beyond simply identifying each topic, this report has attempted to describe the parameters of each issue and suggest what a further exploration and investigation of each theme might look like. In the course of doing so, it became apparent how in several cases the issues themselves are so knotty and entangled that our recommendation for a first step is to get some definitional clarity, and clearly identify the different opinions on the topic. The prime example is the cluster of issues around Scaling, Systems Change and Complexity. At a minimum, we hope that the topics identified resonate with enough CoP members that they, and hopefully other readers, are motivated to explore them further, and that the description is sufficient to serve as a basis for that exploration. Such exploration might begin with CoP workshops, webinars, or debates about these issues. More ambitious would be for the CoP or its members to write or commission papers on the issues discussed herein. Depending on the topic, these could range from concept papers to stocktaking that catalogs existing tools and guidelines, identifies gaps, and proposes or develops new or additional tools to address those gaps. Four things are clear from this exercise. First, scaling is more important than ever to solving the issues of international development, and that is now recognized by a large and growing number of people and institutions. Second, scaling has come a long way in the last twenty years in terms of our understanding of what is good practice; and the development of tools and guidelines to translate that understanding into action. The depth, breadth, and sophistication of the topics identified in this paper stand on the foundations of those accomplishments and deserve acknowledgement. Third, there is a broad, rich, and deep ongoing conversation about scaling. In regard to both the progress and current conversation, the CoP has clearly played an important role, and should continue or even expand its activities. Finally, the topics identified in this paper clearly show that there is more work to be done, and while daunting, the task is also tremendously exciting.
Kohl R. 2021. "Scaling and Systems: Issue Paper." Global Community of Practice on Scaling Development Outcomes Working Paper.
A recent Working Paper for the Scaling Community of Practice (COP) identified a number of crosscutting issues whose further exploration would benefit the entire COP membership. One of these was the relationship between Scaling and Systems. There has been a vigorous debate in the scaling community on the extent to which systems need to be taken into account in scaling efforts, or even if the starting point for change efforts should be systems change rather than scaling innovations.3 Accordingly, the COP decided to organize a Webinar to explore the relationship between systems and scaling. This note serves to provide background for that Webinar, to frame the issues and identify questions for discussion. It draws on a cursory review of some of the literature on scaling and systems as well as discussions with COP members on these topics. A bibliography is provided at the end. Some consideration of systems in scaling is largely uncontroversial amongst scaling practitioners, though unfortunately this consensus is not yet reflected in many actual scaling efforts. The basic argument is that by taking systems into account, scaling efforts can align with systemic constraints and achieve a minimum objective of sustainable impact at scale. The debate arises over whether there should be greater engagement with systems in terms of breadth, depth, and explicit objectives for systems change (rather than taking systems constraints as largely given) in order to increase impact, coverage, reach and sustainability. It is important to note that the scaling and systems discussion is not being conducted in terms of welldefined and commonly agreed upon parameters or dimensions, i.e., the terms of the debate are not clear. One of the goals of this note is to try to provide some common terms to facilitate a productive conversation. However, this is complicated by the fact that participants in this conversation are coming from different disciplines, sit in different types of organizations including funders, innovators, and implementing organizations, and play different roles in scaling. Perhaps most importantly, they likely represent a range of systems “takers” vs. “makers” in terms of their capacity to take systems into account, let alone effect systems change. Indeed, one of the findings of this paper is that a more thorough attempt to frame the parameters of systems and scaling is needed.
Koorts H, Maple J-L, Eakin E, Lawrence M, Salmon J. 2022. "Complexities and Context of Scaling Up: A Qualitative Study of Stakeholder Perspectives of Scaling Physical Activity and Nutrition Interventions in Australia" Frontiers in Public Health 10:771235. doi: 10.3389/fpubh.2022.771235.
Scaling up population health interventions is a context-orientated, dynamic and multi-stakeholder process; understanding its influences is essential to enhance future scaling efforts. Using physical activity and nutrition interventions in Australia as case examples, the aim of this paper is to identify core influences involved in scaling up physical activity and nutrition interventions, and how these may differ by context and stakeholder.

A qualitative study involving semi-structured telephone interviews with individuals representing academic, government and non-government organizations with involvement in scaling up state and national physical activity and nutrition interventions. Interview questions were derived from the WHO report “20 Questions for Developing a Scaling up Case Study”, and mapped against four key principles and five core areas in the WHO ExpandNet framework for scaling up: (1) The innovation; (2) User organization; (3) Environment; (4) Resource team and; (5) Scale up strategy. Data were analyzed thematically.

Nineteen interviews were conducted (government = 3; non-government = 5; and academic = 11 sectors) involving eight scaled up interventions, targeting nutrition (n = 2), physical activity (n = 1) or a combination (n = 5). Most themes aligned to the “Environment”, including: (i) political (e.g., personal agendas); (ii) social (e.g., lack of urgency); and (iii) sector/workforce (e.g., scale up accountability) factors. Themes relating to “Scale up strategy” (e.g., flexibility and evaluation transparency) were next most commonly occurring. Whilst themes were broadly consistent across participants, government participants had a more policy-oriented perspective on the scale up process. Academics discussed a tension between the generation and use of evidence, and the influence of political climates/interest on scale up decisions.

Attributes of the “Environment” and “Scale up strategy” consistently featured as major influences on successful outcomes, while the role of evidence differed greatly between participant groups. A multisector scale up strategy for future interventions may enable the complexities of environmental and political contexts to be incorporated into scale up planning.
Lam D, Martín-López B, Wiek A, Bennett E, Frantzeskaki N, Horcea-Milcu I, Lang D. 2020. "Scaling the impact of sustainability initiatives: a typology of amplification processes." Urban Transformations (2020) 2:3.
Amplifying the impact of sustainability initiatives to foster transformations in urban and rural contexts, has received increasing attention in resilience, social innovation, and sustainability transitions research. We review the literature on amplification frameworks and propose an integrative typology of eight processes, which aim to increase the impact of such initiatives. The eight amplification processes are: stabilizing, speeding up, growing, replicating, transferring, spreading, scaling up, and scaling deep. We aggregated these processes into three categories: amplifying within, amplifying out, and amplifying beyond. This integrative typology aims to stimulate the debate on impact amplification from urban and rural sustainability initiatives across research areas to support sustainability transformations. We propose going beyond an understanding of amplification, which focuses only on the increase of numbers of sustainability initiatives, by considering how these initiatives create transformative change.
Larson RS, Dearing JW, Backer TE. 2017. "Strategies to Scale Up Social Programs: Pathways, Partnerships and Fidelity." The Wallace Foundation.
How to scale (“pathways”), whom to involve (“partnerships”), and retention of program quality (“fidelity”) are three strategic decisions that can be critical to the scale up of beneficial social programs in societies. By social program we mean an initiative intended by its developers to improve some aspect of communities. By scaling up we mean a process for significantly increasing the number of sustained implementations of a successful program, thereby serving more people with comparable benefits. Here’s an example: Gateway to College is a social program that supports communities in building sustainable pathways for disconnected youth to a high school diploma and a meaningful college credential. It was founded in 2000 at Portland Community College, in Oregon, and in 2003, the Bill & Melinda Gates Foundation provided funding to begin replication of the Gateway to College model. In 2004, Riverside Community College, in California, became the second Gateway partner college. By 2008, Gateway had expanded to 17 sites in 13 states. By 2013, 43 sites in 23 states. Did this increasing number of implementations come at a cost of lowered student success? Apparently not. Seventy-three percent of Gateway graduates continue on to post-secondary education and successfully earn college credentials. How does a program like Gateway to College scale up? To help answer this question, we collected 497 documents about 45 social programs to systematically code for strategic decisions made by lead organizations—those initiating a social program or bringing it to scale. We selected the 45 programs based on the advice of expert informants, database searching and web searches, and inclusion criteria such as program effectiveness, evidence of scale up through one of three pathways, and topical focus. We focused on the pathways they use for scaling, how they chose and then work with partner organizations, and how the program was reinvented and adapted—or not—as issues of fidelity were addressed. Because some information in which we were interested was not well represented in the available written documents, we interviewed 100 leaders and implementers associated with the 45 programs through use of a semi-structured interview protocol. Finally, we selected four programs for in-depth study through more interviews and site visits, which enriched our understanding of the work of these leadership teams and the challenges they have overcome. This purposive sampling procedure does not ensure a valid representation for extrapolating our results to all scale up activity in the given domains; rather, it is suggestive of questions and topics for further work. Ours is an exploratory study, as is much case study research.
Leeman J, Boisson A, Go V. 2022. "Scaling Up Public Health Interventions: Engaging Partners Across Multiple Levels." Annual Review of Public Health.
Advancing the science of intervention scale-up is essential to increasing the impact of effective interventions at the regional and national levels. In contrast with work in high-income countries (HICs), where scale-up research has been limited, researchers in low- and middle-income countries (LMICs) have conducted numerous studies on the regional and national scale-up of interventions. In this article, we review the state of the science on intervention scale-up in both HICs and LMICs. We provide an introduction to the elements of scale-up followed by a description of the scale-up process, with an illustrative case study from our own research. We then present findings from a scoping review comparing scale-up studies in LMIC and HIC settings. We conclude with lessons learned and recommendations for improving scale-up research.
Linn JF, Hartmann A, Kharas H, Kohl R, Massler B. 2010. "Scaling up the fight against rural poverty: an institutional review of IFAD’s approach." Global Economy & Development Working Paper 43, Washington, DC: The Brookings Institution.
The International Fund for Agricultural Development (IFAD) has for many years stressed innovation, knowledge and scaling up as essential ingredients of its strategy to combat rural poverty in developing countries. This institutional review of IFAD’s approach to scaling up is the first of its kind: A team of development experts were funded by a small grant from IFAD to assess IFAD’s track record in scaling up successful interventions, its operational policies and processes, instruments, resources and incentives, and to provide recommendations to management for how to turn IFAD into a scaling-up institution. Beyond IFAD, this institutional scaling up review is a pilot exercise that can serve as an example for other development institutions.
Linn JF. 2019. "Opinion: Scaling up development impact – the opportunities and challenges." Devex.
About two decades ago, Jim Wolfensohn, then-President of the World Bank, visited Georgia, and noted a successful rural credit project, supported by the bank, which had improved lives for some 1,500 farmers. Upon his return, he asked the project team what they were doing to ensure the other 50,000 or so farmers in Georgia would reap similar benefits. Wolfensohn did not like the response: “Oh, it’s such a good project, someone will pick it up and run with it; we have other projects under preparation.” Not long after, he started a new initiative at the bank to develop a systematic focus on scaling up successful development interventions. A couple of years later, Wolfensohn retired from the World Bank and his successor had different priorities. This story reflects a couple of troublesome realities: first, development institutions pay too little attention to systematically scale up successful projects; and second, because of the pervasive inclination of new managers to do things differently from their predecessor, it is difficult to maintain a scaling focus in institutions.
Linn JF. 2014. "Scaling Up Development Impact, A summary of current research, advice and outreach." Brookings Institute. Short Paper.
This study reports on a review of the United Nations Development Programme’s (UNDP) country program in Tajikistan in terms of how it addresses the opportunities and challenges to scale up successful development interventions. It assesses to what extent the UNDP pursued well articulated scaling up pathways in its overall program and in specific project areas, including its communities development program, its AIDS/HIV, tuberculosis and anti-malaria program, its support for aid coordination, its disaster risk management program and its energy and environment program. The study concludes that UNDP has incorporated key elements of a scaling up approach in its Tajikistan program, but also identifies additional ways to develop a more systematic approach to scaling up. This study is part of a broader program of research and analysis carried out under the auspices of the Brookings Global Economy and Development Program.
Management Sciences for Health (MSH). 2008. "The Leadership, Management and Sustainability (LMS) Final Report."
Managers who lead improve health services. Strengthening leadership and management practices and improving health management systems, when done effectively, contribute to improved health outcomes. A programmatic assessment of the impact of the leadership development program delivered by LMS Kenya offers evidence that these improvements are achievable and sustainable. Motivated managers make health systems work. Leadership development can bring new energy to address major challenges faced by health practitioners and increase overall performance and effectiveness. The LMS approach produces a critical transformation for health leaders/managers from an activity mind-set to a results mind-set. This transformation is fundamental to motivating teams, overcoming challenges and achieving better health outcomes in resource constrained settings. Multiple pathways exist from leadership and management to improved health services. LMS action-learning approaches include developing the leadership of health teams, empowering communities to create health service demand, increasing sustainability of service delivery programs and other methods. Good governance is about people and systems. LMS teams have helped developing country counterparts in public and NGO sectors to establish systems that promote transparency, social participation, and accountability. However, individuals and teams have a limited capacity to support implementation. Developing good governance, like developing leadership capacity, is a process that takes place over time, where people are challenged, offered feedback, and given support throughout the change. Start developing managers early in their careers. The largest pool of potential health care leaders and managers are the students of today in medical, nursing and public health schools. The LMS experience demonstrates that it is possible to integrate leadership and management into the pre-service curricula of academic institutions. Due to the hierarchical nature of the university system, it often takes more time to integrate these practices into a curriculum. Our experience shows, however, that once they are integrated this approach is highly sustainable. Virtual approaches address needs at scale. One way to reach many more organizations is to tap into the power of information technology. Opportunity exists to innovate and expand virtual approaches using new methods as Internet connectivity increases in the developing world, and as mobilization and support for strengthening health systems grows. Active facilitation and engagement with participants is the key. Sustainability requires practical tools that empower people to act. Approaches and tools must be designed for people to use in their own settings, and they should be practical and action-oriented. Complex, proven practices can be distilled into simple, effective and user-friendly practices that contribute to achievements in service delivery.
Mangham LJ, Hanson K. 2010. "Scaling up in international health: what are the key issues?" Health Policy and Planning, 25(2):85-96.
The term ‘scaling up’ is now widely used in the international health literature, though it lacks an agreed definition. We review what is meant by scaling up in the context of changes in international health and development over the last decade. We argue that the notion of scaling up is primarily used to describe the ambition or process of expanding the coverage of health interventions, though the term has also referred to increasing the financial, human and capital resources required to expand coverage. We discuss four pertinent issues in scaling up the coverage of health interventions: the costs of scaling up coverage; constraints to scaling up; equity and quality concerns; and key service delivery issues when scaling up. We then review recent progress in scaling up the coverage of health interventions. This includes a considerable increase in the volume of aid, accompanied by numerous new health initiatives and financing mechanisms. There have also been improvements in health outcomes and some examples of successful large-scale programmes. Finally, we reflect on the importance of obtaining a better understanding of how to deliver priority health interventions at scale, the current emphasis on health system strengthening and the challenges of sustaining scaling up in the prevailing global economic environment. Health planning, health policy, health systems
Massoud MR, Donohue KL, McCannon CJ. 2010. "Options for large-scale spread of simple, high-impact interventions. Technical report." USAID Health Care Improvement Project. Bethesda, Maryland.
The Surgical Safety Checklist has the potential to save untold lives worldwide and to prevent even more surgical harm. Such success, however, will rest on effective implementation, which in turn will require adoption by many thousands of surgical practitioners, working in different cultures and contexts, many of them in remote, hard-to-reach areas. The World Health Organization Patient Safety Programme and the Harvard School of Public Health commissioned the United States Agency for International Development’s Health Care Improvement Project (HCI), managed by University Research Co., LLC (URC), to present its understanding of and experience with the effective adoption of simple, high-impact interventions, such as the surgical checklist. URC – through HCI and its predecessor project, the Quality Assurance Project – has over 20 years’ experience in fostering the development and spread of such innovations. URC is joined in this effort by the Institute for Healthcare Improvement (IHI), which also has decades of experience in this field. All too often in health care, evidence-based interventions that have been shown to produce superior results in certain locations do not spread to other sites. Therefore, practitioners of health care improvement have broadened their focus to not only develop superior models of care but also to take such models to larger scale by focusing on intentional spread, to more rapidly meet the needs of large numbers of patients. Such spread requires making changes in the organization of care delivery, policies, resources, and other factors that will influence the uptake of the superior model. In planning to spread an evidence-based intervention, we must consider three key questions: What are we trying to spread? To whom do we want to spread it, and by when? How will we spread it? The framework for spread requires a superior model or practice that has proven itself on a small scale through improved system results as well as a group of leaders committed to spreading this superior model. The model needs to be developed and packaged for optimal adoption by members of the social system in question. It is important to understand the social system and its constituent parts, define the full scale of the intended spread efforts, identify the leaders within the social system, and define the channels of communication. It is imperative to identify and develop champions for change. The spread plan can then be organized such that the superior model will be broadly and successfully implemented in the social system.
Massoud MR, Nielsen GA, Nolan K, Schall MW, Sevin C. 2006. "A framework for spread: From local improvements to system-wide change." IHI Innovation Series white paper. Institute for Healthcare Improvement, Cambridge, MA.
A key factor in closing the gap between best practice and common practice is the ability of health care providers and their organizations to rapidly spread innovations and new ideas. Pockets of excellence exist in our health care systems, but knowledge of these better ideas and practices often remains isolated and unknown to others. One clinic may develop a new way to ensure that all diabetics have their HbA1c levels checked on a regular basis, or one medical-surgical unit in a hospital may develop a consistent way to reduce pain for post-operative patients. But too often these improvements remain unknown and unused by others within the organization. Organizations face several challenges in spreading good ideas, including the characteristics of the innovation itself; the willingness or ability of those making the adoption to try the new ideas; and characteristics of the culture and infrastructure of the organization to support change. In 1999, the Institute for Healthcare Improvement (IHI) chartered a team to develop a \"Framework for Spread.\" The stated aim of the team was to \"…develop, test, and implement a system for accelerating improvement by spreading change ideas within and between organizations.\" The team conducted a review of organizational and health care literature on the diffusion of innovations, and interviewed organizations both within and outside of health care that had been successful in spreading new ideas and processes, including Luther Midelfort Health System, Mayo Health System, Virginia Mason Medical Center, and Dean Health System. Since then, the Framework for Spread and our deeper understanding of its content have continued to evolve. This white paper provides a snapshot of IHI’s latest thinking and work on spread. It is divided into two parts: The first part of the white paper describes the major spread projects that IHI has supported through early 2006, and harvests the lessons we have learned about the most effective ways to: Prepare for spread; Establish an aim for spread; and Develop, execute, and refine a spread plan. The second part of the white paper is a reprint of an article published in the June 2005 issue of the Joint Commission Journal on Quality and Patient Safety, describing how the Veterans Health Administration (VHA) used the Framework for Spread to spread improvements in access to care to more than 1,800 outpatient clinics.
Massoud MR. 2014. "A promising approach to scale up health care improvements in low-and middle-income countries: the Wave-Sequence Spread Approach and the concept of the Slice of a System." F1000Research 2014, 3:100.
There are several examples of successes in improving health care. However, many of these remain limited to the sites at which they were originally developed. There are fewer examples of successful spread of the improvement more widely inside or outside the health systems within which they were developed. This article discusses the wave-sequence approach to spread or scale up, which enables take up of the improvement in a systematic and sequential way, using “spread agents” — people who participated in the original demonstration sites. The paper also discusses the concept of the “slice” of a system which is useful for thinking about spread and considers a phenomenon related to the rate of adoption which we have observed in this wave-sequence approach.
Maternal and Child Survival Program. 2020. "Basic Toolkit for Systematic Scale-up." Maternal and Child Survival Program, USAID, Management Systems International.
The tools in this toolkit are those that the Scale-up Coordinators Guide focuses on. We consider this to be the basic minimum set of tools for a scale-up coordinator to guide key stakeholders on a multi-organizational team through iterative cycles of a systematic process of scale-up. That is, the phases of: 1. Engagement and assessment 2. System-oriented co-creation (planning) 3. Implementation with learning and adaptative management. For those with interest, Annex 4 of the Scale-up Coordinator’s Guide has resources with more in-depth information about the tools from which these were adapted as well as many other related tools. Tools for engagement and assessment, and for system-oriented co-creation (planning)
• Tool 1: Define the intervention package 1
• Tool 2: Scalability Checklist 2
• Tool 3: Assess Implementer Capacity
• Tool 4: Assess Scale up Environment 1
• Tool 5: Identify Key Stakeholders and Describe Scale up Management Team 1
• Tool 6: Roles and Responsibilities for Leaders and Managers
• Tool 7: Plan Scale-up Strategies for Institutionalization and Service Expansion 1
• Tool 8: Developing a Vision for Reaching Sustainable Impact at Scale 1
Mattina D. 2006. "Money Isn’t Everything: The Challenge of Scaling Up Aid to Achieve the Millennium Development Goals in Ethiopia." IMF Working Paper, Vol., pp. 1-34.
This paper outlines the challenge of developing an operational macroeconomic framework in Ethiopia consistent with the large envisaged scaling up of aid to achieve the Millennium Development Goals (MDGs). This paper describes an MDG scenario that addresses both microeconomic and macroeconomic constraints, such as the need to boost sustainable growth, limit Dutch disease, formulate an exit strategy from aid dependency, enhance public financial management (PFM), and expand the supply of skilled labor. The paper will argue that a carefully sequenced MDG strategy is essential so that the scaled-up aid and public spending will remain in line with Ethiopia's absorptive capacity. Keywords: Millennium Development Goals, Dutch disease, absorptive capacity, aid scaling up
McCannon C J, Schall MW, Perla RJ. 2008. "Planning for scale: A guide for designing large-scale improvement initiatives." IHI Innovation Series white paper. Institute for Healthcare Improvement, Cambridge, Massachusetts.
This white paper aims to support those that are planning to take effective health care practices from one setting or isolated environment and to make them ubiquitous across a health care system, region, state, or nation. It is a preparation tool which is meant to guide conversation and thinking prior to the launch of a large-scale improvement effort; it considers the motivations, foundations, aims, interventions, social systems, and methods for spreading change that coordinators of such initiatives must understand and select. This white paper does not attempt to describe the rigorous process for executing a large-scale improvement initiative, which entails tight management of logistics and a great deal of focus on tactics for mobilizing involvement, measuring progress, and stimulating sustainable change within a target population. That content will be the subject for future papers and is described in some detail in publications and content on the IHI website. Since 2004, several nations have launched voluntary, large-scale initiatives to improve the quality and safety of their health care systems, with more to come.1-6 Through direct participation in—or consultation to—these initiatives in the United States, Canada, Scandinavia, the United Kingdom, and Japan, the Institute for Healthcare Improvement (IHI) has learned much about the complexities associated with such ambitious work. We have generated—and iteratively tested and refined—the following list of questions that serve as a discussion guide for those contemplating multi-stakeholder improvement initiatives that involve many caregiving organizations in a district, region, or nation. These questions fall into six sections: motivation, foundation, aim, nature of the intervention, nature of the social system, and network building (communication and support). The first three sections have emerged through first-hand learning, while the last three sections have their origins in the work of Everett Rogers and other experts on diffusion of innovation.7-12 The questions should be considered by a core group of stakeholders—those with experience in and influence over the problem in question—in the months leading up to the launch of a large-scale improvement effort. Some questions might not be relevant for some initiatives; negative or incomplete answers to any of them need not halt action. “Large-scale improvement” is a phrase that lends itself to many possible definitions. In this paper it refers to efforts that seek to stimulate change in complete, geopolitical areas through mobilization of hundreds or thousands of constituent organizations. For purposes of illustration, we offer here the example of the 100,000 Lives Campaign—an initiative the IHI led from December 2004 to June 2006 to assist hospitals across the United States in avoiding unnecessary deaths.
McCannon CJ, Berwick DM, Massoud MR. 2007. "The Science of Large-Scale Change in Global Health." JAMA, 298:1937-1939.
Innovation in health care includes important challenges: to find or create technologies and practices that are better able than the prevailing ones to reduce morbidity and mortality and to make those improvements ubiquitous quickly. In many respects in the pursuit of global health, the second challenge—the rapid spread of effective changes—seems to be the greater. Many sound (even powerful) solutions exist, such as new medicines and innovations in health care delivery, but their adoption is unreliable and slow. Often, they remain hidden in pockets around the globe, flourishing locally without reliably reaching those in need elsewhere. Some such solutions come from biomedical research, but even more take shape at the point of care, in settings where local problem solvers create effective new approaches to problems that others who live far away face as well.
McClure D, Gray I. 2015. "Engineering Scale Up in Humanitarian Innovations Missing Middle." IEEE 2015 Global Humanitarian Technology Conference.
The growing ability of the Humanitarian Sector to apply lightweight lean innovation techniques and deliver promising new Pilot programs has not been matched by corresponding capacity to bring these ideas to scale. The initial explanation this shortfall has been the failure of Pilots to adequately apply techniques such as User Centered Design. This focus ignores deeper more systematic challenges in the way an innovation must be transformed from a fast moving Pilot to a mature solution ready for replication. We've identified the gap between these two very different solution states as the Missing Middle of innovation. It is characterized by complex solution architecture challenges across multiple domains. In this perspective, the primary reason innovations fail to scale is not because of bad Pilots, but because of the general omission of a set of complex solution architecture tasks. Little attention has been given to this difficult work. In an effort to provide a framing model around the nature of the challenge, this paper proposes four areas that need to be intentionally addressed as part of a Scale Up initiative. The Four C's include, Completeness, Compromise, Connection and Commercials. The paper describes the nature of each of these Scale Up challenges in the light of the Humanitarian Sector.
McLean R & Gargani J. 2019. "Scaling Impact: Innovation for the Public Good." Routledge.
Scaling Impact introduces a new and practical approach to scaling the positive impacts of research and innova­tion. Inspired by leading scientific and entrepreneurial innovators from across Africa, Asia, the Caribbean, Latin America, and the Middle East, this book presents a syn­thesis of unrivalled diversity and grounded ingenuity. The result is a different perspective on how to achieve impact that matters, and an important challenge to the predomi­nant more-is-better paradigm of scaling. For organizations and individuals working to change the world for the better, scaling impact is a common goal and a well-founded aim. The world is changing rapidly, and seem­ingly intractable problems like environmental degradation or accelerating inequality press us to do better for each other and our environment as a global community. Challenges like these appear to demand a significant scale of action, and here the authors argue that a more creative and critical approach to scaling is both possible and essential. To encourage uptake and co-development, the authors present actionable principles that can help organizations and innovators design, manage, and evaluate scaling strat­egies. Scaling Impact is essential reading for development and innovation practitioners and professionals, but also for researchers, students, evaluators, and policymakers with a desire to spark meaningful change.
Milat AJ, Bauman A, Redman S. 2015. "Narrative review of models and success factors for scaling up public health interventions." Implementation Science, 10(1), 1.
To maximise the impact of public health research, research interventions found to be effective in improving health need to be scaled up and delivered on a population-wide basis. Theoretical frameworks and approaches are useful for describing and understanding how effective interventions are scaled up from small trials into broader policy and practice and can be used as a tool to facilitate effective scale-up. The purpose of this literature review was to synthesise evidence on scaling up public health interventions into population-wide policy and practice, with a focus on the defining and describing frameworks, processes and methods of scaling up public health initiatives.

The review involved keyword searches of electronic databases including MEDLINE, CINAHL, PsycINFO, EBM Reviews and Google Scholar between August and December 2013. Keywords included ‘scaling up’ and ‘scalability’, while the search terms ‘intervention research’, ‘translational research’, ‘research dissemination’, ‘health promotion’ and ‘public health’ were used to focus the search on public health approaches. Studies included in the review were published in English from January 1990 to December 2013 and described processes, theories or frameworks associated with scaling up public health and health promotion interventions.

There is a growing body of literature describing frameworks for scaling health interventions, with the review identifying eight frameworks, the majority of which have an explicit focus on scaling up health action in low and middle income country contexts. Key success factors for scaling up included the importance of establishing monitoring and evaluation systems, costing and economic modelling of intervention approaches, active engagement of a range of implementers and the target community, tailoring the scaled-up approach to the local context, the use of participatory approaches, the systematic use of evidence, infrastructure to support implementation, strong leadership and champions, political will, well defined scale-up strategy and strong advocacy.

Effective scaling up requires the systematic use of evidence, and it is essential that data from implementation monitoring is linked to decision making throughout the scaling up process. Conceptual frameworks can assist both policy makers and researchers to determine the type of research that is most useful at different stages of scaling up processes.
Milat AJ, Newson R, King L, Rissel C, Wolfenden L, Bauman A, Redman S, Giffin M. 2016. "A guide to scaling up population health interventions." Public Health Res Pract. 2016;26(1):e2611604.
The ‘how to’ of scaling up public health interventions for maximum reach and outcomes is receiving greater attention; however, there remains a paucity of practical tools to guide those actively involved in scaling up processes in high-income countries. To fill this gap, the New South Wales Ministry of Health developed Increasing the scale of population health interventions: a guide (2014). The guide was informed by a systematic review of scaling up models and methods, and a two-round Delphi process with a sample of senior policy makers, practitioners and researchers actively involved in scaling up processes. Although it is a practical guide to assist health policy makers, health practitioners and others responsible for scaling up effective population health interventions, it can also be used by researchers in the design of research studies that are potentially suitable for scaling up, particularly where research–practice collaborations are involved. The guide is divided into four steps: step 1, ‘scalability assessment’, aims to determine if an intervention is scalable; step 2, ‘developing a scale up plan’, aims to develop a practical and workable scaling up plan that can be used to convince stakeholders there is a compelling case for action. Step 3, ‘preparing for scale up’, aims to identify ways of securing resources needed for going to scale, operating at scale, and building a foundation of legitimacy and support to sustain the scaling up effort through the implementation stage; and step 4, ‘scaling up the intervention’, involves putting the plan developed in step 2 into place. Although the guide is written as though the user is starting from the point of assessing the scalability of an intervention, later steps can be used by those already involved in scaling up to review their implementation processes. The guide is not intended to be prescriptive. Its purpose is to help policy makers, practitioners, researchers and other decision makers decide on appropriate methodological and practical choices, and balance what is desirable with what is feasible.
Mills A and K Hanson, Eds. 2003. "Expanding Access to Health Interventions in Low and Middle-Income Countries: Constraints and Opportunities for Scaling-Up." Journal of International Development Special Issue: Vol 15(1), Pages 1 – 131.
The Commission on Macroeconomics and Health recommended a significant expansion in funding for health interventions in poor countries. However, there are a range of constraints to expanding access to health services: as well as an absolute lack of resources, access to health interventions is hindered by problems of demand, weak service delivery systems, policies at the health and cross-sectoral levels, and constraints related to governance, corruption and geography. This special issue is devoted to analysis of the nature and intensity of these constraints, and how they can best be overcome
Moore ML, Riddell D, Vocisano D. 2015. "Scaling Out, Scaling Up, Scaling Deep Strategies of Non-profits in Advancing Systemic Social Innovation." Journal of Corporate Citizenship. 2015. 67-84.
How can brilliant but isolated experiments aimed at a solving the most pressing and complex social and ecological problems become more widely adopted and lead to transformative impact? Leaders of social change and innovation often struggle to expand their impact on social systems, and funders of such change are increasingly concerned with the scale and positive impact of their investments. In 1998, the Montreal-based J.W. McConnell Family Foundation pursued a deliberate granting strategy known as Applied Dissemination to reframe approaches to replicating successful projects. A few years later, the Foundation began convening its grantees receiving funding from the Applied Dissemination (AD) program to accelerate the impacts of their initiatives, develop a stronger understanding of the complex systems in which they worked, and to collectively begin to address some of Canada’s most intractable social problems. The AD learning group focused on peer-based learning and application, in an environment that created trust and respect among participants. The AD learning group was successful not only in improving individual and organizational efforts to accelerate and scale impact, but also in catalyzing a field of practice in Canada that focused on generating new social innovations, and scaling up and deepening the impact of those innovative initiatives. More than a decade later, the experience contains valuable lessons about effective scaling strategies, and about how to design applied learning approaches to support social innovators. Part one of this report distills important lessons from a decade of practice in accelerating impact and scaling social innovations, including the strategies used to achieve success. Part two summarizes insights from this cohort of social innovators about the design elements involved in the applied, peerbased learning process and how that ultimately built their personal and organizational capacity. This successful initiative was not without challenges though, and these are also detailed in the report.
Moore, M-L, Riddel, D. 2015. "Scaling out, Scaling up, Scaling deep: Advancing systemic social innovation and the learning processes to support it." Journal of Corporate Citizenship. 10.9774/GLEAF.4700.2015.ju.00009.
How can brilliant but isolated experiments aimed at a solving the most pressing and complex social and ecological problems become more widely adopted and lead to transformative impact? Leaders of social change and innovation often struggle to expand their impact on social systems, and funders of such change are increasingly concerned with the scale and positive impact of their investments. In 1998, the Montreal-based J.W. McConnell Family Foundation pursued a deliberate granting strategy known as Applied Dissemination to reframe approaches to replicating successful projects. A few years later, the Foundation began convening its grantees receiving funding from the Applied Dissemination (AD) program to accelerate the impacts of their initiatives, develop a stronger understanding of the complex systems in which they worked, and to collectively begin to address some of Canada’s most intractable social problems. The AD learning group focused on peer-based learning and application, in an environment that created trust and respect among participants. The AD learning group was successful not only in improving individual and organizational efforts to accelerate and scale impact, but also in catalyzing a field of practice in Canada that focused on generating new social innovations, and scaling up and deepening the impact of those innovative initiatives. More than a decade later, the experience contains valuable lessons about effective scaling strategies, and about how to design applied learning approaches to support social innovators. Part one of this report distills important lessons from a decade of practice in accelerating impact and scaling social innovations, including the strategies used to achieve success. Part two summarizes insights from this cohort of social innovators about the design elements involved in the applied, peer-based learning process and how that ultimately built their personal and organizational capacity. This successful initiative was not without challenges though, and these are also detailed in the report.
Muralidharan K and Niehaus P. 2017. "Experimentation at Scale." Journal of Economic Perspectives Vol. 31, No. 4, Fall 2017 (pp. 103-24).
This paper makes the case for greater use of randomized experiments "at scale." We review various critiques of experimental program evaluation in developing countries, and discuss how experimenting at scale along three specific dimensions—the size of the sampling frame, the number of units treated, and the size of the unit of randomization—can help alleviate the concerns raised. We find that program-evaluation randomized controlled trials published over the last 15 years have typically been "small" in these senses, but also identify a number of examples—including from our own work—demonstrating that experimentation at much larger scales is both feasible and valuable.
Myers RG. 1984. "Going to Scale." A paper prepared for UNICEF for the Second Inter-Agency Meeting on Community-based Child Development, New York, October 29-31, 1984.
The main purpose of this paper is to provide a basis for discussing issues associated with the process of "going to scale" with programs of early childhood development. "Going to Scale" programs are described as those that attempt to reach as many potential beneficiaries as possible at a regional, national, or even worldwide level. The first section of the paper summarizes results from three analyses of successful projects and programs in an attempt to identify barriers to scale and to specify conditions, characteristics, strategies and processes accompanying successful programs. The second section examines advantages and drawbacks of three broad approaches to achieving scale, while the third section draws implications for child development programs from the preceding two sections. The final section discusses costs, organizational issues, the role of communications, evaluation, and some options for international organizations as they consider ways to increase the coverage and impact of early childhood development programs. (HOD)
Nastase A, Rajan A, French B, Bhattacharya D. 2021. "Technical assistance: a practical account of the challenges in design and implementation." [version 2; peer review: 1 approved, 1 approved with reservations]. Gates Open Res 2021, 4:177
Technical assistance is provided to country governments as part of international development programmes to support policymaking or strengthen state capability. This article presents the conceptual evolution of ‘technical assistance’ linked to capacity development, starting with programmes aiming exclusively to enhance individual capacity in the 1950s to 1970s and progressing to complex systems approaches in the past ten years. It also presents some of the frequent challenges in designing and implementing technical assistance, drawing from the existing literature and the authors’ experience in international development. The article summarises the latest thinking about delivering more effective development, including the adaptive management practices and the initiatives to strengthen evidence about what works. Finally, we complement this article with a follow-up open letter reflecting on the current policy options and opportunities for change.
Nguyen DTK, McLaren L, Oelke ND, McIntyre L. 2020. "Developing a framework to inform scale-up success for population health interventions: a critical interpretive synthesis of the literature." Global Health Research and Policy. 2020; 5: 18 doi: 10.1186/s41256-020-00141-8
Population health interventions (PHIs) have the potential to improve the health of large populations by systematically addressing underlying conditions of poor health outcomes (i.e., social determinants of health) and reducing health inequities. Scaling-up may be one means of enhancing the impact of effective PHIs. However, not all scale-up attempts have been successful. In an attempt to help guide the process of successful scale-up of a PHI, we look to the organizational readiness for change theory for a new perspective on how we may better understand the scale-up pathway. Using the change theory, our goal was to develop the foundations of an evidence-based, theory-informed framework for a PHI, through a critical examination of various PHI scale-up experiences documented in the literature.

Methods: We conducted a multi-step, critical interpretive synthesis (CIS) to gather and examine insights from scale-up experiences detailed in peer-reviewed and grey literatures, with a focus on PHIs from a variety of global settings. The CIS included iterative cycles of systematic searching, sampling, data extraction, critiquing, interpreting, coding, reflecting, and synthesizing. Theories relevant to innovations, complexity, and organizational readiness guided our analysis and synthesis.

Results: We retained and examined twenty different PHI scale-up experiences, which were extracted from 77 documents (47 peer-reviewed, 30 grey literature) published between 1995 and 2013. Overall, we identified three phases (i.e., Groundwork, Implementing Scale-up, and Sustaining Scale-up), 11 actions, and four key components (i.e., PHI, context, capacity, stakeholders) pertinent to the scale-up process. Our guiding theories provided explanatory power to various aspects of the scale-up process and to scale-up success, and an alternative perspective to the assessment of scale-up readiness for a PHI.

Conclusion: Our synthesis provided the foundations of the Scale-up Readiness Assessment Framework. Our theoretically-informed and rigorous synthesis methodology permitted identification of disparate processes involved in the successful scale-up of a PHI. Our findings complement the guidance and resources currently available, and offer an added perspective to assessing scale-up readiness for a PHI.

Keywords: Critical interpretive synthesis; Framework; Population health intervention; Readiness; Scale-up.
Omimo A, Taranta D, Ghiron L, Kabiswa C, Aibe S, Kodande M, Nalwoga C, Mugaya S, Onduso P. 2018. "Applying ExpandNet’s Systematic Approach to Scaling Up in an Integrated Population, Health and Environment Project in East Africa." Social Sciences, 7(1), 8; doi:10.3390/socsci7010008
While the importance of pursuing integrated population, health and environment (PHE) approaches and ensuring their sustainable expansion to regional and national levels have been widely affirmed in the development field, little practical experience and evidence exist about how this can be accomplished. This paper lays out the systematic approach to scale up developed by ExpandNet and subsequently illustrates its application in the Health of People and Environment in the Lake Victoria Basin (HoPE-LVB) project, which is an integrated PHE project implemented in Uganda and Kenya from 2012–2017. Results demonstrate not only the perceived relevance of pursuing integrated development approaches by stakeholders but also the fundamental value of systematically designing and implementing the project with focused attention to scale up, as well as the challenges involved in operationalizing commitment to integration among bureaucratic agencies deeply grounded in vertical departmental approaches. Keywords: scaling up; ExpandNet; population; health and environment (PHE); institutionalization; expansion; nine step approach; development; Kenya; Uganda
Paina L, Peters DH. 2011. "Understanding pathways for scaling up health services through the lens of complex adaptive systems." Health Policy and Planning, 1-9, doi: 10.1093/heapol/czr054
Despite increased prominence and funding of global health initiatives, efforts to scale up health services in developing countries are falling short of the expectations of the Millennium Development Goals. Arguing that the dominant assumptions for scaling up are inadequate, we propose that interpreting change in health systems through the lens of complex adaptive systems (CAS) provides better models of pathways for scaling up. Based on an understanding of CAS behaviours, we describe how phenomena such as path dependence, feedback loops, scale-free networks, emergent behaviour and phase transitions can uncover relevant lessons for the design and implementation of health policy and programmes in the context of scaling up health services. The implications include paying more attention to local context, incentives and institutions, as well as anticipating certain types of unintended consequences that can undermine scaling up efforts, and developing and implementing programmes that engage key actors through transparent use of data for ongoing problem-solving and adaptation. We propose that future efforts to scale up should adapt and apply the models and methodologies which have been used in other fields that study CAS, yet are underused in public health. This can help policy makers, planners, implementers and researchers to explore different and innovative approaches for reaching populations in need with effective, equitable and efficient health services. The old assumptions have led to disappointed expectations about how to scale up health services, and offer little insight on how to scale up effective interventions in the future. The alternative perspectives offered by CAS may better reflect the complex and changing nature of health systems, and create new opportunities for understanding and scaling up health services. Complex adaptive systems, health systems, scaling up, health planning
Partners for Health Reform Plus. 2004. "The role of pilot programs: Approaches to health systems strengthening." Bethesda, MD. PHRPlus, Abt Associates.
The U.S. Agency for International Development (USAID) is one of the primary providers of technical assistance for health systems strengthening, and, through its global and country-level projects, it has accumulated significant experience in designing and implementing health sector reforms. One approach to health system strengthening that has been used relatively frequently by USAID-supported projects is the implementation of small-scale pilots. Such pilots enable country decision makers to “try out” a complex health system reform before deciding whether and how to roll out the reform at the national level. Pilots are often supported by evaluations that help inform the decision as to whether or not the reform should be rolled out, and may help fine-tune the reform design. While the use of pilots is not unique to health systems strengthening efforts, the piloting of health system reforms is rather different from piloting of new clinical interventions, or even service delivery modes. Complex health systems strengthening initiatives typically challenge country policymakers and implementers on multiple levels: they involve a diverse range of stakeholders, they address politically sensitive issues, they generally include multiple related components, and they frequently require skills and capacity that are in short supply in-country. Under these circumstances, piloting health system strengthening initiatives may offer particular advantages. Abt Associates, in its work for USAID on the Health Financing and Sustainability project, the Partnerships for Health Reform, and the ZdravReform project, has been involved in multiple health systems strengthening pilot programs. The contract for the Partners for Health Reformplus project (PHRplus) initially required that three pilot sites be established. The PHRplus team therefore decided that it would be important to take stock of what had been learned from previous health system strengthening pilots. Surprisingly, the published literature on this topic is minimal: papers that document pilot experiences are hard to find, and there is nothing in the literature about how to design or implement a pilot project. PHRplus conducted a review of the grey literature on piloting complex health system reforms,1 but also determined that it was important to document our own experiences of piloting. This volume represents the fruit of these efforts. It contains case studies of three health system reform pilots undertaken in Kyrgyzstan, Niger, and Rwanda during the past 15 years. Chapter 1 draws upon these experiences, as well as the broader review previously conducted, to develop a conceptual framework to assist with the design and planning of health system strengthening pilot programs, as well as summarizing the lessons learned from these three case studies and similar experiences
Phillips JF, Awoonor-Williams JK, Bawah AA, Nimako BA, Kanlisi N, Sheff MC, Jackson EF, Asuming PO, Biney A, and Kyei P. 2018. "What do you do with success? The science of scaling up a health systems strengthening intervention in Ghana." BMC Health Services Research 18 (1): 1.
The completion of an implementation research project typically signals the end of research. In contrast, the Ghana Health Service has embraced a continuous process of evidence-based programming, wherein each research episode is followed by action and a new program of research that monitors and guides the utilization of lessons learned. This paper reviews the objectives and design of the most recent phase in this process, known as a National Program for Strengthening the Implementation of the Community-based Health Planning and Services (CHPS) Initiative in Ghana (CHPS+).

A mixed method evaluation strategy has been launched involving: i) baseline and endline randomized sample surveys with 247 clusters dispersed in 14 districts of the Northern and Volta Regions to assess the difference in difference effect of stepped wedge differential cluster exposure to CHPS+ activities on childhood survival, ii) a monitoring system to assess the association of changes in service system readiness with CHPS+ interventions, and iii) a program of qualitative systems appraisal to gauge stakeholder perceptions of systems problems, reactions to interventions, and perceptions of change. Integrated survey and monitoring data will permit multi-level longitudinal models of impact; longitudinal QSA data will provide data on the implementation process.

A process of exchanges, team interaction, and catalytic financing has accelerated the expansion of community-based primary health care in Ghana’s Upper East Region (UER). Using two Northern and two Volta Region districts, the UER systems learning concept will be transferred to counterpart districts where a program of team-based peer training will be instituted. A mixed method research system will be used to assess the impact of this transfer of innovation in collaboration with national and regional program management. This arrangement will generate embedded science that optimizes prospects that results will contribute to national CHPS reform policies and action.
Pitt H, Mathew J. 2016. "Scaling up and out as a Pathway for Food System Transitions." Sustainability. 8. 1025. 10.3390/su8101025.
This paper contributes to the understanding of sustainability transitions by analysing processes of scaling up and out as change pathway. It defines scaling up and out as a distinct form of policy transfer focused on programme implementation, with continuity of actors across jurisdictions. We detail how scaling up and out occurs, introducing a new mechanism to policy transfer frameworks. This is explicated through the case study of Food for Life (FFL), a civil society innovation programme promoting sustainable healthy food in public settings. We highlight why FFL was scaled up and out, how this was achieved, by whom, and the results and success factors. The case study demonstrates the importance of interrogating motivations for transferring policies, and how these influence whether successful outcomes are achieved. This requires a revised framework for analysing policy transfer, with greater attention to the links between motives and outcomes, and a less binary understanding of agents\' roles. Where scaling is the mode of policy transfer, we suggest that continuous involvement of at least one transfer agent across the process is significant to success. We conclude by highlighting implications for future research into policy transfer and food system transitions.
Ramalingam B, Wild L, and Buffardi AL. 2019. "Making adaptive rigor work: Principles and practices for strengthening monitoring, evaluation and learning for adaptive management." Briefing Paper. Overseas Development Institute (ODI). London.
Core development and humanitarian challenges are complex, and require processes of testing, learning and iteration to find solutions – adaptive management offers one approach for this. Yet large bureaucracies and development organisations can have low tolerance for experimentation and learning, and adaptive management can be viewed as an excuse for ‘making things up as you go along’. This briefing from the Global Learning for Adaptive Management (GLAM) initiative argues that adaptive programmes can be accountable, rigorous and high quality in how they use evidence – but this requires rethinking some key assumptions about how they are practised. The paper sets out three key elements of an ‘adaptive rigour’ approach: Strengthening the quality of monitoring, evaluation and learning data and systems. Ensuring appropriate investment in monitoring, evaluation and learning across the programme cycle. Strengthening capacities and incentives to ensure the effective use of evidence and learning as part of decision-making, leading ultimately to improved effectiveness.
Results for Development. 2000. "International Development Innovation Alliance (IDIA)." Website.
Innovation is critical for delivering sustained, scalable solutions to the world\'s complex problems. From a development perspective, IDIA defines innovation is a new solution with the transformative ability to accelerate impact. Innovation can be fueled by science and technology, can entail improved ways of working with new and diverse partners, or can involve new social and business models or policy, creative financing mechanisms, or path-breaking improvements in delivering essential services and products. Innovation thrives best when facilitated by a strong ecosystem of favourable enabling conditions, including better policy and regulatory frameworks, open data and standards, and expanded access for innovators to resources. More effectively tapping into capital flows through blended finance, for instance, will harness additional resources that far exceed the current capital available for poverty reduction. The same is true when the development of science, technology and innovation (STI) capabilities is promoted, by strengthening the regulatory, infrastructure, and human capacity foundations that support the growth of local STI ecosystems. Over the next 30 years, the global community has an extraordinary opportunity to eradicate extreme poverty. That will not be easy, given a global context where: two billion more people will be added to today’s 7.3 billion; 90% of humanity will live in low and middle income countries; and crucial new disruptive influences and demands on natural resources and human capital will need to be confronted. Our world has already been dramatically transformed by the life-saving and life-improving innovations of entrepreneurs from many years ago, who saw where inadequate practices were slowing progress, and who took action to create more effective approaches. With the progress in addressing many of today’s development challenges still lagging far behind our aspirations, finding and supporting innovative solutions in a more efficient manner has become a priority for the international development community in order to enable billions of people to overcome poverty. An agenda of innovation will unleash the power of human enterprise to invent better futures. Innovations in health, agriculture, finance, and other sectors have been vital in the life-saving and life-improving breakthroughs of recent times. Where inadequate practices slow progress, more effective approaches will need to be found. The future depends on how well innovation can spur faster advances.
Robb-McCord J, Voet W. 2003. "Scaling up practices, tools, and approaches in the maternal and neonatal health program." Baltimore: JHPIEGO Corp.
This report documents how the Maternal and Neonatal Health Program (1998-2004) scaled up practices and approaches at the global, regional, and country levels, providing a qualitative description of the Program\'s expanded reach, breadth, impact, and sustainability. Since 1998, the Maternal and Neonatal Health (MN/4) Program has worked to build a strategy responsive to the complexities of programming for safe motherhood. Central to the MNH Program approach is support for the use of key evidence-based maternal and neonatal care practices that build on global lessons learned about how to save the lives of mothers and newborns. The Program\'s three technical components—clinical services, behavior change interventions, and policy—provide interventions to support the appropriate use of these practices. The MNH Program\'s clinical services and policy interventions include establishing clinical standards of care, educating and training providers, and strengthening service delivery sites through performance and quality improvement. The Program\'s behavior change interventions complement the clinical services and policy components by facilitating behavioral and normative change at all levels of the healthcare system and community to increase access to, demand for, and the use of skilled life-saving care. Improving birth preparedness and complication readiness (BP/CR)— a strategy that emphasizes shared responsibility among policymakers, facilities, providers, communities, families, and women for maternal and newborn survival—provides the focus for the Program\'s work in all three technical areas. The MNH Program is currently working in 11 countries in Africa, Asia, and Latin America. Country-level program interventions are mutually supportive and are guided and supported by the Program\'s global agenda. The MNH Program contributes to the international safe motherhood effort by: z. Establishing and promoting international evidence-bawd standards for essential maternal and newborn care through global partnerships 7. Improving the quality of skilled attendance by implementing international evidence-based standards and guidelines in national policy, curricula, and competency-based training 7. Generating shared responsibility and coordinated action among policymakers, health facilities, providers, communities, families, and women through birth preparedness and complication readiness z Scaling up evidence-based safe motherhood practices, tools, and approaches by collaborating with global and national partners z. Building the evidence base for social and behavior change interventions that generate informed demand and collective action for safe motherhood
Rodríguez AMS, MacLachlan M, Brus A. 2020. "The coordinates of scaling: Facilitating inclusive innovation." Systems Research and Behavioral Science, Volume 38, Issue 6, pages 833-850.
The desire to ensure that the benefits of successful small-scale social innovation are more widely available has led to a plethora of frameworks that seek to scale such innovations. We review 20 extant frameworks for scaling and distinguished four directions: up (producing changes in laws, policies, institutions or norms), down (resource allocation to support implementation), in (ensuring organizations have the capacity to deliver the type and number of good practices required) and out (geographically replicating or broadening the range or scope of good practices). In addition to these directions of scaling a generic pathway, or process, to achieve scaling is also discernible across many of the frameworks reviewed. This involves five phases: identifying, planning, implementing, learning and adapting. We stress the need for a more dynamic and systemic approach to scaling, as well as one which anticipates, addresses and assesses the extent to which scaling is inclusive of marginalized groups.
Rogers EM. 1995. "Diffusion of Innovations." Fourth Edition. New York: Free Press.
Now in its fifth edition, Diffusion of Innovations is a classic work on the spread of new ideas. In this renowned book, Everett M. Rogers, professor and chair of the Department of Communication & Journalism at the University of New Mexico, explains how new ideas spread via communication channels over time. Such innovations are initially perceived as uncertain and even risky. To overcome this uncertainty, most people seek out others like themselves who have already adopted the new idea. Thus the diffusion process consists of a few individuals who first adopt an innovation, then spread the word among their circle of acquaintances—a process which typically takes months or years. But there are exceptions: use of the Internet in the 1990s, for example, may have spread more rapidly than any other innovation in the history of humankind. Furthermore, the Internet is changing the very nature of diffusion by decreasing the importance of physical distance between people. The fifth edition addresses the spread of the Internet, and how it has transformed the way human beings communicate and adopt new ideas.
Ross J, Karlage A, Etheridge J, Alade M, Fifield J, Goodwin C, Semrau K, Hirschhorn L. 2021. "Adaptive Learning Guide: A Pathway to Stronger Collaboration, Learning, and Adapting." Washington, DC: USAID MOMENTUM.
This guide provides information and resources to integrate adaptive learning into the design, implementation, and improvements of programs for maternal, newborn and child health services, voluntary family planning, and reproductive health care. It offers a conceptual introduction to adaptive learning, key steps to integrating into your work using links to existing resources and real-world examples of how adaptive learning can drive continuous learning and improvement in project work. MOMENTUM—or Moving Integrated, Quality Maternal, Newborn, and Child Health Services, Voluntary Family Planning, and Reproductive Health Care (MNCH/FP/RH) to Scale—is the flagship, multi-award program at the United States Agency for International Development (USAID) to accelerate reductions in maternal, newborn, and child mortality and morbidity in high-burden USAID partner countries. The purpose of this Adaptive Learning Guide is to provide MOMENTUM project teams with the information and resources to integrate adaptive learning into the design, implementation, and improvement of MNCH/FP/RH programs.
Ruth Simmons, Peter Fajans, Laura Ghiron, B. Ronald Johnson. 2011. "Managing Scaling Up" Chapter 1 - From One to Many: Scaling Up Health Programs in Low Income Countries.
The ExpandNet Scaling Up Framework. Dissemination and advocacy. Organizing the scaling up process. Resource mobilization and cost issues. Monitoring and evaluation. WHO Tools and approaches.
Sartas, Murat, Schut, M., Proietti, C., Thiele, G., Leeuwis, C. 2020. "Scaling readiness: Science and practice of an approach to enhance impact of research for development." Agricultural Systems, 183, 102874.
Scaling of innovations is a key requirement for addressing societal challenges in sectors such as health, agriculture, and the environment. Research for development (R4D) programs, projects and other interventions struggle to make particular innovations go to scale. Current conceptualizations of scaling are often too simplistic; more systemic and multidimensional perspectives, frameworks and measures are needed. There is a gap between new complexity-aware theories and perspectives on innovation, and tools and approaches that can improve strategic and operational decision-making in R4D interventions that aim to scale innovations. This paper aims to bridge that gap by developing the key concepts and measures of Scaling Readiness. Scaling Readiness is an approach that encourages critical reflection on how ready innovations are for scaling and what appropriate actions could accelerate or enhance scaling. Scaling Readiness provides action-oriented support for (1) characterizing the innovation and innovation system; (2) diagnosing the current readiness and use of innovations as a proxy for their readiness to scale; (3) developing strategy to overcome bottlenecks for scaling; (4) facilitating and negotiating multi-stakeholder innovation and scaling processes; and (5) navigating and monitoring the implementation process to allow for adaptive management. Scaling Readiness has the potential to support evidence-based scaling strategy design, implementation and monitoring, and – if applied across multiple interventions – can be used to manage a portfolio of innovation and scaling investments.
Satia J, Fajans P, Elias C, Whittaker M. 2000. "A strategic approach to reproductive health programme development." Asia-Pacific Population Journal, DOI: 10.18356/193e2ac6-en.
Many countries are transforming their efforts to meet their population’s reproductive health needs by refocusing maternal and child health and family planning activities into more comprehensive reproductive health programmes. Clearly, the specific directions and magnitude of the changes involved should depend on the socio-economic context and local epidemiology of reproductive health problems, as well as on the current programmatic situation. In seeking to innovate and expand reproductive health services, programme managers and policy makers are generally advised to follow an approach that is (a) public health based - addressing key reproductive health problems, (b) pragmatic - adding interventions and services in an incremental manner and building on what already exists, and (c) participatory - recognizing what different actors can feasibly do (Fathalla, 1996). The need to identify appropriate service delivery models and subsequently scale-up successful efforts is acute. Over the past decade, the UNDP/UNFPA/WHO/World Bank Special Programme of Research, Development and Research Training in Human Reproduction (HRP), Department of Reproductive Health and Research (RHR), WHO, in collaboration with a variety of other institutions, has developed a strategic approach to contraceptive introduction that focuses on improving the quality of care in a reproductive health context. As the strategic approach was being implemented to address contraceptive introduction in several countries, its utility for addressing other specific reproductive health problems was recognized. Since then, the approach has been used for safe motherhood, abortion, reproductive tract infections/sexually transmitted infections (RTI/STIs) including human immunodeficiency virus/acquired immunodeficiency syndrome HIV/AIDS and adolescent reproductive health programmes. More recently, it has been applied as a framework for developing comprehensive reproductive health programmes. This article describes the strategic approach to contraceptive introduction and experience with its implementation. It then reviews the application of the approach to reproductive health programme development and raises some issues concerning the methodology that needs to be addressed in considering its application to reproductive health programme development.
Scaling Pathways. 2021. "Scaling Through Mass Disruption." CASE at Duke, Skoll Foundation, Mercy Corps Ventures.
In light of the mass disruption caused by COVID-19, we are working with our Scaling Pathways partners on a new video interview series called Scaling Through Mass Disruption, which captures how social enterprises are adapting, pivoting, managing finances, engaging teams, and so much more in times of crisis. The social enterprises interviewed share advice and insights that are not only relevant for the present situation, but also help identify trends and lessons learned that can prepare organizations for inevitable future crises and disruptions.
scalingXchange. 2020. "Southern perspectives on funders' support for scaling : lessons from the scalingXchange." Scaling Science, IDRC.
Discussions aim to unpack, challenge, and develop the four guiding principles of scaling science, which were identified through a comprehensive retrospective study of more than 200 research projects with real-world impact objectives. The guiding principles present an evidence-based framework for scaling impact that offers options for innovation and creativity, rather than outlining a singular pathway to success. Together they form an understanding of scaling as a coordinated effort to achieve a collection of impacts at optimal scale. This report includes discussion points regarding opportunities and challenges that researchers face in putting principles into practice.
Schouten LMT, Hulscher MEJL, Everdingen JJE, Huijsman R, Grol RPTM 2008. "Evidence for the impact of quality improvement collaboratives: systematic review." British Medical Journal. 336:1491-1499.
To evaluate the effectiveness of quality improvement collaboratives in improving the quality of care. Data sources Relevant studies through Medline, Embase, PsycINFO, CINAHL, and Cochrane databases. Study selection Two reviewers independently extracted data on topics, participants, setting, study design, and outcomes. Data synthesis Of 1104 articles identified, 72 were included in the study. Twelve reports representing nine studies (including two randomised controlled trials) used a controlled design to measure the effects of the quality improvement collaborative intervention on care processes or outcomes of care. Systematic review of these nine studies showed moderate positive results. Seven studies (including one randomised controlled trial) reported an effect on some of the selected outcome measures. Two studies (including one randomised controlled trial) did not show any significant effect.

The evidence underlying quality improvement collaboratives is positive but limited and the effects cannot be predicted with great certainty. Considering that quality improvement collaboratives seem to play a key part in current strategies focused on accelerating improvement, but may have only modest effects on outcomes at best, further knowledge of the basic components effectiveness, cost effectiveness, and success factors is crucial to determine the value of quality improvement collaboratives.
Shiffman J. 2000. Agenda setting and reproductive rights into the 21st century. Paper presented at the International Union for the Scientific Study of Population Conference on Family Planning Programmes in the 21st Century, Dhaka, Bangladesh, 17-20 January 2000.
Simmons R, Shiffman J. 2007. "Scaling up health service innovations: a framework for action." In: Simmons R, Fajans P, Ghiron L, eds. Scaling up health service delivery: from pilot innovations to policies and programmes. Geneva, World Health Organization, 2007:1–30.
This chapter provides a conceptual framework for scaling up, with a focus on evidence-based reproductive health service innovations. It cites an extensive literature from several disciplines. The framework links an innovation to be scaled up with four other elements: a resource team that promotes it; a user organization expected to adopt the innovation; a strategy to transfer it; and an environment in which the transfer takes place. The authors discuss key attributes that have been found to facilitate the scaling-up process and identify strategic choices that must be made to ensure success. A final section identifies the diverse environments in which scaling up occurs, arguing that successful scale up requires tailoring strategies to the various dimensions of these settings.
Skibiak J, Mijere P, Zama M. 2007. "Expanding contraceptive choice and improving quality of care in Zambia’s Copperbelt: moving from pilot projects to regional programmes." In: Simmons R, Fajans P, Ghiron L, eds. Scaling up health service delivery: from pilot innovations to policies and programmes. Geneva, World Health Organization, 2007:71–88.
This case-study explores the programmatic challenges of moving from pilot interventions to regional programmes. It documents the history of an initiative to scale up reproductive health interventions, developed and tested between 1996 and 2000 in Zambia’s Copperbelt Province. The interventions included an expansion of the range of contraceptive methods available at health facilities, the development of innovative training approaches for healthcare workers, and the testing of strategies to reach out to communities. This chapter highlights the challenges facing programme designers as they must decide which elements of a pilot study to scale up, the structures most appropriate for managing the process, and the pace and breadth of the expansion effort. Finally, it provides a conceptual framework to guide the scaling-up process and to weigh the potential trade-offs between increasing scale and the need to maintain quality, local values, local relevance and sustainability.
Skoll Foundation, USAID, Mercy Corps Ventures, CASE at Duke. 2022. "Scaling Pathways." Blog Website.
Scaling Pathways is a partnership between the Skoll Foundation, USAID, Mercy Corps Ventures, and CASE at Duke to curate and share scaling insights from the world’s leading social entrepreneurs.
Spicer N, Bhattacharya D, Dimka R, Fanta F, Mangham-Jefferies L, Schellenberg J, Tamire-Woldemariam A, Walt G, and Wickremashinge D. 2014. "‘Scaling-up is a craft not a science’: Catalysing scale-up of health innovations in Ethiopia, India and Nigeria." Social Science and Medicine, 121, 30-38.
Donors and other development partners commonly introduce innovative practices and technologies to improve health in low and middle income countries. Yet many innovations that are effective in improving health and survival are slow to be translated into policy and implemented at scale. Understanding the factors influencing scale-up is important. We conducted a qualitative study involving 150 semi-structured interviews with government, development partners, civil society organisations and externally funded implementers, professional associations and academic institutions in 2012/13 to explore scale-up of innovative interventions targeting mothers and newborns in Ethiopia, the Indian state of Uttar Pradesh and the six states of northeast Nigeria, which are settings with high burdens of maternal and neonatal mortality. Interviews were analysed using a common analytic framework developed for cross-country comparison and themes were coded using Nvivo. We found that programme implementers across the three settings require multiple steps to catalyse scale-up. Advocating for government to adopt and finance health innovations requires: designing scalable innovations; embedding scale-up in programme design and allocating time and resources; building implementer capacity to catalyse scale-up; adopting effective approaches to advocacy; presenting strong evidence to support government decision making; involving government in programme design; invoking policy champions and networks; strengthening harmonisation among external programmes; aligning innovations with health systems and priorities. Other steps include: supporting government to develop policies and programmes and strengthening health systems and staff; promoting community uptake by involving media, community leaders, mobilisation teams and role models. We conclude that scale-up has no magic bullet solution – implementers must embrace multiple activities, and require substantial support from donors and governments in doing so. Keywords Uttar Pradesh, India Northeast Nigeria Ethiopia Scale-up Maternal and newborn health Innovations
Spring Impact 2021. "Journey to Impact: Lessons from our first ten years"
At Spring Impact, we work to strengthen the global practice around scaling social impact, through deep partnerships with organizations, and by developing and sharing insight.

We’ve spent 10 years helping mission-driven organizations across the world to scale their impact. What you’re reading about now – the Journey to Impact report – is part of our commitment to sharing knowledge with the social sector.

Whether you’re a mission-driven organization looking to scale your impact, or a funder interested in supporting their grantees to more effectively achieve the impact they seek, the report will provide you with lots of direct, applicable insight.
Spring Impact. 2020. "Five Stage Process: Our methodology for achieving scale through replication."
There are a number of routes to scaling social impact; our current focus is on replication. Replication refers broadly to taking your organisation, program or set of core principles to new locations. Our methodology is based on five stages of achieving scale through social replication. It is publicly available via our Social Replication Toolkit. The Toolkit is designed to help organisations assess their readiness for replication and to provide practical guidance through our five stages to scale. We are continuously applying this methodology across diverse sectors, geographies and cause areas. We‘ve helped shape the way leading global organisations and grassroots social ventures increase their reach, transforming the lives of those in need.
Starr K, Hattendorf L. 2015. "2015, Aug 21. The doer and the payer: A simple approach to scale [Blog post]." Stanford Social Innovation Review.
At Mulago, we obsess on the notion of social impact that goes to scale. Since we’re usually willing to pay for lunch, people often come to talk about “going to scale” and “scaling up our work.” Most of the time, the word “growth” would better capture what they have in mind. Growth is a fine thing, but scale is what solves problems, and so scale is what we look for. When we talk about “going to scale,” this is what we mean:

Call it exponential, geometric, what have you—the point is that the curve steepens, and impact (I) accelerates dramatically over time (T). We know it takes a while; we know that “the hockey stick” is bullshit, but in the end we need to see something that looks like that curve.

Since we need to talk about scale—and design for it—with lots of organizations doing lots of different things, we wanted to find a simpler, more usable way to talk and think about it. Over time, we realized that if you want to get to real scale, two questions really matter: 1) Who’s the doer, and 2) who’s the payer?
Starr K, Kehrer D. 2021. "Scaling and Systems Change: Chicken & Egg?" Scaling Up Community of Practice.
Join the Global Community of Practice on Scaling Development Outcomes in inviting two accomplished experts, Kevin Starr and Daniel Kehrer, to share their perspectives on large-scale change from a transactional and from a systems perspective, respectively.
Starr K. 2021. "Getting to Scale: Size Matters. Shape Matters More." Stanford Social Innovation Review.
Scale is one of those words that means something different to just about everyone. But to measure it, to figure out concretely whether an intervention has scaled, you have to decide what you mean. So when Michael Kremer et al set out to measure the social return from innovation investments made by USAID’s Development Innovation Ventures (DIV), they selected four interventions that had “scaled” because they had reached at least a million people and used that benchmark to estimate the social return on DIV’s investment (see this excellent blog post by Jocilyn Estes, David Evans and Sarah Rose summarizing that work). As they note, the choice of a million was “an arbitrary cut-off motivated by the costliness of detailed data collection.” And on that basis, a dollar spent by DIV is judged to generate $17 worth of social impact.
Starr K. 2021. "Kevin Starr" Stanford Social Innovation Review.
Kevin Starr directs the Mulago Foundation and the Rainer Arnhold Fellows Program.
Subramanian S, Naimoli J, Matsubayashi T, Peters DH. 2011. "Do we have the right models for scaling up health services to achieve the Millennium Development Goals?" BMC Health Services Research, 11:336.
There is widespread agreement on the need for scaling up in the health sector to achieve the Millennium Development Goals (MDGs). But many countries are not on track to reach the MDG targets. The dominant approach used by global health initiatives promotes uniform interventions and targets, assuming that specific technical interventions tested in one country can be replicated across countries to rapidly expand coverage. Yet countries scale up health services and progress against the MDGs at very different rates. Global health initiatives need to take advantage of what has been learned about scaling up.

A systematic literature review was conducted to identify conceptual models for scaling up health in developing countries, with the articles assessed according to the practical concerns of how to scale up, including the planning, monitoring and implementation approaches.

We identified six conceptual models for scaling up in health based on experience with expanding pilot projects and diffusion of innovations. They place importance on paying attention to enhancing organizational, functional, and political capabilities through experimentation and adaptation of strategies in addition to increasing the coverage and range of health services. These scaling up approaches focus on fostering sustainable institutions and the constructive engagement between end users and the provider and financing organizations.

The current approaches to scaling up health services to reach the MDGs are overly simplistic and not working adequately. Rather than relying on blueprint planning and raising funds, an approach characteristic of current global health efforts, experience with alternative models suggests that more promising pathways involve “learning by doing” in ways that engage key stakeholders, uses data to address constraints, and incorporates results from pilot projects. Such approaches should be applied to current strategies to achieve the MDGs.
The Bridgespan Group. 2022. "Transformative Scale." Website.
Innovative nonprofit leaders and philanthropists have developed several effective social programs over the past decade. Despite this progress, even the most successful programs reach only a small fraction of those in need. Bridging this gap to reach transformative scale requires new strategies that extend beyond the boundaries of any single organization—new forms of collaboration within and across sectors, new platforms for learning and spreading what works, and different types of funding models. We seek to increase the number of problems being addressed at a population-level of scale.
The Challenge Initiative. 2019. "How to Sustainably Scale Up Global Health Programs and Measure Progress." TCI's PAthways to Scale and Sustainability (PASS) Learning Series No. 2.
Complex global health problems abound with devastating impact on the lives of people and the productivity of societies. Despite these challenges, public health programs have demonstrated that major successes are possible, even in the poorest of countries. For example, beginning in the 1950s Sri Lanka’s government committed to extending safe motherhood services throughout the country, including rural areas, leading to remarkable declines in maternal mortality, from an estimated 500–600 maternal deaths per 100,000 live births in the 1950s to 30 deaths per 100,000 live births in 2015. Another case in point: In 20 endemic African and Asian countries, the prevalence of Guinea worm dropped by 99%, from 3.5 million cases in 1986 to fewer than 35,000 in 2003, through behavior change efforts among multiple partners. And within a span of less than 10 years, polio was eliminated as a threat to public health in 1991 in the Americas through a region-wide effort that immunized almost every young child. Many more success cases exist. At the same time, the global health field also struggles to move effective innovations out of pilot phases into large-scale implementation to maximize their impact. On average, it takes nine years for research evidence to be implemented into practice. A recent analysis demonstrated that accelerating scale-up of a hypothetical 20-year global health program by just one year could reach 10% more people, resulting in significant impact on lives saved. Large-scale thinking is necessary to meet the global health challenges of today, yet the scale-up field is typically under-resourced and nascent. Effective and sustainable scale-up requires more than just routine implementation. It requires extra thought, attention and planning, but few programs are focused specifically on scaling up effective interventions and practices.
The International Development Innovation Alliance (IDIA). 2017. "Good Practice Guides for Funders - Scaling Innovation."
This paper in the IDIA Insights series focuses on eight good practices for funders seeking to take promising development innovations to scale. It is designed to accompany the Insights on Scaling Innovation paper that draws on the experience and learning of a wide range of bilateral, multilateral, philanthropic and civil society actors who came together in a Working Group on Scaling Innovation facilitated by the International Development Innovation Alliance (IDIA). While these good practices do not represent the formal strategy or approach of any one single agency in the Working Group or IDIA itself, they do reflect areas of overlapping learning and experience that can be used as a point of reference for interested stakeholders in reflecting on, and enhancing, their own approaches and guidance on scaling innovations. Scaling innovation is a long, complex and dynamic process. The good practices contained herein will therefore benefit from regular review and iteration to accurately capture continuing advances in knowledge and learning. The insights collected in this paper are also likely to be valuable in helping innovators and partner organizations develop their own scaling approaches, thereby acting as a potential catalyst for deeper and more efficient partnerships. The members of IDIA are committed to supporting the co-creation of tools and knowledge products such as these Insights papers to inform and enhance their own innovation-related work and that of others in the global innovation community. The exchange of knowledge, learning and expertise that has characterized the development of this paper is an essential part of ensuring innovations intended to help accelerate achievement of the 2030 Sustainable Development Goals can be pursued and supported.
The International Development Innovation Alliance (IDIA). 2017. "Insights on Scaling Innovation."
This paper in the IDIA Insights series focuses on various challenges, lessons learned and practices of funders seeking to take promising development innovations to scale. It draws on the experience and learning of a wide range of bilateral, multilateral, philanthropic and civil society actors who came together in a Working Group on Scaling Innovation facilitated by the International Development Innovation Alliance (IDIA). While it does not represent the formal strategy or approach of any one single agency in the Working Group or IDIA itself, it does reflect areas of overlapping interest and terminology that can be used as a point of reference for interested stakeholders in reflecting on, and enhancing, their own approaches and guidance on scaling innovations. Scaling innovation is a long, complex and dynamic process. The insights contained herein will therefore benefit from regular review and iteration to accurately capture continuing advances in knowledge and practice. In its current form, this document provides a broad architecture intended to help funders as they navigate the challenging pathways associated with scaling innovation. The insights collected in this paper are also likely to be valuable in helping innovators and partner organizations develop their own scaling approaches, thereby acting as a potential catalyst for deeper and more efcient partnerships. The members of IDIA are committed to supporting the co-creation of tools and knowledge products such as these IDIA Insights papers to inform and enhance their own innovation-related work and that of others in the global innovation community. The exchange of knowledge, learning and expertise that has characterized the development of this paper is an essential part of ensuring innovations intended to help accelerate achievement of the 2030 Sustainable Development Goals can be pursued and supported.
Tom Graham. 2018. "How to design for scale: lessons for ambitious new interventions." Apolitical.
This piece is part of Apolitical’s spotlight series on scaling social impact, in partnership with the Bernard Van Leer Foundation. It also appears in our government innovation newsfeed. In many respects, the so-called Graduation Program has been a roaring success: it’s the only intervention proven to lift people out of extreme poverty. With food parcels, life skills classes and two years of asset transfers, it’s a complete package. Almost 100 countries have experimented with it, providing a rock-solid evidence base. So why isn’t it national policy across the world? The biggest determinant of whether an intervention will succeed at a huge scale, changing the lives of millions rather than dozens or hundreds or even thousands, is not whether it works, but whether it was designed in a way that made it suitable for scale in the first place. The path from developing a program that works to helping huge numbers of people is a long one, often taking 10 to 15 years. Organisations that don’t anticipate the difficulties will develop programs with features that make them inherently difficult or even impossible to scale up. The Graduation Program, for example, is expensive. And now governments that want to implement it are trying to cut costs — but they don’t know what to cut and what to keep without losing the effects. For reasons like this, success on a small scale is no indication, let alone a guarantee, of success on the big stage. Only interventions that are relatively simple, clearly better than the alternatives and not reliant on unique conditions are likely to scale well. But it can be done. The secret is to plan for that long journey right at the beginning and to design something that will not just survive, but thrive as it gets bigger. So how do you do that? The secret sauce The first thing to do is to understand the problem being addressed, and how the intervention solves it. "If you don’t know what that secret sauce is, you really don’t have a chance of scaling it" “Sometimes an innovation catches on, but we don’t actually know why or what is making it work,” said Whitney Pyles Adams, who runs the NGO CARE’s Scale X Design accelerator. “And if you don’t know what that secret sauce is, you really don’t have a chance of scaling it elsewhere.”
Tom Graham. 2018. "The world is scattered with pilot projects trying to work holistically: Q&A with Karen Levy of Evidence Action, the NGO that delivers solutions to hundreds of millions of people." Apolitical.
Evidence Action is an international NGO that bridges the gap between proven interventions and delivery at scale. Or rather, huge scale: they deal in the hundreds of millions. Evidence Action formally launched in 2013, and Levy has been with them from the beginning. She helped bring two of their flagship programs into operation. The first is Deworm the World. This intervention involves school-based, blanket drug administration to children in areas where worms are prevalent, with proven health and educational benefits. In 2016, almost 200 million children were treated across Africa and Asia. The second is No Lean Season. This targets seasonal poverty by offering agricultural households a travel subsidy so they can send a man to a nearby city, where he can find work and send money home. It is currently being tested in Bangladesh and Indonesia. As part of our special series on how to scale up social programs, and help millions or hundreds of millions of people rather a few thousand, we spoke to Levy about her work at Evidence Action. Your interventions are strikingly simple and scalable. How do you arrive at them? "We look for things that are elegant, well-defined and honed" When you’re expanding a program, how do you analyse prospective sites? "The world is scattered with pilot projects" Do you adapt the interventions to new contexts? Do you always design your interventions for adoption by government? "Often you need to design an intervention with government adoption in mind" Working at the scale you do, how do check whether the impact is being maintained? What’s the endgame?
USAID Health Care Improvement Project. 2008. "The Improvement Collaborative: An Approach to Rapidly Improve Health Care and Scale Up Quality Services." Published by the USAID Health Care Improvement Project. Bethesda, MD: University Research Co., LLC (URC).
Overview URC was the lead implementer for the USAID Applying Science to Strengthen and Improve Systems (ASSIST) cooperative agreement of the Office of Health Systems in USAID’s Global Health Bureau. ASSIST’s mission was to improve healthcare, strengthen health systems, and advance the frontier of improvement science in USAID-assisted countries. URC and its partners achieved these aims by: Fostering improvements in a range of healthcare processes through the application of modern improvement methods by host-country providers and managers Building the capacity of host-country systems to improve the effectiveness, efficiency, client-centeredness, safety, accessibility, and equity of the healthcare services provided Generating new knowledge to increase the effectiveness and efficiency of applying improvement methods in low- and middle-income countries Strengthening maternal, newborn, and child health (MNCH) and family planning/reproductive health services in Zika-affected countries in Latin America and the Caribbean USAID ASSIST built on the work of the USAID Health Care Improvement Project. From 2013 to 2020, ASSIST operated in over 40 countries, supporting quality improvement efforts for HIV and AIDS, MNCH, malaria, tuberculosis, Zika, orphans and vulnerable children, and non-communicable disease services at both facility and community levels.
USAID Maternal Child Health Survival Program. 2018. "Scale-up for Success." Website.
If countries want to reach their ambitious Sustainable Development Goals targets, they need to be systematic in identifying solutions with the most potential for high impact to prioritize in their national plans. Some of these high-impact solutions, like comprehensive emergency obstetric care, will already be part of most countries’ routine care, but will be in need of improvements in quality or targeting. However, not all proven, cost-effective solutions – such as chlorhexidine for newborn sepsis prevention and integrated community case management of child illness – are currently part of routine health care practice in many countries. Scale-up is the process of bringing such newer but proven interventions and strategies to more people on a sustained basis. When governments recognize an intervention can improve the population’s health and achieve significant impact, decision-makers decide to #ScaleForSuccess. Countries can begin this process by including these newer, proven interventions in their plans, ensuring that policies are in place to support their implementation, and seeking initial financing. To make solid progress toward widespread and sustained impact, countries will also need to make a multi-year commitment to a systematic but flexible process that can successfully respond to ongoing challenges. This requires leadership, partnership, and solid management at local, district and national levels. Those leaders need timely information for a process of learning and adaptation, and strategies for ongoing resource mobilization. This is neither quick nor easy, but those who engage in such a systematic process will be rewarded with sustained and significant improvements in the health of their populations.
USAID, Skoll Foundation, Mercy Corps. 2018. "Skoll-USAID Innovation Investment Alliance."
The Innovation Investment Alliance (IIA) is a funding and learning partnership between the Skoll Foundation and USAID’s Global Development Lab, with support from Mercy Corps, that has invested over $50 million in eight proven, transformative social enterprises to scale their impact. Program Status Update As of April 2017, the Innovation Investment Alliance has granted over $50 million to eight social enterprises operating in five different sectors. Objectives Together, we aim to create systems-level change across sectors and geographies and draw out lessons on scaling that are applicable to the social enterprise community and inform the ongoing conversation on how to create sustainable impact at scale.
USAID. 2015. "Idea to Impact: A Guide to Introduction and Scale of Global Health Innovations." Website.
The challenges of developing, introducing, and scaling global health products—whether they are medical devices, drugs, diagnostics, vaccines, or consumer products—are innumerable. Many activities are required, across many countries, and with many actors (donors, implementing partners, ministries of health, and manufacturers, to name a few , making coordination and eƍcient execution a tricky proposition. Perhaps more signiƋcantly, developing-country public and private markets lack the resources and health infrastructure typically seen in the developed world. As a result, it often takes years, sometimes decades, for products to reach most of their intended users. In spite of such challenges, signiƋcant progress has been made. Child mortality has been cut in half over the past two decades, thanks in part to signiƋcant advancements in how global health products are developed, introduced, and scaled. Insecticide-treated nets, for example, scaled rapidly in African countries after they were made widely available in campaigns and integrated into routine health care programs. Their widespread use, along with other malaria control eƊorts, reduced malaria mortality rates in children younger than 5 years by an estimated 54 percent between 2000 and 2012 (WHO 2013). Antiretroviral treatment for HIV scaled rapidly in Africa and Asia after several global partners negotiated volume discounts and committed to deploying antiretrovirals through national health systems. By 2012, 9.7 million HIV-positive people in low- and middle-income countries were receiving treatment, an achievement that has saved an estimated 4.2 million lives and prevented 800,000 infections in children (WHO, UNICEF, and UNAIDS 2013). More recently, the Meningitis Vaccine Project orchestrated the development and introduction of a novel, low-cost group A meningococcal meningitis vaccine in Africa. The Meningitis Vaccine Project and its many partners not only developed the vaccine in record time, they deployed it quickly, reaching 100 million people within 24 months of initial regulatory approval. The vaccine has already reduced the incidence of meningitis A in the ten target countries by 95 percent. Idea to Impact: A Guide to Introduction and Scale of Global Health Innovations consolidates and shares best practices and lessons learned from decades of scaling global health innovations and draws on best practices from the private sector, while oƊering a dynamic and ƌexible home for new thinking and advancements still to come. Many of the insights and examples are heavily informed by the learnings and practices of private companies, non-governmental organizations, academia, USAID and other donors, and other public health experts.
Uvin P and Miller D. 1996. "Paths to scaling-up: Alternative strategies for local nongovernmental organizations." Human Organization 55(3): 344-354.
This article proposes an initial scientific look at scaling-up, i.e., the process whereby non-governmental organizations (NGOs) increase their impact. It distinguishes four types of scaling-up—quantitative, functional, political and organizational—and discusses the paths organizations can employ to achieve these types. The article constitutes what can be called a "pre-theory:" the development of some clear definitions and taxonomies, which can constitute the basis for scientific investigation and comparative discussion. The conclusion of this article briefly presents two visions for approaching NGO scaling-up in its larger social context.
Uvin P, Jain PS, & Brown LD. 2000. "Think Large and Act Small: Towards a New Paradigm for NGO Scaling Up." World Development, 28, 1409-1419.
Scaling up is about “expanding impact” and not about “becoming large,” the latter being only one possible way to achieve the former. The experiences of five Indian nongovernment organizations (NGOs) suggest the emergence of a new paradigm of scaling up, in which NGOs become catalysts of policy innovations and social capital, creators of programmatic knowledge that can be spun off and integrated into government and market institutions, and builders of vibrant and diverse civil societies. We detail the mechanisms by which NGO impact can be scaled up without drastically increasing the size of the organization.

Keywords NGOs civil society Asia India advocacy scaling up
Uvin P, Miller D. 1994. "Scaling up: Thinking through the issues." The World Hunger Program, Watson Institute of International Studies, Brown University, Providence, RI.
The World Hunger Program is part of the Watson Institute of International Studies at Brown University. The 1970s and 1980s have witnessed a dramatic growth in the number, the size and the level of activity of non-profit/non-governmental organizations (NGOs) in the Third World. They include peasant associations, neighborhood associations, people's movements, community initiatives, urban action committees and intermediary NGOs, filling the ranks of what is often referred to as the "associative sector." This phenomenon parallels a slow but profound change in the international aid system. Since the middle of the 1970s, donor agencies have be come increasingly preoccupied with the issue of participation and have looked to Northern and Southern NGOs as the vehicles for carrying forward an agenda of participation. As a result, both the numbers of international and local NGOs and the number of do nor funded projects that have a participation component within them have grown remarkably over the past ten years. This growth in size, number and activities of organized participatory initiatives has become recognized within the development community as" scaling-up." It constitutes an objective, or desire, of many funders, field practitioners and scholars alike. However, some important questions about scaling-up need to be answered. What role do the participants that expand the ranks of the scaled up organizations play within these organizations or within their funded projects? Is there a relationship between donor support of scaling-up and constituent participation? In what way will this new interest in scaling-up affect the rank and file of the participation movement? Has the redirection of development assistance toward scaling-up initiatives been effective? As Edwards and Hulme stated it: "how can [NGOs] increase their development impact without losing their traditional flexibility, value-base and effectiveness at the local level?"[1] Together, these questions constitute a research agenda that, to date, has not received adequate attention. Indeed, most of the literature on grassroots development movements is normative in nature; the rest is largely anecdotal. It is as if the usual laws of scientific inquiry, verification and experimentation do not apply when dealing with organizations whose social goals have often eluded the same rigors of measurement administered among the bottom lines of government and for-profit organization.[2] This article proposes a first scientific look at scaling up. It does not present any grand theory of scaling up, nor is it the result of detailed comparative field research. Rather, it represents what can be called a "pre-theory:" the development of some clear definitions and taxonomies, which can constitute the basis for scientific investigation and discussion.
Uvin P. 1995. "Fighting hunger at the grassroots: Paths to scaling up." World Development: 23(6): 927-940.
This article proposes some clear definitions and taxonomies of scaling up, i.e., the processes by which grassroots organizations expand their impact. It then goes on to apply this taxonomy of scaling up to 25 Third World organizations that were nominated for the Alan Shawn Feinstein Hunger Awards, a yearly set of three awards given by Brown University to organizations that have been especially meritorious in combating or preventing hunger. This case study describes a number of paths for scaling up that might have general relevance.
Uvin P. 1999. "Scaling up, scaling down: NGO paths to overcoming hunger." In Scaling Up, Scaling Down – Overcoming Malnutrition in Developing Countries. Ed. Thomas J. Marchione. Australia: Gordon and Breach.
Over the last two decades, two profound changes, one institutional and one conceptual, have taken place in the fight against hunger and poverty. At the institutional level, tens of thousands of new community-based organizations have come into being: they now play key roles in the many facets of the fight to eradicate hunger and poverty. At the conceptual level. the definition of food security, and of the ways to achieve it. has undergone profound changes, away from a technical, primarily food production focus, towards an approach that is at the same time much more holistic and more political, involving empowerment and freedom, health and education. agriculture and off-farm employment. These two processes are linked: new actors propose new definitions. pressure old actors, who are in turn influenced by them. Thus, we now have a much greater institutional and conceptual diversity, providing at the same time new opportunities. challenges, and difficulties in the fight against hunger and poverty. In this respect, the veritable explosion of community-based, participatory. grassroots action in most of the Third World over the last two decades seems to be a most important and encouraging trend. Throughout the world, there are literally millions of grassroots organizations (GROs) and tens of thousands of non-governmental organizations (NG0s) seeking to increase agricultural production. improve basic health care. increase poor people's incomes, design safety nets against exceptional entitlement shun-falls. improve the quality of the environment. change dietary practices. curb fertility rates, and assure access to land, water, and market opportunities. Their creativity and diversity arc unparalleled. Yet. recognizing the importance of these organizations in the fight against hunger does not imply that there is no longer a role for governments, international organizations, private enterprises, research institutions, aid agencies, and the like. All these institutions remain crucial in the fight against hunger: they have financial, legal, and intellectual resources that NGOs and GROs are unlikely ever to possess. Hence, even within a vision that stresses the capacities and creativity of local communities, it remains true that governments and international organizations are crucial actors in the fight against hunger. The issue is not their destruction, or neglect, but rather the creation of links between them and the grassroots. What is required, then, is the creation of interactions between the grass-roots and what can be called "the summit." in ways that are beneficial to local communities and poor people. This question of how to "link the grassroots to the summit" is at the cutting edge of current thinking about development strategies. Linking the grassroots to the summit implies two processes. One is scaling up, a term referring to the process by which grassroots organizations expand their impact and enter into relations with the summit.
Varvasovsszky Z, Brugha R. 2000. "How to do (or not to do) . . . a stakeholder analysis." Health Policy and Planning, 15:338-345.
This paper provides guidance on how to do a stakeholder analysis, whether the aim is to conduct a policy analysis, predict policy development, implement a specific policy or project, or obtain an organizational advantage in one's dealings with other stakeholders. Using lessons learned from an analysis of alcohol policy development in Hungary, it outlines issues to be considered before undertaking the stakeholder analysis concerning the purpose and time dimensions of interest, the time-frame and the context in which the analysis will be conducted. It outlines advantages and disadvantages of an individual or team approach, and of the use of insiders and outsiders for the analysis. It describes how to identify and approach stakeholders and considers the use of qualitative or quantitative data collection methods for estimating stakeholder positions, levels of interest and influence around an issue. A key message is that the process of data collection and analysis needs to be iterative; the analyst needs to revise and deepen earlier levels of the analysis, as new data are obtained. Different examples of ways of analyzing, presenting and illustrating the information are provided. Stakeholder analysis is a useful tool for managing stakeholders and identifying opportunities to mobilize their support for a particular goal. However, various biases and uncertainties necessitate a cautious approach in using it and applying its results.
Vicki B, Huong T, Miranda B, Rachel L, Marj M. 2020. "A narrative review of economic constructs in commonly used implementation and scale-up theories, frameworks and models." Health Research Policy Systems 2020; 18: 115.
Translating research evidence into practice is challenging and, to date, there are relatively few public health interventions that have been effectively and cost-effectively implemented and delivered at scale. Theories, models and frameworks (herein termed ‘frameworks’) have been used in implementation science to describe, guide and explain implementation and scale-up. While economic constructs have been reported as both barriers and facilitators to effective implementation and scale-up of public health interventions, there is currently no published review of how economic constructs are considered within commonly used implementation and scale-up frameworks. This paper aimed to narratively review the economic constructs incorporated in commonly used implementation and scale-up frameworks.

Frameworks for inclusion in the narrative review were identified from the literature and thematic content analysis was undertaken using a recursive deductive approach. Emergent key themes and sub-themes were identified and results were summarised narratively within each theme.

Twenty-six framework publications were included in our analysis, with wide variation between frameworks in the scope and level of detail of the economic constructs included. Four key themes emerged from the data – ‘resources’, ‘benefit’, ‘cost’ and ‘funding’. Only five frameworks incorporated all four identified key themes. Overarching lenses from which to consider key themes included ‘stakeholder perspectives’, ‘stage in the research translation process’ and ‘context’. ‘Resources’ were most frequently considered in relation to the sub-themes of ‘types of resources’ (e.g. labour, time or infrastructure) and ‘availability’ of resources, and the opportunity for ‘economies of scale’. The ‘relative advantage of interventions’ emerged as an interconnecting sub-theme between ‘cost’ and ‘benefit’. ‘Funding’ was most often considered in relation to ‘funding sources’, ‘availability’, ‘sustainability’ or ‘contextual impact’. The concept of ‘opportunity cost’ was considered in relatively few frameworks, despite being fundamental to economic theory.

Implementation and scale-up frameworks provide a conceptual map to inform the effective and cost-effective implementation of public health interventions delivered at scale. Despite evidence of an emerging focus on the economic considerations of implementation and scale-up within some commonly used frameworks, our findings suggest that there is significant scope for further exploration of the economic constructs related to implementation and scale-up. Keywords: Implementation, scale-up, theories, models, frameworks, economic
Victora C, Hanson K, Bryce J, Vaughn JP. 2004. "Achieving universal coverage with health interventions." The Lancet 364(9444): 1541-1548.
Cost-effective public health interventions are not reaching developing country populations who need them. Programmes to deliver these interventions are too often patchy, low quality, inequitable, and short-lived. We review the challenges of going to scale—ie, building on known, effective interventions to achieve universal coverage. One challenge is to choose interventions consistent with the epidemiological profile of the population. A second is to plan for context-specific delivery mechanisms effective in going to scale, and to avoid uniform approaches. A third is to develop innovative delivery mechanisms that move incrementally along the vertical-to-horizontal axis as health systems gain capacity in service delivery. The availability of sufficient funds is essential, but constraints to reaching universal coverage go well beyond financial issues. Accurate estimates of resource requirements need a full understanding of the factors that limit intervention delivery. Sound decisions need to be made about the choice of delivery mechanisms, the sequence of action, and the pace at which services can be expanded. Strong health systems are required, and the time frames and funding cycles of national and international agencies are often unrealistically short.
Watkins A. 2018. "Takeaways and Policy Recommendations: Global Solutions Summit 2018." Global Solutions Summit.
The Global Solutions Summit (GSS 2018) convened at the United Nations Headquarters in New York City on June 4, 2018 with support from the UN Department of Economic and Social Affairs, the UN Conference on Trade and Development, the UN Commission on Science and Technology for Development, and the UN Office of Partnerships. The official theme of GSS 2018 was “From Lab to the Last Mile: Technology Deployment Business Models for the SDGs.” 1 Why is technology deployment relevant for the SDGs? At a recent speech to the UN Food and Agricultural Organization in Rome, Dr. Akinwumi Adesina, President of the African Development Bank, answered this question simply, succinctly, and eloquently when he noted, “Technologies to achieve Africa’s green revolution exist. For the most part, they are all just sitting on the shelves.”
Williams JKA, Phillips JF, Bawah AA. 2019. "Scaling down to scale-up: a strategy for accelerating Community-based Health Service Coverage in Ghana." 1. 10.35500/jghs.2019.1.e9.
Calls for evidence-based action are as old as writing itself. Exhortations to learn-by-doing recur whenever health system change is proposed. Learning by doing benefits from scaling operations down to tractable levels of management, testing changes that may or may not improve operations, and scaling up innovation based on lessons learned. However, the many small-scale projects and pilot studies that have proliferated in the health sector in recent decades often end without their large-scale utilization, in part because scaling up is a label signaling the end of research rather than a paradigm for generating continuous evidence to guide a sustained process of organizational change. This paper presents Ghana's experience with sequential “scaling down” of operations as a continuous process of evidence-driven scaling up. While the Sophoclean process of “learning by doing” is more complex than this phrase connotes, the Ghana example represents a practical demonstration of a paradigm for scaling up that retains the advantages of small scale “learning by doing” as a means of catalyzing the pace, operational integrity, and coverage of scaling-up. Commencing with policy responses to the 1978 Alma Ata Global Health Conference, improving access to primary health care has remained a pillar of health policy in Ghana ever since. Despite a full decade of investment in large scale policy action during the 1980s, evidence emerging in the early 1990s established that implementation of policies had failed.3 In 1992, the Ministry of Health convened an advisory committee to review problems and seek solutions that would define a new course for community health care programming. But policy debate ensued over the optimum configuration of community worker cadres and feasible means of financing community worker deployment costs. To resolve debate, a process of learning-by-doing was launched that has continued to the present (Fig. 1). The process that has been applied is adapted from a paradigm sponsored by World Health Organization to scaling up pilot project innovations.4 Specific attention in Ghana was directed to transferring the Bangladesh model for phased in research to its program development needs.5 However, the operational history of the Ghana model contrasts with antecedent Asian programs in important ways. We are therefore providing a synopsis of its history with the goal of extracting lessons that could inform program development elsewhere in Africa. This national program, founded in 1999 and implemented since 2000, is known as the Ghana Community-based Health Planning and Services (CHPS) Initiative
Wils F. 1995. "Scaling-up, mainstreaming and accountability: The challenge for NGOs." In Non-Governmental Organisations: Performance and Accountability beyond the Magic Bullet. Eds. Michael Edwards and David Hulme. London, England: Earthscan.
The last decade has seen some significant changes in international development and in the status of non-governmental organisations operating in the field. Not only has the number of NGOs virtually doubled; many of them have seen a considerable growth in their budgets, and have grown closer to governments and official aid agencies. NGOs are acknowledged by many to be more effective agents of development than governments or commercial interests ? even as a ?magic bullet? for development problems. Despite these positive trends, the real impact of the NGO sector is not well documented. This is partly because NGO performance-assessment and accountability methods are weak, and partly because NGOs are caught up increasingly in the world of official aid, which pushes them towards certain forms of evaluation at the expense of others. This unique book takes a hard and critical look at these issues, and describes how NGOs can, and must, improve the way they measure and account for their performance if they are to be truly effective.
World Health Organization, Geneva. 2007. "Introducing WHO's guidelines and tools in reproductive health into national programmes: principles and process of adaptation and implementation."
It is important that those embarking on a process to introduce a new WHO guideline or tool into their sexual and reproductive health-care programme consider that these materials are not simply documents that must be distributed, but rather that they contain health-care practices which must be introduced to providers through a well-planned process of adaptation and implementation. Procedures to implement guidance and change practices can seem long and overwhelming, but it is important to follow the principles and processes suggested in this document if change is to be sustained at the service delivery level. Even the simple updating of national guidelines with new recommendations may require other programmatic interventions, such as training or the development of supervisory materials, to ensure that updates are implemented. When the implementation of new guidance requires changes in the way services are delivered, a comprehensive introductory process is usually necessary. A collaborative and participatory adaptation process fosters ownership and buy-in among policy-makers, professional bodies and other national experts. Once a national version has been created or updated and then endorsed, the process to implement the new guidance at the service delivery level must be carefully planned. Providers can only implement a new practice it (a) they have been convinced of the need for it; (b) they have been trained in how to carry it out; (c) they are encouraged by supervisors to adopt it; (d) they have materials (guidelines, job aids, checklists) to support them in implementing the practice; and (e) their service delivery environment enables them to implement it. It is often tempting to disseminate guidelines without taking these issues into consideration, but experience has shown that, without due attention to the points mentioned, changes in quality of care will not be sustained.
World Health Organization, Geneva. 2007. "World Health Organization Strategic Approach."
Beginning with the end in mind: planning pilot projects and other programmatic research for successful scaling up Nine steps for developing a scaling-up strategy A strategic approach to strengthening control of reproductive tract and sexually transmitted infections: use of the programme guidance tool Practical guidance for scaling up health service innovations Scaling up health service delivery: from pilot innovations to policies The WHO Strategic Approach to strengthening sexual and reproductive health policies and programmes A strategic assessment of reproductive health in the Lao People’s Democratic Republic Abortion in Viet Nam: An assessment of policy, programme and research issues An Assessment of the contraceptive method mix in Myanmar An Assessment of the need for contraceptive introduction in Zambia An Assessment of the need for contraceptive introduction in Viet Nam An Assessment of the need for contraceptive introduction in Brazil Contraceptive introduction reconsidered : a review and conceptual framework
World Health Organization. 2016. "Scaling up projects and initiatives for better health: from concepts to practice." Copenhagen: World Health Organization Regional Office for Europe.
Scaling up means to expand or replicate innovative pilot or small-scale projects to reach more people and/ or broaden the effectiveness of an intervention. Based on a narrative literature review and a survey targeting key informants from 10 WHO Member States that are also members of the Regions for Health Network (RHN), this publication addresses practical challenges and provides a tool box for scaling up activities. This publication integrates and describes tools from different practical guidelines. It is structured in line with a scaling-up guideline developed for New South Wales (Australia). Using all of the presented tools in a systematic manner is often not possible for practitioners. But with references to frameworks, models and practical experiences, WHO and RHN hope to raise awareness of critical promoting or hindering factors, to encourage utilization of supportive tools, and to promote the further exchange of experiences and practical knowledge.

World Health Organization. 2009. "The WHO Strategic Approach to strengthening sexual and reproductive health policies and programmes." Geneva.
Faced with the challenge of putting into practice the ideals of the Millennium Development Goals, the International Conference on Population and Development (ICPD), and other global summits of the last decade, decision-makers and programme managers responsible for sexual and reproductive health ask how they can: improve access to and the quality of family planning and other sexual and reproductive health services; increase skilled attendance at birth and strengthen referral systems; reduce the recourse to abortion and improve the quality of existing abortion services; provide information and services that respond to young people’s needs; and provide information and services that respond to young people’s needs; and integrate the prevention and treatment of reproductive tract infections, including HIV/AIDS, with other sexual and reproductive health services. To help answer these and other such questions, 25 countries have used the WHO-sponsored Strategic Approach to Strengthening and Reproductive Health Policies and Programmes. Public-sector health-care programmes, in collaboration with non-governmental organizations and international agencies, typically use the Strategic Approach. It involves a three-stage process for assisting countries to assess reproductive health needs and priorities, test policies and programme adaptations to address these needs, and then scale up successful interventions. The Strategic Approach unites concepts and practices from public health and the social and management sciences with the principles of the ICPD in its essential features: a staged implementation process that links assessment, pilot-testing, and scaling-up a systems framework to highlight the relevant factors for decision-making about appropriate services; a reproductive health philosophy of reproductive rights, gender equity, and empowerment; a focus on improving equitable access to and quality of care so that services are client-centred and responsive to community needs; a participatory process to consider the concerns of all relevant stakeholders; and country ownership of the process and the results.
Worsham E, Langsam K, Martin E. 2019. "People Matter: Evolving Talent to Drive Impact at Scale." Scaling Pathways, Innovation Investment Alliance, Skoll Foundation, CASE at Duke.
“How much time does your leadership team spend on culture and people? Are these topics on the agenda of your management meetings?” These are questions that Maryana Iskander, CEO of Harambee Youth Employment Accelerator, encourages scaling organizations to ask themselves. She strongly believes that the work of defining culture and growing people ultimately drives every other performance metric in the organization— from strategy and partnerships to operational success and delivery. But what are the key talent-related topics and questions that management teams need to add to the agenda as they work to achieve impact at scale? Hint: they are not the same ones that the organization tackled during start-up and validation. As we spoke with social enterprise leaders, we found that their stories and advice on this talent evolution fell into three major categories:
Worsham E, Langsam K, Martin E. 2020. "Using Data to Power Scale." Scaling Pathways, Innovation Investment Alliance, Skoll Foundation, CASE at Duke.
Drawing on the perspectives and experience of some of the world’s leading social enterprises, this paper lays out key strategies and advice on how to use data to more effectively and efficiently scale impact. Our interviewees’ advice was three-fold: 1) to lay the foundation for data efforts by carefully considering equity and client voice; 2) to set the data building blocks—the how, what, and who of data—that will be critical for driving scale; and, with all that in place, 3) to pursue more advanced data approaches that align with the specific scaling strategies that you are pursuing.
Yale University. 2022. "Y-RISE." Website.
The Yale Research Initiative on Innovation and Scale (Y-RISE) advances research on the effects of policy interventions when delivered at scale. While evaluation techniques for pilot-scale programs are well developed, complexities arise when we contemplate scaling up interventions to create policy change.
Yamey G. 2011. "Scaling up global health interventions: A proposed framework for success." PLoS Med. 8:e1001049
The rise in international aid to fund large-scale global health programs over the last decade has catalyzed interest in improving the science of scale-up. This Essay draws upon key themes in the emerging science of large-scale change in global health to propose a framework for explaining successful scale-up. Success factors for scaling up were identified from interviews with implementation experts and from the published literature. These factors include the following: choosing a simple intervention widely agreed to be valuable, strong leadership and governance, active engagement of a range of implementers and of the target community, tailoring the scale-up approach to the local situation, and incorporating research into implementation. The adoption of the Millennium Development Goals—coupled with the recent rise in international aid for health—has catalyzed interest in improving the science of scale-up [1]. Global health researchers have realized the need for “a quantitative, scientific framework to guide health-care scale-up in developing countries” [2], a need that has begun to draw the attention of donors [3]. Low- and middle-income countries (LMICs) have begun to study effective ways to deliver proven interventions at scale [4],[5]. Thus, there are promising signs that a “science of large-scale change in global health” is emerging [5]. In this Essay, I draw upon key themes in this emerging science to propose a framework for explaining successful scale-up. This framework is aimed at planners of scale-up processes to use in thinking about strategies for implementing a new program, policy, or intervention to scale. The term “scaling up” is now widely used in the public health literature, but there is no agreed definition. The term is primarily used, say Mangham and Hanson, to describe “the ambition or process of expanding the coverage of health interventions” [6], a working definition that I use in this article.
Yamey G. 2012. "What are the barriers to scaling up health interventions in low and middle income countries? A qualitative study of academic leaders in implementation science." Globalization and Health, 8:11.
Most low and middle income countries (LMICs) are currently not on track to reach the health-related Millennium Development Goals (MDGs). One way to accelerate progress would be through the large-scale implementation of evidence-based health tools and interventions. This study aimed to: (a) explore the barriers that have impeded such scale-up in LMICs, and (b) lay out an “implementation research agenda”—a series of key research questions that need to be addressed in order to help overcome such barriers.

Interviews were conducted with fourteen key informants, all of whom are academic leaders in the field of implementation science, who were purposively selected for their expertise in scaling up in LMICs. Interviews were transcribed by hand and manually coded to look for emerging themes related to the two study aims. Barriers to scaling up, and unanswered research questions, were organized into six categories, representing different components of the scaling up process: attributes of the intervention; attributes of the implementers; scale-up approach; attributes of the adopting community; socio-political, fiscal, and cultural context; and research context.

Factors impeding the success of scale-up that emerged from the key informant interviews, and which are areas for future investigation, include: complexity of the intervention and lack of technical consensus; limited human resource, leadership, management, and health systems capacity; poor application of proven diffusion techniques; lack of engagement of local implementers and of the adopting community; and inadequate integration of research into scale-up efforts.

Key steps in expanding the evidence base on implementation in LMICs include studying how to: simplify interventions; train “scale-up leaders” and health workers dedicated to scale-up; reach and engage communities; match the best delivery strategy to the specific health problem and context; and raise the low profile of implementation science.
Zamboni K, Schellenberg J, Hanson C, Betran AP, Dumont A. 2019. "Assessing scalability of an intervention: why, how and who?" Health Policy and Planning, 2019, 1–9 doi: 10.1093/heapol/czz068
Public health interventions should be designed with scale in mind, and researchers and implementers must plan for scale-up at an early stage. Yet, there is limited awareness among researchers of the critical value of considering scalability and relatively limited empirical evidence on assessing scalability, despite emerging methodological guidance. We aimed to integrate scalability considerations in the design of a study to evaluate a multi-component intervention to reduce unnecessary caesarean sections in low- and middle-income countries. First, we reviewed and synthesized existing scale up frameworks to identify relevant dimensions and available scalability assessment tools. Based on these, we defined our scalability assessment process and adapted existing tools for our study. Here, we document our experience and the methodological challenges we encountered in integrating a scalability assessment in our study protocol. These include: achieving consensus on the purpose of a scalability assessment; and identifying the optimal timing of such an assessment, moving away from the concept of a one-off assessment at the start of a project. We also encountered tensions between the need to establish the proof of principle, and the need to design an innovation that would be fit-for-scale. Particularly for complex interventions, scaling up may warrant rigorous research to determine an efficient and effective scaling-up strategy. We call for researchers to better incorporate scalability considerations in pragmatic trials through greater integration of impact and process evaluation, more stringent definition and measurement of scale-up objectives and outcome evaluation plans that allow for comparison of effects at different stages of scale-up.

Scale-up, scalability, evaluation

Tools to Support Scale Up

Aytekin D, Birol E. 2021. "HarvestPlus Updates Biofortification Priority Index with More Crops and Features." HarvestPlus.
What is the Biofortification Priority Index (BPI)? The BPI is an analytical tool developed by Harvestplus to inform data-driven decisions for scaling up biofortification. It is also recognized as one of the notable innovations of the CGIAR global agricultural research partnership over the past 50 years. The BPI helps identify where investments in biofortification can have the biggest bang for the buck among 128 countries that the Index covers. The BPI provides rankings of country-level investment potential for 13 types of staple crops. What is new on the BPI? The BPI has been upgraded and expanded to include: - more biofortified crops: iron lentil, cowpea, and Irish potato; vitamin A banana/plantain; and zinc sorghum. They add to the previously available iron bean and pearl millet; vitamin A cassava, maize, and orange sweet potato; and zinc maize, rice, and wheat; - new releases of biofortified crop varieties available for scaling in the 128 countries, - new evidence on the nutrition impact of biofortified crops; and - more functions, including downloadable maps and data. Who uses the BPI? Interactive BPI website is a one-stop shop for anyone involved or interested in the development of biofortified varieties and scaling up biofortification. It is of particular use to: - Crop breeders, to guide research priorities for biofortification breeding; - NGOs and humanitarian organizations, to identify target locations for programs; - Private sector actors, to identify profitable investment opportunities for biofortified products (e.g., seed and food); - International financial institutions and governments, to guide decision-making on the introduction and scaling of biofortified crops.
Ben Charif, A, Zomahoun, HTV, Gogovor, A. Abdoulaye Samri M, Massougbodji J, Wolfenden L, Zwarenstein M, Milat AJ, Rheult N, Ousseine YM, Salerno J, Markle-Reid M, Légaré F. 2022. "Tools for assessing the scalability of innovations in health: a systematic review." Health Res Policy Sys 20, 34 (2022).
The last decade has seen growing interest in scaling up of innovations to strengthen healthcare systems. However, the lack of appropriate methods for determining their potential for scale-up is an unfortunate global handicap. Thus, we aimed to review tools proposed for assessing the scalability of innovations in health. Methods We conducted a systematic review following the COSMIN methodology. We included any empirical research which aimed to investigate the creation, validation or interpretability of a scalability assessment tool in health. We searched Embase, MEDLINE, CINAHL, Web of Science, PsycINFO, Cochrane Library and ERIC from their inception to 20 March 2019. We also searched relevant websites, screened the reference lists of relevant reports and consulted experts in the field. Two reviewers independently selected and extracted eligible reports and assessed the methodological quality of tools. We summarized data using a narrative approach involving thematic syntheses and descriptive statistics. Results We identified 31 reports describing 21 tools. Types of tools included criteria (47.6%), scales (33.3%) and checklists (19.0%). Most tools were published from 2010 onwards (90.5%), in open-access sources (85.7%) and funded by governmental or nongovernmental organizations (76.2%). All tools were in English; four were translated into French or Spanish (19.0%). Tool creation involved single (23.8%) or multiple (19.0%) types of stakeholders, or stakeholder involvement was not reported (57.1%). No studies reported involving patients or the public, or reported the sex of tool creators. Tools were created for use in high-income countries (28.6%), low- or middle-income countries (19.0%), or both (9.5%), or for transferring innovations from low- or middle-income countries to high-income countries (4.8%). Healthcare levels included public or population health (47.6%), primary healthcare (33.3%) and home care (4.8%). Most tools provided limited information on content validity (85.7%), and none reported on other measurement properties. The methodological quality of tools was deemed inadequate (61.9%) or doubtful (38.1%). Conclusions We inventoried tools for assessing the scalability of innovations in health. Existing tools are as yet of limited utility for assessing scalability in health. More work needs to be done to establish key psychometric properties of these tools.
Center for Accelerating Innovation and Impact 2016. "Pathways to scale: A guide for early-stage global health innovators on business models and partnership approaches to scale-up." USAID.
Pathways to Scale aims to help early-stage innovators develop business models and partnership approaches that align with the development of their products, and envision potential pathways to bring products to scale. It introduces the most commonly found models for scaling up global health innovations, and features case studies that highlight and explain pathways taken by innovations that have begun to scale-up. It also offers a toolkit with exercises, structured questions, key considerations, and curated resources that innovators can use to identify the most suitable scaling model(s) to forge their path.
Clark C, Massersky C, Schweitzer Raben T, Worsham E. 2012. "Scaling Social Impact: A Literature Toolkit for Funders." Growth Philanthropy Network and Duke University.
With support from Grantmakers for Effective Organizations (GEO), the authors set out to document and analyze the currently available literature on scaling, in order to both highlight the best set of resources available that are useful to funders actively pursuing grantmaking strategies around scaling impact, and to shine a light on what still needs to be studied and explored. This report is a compendium of the main findings of that work, and is arranged in the form of a funder-facing literature review (with materials selected based on their usefulness to funders and grantmakers interested in supporting scaling initiatives and listed in priority order), with links and abstracts, along with recommendations for future work.
Cooley L, Ved R, Fehlenberg K. 2021. "Scaling up--From vision to large-scale change: Tools for practitioners" 2nd Edition, Management Systems International (MSI), Washington DC.
Cooley L, Ved R, Fehlenberg K. 2012. "Scaling up—from vision to large scale change: Tools and techniques for practitioners." Management Systems International (MSI), Washington DC.
The FRAMEWORK and various tools have also been applied in fields as diverse as food security, livelihoods, local government, early childhood education, judicial sector reform, and community policing. Field managers working in development implementing agencies (government or non); Staff and managers at funding agencies (governments, international donors, and private foundations) interested in scaling up their programs or integrating scaling up into the design of new programs; Academics in professional fields like public health, public policy, social welfare, international affairs, and international development; Monitoring and evaluation practitioners interested in integrating scaling up into the design and implementation of their monitoring and evaluation work or in managing the quality of the scaling up process.
Cooley L, Ved R, Fehlenberg. 2021. "Tool 13: Institutionalization Tracker" "Scaling up--From vision to large-scale change: Tools for practitioners," 2nd Edition, Management Systems International (MSI), Washington DC
ExpandNet website/tools. 2005. "Our tools" Website.
Scaling up health service delivery presents the ExpandNet framework based on extensive literature review followed by seven in-depth country case studies of scaling up in Africa; Asia and Latin America. The Implementation Mapping Tool (IMT) helps project teams use learning questions to conduct participatory reflection, adaptive management and documentation of the process of scaling up health and development interventions. 20 questions for developing a scaling-up case study can be used retrospectively or prospectively to document scaling-up experience. Our Practical Guidance tool can support preparing documentation related to these questions. The Beginning with the end in mind tool offers recommendations for designing and implementing a pilot, demonstration or implementation research project with scaling up in mind. It is also helpful to make mid-course corrections. The Nine Steps for developing a scaling-up strategy tool and associated worksheets are based on the ExpandNet framework and provide stepwise guidance to develop a scaling-up strategy. It can also support the management of the scaling-up process. The Practical Guidance tool identifies general scaling-up principles and provides examples from case studies of successful scale-up initiatives. It is helpful at the design stage, during implementation and to manage the scaling-up process.
ExpandNet, Management Systems International, World Health Organization. 2007. "20 questions for developing a scaling-up case study."
Scaling up is a major challenge facing many donors, implementing agencies, and developing country governments. This challenge has stimulated operations research on how to do scaling up more effectively. In response to this challenge, two teams have recently developed strategic and operational frameworks for scaling up. One framework is the product of a collaborative effort by the WHO and the University of Michigan School of Public Health, the other was created by Management Systems International, an international development management consulting firm. The two frameworks were developed independently but with a great deal of collaboration and sharing of ideas over the last few years, and the results show a substantial consensus on what are the key elements and steps in scaling up. While these frameworks represent a significant advance in the field, both teams recognize that there is still much more to be learned about how to improve strategic management of scaling up. That is because in large part, as of this writing it has not been easy to learn lessons from existing field experience. In developing their respective frameworks, both teams have found that the existing case study literature on scaling up generally, and health services in developing countries in particular, is weak. There are few documented cases that have been published, and those that do exist are so-called “gray market” literature – internal documents, reports to donors or funders, or at best published on organizational websites. As many tend to be summaries or reports on projects as a whole which happen to be scaled up, rather than focusing on scaling up per se, it is often difficult to locate them with standard search methods. (This is why the WHO/Univ. of Michigan collaboration have created ExpandNet/WHO, a global network of public health professionals and scientists seeking to advance the practice and science of scaling-up successful health service innovations. A major focus of ExpandNet’s efforts is increasing the case literature and knowledge sharing around scaling up). Even those cases which do exist and focus on scaling up are usually not documented in ways which allows the reader to identify lessons learned about the process and management of scaling up itself. They tend, following the trend in scientifically-oriented academic medical and health journals, to emphasize a description of the project, model or innovation which was being tested, the results of the pilot project, and the success or failure of efforts to scale up that project. They do not talk about what the goals of scaling up were (and who determined them), how scaling up was done, and who did it. They rarely describe the political, health sector, or social environment in which scaling up occurred, the challenges and opportunities that environment presented, and how those challenges and opportunities were overcome and leveraged, respectively.
ExpandNet, World Health Organization. 2009. "Worksheets for developing a scaling-up strategy." Geneva.
The Nine Steps for developing a scaling-up strategy tool and associated worksheets are based on the ExpandNet framework and provide stepwise guidance to develop a scaling-up strategy. It can also support the management of the scaling-up process.
International Fund for Agricultural Development (IFAD). 2021. "Scaling Up eLearning."
Jacobs F, Ubels J, Woltering L, Boa-Alvarado M. 2021. "The Scaling Scan- A practical tool to determine the strengths and weaknesses of your scaling ambition (2nd edition)." Published by the PPPlab and CIMMYT.
Scaling of innovations is the process of expanding 2 the use of beneficial technologies or practices over geographies and across organizations to impact larger numbers of people. Reaching impact at scale is gaining importance, especially in light of the ambitions of the Sustainable Development Goals (SDGs). However, the positive outcomes of innovations in (pilot) projects are hardly ever replicated at scale, and if they do, the outcomes achieved often fade out after the project ends. A few important learnings about scaling are:
• Scaling of innovations requires a conducive policy, financial, social and busines environment and if these are not present, it needs supporting innovations to improve those conditions.
• Scaling of innovations should be a means to an end. More is not always better; agriculture is the largest user and polluter of water, and one of the biggest threats to biodiversity, for example.
• Reaching high numbers of adopters during projects is important but how they are reached is more important. After all, we want impact at scale to sustain and grow beyond the project boundaries (time and space). Meaningful scaling is about designing for sustainable and responsible change at scale from the beginning, nurturing local ownership and leadership, and learning and adjusting along the journey. Scaling of innovations is one the key pieces of the puzzle to achieve impact at scale, along with changing in conditions that hold problems in place (root causes) and supporting innovations that contribute to an enabling environment. Why the Scaling Scan? The multi-dimensional nature of scaling an innovation has discouraged scaling practitioners to engage in meaningful scaling, count adopters at the end of the project and assume others will continue scaling when project support cedes. The Scaling Scan provides a user-friendly tool to explore what is required to scale an innovation in a particular context, the implications this has for project management and collaborations and the potential trade-offs on the environment as well as social dynamics. At the core of the Scaling Scan are ten scaling ingredients that constitute the enabling environment around an innovation, and largely influence whether a scaling ambition can be realistically achieved. The Scaling Scan can be used to:
• facilitate discussion and develop capacities on scaling of a range of stakeholders;
• help formulate a realistic, context-specific and responsible scaling ambition for a selected innovation; • rapidly scan for bottlenecks and opportunities of a scaling initiative and generate immediate information to adjust strategies or identify needs for new collaborations;
• check whether project proposals, implementation plans, and evaluations misses important scaling elements. Who is the Scaling Scan for? The Scaling Scan is designed for anyone involved in pro-poor and sustainable development programs looking to scale impact. Project coordinators, managers and teams who manage project resources and priorities will be those most able to take advantage of the tool. It is specifically designed to develop capacity of nonscaling experts to recognize, and make sense of, scaling in their context. We highly recommend involving a diverse set of stakeholders for richer and more deliberate choices on what needs to be addressed and done to achieve the scaling ambition. Furthermore, the Scaling Scan can be applied:
• Within a range of sectors, despite being based on experience from the agriculture and the water sector
• By individuals as well as (project) teams
• By individual organizations and partnerships
Kohl R, Foy C. 2018. "Agricultural scalability assessment tool (ASAT)." USAID.
USAID’s Bureau for Food Security (BFS) and country missions have been implementing the Feed the Future (FTF) food security initiative since 2010. In many cases, small-scale innovations developed and introduced by FTF have since scaled up or are in the process of doing so. However, some innovations that could have gone to scale have not done so, have not reached their full-scale potential, or are not fully sustainable at scale. At the same time, the BFS has funded research by the Consortium of International Agronomic Research Centers (CGIAR) and innovation laboratories at major U.S. agricultural universities. This research has produced hundreds of innovations with varying potential to transform agriculture in developing countries, as well as more that are moving through the research pipeline. The Agency needs to be able to decide which innovations have the greatest potential for both successful scaling and significantly improving food security and reducing malnutrition across FTF countries and elsewhere. To this end, BFS asked the E3 Analytics and Evaluation Project, led by Management Systems International (MSI), to develop a toolkit to assess the scalability of agricultural innovations. The resulting Agricultural Scalability Assessment Toolkit (ASAT) draws on 15 years of experience by MSI and its team lead for this research, Dr. Richard Kohl, in scaling innovations and programs in the developing world, as well as on the literature on scaling and diffusion of innovation. This work includes extensive experience assisting FTF project design and strengthening scaling strategies, and five case studies the MSI team conducted of successful scaling up of agricultural innovations through commercial pathways in developing countries.
Management Systems International. 2021. "Scaling Development Outcomes." Webpage.
Since 2003, MSI has been deeply engaged in efforts to study and advance the successful and sustainable scaling of social, economic and humanitarian outcomes. MSI serves as the secretariat for the Global Community of Practice on Scaling Development Outcomes, and MSI experts have authored many of the pioneering contributions on scaling. MSI representatives also frequently serve as scaling advisors to a wide range of foundations, official donors and social entrepreneurs. Our scaling efforts span a variety of sectors with application in education, agriculture, health, governance, youth employment and child welfare.
Maternal and Child Survival Program. 2020. "Basic Toolkit for Systematic Scale-up: A companion to the Scale-up Coordinator’s Guide for Supporting Country-Led Efforts to Systematically Scale-up and Sustain Reproductive, Maternal, Newborn, Child and Adolescent Health Interventions." United States Agency for International Development (USAID).
This guide is for those supporting a systematic process of scale-up. Although the process can be managed successfully in various ways, we wrote this guide specifically with the perspective that there is a “scale-up coordinator” or scale-up manager. The concept for this figure is based on that used by the United States Agency for International Development’s (USAID’s) Center for Innovation and Impact (CII), who calls this person an “Uptake Coordinator” or “Product Manager.” It is described in the text box. CII, in turn, adapted this idea from the successful experiences of the Chlorhexidine (CHX) Work Group and the US pharmaceutical industry which often employs product managers to facilitate the rollout of a new drug or vaccine and see it through to rapid and widespread use. We generalize the concept to include scale-up of a service or approach. In global health, we feel that having a specific person in charge of the various processes and tasks is critical, because there are needs for coordination that span across various roles and touch on multiple routine systems, multiple Ministry of Health (MOH) departments, other public sector institutions, various partner agencies, and private sector partners. Someone supporting scale-up needs the scope and authority to bridge these divides. In order to be effective, this person must juggle various types of activities including technical and management roles. In the experiences used illustratively in this guide, the scale-up coordinator was someone working in the country office of an agency giving technical support to country-led MOH scale-up efforts, but with the right terms of reference and level of authority, this person could also be someone within the government structure itself.
McClure D, Gray I. 2016. "Scaling Assessment Map: An Evolving Tool Supporting Innovation Scale Up." Thoughtworks.
Nearly three years ago we wrote about the “Missing Middle” in the innovation lifecycle[i], a gap that kept successful pilot programs from reaching the goal of replication and optimization in multiple contexts. Since then, scaling humanitarian innovation has received a great deal of attention from the sector, with a number of new initiatives specifically focused on the scaling challenge. This, in turn, has created a growing need for working tools to assess the readiness of a pilot innovation to proceed into the scale-up process. The Scaling Assessment Map was developed by Ian Gray and Dan McClure to support the evaluation and planning for pilot innovation programs embarking on the Scale Up journey. This particular design for a scaling assessment is based on our prior work exploring the diverse challenges an innovation team must meet when working to create a sustainable ecosystem around their idea. The map is an evolving design that is in the process of being used and refined with innovation teams and mentors. In the spirit of collaboration, we are presenting the map with the hope that additional perspectives and experience can help refine both the tool and scaling practices as a whole.
Morinière LC, Turnbull M, Bremaud I, Vaughan-Lee H, Xaxa V, Farheen SA. 2018. "Toolkit: Scalability Assessment and Planning (SAP)" Save the Children Switzerland, Save the Children Asia Regional Office.
The SAP Toolkit guides Scaling and Assessment Planning with corrective actions to strengthen or enable scaling up. While scaling up is complex and fraught with challenges, using the SAP Toolkit ensures your initiative has the best possible chances for scalability and will set you on the path to effectively scaled initiatives. The Toolkit is designed for a facilitator in a 1.5-day workshop. In some cases (if evidence-based documentation is available), certain components could be completed as a desk review.
NIH Fogarty International Center. 2000. "Overcoming Barriers to Implementation in Global Health: A Toolkit for Engaging Diverse Stakeholders in Implementation Science." National Institutes of Health, Maryland.
This toolkit for researchers, policymakers and program implementers working in low- and middle-income countries (LMICs) provides resources to further implementation science, and strengthen stakeholder interactions, community participation and dissemination. It includes recognized and successful models, frameworks, strategies and approaches. Overcoming Barriers to Implementation in Global Health: A Toolkit for Engaging Diverse Stakeholders in Implementation Science Resources for Implementation Science Researchers Part 1: Implementation Science Methodologies and Frameworks Part 2: Participatory Research Models and Building Stakeholder Relationships Part 3: Dissemination Strategies in Evidence-based Policy and Practice
Price-Kelley H, Van Haeren L, & McLean R. 2020. "The Scaling Playbook A Practical Guide for Researchers."
What is scaling science? the term ‘scaling science’ purposefully embraces two meanings: • First, it means scaling scientific research results to optimize impacts. That is, scaling the impacts of research for the public good. • Second, it refers to a systematic, principle-based science of scaling that can increase the likelihood that innovations will benefit society. All approaches to scaling should be questioned, tested, refined and used thoughtfully. Innovators working with IDRC find that scaling in research for development aims to achieve a scale of impact important to people and environment, and contribute to a broader system of development change. In other words, scaling means understanding how to position research results so that the solutions generated reach the people who can use them, and in a way they can endorse. At the same time, our work to synthesize this experience and apply it to advance scaling practice is one contribution to a science of scaling. Your efforts to test, refine, and improve these ideas are just as important. The IDRC scaling science exploration yielded unique information about what creates desirable change and meaningful impact. From this perspective
Rick Davies and Jess Dart. 2003. "A Dialogical, Story-Based Evaluation Tool: The Most Significant Change Technique." American Journal of Evaluation - AM J EVAL. 24. 137-155. 10.1177/109821400302400202.
The Most Significant Change (MSC) technique is a dialogical, story-based technique. Its primary purpose is to facilitate program improvement by focusing the direction of work towards explicitly valued directions and away from less valued directions. MSC can also make a contribution to summative evaluation through both its process and its outputs. The technique involves a form of continuous values inquiry whereby designated groups of stakeholders search for significant program outcomes and then deliberate on the value of these outcomes in a systematic and transparent manner. To date, MSC has largely been used for the evaluation of international development programs, after having been initially developed for the evaluation of a social development program in Bangladesh (Davies, 1996). This article provides an introduction to MSC and discusses its potential to add to the basket of choices for evaluating programs in developed economies. We provide an Australian case study and outline some of the strengths and weaknesses of the technique. We conclude that MSC can make an important contribution to evaluation practice. Its unusual methodology and outcomes make it ideal for use in combination with other techniques and approaches.
Rick Davies and Jess Dart. 2005. "The ‘Most Significant Change’ (MSC) Technique: A Guide to Its Use." 10.13140/RG.2.1.4305.3606.
The most significant change (MSC) technique is a form of participatory monitoring and evaluation. It is participatory because many project stakeholders are involved both in deciding the sorts of change to be recorded and in analysing the data. It is a form of monitoring because it occurs throughout the program cycle and provides information to help people manage the program. It contributes to evaluation because it provides data on impact and outcomes that can be used to help assess the performance of the program as a whole. Essentially, the process involves the collection of significant change (SC) stories emanating from the field level, and the systematic selection of the most significant of these stories by panels of designated stakeholders or staff. The designated staff and stakeholders are initially involved by ‘searching’ for project impact. Once changes have been captured, various people sit down together, read the stories aloud and have regular and often in-depth discussions about the value of these reported changes. When the technique is implemented successfully, whole teams of people begin to focus their attention on program impact.
Sartas M, Schut M, van Schagen B, Velasco C, Thiele G, Proietti C, Leeuwis C. 2020. "Scaling readiness: Concepts, practices, and implementation." CGIAR, International Potato Center on behalf of RTB.
Scaling Readiness is an approach that can support organizations, projects, and programs in achieving their ambitions to scale innovations and achieve impact. Scaling Readiness encourages critical reflection on how ready innovations are for scaling, and what appropriate actions could accelerate or enhance scaling.
Simmons R, Fajans P, Ghiron L, Eds. 2007. "Scaling up Health Service Delivery: From Pilot Innovations to Policies and Programmes." World Health Organization.
This book has brought together insights from a comprehensive review of relevant literature, as well as the experience of major scalingup initiatives in family planning and primary care services from Africa, Asia and Latin America. We hope that the value of conducting systematic analysis of the determinants of successful scaling up has been demonstrated by this effort. Most of the understanding about scaling up presented here stems from experiences with the expansion of family planning and related reproductive health services. In all of these cases, efforts were focused on improving public sector programmes. The relevance of the conceptual frameworks and the lessons that have emerged from the authors’ experiences, however, extend beyond these areas of application. As Skibiak et al. argued in discussing the Zambian experience in Chapter 4, “the greatest challenges in scaling up reside in the practical, organizational transformation of a small pilot study to a broad-based programmatic intervention”. The strategic choices that have to be made and the determinants of success apply across sectors and across different types of implementing agencies. Therefore the principles and lessons discussed here are not limited to reproductive health or to the public sector, but can also be of value when adapted to other areas of health and development. Because work on this book has benefited from several opportunities for ongoing intellectual exchange over a period of years, those who participated have been able to use the lessons learned to shape scalingup activities in the field. At the same time we wish to clarify that this is not a cookbook from which project managers can select specific, stepby-step recipes. It can, however, provide general principles and examples to be used in the development of scaling-up strategies uniquely appropriate to their context. The same type of marriage between universal principles and the need for local relevance and adaptation applies to the innovations discussed here. New ways of improving equitable access to good health services or of implementing strategies that empower women, communities or young people to demand quality of care, for example, must be backed by locally generated evidence. Concepts and case-studies, or internationally accepted best practices, can offer guidance on what general principles are relevant, but they do not provide detailed operational plans for how quality of care and service access can be enhanced in a specific country, province or district. Such planning requires context-specific diagnostic assessments, designs and testing through pilot or experimental projects.
Simmons R, Fajans P, Ghiron L. 2011. "Beginning with the end in mind: planning pilot projects and other programmatic research for successful scaling up." ExpandNet & WHO.
This guide contains 12 recommendations on how to design pilot projects with scaling up in mind, as well as a checklist that provides a quick overview of the scalability of a project that is being planned, proposed, or in the process of implementation. Based on a combination of a comprehensive review of multiple literatures, field experience and a conceptual framework, the guide is intended for use by researchers, policy-planners, programme managers, technical-assistance providers, donors and others who seek to ensure that pilot or other programmatic research is designed in ways that lead to lasting and larger-scale impact.
Simmons R, Ghiron L, Fajans P, Lundgren R, Finkle C. 2020. "The Implementation Mapping Tool A tool to support adaptive management and documentation of scale up." ExpandNet.
The Implementation Mapping Tool (IMT) is a methodology to support reflection, corrective action and documentation as part of the process of scaling up – or planning for scale up – development interventions. It offers guidance for discussion and reflection, followed by planning for needed adaptations or other actions, while at the same time creating a record that explains how scaleup was implemented and results were achieved. In this methodology, members of a project team meet regularly with relevant stakeholders to discuss learning questions related to the scale-up process. They agree on emerging lessons and their implications for needed changes in the package of interventions or their implementation.These lessons and required adaptations are documented in a simple template (Appendix 1), creating a narrative record of why and how scaling up evolved over time. When testing and scaling up an innovation, it is important to apply the principles of adaptive management (3-7). At the center of adaptive management are responsive feedback loops in which project teams use a cyclical process of reflection, learning and necessary actions to ensure scale up progresses successfully (8-9). Use of the IMT facilitates this process and provides a qualitative record that illuminates how and why results were achieved or not. The nuances of these processes are often not fully remembered once projects have ended and key personnel with familiarity are no longer available. ExpandNet1 developed the IMT in collaboration with the Sukh Initiative in Pakistan and the Institute for Reproductive Health of Georgetown University for application in the area of health but it can be used for other areas of development as well. Although originally intended for projects that have reached the scale-up stage, it is relevant for any stage of the process – when initially testing an intervention, when developing a scale-up strategy, or for managing the scale-up process. The IMT shares some characteristics with other methods of assessing and documenting implementation, including the ‘Most Significant Change’ Technique, monthly learning diaries, and guided periodic reflection (10-13). Appendix 4 briefly describes these methods and how the IMT draws from them.
Simmons R, Ghiron L, Fajans P, Newton N. 2009. "Practical guidance for scaling up health service innovations." ExpandNet & WHO.
Calls for scaling up successfully tested health service innovations have multiplied over the past several years. Many acknowledge that pilot or experimental projects are of limited value unless they have larger policy and programme impact. Moreover, there is increasing recognition that proven innovations cannot simply be handed over with the expectation that they will automatically become part of routine programme implementation. While there has been progress, there is still little practical guidance on how to proceed with scaling up. This document can begin to fill this gap.
Simmons R, Ghiron L, Fajans P. 2010. "Nine steps for developing a scaling-up strategy." ExpandNet & WHO.
The purpose of this document is to outline a concise, step-by-step process for developing a scaling up strategy. The rationale behind such an undertaking is twofold. First, strategic planning for the expansion and institutionalization of successfully tested health systems innovations is essential, but often does not happen. As a result, effective new practices and products remain underutilized. It is hoped that the availability of this guide will encourage broader attention to systematic planning once pilot innovations have been successfully tested. Second, even when there is interest among programme managers and others in engaging in a systematic planning process, the experience, know-how and resources for doing so are often lacking. As a consequence planning remains ad hoc and is often limited to statements about broad goals and the extent of scaling up that is to be accomplished. Such expectations tend to be unrealistic unless they take into account the nature of the innovation, the capacity of the implementing organizations, the characteristics of the larger environment within which scaling up takes place and the resources available to support the process. Attention to scaling up requires systematic planning of how pilot-tested innovations can be implemented on a larger scale and achieve broad impact. Typically innovations are tested in pilot projects with special organizational, financial and human resources, which will not be available when innovations are being taken to scale. As a consequence, programme managers responsible for leading the process of “rolling out” the innovation to a sub-national or national level are faced with an enormous challenge: they have to implement the innovation on a large scale with few resources and in health systems that may be characterized by weak capacities and multiple, pressing priorities. Under such circumstances, success with scaling up calls for a careful balancing act between desired outcomes and practical realities and constraints. It also requires a planning process that is consistent with building national health systems’ capacities rather than imposing additional burdens on fragile public sector systems. This document provides guidance for such systematic strategy development. It should be kept in mind, however, that developing a strategy is only the beginning. Strategic thinking needs to continue throughout the process of implementation, requiring ongoing attention to the multiple factors that affect scaling up, as well as adjustments to the strategy whenever necessary. A scaling-up strategy, once developed, is not something that should be rigorously adhered to. Parts of it can quickly become obsolete as circumstances change, or because some factors were not adequately considered in the planning process. However, the initial plan can provide the foundation for the necessary adjustments that have to be made as scaling up proceeds.
VVOB, Brookings Center for Universal Education, Educate!, Management Systems International (MSI), Pratham, STiR Education. 2021. "Maximising the impact of innovations with the Education Scalability Checklist."
If we want to achieve widespread educational effectiveness, we need to prevent every school and every teacher to “reinvent the wheel”. Locally effective education innovations already exist, the more pressing challenge is that of scaling them up to different contexts, all the while sustaining the shifts in practice, policy and structures these improvements require. There are some hurdles on the path to scaling that are inherent to the education sector: Effective learning is a highly personal issue. There are many individual and contextual factors that impact learning for schools and teachers to reckon with in adapting effective practices from elsewhere to their own setting. A change in school or district leadership, in Ministry staff or a Minister of Education, can be enough to sweep away carefully tested solutions. Sustaining and institutionalising innovations in education is difficult.

Implementation Science

2006. "Implementation Science" An open access, peer-reviewed online journal, since 2006.
Aims and scope Implementation Science publishes research relevant to the scientific study of methods to promote the uptake of research findings into routine healthcare in clinical, organizational, or policy contexts. Applied health related research constantly produces new findings but often these are not routinely translated into healthcare practice. Implementation research is the scientific study of methods to promote the systematic uptake of proven clinical treatments, practices, organizational, and management interventions into routine practice, and hence to improve health. This also encompasses the de-implementation of interventions demonstrated to be of low or no clinical benefit and the study of influences on patient, healthcare professional, and organizational behavior in either healthcare or population settings. The lack of routine uptake of research findings is strategically important for the development of healthcare because it clearly places an invisible ceiling on the potential for research to enhance health. Further, it is scientifically important because it identifies the behavior of healthcare professionals and healthcare organizations as key sources of variance requiring improved empirical and theoretical understanding before effective uptake can be reliably achieved. Implementation science is an inherently interdisciplinary research area, and the journal is not constrained by any particular research tradition. Implementation Science publishes articles of high scientific rigor using the most appropriate methods to produce valid, generalizable answers to research questions. As well as hosting papers describing the effectiveness of implementation interventions, Implementation Science provides a unique home for rigorous and large-scale intervention development, evaluations of the process by which effects are achieved, economic evaluations of implementation, and the role of theory relevant to implementation research. The journal is also interested in publishing articles that present novel methods (particularly those that have a theoretical basis) for studying implementation processes and interventions. We are also interested in receiving articles that address methodologically robust study of the de-implementation of ineffective clinical and organizational practices. We welcome study protocols of large and innovative research, but these will only be considered if the study is received within 12 months of ethics approval and been approved for funding through external peer review via an established funding body at the national level in the respective country. We do not consider protocols for systematic reviews.
Bennett S, Mahmood SS, Edward A, Tetui M, Ekirapa-Kiracho E. 2017. "Strengthening scaling up through learning from implementation: comparing experiences from Afghanistan, Bangladesh and Uganda." Health Res Policy Syst. 2017;15(2):108.
Many effective innovations and interventions are never effectively scaled up. Implementation research (IR) has the promise of supporting scale-up through enabling rapid learning about the intervention and its fit with the context in which it is implemented. We integrate conceptual frameworks addressing different dimensions of scaling up (specifically, the attributes of the service or innovation being scaled, the actors involved, the context, and the scale-up strategy) and questions commonly addressed by IR (concerning acceptability, appropriateness, adoption, feasibility, fidelity to original design, implementation costs, coverage and sustainability) to explore how IR can support scale-up.

We draw upon three IR studies conducted by Future Health Systems (FHS) in Afghanistan, Bangladesh and Uganda. We reviewed project documents from the period 2011–2016 to identify information related to the dimensions of scaling up. Further, for each country, we developed rich descriptions of how the research teams approached scaling up, and how IR contributed to scale-up. The rich descriptions were checked by FHS research teams. We identified common patterns and differences across the three cases.

The three cases planned quite different innovations/interventions and had very different types of scale-up strategies. In all three cases, the research teams had extensive prior experience within the study communities, and little explicit attention was paid to contextual factors. All three cases involved complex interactions between the research teams and other stakeholders, among stakeholders, and between stakeholders and the intervention. The IR planned by the research teams focussed primarily on feasibility and effectiveness, but in practice, the research teams also had critical insights into other factors such as sustainability, acceptability, cost-effectiveness and appropriateness. Stakeholder analyses and other project management tools further complemented IR.

IR can provide significant insights into how best to scale-up a particular intervention. To take advantage of insights from IR, scale-up strategies require flexibility and IR must also be sufficiently flexible to respond to new emerging questions. While commonly used conceptual frameworks for scale-up clearly delineate actors, such as implementers, target communities and the support team, in our experience, IR blurred the links between these groups.
Brownson RC, Colditz GA, Proctor EK. 2012. "Dissemination and Implementation Research in Health: Translating Science to Practice." Oxford University Press.
Fifteen to twenty years is how long it takes for the billions of dollars of university-based research to translate into evidence-based policies and programs suitable for public use. Over the past decade, an exciting science has emerged that seeks to narrow the gap between the discovery of new knowledge and its application in public health, mental health, and health care settings. Dissemination and implementation (D&I) research seeks to understand how to best apply scientific advances in the real world, by focusing on pushing the evidence-based knowledge base out into routine use. To help propel this crucial field forward, this book aims to address a number of key issues, including: how to evaluate the evidence base on effective interventions; which strategies will produce the greatest impact; how to design an appropriate study; and how to track a set of essential outcomes. D&I studies must also take into account the barriers to uptake of evidence-based interventions in the communities where people live their lives and the social service agencies, hospitals, and clinics where they receive care. The challenges of moving research to practice and policy are universal, and future progress calls for collaborative partnerships and cross-country research. The fundamental tenet of D&I research—taking what we know about improving health and putting it into practice—must be the highest priority. Keywords: university-based research, evidence-based politics, dissemination and implementation, D&I, health, policy
Cardoso-Weinberg A, Alley C, Kupfer LE, Aslanyan G, Makanga M, Zicker F, Olesen O. 2022. "Funders’ perspectives on supporting implementation research in low-and middle-income countries." Glob Health Sci Pract. 2022;10(2):e2100497.
Implementation research (IR) is an emerging research area that helps research and public health programs achieve impact at scale. We sought to collate and share health research funders’ experiences with funding of IR activities in low- and middle-income countries (LMICs). We identified and grouped the funders’ strategies into 7 approaches that are considered important when supporting IR in LMICs. Our analysis offers broad direction and guidance to funding agencies and related partner organizations on important elements to consider when funding and implementing IR in LMICs.Ultimately, this work could help improve the impact of research investments in LMICs.
Durlak JA, and DuPre EP. 2008. "Implementation Matters: A Review of Research on the Influence of Implementation on Program Outcomes and the Factors Affecting Implementation." American Journal of Community Psychology 41: 327-350.
The first purpose of this review was to assess the impact of implementation on program outcomes, and the second purpose was to identify factors affecting the implementation process. Results from over quantitative 500 studies offered strong empirical support to the conclusion that the level of implementation affects the outcomes obtained in promotion and prevention programs. Findings from 81 additional reports indicate there are at least 23 contextual factors that influence implementation. The implementation process is affected by variables related to communities, providers and innovations, and aspects of the prevention delivery system (i.e., organizational functioning) and the prevention support system (i.e., training and technical assistance). The collection of implementation data is an essential feature of program evaluations, and more information is needed on which and how various factors influence implementation in different community settings.
Evidence Project. 2016. "Implementation Science Approaches to Family Planning and Reproductive Health: Experiential Learning and Sharing for Implementers, Policy-makers, Researchers, and Advocates." Meeting Report. Washington D.C., Population Council.
John Townsend wrapped up the workshop noting two important topics the workshop did not have time to discuss: 1) ethics and 2) rights. After providing a brief summary of the day, he closed the session by stressing that partnership is key for scale up. The presentations and rich discussions at the meeting highlighted several areas that require further consideration and attention in order to move the field of implementation science forward. Some of these include:
▪ Better documentation of the context in which interventions are implemented in order to better understand implementation barriers and facilitators as well as to inform replication of interventions to different settings and their scale up. This calls for careful documentation of examples of IS in FP/ RH shared through peer-reviewed literature and other avenues in order to make them widely available and accessible.
▪ Clear examples of what is successful implementation science and how it is measured (i.e., feasibility, sustainability, scale up). Engaging donors and funders in the development and refinement of these measures may help to identify new investment opportunities in implementation science.
▪ Development of the capacity building agenda for IS at different levels (e.g. local researchers, national policymakers, global and national implementers, advocates) will strengthen the influence of implementation science.
▪ Finding opportunities to strengthen partnership between the research and advocacy communities to generate additional avenues to ensure evidence utilization.
▪ Determining how best we can use IS to get better costing data so that we can publicize “best buys” within FP/RH.
Fixsen D, Naoom SF, Blase KA, Friedman RM, Wallace F. 2005. "Implementation Research: A Synthesis of the Literature." University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231).
knowledge gap, service delivery, program efficiency and program effectiveness
George A, Menotti EP, Rivera D, Marsh DR. 2011. "Community case management in Nicaragua: Lessons in fostering adoption and expanding implementation." Health Policy and Planning, 26: 327-337.
Community case management (CCM) as applied to child survival is a strategy that enables trained community health workers or volunteers to assess, classify, treat and refer sick children who reside beyond the reach of fixed health facilities. The Nicaraguan Ministry of Health (MOH) and Save the Children trained and supported brigadistas (community health volunteers) in CCM to improve equitable access to treatment for pneumonia, diarrhoea and dysentery for children in remote areas. In this article, we examine the policy landscape and processes that influenced the adoption and implementation of CCM in Nicaragua. Contextual factors in the policy landscape that facilitated CCM included an international technical consensus supporting the strategy; the role of government in health care provision and commitment to reaching the poor; a history of community participation; the existence of community-based child survival strategies; the decentralization of implementation authority; internal MOH champions; and a credible catalyst organization. Challenges included scepticism about community-level cadres; resistance from health personnel; operational gaps in treatment norms and materials to support the strategy; resource constraints affecting service delivery; tensions around decentralization; and changes in administration. In order to capitalize on the opportunities and overcome the challenges that characterized the policy landscape, stakeholders pursued various efforts to support CCM including sparking interest, framing issues, monitoring and communicating results, ensuring support and cohesion among health personnel, supporting local adaptation, assuring credibility and ownership, joint problem solving, addressing sustainability and fostering learning. While delineated as separate efforts, these policy and implementation processes were dynamic and interactive in nature, balancing various tensions. Our qualitative analysis highlights the importance of supporting routine monitoring and documentation of these strategic operational policy and management issues vital for CCM success. We also demonstrate that while challenges to CCM adoption and implementation exist, they are not insurmountable. Community case management, child health, policy analysis, scaling up implementation, Nicaragua
Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. 2004. "Diffusion of innovation in service organizations, systematic review and recommendation." Millbank Quarterly 82:581-629.
This article summarizes an extensive literature review addressing the question, How can we spread and sustain innovations in health service delivery and organization? It considers both content (defining and measuring the diffusion of innovation in organizations) and process (reviewing the literature in a systematic and reproducible way). This article discusses (1) a parsimonious and evidence-based model for considering the diffusion of innovations in health service organizations, (2) clear knowledge gaps where further research should be focused, and (3) a robust and transferable methodology for systematically reviewing health service policy and management. Both the model and the method should be tested more widely in a range of contexts. Keywords: Diffusion of innovation, systematic review, implementation
Kilbourne AM, Neumann MS, Pincus HA, Bauer MS, & Stall R. 2007. "Implementing evidence-based interventions in health care: application of the replicating effective programs framework." Implementation Science, 2(1), 42.
We describe the use of a conceptual framework and implementation protocol to prepare effective health services interventions for implementation in community-based (i.e., non-academic-affiliated) settings.

The framework is based on the experiences of the U.S. Centers for Disease Control and Prevention (CDC) Replicating Effective Programs (REP) project, which has been at the forefront of developing systematic and effective strategies to prepare HIV interventions for dissemination. This article describes the REP framework, and how it can be applied to implement clinical and health services interventions in community-based organizations.

REP consists of four phases: pre-conditions (e.g., identifying need, target population, and suitable intervention), pre-implementation (e.g., intervention packaging and community input), implementation (e.g., package dissemination, training, technical assistance, and evaluation), and maintenance and evolution (e.g., preparing the intervention for sustainability). Key components of REP, including intervention packaging, training, technical assistance, and fidelity assessment are crucial to the implementation of effective interventions in health care.

REP is a well-suited framework for implementing health care interventions, as it specifies steps needed to maximize fidelity while allowing opportunities for flexibility (i.e., local customizing) to maximize transferability. Strategies that foster the sustainability of REP as a tool to implement effective health care interventions need to be developed and tested.
Kris Putnam-Walkerly and Elizabeth Russell. 2016. "What the Heck Does ‘Equity’ Mean?" Stanford Social Innovation Review, Sept. 15, 2016.
Implementation science has not advanced equitable outcomes routinely, explicitly, or intentionally. Here’s how it can.
Légaré F, Plourde K, Ben Charif A, Gogovor A, Brundisini FK, McLean R, Milat A, Rheault N, Wolfenden L, Zomahoun HTV. 2021. "Evidence on Scaling in Health and Social Care: Protocol for a Living Umbrella Review." Syst Rev 10, 261 (2021).
There is a growing interest in scaling effective health innovations to promote equitable access to high-quality health services worldwide. However, multiple challenges persist in scaling innovations. In this study, we aim to summarize the scaling evidence in the health and social care literature and identify current knowledge gaps.

We will conduct a living umbrella review according to the Joanna Briggs Institute Reviewers’ Manual. We will consider all knowledge syntheses addressing scaling in health or social care (e.g., any setting, any clinical area) and conducted in a systematic way. We will search the following electronic databases: MEDLINE (Ovid), Embase, PsychINFO (Ovid), CINAHL (EBSCO), Web of Science, The Cochrane Library, Sociological Abstract (Proquest), Academic Search Premier (EBSCO), and Proquest Dissertations & Theses Global, from inception. Furthermore, we will conduct searches of the grey literature. No restriction regarding date or language will be applied. Each phase of the review will be processed by two independent reviewers. We will develop a data extraction form on Covidence. We will assess the methodological quality of the included reviews using AMSTAR2 and the risk of bias using ROBIS. Results will be presented in tabular form and accompanied by a narrative synthesis covering the traditional themes of scaling science that emerge from the analysis, such as coverage, range, and sustainability, as well as themes less covered in the literature, including reporting guidance, models, tools, barriers, and/or facilitators to scaling innovations, evidence regarding application in high-income or low-income countries, and end-user engagement. We will disseminate the findings via publications and through relevant networks.

The findings of the umbrella review will facilitate access to scaling evidence in the literature and help strengthen the science of scaling for researchers, policy makers, and program managers. Finally, this work will highlight important knowledge gaps and help prioritize future research questions.
Madon T, Hofman KJ, Kupfer L, Glass RI. 2007. "Public Health: Implementation Science." Science. 318(5857):1728-1729.
Background Recently mobile health (mHealth) has been implemented in Kenya to support family planning. Our objectives were to investigate disparities in mobile phone ownership and to examine the associations between exposure to family planning messages through mHealth (stand-alone or combined with other channels such as public forums, informational materials, health workers, social media and political/religious/community leaders’ advocacy) and contraceptive knowledge and use. Methods Logistic and Poisson regression models were used to analyze the 2014 Kenya Demographic and Health Survey. Results Among 31 059 women, 86.7% had mobile phones and were more likely to have received higher education, have children ≤5 y of age and tended to be wealthier or married. Among 7397 women who were sexually active, owned a mobile phone and received family planning messages through at least one channel, 89.8% had no exposure to mHealth. mHealth alone was limited in improving contraceptive knowledge and use but led to intended outcomes when used together with four other channels compared with other channels only (knowledge: incidence rate ratio 1.084 [95% confidence interval {CI} 1.063–1.106]; use: odds ratio 1.429 [95% CI 1.026–1.989]). Conclusions Socio-economic disparities existed in mobile phone ownership, and mHealth alone did not improve contraceptive knowledge and use among Kenyan women. However, mHealth still has potential for family planning when used with existing channels. family planning, Kenya, mHealth, mobile phone access
McKay H, Mackey D, Gray S, Hoy C, Rei A. 2019. "Translational Formative Evaluation before Scale-up of a Physical Activity Intervention for Older Men." Translational Journal of the American College of Sports Medicine.
Despite irrefutable health benefits of physical activity, older adults remain among the least active Canadians. To achieve population health, physical activity interventions that proved effective in controlled research settings must be delivered at scale to reach broader populations of older adults across multiple settings. Formative evaluations are essential, as they identify barriers and enablers to implementation across levels of stakeholder groups and settings. Thus, we conducted a formative evaluation of a choice- and evidence-based physical activity intervention (Men on the Move) designed for scalability.

We adopted key elements of two implementation frameworks that place characteristics of the innovation, prevention delivery system, prevention support system, and prevention synthesis and translation system at the core of implementation success. Guided by the Interactive Systems Framework for Dissemination and Implementation, data were collected from delivery partners, including 1 leader from a key provincial recreation organization, 6 recreation directors/coordinators and 3 activity coaches, and 14 participants (older men). This research team participated in prevention support and prevention synthesis and translation systems. Two trained interviewers conducted telephone interviews with delivery partners, and five trained interviewers and a notetaker conducted in-person interviews with participants.

Five themes emerged from analyses of delivery partner interviews: support, activity coaches, intervention delivery, Men on the Move continuation, and the absence of men. Two themes emerged from our analyses of participant data: monitoring and connectedness.

Lessons learned from this formative evaluation will guide the adaptation of the intervention to context and population for scale-up across British Columbia, Canada. In so doing, we aim to bridge the know–do–scale-up gap, which is imperative as we seek to improve older adult health at the population level.
Meyers DC, Durlak JA, & Wandersman A. 2012. "The quality implementation framework: A synthesis of critical steps in the implementation process." American Journal of Community Psychology.
Implementation science is growing in importance among funders, researchers, and practitioners as an approach to bridging the gap between science and practice. We addressed three goals to contribute to the understanding of the complex and dynamic nature of implementation. Our first goal was to provide a conceptual overview of the process of implementation by synthesizing information from 25 implementation frameworks. The synthesis extends prior work by focusing on specific actions (i.e., the “how to”) that can be employed to foster high quality implementation. The synthesis identified 14 critical steps that were used to construct the Quality Implementation Framework (QIF). These steps comprise four QIF phases: Initial Considerations Regarding the Host Setting, Creating a Structure for Implementation, Ongoing Structure Once Implementation Begins, and Improving Future Applications. Our second goal was to summarize research support for each of the 14 QIF steps and to offer suggestions to direct future research efforts. Our third goal was to outline practical implications of our findings for improving future implementation efforts in the world of practice. The QIF's critical steps can serve as a useful blueprint for future research and practice. Applying the collective guidance synthesized by the QIF to the Interactive Systems Framework for Dissemination and Implementation (ISF) emphasizes that accountability for quality implementation does not rest with the practitioner Delivery System alone. Instead, all three ISF systems are mutually accountable for quality implementation.
Milat A, Lee K, Conte K, Grunseit A, Wolfenden L, Van Nassau F, Orr N, Sreeram P, Bauman A. 2020. "Intervention Scalability Assessment Tool: A decision support tool for health policy makers and implementers." Health Res Policy Sys 18, 1.
Promising health interventions tested in pilot studies will only achieve population-wide impact if they are implemented at scale across communities and health systems. Scaling up effective health interventions is vital as not doing so denies the community the most effective services and programmes. However, there remains a paucity of practical tools to assess the suitability of health interventions for scale-up. The Intervention Scalability Assessment Tool (ISAT) was developed to support policy-makers and practitioners to make systematic assessments of the suitability of health interventions for scale-up.

The ISAT was developed over three stages; the first stage involved a literature review to identify similar tools and frameworks that could be used to guide scalability assessments, and expert input to develop draft ISAT content. In the second stage, the draft ISAT tool was tested with end users. The third stage involved revising and re-testing the ISAT with end users to further refine the language and structure of the final ISAT.

A variety of information and sources of evidence should be used to complete the ISAT. The ISAT consists of three parts. Part A: ‘setting the scene’ requires consideration of the context in which the intervention is being considered for scale-up and consists of five domains, as follows: (1) the problem; (2) the intervention; (3) strategic/political context; (4) evidence of effectiveness; and (5) intervention costs and benefits. Part B asks users to assess the potential implementation and scale-up requirements within five domains, namely (1) fidelity and adaptation; (2) reach and acceptability; (3) delivery setting and workforce; (4) implementation infrastructure; and (5) sustainability. Part C generates a graphical representation of the strengths and weaknesses of the readiness of the proposed intervention for scale-up. Users are also prompted for a recommendation as to whether the intervention (1) is recommended for scale-up, (2) is promising but needs further information before scaling up, or (3) does not yet merit scale-up.

The ISAT fills an important gap in applied scalability assessment and can become a critical decision support tool for policy-makers and practitioners when selecting health interventions for scale-up. Although the ISAT is designed to be a health policy and practitioner tool, it can also be used by researchers in the design of research to fill important evidence gaps.
Norton WE, McCannon CJ, Schall MW, Mittman BS. 2012. "A stakeholder-driven agenda for advancing the science and practice of scale-up and spread in health." Implementation Science, 7:118.
Although significant advances have been made in implementation science, comparatively less attention has been paid to broader scale-up and spread of effective health programs at the regional, national, or international level. To address this gap in research, practice and policy attention, representatives from key stakeholder groups launched an initiative to identify gaps and stimulate additional interest and activity in scale-up and spread of effective health programs. We describe the background and motivation for this initiative and the content, process, and outcomes of two main phases comprising the core of the initiative: a state-of-the-art conference to develop recommendations for advancing scale-up and spread and a follow-up activity to operationalize and prioritize the recommendations. The conference was held in Washington, D.C. during July 2010 and attended by 100 representatives from research, practice, policy, public health, healthcare, and international health communities; the follow-up activity was conducted remotely the following year.

Conference attendees identified and prioritized five recommendations (and corresponding sub-recommendations) for advancing scale-up and spread in health: increase awareness, facilitate information exchange, develop new methods, apply new approaches for evaluation, and expand capacity. In the follow-up activity, ‘develop new methods’ was rated as most important recommendation; expanding capacity was rated as least important, although differences were relatively minor.

Based on the results of these efforts, we discuss priority activities that are needed to advance research, practice and policy to accelerate the scale-up and spread of effective health programs.
Pérez D, Lefevre P, Castro M, Sánchez L, Toledo ME, Vanlerberghe V, Van der Stuyft P. 2011. "Process-oriented fidelity research assists in evaluation, adjustment and scaling-up of community-based interventions." Health Policy and Planning, 26: 413–422.
Fidelity research can help to answer essential questions about the diffusion process of innovative health interventions and provide insights for further scaling-up and institutionalization. This study assessed fidelity and reinvention in the implementation of a community-based control strategy for Aedes aegypti control. The intervention was implemented in 16 study areas in La Lisa, a municipality of Havana, Cuba. Its major components were: organization & management, capacity-building, community work and surveillance. A participatory assessment of process data was performed to determine whether the components and subcomponents were implemented, not-implemented or modified. Frequencies were tabulated over all circumscriptions (lowest level of local government) and the average was calculated for the four components. Spearman Rank correlation coefficients were calculated to explore the relationships between components. In addition, semi-structured interviews were conducted with co-ordinators of the strategy at different levels to identify difficulties encountered in the strategy’s implementation. Surveillance was the most implemented component (72.9%) followed by capacity-building (54.7%). Community work and organization & management were less implemented or modified (50% and 45%, respectively). Apart from surveillance and capacity-building, all components are significantly and strongly correlated (Spearman Rank correlation coefficient > 0.70, P < 0.01). If one component is implemented in a circumscription, the other components are also likely to be implemented. It is noticeable that areas which did not undergo organizational changes commonly did not implement community work activities. Within the whole strategy, few activities were added. Scarcely implemented subcomponents were the most innovative. The difficulties encountered during implementation were related to appropriate training and skills, available time, lack of support and commitment to the strategy, lack of motivation of local leadership, and integration of actors and resources. The study showed a wide variability of fidelity in the implementation of the intervention and highlighted challenges for scaling-up and institutionalization of the community-based intervention. Cuba, dengue, community participation, fidelity research, diffusion of innovations, implementation, process evaluation
Peters DH, Adam T, Alonge O, Agyepong IA, Tran N. 2013. "Implementation research: what it is and how to do it." BMJ, 347:f6753.
The field of implementation research is growing, but it is not well understood despite the need for better research to inform decisions about health policies, programmes, and practices. This article focuses on the context and factors affecting implementation, the key audiences for the research, implementation outcome variables that describe various aspects of how implementation occurs, and the study of implementation strategies that support the delivery of health services, programmes, and policies. We provide a framework for using the research question as the basis for selecting among the wide range of qualitative, quantitative, and mixed methods that can be applied in implementation research, along with brief descriptions of methods specifically suitable for implementation research. Expanding the use of well designed implementation research should contribute to more effective public health and clinical policies and programmes. Defining implementation research Implementation research attempts to solve a wide range of implementation problems; it has its origins in several disciplines and research traditions (supplementary table A). Although progress has been made in conceptualising implementation research over the past decade,1 considerable confusion persists about its terminology and scope.2–,4 The word “implement” comes from the Latin “implere,” meaning to fulfil or to carry into effect.5 This provides a basis for a broad definition of implementation research that can be used across research traditions and has meaning for practitioners, policy makers, and the interested public: “Implementation research is the scientific inquiry into questions concerning implementation—the act of carrying an intention into effect, which in health research can be policies, programmes, or individual practices (collectively called interventions).”
Peters DH, Tran NT, Adam T. 2013. "Implementation research in health: a practical guide." Alliance for Health Policy and Systems Research, World Health Organization, Geneva.
A key challenge faced by the global health community is how to take proven interventions and implement them in the real world. Affordable, life-saving interventions exist to confront many of the health challenges we face, but there is little understanding of how best to deliver those interventions across the full range of existing health systems and in the wide diversity of possible settings. Our failure to effectively implement interventions carries a price. Each year more than 287,000 women die from complications related to pregnancy and child birth, for example, while approximately 7.6 million children, including 3.1 million newborns, die from diseases that are preventable or treatable with existing interventions. Understanding implementation in the real world Implementation issues arise as a result of a range of factors including ‘real world’ contextual factors that are either overlooked or not captured by other research disciplines. Implementation research shines a light on those factors, providing the basis for the kind of context-specific and evidence-informed decision-making that is crucial to making what is possible in theory a reality in practice. Because implementation research is embedded in reality, people working in the real world (practitioners as opposed to people ‘doing research’) often ask the questions that are the starting point for new thinking. Making sure that those questions are heard, and that the research undertaken is directed at finding answers to the questions asked rather than at the topics researchers themselves may find interesting is one of the key challenges implementation researchers face. A practical tool Embedded in the real world, implementation research is also a powerful tool for capturing and analysing information in real time, allowing for the assessment of performance, for example, and facilitating health systems strengthening. Implementation research is particularly important in supporting the scale-up of interventions and their integration into health systems at the national level. Too often interventions that work in small-scale pilot studies fail to live up to expectations when rolled out in national strategies, or fail to transfer from one country to another as a result of contextual differences. Implementation research not only helps to clarify why that happens, but can be used to support the process of re-iterative refinement needed for successful adaptation. The same capacities make implementation research a useful tool for helping organizations develop the capacity to learn, enabling them to assimilate and put into effect knowledge developed on an iterative basis.
Rotteau L, Albert M, Bhattacharyya O, Berta W, Webster F. 2020. "Understanding decisions to scale up: a qualitative case study of three health service intervention evaluations." Journal of Health Services Research & Policy. May 2020. doi:10.1177/1355819620921892
Efforts to scale up evidence-based health care interventions are seen as a key strategy to address complex health system challenges. However, scale-up efforts have shown significant variability. We address the gap between scale-up theory and practice by exploring the socio-cultural factors at play in the evaluation and scale-up of three interventions within the clinical field.

A qualitative multiple case study was conducted to characterize the evaluation and scale-up efforts of three interventions. We interviewed 18 participants, including clinicians and researchers across the three cases. Using Pierre Bourdieu\'s concepts of field and capital as a theoretical lens, we conducted a thematic analysis of the data.

Despite the espoused goals of ensuring that health service interventions are always based on high-quality evidence within the clinical field, this study demonstrates that the outcomes of the evaluations are not the only factor in the decision to engage in scale-up efforts. Important socio-cultural factors also come into play. Bourdieu uses the term capital to refer to the resources that agents compete for and with their acquisition, accumulate power and/or social standing. The type of evidence valued in the clinical field and the ability to leverage capital in demonstrating that value are also important factors.

Determining if an intervention is effective and should be scaled up is more complex in practice than described in the literature. Efforts are needed to explicitly include the role of social processes in the current frameworks guiding scaling-up efforts. Keywords: capital; evaluation of complex interventions; field; pilot projects; scale up.
Seelos C, Mair J. 2016. "When innovation goes wrong" Stanford Social Innovation Review, Fall 2016
Efforts by social enterprises to develop novel interventions receive a great deal of attention. Yet these organizations often stumble when it comes to turning innovation into impact. As a result, they fail to achieve their full potential. Here’s a guide to diagnosing and preventing several “pathologies” that underlie this failure.
Shelton R, Nathan N. 2022. "Sustaining evidence-based interventions" In Weiner B, Lewis C, Sherr K (Eds.), Practical implementation science: Moving evidence into action (pp. 277-308). Springer.
If the long-term benefits of public health interventions are to be realized, then ongoing program implementation or sustainability is required. Sustained implementation of public health programs, however, is a considerable challenge in complex, real-world settings. As such, understanding how to best sustain evidence-based interventions is an important area of focus within implementation science. This textbook chapter provides researchers and practitioners with the most current evidence base in sustainability research and practice, while also providing practical methods, frameworks, and tools to use when planning for, or designing, sustainability interventions and strategies. The chapter first defines sustainability and explains why it is important. The rest of the chapter presents the multi-dimensional factors that may impact sustainability and gives practitioners guidance as to how they may plan for and evaluate the sustainability of their programs. Through the use of case studies that illustrate these points, practitioners will identify how to apply sustainability frameworks, assessments, and tools in their own settings.
Simmons R, Fajans P, Ghiron L. 2022. "Scaling Up Evidence-Based Interventions." In Weiner B, Lewis C, Sherr K (Eds.), Practical implementation science: Moving evidence into action (pp. 253-275). Springer.
Implementation science seeks to address the health and development needs of populations. To have impact, these efforts must be undertaken with a focus on scale up and sustainability. This chapter discusses the definition of scale up, as well as the common characteristics of major scale-up frameworks and presents the ExpandNet/World Health Organization (WHO) systematic approach, which consists of three phases: 1. Planning and conducting implementation research and programmatic projects to increase the potential for future expansion and institutionalization of evidence-based interventions (EBIs) 2. Using knowledge about the determinants of success to guide the development of a scaling-up strategy 3. Managing the implementation of scale up These phases are discussed with emphasis on ExpandNet/WHO guidance tools and lessons learned from their application across a range of health and development interventions, providing readers with an understanding of the principles of systematic and sustainable scale up and their contribution to implementation science.
The Center for Implementation. 2000. "Implementation in Action Bulletin."
The Global Fund to Fight AIDS, Tuberculosis and Malaria. 2008. "Framework for operations and implementation research in health and disease control programs." Geneva.
Operations research (OR) is critical in providing scientific evidence for health and disease control programs to improve their quality and “learning” as they scale up. In the context of aligning international health support, the need to develop a framework endorsed and recognized by a wider professional community as a commonly-used instrument for designing, planning, implementing and taking full advantage of effective OR has been well recognized. The Framework for Operations and Implementation Research in Health and Disease Control Programs is a result of a collaborative effort between the Global Fund to Fight AIDS, Tuberculosis and Malaria, the Special Program for Research and Training in Tropical Diseases (TDR) and an inter-agency technical working group. The culmination of this collaboration was a three-day meeting held in Geneva in April 2008 and attended by over fifty participants representing the Global Fund, TDR, the World Health Organization (WHO), the Joint United Nations Program on HIV/AIDS (UNAIDS), United States Agency for International Development (USAID), the World Bank, field-based programs, policy-makers and research communities from all over the world, which finalized and endorsed the framework. The overall goal of the document is two-fold: to standardize the practice of OR across the international health community and to stimulate the integration of OR into health programs. In general, OR needs to be integrated as an essential part of monitoring and evaluation (M&E) efforts. Thus, the concept of M’OR’E could become a new paradigm enhancing the practice of integrating monitoring, research and evaluation dimensions as one common component into program management systems. It would not only strengthen program implementation, but would also facilitate more effective utilization of M&E resources (currently recommended at five to ten percent of overall Global Fund grant budgets). The range of target audiences for this document is wide and varies from policy-makers to program managers, from researchers to program implementers, from donors to government agencies, from technical organizations to civil society and other stakeholders. The document is divided into three main sections. Section A contains an overview of OR definitions, scope and uses. Section B is the OR process flowchart and offers a step-by-step 16-item checklist of major activities required in the planning, implementation and follow-through (dissemination and use) of OR at the country level. Section C provides case studies of OR activities from the field and an annotated reference list of available handbooks, guidelines and other tools for OR. Feedback from people who use this document will help make improvements in future editions. We strongly suggest that programs and partners use this tool to incorporate OR in a systematic way, so that we can maximize the “learning” and quality of the scale-up of health services.
Viswanath K, Synowiec C, Agha S. 2019. "Responsive feedback: Towards a new paradigm to enhance intervention effectiveness." Gates Open Research Vol. 3 781. 28 May. 2019, doi:10.12688/gatesopenres.12937.2
The current dominant models of intervention design in the development sector do not account for the complexity and unpredictability of implementation challenges. Decision makers and implementers need timely feedback during implementation to respond to field realities and to course-correct. This letter calls for a new approach of “responsive feedback” or “feedback loops” that promotes interactions between project designers, implementers, researchers and decision-makers to enable course corrections needed to achieve intended outcomes. A responsive feedback approach, in theory, should be agile, flexible, adaptive, iterative, and actionable. There can be multiple challenges associated with incorporating this approach into practice including donor requirements, organizational structure and culture, concerns about the additional time required to adopt such an approach, resource and operational constraints, the absence of skill sets needed for such an approach within smaller organizations and inadequate inter-departmental communication. However, these barriers to adaptation can be overcome. For responsive feedback to become a part of the culture of development organizations, commitment is needed from donors, decision-makers, project designers and implementers. We believe that, to generate opportunities for learning and adaptation, donors should provide the stimulus to break down silos between implementers and researchers. Keywords: Implementation science, responsive feedback, feedback loops, adaptive implementation, theory of change, monitoring and evaluation
Weiner B, Lewis C, Sherr K (Eds.). 2022. "Practical implementation science: Moving evidence into action." Springer.
Practical Implementation Science is designed for graduate health professional and advanced undergraduate students who want to master the steps of using implementation science to improve public health. Engaging and accessible, this textbook demonstrates how to implement evidence-based practices effectively through use of relevant theories, frameworks, models, tools, and research findings. Additional real-world case studies across public health, global health, and health policy provide essential context to the major issues facing implementation domestically and globally with consideration of communities in low-to-middle-income countries (LMIC). The textbook is organized around the steps involved in planning, executing, and evaluating implementation efforts to improve health outcomes in communities. Coverage spans assessing the knowledge-practice gap; selecting an evidence-based practice (EBP) to reduce the gap; assessing EBP fit and adapting the EBP; assessing barriers and facilitators of implementation; engaging stakeholders; creating an implementation structure; implementing the EBP; and evaluating the EBP effort. Each chapter includes a "how to" approach to conducting the task at hand. The text also addresses the practical importance of implementation science through disseminating EBPs; scaling up EBPs; sustaining EBPs; and de-implementing practices that are no longer effective. All chapters include learning objectives and summaries with emphasized Key Points for Practice, Common Pitfalls in Practice, and discussion questions to direct learning and classroom discussion. Fit for students of public health, health policy, nursing, medicine, mental health, behavioral health, allied health, and social work, Practical Implementation Science seeks to bridge the gap from scientific evidence to effective practice.
World Bank and Interntational Monetary Fund. 2004. "Poverty reduction strategy papers: progress in implementation." Washington, DC.
Although the Poverty Reduction Strategy (PRS) approach was introduced just over four years ago, it is now seen as the country-level operational framework for progress towards the MDGs. Although implementation experience has varied with regard to process and content— both across countries as well as within individual countries’ strategies—a key contribution of the PRS approach has been to focus attention on country-specific constraints to development. In many low-income countries the PRS initiative has also resulted in a sharper focus on poverty reduction, a more open participatory process and greater attention to monitoring poverty-related outcomes. Over the past year (through end-June 2004), ten more countries have finalized full PRSPs, bringing to 42 the number of countries implementing PRSs. Twenty-three countries now have completed one or more annual progress reports (APR).

The main findings of this progress report are twofold. First, countries have made good progress in addressing the more straightforward challenges inherent in the approach. For example, poverty analysis is relatively good, strategies recognize the importance of growth and macroeconomic stability, indicators lists are being rationalized, and sectoral coverage is broadening. Second, the challenges that remain are technically difficult and institutionally complex. For example, the analysis of the sources of growth and its distributional impact remains relatively weak and countries have also experienced difficulties in marrying their aspirations for the future with the resource and capacity constraints of the present in the context of their PRSs. To better realize the potential of the PRS approach, sustained efforts on the part of countries and their development partners are, therefore, needed to: (i) reinforce the PRS as a country-driven approach; (ii) enhance the analysis that underpins a PRS; (iii) strengthen the institutional capacity for successful implementation; and (iv) enhance aid effectiveness. Given the scope of these challenges, expectations need to be ambitious yet realistic, while capacity building and analytical support must be appropriately prioritized and sequenced at the country level.
Zomahoun HTV, Ben Charif A, Freitas A, Garvelink MM, Menear M, Dugas M, Adekpedjou R, Légaré F. 2019. "The pitfalls of scaling up evidence-based interventions in health." Journal, Global Health Action , Volume 12, 2019 - Issue 1.
Policy-makers worldwide are increasingly interested in scaling up evidence-based interventions (EBIs) to larger populations, and implementation scientists are developing frameworks and methodologies for achieving this. But scaling-up does not always produce the desired results. Why not? We aimed to enhance awareness of the various pitfalls to be anticipated when planning scale-up. In lower- and middle-income countries (LMICs), the scale-up of health programs to prevent or respond to outbreaks of communicable diseases has been occurring for many decades. In high-income countries, there is new interest in the scaling up of interventions that address communicable and non-communicable diseases alike. We scanned the literature worldwide on problems encountered when implementing scale-up plans revealed a number of potential pitfalls that we discuss in this paper. We identified and discussed the following six major pitfalls of scaling-up EBIs: 1) the cost-effectiveness estimation pitfall, i.e. accurate cost-effectiveness estimates about real-world implementation are almost impossible, making predictions of economies of scale unreliable; 2) the health inequities pitfall, i.e. some people will necessarily be left out and therefore not benefit from the scaled-up EBIs; 3) the scaled-up harm pitfall, i.e. the harms as well as the benefits may be amplified by the scaling-up; 4) the ethical pitfall, i.e. informed consent may be a challenge on a grander scale; 5) the top-down pitfall, i.e. the needs, preferences and culture of end-users may be forgotten when scale-up is directed from above; and 6) the contextual pitfall, i.e. it may not be possible to adapt the EBIs to every context. If its pitfalls are addressed head on, scaling-up may be a powerful process for translating research data into practical improvements in healthcare in both LMICs and high-income countries, ensuring that more people benefit from EBIs. Keywords: Scaling-up; cost-effectiveness; equity; ethics; evidence-based intervention; harms; health.

Research Utilization

2000. "Reach, Effectiveness, Adoption, Implementation, Maintenance (RE-AIM)." Website.
The following blog post is the twelth in our journey through the special edition of RE-AIM in the Frontiers in Public Health special issue! The article titled “RE-AIM in the Real World: Use of the RE-AIM Framework for Program Planning and Evaluation in Clinical and Community Settings” written by Kwan et al. (2019), shows the use of RE-AIM in the use of RE-AIM n both clinical and community-based projects in regards to understanding intervention planning, evaluation, and outcomes. This is a unique publication that shows the use of RE-AIM in a variety of samples (snowball sampling) to look at the use of dimensions and understanding of RE-AIM through a 5-point scale. To read more about this work, please click the image below to read the full text!
Askew I, Matthews Z, Partridge R. 2001. "Going Beyond Research: A key issues paper raising discussion points related to dissemination, utilisation and impact of reproductive and sexual health research." from the Moving Beyond Research to Influence Policy Workshop, January 23-24, 2001, University of Southampton.
This report is the summary of a two-day meeting of researchers and policy makers that focussed on the process of getting research on reproductive health issues into practice. This was achieved by examining the dynamics of knowledge transfer and by identifying key actors that are involved and their roles in the dissemination and utilisation of research results. The actors who may be involved are numerous and varied, and depend on whether the results are of use in policy development or in service delivery or both. The most direct process of getting research findings into policy development could involve researchers liasing directly with macro level decision makers for example directors of public health or the heads of private, NGO or religious based health service programmes. Alternatively this liaison may be necessary at the micro level for example with district level health officers if the intention is more likely to influence programme implementation. In many instances it is possible that the expertise of communicating with decision makers is not available within research organisations and therefore researchers may need to use others, for example journalists or specialist communications organisations to communicate research results to decision makers in an effective and appropriate way. How research results are communicated to these carefully identified actors is another key area. The traditional outputs of research projects, such as final reports and peerreviewed papers, are often inaccessible to the key decision makers, either due to constraints in accessing them or the language in which they are written. However, over-condensing and the simplification of the results in an effort to improve the effectiveness of their communication, often denies the decision maker sufficient information with which to design coherent policies and practical programmes. The priority of academic researchers to publish in peer-reviewed journals for career advancement has advantages and disadvantages in terms of communicating with decision makers. Micro level decision makers often complain about lack of access to such journals, and about the lack of practical guidance in how to operationalise the findings in a service delivery programme. An advantage of communicating through journals is that being peer reviewed enables decision makers to access quality research whose methodological merits have been thoroughly scrutinised. Evaluation of the utilisation of policy- and programme-orientated research is also important. The utilisation of research by decision makers at policy and programme level is particularly important to donors, who are increasingly required to justify the funding of research programmes to their paymasters, in terms of the impact of the research on health services, and ultimately the impact on general health status. Evaluating research quality and its impact on decision-making is not straightforward and varies according to the type of research undertaken and its objectives.
Askew I. 2006. "Factors commonly influencing research utlization." Turning research into practice: suggested actions from case-studies of sexual and reproductive health research. Geneva, World Health Organization 2006:30-32.
The Population Council’s Frontiers in Reproductive Health program (FRONTIERS) provides technical and financial support to research that is intended explicitly to be utilized for strengthening the delivery of sexual and reproductive health services. As such, this research programme sponsors research studies for which one of the main criteria of success is whether or not the results have been used. This paper describes the findings from an assessment of eight case-studies of operations research projects undertaken during the 1990s in four countries of sub-Saharan Africa (1). The case-studies sought to identify the determinants that most commonly influenced whether and how the research results were utilized. Utilization was measured in terms of changes in health-care delivery procedures or policies by service organizations, and the extent to which the results were used by additional organizations external to the research. The assessment identified eight factors that influenced the extent to which research findings were used in all of the eight case-studies. These eight factors are described below. For each factor, the questions used to determine the extent to which utilization occurred are given, followed by a brief description of why and how each factor was influential. DEFINITION OF THE PROBLEM The more the programme managers were in a position to determine the research question the more likely they were to use the results. Utilization was greatest if they were, in effect, commissioning the research by determining the nature of the problem, specifying the research question, and determining how they would use the results, rather than simply approving a research project presented to them with limited or no consultation. In defining the problem, the programme managers considered the following questions.
Bero LA, Grill RI, Grimshaw JW, Harvey E, Oxman AD, and Thomson MA. 1998. "Closing the gap between research and practice: An overview of systematic reviews of interventions to promote the implementation of research findings." British Medical Journal 317(7156): 465-468.
Despite the considerable amount of money spent on clinical research relatively little attention has been paid to ensuring that the findings of research are implemented in routine clinical practice.1 There are many different types of intervention that can be used to promote behavioural change among healthcare professionals and the implementation of research findings. Disentangling the effects of intervention from the influence of contextual factors is difficult when interpreting the results of individual trials of behavioural change.2 Nevertheless, systematic reviews of rigorous studies provide the best evidence of the effectiveness of different strategies for promoting behavioural change. 3 4 In this paper we examine systematic reviews of different strategies for the dissemination and implementation of research findings to identify evidence of the effectiveness of different strategies and to assess the quality of the systematic reviews. Summary points Systematic reviews of rigorous studies provide the best evidence on the effectiveness of different strategies to promote the implementation of research findings Passive dissemination of information is generally ineffective It seems necessary to use specific strategies to encourage implementation of research based recommendations and to ensure changes in practice Further research on the relative effectiveness and efficiency of different strategies is required
Canoutas E, Hart L and Zan T. 2012. "Eight Strategies for Research to Practice. Moving Evidence to Action." FHI 360. Research Triangle Park, North Carolina.
An important goal of human development research is to generate evidence to guide improvements in policies and practices. Increasingly pressed to use proven approaches and to terminate strategies that do not work, development agencies look to research to better understand problems, to inform decision making and to identify effective solutions. Despite the critical role of research, a large gap often exists between the evidence and its widespread use in development programs.1 This “research-to-practice gap” is augmented by several factors, including limited stakeholder involvement in research, pilot project designs with little consideration for scale-up, feeble attempts to disseminate research findings and advocate their use, and the absence of tools and systematic efforts to replicate and expand evidence-based interventions. These barriers may be particularly evident in resource-constrained settings, where weak health systems further challenge the incorporation of research into practice. Designed for both researchers and program designers, this primer introduces a set of eight strategies that address these challenges and help close the gap between research and practice. The strategies are based on a growing body of evidence, theoretical frameworks, case studies and published guidance. The strategies explain how to plan, implement and disseminate research to facilitate its translation into practice. They also describe the most effective ways to incorporate research results into policies and programs. Although not exhaustive, each strategy includes general recommendations and resources for further reading.
Cernada GP. 1982. "Knowledge into action: a guide to research utilization." Farmingdale, NY Baywood Publishing Company.
The purpose of this slender volume is to describe why and how applied research carried out in a national public health program sometimes influenced program action in the field — and sometimes did not. A number of modified case studies are presented and analyzed to draw some practical lessons and to provide a theoretical basis for future program action. The setting is the Taiwan family planning program. The time period is from the mid 1960’s through the mid 1970’s. The focus is on the productive integration of research findings into community health education programs and the dissemination of these findings to stimulate other Asian countries at earlier stages of program planning and implementation. The viewpoint is that of a former program advisor and community health educator who spent some ten years on site and observed or participated in the events described. The results of quantitative research, particularly operations research and social surveys, recorded documents, and interviews comprise the methodology used to collect data. Aims to describe why and how applied research carried out in a national public health program sometimes influenced program action in the field - and sometimes did not. This title presents and analyses a number of modified case studies to draw some practical lessons and to provide a theoretical basis for future program action.
Crewe E and Young J. 2002. "Bridging Research and Policy: Context, Evidence and Links." Overseas Development Institute Working Paper No. 173. London, England: Overseas Development Institute.
Can we make sense of the relationship between research and policy? How can policy makers and researchers make better use of research to contribute to more evidence-based policies that reduce poverty, alleviate suffering and save lives? This paper written on behalf of the Overseas Development Institute (ODI) explores the relationship between research and policy. In the context of policy, research has often been ignored, inaccurate or neglected the concerns of poor and marginalised people. Research could have greater impact upon policy than it has had to date. Policy makers could make more extensive use of research and researchers could communicate their findings more effectively: Research may be ignored for a number of reasons. These range from ineffective communication, to the ignorance/anti-intellectualism of politicians. Understanding of the relationship between policy and research needs to be deepened if evidence-based policy making that responds to the demands of poor people is to be promoted. There are various models of policy making. Linear models have now been replaced by incremental and latterly ‘enlightenment’ methods. Contextual analysis and credibility are important dimensions when looking at the impact research has upon policy. In terms of context, both policy makers and researchers are influenced by political, social and economic structures, but more importantly the assumptions that underlie them. In terms of credibility, in aiming for more pro-poor evidence based policy, local involvement and effective dissemination are fundamental. It seems that researchers should have more influence on policy makers, and policy makers could make better use of research: Both research and policy have the obligation to establish chains of legitimacy with their informants. Research is more likely to contribute to evidence-based policy making if it fits within the political and institutional limits of policy makers and resonates with their ideological assumptions. This is also true if outputs are based on credible evidence, and are communicated via the most important communicators.
Davis P and Howden-Chapman P. 1996. "Translating research findings into health policy." Social Science and Medicine (43)5: 865-872.
Evidence of the influence of research on health policy is paradoxical. While there is scant evidence that research has had any impact on the direction or implementation of widespread health reforms, research on evidence-based medicine has dramatically increased, despite limited evidence that it has affected clinical practice. These developments have occurred in the context of a general decline in state intervention and provision and a post-modern questioning of researchers\' authority. Models of the relationship between research and policy range from one where empirical research rationally informs decision-making, through research incrementally affecting policy, to an \"enlightenment\" or \"infiltration\" model, which may operate on a conceptual level. Health research that contributes to large-scale socio-political change may require more methodological pluralism and greater focus on key institutional structures. Case studies reviewed suggest that dissemination is enhanced if researchers involve managers and policy-makers in the development of the framework for and focus of research and if investigators assume a responsibility for seeing their research translated into policy. Public health research is more influential if topical, timely, well-funded and carried out by a collaborative team that includes academics. Evaluations are more influential if, in addition, they are commissioned by health authorities but based on local collection of data, and instruments and incentives to implement policy are available. In some areas, such as the recent policy focus on careers in the community, researchers were largely responsible for raising this policy issue, whereas in other areas, such as the relationship between unemployment and health, researchers are just one of the groups of experts making competing claims about causality. In conclusion, clear research findings are not always a passport to policy, but researchers can reframe the way health policy issues are seen, and collaboration with policy-makers initially can enhance implementation later.
FHI360. 2022. "Research Utilization." Website.
FHI 360 is committed to moving evidence into practice to improve the health and development of the people and communities we serve. We purposefully employ a range of strategies across the research-to-practice continuum to achieve maximum impact. As a global research and implementation organization, FHI 360 is uniquely positioned to implement research utilization.
Glaser EM, Abelson HH, Garrison KN. 1983. "Putting knowledge to use." San Francisco, Jossey-Bass Publishers.
The knowledge industry must surely be considered one of the majors in the nation. In research and development alone the Federal investment amounts to some $20-billion annually. Curiously, it is an industry where astonishingly little attention has been dedicated to the marketing of its product. In these days of accountability, many persons are asking "How have these products been used to help us?" For example, public interest groups and their Washington lobbyists are formally asking the question with implications of skepticism. But there is a still more compelling reason for defending a more systematic facilitation of knowledge utilization: in human services the well-being of millions of American citizens depends in part on the effectiveness and efficiency of those services. The effectiveness and efficiency, in turn, depend in part upon optimum input of relevant knowledge. The record has it that common influencers of policies and practices are socio-political pressures, personal predilections, and persistence of the way things have been done before. The use of new knowledge trails along behind. There is a responsibility for better knowledge transfer on the parts of both the knowledge producers and the policy and practice people. But how can responsibilities for improved knowledge utilization be carried out? The literature on the topic offers advice. In fact, a surfeit of it. In the past 20 years the number of citations in the knowledge utilization field has grown from some 400 to an estimated 20,000 plup. If you venture into this literature in a quest for guidance, you stagger out reeling. The field abounds with assertions, conceptual models for analysis, and contradictory observations. But sound research information seems to remain in hiding. Dr. Edward Glaser and his colleagues at the Human Interaction Research Institute (HIRI)-Drs. Harold Abelson, Michael McKee, Goodwin Watson, Ms. Kathalee Garrison, and Ms. Molly Lewin-have marshalled an assault, as it were, upon literature. They have ferreted out sounder facts pertaining to the process of knowledge utilization and driven out the seemingly useless materiaL And beyond that, they have distilled the essence of knowledge on knowledge utilization. The result is perhaps not quite a technology manual in itself, but it certainly does represent an orderly presentation of rich information that will be essential to anyone planning and developing better ways of knowledge transfer, From the standpoint of NIMH\'s services research and development program, I should like to point out that HIRI material of this sort has been used as a basis for research utilization policy and practice decisions for eight years. During that time the "utilization rate" of projects has increased eight-fold. In this program alone, it has led to fruitfulness of multiple millions in research investments. We are grateful to HIRI for its signal contribution. We hope others will find this newly revised material to be similarly beneficial.
Haines A and Donald A. 1998. "Getting Research Findings into Practice: Using Research Findings in Clinical Practice." BMJ: 317:339-342 (1 August).
In clinical practice caring for patients generates many questions about diagnosis, prognosis, and treatment that challenge health professionals to keep up to date with the medical literature. A study of general practitioners in North America found that two clinically important questions arose for every three patients seen.1 The challenge in keeping abreast of the medical literature is the volume of literature. General physicians who want to keep up with relevant journals face the task of examining 19 articles a day 365 days a year.2 One approach to meeting these challenges and avoiding clinical entropy is to learn how to practise evidence based medicine. Evidence based medicine involves integrating clinical expertise with the best available clinical evidence derived from systematic research.3 Individual clinical expertise is the proficiency and judgment that each clinician acquires through clinical experience and practice. Best available clinical evidence is clinically relevant research which may be from the basic sciences of medicine, but especially that derived from clinical research that is patient centred, that evaluates the accuracy and precision of diagnostic tests and prognostic markers, and the efficacy and safety of therapeutic, rehabilitative, and preventive regimens. This paper focuses on what evidence based medicine is and how it can be practised by busy clinicians. The practice of evidence based medicine is a process of lifelong self directed learning in which caring for patients creates a need for clinically important information about diagnoses, prognoses, treatment, and other healthcare issues. The box at the bottom of the next page illustrates the five steps necessary to the practice of evidence based medicine. Practising evidence based medicine allows clinicians to keep up with the rapidly growing body of medical literature Evidence based medicine improves clinicians\' skills in asking answerable questions and finding the best evidence to answer these questions …
Hanson K, Cleary S, Schneider H, Tantivess S, Gilson L. 2010. "Scaling up health policies and services in low-and middle-income settings." BMC Health Services Research, 10(Suppl 1):11.
“Scaling up” effective health services is high on the policy agendas of many countries and international agencies. The current concern has been driven by growing recognition both of the challenges of achieving the health-related Millennium Development Goals (MDGs) in many countries, and of the need to ensure that the increased resources for health channelled through disease-specific health initiatives are able generate health gain at scale. Effective and cost-effective interventions exist to address many of the major causes of disease burden in the developing world, but coverage of many of these services remains low. There is a substantial gap between what could be achieved and what is actually being achieved in terms of health improvement in low- and middle-income countries. The term “scaling up” is widely used as shorthand to describe the objective or process of expanding service delivery. As this broad definition suggests, the term scaling up has been used in a variety of ways and contexts. For example, it can refer to the outcome, in terms of increased coverage, or the inputs required, whether financial, human or capital resources. Similarly, scaling up can also refer to a policy, strategy or the process of expansion [1]. The object of scaling up can be particular health interventions (e.g. attended delivery, insecticide treated mosquito nets, or integrated management of childhood illness), or health systems interventions such as health financing mechanisms (community-based health insurance), incentive mechanisms (e.g. pay-for-performance or contracting) or approaches to service delivery (e.g. training shopkeepers to supply antimalarial drugs).
Hennink M, McEachran J. 2006. "How to use the conceptual framework to increase research utilization." Turning research into practice: suggested actions from case-studies of sexual and reproductive health research. Geneva, World Health Organization.
The best use of the conceptual framework is as a comprehensive planning tool. When used as such, it can allow researchers to plan activities to enhance research utilization through the life of the project, and beyond. However, the conceptual framework can also be used once the project has been completed: (i) to identify and analyse the various factors and activities (communication, collaboration, etc.) undertaken during the research process to enhance utilization; or (ii) in cases where research utilization was less than expected, to identify and analyse factors and activities that may have been neglected or inadequately executed. It should be kept in mind that it is often difficult to prove a causal relationship between activities to enhance research utilization and actual utilization. This chapter discusses the factors researchers need to consider during the three main phases of the research process: pre-research, during research and postresearch. Although some factors are discussed individually in relation to a single phase of the research process, in practice they may need to be applied such that the three phases are seen as a single continuum. In applying the activities recommended in the conceptual framework, researchers need to be aware of what they can influence and what they cannot. For example, researchers can influence certain external factors to strengthen research utilization, in particular, by employing effective communication strategies and developing collaborative links with key stakeholders and end users of the research. However, they may not be able to influence such things as the current political, legal or programmatic climate in the country. In this chapter, individual sections discuss factors in each phase of the research process. This is followed by a review of additional parallel processes of stakeholder involvement, communication and macro-contextual factors. The importance of these additional processes is described in the discussion of each stage of the research and scaling-up activities.
Hunt SM. 1993. "The relationship between research and policy: Translating knowledge into action." In Healthy Cities: Research and Practice. Eds. John Davies and Michael Kelly. London, England: Routledge.
The growth of health promotion as a topic for discussion and a principle for practice is widespread, and affects all groups of health professionals. The Healthy Cities project, like Health for All, was inaugurated by the World Health Organization and has informed policy throughout the world. Healthy Cities: Research and Practice examines the application of the project in a number of countries. The contributors explore problems in the relationship between policy makers, communities, and academic researchers, and discuss how the Healthy Cities program affects housing policy, community development, scientific interchange and health education. In addition, the Editors, John Davies and Michael Kelly, provide a context by tracing the history of the WHO projects and discuss them in the broader context of scientific and philosohical debates about modernism and post-modernism. The contributors are drawn from practitioners and scientists with wide experience in the area from the United Kingdom, Canada, Australia and the United States. Healthy Cities will be invaluable to all those working at community level and in government with an interest in health, as well as students of health promotion.
Ir P, Bigdeli M, Meessen B, Van Damme W. 2010. "Translating knowledge into policy and action to promote health equity: The Health Equity Fund policy process in Cambodia 2000-2008." Health Policy, 96: 200-209.
To understand how knowledge is used to inform policy on Health Equity Funds (HEFs) in Cambodia; and to draw lessons for translating knowledge into health policies that promote equity.

We used a knowledge translation framework to analyse the HEF policy process between 2000 and 2008. The analysis was based on data from document analysis, key informant interviews and authors' observations.

The HEF policy-making process in Cambodia was both innovative and incremental. Insights from pilot projects were gradually translated into national health policy. The uptake of HEF in health policy was determined by three important factors: a policy context conducive to the creation, dissemination and adoption of lessons gained in HEF pilots; the credibility and timeliness of HEF knowledge generated from pilot projects; and strong commitment, relationships and networks among actors.

Knowledge locally generated through pilot projects is crucial for innovative health policy. It can help adapt blueprints and best practices to a local context and creates ownership. While international organisations and donors can take a leading role in innovative interventions in low-income countries, the involvement of government policy makers is necessary for their scaling-up.
Kim C, Wilcher R, Petruney T, Krueger K, Wynne L, Zan T. 2018. "A research utilisation framework for informing global health and development policies and programmes." Health research policy and systems, 16(1), 9.
A shift in the culture and practice of health and development research is required to maximise the real-world use of evidence by non-academic or non-research-oriented audiences. Many frameworks have been developed to guide and measure the research utilisation process, yet none have been widely applied. Some frameworks are simplified to an unrealistic linear representation while others are rendered overly complex and unusable in an attempt to capture all aspects of the research utilisation process. Additionally, many research utilisation frameworks have focused on the policy development process or within a clinical setting, with less application of the translation process at the programme level. In response to this gap – and drawing from over a decade of experience implementing research utilisation strategies – we developed a simple, four-phase framework to guide global health and development efforts that seek to apply evidence to policies and programmes. We present a detailed description of each phase in our framework, with examples of its relevance and application illustrated through our own case study experiences in global health. We believe the utility of this framework extends beyond the health sector and is relevant for maximising use of evidence to achieve the Sustainable Development Goals. Keywords: Research utilisation, Evidence-to-practice, Knowledge translation, Knowledge brokers, Research uptake, Evidence-informed policy, Research utilisation framework, Research utilisation case study
Kim J. 2006. "Conceptual framework for research utilization." Turning research into practice: suggested actions from case-studies of sexual and reproductive health research. Geneva, World Health Organization 2006:16-19.
The following conceptual framework for research utilization was developed during the course of the WHO and Partners Technical Consultation on Turning Research into Practice (TRIP) held in Geneva, Switzerland, in March 2003, a subsequent TRIP Working Group meeting held in London, United Kingdom, in July 2003 and a TRIP Workshop to Review Case-Studies (held in Johannesburg, South Africa, in June 2004). The development process involved an analysis of definitions and determinants of research utilization, as well as an examination of existing case-studies, conceptual pathways and key elements of research utilization. The utility of the conceptual framework as a tool for documenting and examining research utilization through analysis of case-studies was tested and refined during the June 2004 TRIP Workshop, with input from policymakers, researchers, and programme managers in the field of sexual and reproductive health. The conceptual framework may be applied both prospectively and retrospectively. Prospectively, donors, researchers, or programme managers can use it to assess and potentially influence factors that may enhance research utilization. Retrospectively, it may be used to analyse case-studies (of either success or failure) in order to learn from them. The conceptual framework is intended to be applicable across a range of research domains (e.g. basic, clinical, epidemiological, social science, or operations research), although it is expected that the research questions, stakeholders, communication strategies, and utilization goals may all vary. Although the conceptual framework may be applied prospectively or retrospectively to single research initiatives, in practice it is more productive to use it in the context of a broader body of existing and accumulating research evidence (i.e. a series of 17TRIP related research studies that are contributing, or have contributed, to an applicable body of knowledge). Thus, in some cases, utilization of research may be measured by its contribution to a developing theoretical knowledge base, or by its influence in stimulating further areas for investigation. Finally, although the conceptual framework may be useful for highlighting where further attention to certain factors may be critical to achieve research utilization, it does not necessarily follow that these factors lie within the responsibility or sphere of influence of the researcher. Thus, in such cases, the conceptual framework may be a useful tool for alerting or involving other stakeholders, including donors, government, advocacy groups, policy-makers and programme managers.
Martin A, McEvoy M, & Townsend JW. 1991. "Approaches to strengthening the utilization of OR results: Dissemination." In Operations Research: Helping Family Planning Programs Work Better. Eds. Marjorie Horn and Myrna Seidman. New York: Wiley-Liss, Inc.
National Center for the Dissemination of Disability Research. 1996. "A review of the literature on dissemination and knowledge utilization." Austin, TX.
(a) The U.S. system is deeply indebted to the extension model developed in the agricultural tradition; (b) it has traditionally focused on the dissemination side of the equation, rather than on the knowledge use side; (c) it has grown up as a set of uncoordinated—and even competitive—activities; (d) the resulting approach is largely top down, research-to-practice focused, rather than bottom up, problem-solving focused. (p. 290) A number of researchers have noted that, while the extension model has worked successfully in the field of agriculture, it has not been particularly effective National Center for the Dissemination of Disability Research (NCDDR) July 1996 4 in other areas. The reasons for this circumstance include issues of funding and coordination; also important are differences in the user orientation of agricultural and other research and in the kinds of information being disseminated. Rogers (1988) points out that agricultural researchers have always geared their work toward farmers’ use of their results. He also notes that the agricultural extension system has been less successful when the subject matter to be disseminated strays from agricultural production technology. The agricultural extension model reflects a rational, linear conception of the process of knowledge utilization; the focus of this model is on getting the word out, with the assumption that good ideas will be used by those who hear about them. As Louis (1992) explains, a key assumption of this approach to dissemination “is that knowledge is a ‘thing’ that simply needs to find a good home” (p. 288). However, the understandings about knowledge use emerging from the recent literature reveal that the process is complex, transactional, and heavily dependent on the potential user’s pre-existing knowledge, beliefs, and experiences. The focus on the user of research has come to the forefront during a period when the target audiences for disability research have broadened to include stronger attention to direct service providers and to persons with disabilities and their families. Although little empirical research has been conducted to assess the effectiveness of specific dissemination approaches with diverse user audiences, it is clear that consideration needs to be given to demographic and psychographic differences. Many improvements have been made in the dissemination of disability research (Blasiotti, 1992). NIDRR and other branches of the federal government are working to establish common perspectives as well as coordinated approaches to dissemination, and to encourage the incorporation of dissemination into all stages of the research, development, and utilization process. This literature review, along with a series of guides to effective practice that will be developed from it, is intended to support such changes.
Nutbeam N. 1996. "Achieving ‘best practice’ in health promotion: Improving the fit between research and practice." Health Education Research 11(3): 317-326.
This paper is based on the proposition that transfer of knowledge between researchers and practitioners concerning effective health promotion interventions is less than optimal. It considers how evidence concerning effectiveness in health promotion is established through research, and how such evidence is applied by practitioners and policy makers in deciding what to do and what to fund when addressing public health problems. From this examination it is concluded that there are too few rewards for researchers which encourage research with potential for widespread application and systematic development of promising interventions to a stage of field dissemination. Alternatively, practitioners often find themselves in the position of tackling a public health problem where evidence of efficacy is either lacking, or has to be considered alongside a desire to respond to expressed community needs, or the need to respond to political imperative. Several different approaches to improving the fit between research and practice are proposed, and they include improved education and training for practitioners, outcomes focussed program planning, and a more structured approach to rewarding research development and dissemination.
Nyonator FK, Akosa AB, Awoonor-Williams JK, Phillips JF, Jones TC. 2007. "Scaling-up experimental project success with the Community-based Health Planning and Services initiative in Ghana." In: Simmons R, Fajans P, Ghiron L, eds. Scaling up health service delivery: from pilot innovations to policies and programmes. Geneva, World Health Organization, 89–111.
The Community-based Health Planning and Services (CHPS) initiative in Ghana is an example of a strategy for scaling up a field trial to become a national programme. Representing a response to the problem that research projects can inadvertently produce nonreplicable service delivery capabilities, CHPS develops mechanisms for expanding national understanding and use of research findings to serve the health service needs of all Ghanaian households. This chapter describes strategies for introducing and developing community health services that were successfully tested in a Navrongo Health Research Centre trial and validated in Nkwanta District for a national programme of reorienting primary health care from clinics to communities. Nurses, once confined to clinical duties, are relocated to community-constructed clinics where they live and work. Volunteers support their services by mobilizing traditional social institutions to foster community support. Strategies for decentralized planning ensure that operational details of the programme are adapted to local circumstances. Strengths and limitations of the programme are reviewed and discussed.
Orosz E. 1994. "The impact of social science research on health policy." Social Science and Medicine 39(9): 1287-93.
The relationship between research and health policy is discussed from a policy process perspective, describing communication problems in the course of policy formulation, implementation and evaluation. Policy process is often expected by researchers to be rational, having logical sequence of steps and the objective evaluation of alternatives based on scientific knowledge. In fact, policies are often formulated without clear problem identification or based on wrong assumption. The timing of research and policy-making differs. Policy-makers need to respond quickly. Evaluations may be regarded by politicians as embarrassing if they point to a need for significant change. It is not satisfactory to consider only research and policy-making: their relationship is influenced by the media, different interest groups and by the general public. Health policy formulation is embedded in the general policy environment of particular societies. Some countries have a long tradition of consensus-building, while in others health reforms have been formulated and introduced in a centralized way. Traditional bio-medical thinking influences health policy-makers. The importance of social and political acceptability tends to be overlooked. The paper emphasizes that we are experiencing an era of scarcity of resources and growing tension concerning allocation decisions. Existing institutions provide insufficient incentives for policy-makers and researchers to promote public dialogue about such issues. The paper concludes that there is a need for new approaches to policy development and implementation, new structures in policy-making, changes in research financing and co-operation between disciplines and new structures for public participation in policy-making. Research should facilitate more open and democratic dialogue about policy options and the consequences of alternative choices.
Phillips JF, Nyonator FK, Jones TC, Ravikumar S. 2007. "Evidence-based scaling-up of health and family planning service innovations in Bangladesh and Ghana." In: Simmons R, Fajans P, Ghiron L, eds. Scaling up health service delivery: from pilot innovations to policies and programmes. Geneva, World Health Organization. Geneva, World Health Organization, 2007:113–134.
This chapter describes two initiatives that have utilized research to guide the development and scaling up of community-based health and family planning programmes. In Bangladesh and Ghana, evidence was accumulated in stages, beginning with an exploratory investigation, followed by an experimental trial testing potential interventions and a replication phase for validating research results in a non-research programme setting. The process concluded with research-guided programme expansion. Each stage was associated with shifts in generations of questions, mechanisms and outcomes as the process unfolded. Large-scale health systems development was achieved in both countries, not because the scaling-up strategies were alike but because similar research approaches led to different strategies adapted to contrasting societal and institutional contexts
Pokhrel S. 2006. "Scaling up health interventions in resource-poor countries: what role does research in stated-preference framework play?" Health Research Policy and Systems: Mar 30;4:4.
Despite improved supply of health care services in low-income countries in the recent past, their uptake continues to be lower than anticipated. This has made it difficult to scale-up those interventions which are not only cost-effective from supply perspectives but that might have substantial impacts on improving the health status of these countries. Understanding demand-side barriers is therefore critically important. With the help of a case study from Nepal, this commentary argues that more research on demand-side barriers needs to be carried out and that the stated-preference (SP) approach to such research might be helpful. Since SP techniques place service users' preferences at the centre of the analysis, and because preferences reflect individual or social welfare, SP techniques are likely to be helpful in devising policies to increase social welfare (e.g. improved service coverage). Moreover, the SP data are collected in a controlled environment which allows straightforward identification of effects (e.g. that of process attributes of care) and large quantities of relevant data can be collected at moderate cost. In addition to providing insights into current preferences, SP data also provide insights into how preferences are likely to respond to a proposed change in resource allocation (e.g. changing service delivery strategy). Finally, the SP-based techniques have been used widely in resource-rich countries and their experience can be valuable in conducting scaling-up research in low-income countries.
Population Council. 2000. "Maximizing utilization of research." FRONTIERS in Reproductive Health Legacy Series.
One important first step is clarifying terms used, almost interchangeably, in research utilization, as it incorporates a range of ways in which research can be used for making decisions to strengthen RH/FP policies and programs. (Concepts underlying terms used are defined in Box 1.) Achieving, and measuring, utilization of research depends in part on what type(s) of utilization is envisioned, and so it is critical the specific type needs to be considered and specified before research begins. Box 1. Clarifying Research Utilization Language Research utilization: making decisions concerning policy, advocacy and resource allocation, planning and management, and program systems development and strengthening, using information generated from research. Institutionalization: incorporation of a practice or intervention proven to be effective (sometimes termed a ‘best practice’) within the routine activities of a facility, program or organization. Replication: introduction of a proven intervention or practice into another setting; this may be another program or another country. Scale-up: extension of an intervention or proven practice beyond the original project site. Moreover, an OR project may be expected to achieve several types of utilization, depending on project nature, generalizability of findings to other programs and settings, and availability of resources for more than one type of utilization. For example, an intervention proven successful in a district may first become institutionalized within that district’s health program and then scaled up to other districts in the same province and to other provinces. If appropriate, it may also be replicated in another country. Most of the principles described below are relevant whether research concerns introduction of a new or improved RH/FP technology to a country or program (such as emergency contraception), or of a new / revised service delivery guideline or tool (such as the Balanced Counseling Strategy or Systematic Screening or an educational or training curriculum), or the reorganization of service delivery systems (such as integrating FP with HIV services, or using community‐level workers to provide services usually offered in clinics).
Porter RW and Prysor-Jones S. 1997. "Making a Difference to Policies and Programs: A Guide for Researchers." Support for Analysis and Research in Africa (SARA Project), Academy for Educational Development, Washington DC: SARA.
This manual for researchers is based on the premise that research can have the greatest impact on policies and programs when effective communication exists among researchers decision-makers and the affected community. It follows that steps can be taken during the design and implementation of research projects to systematically encourage effective communication from the beginning. The four basic stages in a research process (defining the question developing the proposal conducting the study and communicating results) serve as a framework for recommendations that focus attention from the start on the desired impact of the research. Using this framework the manual considers how researchers can 1) involve decision-makers program managers front-line workers and intended beneficiaries in identifying research needs and reviewing proposals and plans; 2) develop a relationship of trust with those involved in policies and programs and with beneficiaries; 3) involve program staff and community members in data collection and analysis; 4) review research progress periodically with these stakeholders; 5) involve managers staff and communities in drawing conclusions and making recommendations; 6) focus research reports on specific program and policy issues; 7) suggest specific actions based on findings; 8) prepare different types of reports and presentations for different audiences; and 9) use time effectively and strategically.
Pronovost P, Berenholtz S, Needham D. 2008. "Translating evidence into practice: a model for large scale knowledge translation." BMJ, 337:a1714:963-965.
Evidence based therapies that prevent morbidity or death are often not translated into clinical practice. One reason is that research often neglects how to deliver therapies to patients.1 Consequently, errors of omission are prevalent and cause substantial preventable harm.2 Attempts to increase the reliable use of evidence based therapies have generally focused on changing doctors’ behaviour.3 However, doctors work in a healthcare team within a larger hospital system, which must be considered when attempting to improve the reliability of patient care. Models to increase the reliable use of evidence based therapies typically focus on translating evidence into practice or on the best methods to run a collaborative; few if any have done both.4 Our model embeds an explicit method for knowledge translation in a collaborative model for broader dissemination of knowledge into practice. Model to translate evidence into practice We have described an integrated approach to improve the reliability of care5 that has been associated with substantial and sustained reductions in bloodstream infections associated with central lines.6 The approach has five key components: A focus on systems (how we organise work) rather than care of individual patients Engagement of local interdisciplinary teams to assume ownership of the improvement project Creation of centralised support for the technical work Encouraging local adaptation of the intervention Creating a collaborative culture within the local unit and larger system. This approach has matured into the Johns Hopkins Quality and Safety Research Group translating evidence into practice model (figure⇓). The resources required to develop, implement, and evaluate programmes using this model are substantial
Shelton JD. 2014. "Evidence-based public health: not only whether it works, but how it can be made to work practicably at scale." Global Health Science and Practice 2(3) 253-257.
Because public health must operate at scale in widely diverse, complex situations, randomized controlled trials (RCTs) have limited utility for public health. Other methodologies are needed. A key conceptual backbone is a detailed ‘‘theory of change’’ to apply appropriate evidence for each operational component. Synthesizing patterns of findings across multiple methodologies provides key insights. Programs operating successfully across a variety of settings can provide some of the best evidence. Challenges include judging the quality of such evidence and assisting programs to apply it. WHO and others should shift emphasis from RCTs to more relevant evidence when assessing public health issues.
Social Impact Exchange. 2019. "Knowledge Center."
SIE has compiled and organized hundreds of articles, papers, books, and other resource on scaled social impact. With our shift in strategy to systems change, we have amassed a collection of resources on this and related topics that we are continuously building.
Stone D, Maxwell S, and Keating M. 2001. "Bridging Research and Policy." Background paper for International Workshop Bridging Research and Policy, Warwick University, England, 16-17 July. England: Overseas Development Institute.
This paper is about the relationship between research and policy – specifically about how research impacts on policy, and about how policy draws on research. It might be thought that the relationship is straightforward, with good research designed to be relevant to policy, and its results delivered in an accessible form to policy-makers – and with good policy-making securely and rationally based on relevant research findings. In fact, this is far from the case. As a taster, Box 1 gives ten reasons why the link from research to policy might not be straightforward. Sometimes research is not designed to be relevant to policy. Sometimes it is so designed, but fails to have an impact because of problems associated with timeliness, presentation, or manner of communication. Sometimes (probably quite often) policy-makers do not see research findings as central to their decision-making. The relationship between research and policy is often tenuous, quite often fraught. To observe as much is not new. There are literatures on the question in many social science disciplines – in political science, sociology, anthropology, and management, to name a few. Our purpose here is to review some of these literatures and to draw out the implications for both researchers and policy-makers.1 The starting point is a discussion of what is meant by ‘policy’ and the ‘policy process’. The rational, linear model of policymaking – which summarises a logical sequence from problem definition, through analysis of alternatives, to decision, implementation, and review – is the traditional approach. We will see shortly what is wrong with this. Accordingly, the paper begins (Section 2) with a brief review of thinking on policy, presenting alternative models, and setting out a framework for thinking about the interaction between research and policy. It then deals successively with the challenge facing researchers (Section 3) and policy-makers (Section 4). Can the range of advice already offered to researchers be extended? And can policymakers be helped by new ideas such as evidence-based policy-making and performancebased evaluation? The Conclusion (Section 5) draws these threads together, suggesting that the impact of research is uncertain and contingent on social and political context.
Syed S, Hyder A, Bloom G, Sundaram S, Bhuiya A, Zhenzhong Z, Kanjilal B, Oladepo O, Pariyo G, Peters DH, Future Health Systems: Innovation for Equity. 2008. "Exploring evidence-policy linkages in health research plans: A case study from six countries." Health Research Policy and Systems, 6:4.
The complex evidence-policy interface in low and middle income country settings is receiving increasing attention. Future Health Systems (FHS): Innovations for Equity, is a research consortium conducting health systems explorations in six Asian and African countries: Bangladesh, India, China, Afghanistan, Uganda, and Nigeria. The cross-country research consortium provides a unique opportunity to explore the research-policy interface. Three key activities were undertaken during the initial phase of this five-year project. First, key considerations in strengthening evidence-policy linkages in health system research were developed by FHS researchers through workshops and electronic communications. Four key considerations in strengthening evidence-policy linkages are postulated: development context; research characteristics; decision-making processes; and stakeholder engagement. Second, these four considerations were applied to research proposals in each of the six countries to highlight features in the research plans that potentially strengthen the research-policy interface and opportunities for improvement. Finally, the utility of the approach for setting research priorities in health policy and systems research was reflected upon. These three activities yielded interesting findings. First, developmental consideration with four dimensions – poverty, vulnerabilities, capabilities, and health shocks – provides an entry point in examining research-policy interfaces in the six settings. Second, research plans focused upon on the ground realities in specific countries strengthens the interface. Third, focusing on research prioritized by decision-makers, within a politicized health arena, enhances chances of research influencing action. Lastly, early and continued engagement of multiple stakeholders, from local to national levels, is conducive to enhanced communication at the interface. The approach described has four main utilities: first, systematic analyses of research proposals using key considerations ensure such issues are incorporated into research proposals; second, the exact meaning, significance, and inter-relatedness of these considerations can be explored within the research itself; third, cross-country learning can be enhanced; and finally, translation of evidence into action may be facilitated. Health systems research proposals in low and middle income countries should include reflection on transferring research findings into policy. Such deliberations may be informed by employing the four key considerations suggested in this paper in analyzing research proposals.
Walley J, Khan MA, Shah SK, Witter S, Xiaolin W. 2007. "How to get research into practice: first get practice into research." Bulletin of the World Health Organization, 85 (6):424-5.
Discovering ways to increase access to and delivery of interventions is a major challenge. Typically research is divorced from implementation, which has led to a growing literature about how to get research into practice. However, operational research is best prioritized, designed, implemented and replicated from within national programmes. The current model for most international health service research is based on the assumption that the research community “discovers” solutions and then tries to market them to busy decision-makers and practitioners. The problem of failing to get research into policy and practice is well known. Much debate focuses on the effectiveness of different approaches to dissemination and behaviour change.1–4 This is a significant issue when trying to influence individual practitioners. Another focus is on developing the capacity of research institutions in developing countries, with the expectation that this will increase the relevance and local ownership of results.5 We argue that these two approaches are necessary but not sufficient. The aim should not be to perfect techniques of feeding results to decision-makers, but to start from the perspective of the decision-makers even before devising the questions. This means “getting practice into research”. This approach is not appropriate for research into new and untried treatments where efficacy has not been established, but should become the norm for operational research, by which we understand research into how an intervention is implemented. It is an approach that is gaining ground in the developed north, but which has even greater application in resource-constrained settings. Here, based on our experience in China, Pakistan and elsewhere, are some key considerations: Operational research should be embedded in local programmes. Operational research should emerge out of an ongoing partnership with a national programme. This includes the process of prioritizing, developing, conducting and disseminating research, and is part of national expansion of services. Operational research should focus on local opportunities for going to scale. The first stage is to explore the options that are under consideration for implementation and then design research to inform the choice of how that implementation should best be carried out. For maximum effect, it is often useful to focus attention on situations where there are resources available from international or national agencies, but where some technical or organizational block has prevented them from being used effectively. The research questions may be based on an understanding of the barriers to large-scale access. 6 Then trials and social and economic studies can be embedded within programme sites, and provide knowledge on how to overcome these barriers and deliver effective interventions, as in Pakistan.7–9 Because these operational issues are commonly relevant to other high-burden countries, the publication of the results should have international as well as national influence.
World Health Organization. 2006. "Bridging the ‘Know-Do’ Gap." Meeting on Knowledge Translation in Global Health, 10-12 October, 2005, Geneva, Switzerland.
Bridging the know–do gap is one of the most important challenges for public health in this century. It also poses the greatest opportunity for strengthening health systems and ultimately achieving equity in global health. Knowledge translation (KT) is emerging as a paradigm to learn and act towards closing the gap. While knowledge is more than research evidence, knowledge translation strategies can harness the power of scientific evidence and leadership to inform and transform policy and practice. There are pioneering efforts as well as exciting new initiatives in various developed and developing countries with respect to knowledge translation. Countries (policy-makers, health workers, researchers and the community) can work together and share experiences and lessons in bridging the gap. Although there are ongoing innovations and learning by doing, there is still no comprehensive framework or common platform for better understanding the know–do gap and systems to address it. WHO has a major role to play in bridging the know–do gap and supporting countries through better knowledge management. Given the breadth and scope of this great challenge, WHO should focus on the following: strategic advocacy for KT; platforms for knowledge exc change and sharing among countries and within WHO; resource mobilization; support country initiatives on KT strategies for health systems strengthening. For countries and the global community alike, some initial recommendations for action are: capacity development for KT, focusing on knowledge exchange and demand-side awareness-building; joint learning platforms for KT; research on improved methodologies for knowledge synthesis and exchange, and best practices on KT; KT-sensitive peer review and funding systems.
World Health Organization. 2006. "Turning Research into Practice." Suggested actions from case-studies of sexual and reproductive health research.
This document is the outcome of two meetings. The first meeting reviewed case-studies and other evidence related to research utilization from the standpoint of researchers and donors, while the second meeting reviewed additional materials from the standpoint of policy-makers and sexual and reproductive health programme managers. It is hoped that the information in this document will not only help researchers and others to increase the utilization of research findings, but also help them to monitor the extent to which research-based evidence is used for policy change and adoption of best practices to improve sexual and reproductive health. Findings from well-designed and ethically sound research should contribute to the formulation of policies and the development and strengthening of programmes for improving the sexual and reproductive health and well-being of communities. In order to ensure maximum utilization of research findings, researchers need to be adept in a range of communication skills and information dissemination strategies, including the ability to identify and engage with relevant stakeholders. For their part, policy-makers and service providers need to have a sound appreciation of how research can contribute to the development and modification of policies and practices, including implementation of interventions. A key obstacle to the utilization of research is the lack of dialogue between the various stakeholders. The gap between knowledge generation and its use is now well recognized by many researchers, donors, policy-makers and service providers. The challenge before all stakeholders is how to develop strong communication linkages between the various parties in order to facilitate the uptake of research findings. To do this effectively, stakeholders will need to identify the barriers to communication and learn from successful examples of research utilization. Research findings can contribute greatly to improving the reproductive health of people. Findings can be used to make decisions on new policies about provision of services (e.g. instituting new procedures, practices and interventions, including those for prevention) related to reproductive health-care delivery. They can equally contribute to the strengthening of existing programmes in terms of discontinuing practices found to be ineffective or harmful. Furthermore, research findings can also be used for advocacy for reproductive health or promoting the adoption of necessary interventions or for models of best practice to prevent or mitigate consequences of risks to health.

Adaptation, Adaptive Management, Complexity and Fidelity

Aarons AG, Green AE, Palinkas LA, Self-Brown S, Whitaker DJ, Lutzker JR, Silovsky JF, Hecht DB, Chaffin MJ. 2012. "Dynamic adaptation process to implement an evidence-based child maltreatment intervention." Implementation Science 2012,7:32.
Adaptations are often made to evidence-based practices (EBPs) by systems, organizations, and/or service providers in the implementation process. The degree to which core elements of an EBP can be maintained while allowing for local adaptation is unclear. In addition, adaptations may also be needed at the system, policy, or organizational levels to facilitate EBP implementation and sustainment. This paper describes a study of the feasibility and acceptability of an implementation approach, the Dynamic Adaptation Process (DAP), designed to allow for EBP adaptation and system and organizational adaptations in a planned and considered, rather than ad hoc, way. The DAP involves identifying core elements and adaptable characteristics of an EBP, then supporting implementation with specific training on allowable adaptations to the model, fidelity monitoring and support, and identifying the need for and solutions to system and organizational adaptations. In addition, this study addresses a secondary concern, that of improving EBP model fidelity assessment and feedback in real-world settings.

This project examines the feasibility, acceptability, and utility of the DAP; tests the degree to which fidelity can be maintained using the DAP compared to implementation as usual (IAU); and examines the feasibility of using automated phone or internet-enabled, computer-based technology to assess intervention fidelity and client satisfaction. The study design incorporates mixed methods in order to describe processes and factors associated with variations in both how the DAP itself is implemented and how the DAP impacts fidelity, drift, and adaptation. The DAP model is to be examined by assigning six regions in California (USA) to either the DAP (n = 3) or IAU (n = 3) to implement an EBP to prevent child neglect.

The DAP represents a data-informed, collaborative, multiple stakeholder approach to maintain intervention fidelity during the implementation of EBPs in the field by providing support for intervention, system, and organizational adaptation and intervention fidelity to meet local needs. This study is designed to address the real-world implications of EBP implementation in public sector service systems and is relevant for national, state, and local service systems and organizations.
Aarons G, Sklar M, Mustanski B, Benbow N, Brown CH. 2017. "Scaling-out evidence-based interventions to new populations or new health care delivery systems." Implementation Science (2017) 12:11.
Implementing treatments and interventions with demonstrated effectiveness is critical for improving patient health outcomes at a reduced cost. When an evidence-based intervention (EBI) is implemented with fidelity in a setting that is very similar to the setting wherein it was previously found to be effective, it is reasonable to anticipate similar benefits of that EBI. However, one goal of implementation science is to expand the use of EBIs as broadly as is feasible and appropriate in order to foster the greatest public health impact. When implementing an EBI in a novel setting, or targeting novel populations, one must consider whether there is sufficient justification that the EBI would have similar benefits to those found in earlier trials.

In this paper, we introduce a new concept for implementation called “scaling-out” when EBIs are adapted either to new populations or new delivery systems, or both. Using existing external validity theories and multilevel mediation modeling, we provide a logical framework for determining what new empirical evidence is required for an intervention to retain its evidence-based standard in this new context. The motivating questions are whether scale-out can reasonably be expected to produce population-level effectiveness as found in previous studies, and what additional empirical evaluations would be necessary to test for this short of an entirely new effectiveness trial. We present evaluation options for assessing whether scaling-out results in the ultimate health outcome of interest.

In scaling to health or service delivery systems or population/community contexts that are different from the setting where the EBI was originally tested, there are situations where a shorter timeframe of translation is possible. We argue that implementation of an EBI in a moderately different setting or with a different population can sometimes “borrow strength” from evidence of impact in a prior effectiveness trial. The collection of additional empirical data is deemed necessary by the nature and degree of adaptations to the EBI and the context. Our argument in this paper is conceptual, and we propose formal empirical tests of mediational equivalence in a follow-up paper.
Andrews M, Pritchett L, Woolcock M. 2012. "Escaping Capability Traps through Problem-Driven Iterative Adaptation (PDIA)." Working Paper 299. Center for Global Development.
Many reform initiatives in developing countries fail to achieve sustained improvements in performance because they are merely isomorphic mimicry—that is, governments and organizations pretend to reform by changing what policies or organizations look like rather than what they actually do. In addition, the flow of development resources and legitimacy without demonstrated improvements in performance undermines the impetus for effective action to build state capability or improve performance. This dynamic facilitates “capability traps” in which state capability stagnates, or even deteriorates, over long periods of time even though governments remain engaged in developmental rhetoric and continue to receive development resources. How can countries escape capability traps? We propose an approach, Problem-Driven Iterative Adaptation (PDIA), based on four core principles, each of which stands in sharp contrast with the standard approaches. First, PDIA focuses on solving locally nominated and defined problems in performance (as opposed to transplanting preconceived and packaged “best practice” solutions). Second, it seeks to create an authorizing environment for decision-making that encourages positive deviance and experimentation (as opposed to designing projects and programs and then requiring agents to implement them exactly as designed). Third, it embeds this experimentation in tight feedback loops that facilitate rapid experiential learning (as opposed to enduring long lag times in learning from ex post “evaluation”). Fourth, it actively engages broad sets of agents to ensure that reforms are viable, legitimate, relevant, and supportable (as opposed to a narrow set of external experts promoting the top-down diffusion of innovation).
Atun R, de Jong T, Secci F, Ohiri K, Adeyi O. 2010. "Integration of targeted health interventions into health systems: a conceptual framework for analysis." Health Policy and Planning, 24:104-111.
The benefits of integrating programmes that emphasize specific interventions into health systems to improve health outcomes have been widely debated. This debate has been driven by narrow binary considerations of integrated (horizontal) versus non-integrated (vertical) programmes, and characterized by polarization of views with protagonists for and against integration arguing the relative merits of each approach. The presence of both integrated and non-integrated programmes in many countries suggests benefits to each approach. While the terms \'vertical\' and \'integrated\' are widely used, they each describe a range of phenomena. In practice the dichotomy between vertical and horizontal is not rigid and the extent of verticality or integration varies between programmes. However, systematic analysis of the relative merits of integration in various contexts and for different interventions is complicated as there is no commonly accepted definition of \'integration\'-a term loosely used to describe a variety of organizational arrangements for a range of programmes in different settings. We present an analytical framework which enables deconstruction of the term integration into multiple facets, each corresponding to a critical health system function. Our conceptual framework builds on theoretical propositions and empirical research in innovation studies, and in particular adoption and diffusion of innovations within health systems, and builds on our own earlier empirical research. It brings together the critical elements that affect adoption, diffusion and assimilation of a health intervention, and in doing so enables systematic and holistic exploration of the extent to which different interventions are integrated in varied settings and the reasons for the variation. The conceptual framework and the analytical approach we propose are intended to facilitate analysis in evaluative and formative studies of-and policies on-integration, for use in systematically comparing and contrasting health interventions in a country or in different settings to generate meaningful evidence to inform policy.
BOND Network for International Development. 2006. "Working With Barriers To Organisational Learning."
This paper has focused on some of the defences and barriers to organisational learning that face many NGOs. It has offered some initial pointers and examples of how these can be worked with. The intention has been to open up a conversation and further inquiry into this key area of organisational life, out of which new meanings and actions may emerge. The approach to learning conveyed in this paper inevitably challenges the dominant world view that outcomes need to be predicted and, we are judged solely by our ability to achieve these predetermined outcomes. This is not a conducive environment for learning. Whereas, if we inhabit a world view, where things are not in our control, where we do the best we can, at that moment, but even when we bring all our collective intelligence to bear, things may still not turn out the way we thought, where we operate from a position of humility and compassion for human frailty (including our own) then it offers a very different environment for learning. We need to recognise and work with the barriers to organisational learning, in order to release the vital benefits that it brings. Indeed being able to work with some of the ‘stuckness’ is one part of what organisational learning is about. It takes a strong belief in learning to make a case for funding support/procedures and an organisational culture to support it, for both NGOs and their partners. We hope this paper encourages this.
Brinkerhoff DW, Frazer S, McGregor-Mirghani L. 2018. "Adapting to Learn and Learning to Adapt: Practical Insights from International Development Projects." RTI Press. Policy Brief. ISSN 2378-7937. RTI International.
Adaptive programming and management principles focused on learning, experimentation, and evidence-based decision making are gaining traction with donor agencies and implementing partners in international development. Adaptation calls for using learning to inform adjustments during project implementation. This requires information gathering methods that promote reflection, learning, and adaption, beyond reporting on pre-specified data. A focus on adaptation changes traditional thinking about program cycle. It both erases the boundaries between design, implementation, and evaluation and reframes thinking to consider the complexity of development problems and nonlinear change pathways.Supportive management structures and processes are crucial for fostering adaptive management. Implementers and donors are experimenting with how procurement, contracting, work planning, and reporting can be modified to foster adaptive programming. Well-designed monitoring, evaluation, and learning systems can go beyond meeting accountability and reporting requirements to produce data and learning for evidence-based decision making and adaptive management. It is important to continue experimenting and learning to integrate adaptive programming and management into the operational policies and practices of donor agencies, country partners, and implementers. We need to devote ongoing effort to build the evidence base for the contributions of adaptive management to achieving international development results.
Cancer Prevention and Control Research Network. 2007. "Adaptation Guidance Tool."
In choosing an evidence-based intervention you may have to make changes to increase fit or compatibility with your audience and/or community. Here is general guidance in terms of things that can and cannot be changed from the original intervention. Remember to refer to any adaptation suggestions from the original developer(s) in making these adaptation decisions.
Carvalho ML, Honeycutt S, Escoffery C, Glanz K, Sabbs D, Kegler MC. 2013. "Balancing Fidelity and Adaptation: Implementing Evidence-Based Chronic Disease Prevention Programs." Journal of Public Health Management and Practice. 19(4):348–356. doi: 10.1097/PHH.0b013e31826d80eb
To describe adaptations that community-based organizations (CBOs) made to evidence-based chronic disease prevention intervention programs and to discuss reasons for those adaptations.

The process evaluation used project report forms, interviews, and focus groups to obtain information from organizational staff.

Programs were conducted in community-based organizations (n = 12) in rural southwest Georgia including churches, worksites, community coalitions, a senior center, and a clinical patient setting.

Site coordinators (n = 15), organizational leaders (n = 7), and project committee members (n = 25) involved in program implementation at 12 funded organizations. Intervention: The Emory Cancer Prevention and Control Research Network awarded mini grants to rural CBOs to implement one of 5 evidence-based nutrition or physical activity programs. These sites received funding and technical assistance from Emory and agreed to conduct all required elements of the selected evidence-based program.

Main Outcome Measures:
Program implementation and context were explored, including completion of core elements, program adaptation, and reasons for adaptation that occurred at sites implementing evidence-based chronic disease prevention programs.

Five major types of adaptations were observed: changing educational materials, intended audience, and program delivery; adding new activities; and deleting core elements. Sites had intentional or unintentional reasons for making program adaptations including enhancing engagement in the program, reaching specific audiences, increasing program fit, and reinforcing program messages. Reasons for not completing core elements (program deletions) included various types of “turbulence” or competing demands (eg, leadership/staff transitions and time constraints).

The types of adaptations and reasons described in this evaluation support the idea that adaptation is a natural element of implementing evidence-based interventions. Building this understanding into dissemination strategies may help researchers and funders better reach communities with evidence-based interventions that are a relevant fit, while striving for fidelity.
Castro FG, Barrera M Jr, Holleran Steiker LK. 2010. "Issues and Challenges in the Design of Culturally Adapted Evidence-Based Interventions." Annual Review of Clinical Psychology. Vol. 6:213-239. doi: 10.1146/annurev-clinpsy-033109-132032
This article examines issues and challenges in the design of cultural adaptations that are developed from an original evidence-based intervention (EBI). Recently emerging multistep frameworks or stage models are examined, as these can systematically guide the development of culturally adapted EBIs. Critical issues are also presented regarding whether and how such adaptations may be conducted, and empirical evidence is presented regarding the effectiveness of such cultural adaptations. Recent evidence suggests that these cultural adaptations are effective when applied with certain subcultural groups, although they are less effective when applied with other subcultural groups. Generally, current evidence regarding the effectiveness of cultural adaptations is promising but mixed. Further research is needed to obtain more definitive conclusions regarding the efficacy and effectiveness of culturally adapted EBIs. Directions for future research and recommendations are presented to guide the development of a new generation of culturally adapted EBIs.
Century J, Rudnick M, Freeman C. 2010. "A Framework for Measuring Fidelity of Implementation: A Foundation for Shared Language and Accumulation of Knowledge." American Journal of Evaluation 2010; 31; 199. DOI: 10.1177/1098214010366173
There is a growing recognition of the value of measuring fidelity of implementation (FOI) as a necessary part of evaluating interventions. However, evaluators do not have a shared conceptual understanding of what FOI is and how to measure it. Thus, the creation of FOI measures is typically a secondary focus and based on specific contexts and programs. This article describes a project that holds the development of FOI measures as its primary goal and has developed a suite of data collection tools designed to be used across multiple programs. It describes the foundation of the suite—a conceptual framework for clearly and specifically describing FOI and the need for the framework. It also describes where the framework resides in existing literature and how it can be used to support measurement of interventions in education and other fields.
Chambers DA, Norton WE. 2016. "The Adaptome. Advancing the Science of Intervention Adaptation." American Journal of Preventive Medicine. Volume 51, Issue 4, Supplement 2, Pages S124–S131. doi: 10.1016/j.amepre.2016.05.011
In the past few decades, prevention scientists have developed and tested a range of interventions with demonstrated benefits on child and adolescent cognitive, affective, and behavioral health. These evidence-based interventions offer promise of population-level benefit if accompanied by findings of implementation science to facilitate adoption, widespread implementation, and sustainment. Though there have been notable examples of successful efforts to scale up interventions, more work is needed to optimize benefit. Although the traditional pathway from intervention development and testing to implementation has served the research community well—allowing for a systematic advance of evidence-based interventions that appear ready for implementation—progress has been limited by maintaining the hypothesis that evidence generation must be complete prior to implementation. This sets up the challenging dichotomy between fidelity and adaptation and limits the science of adaptation to findings from randomized trials of adapted interventions. The field can do better. This paper argues for the development of strategies to advance the science of adaptation in the context of implementation that would more comprehensively describe the needed fit between interventions and their settings, and embrace opportunities for ongoing learning about optimal intervention delivery over time. Efforts to build the resulting adaptome (pronounced “adapt-ohm”) will include the construction of a common data platform to house systematically captured information about variations in delivery of evidence-based interventions across multiple populations and contexts, and provide feedback to intervention developers, as well as the implementation research and practice communities. Finally, the article identifies next steps to jumpstart adaptome data platform development.
Core Group. 2016. "A Call To Action. Complexity Matters: Aligning the Monitoring and Evaluation of Social and Behavior Change with the Realities of Implementation."
Advocate for the support of M&E of SBC interventions that capture the unpredictability of the change process and reflect the realities of project implementation. Show that proven M&E methods that are appropriately adaptive in response to emergent needs and opportunities can help monitor complexity. Argue that M&E methods that do not address the complexity of both behavior and of program implementation can distort our understanding of SBC Support the claim that data gathered through community feedback and iteration not only can accelerate individual behavior change, but can propel social change. Champion the use of participatory, narrative, mixed methods and learning-based approaches that align with what we know about the context and complexity of SBC program implementation.
Dennis Whittle. 2013. "How Feedback Loops Can Improve Aid (and Maybe Governance)." Center for Global Development.
If private markets can produce the iPhone, why can’t aid organizations create and implement development initiatives that are equally innovative and sought after by people around the world? The key difference is feedback loops. Well-functioning private markets excel at providing consumers with a constantly improving stream of high-quality products and services. Why? Because consumers give companies constant feedback on what they like and what they don’t. Companies that listen to their consumers by modifying existing products and launching new ones have a chance of increasing their revenues and profits; companies that don’t are at risk of going out of business. Is it possible to create analogous mechanisms that require aid organizations to listen to what regular citizens want—and then act on what they hear? This essay (part first-person account, part research and analysis) argues that the answer is yes. I first look at the current mental model in development, in which experts are largely responsible for determining which projects are implemented and for evaluating the impact of those projects. Partly because of information problems and partly because of incentives, this approach results in slow innovation and little responsiveness to what citizens really want. I then review the potential for new approaches such as randomized controlled trials, concluding that their applicability (and even desirability) is limited for cost, technical, and theoretical reasons. I conclude (along with others such as Pritchett, Samji, Hammer, Woolcock, and Easterly), that only approaches that provide a faster and more steady stream of information from varied sources—especially citizens—are likely to improve the quality of aid. In the future, the default model should be that aid agencies need to demonstrate (a) why they believe regular citizens actually want each proposed project and (b) how citizen voice will be used to ensure high-quality implementation. Rather than creating an immediate mandate, however, the best approach in the near term may be to find out what works and then gradually phase in a series of requirements. Fortunately, many examples have emerged in which direct feedback from citizens has been solicited as input into both the selection and the implementation of development initiatives. Not all have been successful, but some have led to significant improvements in outcomes. Being successful—having a closed feedback loop—requires not only that citizens be listened to but that their voices be acted upon in the form of changes to aid programs. The essay provides a set of principles that can be used by practitioners to design feedback loops with a higher probability of success. It also provides a set of key conceptual issues that remain to be explored in depth by researchers, as well as a potential implementation road map for leaders of aid agencies.
Derbyshire H, Donovan E. 2016. "Adaptive programming in practice: shared lessons from the DFID-funded LASER and SAVI programmes." UKAID.
Influenced by a wealth of research and emerging international consensus, key donors including UK Department for International Development (DFID), USAID and the World Bank are encouraging the increased use of adaptive programme approaches to address complex development challenges such as institutional reform and work in fragile and conflict affected states, in an attempt to increase impact. Donors and service providers are however grappling with the complex practicalities of how to design, implement and monitor adaptive programmes – a challenge compounded by the recognised lack of practical case study material to draw on. This paper has been produced jointly by two DFID-funded programmes – LASER (Legal Assistance for Economic Reform) and SAVI (State Accountability and Voice Initiative, Nigeria). Both programmes have been taking an adaptive approach to delivering development support for some time, and are achieving results. Although LASER and SAVI are very different – including in size of budget, sector, location, and aid modality – both deal with significant issues of institutional change in complex environments. It is striking that despite programmatic differences, key aspects of their respective approaches to adaptive programming, as well as many lessons and challenges, are similar. As a contribution to current thinking and practice, this joint paper provides on overview of LASER and SAVI’s practical experience as suppliers and practitioners in designing, contracting, managing, implementing and monitoring flexible, adaptive programmes – complementing the more widely available perspective of academics and think tanks.
Desai H, Pellfolk E, Maneo G, Schlingheider A. 2018. "Managing to Adapt: Analysing adaptive management for planning, monitoring, evaluation, and learning." Oxfam. DOI 10.21201/2017.2159
CARE Cooperative for Assistance and Relief Everywhere CSO Civil society organization DFID Department for International Development DRC Democratic Republic of Congo GEM Gendered enterprise and markets IRC International Rescue Committee LDP Law and development partnership LSE London School of Economics and Political Science M&E Monitoring and evaluation MEL Monitoring, evaluation, and learning MCP Multi-country programme NGO Non-government organization ODI Overseas Development Institute PbR Payment by results PDIA Problem-driven iterative adaptation PMEL Planning, monitoring, evaluation, and learning PMU Programme Management Unit RCDA Rural Communities Development Agency RTE Real-time evaluation Sida Swedish International Development Cooperation Agency SNA Social network analysis ToC Theory of change USAID United State Agency for International Development 6 Managing to Adapt: Analysing adaptive management for planning, monitoring, evaluation, and learning EXECUTIVE SUMMARY Adaptive management is at the heart of ‘doing development differently’ (Wild et al., 2016). Whether it is here to stay depends on how much it is mainstreamed into existing development programming by donors and implementers alike, especially in planning, monitoring, evaluation, and learning (PMEL) cycles. In this report, we find that mainstreaming adaptive management in PMEL involves three strategies: 1. planning for flexibility; 2. developing locally owned monitoring and evaluation (M&E); and 3. creating an enabling environment for learning. Adopting these strategies contributes to virtuous cycles of PMEL
Dexis Consulting Group. 2016. "Evidence Base For Collaborating, Learning, And Adapting. Summary of the Literature Review." USAID.
CLA Collaborating, Learning, and Adapting CoP Community of Practice EB4CLA Evidence Base for CLA ICT Information and Communication Technology IP Implementing Partner IR Intermediate Result KM Knowledge Management M&E Monitoring and Evaluation NGOs Non-governmental Organizations PPL USAID’s Bureau of Policy, Planning, and Learning TOC Theories of Change USAID/PPL United States Agency for International Development Bureau of Policy, Planning, and Learning
Escoffery C, Lebow-Skelley E, Haardoerfer E, Boing E, Udelson H, Wood R, Hartman M, Fernandez ME, Mullen PD. 2018. "Systematic review of adaptations of public health evidence-based interventions globally." Implementation Science volume 13, Article number: 125. doi: 10.1186/s13012-018-0815-9
Adaptations of evidence-based interventions (EBIs) often occur. However, little is known about the reasons for adaptation, the adaptation process, and outcomes of adapted EBIs. To address this gap, we conducted a systematic review to answer the following questions: (1) What are the reasons for and common types of adaptations being made to EBIs in community settings as reported in the published literature? (2) What steps are described in making adaptations to EBIs? and (3) What outcomes are assessed in evaluations of adapted EBIs?

We conducted a systematic review of English language publications that described adaptations of public health EBIs. We searched Ovid PubMed, PsycINFO, PsycNET, and CINAHL and citations of included studies for adapted public health EBIs. We abstracted characteristics of the original and adapted populations and settings, reasons for adaptation, types of modifications, use of an adaptation framework, adaptation steps, and evaluation outcomes. Results Forty-two distinct EBIs were found focusing on HIV/AIDS, mental health, substance abuse, and chronic illnesses. More than half (62%) reported on adaptations in the USA. Frequent reasons for adaptation included the need for cultural appropriateness (64.3%), focusing on a new target population (59.5%), and implementing in a new setting (57.1%). Common adaptations were content (100%), context (95.2%), cultural modifications (73.8%), and delivery (61.9%). Most study authors conducted a community assessment, prepared new materials, implemented the adapted intervention, evaluated or planned to evaluate the intervention, determined needed changes, trained staff members, and consulted experts/stakeholders. Most studies that reported an evaluation (k = 36) included behavioral outcomes (71.4%), acceptability (66.7%), fidelity (52.4%), and feasibility (52.4%). Fewer measured adoption (47.6%) and changes in practice (21.4%).

These findings advance our understanding of the patterns and effects of modifications of EBIs that are reported in published studies and suggest areas of further research to understand and guide the adaptation process. Furthermore, findings can inform better reporting of adapted EBIs and inform capacity building efforts to assist health professionals in adapting EBIs.
Escoffery C, Lebow-Skelley E, Udelson H, Böing EA, Wood R, Fernandez ME, Mullen PD. 2019. "A scoping study of frameworks for adapting public health evidence-based interventions." Translational Behavioral Medicine, Volume 9, Issue 1, February 2019, Pages 1–10. doi: 10.1093/tbm/ibx067
Evidence-based public health translation of research to practice is essential to improve the public’s health. Dissemination and implementation researchers have explored what happens once practitioners adopt evidence-based interventions (EBIs) and have developed models and frameworks to describe the adaptation process. This scoping study identified and summarized adaptation frameworks in published reports and grey literature. We followed the recommended steps of a scoping study: (a) identifying the research question; (b) identifying relevant studies; (c) selecting studies; (d) charting the data; (e) collating, summarizing, and reporting the results; and (f) consulting with experts. We searched PubMed, PsycINFO, PsycNET, and CINAHL databases for articles referencing adaptation frameworks for public health interventions in the published and gray literature, and from reference lists of framework articles. Two reviewers independently coded the frameworks and their steps and identified common steps. We found 13 adaptation frameworks with 11 program adaptation steps: (a) assess community, (b) understand the EBI(s), (c) select the EBI, (d) consult with experts, (e) consult with stakeholders, (f) decide on needed adaptations, (g) adapt the original EBI, (h) train staff, (i) test the adapted materials, (j) implement the adapted EBI, and (k) evaluate. Eight of these steps were recommended by more than five frameworks: #1–3, 6–7, and 9–11. This study is the first to systematically identify, review, describe, and summarize frameworks for adapting EBIs. It contributes to the literature by consolidating key steps in program adaptation of EBIs and describing the associated tasks in each step.
Evidence to Action. 2019. "Pioneering Tools for Adapting Family Planning and Reproductive Health Interventions in Complex, Dynamic Environments (Brief)." Pathfinder.
At Abdou Moumouni University in Niamey, Niger, the Evidence to Action (E2A) Project partnered with stakeholders in the government and university to successfully: - Equip thousands of students with the knowledge, skills, and resources they need to become agents of change. - Build the capacity of providers to meet students’ reproductive health needs, including a full range of contraceptive options. - Make this progress sustainable. This University Leadership for Change initiative was so effective, we scaled it up to universities in Zinder, Maradi, and Tahoua—and didn’t stop there. As part of the Resilience in Sahel Enhanced-Family Planning (RISE-FP) Project, we joined local partners to adapt our approach to reach young people in a challenging new environment: rural communities in Zinder. Now you can get insights from our adaptation experience and see the innovative tools E2A produced in partnership with Syntegral and used to deliver results, including a 30% increase in new family planning acceptors among young people.
Evidence to Action. 2019. "University Leadership for Change: Adapting and Scaling to the Community Level." Pathfinder.
From 2014 to 2016, the USAID-funded Evidence to Action (E2A) Project implemented the University Leadership for Chance (ULC) project in Niger to promote youth leadership and strengthen health systems to meet family planning and reproductive health (FP/RH) needs. From 2017 to 2019, E2A and partner Syntegral adapted this project to the community setting and implemented it as Community Leadership for Change (CLC) as part of the Resilience in the Sahel Enhanced-Family Planning (RISE-FP) program. E2A and Syntegral developed a set of tools for adapting programs to different contexts to systematically identify needs for adaptation and monitor implementation of these changes in the RISE-CLC project. This report describes the use of the Context of Implementation and Adaptation (COIA) Analysis and Frontline Aggregated Monitoring and Evaluation (FrAME) Adaptive Management System tools throughout project implementation.
Finley EP, Huynh AK, Farmer MM, Bean-Mayberry B, Moin T, Oishi SM, Moreau JL, Dyer KE, Lanham HJ, Leykum L & Hamilton AB. 2018. "Periodic reflections: a method of guided discussions for documenting implementation phenomena." BMC Med Res Methodol 18, 153 (2018).
Ethnography has been proposed as a valuable method for understanding how implementation occurs within dynamic healthcare contexts, yet this method can be time-intensive and challenging to operationalize in pragmatic implementation. The current study describes an ethnographically-informed method of guided discussions developed for use by a multi-project national implementation program.

The EMPOWER QUERI is conducting three projects to implement innovative care models in VA women’s health for high-priority health concerns – prediabetes, cardiovascular risk, and mental health – utilizing the Replicating Effective Programs (REP) implementation strategy enhanced with stakeholder engagement and complexity science. Drawing on tenets of ethnographic research, we developed a lightly-structured method of guided “periodic reflections” to aid in documenting implementation phenomena over time. Reflections are completed as 30–60 min telephone discussions with implementation team members at monthly or bi-monthly intervals, led by a member of the implementation core. Discussion notes are coded to reflect key domains of interest and emergent themes, and can be analyzed singly or in triangulation with other qualitative and quantitative assessments to inform evaluation and implementation activities.

Thirty structured reflections were completed across the three projects during a 15-month period spanning pre-implementation, implementation, and sustainment activities. Reflections provide detailed, near-real-time information on projects’ dynamic implementation context, including characteristics of implementation settings and changes in the local or national environment, adaptations to the intervention and implementation plan, and implementation team sensemaking and learning. Reflections also provide an opportunity for implementation teams to engage in recurring reflection and problem-solving.
Green D. 2015. "The Adaptation Gap (and how to deal with it)." Oxfam.
In practical terms, that means individuals, groups, organisations and networks need to invest in enhanced capabilities to: specify interventions that are relevant to context, drawing on insights of those in that context implement interventions in ways that support the ongoing and real-time sensing of information, insights and ideas from the internal organisational system, from partners and peers, and most importantly from communities and others embedded in operational contexts; make sense of this information, insights and ideas in ways that is relevant for the programme or policy, to support more appropriate, contextually relevant decision making make appropriate changes and adjustments at a strategic and tactical level. Do all of the above on an ongoing basis, in continuous cycles of ‘learning by doing’. At a minimum, aid organisations that addressed the adaptation gap would,: Operate from the “end-user-back”, and not from the “organisation-forward” Develop knowledge, information and data capabilities and tools to anticipate and interpret problems, emerging needs, and to respond to uncertainty and change Empower organisations and teams to make decentralized decisions based on a shared understanding of organizational purpose and values Foster new kinds of networks and partnerships to achieve goals in a highly collaborative fashion Develop and adapt business models as necessary to ensure relevance in a highly fluid and dynamic world
Hansen WB. 2013. "Introduction to the special issue on adaptation and fidelity." Health Education, 113(4), 260-263.
The editorial aims to provide a brief overview of the individual contributions to this special issue, and a commentary on the contributions in terms of their contribution to the broader issue relating fidelity and adaptation to health promotion research, policy and practice.

This is the first special issue with a focus on fidelity and adaptation. Researchers who have recently examined these issues were invited to submit papers that described recent findings to this special issue of Health Education. Following the traditional double blinded peer review process, five submissions were accepted for publication.

The papers in this issue contribute each in their distinctive way to advancing our understanding of the relative influence that fidelity and adaptation have on moderating outcomes of health education programmes. Fidelity and adaptation must be thought of as independent constructs, each of which influences the outcomes of interventions delivered in school and community settings. Originality/value This compilation of papers is the first to systematically address both fidelity and adaptation. Practitioners and researchers alike will gain increased understanding of the potential for fidelity and adaptation to affect outcomes.
Haugh K, Salib M. 2017. "What difference does CLA make to development? Key Findings From A Recent Literature Review." USAID. Evidence Base For CLA.
ADAPT Analysis Driven Agile Programming Techniques CLA Collaborating, Learning, and Adapting CoP Community of practice DAC Development Assistance Committee DFAT Australian Department of Foreign Affairs and Trade DFID UK Department for International Development EB4CLA Evidence Base for CLA GIZ German Federal Enterprise for International Cooperation ICT Information and Communication Technology IDB Inter-American Development Bank IRC International Rescue Committee KM Knowledge management M&E Monitoring and evaluation NGO Nongovernmental organization PPL USAID’s Bureau of Policy, Planning, and Learning SIDA Swedish International Development Agency SRH Self-rated health TOC Theories of Change USAID/PPL United States Agency for International Development Bureau of Policy, Planning, and Learning
Hummelbrunner R, Jones H. 2013. "Background Note: A guide for planning and strategy development in the face of complexity." Overseas Development Institute.
The challenges to economic, social and political development are complex and, therefore, unpredictable (Ramalingam and Jones, 2008). As many commentators have argued, effective programming by governments, non-governmental organisations and international agencies requires a shift in emphasis – moving away from a heavy reliance on planning and ex-ante analysis towards monitoring, learning and adaptation (Jones, 2011). How, then, can policy makers, managers and practitioners best plan in the face of complexity? Does complexity make planning an irrelevant exercise? This background note is a guide, elaborating how planning and strategy development can be carried out despite complexity. While it is true that complex situations require a greater focus on learning and adaptation, this does not render planning irrelevant. In fact, there are ways in which the processes and products of planning can respect the realities of the situation and set up interventions (policies, programmes and projects) to give them the best chance of success. The guide builds on academic, policy and programmatic literature related to themes around systems and complexity (such as an in-depth study by Jones, 2011, which synthesises much of the material), and draws on the authors’ experience of advising development agencies and governments in both developed and developing countries. First, this guide describes the features of complex situations, and explains why they pose a challenge for traditional planning approaches. This should give the reader the necessary tools to assess whether and in what way they are facing a complex problem (and, therefore, whether the guide is relevant for them). Second, it outlines key principles for planning in the face of complexity. This should give the reader an understanding of how planning and strategy development need to differ from more traditional approaches when confronted by complex problems. Third, the guide provides examples of approaches that have been used for planning in situations of complexity. This should give the reader a deeper understanding of the principles involved, and some ideas about how they can be applied in practice
Husain L. 2017. "Policy experimentation and innovation as a response to complexity in China’s management of health reforms." Global Health 13, 54 (2017).
There are increasing criticisms of dominant models for scaling up health systems in developing countries and a recognition that approaches are needed that better take into account the complexity of health interventions. Since Reform and Opening in the late 1970s, Chinese government has managed complex, rapid and intersecting reforms across many policy areas. As with reforms in other policy areas, reform of the health system has been through a process of trial and error. There is increasing understanding of the importance of policy experimentation and innovation in many of China’s reforms; this article argues that these processes have been important in rebuilding China’s health system. While China’s current system still has many problems, progress is being made in developing a functioning system able to ensure broad population access. The article analyses Chinese thinking on policy experimentation and innovation and their use in management of complex reforms. It argues that China’s management of reform allows space for policy tailoring and innovation by sub-national governments under a broad agreement over the ends of reform, and that shared understandings of policy innovation, alongside informational infrastructures for the systemic propagation and codification of useful practices, provide a framework for managing change in complex environments and under conditions of uncertainty in which ‘what works’ is not knowable in advance. The article situates China’s use of experimentation and innovation in management of health system reform in relation to recent literature which applies complex systems thinking to global health, and concludes that there are lessons to be learnt from China’s approaches to managing complexity in development of health systems for the benefit of the poor.
Kirk MA, Moore JE, Wiltsey Stirman S, Birken, S. 2020. "Towards a comprehensive model for understanding adaptations’ impact: the model for adaptation design and impact (MADI)."
Implementation science is shifting from qualifying adaptations as good or bad towards understanding adaptations and their impact. Existing adaptation classification frameworks are largely descriptive (e.g., who made the adaptation) and geared towards researchers. They do not help practitioners in decision-making around adaptations (e.g., is an adaptation likely to have negative impacts? Should it be pursued?). Moreover, they lack constructs to consider “ripple effects” of adaptations (i.e., both intended and unintended impacts on outcomes, recognizing that an adaptation designed to have a positive impact on one outcome may have unintended impacts on other outcomes). Finally, they do not specify relationships between adaptations and outcomes, including mediating and moderating relationships. The objective of our research was to promote systematic assessment of intended and unintended impacts of adaptations by using existing frameworks to create a model that proposes relationships among constructs.

Materials and methods
We reviewed, consolidated, and refined constructs from two adaptation frameworks and one intervention-implementation outcome framework. Using the consolidated and refined constructs, we coded qualitative descriptions of 14 adaptations made to an existing evidence-based intervention; the 14 adaptations were designed in prior research by a stakeholder panel using a modified Delphi approach. Each of the 14 adaptations had detailed descriptions, including the nature of the adaptation, who made it, and its goal and reason. Using coded data, we arranged constructs from existing frameworks into a model, the Model for Adaptation Design and Impact (MADI), that identifies adaptation characteristics, their intended and unintended impacts (i.e., ripple effects), and potential mediators and moderators of adaptations’ impact on outcomes. We also developed a decision aid and website ( to help implementation scientists apply MADI in their work.

Results and conclusions
Our model and associated decision aids build on existing frameworks by comprehensively characterizing adaptations, proposing how adaptations impact outcomes, and offering practical guidance for designing adaptations. MADI encourages researchers to think about potential causal pathways of adaptations (e.g., mediators and moderators) and adaptations’ intended and unintended impacts on outcomes. MADI encourages practitioners to design adaptations in a way that anticipates intended and unintended impacts and leverages best practice from research.
Koorts H & Rutter H. 2021. "A systems approach to scale-up for population health improvement." Health Research Policy and Systems.
Despite a number of important global public health successes, for many health behaviours there is a continued lack of interventions that have been sufficiently scaled up to achieve system-wide integration. This has limited sustainable and equitable population health improvement. Systems change plays a major role in the relation between implementation processes and at-scale institutionalisation of public health interventions. However, in research, systems approaches remain underutilised in scaling up. Public health scale-up models have typically centred on intervention replication through linear expansion. In this paper, we discuss current conceptualisations and approaches used when scaling up in public health, and propose a new perspective on scaling that shifts attention away from the intervention to focus instead on achieving the desired population-level health outcomes. In our view, ‘scaling up’ exists on a continuum. At one end, effective scaling can involve a linear, intervention-orientated expansive approach that prioritises the spread of evidence-based interventions into existing systems in order to drive expansion in the application of that intervention. At the other end, we contend that scale-up can sit within a complex systems paradigm in which interventions are conceptualised as events in systems. In this case, implementation and scale-up activities should focus on generating changes within the system itself to achieve the desired outcome. This we refer to as ‘systems-orientated scale-up’ to achieving population health improvement, which can complement traditional approaches in relevant situations. We argue that for some health behaviours, our proposed approach towards scaling up could enhance intervention implementation, sustainability and population health impact.
Larson A, McPherson R, Posner J, LaFond A, Ricca J. 2015. "Scaling Up High-Impact Health Interventions in Complex Adaptive Systems: Lessons from MCHIP." Maternal and Child Survival Program.
Many global health programs in low- and middle-income countries focus on supporting scale-up to accelerate health gains. Scale-up efforts aim to expand and institutionalize proven health interventions so that they become a part of routine practice within national health systems and available to everyone who needs them. At the same time as greater attention is being paid to taking interventions to scale, practitioners, donors, and researchers in global health are changing their thinking about how national health systems work. Replacing the traditional view of Ministries of health as unified institutions that can be molded into new forms, national health systems are increasingly being recognized as complex adaptive systems (CASs) with diverse components and actors that interact in multiple ways with each other and with the external environment. When interventions are scaled up within such systems, there are multiple interactions with these various components and actors in complex ways, making outcomes unpredictable. There are potential tensions between the goal of scaling up specific, well-defined health interventions in every hospital, clinic, and community across a country and the complex reality of how health systems deliver services. Although scale-up efforts frequently attempt to make the desired intervention as simple as possible to implement, even simple interventions require replacing an old behavior with a new one which is an inherently complex endeavor. The variable results of efforts to increase and maintain the use of new health technologies and practices suggest that we have more to learn about how to achieve scale. Researchers have drawn on practical experience and complexity science to offer insights into how scale-up efforts can best take account of the complex nature of health systems (Paina & Peters 2012; Ramalingam 2013). However, there has been relatively little empirical evidence on how CASs positively and negatively affect scale-up processes and outcomes and, more importantly, how scale-up efforts can harness aspects of CASs to produce desired changes.
Lau AS. 2006. "Making the Case for Selective and Directed Cultural Adaptations of Evidence-Based Treatments: Examples From Parent Training." University of California. Los Angeles.
With prevailing concerns about the generalizability of evidence-based treatments (EBTs) in real-world practice settings, there has been increased attention to the potential of cultural adaptations of treatments to ensure fit with diverse consumer populations. However, it could also be argued that there has been insufficient dissemination and evaluation of our existing EBTs with minority populations to warrant and guide adaptation efforts. This article discusses a framework (a) for identifying instances where cultural adaptation of EBTs may be most indicated, and (b) for using research to direct the development of treatment adaptations to ensure community engagement and the contextual relevance of treatment content. Ongoing work in the area of parent training is highlighted to illustrate key issues and recommendations. Key words: cultural adaptation, evidence-based treatments, minority children and families, parent training
MacDonald M. 2012. "Complexity Science in Brief." University of Victoria.
As an emerging approach to research, complexity science is a study of a system. It is not a single theory, but a collection of theories and conceptual tools from an array of disciplines (Benham-Hutchins & Clancy, 2010; Paley & Gail, 2011). For example, complexity science has been taken up in both natural (i.e. mathematics) and social sciences (ecology), and has become increasingly popular in health care literature. Complexity science is concerned with complex systems and problems that are are dynamic, unpredictable and multi-dimensional, consisting of a collection of interconnected relationships and parts. Unlike traditional “cause and effect” or linear thinking, complexity science is characterized by nonlinearity. According to Miles (2009), complex systems and problems require more than simplistic linear thinking. With a complexity science perspective, there is an appreciation of the complex, dynamic and interconnected relationships occurring within a complex system or problem. Considering the public and population health issues of obesity or chronic disease, there are a multitude of factors and relationships that contribute to the problem. Therefore, a public and population health intervention requires an approach that can account for the complexity of the issue. In other words, public health professionals can be more effective if they understand the complex relationships that are occurring, rather than reducing problems to their smaller parts (Miles, 2009).
Manuel, Clare. 2016. "Delivering institutional reform at scale: Problem-driven approaches supported by adaptive programming." LASER: Legal Assistance for Economic Reform. KPMG.
This synthesis paper suggests how problem-driven approaches supported by adaptive programme management can be implemented at scale in relation to donor programming aimed at institutional reform and improving state capability. It suggests changes to the way standard programme models are applied in practice by DFID and other donors, in light of new thinking about how donors can best support the complex issue of institutional reform, drawing on LASER\\\'s experience on the ground. Two key adjustments to current donor programming approaches are suggested: (1) Put programme function before form: Take time to identify and interrogate institutional problems that local people care about before determining what type of scaled-up donor programming is appropriate – in particular what kind of delivery mechanism and what level of funding. (2) Scope and design the form of donor programming by discovery and doing: Systems thinking suggests that complex institutional problems are best understood, and solutions identified by a process of discovery, rather than primarily by analysis. Scoping/design before a major donor programme ‘crystallises’ and programme form and budget are set, should focus on learning by doing – facilitating local people to iterate around problems to deconstruct them and identify solutions that work for them, and which scaled up donor programming may be able to support. This approach is not appropriate for all donor programming, but it is suggested that there should be space within donor programme portfolios for this approach to be applied to complex institutional reform issues, and that some programming should therefore incorporate: a much longer ‘scoping/design’ process (typically 1-2 years), before programme ‘crystallisation’ i.e. before a scaled up programme delivery mode and budget are determined; a new conceptualisation of scoping/design - involving ‘getting stuck in’ and working with local people on problems they care about without putting money on the table. This requires soft rather than primarily analytical skills; and a narrow entry point to scoping/design through a problem that local people who are able to bring about change wish to engage with. This conceptualisation of scoping/design as an extended process of discovery and learning by doing is likely to be undertaken by contractors with the appropriate soft skills, as well as technical ability, but before the form of scaledup programming has been determined. The paper suggests a range of contracting models to enable this, including: the development of flexible country-level or centrally managed flexible programmes mandated to provide responsive technical assistance for complex institutional reform issues and scope/design by doing in response to emerging opportunities. Such programmes could be sector specific e.g. for the justice sector or for institutional reform issues across the board; or the use of existing framework arrangements. This type of flexible programming for scoping/design, which provides relatively low-key technical assistance, and does not put large-scale funding on the table up front, could lead to the development of more conventional scaledup programmes, once time has been taken to interrogate and iterate around problems, and determine the most appropriate form and funding level for new programming. Once a new programme has ‘crystallised’ and its delivery model and funding levels determined, the appropriate degree of flexibility can be achieved through adaptive programme management and ongoing re-design through learning and adapting, consistent with DFID Smart Rules and requirements for programme management.
Matts M. 2016. "Drafting A Collaborating, Learning, and Adapting Plan." Program Cycle. ADS 201 Additional Help. Bureau for Policy Planning and Learning. USAID.
CLA Activities and Processes These are some of the types of activities and processes that Missions may consider including in their CLA Plans, as appropriate: Strategic Collaboration • Coordinating site visits among technical teams to encourage collaboration • Facilitating implementing partners’ efforts to collaborate and share information with each other through joint work planning sessions or other means Learning • Developing learning agendas or questions around critical knowledge gaps at the Development Objective, project, or activity level • Identifying trends or changes in the context that could impact the portfolio and processes for tracking and responding to them • Participate in knowledge sharing forums, such as communities of practice and learning networks Reflection to Inform Adaptation • Planning for facilitated reflection sessions with local stakeholders to regularly gather information on context changes • Instituting stocktaking and other reflection opportunities to consider new learning, shifting priorities, and necessary adaptations
Mercy Corps. 2015. "Managing Complexity: Adaptive Management at Mercy Corps."
Development actors increasingly agree that a confluence of new challenges and greater complexity require aid agencies to be more agile. We need to be able to adapt in a timely and intentional way, by better understanding, and responding to, contextual dynamics. How, then, can we set up our organizations, country offices and programs to be better able to adapt? This paper pulls out practical tips and short examples for what we have found to be the most important elements of adaptive management at Mercy Corps: Organizational culture provides the cues, expectations and incentives to prioritize learning and adaptation. This section provides suggestions for leadership, teamwork, physical cues and the importance of reinforcing culture through formal mechanisms. People and skills: Adaptive management depends upon a respected, empowered and accountable team, equipped with the skills of critical thinking, analysis and creativity who are expected to gather and use data and information in their work. This section looks at team composition, recruitment, skills building and coaching, and accountability. Tools and systems: A foundational culture of investigation, debate and agility needs to be supported and reinforced by a broad set of tools (both technical and managerial), processes (such as recruitment) and systems (such as finance, procurement and M&E). This section looks at tools and systems for planning and monitoring, regular analysis, providing space for reflection, piloting approaches and internal systems, norms and policies. Enabling environment: To do adaptive management, and work on the changes needed in the first three components often goes beyond the vision of an agency. The ability to be lean and nimble requires buy‐in and flexibility from the broader enabling environment, such as donors and host governments. This section talks about how donors’ systems and expectations can support adaptation at initial program design and implementation stages.
Mshelia C, Lê G, Mirzoev T, Amon S, Kessy A, Baine SO, Huss R. 2016. "Developing learning diaries for action research on healthcare management in Ghana, Tanzania and Uganda." Action Research. 2016;14(4):412-434. doi:10.1177/1476750315626780
Abstract Action research (AR) can be an effective form of ‘on the job’ training. However, it is critical that AR cycles can be appropriately recorded in order to contribute to reflection and learning. One form of recording is for coresearchers to keep a diary. We found no previous literature describing the use of diaries in AR in sub-Saharan Africa. We therefore use this paper to reflect on how diaries were used by district health management teams in the PERFORM project. We share five lessons from our experience. First, it is important to foster ownership of the diary by the people who are responsible for filling it in. Second, the purpose of keeping a diary needs to be clear and shared between researchers and practitioners from the very beginning. Third, diaries should be allowed to evolve. Fourth, it is a challenge for busy practitioners to record the reflection and learning processes that they go through. Last, diaries on their own are not sufficient to capture reflection and learning. In conclusion, there is no best way for practitioners to keep a diary; rather the focus should be on ensuring that an AR recording process (whether diary or otherwise) is locally owned and complements the specific practice setting.

Keywords Diaries, action research, district health management teams, Ghana, Tanzania, Uganda
O'Donnell, Michael. 2016. "Adaptive management: what it means for CSOs." Published by Bond, Society Building, 8 All Saints Street, London N1 9RL, UK.
Adaptive management is an approach to tackling international development challenges that are complex. The starting point is an assumption of uncertainty about what will work to address the challenge. It is then characterised by a flexible approach involving testing, monitoring, getting feedback and – crucially – making course-corrections if necessary. Adaptation is often defined as an alternative to approaches that emphasise adherence to detailed plans to solve development problems in a more linear, mechanistic way. In practice, the management of most interventions falls somewhere along a spectrum between linear and adaptive. Adaptive management is best understood as a broad approach, rather than as a specific method or set of tools. This introductory paper is intended for managers and leaders in civil society organisations and funders, who are not already immersed in the issue. It provides insight into what adaptive management is, when and why it may be appropriate, and what may be required for organisations to adopt adaptive approaches. Working adaptively may not be appropriate in all circumstances. There are relatively simple problems and simple contexts in which more traditional, linear programming approaches are valid. However, it is increasingly recognised that many key development challenges have complex features – from addressing climate change to institution-building to migration to addressing inequality. And the contexts in which those challenges need to be addressed are often also complex. Even many “simple” interventions (such as vaccinating children) take on complex dimensions when they need to be implemented in difficult environments such as fragile and conflict-affected states. CSOs and funders will need to work in more adaptive ways than they currently do to respond appropriately and effectively. There are many existing tools and techniques that can be used for adaptive management (participatory appraisal, political economy analysis, feedback mechanisms, etc.). However, to increase adaptive management practice, arguably a greater focus is needed on creating the right institutional and funding conditions to enable and facilitate it, including more widespread acceptance of uncertainty and risk in programming. This may require quite profound changes for some organisations, meaning that the challenge of increasing the use of adaptive management should not be underestimated.
Ørnemark C. 2016. "GPSA Note: ‘Learning Journeys’ For Adaptive Management-Where Does It Take Us?" GPSA.
Learning-by-doing, SOCIAL ACCOUNTABILITY, feedback loops, learning loops, World Bank, M&E, Monitoring and Evaluation
Paina L, Peters DH. 2012. "Understanding pathways for scaling up health services through the lens of complex adaptive systems." Health Policy and Planning 2012;27:365–373 doi:10.1093/heapol/czr054
Despite increased prominence and funding of global health initiatives, efforts to scale up health services in developing countries are falling short of the expectations of the Millennium Development Goals. Arguing that the dominant assumptions for scaling up are inadequate, we propose that interpreting change in health systems through the lens of complex adaptive systems (CAS) provides better models of pathways for scaling up. Based on an understanding of CAS behaviours, we describe how phenomena such as path dependence, feedback loops, scale-free networks, emergent behaviour and phase transitions can uncover relevant lessons for the design and implementation of health policy and programmes in the context of scaling up health services. The implications include paying more attention to local context, incentives and institutions, as well as anticipating certain types of unintended consequences that can undermine scaling up efforts, and developing and implementing programmes that engage key actors through transparent use of data for ongoing problem-solving and adaptation. We propose that future efforts to scale up should adapt and apply the models and methodologies which have been used in other fields that study CAS, yet are underused in public health. This can help policy makers, planners, implementers and researchers to explore different and innovative approaches for reaching populations in need with effective, equitable and efficient health services. The old assumptions have led to disappointed expectations about how to scale up health services, and offer little insight on how to scale up effective interventions in the future. The alternative perspectives offered by CAS may better reflect the complex and changing nature of health systems, and create new opportunities for understanding and scaling up health services. Complex adaptive systems, health systems, scaling up, health planning Topic: developing countriesfeedbackhealth serviceslens, crystallinelens (device)world health
Petraglia J. 2020. "Should Adaptive Management be Entrusted to Managers?" Pathfinder.
Perceptions matter. And COVID-19 reminds adaptive managers that frontline perceptions may matter the most.
Pett J. 2020. "Navigating adaptive approaches for development programmes: a guide for the uncertain." ODI Working Paper 589.
This working paper compares six of the most prominent adaptive approaches to emerge over the past two decades. Three come from the world of innovation, largely in the private sector (agile, lean startup and human-centred design), and three from the global development sector (thinking and working politically, forms of adaptive management and problem-driven iterative adaptation). While all of these approaches are valuable when used in the right context, practitioners may be perplexed by the multiplicity of methods and jargon. This paper aims to address some of this confusion by mapping where these approaches have come from and showing how they can be applied across the adaptive programme cycle. Armed with this knowledge, practitioners might experiment with different combinations and sequences of adaptive approaches according to the kind of problem and context faced. In turn, this may help us move beyond a siloed view of approaches linked to innovation, adaptive management or more politically smart ways of working.
Plsek PE, Greenhalgh T. 2001. "The challenge of complexity in health care." Complexity Science. BMJ 2001;323:625.
The science of complex adaptive systems provides important concepts and tools for responding to the challenges of health care in the 21st century Clinical practice, organisation, information management, research, education, and professional development are interdependent and built around multiple self adjusting and interacting systems In complex systems, unpredictability and paradox are ever present, and some things will remain unknowable New conceptual frameworks that incorporate a dynamic, emergent, creative, and intuitive view of the world must replace traditional “reduce and resolve” approaches to clinical care and service organisation
Power J, Gilmore B, Vallières F, Toomey E, Mannan H, McAuliffe E. 2019. "Adapting health interventions for local fit when scaling-up: a realist review protocol." BMJ Open.
Abstract Introduction Scaling-up is essential to ensure universal access of effective health interventions. Scaling-up is a complex process, which occurs across diverse systems and contexts with no one-size-fits-all approach. To date, little attention has been paid to the process of scaling-up in how to make adaptations for local fit. The aim of this research is to develop theory on what actions can be used to make adaptations to health interventions for local fit when scaling-up across diverse contexts that will have practical application for implementers involved in scaling-up. Methods and analysis Given the complexity of this subject, a realist review methodology was selected. Specifically, realist review emphasises an iterative, non-linear process, whereby the review is refined as it progresses. The identification of how the context may activate mechanisms to achieve outcomes is used to generate theories on what works for whom in what circumstances. This protocol will describe the first completed stage of development of an initial programme theory framework, which identified potential actions, contexts, mechanisms and outcomes that could be used to make adaptations when scaling-up. It will then outline the methods for future stages of the review which will focus on identifying case examples of scale-up and adaptation in practice. This realist review consists of six stages: (i) clarifying scope and development of a theoretical framework, (ii) developing a search strategy, (iii) selection and appraisal, (iv) data extraction, (v) data synthesis and analysis and (vi) further theory refinement with stakeholders.
Ramalingam B, Laric M, Primrose J. 2014. "From best practice to best fit: Understanding and navigating wicked problems in international development." Overseas Development Institute.
The tools of complex systems research, which emerged from scientific research, are already used and valued by the private and public sectors to better analyse and navigate a range of wicked problems across many disciplines. International development is starting to catch on, with a number of initiatives and projects in this area. Many development partner tools and business processes deal with static, simple or linear problems. There is considerable demand for new methods and principles that can help development partners better navigate the complex, dynamic realities they face on a day-to-day basis. What we did This project looked at the appetite for these new methods in DFID and tested a number of tools and principles in four small-scale pilots: looking at system dynamics in trade; adaptive management and complexity-informed theories of change in private sector development; network analysis in girls’ empowerment; and systems thinking in programme management. What we found There is a significant appetite for improved tools and principles that can help DFID better deliver on its programming. The pilots contributed to improved analysis and understanding of problems, provided a valuable means by which to engage with the wicked nature of challenges, and created sound insights about the kinds of interventions that might be appropriate. The pilots generated tangible findings that were directly utilised in corporate and programmatic decisions. They played a significant role in the design of two large programmes, and provided the evidence base for a root and branch review of DFID processes. The findings were not unanimously positive, however: the pilot recommendations were not always tailored to DFID’s organisational realities, and needed some reworking. Moreover, some of the methods did not fit easily within DFID processes and required some adaptation. The terminology and visualisations were in some places hard for DFID staff to understand. FROM BEST PRACTICE TO BEST FIT v  improve their understanding of wicked problems in different areas of development work  trial and adapt new tools to improve analysis and programming in the face of these problems  ensure their internal systems, processes, skills and capacities can support these improved analytical and programming approaches  build linkages with complexity specialists in different sectors to support all of the above. Complex systems specialists should:  work at adapting tools from other sectors to development work  simplify terminology and make methods more accessible  build the evidence of benefits and costs of applying complex systems methods in different contexts
Schreiber ES, Bearlin AR, Nicol SJ, Todd CR. 2004. "Adaptive management: a synthesis of current understanding and effective application." Wiley.
Adaptive management (AM) remains a commonly cited, yet frequently misunderstood, management approach. The aim of AM is to improve environmental management through ‘learning by doing’ and understand the impact of incomplete knowledge, but AM more commonly consists of ad hoc changes in managing environmental resources in the absence of adequate planning and monitoring. Here, we trace and review the development of AM, the central roles of consultation, collaboration and of monitoring, and of quantitative models and simulations. We identify a series of formalized, structured steps included in one AM cycle and review how current AM programs build upon such cycles. We conclude that the best AM outcomes require rigorous and formalized approaches to planning, collaboration, modelling and evaluation. Finally, simulating potential outcomes of an AM cycle in the presence of existing uncertainty can help to identify management strategies that are most likely to succeed in relation to clearly articulated goals. Key words adaptive environmental assessment and management, modelling, uncertainty
Shilomboleni, De Plaen. 2019. "Scaling up research-for-development innovations in food and agricultural systems." Development in Practice, 29:6, 723-734, DOI: 10.1080/09614524.2019.1590531
The last decade has seen a growing interest in scaling up innovations to realise wider benefits from development investments. While numerous proven technologies, products and models have been successfully piloted, scaling them up through expansion, adoption and replication has proved challenging, particularly in poor regions of the world. The low uptake of innovations is partially attributed to the design of technologies, in a manner that is not compatible with local farming practices. At the same time, proven innovations fail to generate large impacts at scale because implementing actors have not sufficiently understood or effectively engaged with the scaling process. This article shares lessons from the Canadian International Food Security Research Fund (CIFSRF) that supported applied research to develop, test and scale up promising food and nutrition security innovations. Key lessons include ensuring that innovations are embedded within local socio-ecological systems; engaging end users throughout the research process and enabling participatory decision-making; and considering the investment returns of innovations for end-users. KEYWORDS: Aid – Capacity DevelopmentCivil society – Participation, PartnershipEnvironment (built and natural) – Food security
Sterman JD. 2006. "Learning from Evidence in a Complex World." American Journal of Public Health 96, no. 3: pp. 505-514. doi: 10.2105/AJPH.2005.066043
Policies to promote public health and welfare often fail or worsen the problems they are intended to solve. Evidence-based learning should prevent such policy resistance, but learning in complex systems is often weak and slow. Complexity hinders our ability to discover the delayed and distal impacts of interventions, generating unintended “side effects.” Yet learning often fails even when strong evidence is available: common mental models lead to erroneous but self-confirming inferences, allowing harmful beliefs and behaviors to persist and undermining implementation of beneficial policies. Here I show how systems thinking and simulation modeling can help expand the boundaries of our mental models, enhance our ability to generate and learn from evidence, and catalyze effective change in public health and beyond. The United States spends more on health care than any other nation (15.3% of gross domestic product [GDP] in 2003, up from 5.1% in 1960).1,2 Yet the return on this huge investment is discouraging: the United States ranks 33rd in life expectancy and 35th in infant mortality.2 More than 40 million have no health insurance. Minorities and the poor have significantly lower life expectancy than others.3 Nearly two thirds of US adults are overweight, and almost one third are obese.4 Diabetes and cardiovascular disease are rampant. The number of unhealthy days Americans experience is growing.5 Preventable medical errors kill tens of thousands each year.6 From Staphylococcus aureus to malaria to HIV, morbidity and mortality from drug-resistant pathogens grows.7 Most disturbing, many of these afflictions are the unintended consequences of the extraordinary prosperity and technical progress that enabled us to treat disease and decrease daily toil so successfully over the past century. Health care is not unique. Thoughtful leaders throughout society increasingly suspect that the policies we implement to address difficult challenges have not only failed to solve the persistent problems we face, but are in fact causing them. All too often, well-intentioned programs create unanticipated “side effects.” The result is policy resistance, the tendency for interventions to be defeated by the system’s response to the intervention itself. From overuse of antibiotics that spread resistant pathogens, to the obesity caused by the sedentary lifestyles and cheap calories our prosperity affords, our best efforts to solve problems often make them worse (box next page).
Stirman SW, Miller CJ, Toder K, Calloway A. 2013. "Development of a framework and coding system for modifications and adaptations of evidence-based interventions." Implementation Science volume 8, Article number: 65. doi: 10.1186/1748-5908-8-65
Evidence-based interventions are frequently modified or adapted during the implementation process. Changes may be made to protocols to meet the needs of the target population or address differences between the context in which the intervention was originally designed and the one into which it is implemented [Addict Behav 2011, 36(6):630–635]. However, whether modification compromises or enhances the desired benefits of the intervention is not well understood. A challenge to understanding the impact of specific types of modifications is a lack of attention to characterizing the different types of changes that may occur. A system for classifying the types of modifications that are made when interventions and programs are implemented can facilitate efforts to understand the nature of modifications that are made in particular contexts as well as the impact of these modifications on outcomes of interest.

We developed a system for classifying modifications made to interventions and programs across a variety of fields and settings. We then coded 258 modifications identified in 32 published articles that described interventions implemented in routine care or community settings.

We identified modifications made to the content of interventions, as well as to the context in which interventions are delivered. We identified 12 different types of content modifications, and our coding scheme also included ratings for the level at which these modifications were made (ranging from the individual patient level up to a hospital network or community). We identified five types of contextual modifications (changes to the format, setting, or patient population that do not in and of themselves alter the actual content of the intervention). We also developed codes to indicate who made the modifications and identified a smaller subset of modifications made to the ways that training or evaluations occur when evidence-based interventions are implemented. Rater agreement analyses indicated that the coding scheme can be used to reliably classify modifications described in research articles without overly burdensome training.

This coding system can complement research on fidelity and may advance research with the goal of understanding the impact of modifications made when evidence-based interventions are implemented. Such findings can further inform efforts to implement such interventions while preserving desired levels of program or intervention effectiveness.
USAID. 2016. "Collaborating, Learning, and Adapting Framework & Key Concepts."
Continuous Learning & Improvement Relationships & Networks Openness Decision-Making Institutional Memory Knowledge Management CLA in Implementing Mechanisms Resources Mission Internal Collaboration External Collaboration Pause & Reflect Adaptive Management Technical Evidence Base Theories of Change Scenario Planning M&E for Learning
USAID. 2018. "Discussion Note: Adaptive Management." Program Cycle.
This Discussion Note complements ADS Program Cycle Principles by elaborating on Principle 2: Manage Adaptively through Continuous Learning. This Discussion Note is intended for USAID staff interested in learning about recent and promising practices in adaptive management across the Program Cycle. USAID’s work takes place in environments that are often unstable and in transition. Even in more stable contexts, circumstances evolve and may affect programming in unpredictable ways. For its programs to be effective, USAID must be able to adapt in response to changes and new information. The ability to adapt requires an environment that promotes intentional learning and flexible project and activity design, minimizes the obstacles to modifying programming and creates incentives for managing adaptively. Adaptive management is defined in ADS 201.6 as “an intentional approach to making decisions and adjustments in response to new information and changes in context.” Adaptive management is not about changing goals during implementation, it is about changing the path being used to achieve the goals in response to changes. Like other donors and development organizations (see, for example, the following initiatives: Doing Development Differently, Problem-Driven Iterative Adaptation, Thinking and Working Politically, and The World Bank’s Global Delivery Initiative), USAID is increasingly recognizing the importance of adaptability for its work to be effective. ADS 201 now integrates adaptive management approaches throughout the Program Cycle. “Manage adaptively through continuous learning” is one of the four core principles that serve as the foundation for Program Cycle implementation. This Discussion Note is organized around the phases of the Program Cycle (strategy, project, and activity design and implementation; monitoring and evaluation; and learning and adapting); While the adaptive management approaches described here are examples of initial entry points associated with a specific phase of the Program Cycle, many of these approaches lead to adjustment in other areas. The note concludes with sections on enabling conditions and a description of the skills and attributes of adaptive managers.
USAID. 2013. "Discussion Note: Complexity-Aware Monitoring." Monitoring & Evaluation Series. Version 2. USAID Office of Learning Evaluation and Research (LER).
Performance monitoring is intended to “reveal whether implementation is on track and whether expected results are being achieved (ADS” Complexity-aware monitoring complements performance monitoring for aspects of strategies, project and activities where cause-effect relationships are uncertain and agreement on problems and solutions is low. When USAID staff identify components of strategies, projects, and activities that meet these criteria, they may consider using complementary monitoring approaches that are complexity-aware in order to address performance monitoring’s three blind spots (unintended outcomes, alternative causes and feedback loops), synchronize with the pace of change, and consider key systems concepts, such as interrelationships, perspectives, and boundaries. Complexity-aware monitoring can be used in conjunction with performance and context monitoring keeping in mind the distinctive strengths of each.
USAID. 2000. "Understanding CLA."
Collaborating, Learning, and Adapting (CLA) is a set of practices that help us improve our development effectiveness. Learning has always been part of USAID’s work, and most USAID missions and implementing partners are already practicing CLA in some way. Our aim now is to make CLA more systematic and intentional throughout the Program Cycle, and to dedicate the resources necessary to make it happen. According to USAID’s Program Cycle guidance (ADS 201.3.7), “Strategic collaboration, continuous learning, and adaptive management link together all components of the Program Cycle.” Integrating CLA into our work helps to ensure that our programs are coordinated with others, grounded in a strong evidence base, and iteratively adapted to remain relevant throughout implementation. The systematic application of CLA approaches, led by people who have the knowledge and resources to carry them out, enables USAID to be an effective learning organization and thereby a more effective development organization. In the simplest terms, integrating collaborating, learning, and adapting throughout the Program Cycle can help development practitioners address the above challenges by thinking through: Collaborating: Are we collaborating with the right partners at the right time to promote synergy over stove-piping? Learning: Are we asking the most important questions and finding answers that are relevant to decision making? Adapting: Are we using the information that we gather through collaboration and learning activities to make better decisions and make adjustments as necessary? Enabling Conditions: Are we working in an organizational environment that supports our collaborating, learning, and adapting efforts?
Valters C, Cummings C, Nixon H. 2016. "Putting learning at the centre: Adaptive development programming in practice." Overseas Development Institute.
Adaptive programming suggests, at a minimum, that development actors react and respond to changes in the political and socio-economic operating environment. It emphasises learning and the development practitioner is encouraged to adjust their actions to find workable solutions to problems that they may face. Being prepared to react to change may seem like common sense – and indeed it is. However much development thinking and practice remains stuck in a linear planning model which discourages learning and adaptation, in part because projects are seen as ‘closed, controllable and unchanging systems’ (Mosse, 1998: 5). This paper critically engages with this problem and makes clear why and how learning needs to be at the centre of adaptive development programming. It begins by clarifying why and what kind of learning matters for adaptive programming. The paper then turns its focus to how strategies and approaches applied throughout a programme’s conception, design, management and M&E can enable it to continually learn and adapt. Global Monitoring, evaluation and learning Fragile states Aid
Wang Z, Norris SL, Bero L. 2018. "The advantages and limitations of guideline adaptation frameworks." Implementation Science (2018) 13:72.
The implementation of evidence-based guidelines can improve clinical and public health outcomes by helping health professionals practice in the most effective manner, as well as assisting policy-makers in designing optimal programs. Adaptation of a guideline to suit the context in which it is intended to be applied can be a key step in the implementation process. Without taking the local context into account, certain interventions recommended in evidence-based guidelines may be infeasible under local conditions. Guideline adaptation frameworks provide a systematic way of approaching adaptation, and their use may increase transparency, methodological rigor, and the quality of the adapted guideline. This paper presents a number of adaptation frameworks that are currently available. We aim to compare the advantages and limitations of their processes, methods, and resource implications. These insights into adaptation frameworks can inform the future development of guidelines and systematic methods to optimize their adaptation.

Recent adaptation frameworks show an evolution from adapting entire existing guidelines, to adapting specific recommendations extracted from an existing guideline, to constructing evidence tables for each recommendation that needs to be adapted. This is a move towards more recommendation-focused, context-specific processes and considerations. There are still many gaps in knowledge about guideline adaptation. Most of the frameworks reviewed lack any evaluation of the adaptation process and outcomes, including user satisfaction and resources expended. The validity, usability, and health impact of guidelines developed via an adaptation process have not been studied. Lastly, adaptation frameworks have not been evaluated for use in low-income countries.

Despite the limitations in frameworks, a more systematic approach to adaptation based on a framework is valuable, as it helps to ensure that the recommendations stay true to the evidence while taking local needs into account. The utilization of frameworks in the guideline implementation process can be optimized by increasing the understanding and upfront estimation of resource and time needed, capacity building in adaptation methods, and increasing the adaptability of the source recommendation document.
Wild L, Booth D, Cummings C, Foresti M, Wales J. 2015. "Adapting development Improving services to the poor." Overseas Development Institute.
This report argues that if we are to avoid reproducing the pattern of uneven progress that has characterised the MDG campaign, there must be more explicit recognition of the political conditions that sometimes enable, but so often obstruct, development progress. In this context, domestic reformers and their international partners must pursue innovative and politically smart ways to tackle the most intractable problems.

Key findings:
The global discussion around the post-2015 agenda has recognised the need to adjust to a changed and changing global context. However, there are key areas where the SDG discussion is not inviting different commitments, but is still focused on more of the same. A key danger is that discussions of what it will take to achieve the new goals will, once again, centre on financing gaps. During the pursuit of the MDGs, this focus reinforced the belief that inadequacies in provision could be dealt with easily enough if there was enough new funding on the table. Today, with higher rates of economic growth in many parts of the developing world, there is a new temptation to assume that growth, by itself, will take care of the problem. There is also a growing recognition that it is the function as well as the form of institutions that matters for translating resources into results. But, on international platforms, there is a lack of realism about the type of change processes and the institutional adjustments that have been linked to development breakthroughs in recent times and in past history. This fuels illusions about how easy it is going to be, in a typical poor country, to tackle gaps and inequities. The evidence base for doing things differently needs to become stronger, and more needs to be said and debated about the scope for taking these different approaches to scale. Changes in domestic politics and policy processes are by far the most important drivers of development outcomes and improvements in service delivery. Donors can help reform processes to adopt a problem-driven and adaptive approach, but if they are to be effective they must act as facilitators and brokers of locally led processes of change, not as managers.

Aid should do more to support initiatives that are problem-driven, adaptive and locally led. These initiatives need financial and other support that is fit for that purpose. This means not only tracking MDG-type development outcomes but also monitoring and building up an understanding of the intermediate changes in process that are most effective in improving those outcomes. Measures of how ‘adaptive’ or ‘locally led’ aid programmes are would be a good start. There are many areas where spending that benefits poor countries could be increased, but the current debate about targets for aid spending is too focused on the ability of the donor country to pay, rather than whether those funds are used effectively. Looking at how aid works is more important than how much to spend.
Wilson GJ. 2016. "What is adaptive management?"
Recommendations – How could Adaptive Management be encouraged and addressed by Donors and Host Governments? Encourage implementing partners to embrace AM and the ways and means to tackle complexity. Strongly support the host Government’s drive for more information and data on their current situation and development needs, not just at the project level but improvements to national statistics, and poverty data Share and encourage open access policies; free and open access to publicly-funded research offers significant social and economic benefits. Development bodies must not be allowed to hoard data Encourage adaptive management with innovative funding mechanisms, encouragement for innovative research, and development of new technology solutions to managing aid and improving VFM. Support local universities to develop their capacity, encourage partnerships, north-south, south-south Ensure all programme designs and implementation plans draw on the perspectives of local partners and those with whom they work. Gateways in the programme and project approval process will need to demand that provision is made for systematic MEL. In situations where Development Trust Funds operate ensure that all proposed projects incorporate a clear statement on MEL and how the programme/project will provide a steady stream of information that is used to understand the context and programme performance, and how the information will be stored and made accessible. In short, a closer focus on accountability for learning Establish more joint programmes for both DPs and other partners to learn about Adaptive Management and PDIA type approaches; adopt a joint commitment to collaborative learning. In complex emergencies where humanitarian and development action co-exists, demand joint approaches to understanding and responding.

Country Government Ownership

Atun R, Kazatchkine M. 2009. "Promoting Country Ownership and Stewardship of Health Programs: The Global Fund Experience." JAIDS Journal of Acquired Immune Deficiency Syndromes. 52():S67-S68. DOI: 10.1097/QAI.0b013e3181bbcd58
The Global Fund to Fight AIDS, Tuberculosis and Malaria was established in 2002 to provide large-scale financing to middle- and low-income countries to intensify the fight against the 3 diseases. Its model has enabled strengthening of local health leadership to improve governance of HIV programs in 5 ways. First, the Global Fund has encouraged development of local capacity to generate technically sound proposals reflecting country needs and priorities. Second, through dual-track financing-where countries are encouraged to nominate at least one government and one nongovernment principal recipient to lead program implementation-the Global Fund has enabled civil society and other nongovernmental organizations to play a critical role in the design, implementation, and oversight of HIV programs. Third, investments to strengthen community systems have enabled greater involvement of community leaders in effective mobilization of demand and scale-up for services to reach vulnerable groups. Fourth, capacity building outside the state sector has improved community participation in governance of public health. Finally, an emphasis on inclusiveness and diversity in planning, implementation, and oversight has broadly enhanced country coordination capacity. Strengthening local leadership capacity and governance are critical to building efficient and equitable health systems to deliver universal coverage of HIV services.
Bao J, Rodriguez DC, Paina L, Ozawa S, Bennett S. 2015. "Monitoring and Evaluating the Transition of Large-Scale Programs in Global Health." Global Health: Science and Practice December 2015, 3(4):591-605;
Donors are increasingly interested in the transition and sustainability of global health programs as priorities shift and external funding declines. Systematic and high-quality monitoring and evaluation (M&E) of such processes is rare. We propose a framework and related guiding questions to systematize the M&E of global health program transitions.

We conducted stakeholder interviews, searched the peer-reviewed and gray literature, gathered feedback from key informants, and reflected on author experiences to build a framework on M&E of transition and to develop guiding questions.

The conceptual framework models transition as a process spanning pre-transition and transition itself and extending into sustained services and outcomes. Key transition domains include leadership, financing, programming, and service delivery, and relevant activities that drive the transition in these domains forward include sustaining a supportive policy environment, creating financial sustainability, developing local stakeholder capacity, communicating to all stakeholders, and aligning programs. Ideally transition monitoring would begin prior to transition processes being implemented and continue for some time after transition has been completed. As no set of indicators will be applicable across all types of health program transitions, we instead propose guiding questions and illustrative quantitative and qualitative indicators to be considered and adapted based on the transition domains identified as most important to the particular health program transition. The M&E of transition faces new and unique challenges, requiring measuring constructs to which evaluators may not be accustomed. Many domains hinge on measuring “intangibles” such as the management of relationships. Monitoring these constructs may require a compromise between rigorous data collection and the involvement of key stakeholders.

Monitoring and evaluating transitions in global health programs can bring conceptual clarity to the transition process, provide a mechanism for accountability, facilitate engagement with local stakeholders, and inform the management of transition through learning. Further investment and stronger methodological work are needed.
Bennett S, Rodriguez D, Ozawa S, Singh K, Bohren M, Chhabra V, Singh S. 2015. "Management practices to support donor transition: lessons from Avahan, the India AIDS Initiative." BMC Health Services Researchvolume 15, Article number: 232.
During 2009-2012, Avahan, a large donor funded HIV/AIDS prevention program in India was transferred from donor support and operation to government. This transition of approximately 200 targeted interventions (TIs), occurred in three tranches in 2009, 2011 and 2012. This paper reports on the management practices pursued in support of a smooth transition of the program, and addresses the extent to which standard change management practices were employed, and were useful in supporting transition.

We conducted structured surveys of a sample of 80 TIs from the 2011 and 2012 rounds of transition. One survey was administered directly before transition and the second survey 12 month after transition. These surveys assessed readiness for transition and practices post-transition. We also conducted 15 case studies of transitioning TIs from all three rounds, and re-visited 4 of these 1-3 years later.

Considerable evolution in the nature of relationships between key actors was observed between transition rounds, moving from considerable mistrust and lack of collaboration in 2009 toward a shared vision of transition and mutually respectful relationships between Avahan and government in later transition rounds. Management practices also evolved with the gradual development of clear implementation plans, establishment of the post of “transition manager” at state and national levels, identified budgets to support transition, and a common minimum programme for transition. Staff engagement was important, and was carried out relatively effectively in later rounds. While the change management literature suggests short-term wins are important, this did not appear to be the case for Avahan, instead a difficult first round of transition seemed to signal the seriousness of intentions regarding transition.

In the Avahan case a number of management practices supported a smooth transition these included: an extended and sequenced time frame for transition; co-ownership and planning of transition by both donor and government; detailed transition planning and close attention to program alignment, capacity development and communication; engagement of staff in the transition process; engagement of multiple stakeholders post transition to promote program accountability and provide financial support; signaling by actors in charge of transition that they were committed to specified time frames.
Burrows D, Oberth G, Parsons D, McCallum L. 2016. "Transitions from donor funding to domestic reliance for HIV responses: Recommendations for transitioning countries." APM Global Health.
Key messages To aid in the transparency and predictability of transition processes, we need:
Systematic transition criteria: A clear set of criteria needs to be developed for assessment of a country’s transition preparedness. Publicly available transition schedules: Transition should be discussed between donors and representatives of the country to determine start and end dates and duration of transition. Coordinated donor decisions: Donors need a clearer mechanism to communicate their transition plans about a particular country with each other. March 2016 Transitions from donor funding to domestic reliance for HIV responses 6 “Good practice” transitions require: Time: Not only is a period of several (5-10) years required, but also a phased roadmap to achieve various specified financial and operational targets is needed. High-level political commitment: Without commitment at the highest political levels, transitions can be easily derailed by changes in staffing, in political parties, in economic circumstances, etc. Country ownership: Aligning donor-funded projects with national policy as well as with the national context is important if projects are to be absorbed by domestic bill-payers. Built-in monitoring and evaluation: M&E is needed to assess progress against the roadmap targets, as well as to track changes to the epidemic, issues affecting the testing and treatment cascade, access by key populations to essential services, and other important considerations. Transitions that promote and protect human rights are most likely to maintain and expand access to essential HIV services by key populations through: Funding mechanisms for NGOs, which must be in place and working effectively to enable access to sufficient funds for key population service delivery programs. High-level political engagement, specifically related to the costs and benefits of excluding or including specific key populations in national HIV responses. Improved in-country capacity for advocacy based on data collection and analysis by NGOs or community-based networks representing each relevant key population. Increased capacity of NGOs to demonstrate specifically the level and types of activities they will undertake in the HIV prevention and treatment cascade to justify the sustained allocation. Ensured funding for police, security, and criminal justice reform programs because these structural elements have the strongest influence in most countries over access of key populations to needed services.
Gbeleou SC, Schechter J. 2020. "How NGOs Can Work With Governments to Build Partnerships That Will Scale." Stanford Social Innovation Review.
Four lessons from Togo on scaling health care innovation through the public sector. On the night before Integrate Health’s first Community Health Worker training in 2015, we got a startling call: “Please take the Ministry of Health logo off the training materials.” Integrate Health had worked with the Ministry of Health’s District Health Director, from day one, to design the program. But she had gotten cold feet. Gbeleou went to her office the next morning, and emerged after an hour, shaking his head. We had secured permission to proceed with the training and launch the program, with only the NGO logo on the training materials. The program that Integrate Health and the Ministry of Health were piloting together, the Integrated Primary Care Program, had acquired all of the necessary approvals and was even part of a joint study. But it was also different—Community Health Workers were being paid to work full-time, unlike the country\'s volunteer model—and different can be risky. Since that initial rocky start in 2015, Integrate Health has made significant progress in strengthening our partnership with the Ministry of Health. In fact, Gbeleou and Dr. Sibabe Agoro, Regional Health Director for the Ministry of Health in Togo, had just returned from obtaining their UK visas in March of this year when they learned that the Skoll World Forum had been canceled due to COVID-19. Gbeleou and Dr. Agoro had planned to travel to Oxford to participate in a panel discussion on government and NGO partnership at scale. But as the novel coronavirus grew to a pandemic, the critical need for effective NGO and government collaboration rapidly accelerated. 1. Center the voice of government. 2. Implement within existing government delivery and data systems. 3. Build relationships with government partners at all levels. 4. Progressively transfer real ownership to government partners.
Germain A. 2011. "Ensuring the complementarity of country ownership and accountability for results in relation to donor aid: a response." Reproductive Health Matters, 19:38, 141-145, DOI: 10.1016/S0968-8080(11)38595-3
Aid allocations by donor countries to sexual and reproductive health and rights and the values that underlie that aid are a timely and important topic. In general, data about such funding are weak and the field would benefit significantly from better and more comprehensive data on what the funds are earmarked for and how they are used. In the absence of such data, it is extremely difficult to address the issue objectively. This paper is a response to the paper by Sara Seims on improving the impact of sexual and reproductive health development assistance from the seven like-minded European donors.1 It offers a different perspective on several of the key issues she raises.
Ghebreyesus TA. 2010. "Achieving the health MDGs: country ownership in four steps." The Lancet 376(9747): 1127-1128.
Monitoring and Evaluation. The first step is planning. Countries must start with a clear development vision, but they also need to elaborate a detailed roadmap for realising it. In Ethiopia, our vision is to become a middle-income country over the next 10–15 years, and our government has clearly articulated strategies for how to get there. For country ownership to be realised, development partners must allow countries the space to identify their own needs and priorities, and develop their own plans as they see fit. But countries should also be open to ideas and seek to tailor proven practices to their particular circumstances. Once a well-considered national plan is in place, however, partners need to support that plan if country ownership is to thrive. We remained open to ideas throughout the planning process. We invited partners\' contributions and benchmarked best practices from other countries. This is the most decisive step towards real ownership. The second step is resourcing the plan. Here too, countries must take the lead. And because resources are limited, careful prioritisation is crucial. In crafting our health plan, we defined two alternative versions. If resource constraints mean that we cannot implement our broader and more ambitious plan, we go with our contingency plan, which focuses on the most pressing priorities. Even more important is the way in which resources are channelled. Flexible and predictable funding fosters accountability and ownership by allowing countries greater leverage in responsibly managing resources. Direct budget support is the ideal mechanism, in view of the enhanced flexibility and control it affords countries. In cases where our partners\' chosen mechanism is not budgetary support, we have negotiated ways in which the funding can be used to benefit the whole health system. We have even used vertically raised funds—ie, those earmarked for disease-specific services—to strengthen our health system. For example, about 25–30% of HIV/AIDS grants from the Global Fund to Fight AIDS, Tuberculosis and Malaria and 15% of resources from the US President\'s Emergency Plan for AIDS Relief have been used to build system capacities in many areas, including an information system for health management, a supply-chain management system, and major improvements in human resources. The third step is implementation, in which countries must also be fully engaged. Some have argued that countries lack the capacity to implement. If so, the most efficient and sustainable solution is for partners to strengthen existing capacities within the country rather than replacing them with parallel structures. If existing national systems and procedures are inadequate, partners should work with countries to fix them.
Gillespie D, Fredrick B, Karklins S, Whitmarsh S. 2020. "The Future of Advocacy - Local Ownership, Sustainability, and Grant-making." Johns Hopkins Bloomberg School of Public Health, Advance Family Planning (AFP) initiative.
Donors have long wanted to shift programs to local ownership. But the shift has been problematic. Programs are still largely donor-driven. Full local ownership is unlikely, but collaborative partnership of donors and local entities is a proven model that can both improve health and promote civil society. If donors expect sustained advocacy after their support ends, they need to support capacity building directly or through higher indirect cost allowances. Depending on their investment aims, donors can choose from four options for strengthening local capacity for partnership. A proposed model for strengthening local advocacy capacity worldwide envisions a global project, regional advocacy hubs, and networks of local NGOs.
Goldberg J, Bryant M. 2012. "Country ownership and capacity building: the next buzzwords in health systems strengthening or a truly new approach to development." BMC Public Health volume 12, Article number: 531.
During the last decade, donor governments and international agencies have increasingly emphasized the importance of building the capacity of indigenous health care organizations as part of strengthening health systems and ensuring sustainability. In 2009, the U.S. Global Health Initiative made country ownership and capacity building keystones of U.S. health development assistance, and yet there is still a lack of consensus on how to define either of these terms, or how to implement “country owned capacity building”.

Concepts around capacity building have been well developed in the for-profit business sector, but remain less well defined in the non-profit and social sectors in low and middle-income countries. Historically, capacity building in developing countries has been externally driven, related to project implementation, and often resulted in disempowerment of local organizations rather than local ownership. Despite the expenditure of millions of dollars, there is no consensus on how to conduct capacity building, nor have there been rigorous evaluations of capacity building efforts. To shift to a new paradigm of country owned capacity building, donor assistance needs to be inclusive in the planning process and create true partnerships to conduct organizational assessments, analyze challenges to organizational success, prioritize addressing challenges, and implement appropriate activities to build new capacity in overcoming challenges. Before further investments are made, a solid evidence base should be established concerning what works and what doesn’t work to build capacity.

Country-owned capacity building is a relatively new concept that requires further theoretical exploration. Documents such as The Paris Declaration on Aid Effectiveness detail the principles of country ownership to which partner and donor countries should commit, but do not identify the specific mechanisms to carry out these principles. More evidence as to how country-owned capacity building plays out in practice is needed to guide future interventions. The Global Health Initiative funding that is currently underway is an opportunity to collect evaluative data and establish a centralized and comprehensive evidence base that could be made available to guide future country-owned capacity building efforts.
Mai M, Hassen E, Ntabona AB, Bapura J, Sarathy M, Yodi R, Mujani Z. 2019. "Government Ownership and Adaptation in Scale - Up: Experiences from Community - Based Family Planning Program in the Democratic Republic of the Congo." Afr J Reprod Health. 2019 Dec;23(4):35-45. PMID: 32227738. doi: 10.29063/ajrh2019/v23i4.5
A systematic approach to scale-up was applied to expand an integrated package of family planning and primary healthcare services from the Democratic Republic of the Congo\\\\\\\'s South Kivu province to health zones in Lomami, Lualaba, and Kasai Central provinces. This approach was based on recommendations from the ExpandNet/WHO guide Beginning with the end in mind. The approach emphasized application of three recommendations: engaging government stakeholders, ensuring the relevance of the intervention, and tailoring the innovation to the setting. This approach led to successful scale-up of community-based family planning, increasing access to and uptake of contraception and demonstrating potential for sustainability; 231,566 new acceptors were recruited and 149,826 couple-years of protection were generated. The systematic scale-up approach led to integration of community-based family planning indicators in the national health information system and transferred ownership of the interventions to the government, creating and strengthening government platforms with potential to sustain the interventions. Keywords: DRC; adaptation; communities; family planning; integration; systematic scale-up.
Ministerial Leadership Initiative for Global Health. 2015. "The MLI Model for Advancing Country Ownership." Aspen Global Health and Development, The Aspen Institute.
In the world of global aid and development, country ownership has become one of the most debated issues. But what does country ownership mean in practical terms to donors, development partners, civil society, and country governments? If the goal is to put each country into the driver's seat, how can we get there? For nearly five years, the Ministerial Leadership Initiative for Global Health - MLI - has forged new pathways for advancing this critical development issue, working with five countries in Africa and Asia. What emerged is a distinctive MLI Model to advance country ownership.
Mwale PM, Wa-Chizuma Msiska T. 2020. "Government-led Community Score Card for Family Planning Services." CARE.
Social accountability approaches have been gaining popularity and attention in the international development community as both donors and governments have responded to demands from citizens and civil society organizations to establish more transparent, accountable, and responsive public service processes. At their core, these social accountability approaches provide ways for citizens to hold their governments accountable for the deli very and provision of quality public services by engaging directly with policy makers, civil servants, and service providers.

The CSC brings service users and service providers together along with local government to identify service utilization and provision challenges, mutually generate and execute shared solutions, and track effectiveness of those solutions in an ongoing process of quality improvement. Since its initial development by CARE Malawi in 2002, the CSC has been used in more than 15 countries in Africa, Asia and South America across service sectors including natural resource management, education, and water and sanitation.
Nanyonjo A, Kertho E, Källander K. 2020. "District Health Teams’ Readiness to Institutionalize Integrated Community Case Management in the Uganda Local Health Systems: A Repeated Qualitative Study." Global Health: Science and Practice 8(2): 190–204.
Several countries have adopted integrated community case management (iCCM) as a strategy for improved health service delivery in areas with poor health facility coverage. Early implementation of iCCM is often run by nongovernmental organizations financed by donors through projects. Such projects risk failure to transition into programs run by the local health system upon project closure. Engagement of subnational health authorities such as district health teams (DHTs) is essential for a smooth transition.

We used a repeated qualitative study design to assess the readiness of and progress made by DHTs in institutionalizing iCCM into the functions of locally decentralized health systems in 9 western Uganda districts. Readiness data were derived from structured group interviews with DHTs before iCCM policy adoption in 2010 and again in 2015. Progressive institutionalization achievements were assessed through key informant interviews with targeted DHT members and local government district planners in the same areas.

In the readiness study, DHTs expressed commitment to institutionalize iCCM into the local health system through the development of district-specific iCCM activity work plans and budgets. The DHTs further suggested that they would implement district-led training, motivation, and supervision of community health workers; procurement of iCCM medicines and supplies; and advocacy activities for inclusion of iCCM indicators into the national health information systems. After iCCM policy adoption, follow-up study data findings showed that iCCM was largely not institutionalized into the local district health system functions. The poor institutionalization was attributed to lack of stewardship on how to transition from externally supported implementation to district-led programming, conflicting guidelines on community distribution of medicines, poor community-level accountability systems, and limited decision-making autonomy at the district level. Conclusion: Successful institutionalization of iCCM requires local ownership with increased coordination and cooperation among governmental and nongovernmental actors at both the national and district levels.
Noor AM. 2022. "Country ownership in global health." PLOS Glob Public Health 2(2): e0000113.
If you consider global health as something you do to help those ‘vulnerable others’, then you might find the lessons in this piece burdensome. The fact is global health is not charity, and you are its first client, the abiding lesson from the COVID-19 pandemic. Understanding this will ensure you practice global health with the most powerful tool you have–empathy. If your approach to supporting countries and communities in need is one that you would find unacceptable if circumstances were reversed, stop and recalibrate. Donor investment in global health, unfortunately, is often driven by geopolitics, as powerful countries and entities try to construct or retain spheres of influence. At times, the geopolitical objectives may conflict with your conscience. Speak up when you can, importantly however, do your best to behave conscientiously, as it is the collective, sustained behaviors of individuals that shape institutional culture.
Olsen B, Hannahan P, Arcia G. 2021. "How do government decisionmakers identify and adopt innovations for scale?" Brookings.
When it comes to supporting innovations at large scale, governments play a central role. But nonstate actors, such as researchers or project implementers, are also essential. Often, they’re the ones who design, pilot, and promote the innovations—hoping one day to hand the initiative over to the government for collaborative, long-term adoption. As evidence: In the Center for Universal Education’s global catalog of nearly 3,000 education innovations, two-thirds of them were started in the nonprofit sector, while only 12 percent originated in government. This means that nonstate education actors must become adept at presenting, proving, and pitching innovations to government. They need to learn how government decisionmakers identify and adopt innovations for scale, and the more the implementers and researchers know about the decisionmaking process, the more effective they can be.
Olsen B. 2021. "Minding the gap: The disconnect between government bureaucracies and cultures of innovation in scaling." Brookings.
Many contemporary practitioners and researchers tasked with bringing proven education innovations to scale around the world know that scaling is less a technical activity, but a mindset as much as an implementation process. As an adaptive mindset, scaling shares myriad characteristics with its close cousin: innovation. Both are complex and demand creative thinking, their outcomes are never fully predictable, and both require flexibility and engagement with the “what-ifs?” of life. And, yet, to be supported at scale by government, most education innovations first must be adopted by public-sector decisionmakers—a group that lives within a decidedly bureaucratic culture. The contradiction between the government mechanics of adopting innovations and the culture of implementing them becomes a central barrier to education innovations being adopted at scale.
Sieleunou I, Turcotte-Tremblay AM, Yumo HA, Kouokam E, Fotso J-CT, Tamga DM, Ridde V. 2017. "Transferring the Purchasing Role from International to National Organizations During the Scale-Up Phase of Performance-Based Financing in Cameroon." Health Systems & Reform, 3(2):91–104, DOI: 10.1080/23288604.2017.1291218
The World Bank and the government of Cameroon launched a performance-based financing (PBF) program in Cameroon in 2011. To ensure its rapid implementation, the performance purchasing role was sub-contracted to a consultancy firm and a nongovernmental organization, both international. However, since the early stage, it was agreed upon that this role would later be transferred to a national entity. This explanatory case study aims at analyzing the process of this transfer using Dolowitz and Marsh’s framework. We performed a document review and interviews with various stakeholders (n D 33) and then conducted thematic analysis of interview recordings. Sustainability, ownership, and integration of the PBF intervention into the health system emerged as the main reasons for the transfer. The different aspects of transfer from international entities to a national body consisted of (1) the decision-making power, (2) the “soft” elements (e.g., ideas, expertise), and (3) the “hard” elements (e.g., computers, vehicles). Factors facilitating the transfer included the fact that it was planned from the start and the modification of the legal status of the national organization that became responsible for strategic purchasing. Other factors hindered the transfer, such as the lack of a legal act clarifying the conditions of the transfer and the lack of post-transition support agreements. The Cameroonian experience suggests that key components of a successful transfer of PBF functions from international to national organizations may include clear guidelines, co-ownership and planning of the transition by all parties, and post-transition support to new actors.
Spicer N, Berhanu D, Bhattacharya D, Tilley-Gyado RD, Gautham M, Schellenberg J, Tamire-Woldemariam A, Umar N, Wickremasinghe D. 2016. "The stars seem aligned’: a qualitative study to understand the effects of context on scale-up of maternal and newborn health innovations in Ethiopia, India and Nigeria." Globalization and Health volume 12, Article number: 75 (2016).
Donors commonly fund innovative interventions to improve health in the hope that governments of low and middle-income countries will scale-up those that are shown to be effective. Yet innovations can be slow to be adopted by country governments and implemented at scale. Our study explores this problem by identifying key contextual factors influencing scale-up of maternal and newborn health innovations in three low-income settings: Ethiopia, the six states of northeast Nigeria and Uttar Pradesh state in India. Methods We conducted 150 semi-structured interviews in 2012/13 with stakeholders from government, development partner agencies, externally funded implementers including civil society organisations, academic institutions and professional associations to understand scale-up of innovations to improve the health of mothers and newborns these study settings. We analysed interview data with the aid of a common analytic framework to enable cross-country comparison, with Nvivo to code themes.

We found that multiple contextual factors enabled and undermined attempts to catalyse scale-up of donor-funded maternal and newborn health innovations. Factors influencing government decisions to accept innovations at scale included: how health policy decisions are made; prioritising and funding maternal and newborn health; and development partner harmonisation. Factors influencing the implementation of innovations at scale included: health systems capacity in the three settings; and security in northeast Nigeria. Contextual factors influencing beneficiary communities’ uptake of innovations at scale included: sociocultural contexts; and access to healthcare.

We conclude that context is critical: externally funded implementers need to assess and adapt for contexts if they are to successfully position an innovation for scale-up.
Spicer N, Hamza YA, Berhanu D, Gautham M, Schellenberg J, Tadesse F, Umar N, Wickremasinghe D. 2018. "The development sector is a graveyard of pilot projects!’ Six critical actions for externally funded implementers to foster scale-up of maternal and newborn health innovations in low and middle-income countries." Globalization and Health volume 14, Article number: 74 (2018).
Donors often fund projects that develop innovative practices in low and middle-income countries, hoping recipient governments will adopt and scale them within existing systems and programmes. Such innovations frequently end when project funding ends, limiting longer term potential in countries with weak health systems and pressing health needs. This paper aims to identify critical actions for externally funded project implementers to enable scale-up of maternal and newborn child health innovations originally funded by the Bill & Melinda Gates Foundation (‘the foundation’), or influenced by innovations that were originally funded by the foundation in three low-income settings: Ethiopia, the state of Uttar Pradesh in India and northeast Nigeria. We define scale-up as the adoption of donor-funded innovations beyond their original project settings and time periods. Methods We conducted 71 in-depth, semi-structured interviews with representatives from government, donors and other development partner agencies, donor-funded implementers including frontline providers, research organisations and professional associations. We explored three case study maternal and newborn innovations. Selection criteria were: a) innovations originally funded by the Bill & Melinda Gates Foundation (‘the foundation’), or influenced by innovations that were originally funded by the foundation; b) innovations for which a decision to scale-up had been made, allowing us to reflect on the factors influencing those decisions; c) innovations with increased geographical reach, benefitting a greater number of people, beyond districts where foundation-funded implementers were active. Our data were analysed based on a common analytic framework to aid cross-country comparisons.

Based on study respondents’ accounts, we identified six critical steps that donor-funded implementers had taken to enable the adoption of maternal and newborn health innovations at scale: designing innovations for scale; generating evidence to influence and inform scale-up; harnessing the support of powerful individuals; being prepared for scale-up and responsive to change; ensuring continuity by being part of the transition to scale; and embracing the aid effectiveness principles of country ownership, alignment and harmonisation.

Six critical actions identified in this study were associated with adopting and scaling maternal and newborn health innovations. However, scale-up is unpredictable and depends on factors outside implementers’ control.
Spring Impact. 2020. "Government Insights on the Journey to Sustained Impact at Scale."
Sustaining impact through government is a critical pathway to scale for many social impact organizations. But what do government partners advise on how to do so successfully? Governments bring networks and infrastructure to reach more people, authority over public spending and an understanding of their population’s needs and values. As such, many social impact organizations believe the government is best placed to scale and sustain the impact of solutions over the long-term, particularly in low-resource settings. However, effectively and sustainably transitioning solution ownership to government is complex, particularly when navigating the challenges of shifting resource-constrained settings. There is recognition that to do so successfully requires strong partnership between all the stakeholders involved – government, social impact organizations and funders – but these different organisations do not always have the same priorities, and government perspectives can be diluted or lost. Through Spring Impact’s work with VillageReach – an international non-profit that transforms health care delivery to reach everyone – we have been developing a Learning Network to bolster knowledge around how to successfully embed solutions into government systems. Recognising that government voices are insufficiently heard, we have a particular goal to ensure government input into building this knowledge. We recently hosted a webinar – in partnership with VillageReach and Devex – which shared insights from voices across governments in Cameroon, Ethiopia and Uganda, on how to partner effectively with government to scale social impact.
Teshome SB, Hoebink P. 2018. "Aid, ownership, and coordination in the health sector in Ethiopia." Development Studies Research, 5:sup1, S40-S55, DOI: 10.1080/21665095.2018.1543549
The Government of Ethiopia is seen as a owner of its national programs and policies and thus also as a strong coordinator of the foreign aid it receives. This is also the case in the health sector in Ethiopia, where the Ministry of Health have shown leadership in the last two decades. National health plans have been clear-cut and had ambitious objectives, to which the international donor community has adhered. The government-led coordination structures and joint health financing arrangements have been instrumental for improved donor coordination and aid effectiveness in the sector. This has led to impressive results, looking at the poor state of health that the government inherited from former regimes. However, the sector has at once been heavily dependent on foreign sources and characterized by high aid fragmentation. In this paper, we describe the health plans and health financing between 1990 and 2015. We also look at health leadership, donor coordination, and the results of investments in health. KEYWORDS: Health aid donor coordination aid effectiveness Ethiopia
Vogus A, Graff K. 2015. "PEPFAR Transitions to Country Ownership: Review of Past Donor Transitions and Application of Lessons Learned to the Eastern Caribbean." Global Health: Science and Practice June 2015, 3(2):274-286.
The US President’s Emergency Plan for AIDS Relief (PEPFAR) has shifted from an emergency response to a sustainable, country-owned response. The process of transition to country ownership is already underway in the Eastern Caribbean; the Office of the US Global AIDS Coordinator (OGAC) has advised the region that PEPFAR funding is being redirected away from the Eastern Caribbean toward Caribbean countries with high disease burden to strengthen services for key populations. This article seeks to highlight and apply lessons learned from other donor transitions to support a successful transition of HIV programs in the Eastern Caribbean. Based on a rapid review of both peer-reviewed and gray literature on donor transitions to country ownership in family planning, HIV, and other areas, we identified 48 resources that addressed key steps in the transition process and determinants of readiness for transition. Analysis of the existing literature revealed 6 steps that could help ensure successful transition, including developing a clear roadmap articulated through high-level diplomacy; investing in extensive stakeholder engagement; and supporting monitoring and evaluation during and after the transition to adjust course as needed. Nine specific areas to assess a country’s readiness for transition include: leadership and management capacity, political and economic factors, the policy environment, identification of alternative funding sources, integration of HIV programs into the wider health system, the institutionalization of processes, the strength of procurement and supply chain management, identification of staffing and training needs, and engagement of civil society and the private sector. In the Caribbean, key areas requiring strengthening to ensure countries in the region can maintain the gains made under PEPFAR include further engaging civil society and the private sector, building the capacity of NGOs to take on essential program functions, and maintaining donor support for targeted capacity building and long-term monitoring and evaluation efforts.
Waiswa P, Wanduru P, Okuga M, Kajjo D, Kwesiga D, Kalungi J, Nambuya H, Mulowooza J, Tagoola A, Peterson S. 2021. "Institutionalizing a Regional Model for Improving Quality of Newborn Care at Birth Across Hospitals in Eastern Uganda: A 4-Year Story" Global Health: Science and Practice.
Despite the rapid increase in facility deliveries in Uganda, the number of adverse birth outcomes (e.g., neonatal and maternal deaths) has remained high. We aimed to codesign and co-implement a locally designed package of interventions to improve the quality of care in hospitals in the Busoga region.

Design and Implementation:
This project was designed and implemented in 3 phases in the 6 main hospitals in east-central Uganda from 2013 to 2016. First, the inception phase engaged health system managers to codesign the intervention. Second, the implementation phase involved training health providers, strengthening the data information system, and providing catalytic equipment and medicines to establish newborn care units (NCUs) within the existing infrastructure. Third, the hospital collaborative phase focused on clinical mentorship, maternal and perinatal death reviews (MPDRs), and collaborative learning sessions.

In all 6 participating hospitals, we achieved institutionalization of NCUs in maternity units by establishing kangaroo mother care areas, resuscitation corners, and routine MPDRs. These improvements were associated with reduced maternal and neonatal deaths. Facilitators of success included a simple, low-cost, and integrated package designed with local health managers; the emergence of local neonatal care champions; implementation and support over a reasonably long period; decentralization of newborn care services; and use of mainly existing local resources (e.g., physical space, human resources, and commodities). Barriers to success related to limited hospital resources, unstable electricity, and limited participation from doctors. More advanced NCUs have been established in 3 of the 6 hospitals, and 7 high-volume comprehensive health centers have been established with functional NCUs.

The involvement of local health workers and leaders was the foundation for designing, sustaining, and scaling up feasible interventions by harnessing available resources. These findings are relevant for the quality of care improvement efforts in Uganda and other resource-restrained settings.
Waiswa P. 2020. "Institutionalization of Projects Into Districts in Low- and Middle-Income Countries Needs Stewardship, Autonomy, and Resources." Global Health: Science and Practice 8(2): 144–146.
Driven by high morbidity and mortality, weak health systems, weak governance, and poverty, many countries in sub-Saharan Africa have a multitude of projects led by government, nongovernmental organizations, and researchers trying to fill gaps. Unfortunately, although the rhetoric is usually to “institutionalize” the project, many of these projects often fail in what others call “pilotitis”1 —a situation in which projects are first piloted but not sustained or scaled up. This is a practice that many governments claim that they are tired of, but business as usual still continues. In this way, Uganda is no exception. As a result, despite proliferation of “high-impact” projects, child mortality remains high in Uganda, and the country is unlikely to achieve the Sustainable Development Goals related to child health. A recent United Nations report estimated that, despite marked progress, 74,000 children die every year in Uganda—the majority from preventable diseases.2 One of the main strategies that has the potential to make significant improvements to child health in sub-Saharan Africa is a strong district-led integrated community case management (iCCM) implementation.
Watson-Grant S, Xiong K, Thomas JC. 2016. "Country Ownership in International Development: Toward a Working Definition." MEASURE Evaluation.
Country ownership in development aid assumes that with recipient countries’ interest and participation, “owning” aid-funded initiatives will lead to more successful outcomes. But there is no universally accepted definition of country ownership. We conducted a systematic literature review to identify aspects of country ownership mentioned in peer-reviewed and gray literature, and identified four themes: 1) power and legitimacy; 2) commitment and responsibility 3) capacity; and 4) accountability. We also analyzed and compared similarities and differences in how different documents define these dimensions, with the results providing a framework for measuring country ownership of development initiatives. Measurement approaches must recognize that factors within and among these dimensions interact with one another in complex ways. Use of measures to identify steps toward more ownership entails conversations among stakeholders about necessary change.
Worsham E, Langsam K, Martin E. 2018. "Leveraging Government Partnerships for Scaled Impact." Scaling Pathways.
Social enterprises driving toward systems change must undertake various partnerships along the way. The one partnership most reported as being key to the ability to achieve such change is one with government. So thousands of social enterprises embark, every day, on this journey—filled with opportunities and challenges now documented in a newly released Scaling Pathways study.

“Business innovation produces some kinds of transformation well, and government policy innovation does others. Each has limits. But many imperatives sit in the space between the two modes.”

– Roger Martin and Sally Osberg in Getting Beyond Better

So how, within this space, can social enterprises and government partner together to dramatically scale impact? CASE explored this question with leading social ventures from the Skoll Foundation and USAID portfolios in its newest contribution to the Scaling Pathways theme study series, Leveraging Government Partnerships for Scaled Impact.


Apanga PA, Freeman MC, Sakas Z, Garn JV. 2022. "Assessing the Sustainability of an Integrated Rural Sanitation and Hygiene Approach: A Repeated Cross-Sectional Evaluation in 10 Countries." Glob Health Sci Pract. 2022;10(4): e2100564.
While many studies have implemented programs to increase sanitation coverage throughout the world, there are limited rigorous studies on the sustainability of these sanitation programs.

Between 2014 and 2018, the rural Sustainable Sanitation and Hygiene for All (SSH4A) approach was implemented by SNV in sub-Saharan Africa and Asia. Repeated cross-sectional household surveys were administered annually throughout program implementation and 1 to 2 years following completion of program activities. We characterize to what extent sanitation coverage was sustained 1 to 2 years after implementation of this SSH4A intervention.

Surveys were conducted in 12 program areas in 10 countries, with 22,666 households receiving a post-implementation survey. Six of 12 program areas (Bhutan, Ghana, Kenya, both Nepal sites, Tanzania) had similar coverage levels of basic sanitation 1–2 years post-implementation, whereas there were varying levels of slippage in the other program areas (both Ethiopia sites, Indonesia, Mozambique, Uganda, Zambia), ranging from a drop of 63 percentage points in coverage in Ethiopia to a drop of only 4 percentage points in Indonesia. In countries that experienced losses in the coverage of household sanitation, sanitation sharing among neighbors generally did not increase, whereas open defecation did increase. In each of the areas where slippage occurred, the sanitation coverage levels at the final time point were all still higher than the initial time point before SNV started working in these areas. We found several factors to be associated with the sustainability of sanitation coverage, including household socioeconomic status, having household members with disabilities, baseline sanitation coverage levels of the program areas, and rate of change of coverage during program activities.

Data revealed sustained gains in sanitation coverage in some program areas, yet slippage in other areas. This work may serve to benchmark the sustainability of sanitation interventions in sub-Saharan Africa and Asia.
Bandali S, Style S, Thiam L, Omar OA, Sabino A , Hukin E. 2021. "Pathways of change for achieving sustainability results: A tool to facilitate adaptive programming." Global Public Health, DOI: 10.1080/17441692.2020.1868016
Traditional approaches to development programming with fixed targets and outcomes do not fit complex problems where the pathway to achieve results differs in each context and evolves constantly. Adaptive programming improves responses to complex problems by identifying which solutions bring change. This paper reviews the theory behind adaptive programming approaches and introduces the ‘Pathways of Change’ tool for achieving sustainability results, developed for the multi-country Women’s Integrated Sexual Health programme. Qualitative data, using semi-structured interviews and group discussions from teams in over 17 countries in Africa and South Asia, are presented which examine the application of the Pathways of Change (PoC) tool focusing on successes and challenges across different intervention areas. The PoC responds to the need for a more practical adaptive programming tool that can be tailored to support flexibility in global health programme implementation while meeting donor requirements. Findings suggest that the PoC tool provides a flexible yet robust alternative to traditional monitoring frameworks and is able to facilitate adaptive, contextualised planning and monitoring for multi-country programmes. The PoC tool offers a solution to realise the ambitions of implementing adaptive programming within global health programmes and potentially beyond. KEYWORDS: Adaptive programming monitoring tools global health measurement
Bossert TJ. 1990. "Can they get along without us? Sustainability of donor-supported health projects in Central America and Africa." Social Science & Medicine. Volume 30, Issue 9, 1990, Pages 1015-1023.
This article presents a synthesis of five country studies of the sustainability of U.S. government-funded health projects in Central America and Africa. The studies reviewed health projects with a comparative framework to determine which project activities had continued after the donor funding ceased. This review found that health projects in Africa were less firmly sustained than those in Central America. The studies then evaluated context factors and project characteristics that were related to the sustainability of the projects. The weak economic and political context of the African cases was found to inhibit sustainability in those countries, suggesting that broader development issues be addressed before donors expect significant sustainability of health projects in Africa. Even in Central America it was found that the strength of the institution implementing the project was an important variable for sustainability, suggesting that donor attention also be shifted toward strengthening institutional development in order to assure sustainability. In addition to context factors, several project characteristics were related to sustainability in most cases and suggest sustainability guidelines for project design and implementation. The article concludes that projects should be designed and managed so as to: (1) demonstrate effectiveness in reaching clearly defined goals and objectives; (2) integrate their activities fully into established administrative structures; (3) gain significant levels of funding from national sources (budgetary and cost-recovery) during the life of the project; (4) negotiate project design with a mutually respectful process of give and take; and (5) include a strong training component. sustainability Central America Africa donor assistance
Brinkerhoff DW, Goldsmith AA. 1992. "Promoting the sustainability of development institutions: A framework for strategy." World Development. Volume 20, Issue 3, March 1992, Pages 369-383.
This article presents a generic framework for understanding institutional sustainability in development. Its lessons draw from the agriculture and health sectors. The framework treats institutions as: (a) systems that function in relationship to their environments; (b) organized and managed entities whose organizational structures and procedures must match the tasks, products, people, resources, and contexts they deal with; and (c) settings intimately concerned with the exchange of resources where economic and political relationships intertwine to create varying patterns of power and incentive. Application of the analytic framework has shown that institutional sustainability depends upon maintaining: responsive output flows (high quality and valued goods and services); cost-effective goods and services delivery mechanisms (organization and management); and resource flows (recurrent costs, capital investments, human resources).
Chambers DA, Glasgow RE, Stange KC. 2013. "The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change." Implementation Science. Volume 8, Article number: 117.
Despite growth in implementation research, limited scientific attention has focused on understanding and improving sustainability of health interventions. Models of sustainability have been evolving to reflect challenges in the fit between intervention and context.

We examine the development of concepts of sustainability, and respond to two frequent assumptions —'voltage drop,’ whereby interventions are expected to yield lower benefits as they move from efficacy to effectiveness to implementation and sustainability, and 'program drift,’ whereby deviation from manualized protocols is assumed to decrease benefit. We posit that these assumptions limit opportunities to improve care, and instead argue for understanding the changing context of healthcare to continuously refine and improve interventions as they are sustained. Sustainability has evolved from being considered as the endgame of a translational research process to a suggested 'adaptation phase’ that integrates and institutionalizes interventions within local organizational and cultural contexts. These recent approaches locate sustainability in the implementation phase of knowledge transfer, but still do not address intervention improvement as a central theme. We propose a Dynamic Sustainability Framework that involves: continued learning and problem solving, ongoing adaptation of interventions with a primary focus on fit between interventions and multi-level contexts, and expectations for ongoing improvement as opposed to diminishing outcomes over time.

A Dynamic Sustainability Framework provides a foundation for research, policy and practice that supports development and testing of falsifiable hypotheses and continued learning to advance the implementation, transportability and impact of health services research.
Eleanor Crook Foundation. 2018. "Grantee Guidance Series: Theory of Sustainability And Theory Of Scale."
Despite many global efforts, malnutrition continues to persist worldwide. Malnutrition contributes to nearly one half of all deaths of children under five years of age (CU5) in developing countries.1,2 There is a need for those working in global nutrition to make coordinated investments in improving and testing large-scale and sustainable interventions if we are to reverse this trend and achieve Sustainable Development Goal (SDG) 2 (to end hunger, achieve food security and improved nutrition and promote sustainable agriculture)3. This will require testing ideas that are not only innovative, but also simple enough to scale in a variety of challenging contexts and that are cost-effective enough to be sustained beyond the life cycle of a single grant. Too many innovative ideas never leave the pilot phase because, while effective in isolated testing, they would be impossibly resource-intensive to implement on a large scale over time. The Eleanor Crook Foundation (ECF) wants to ensure its’ grantees are incorporating considerations of scalability and sustainability into their study designs so that potentially impactful ideas can move beyond the pilot phase and elicit large-scale reductions in malnutrition.
Eriksson L, Bergström A, Hoa DTP, Nga NT, Eldh AC. 2017. "Sustainability of knowledge implementation in a low- and middle- income context: Experiences from a facilitation project in Vietnam targeting maternal and neonatal health." PLoS ONE 12(8): e0182626.
In a previous trial in Vietnam, a facilitation strategy to secure evidence-based practice in primary care resulted in reduced neonatal mortality over a period of three years. While little is known as to what ensures sustainability in the implementation of community-based strategies, the aim of this study was to investigate factors promoting or hindering implementation, and sustainability of knowledge implementation strategies, by means of the former Neonatal Knowledge Into Practice (NeoKIP) trial.

In 2014 we targeted all levels in the Vietnamese healthcare system: six individual interviews with representatives at national, provincial and district levels, and six focus group discussions with representatives at the commune level. The interviews were transcribed verbatim, translated to English, and analysed using inductive and deductive thematic analysis.

To achieve successful implementation and sustained effect of community-based knowledge implementation strategies, engagement of leaders and key stakeholders at all levels of the healthcare system is vital–prior to, during and after a project. Implementation and sustainability require thorough needs assessment, tailoring of the intervention, and consideration of how to attain and manage funds. The NeoKIP trial was characterised by a high degree of engagement at the primary healthcare system level. Further, three years post trial, maternal and neonatal care was still high on the agenda for healthcare workers and leaders, even though primary aspects such as stakeholder engagement at all levels, and funding had been incomplete or lacking.

The current study illustrates factors to support successful implementation and sustain effects of community-based strategies in projects in low- and middle-income settings; some but not all factors were represented during the post-NeoKIP era. Most importantly, trials in this and similar contexts require deliberate management throughout and beyond the project lifetime, and engagement of key stakeholders, in order to promote and sustain knowledge implementation.
Gardner A, Greenblott K, Joubert E. 2005. "What We Know About Exit Strategies: Practical Guidance For Developing Exit Strategies in the Field." Report from Consortium for Southern Africa Food Security Emergency.
This document is intended to reflect on C-SAFE\'s collective experience with exit strategies, improve our understanding, and provide guidance to NGO staff for developing effective exit strategies in the field. The goal of this guidance document is to improve our collective understanding and ability to develop and implement sound exit strategies from developmental relief programs by: looking at some key concepts and terminology related to Exit Strategies (borrowed primarily from the FANTA technical note on Exit Strategies, November 2004); discussesing the challenges associated with Exit Strategies; incorporating special consideration of the southern Africa context - which includes a high prevalence of HIV/AIDS, recurrent droughts, unstable political environments, increasing poverty, and typically an environment of funding constraints; providing step-by-step guidance on how to develop, implement, and monitor sound Exit Strategies for Title II developmental relief programs; and reflecting on the experiences of C-SAFE member agencies in the region. Along with the case studies and examples, this document will assist C-SAFE NGO members in improving their understanding of exit strategies.
Geels FW. 2011. "The multi-level perspective on sustainability transitions: Responses to seven criticisms." Environmental Innovation and Societal Transitions. Volume 1, Issue 1, June 2011, Pages 24-40.
The multi-level perspective (MLP) has emerged as a fruitful middle-range framework for analysing socio-technical transitions to sustainability. The MLP also received constructive criticisms. This paper summarises seven criticisms, formulates responses to them, and translates these into suggestions for future research. The criticisms relate to: (1) lack of agency, (2) operationalization of regimes, (3) bias towards bottom-up change models, (4) epistemology and explanatory style, (5) methodology, (6) socio-technical landscape as residual category, and (7) flat ontologies versus hierarchical levels. Transitions Sustainable development Multi-level perspective Response to critics
Gruen RL, Elliott JH, Nolan ML, Lawton PD, Parkhill A, McLaren CJ, Lavis JN. 2008. "Sustainability science: an integrated approach for health-programme planning." Public Health. Volume 372, ISSUE 9649, P1579-1589, November 01, 2008.
Planning for programme sustainability is a key contributor to health and development, especially in low-income and middle-income countries. A consensus evidence-based operational framework would facilitate policy and research advances in understanding, measuring, and improving programme sustainability. We did a systematic review of both conceptual frameworks and empirical studies about health-programme sustainability. On the basis of the review, we propose that sustainable health programmes are regarded as complex systems that encompass programmes, health problems targeted by programmes, and programmes\' drivers or key stakeholders, all of which interact dynamically within any given context. We show the usefulness of this approach with case studies drawn from the authors\' experience.
Hardee K. 2017. "Guidance on assessing the potential sustainability of Practices as part of an evidence review: Considerations for High Impact Practices in Family Planning." Washington, DC: USAID.
The Need to Focus on Sustainability The Family Planning High Impact Practice (HIP) Initiative is focused on synthesizing evidence and learning on “what works” in family planning. The HIP organizes practices into three broad categories: service delivery, social and behavior change communication, and enabling environment (see website for more detail). The main role of the HIP Technical Advisory Group (TAG) is to offer an unbiased review of evidence on a specific practice in order to assess that practice’s potential to significantly improve family planning programs. The assessment of the evidence is based on a pre-determined set of criteria, such as impact on modern contraceptive use and sustainability (see HIP list for complete list of criteria). The TAG recognizes that the term, “sustainability,” is ill-defined and, to that end, requested a small group of TAG members to provide more specific guidance on how the HIP TAG should consider sustainability when determining if a practice meets the criteria to be labeled a High Impact Practice. Clearly, sustainability is a key concern among decision makers when deciding whether to invest in any specific HIP. The authors considered this issue by asking three key questions: 1) How is sustainability defined? 2) What evidence is required to demonstrate a practice is sustainable? 3) How can HIPs be implemented to increase the potential for sustainability for as long as the practice is relevant for the program? Evolving Definitions of Sustainability Over time, the definitions of sustainability have changed. In the 1990s, the term was used to refer to the ability of country family planning programs1-4 and non-governmental organizations (NGOs)5-7 to maintain gains after donors phased out. In the 1990s and 2000s, questions of sustainability focused on moving from pilot projects to scale up of and integration into existing programs or standardized practice. This includes the geographic expansion of programs, referred to as “horizontal scale-up,” and the inclusion of key implementation inputs into existing systems—such as training, tracking, and policies—referred to as “vertical scale-up” (see Figure 1). 8 The importance of “starting with the end in mind,” contributed to the development of approaches and tools.9-15 More recently, systems frameworks and tools have been developed for promoting “sustainable development,” defined by the World Commission on Environment and Development (WCED) as “development that meets the needs of the present without compromising the ability of future generations to meet their own needs.” 16 Sustainable development has taken on increased salience in the face of climate change issues and the growing importance of building resilient systems. 2, 17-20 Appendix 1 provides a table showing the evolution of definitions of sustainability.
Hartmann A, Kharas H, Kohl R, Linn JF, Massler B, Sourang C. 2013. "Scaling up Programs for the Rural Poor: IFAD's Experience, Lessons, and Prospects (Phase 2)." Brookings Global Economy and Development Working Paper 54.
The challenge of rural poverty and food insecurity in the developing world remains daunting. Recent estimates show that “there are still about 1.2 billion extremely poor people in the world. In addition, about 870 million people are undernourished, and about 2 billion people suffer from micronutrient deficiency. About 70 percent of the world’s poor live in rural areas, and many have some dependency on agriculture,” (Cleaver 2012). Addressing this challenge by assisting rural small-holder farmers in developing countries is the mandate of the International Fund for Agricultural Development (IFAD), an international financial institution based in Rome. The International Fund for Agricultural Development is a relatively small donor in the global aid architecture, accounting for approximately one-half of 1 percent of all aid paid directly to developing countries in 2010. Although more significant in its core area of agricultural and rural development, IFAD still accounts for less than 5 percent of total official development assistance in that sector.1 Confronted with the gap between its small size and the large scale of the problem it has been mandated to address, IFAD seeks ways to increase its impact for every dollar it invests in agriculture and rural development on behalf of its member states. One indicator of this intention to scale up is that it has set a goal to reach 90 million rural poor between 2012 and 2015 and lift 80 million out of poverty during that time. These numbers are roughly three times the number of poor IFAD has reached previously during a similar time span. More generally, IFAD has declared that scaling up is “mission critical,” and this scaling-up objective is now firmly embedded in its corporate strategy and planning statements. Also, increasingly, IFAD’s operational practices are geared towards helping its clients achieve scaling up on the ground with the support of its loans and grants. This was not always the case. For many years, IFAD stressed innovation as the key to success, giving little attention to systematically replicating and building on successful innovations. In this regard, IFAD was not alone. In fact, few aid agencies have systematically pursued the scaling up of successful projects. However, in 2009, IFAD management decided to explore how it could increase its focus on scaling up. It gave a grant to the Brookings Institution to review IFAD’s experience with scaling up and to assess its operational strategies, policies and processes with a view to strengthening its approach to scaling up. Based on an extensive review of IFAD documentation, two country case studies and intensive interactions with IFAD staff and managers, the Brookings team prepared a report that it submitted to IFAD management in June 2010 and published as a Brookings Global Working Paper in early 2011 (Linn et al. 2011).
Healey J, Conlon CM, Malama K, Hobson R, Kaharuza F, Kekitiinwa A, Levitt M, Zulu DW, Marum L. Saving Mothers, Giving Life Working Group. 2019. "Sustainability and scale of the Saving Mothers, Giving Life approach in Uganda and Zambia." Glob Health Sci Pract. 2019;7(suppl 1):S188-206.
Saving Mothers, Giving Life (SMGL) significantly reduced maternal and perinatal mortality in Uganda and Zambia by using a district health systems strengthening approach to address the key delays women and newborns face in receiving quality, timely, and appropriate medical care. This article documents the transition of SMGL from pilot to scale in Uganda and Zambia and analyzes the sustainability of the approach, examining the likelihood of maintaining positive trends in maternal and newborn health in both countries.

We analyzed the potential sustainment of SMGL achievements using a tool adapted from the HIV-focused domains and elements of the U.S. President's Emergency Plan for AIDS Relief Sustainability Index and Dashboard for maternal and neonatal health pro-gramming adding a domain on community normative change. Information for each of the 5 resulting domains was drawn from SMGL and non-SMGL reports, individual stakeholder interviews, and group discussions.

In both Uganda and Zambia, the SMGL proof-of-concept phase catalyzed commitment to saving mothers and newborns and a renewed belief that significant change is possible. Increased leadership and accountability for maternal and newborn health, particularly at the district and facility levels, was bolstered by routine maternal death surveillance reviews that engaged a wide range of local leadership. The SMGL district-strengthening model was found to be cost-effective with cost of death averted estimated at US$177-206 per year of life gained. When further considering the ripple effect that saving a mother has on child survival and the household economy, the value of SMGL increases. Ministries of health and donor agencies have already demonstrated a willingness to pay this amount per year of life for other programs, such as HIV and AIDS.

As SMGL scaled up in both Uganda and Zambia, the intentional integration of SMGL interventions into host country systems, alignment with other large-scale programs, and planned reductions in annual SMGL funding all contributed to increasing host government ownership of the interventions and set the SMGL approach on a path more likely to be sustained following the close of the initiative. Lessons from the learning districts resulted in increased efficiency in allocation of resources for maternal and newborn health, better use of strategic information, improved management capacities, and increased community engagement.
Iwelunmor J, Blackstone S, Veira D, Nwaozuru U, Airhihenbuwa C, Munodawafa D, Kalipeni E, Jutal A, Shelley D, Ogedegbe G. 2016. "Toward the sustainability of health interventions implemented in sub-Saharan Africa: a systematic review and conceptual framework." Implement Sci. 2016; 11: 43.
Sub-Saharan Africa (SSA) is facing a double burden of disease with a rising prevalence of non-communicable diseases (NCDs) while the burden of communicable diseases (CDs) remains high. Despite these challenges, there remains a significant need to understand how or under what conditions health interventions implemented in sub-Saharan Africa are sustained. The purpose of this study was to conduct a systematic review of empirical literature to explore how health interventions implemented in SSA are sustained.

We searched MEDLINE, Biological Abstracts, CINAHL, Embase, PsycInfo, SCIELO, Web of Science, and Google Scholar for available research investigating the sustainability of health interventions implemented in sub-Saharan Africa. We also used narrative synthesis to examine factors whether positive or negative that may influence the sustainability of health interventions in the region.

The search identified 1819 citations, and following removal of duplicates and our inclusion/exclusion criteria, only 41 papers were eligible for inclusion in the review. Twenty-six countries were represented in this review, with Kenya and Nigeria having the most representation of available studies examining sustainability. Study dates ranged from 1996 to 2015. Of note, majority of these studies (30 %) were published in 2014. The most common framework utilized was the sustainability framework, which was discussed in four of the studies. Nineteen out of 41 studies (46 %) reported sustainability outcomes focused on communicable diseases, with HIV and AIDS represented in majority of the studies, followed by malaria. Only 21 out of 41 studies had clear definitions of sustainability. Community ownership and mobilization were recognized by many of the reviewed studies as crucial facilitators for intervention sustainability, both early on and after intervention implementation, while social and ecological conditions as well as societal upheavals were barriers that influenced the sustainment of interventions in sub-Saharan Africa.

The sustainability of health interventions implemented in sub-Saharan Africa is inevitable given the double burden of diseases, health care worker shortage, weak health systems, and limited resources. We propose a conceptual framework that draws attention to sustainability as a core component of the overall life cycle of interventions implemented in the region.
Johnson K, Hays C, Center H, Daley C. 2004. "Building capacity and sustainable prevention innovations: a sustainability planning model." Evaluation and Program Planning. Volume 27, Issue 2, May 2004, Pages 135-149.
This article presents an informed definition of sustainability and an associated planning model for sustaining innovations (pertinent to both infrastructure and interventions) within organizational, community, and state systems. The planning model stems from a systematic review of the literature and from concepts derived from a series of ‘think tanks’ made up of key substance abuse prevention professionals. The model assumes a five-step process (i.e. assessment, development, implementation, evaluation, and reassessment/modification) and addresses factors known to inhibit efforts to sustain an innovation. One set of factors concerns the capacity of prevention systems to support sustainable innovations. The other pertains to the extent to which a particular innovation is sustainable. A sustainability action strategy is presented that includes goals with corresponding sets of objectives, actions, and results that determine the extent of readiness to sustain an innovation. Sustainability tools to assist in implementing the planning model are illustrated, and next steps for the model are discussed. This planning model provides a conceptual and practical understanding of sustainability that can lead to further investigation. Sustainability Planning Evaluation Innovation Prevention
Kutzin J, Sparkes S, Soucat A, Barroy H. 2018. "From silos to sustainability: transition through a UHC lens." Correspondence. Volume 392, Issue 10157, P1513-1514.
The transition to higher-income status is a positive step forward for countries, but this transition brings with it the prospect of declining external assistance, both in general and in particular for health. Most health donor agencies rely at least in part on an income threshold to establish eligibility for support. Such a donor transition implies that government is increasingly responsible for the financing of a health programme and its supported interventions.1 However, focusing attention only on replacing external assistance with domestic revenues for the programmes concerned is problematic in two ways: firstly, this approach limits the sustainability question to revenues, and secondly, it limits the scope for action to the specific health programme that was receiving external support. The commitment countries have made to universal health coverage (UHC) is an opportunity to reframe the transition agenda towards sustaining coverage results rather than externally funded programmes per se. This perspective has implications for the overall approach to transition taken at both national and global levels. UHC embeds the goals of equity in service use, quality, and financial protection at the level of the entire health system and population.2 The way external resources are often channelled, as a legacy of the Millennium Development Goals era, creates or reinforces vertical structures focused on specific diseases or interventions. In many countries, these subsystems operate independently of the rest of the health system, with separate plans, budgets, funding, procurement, supply chains, and information systems.3 When viewed through a UHC lens (ie, across the health system, within which programmes are embedded), it is apparent that these separate subsystems duplicate responsibilities, compromising efficiency in resource use and sometimes effective case management—eg, when service use data on a pregnant woman who has HIV is managed separately by the HIV programme and the maternal health programme.4 Consolidating underlying subsystems can help sustain progress. External assistance should support such real system-building actions. Purely financial solutions to the challenges posed by donor transition, such as blended financing arrangements, should not mask the need to address these efficiency challenges that are at the core of putting national health systems on more sustainable trajectories. There is no need to wait for transition; the time to initiate such change is now.
Lambin E, Kim H, Leape J, Lee K. 2020. "Scaling up solutions for a sustainability transition." One Earth 3 89-96.
The main challenge for a sustainability transition is to scale up successful solutions. Upscaling requires coalitions of public, private, and civil society actors who align their motivations. Pathways to upscaling may involve leveraging a dominant player’s market power, integrating successful initiatives into public policy, or reinforcing government-led change with private efforts. Various actors agree to collaborate to take advantage of their complementary capabilities, e.g., government policies facilitate private action, market incentives reward progressive actors while government sanctions punish laggards, actors take up different tasks of the policy cycle, and large players absorb and disseminate pioneer efforts. To achieve durable impacts, the upscaling of solutions to reach sustainability must continually maintain a balance of incentives among key actors. We identify general lessons for successful upscaling that provide insights on the importance of motivating actors, designing collaborations for lasting success, and incorporating concerns of developing countries.
Lee H. 2017. "Sustainability in International Aid Programs; Identification of Working Concepts of Sustainability and Its Contribution Factors." International Journal of Social Science. Studies. Vol. 5, No. 1; January 2017. ISSN 2324-8033 E-ISSN 2324-8041.
An international aid program is a process that takes complex strategic planning with persistent collective efforts. Although dependent on a program’s nature, many international aid programs seek sustained benefits and services as their eventual goal after a donor funding ceases. Accordingly, sustainability of international aid programs has been much discussed for aid effectiveness and efficiency. Despite its well-recognized importance, and due to its complexity, the term ‘sustainability’ has yet to be clearly conceptualized across even similar programs. Additionally, there seems to be a lack of consensus on what common factors may contribute to sustainability. The current study reviewed 16 select papers of international aid programs related to health, food or rural development, and identified commonly utilized working concepts of sustainability and its key factors. The most cited concept of sustainability in the select papers was sustained delivery of program services and outcomes. For its contributing factors, eleven factors were identified in the order of frequency: capacity building (16), political commitment (10), continuous funding resource (8), community participation (6), linkage or connectedness (5), acceptance in socio-culture (5), program effectiveness (5), institutionalization (3), transition of responsibility (3), negotiation (3) and communication (3). The findings can help plan more sustainable programs in relevant fields. Key words: sustainability, factors, sustained benefit, continued impact, aid program, international development
Lee K, van Nassau F, Grunseit A, Conte K, Milat A, Wolfenden L, Bauman A. 2020. "Scaling up population health interventions from decision to sustainability – a window of opportunity? A qualitative view from policy-makers." Health Res Policy Sys 18, 118.
While known efficacious preventive health interventions exist, the current capacity to scale up these interventions is limited. In recent years, much attention has focussed on developing frameworks and methods for scale-up yet, in practice, the pathway for scale-up is seldom linear and may be highly dependent on contextual circumstances. Few studies have examined the process of scaling up from decision to implementation nor examined the sustainability of scaled-up interventions. This study explores decision-makers’ perceptions from real-world scaled-up case studies to examine how scale-up decisions were made and describe enablers of successful scale-up and sustainability.

This qualitative study included 29 interviews conducted with purposively sampled key Australian policy-makers, practitioners and researchers experienced in scale-up. Semi-structured interview questions obtained information regarding case studies of scaled-up interventions. The Framework Analysis method was used as the primary method of analysis of the interview data to inductively generate common and divergent themes within qualitative data across cases.

A total of 31 case studies of public health interventions were described by interview respondents based on their experiences. According to the interviewees’ perceptions, decisions to scale up commonly occurred either opportunistically, when funding became available, or when a deliberate decision was made and funding allocated. The latter scenario was more common when the intervention aligned with specific political or strategic goals. Decisions to scale up were driven by a variety of key actors such as politicians, senior policy-makers and practitioners in the health system. Drivers of a successful scale-up process included good governance, clear leadership, and adequate resourcing and expertise. Establishing accountability structures and appropriate engagement mechanisms to encourage the uptake of interventions were also key enablers. Sustainability was influenced by evidence of impact as well as good acceptability among the general or target population.

Much like Kingdon’s Multiple Streams Theory of ‘policy windows’, there is a conceptually similar ‘window for scale-up’, driven by a complex interplay of factors such as political need, strategic context, funding and key actors. Researchers and policy-makers need to consider scalability from the outset and prepare for when the window for scale-up opens. Decision-makers need to provide longer term funding for scale-up to facilitate longer term sustainability and build on the resources already invested for the scale-up process.
Moore JE, Mascarenhas A, Bain J, Straus SE. 2017. "Developing a comprehensive definition of sustainability." Implementation Science volume 12, Article number: 110 (2017).
Understanding sustainability is one of the significant implementation science challenges. One of the big challenges in researching sustainability is the lack of consistent definitions in the literature. Most implementation studies do not present a definition of sustainability, even when assessing sustainability. The aim of the current study was to systematically develop a comprehensive definition of sustainability based on definitions already used in the literature.

We searched for knowledge syntheses of sustainability and abstracted sustainability definitions from the articles identified through any relevant systematic and scoping reviews. The constructs in the abstracted sustainability definitions were mapped to an existing definition. The comprehensive definition of sustainability was revised to include emerging constructs.

We identified four knowledge syntheses of sustainability, which identified 209 original articles. Of the 209 articles, 24 (11.5%) included a definition of sustainability. These definitions were mapped to three constructs from an existing definition, and nine new constructs emerged. We reviewed all constructs and created a revised definition: (1) after a defined period of time, (2) a program, clinical intervention, and/or implementation strategies continue to be delivered and/or (3) individual behavior change (i.e., clinician, patient) is maintained; (4) the program and individual behavior change may evolve or adapt while (5) continuing to produce benefits for individuals/systems. All 24 definitions were remapped to the comprehensive definition (percent agreement among three coders was 94%). Of the 24 definitions, 17 described the continued delivery of a program (70.8%), 17 mentioned continued outcomes (70.8%), 13 mentioned time (54.2%), 8 addressed the individual maintenance of a behavior change (33.3%), and 6 described the evolution or adaptation (25.0%).

We drew from over 200 studies to identify 24 existing definitions of sustainability. Based on these definitions, we identified five key sustainability constructs, which can be used as the basis for future research on sustainability. Our next step is to identify sustainability frameworks and develop a meta-framework using a concept mapping approach to consolidate the factors and considerations across sustainability frameworks.
Moucheraud C, Schwitters A, Boudreaux C, Giles D, Kilmarx PH, Ntolo N, Bangani Z, St. Louis ME, Bossert TJ. 2017. "Sustainability of health information systems: a three-country qualitative study in southern Africa." BMC Health Services Research volume 17, Article number: 23.
Health information systems are central to strong health systems. They assist with patient and program management, quality improvement, disease surveillance, and strategic use of information. Many donors have worked to improve health information systems, particularly by supporting the introduction of electronic health information systems (EHIS), which are considered more responsive and more efficient than older, paper-based systems. As many donor-driven programs are increasing their focus on country ownership, sustainability of these investments is a key concern. This analysis explores the potential sustainability of EHIS investments in Malawi, Zambia and Zimbabwe, originally supported by the United States President’s Emergency Plan for AIDS Relief (PEPFAR).

Using a framework based on sustainability theories from the health systems literature, this analysis employs a qualitative case study methodology to highlight factors that may increase the likelihood that donor-supported initiatives will continue after the original support is modified or ends. Results Findings highlight commonalities around possible determinants of sustainability. The study found that there is great optimism about the potential for EHIS, but the perceived risks may result in hesitancy to transition completely and parallel use of paper-based systems. Full stakeholder engagement is likely to be crucial for sustainability, as well as integration with other activities within the health system and those funded by development partners. The literature suggests that a sustainable system has clearly-defined goals around which stakeholders can rally, but this has not been achieved in the systems studied. The study also found that technical resource constraints – affecting system usage, maintenance, upgrades and repairs – may limit EHIS sustainability even if these other pillars were addressed.

The sustainability of EHIS faces many challenges, which could be addressed through systems’ technical design, stakeholder coordination, and the building of organizational capacity to maintain and enhance such systems. All of this requires time and attention, but is likely to enhance long-term outcomes.
Pluye P, Potvin L, Denis JL. 2004. "Making public health programs last: conceptualizing sustainability." Evaluation and Program Planning. Volume 27, Issue 2, May 2004, Pages 121-133.
In public health, programs constitute an important method of improving health, and program sustainability is critical. Knowledge on sustainability raises nevertheless two major issues. The first concerns the social structures within which programs are sustained. The literature suggests different structures however only organizational structures, namely routines, are used for analysis. The second issue concerns the temporal aspect of sustainability that is typically conceived as the final phase of program development after the planning, implementation, and evaluation phases. This ‘stage’ model does not allow one to consider that sustainability must be prepared in advance, concomitantly with implementation. These structural and temporal dimensions ground our proposal to re-conceive sustainability. The literature on organizations defines two relevant social structures, one organizational (routines), and one institutional (standards). This in turn suggests three degrees of sustainability. We then emphasize how sustainability is concomitant with the implementation process, by exploring events that characterize these processes. Health promotion programs Implementation Institutionalization Organizational learning Routinization Sustainability
Proctor E, Luke D, Calhoun A, McMillen C, Brownson R, McCrary S, Padek M. 2015. "Sustainability of evidence-based healthcare: research agenda, methodological advances, and infrastructure support." Implementation Science.
Little is known about how well or under what conditions health innovations are sustained and their gains maintained once they are put into practice. Implementation science typically focuses on uptake by early adopters of one healthcare innovation at a time. The later-stage challenges of scaling up and sustaining evidence-supported interventions receive too little attention. This project identifies the challenges associated with sustainability research and generates recommendations for accelerating and strengthening this work.

A multi-method, multi-stage approach, was used: (1) identifying and recruiting experts in sustainability as participants, (2) conducting research on sustainability using concept mapping, (3) action planning during an intensive working conference of sustainability experts to expand the concept mapping quantitative results, and (4) consolidating results into a set of recommendations for research, methodological advances, and infrastructure building to advance understanding of sustainability. Participants comprised researchers, funders, and leaders in health, mental health, and public health with shared interest in the sustainability of evidence-based health care.

Prompted to identify important issues for sustainability research, participants generated 91 distinct statements, for which a concept mapping process produced 11 conceptually distinct clusters. During the conference, participants built upon the concept mapping clusters to generate recommendations for sustainability research. The recommendations fell into three domains: (1) pursue high priority research questions as a unified agenda on sustainability; (2) advance methods for sustainability research; (3) advance infrastructure to support sustainability research.

Implementation science needs to pursue later-stage translation research questions required for population impact. Priorities include conceptual consistency and operational clarity for measuring sustainability, developing evidence about the value of sustaining interventions over time, identifying correlates of sustainability along with strategies for sustaining evidence-supported interventions, advancing the theoretical base and research designs for sustainability research, and advancing the workforce capacity, research culture, and funding mechanisms for this important work.
Sarriot E, Morrow M, Langston A, Weiss J, Landegger J, Tsuma L. 2015. "A causal loop analysis of the sustainability of integrated community case management in Rwanda." Social Science & Medicine. Volume 131, April 2015, Pages 147-155.
Expansion of community health services in Rwanda has come with the national scale up of integrated Community Case Management (iCCM) of malaria, pneumonia and diarrhea. We used a sustainability assessment framework as part of a large-scale project evaluation to identify factors affecting iCCM sustainability (2011). We then (2012) used causal-loop analysis to identify systems determinants of iCCM sustainability from a national systems perspective. This allows us to develop three high-probability future scenarios putting the achievements of community health at risk, and to recommend mitigating strategies. Our causal loop diagram highlights both balancing and reinforcing loops of cause and effect in the national iCCM system. Financial, political and technical scenarios carry high probability for threatening the sustainability through: (1) reduction in performance-based financing resources, (2) political shocks and erosion of political commitment for community health, and (3) insufficient progress in resolving district health systems--“building blocks”--performance gaps. In a complex health system, the consequences of choices may be delayed and hard to predict precisely. Causal loop analysis and scenario mapping make explicit complex cause-and-effects relationships and high probability risks, which need to be anticipated and mitigated. Sustainability Causal loop analysis Community case management Evaluation Health systems Rwanda Systems thinking Community health
Sarriot E, Ricca J, Yourkavitch J, Ryan L, Sustained Health Outcomes (SHOUT) Group. 2008. "Taking the Long View: A Practical Guide to Sustainability Planning and Measurement in Community-Oriented Health Programming." Calverton, MD: Macro International Inc.
This manual is designed to assist project managers, planners, and evaluators in their efforts to improve their approaches to planning for and assessing sustainability in health projects implemented in developing countries. It is intended as a practical guide for health project managers, especially those implementing community health projects in resource-constrained settings. It focuses on a specific framework, the Sustainability Framework (SF), developed through the U.S. Agency for International Development’s (USAID) Child Survival and Health Grants Program (CSHGP). This manual represents the collective learning of about 30 projects that have applied and helped refine it over a 7-year period, many of them CSHGP-funded projects. It has been used for project planning, monitoring, end-of-project evaluation, and post-project evaluation.
Sarriot EG, Kouletio M, Jahan S, Rasul I, Musha AKM. 2014. "Advancing the application of systems thinking in health: sustainability evaluation as learning and sense-making in a complex urban health system in Northern Bangladesh." Health Research Policy and Systems. Volume 12, Article number: 45.
Starting in 1999, Concern Worldwide Inc. (Concern) worked with two Bangladeshi municipal health departments to support delivery of maternal and child health preventive services. A mid-term evaluation identified sustainability challenges. Concern relied on systems thinking implicitly to re-prioritize sustainability, but stakeholders also required a method, an explicit set of processes, to guide their decisions and choices during and after the project.

Concern chose the Sustainability Framework method to generate creative thinking from stakeholders, create a common vision, and monitor progress. The Framework is based on participatory and iterative steps: defining (mapping) the local system and articulating a long-term vision, describing scenarios for achieving the vision, defining the elements of the model, and selecting corresponding indicators, setting and executing an assessment plan,, and repeated stakeholder engagement in analysis and decisions . Formal assessments took place up to 5 years post-project (2009).

Strategic choices for the project were guided by articulating a collective vision for sustainable health, mapping the system of actors required to effect and sustain change, and defining different components of analysis. Municipal authorities oriented health teams toward equity-oriented service delivery efforts, strengthening of the functionality of Ward Health Committees, resource leveraging between municipalities and the Ministry of Health, and mitigation of contextual risks. Regular reference to a vision (and set of metrics (population health, organizational and community capacity) mitigated political factors. Key structures and processes were maintained following elections and political changes. Post-project achievements included the maintenance or improvement 5 years post-project (2009) in 9 of the 11 health indicator gains realized during the project (1999–2004). Some elements of performance and capacity weakened, but reductions in the equity gap achieved during the project were largely maintained post-project.

Sustainability is dynamic and results from local systems processes, which can be strengthened through both implicit and explicit systems thinking steps applied with constancy of purpose.
Scheirer MA, Dearing JW. 2011. "An Agenda for Research on the Sustainability of Public Health Programs." Am J Public Health. 2011 November; 101(11): 2059–2067.
Funders of programs in public health and community health are increasingly concerned about the sustainability of changes they initiate. Despite a recent increase in sustainability research and evaluation, this literature has not developed a widely used paradigm for conducting research that can accumulate into generalizable findings. We provide guidance for research and evaluation of health program sustainability, including definitions and types of sustainability, specifications and measurements of dependent variables, definitions of independent variables or factors that influence sustainability, and suggestions for designs for research and data collection. We suggest viewing sustainability research as a further stage in the translation or dissemination of research-based interventions into practice. This perspective emphasizes ongoing relationships with earlier stages of a broader diffusion framework, including adoption and implementation processes.
Scheirer MA, Hartling G, Hagerman D. 2008. "Defining sustainability outcomes of health programs: Illustrations from an on-line survey." Evaluation and Program Planning. Volume 31, Issue 4, November 2008, Pages 335-346.
Evaluative research for questions of program sustainability has expanded substantially in recent years, but definitional issues remain. This paper presents definitions for four different types of sustainability as potential outcomes of health programs. We then illustrate these definitions with descriptive findings from an on-line survey to “look back” at the extent and types of sustainability that occurred among 48 community-based projects that had received short-term funding from a foundation-funded health program in New Jersey. We found that large percentages of respondents reported positively to each of four types of sustainability measures—maintaining program activities, continuing to serve substantial numbers of clients, building and sustaining collaborative structures, and maintaining attention to the ideas underlying the projects by disseminating them to others. Strengths and limitations of this methodology for future evaluation are also discussed. Evaluating sustainability Health programs On-line survey Organizational capacity Dissemination
Shediac-Rizkallah MC, Bone LR. 1998. "Planning for the sustainability of community-based health programs: conceptual frameworks and future directions for research, practice and policy." Health Education Research, Volume 13, Issue 1, March 1998, Pages 87–108.
Attention to the sustainability of health intervention programs both in the US and abroad is increasing, but little consensus exists on the conceptual and operational definitions of sustainability. Moreover, an empirical knowledge base about the determinants of sustainability is still at an early stage. Planning for sustainability requires, first, a clear understanding of the concept of sustainability and operational indicators that may be used in monitoring sustainability over time. Important categories of indicators include: (1) maintenance of health benefits achieved through an initial program, (2) level of institutionalization of a program within an organization and (3) measures of capacity building in the recipient community. Second, planning for sustainability requires the use of programmatic approaches and strategies that favor long-term program maintenance. We suggest that the potential influences on sustainability may derive from three major groups of factors: (1) project design and implementation factors, (2) factors within the organizational setting, and (3) factors in the broader community environment. Future efforts to develop sustainable health intervention programs in communities can build on the concepts and strategies proposed here.
Shelton RC, Cooper BR, Stirman SW. 2018. "The Sustainability of Evidence-Based Interventions and Practices in Public Health and Health Care." Annual Reviews of Public Health. 2018 Apr 1;39:55-76. doi: 10.1146/annurev-publhealth-040617-014731
There is strong interest in implementation science to address the gap between research and practice in public health. Research on the sustainability of evidence-based interventions has been growing rapidly. Sustainability has been defined as the continued use of program components at sufficient intensity for the sustained achievement of desirable program goals and population outcomes. This understudied area has been identified as one of the most significant translational research problems. Adding to this challenge is uncertainty regarding the extent to which intervention adaptation and evolution are necessary to address the needs of populations that differ from those in which interventions were originally tested or implemented. This review critically examines and discusses conceptual and methodological issues in studying sustainability, summarizes the multilevel factors that have been found to influence the sustainability of interventions in a range of public health and health care settings, and highlights key areas for future research. adaptation, evidence-based interventions, implementation science, maintenance, sustainability
Stirman SW, Kimberly J, Cook N, Calloway A, Castro F, Charns M. 2012. "The sustainability of new programs and innovations: a review of the empirical literature and recommendations for future research." Implementation Science. Volume 7, Article number: 17.
The introduction of evidence-based programs and practices into healthcare settings has been the subject of an increasing amount of research in recent years. While a number of studies have examined initial implementation efforts, less research has been conducted to determine what happens beyond that point. There is increasing recognition that the extent to which new programs are sustained is influenced by many different factors and that more needs to be known about just what these factors are and how they interact. To understand the current state of the research literature on sustainability, our team took stock of what is currently known in this area and identified areas in which further research would be particularly helpful. This paper reviews the methods that have been used, the types of outcomes that have been measured and reported, findings from studies that reported long-term implementation outcomes, and factors that have been identified as potential influences on the sustained use of new practices, programs, or interventions. We conclude with recommendations and considerations for future research.

Two coders identified 125 studies on sustainability that met eligibility criteria. An initial coding scheme was developed based on constructs identified in previous literature on implementation. Additional codes were generated deductively. Related constructs among factors were identified by consensus and collapsed under the general categories. Studies that described the extent to which programs or innovations were sustained were also categorized and summarized.

Although "sustainability" was the term most commonly used in the literature to refer to what happened after initial implementation, not all the studies that were reviewed actually presented working definitions of the term. Most study designs were retrospective and naturalistic. Approximately half of the studies relied on self-reports to assess sustainability or elements that influence sustainability. Approximately half employed quantitative methodologies, and the remainder employed qualitative or mixed methodologies. Few studies that investigated sustainability outcomes employed rigorous methods of evaluation (e.g., objective evaluation, judgement of implementation quality or fidelity). Among those that did, a small number reported full sustainment or high fidelity. Very little research has examined the extent, nature, or impact of adaptations to the interventions or programs once implemented. Influences on sustainability included organizational context, capacity, processes, and factors related to the new program or practice themselves.

Clearer definitions and research that is guided by the conceptual literature on sustainability are critical to the development of the research in the area. Further efforts to characterize the phenomenon and the factors that influence it will enhance the quality of future research. Careful consideration must also be given to interactions among influences at multiple levels, as well as issues such as fidelity, modification, and changes in implementation over time. While prospective and experimental designs are needed, there is also an important role for qualitative research in efforts to understand the phenomenon, refine hypotheses, and develop strategies to promote sustainment.
The Challenge Initiative (TCI). 2020. "Sustainable Provision of Quality Family Planning Services in Poor, Urban Areas." The Bill and Melinda Gates Foundation.
Scale, impact, cost-efficiency and sustainability are the four interlocking tenets that characterize The Challenge Initiative (TCI). At the heart of TCI’s philosophy is the belief that scaling global health interventions without impact is empty scale; that impact at scale without simultaneously increasing cost-efficiencies is not viable; and that cost-efficient impact at scale that is not sustained will not produce lasting change. TCI delivers on all four tenets, understanding that one without the other three is inadequate to achieve enduring progress.
Torpey K, Mwenda L, Thompson C, Wamuwi E, Van Damme W. 2010. "From project aid to sustainable HIV services: A case study from Zambia." Journal of the International AIDS Society.
Sustainable service delivery is a major challenge in the HIV response that is often not adequately addressed in project implementation. Sustainable strategies must be built into project design and implementation to enable HIV efforts to continue long after donor-supported projects are completed. This paper presents the experiences in operational sustainability of Family Health International\'s Zambia Prevention, Care and Treatment Partnership in Zambia, which is supported by the US President\'s Emergency Plan for AIDS Relief through United States Agency for International Development (October 2004 to September 2009). The partnership worked with Zambia\'s Ministry of Health to scale up HIV clinical services in five of the country\'s nine provinces, reaching 35 districts and 219 facilities. It provided technical and financial support from within the ministry\'s systems and structures. By completion of the project, 10 of the 35 districts had graduated beyond receiving ongoing technical support. By working within the ministry\'s policies, structures and systems, the partnership was able to increase the ministry\'s capacity to add a comprehensive HIV service delivery component to its health services. Ministry structures were impro