by Aniket Bhushan and Rachael Calleja
Published: April 24, 2015
Below, we summarize key messages from our research paper, “Paying for Impact: Results-based Approaches in Development Finance, Situating Canadian Efforts in a Global Context”.
This paper is a primer on results-based approaches in development finance. It fills a gap in the availability of basic, accessible information on results-based approaches, and is centered on five main questions:
- What are the different types of results based approaches and how can they be classified?
- How do they measure in terms of scale?
- What are some of their strengths and weaknesses?
- What is Canada doing in this space and how does it compare with other donors?
- How do results-based approaches communicate results and impacts?
In addition, we conclude with recommendations aimed at three sets of stakeholders.
While large and growing in importance, data on results-based approaches is rarely consolidated. In many cases these initiatives combine public and private resources, and not all inputs qualify as “overseas development assistance”. As a result key information is not always available from conventional foreign aid data sources (be it the OECD-DAC or IATI open data). The paper is based on a desk-based quantitative analysis of 20 results based initiatives which include some of the largest and most well known in the space. Data and other information collected as part of the research is consolidated and made available through the Canadian International Development Platform (www.cidpnsi.ca). Special attention is paid to initiatives where Canada is active.
What are the different types of results-based approaches and how can they be classified?
- Results-based approaches can be defined as “models that aim to alter the incentive structure of aid allocation to link aid more directly to the achievement of quantifiable results, motivate behavioral change and or to catalyze innovation”. Results-based approaches in development finance can be classified into four main types: results-based aid, results-based financing, hybrids and challenge-linked financing.
How do they stack up in terms of scale, both financial size as well as impacts?
- Based on the sample of results-based initiatives analyzed, we estimate the total global marketplace to be between $24 billion and $47 billion. These figures should be interpreted with caution, they are only meant to provide a sense of scale. Results-based approaches are a small fraction of the total foreign aid and development financing marketplace. Most initiatives are recent, i.e. launched after the Millennium Development Goals (MDGs) and influenced by the same. Results-based approaches have been growing in importance and scale in recent years.
What are their strengths and weaknesses?
- The theory of change underpinning results-based approaches to delivering assistance outlines several strengths: for instance, linking payments to recipient performance helps bring greater attention to shared goals; forges new thinking on measurement and measurability; and can enhance country ownership and policy space. However, these approaches are not without challenges. Most efforts are slow to take off as the logic of results-targeting and incentivizing change takes more time to materialize than is often anticipated. Other issues such as the risk of gaming, impact on aid predictability, ownership and capacity, tend to be more challenging than proponents argue. In general, despite significant recent experience, results-based approaches remain largely untested in terms of their efficacy and the evidence based on discernable ‘incentive effects’ is far from conclusive.
What is Canada doing in this space and how does it compare with other donors?
- The mainstay of Canada’s efforts in this space are in the health sector. Canada’s contributions are largely through multilateral initiatives, including the Global Fund and GAVI – Canada ranks among the top 10 donors in both. However, in recent years Canada has played a leadership role in experimenting with instruments such as ‘pull mechanism’ in the agriculture sector through AgResults, and challenge-linked financing such as through the SME Challenge and Trust Fund. The signature Canadian initiative in this space however is Grand Challenges Canada (GCC). Relative to donors like the US and UK, Canada is a relatively small player in the results-based space.
How do results-based approaches communicate results and impacts?
- We found significant variation in the level and type of results, outputs, outcomes and impacts data and information that is made available across the 20 initiatives analyzed. Three initiatives stand out in terms of good practice: The Global Fund, the Millennium Challenge Corporation (MCC) and the UK’s Results-based Aid pilot projects (or Cash-on-Delivery). What stands out about these and other examples of good practice is that quantitative performance data are combined with qualitative narrative information, but presented in a systematic, machine-readable and relatively predictable manner, with appropriate identifiers and contextual information such that they can be combined with other data.
We conclude with recommendations aimed at three sets of stakeholders: public and public-private initiatives in Canada, civil society stakeholders advocating for greater innovation, and the open data, transparency and accountability community.
- Canada could use a white paper or strategy document on its approach to results targeting in development finance, much like other major bilateral donors in the space (DFID, 2014). Such a tool could help guide future efforts, which, from recent pronouncements, are expected to play an increasingly important role in Canadian aid going forward (Economic Action Plan, 2015).
- Specifically, Canada’s signature initiative, GCC, should make more effort to standardize and consolidate information on its initiatives and their results and impacts. This could be done through a modernized data portal, or by publishing to international open data standards.
- Civil society stakeholders advocating for greater innovation and adoption of results-based approaches can and should do more to follow through, and not merely chase ideas for their novelty factor. A concrete example of follow through could be tapping their partners and networks in developing countries to collect data through citizen engagement and direct feedback, which could serve as a valuable validation exercise but also help drive new and more compelling narratives around the impacts of innovative aid modalities.
- Results-based models should be of particular interest to the open data, transparency and accountability community, as expectations surrounding openly available information regarding these efforts are high. The open data community is at an important inflection point. The amount of open development data in the public domain has grown dramatically. However, its usage has lagged. One of the key reasons for this is that qualitative and quantitative results information and data are not being captured adequately through open data standards. The open data community should provide further guidance on how data standards could be leveraged to track, aggregate, communicate and better link results data.
Figure 1. Summary Data on Results-Based initiatives