Evaluation Systems in Development Co-operation

Evaluation Systems in Development Co-operation

2016 Review You do not have access to this content

Click to Access: 
  • PDF
  • http://www.keepeek.com/Digital-Asset-Management/oecd/development/evaluation-systems-in-development-co-operation_9789264262065-en
  • READ
30 Sep 2016
9789264262065 (PDF) ;9789264262058(print)

Hide / Show Abstract

Evaluation is widely recognised as an important component for learning and improving development effectiveness. Evaluation responds to public and taxpayer demands for credible information and independent assessment of development co-operation activities. The Development Assistance Committee’s Network on Development Evaluation supports members in their efforts to strengthen and continuously improve evaluation systems.

The 2016 review of evaluation systems in development co-operation looks at the changes and trends in evaluation systems over the last five years. The report describes the role and management of evaluation in development agencies, ministries and multilateral banks. It provides information about the specific institutional settings, resources, policies and practices of DAC Evaluation Network members, and includes specific profiles on each member’s evaluation system. The study identifies major trends and current challenges in development evaluation. It covers issues such as human and financial resources, institutional setups and policies, independence of the evaluation function, reporting and use of evaluation findings, joint evaluation, and the involvement of partner countries in evaluation work.

This report is part of the DAC Network on Development Evaluation’s ongoing efforts to increase the effectiveness of development co-operation policies and programmes by promoting high-quality, independent evaluation.

loader image

Expand / Collapse Hide / Show all Abstracts Table of Contents

  • Mark Click to Access
  • Foreword and Acknowledgements

    Evaluation of development co-operation helps to meet public demands for accountability, and supports transparency by providing evidence about the effectiveness and impact of development policies and programmes. Evaluation also contributes to institutional learning and global knowledge, bringing to light key success factors and obstacles to effective development. High quality evaluation processes and products are important, but strong evaluation systems are needed to encourage the use of findings, ensure the integrity of the evaluation function, and make evaluative evidence readily available for learning. Changes in evaluation systems in recent years reflect evolutions in the development cooperation landscape, including the merger of ministries of international development, foreign affairs, and trade; and the establishment of new evaluation and development oversight bodies. Demand for evaluative evidence is likely to increase with the implementation of the Sustainable Development Goals; in this context growing attention must be given to supporting evaluation capacity development in line with SDG follow-up and review processes.

  • Acronyms and abbreviations
  • Executive summary

    The purpose of this review is to provide information to assist development organisations in building effective evaluation systems and processes to deliver high quality, credible and useful evaluations. It provides information about evaluation systems in development agencies and multilateral organisations, and analyses experiences from managing evaluation systems across organisations. The study covers the evaluation systems of the 37 members of the DAC Network on Development Evaluation (EvalNet) and 9 multilateral organisations (six development banks, the European Commission, the IMF and the UNDP).

  • Add to Marked List
  • Expand / Collapse Hide / Show all Abstracts Evaluation systems in development co-operation

    • Mark Click to Access
    • Overview of the 2016 Review of Evaluation Systems in Development Co-operation

      The Development Assistance Committee (DAC) Network on Development Evaluation works to foster learning and to support accountability needs through robust, independent evaluation of development co-operation activities. The members and participants of the Network have evaluation units and systems with considerable variety in respect to structures, mandates, polices, and resources. This study looks at how evaluation units are resourced and managed, updating the 2010 study undertaken by the network. This chapter describes the background, motivation and scope of the study, including the methodology and data sources. Overall trends and conclusions offer an overview of the results and main findings.

    • Evaluation systems and governance

      This chapter explores the institutional arrangements and environments for evaluation across the DAC Network on Development Evaluation. The chapter highlights recent changes to evaluation systems since 2010, such as the creation of new evaluation entities and the integration of ministries of development co-operation with ministries of foreign affairs and trade. The chapter outlines the human and financial resources of evaluation units and examines evaluation policies as well as the independence of evaluation systems. The data and analysis is based on a survey of network members, document reviews, and interviews with evaluation units of DAC members and multilateral development banks.

    • Evaluation processes

      This chapter examines evaluation processes, including the approaches used to ensure the quality and rigor of evaluations, evaluation planning and programming, and how members ensure that evaluation results and findings are fed into future programming. The chapter discusses the use of supplementary evaluation tools such as evaluability assessments, ex-ante evaluations and self-evaluation. Capacity issues and constraints are addressed from both the organisational perspective and in relation to partners and stakeholders, as is participation and joint and collaborative evaluation work. Finally, the chapter discusses evaluation use and demand including dissemination, management response and follow up, and institutional knowledge management.

    • Add to Marked List
  • Expand / Collapse Hide / Show all Abstracts Profiles of members of the DAC Network on Development Evaluation

    • Mark Click to Access
    • Introduction and key for the member profiles

      While the core functions of all development evaluation units are similar, the management and institutional setups vary widely. To complement the broad picture, this section captures the diversity of approaches to evaluation in development institutions. An individual evaluation profile for each member of the DAC Network on Development Evaluation is provided.

    • African Development Bank Group (AfDB)
    • Asian Development Bank (ADB)

      The Independent Evaluation Department (IED) of the Asian Development Bank (ADB) is mandated to evaluate sovereign and sovereign-guaranteed operations (public sector operations) and non-sovereign operations supported by the Bank; and the policies and strategies, practices, and procedures that govern them. The objective of evaluation in ADB is to assess development effectiveness and the long-term results of (i) ADB operations; (ii) country partnership strategies; and (iii) ADB policies, plans, practices, and procedures.

    • Australia

      In late 2013 the Australian Government decided to integrate management of the aid programme into the Department of Foreign Affairs and Trade (DFAT) in order to better align Australia’s diplomatic, trade and development efforts and provide a stronger platform for an improved aid programme. This was followed in June 2014 with the announcement of a new aid policy and performance reporting system. Policies and procedures relating to aid evaluation remained relatively consistent throughout this period, with the exception that Office of Development Effectiveness (ODE) mandate was expanded to include a role in quality assuring the assessments made in DFAT’s annual Performance of Australian Aid report.

    • Austria

      The responsibility for evaluations of Austrian development assistance is shared between the Federal Ministry for European and International Affairs (FMEIA) and the Austrian Development Agency (ADA). The ADA is the operations unit for Austrian Development Cooperation (ADC) and includes a separate Evaluation Unit under the Executive Unit for Evaluation and Statistics. Overall, Austria distinguishes between internal and external evaluations, where ‘internal’ refer to evaluations managed by project partners themselves and ‘external’ to evaluations managed by ADA and FMEIA.

    • Belgium

      The 2013 Law on Development Co-operation provides the mandate to the Special Evaluation Office (SEO) to undertake centralised evaluations. Decentralised evaluations are performed by the project or programme partners themselves, including the Belgian Development Agency (BTC).

    • Canada

      In July 2013, Canadian International Development Agency (CIDA) and the Department of Foreign Affairs and International Trade (DFAIT) amalgamated to create the Department of Foreign Affairs, Trade and Development (DFATD). In November 2015, the DFATD was renamed Global Affairs Canada.

    • Czech Republic

      The Czech Republic was the first Central European country to become an OECD member state in 1995. It was approved as the 26th OECD DAC member in 2013 following a review conducted by the OECD DAC in 2006.

    • Denmark

      Danish development co-operation is evaluated by the Evaluation Department (EVAL) located in the Ministry of Foreign Affairs (MFA). Its primary task is to plan, design and implement evaluations of development co-operation implemented by Danida. EVAL disseminates evaluation results and lessons internally and externally.

    • European Bank for Reconstruction and Development (EBRD)

      The EBRD has introduced significant changes to its evaluation function since 2010. With the appointment of the current Chief Evaluator, the emphasis was placed on ensuring the rigour of evaluations undertaken by the Bank. A new evaluation policy was introduced and approved in 2013, replacing the existing policy update from 2010.

    • European Commission (EC)

      The Evaluation Unit covers the evaluation of the development co-operation policy for the Directorate-General for International Cooperation and Development (DG DEVCO) and the European External Action Service (EEAS). The Evaluation Unit is in charge of the evaluation of the European Union’s (EU) co-operation and development programmes in third countries, with the exception of enlargement candidate countries, neighbourhood countries and humanitarian aid. It covers the following regions and their corresponding EC external co-operation instruments: Africa, Caribbean and Pacific Countries (ACP), Asia (including Central Asia) and Latin America.

    • European Investment Bank (EIB)

      The EIB Group consists of the European Investment Bank and the European Investment Fund. 2015 saw the 20th anniversary of the European Investment Bank’s (EIB) evaluation function Operations Evaluation (EV). EV’s activities have been extended to all areas of the EIB Group since 2005.

    • Finland

      The Ministry for Foreign Affairs (MFA) manages and co-ordinates most of the Finnish development co-operation programme. Under the MFA, the Department for Development Policy is responsible for providing overall guidance on the implementation, planning and monitoring of Finland’s development co-operation policy, and holds direct responsibility for the operational activities for multilateral and civil society organisations development co-operation as well as humanitarian aid. Regional departments are responsible for the implementation of bilateral co-operation. The institutional arrangements and responsibilities for evaluation are defined in the Decree on the Ministry for Foreign Affairs 550/2008(1280/2013) and the Evaluation Norm 1/2015. The Unit for Development Evaluation (EVA-11) is an independent administrative unit formally responsible for the development of the evaluation system, commissioning large-scale evaluations, and ensuring its effective use.

    • France
    • Germany

      At the request of the Federal Ministry for Economic Cooperation and Development (BMZ) an external evaluation of the German development evaluation system was carried out and published in 2009, which found the German system to be incoherent, fragmented and lacking sufficient capacity at the strategic level. This led in 2012 to the creation of a separate public entity, the German Institute for Development Evaluation (DEval), which is primarily tasked to perform strategically relevant and mostly complex evaluations for all German development activities.

    • Iceland

      As of 1t January 2016, following the adoption of an Act of Parliament amending the Icelandic Act on Development Cooperation (2008), all activities of the Icelandic International Development Agency (ICEIDA) were transferred to the Ministry for Foreign Affairs (MFA) and ICEIDA ceased operations. The purpose of the amendment to the Act was to simplify the organisational framework that supports the Icelandic Government’s development assistance in order to maximise results and ensure value for money.

    • Inter-American Development Bank (IADB)

      The Office of Evaluation and Oversight (OVE) is an independent body of the Inter- American Development Bank (IDB), responsible for externally evaluating IDB’s projects and performance.

    • International Monetary Fund (IMF)

      The Independent Evaluation Office (IEO) was established to conduct objective and independent evaluations on issues relevant to the mandate of the Fund. It complements the review and evaluation work within the Fund. It thus improves the Fund’s ability to draw lessons from its experience and more quickly integrate improvements into its future work.

    • Ireland

      Irish Aid is the Irish Government’s official aid programme and is a division of the Department of Foreign Affairs and Trade (the Department). The Irish Aid Evaluation Policy (2007) defines evaluation in the Irish Aid context as being the systematic and objective assessment of the design, implementation and results of an ongoing or completed project, programme or policy by assessing the effectiveness of the intervention against its stated objectives.

    • Italy

      In 2014, the new law on General Rules Governing International Development Cooperation came into force, aiming at systematically updating the Italian development cooperation system and effectively establishing a new management structure. This includes the creation of the Italian Agency for Development Cooperation under the Ministry of Foreign Affairs and International Cooperation.

    • Japan

      Korea’s development aid is provided through the Korean International Cooperation Agency (KOICA) and the Economic Development Cooperation Fund (EDCF). KOICA is responsible for bilateral grants and the EDCF for bilateral loans. Both agencies have their own evaluation unit.

    • Korea

      In 2015, a new overall Evaluation Policy was developed and launched by the Directorate for Development Cooperation and Humanitarian Action at the Ministry of Foreign and European Affairs (MFEA) in consultation with Luxembourg Development Cooperation Agency (LuxDev), the Ministry’s implementing agency for development co-operation. The Policy is an overarching framework for the entire evaluation system of Luxembourg’s development interventions, determining key objectives and priorities. The Policy sets out the distribution of evaluation tasks between the MFEA and LuxDev, with the MFEA handling external evaluations and LuxDev delivering internal, independent evaluations. Internal and external evaluations have different but complementary objectives which mean that there is close consultation between the two actors in the evaluation process (OECD 2012).

    • Luxembourg

      The Policy and Operations Evaluation Department (IOB) is an independent unit responsible for evaluation activities for the Ministry of Foreign Affairs (MFA). Given the mandate and independence of IOB, the directorates general remain responsible for the evaluation of foreign policy, foreign trade and development co-operation. In addition to evaluations led by IOB, departments and embassies commission decentralised evaluations.

    • Netherlands

      The last five years have seen a period of significant change for the organisation and management of New Zealand’s development co-operation. In 2009, the management of the New Zealand Aid Programme (the Aid Programme) was transferred from a semiautonomous unit (known as NZ Aid) and integrated into the Ministry of Foreign Affairs and Trade (MFAT).

    • New Zealand

      The Evaluation Department is the unit responsible for initiating and implementing independent evaluations of development co-operation activities financed under the Norwegian development co-operation budget, and communicating these results to the public and decision/policy makers. The Evaluation Department is located in the Norwegian Agency for Development Cooperation (Norad). The key objective of evaluations is to identify lessons learned in a systematic way, so that they can be used in policy development and serve as the basis for operations activities.

    • Norway

      Poland joined the OECD DAC as the 28th member in 2013. Since the Special Review performed by the DAC in 2010, Poland has made efforts to establish the legal and political foundations, identifying a clear focal point, and creating systems for monitoring and evaluation. The Development Cooperation Act was drafted in 2011 and the Multiannual Development Cooperation Programme 2012-2015 was prepared, which serves as the basis for the planning and implementation of the Polish development assistance including monitoring and evaluation activities.

    • Poland

      In 2012 Portugal underwent a major organisational reform. The Portuguese development agency, the Portuguese Institute for Development Support (IPAD), was merged with into Camões, the Institute for Cooperation and Language (Camões, I.P.). Since the merger, Camões, I.P. is responsible for both development co-operation and Portuguese language and cultural activities.

    • Portugal

      Slovakia first joined the donor community with its accession to the OECD and the European Union and became the third OECD member to join the DAC in 2013.

    • Slovakia

      Slovenia became a donor country in 2004 and has been putting the legal and strategic frameworks in place since then. The country adopted the International Development Cooperation of the Republic of Slovenia Act in 2006, which was followed by the Resolution on International Development Cooperation in 2008. The Resolution set out various goals in development co-operation for the country to achieve by 2015. The Resolution is to be updated in order to further guide Slovenian development co-operation. A Special Review of the Slovenian development co-operation programme and systems was conducted in 2012, after which the country was accepted as the 29th member of the OECD DAC. The Evaluation Policy and the Evaluation Guidelines were prepared in 2014 and 2015, and a peer review by other DAC members is envisaged after 2015.

    • Slovenia

      In 2012 the Ministry of Foreign Affairs and Cooperation (MAEC) was restructured. As a result, the Directorate-General for Development Policy Planning and Evaluation (DGPOLDE), to which the evaluation unit the Division of Development Policy Evaluation and Knowledge Management (DEGCO) was attached, was dissolved. The General Secretariat for International Development Cooperation (SGCID) assumed the mandates and competencies of the former DGPOLDE, including the evaluation function.

    • Spain

      The government bodies in charge of evaluation are Sida and the Expert Group for Aid Studies (EBA), which was established in 2013 to replace the earlier Swedish Agency for Development Evaluation (SADEV). Sida reports to the Government, Ministry of Foreign Affairs (MFA), Department for International Development Cooperation. So does EBA, albeit with a more independent mandate. In addition, the Agency for Public Management (Statskontoret) may at the request of the Swedish government carry out studies and evaluations of the management of International Development Cooperation. The National Audit Office, reporting to the Swedish Parliament audits governmental activities including development co-operation.

    • Sweden

      SECO’s evaluation system has since 2009 shifted from a strong accountability focus to one more on learning (OECD 2013). The Evaluation Policy drafted in the same year declares that evaluation serves for learning and accountability, but it also indicates that evaluation is not just about producing reports, but to contribute to SECO’s decision-making process and to foster continuous improvements. The Policy also defines the responsibilities and organisational arrangements guiding the evaluation function in the Department of Economic Cooperation and Development (WE).

    • Switzerland

      In 2011, DFID implemented a new model of decentralised programme evaluation and embedded it throughout the organisation. Placing responsibility for evaluation within programme areas emphasises the use of evaluation for improving the design and delivery of policies and programmes and hence development impact.

    • United Kingdom

      UNDP conducts evaluations within two different categories: independent centralised evaluations conducted by the Independent Evaluation Office (IEO) and decentralised evaluations commissioned by programme units, including country offices, regional bureaus, and practice and policy bureaus.

    • United Nations Development Programme (UNDP)

      Three US state agencies are tasked with evaluation responsibilities within development aid assistance; the US Department of State, the US Agency for International Development (USAID), and the US Millennium Challenge Corporation (US MCC).

    • United States

      The Independent Evaluation Group (IEG) is responsible for assessing the relevance, efficacy, and efficiency of the World Bank Group’s operational programmes and activities and their contributions to development effectiveness. IEG’s mandate is to carry out independent and objective evaluation of the strategies, policies, programmes, projects, and corporate activities of the World Bank Group, which includes the International Bank for Reconstruction and Development (IBRD), the International Development Association (IDA), the International Finance Corporation (IFC), and the Multilateral Investment Guarantee Agency (MIGA). Independent evaluation is undertaken to improve accountability and inform the formulation of new directions; policies and procedures; country, sector, and thematic strategies; lending operations; and technical co-operation.

    • World Bank Group (WBG)
    • Add to Marked List
Visit the OECD web site