6. Japan’s results, evaluation and learning

Japan’s projects are thoroughly planned and evaluated following the “Plan, Do, Check, Action” cycle drawn from Japanese manufacturing. Ex-ante evaluations are required for all ODA loan projects and outline the rationale for the project and a description of target outcomes, such as improved living standards for a regional infrastructure project or reduced pollutants in channels for a sewerage system project.1 The evaluation looks at expected outputs, external risk factors and lessons learned from similar past projects.

The broader development objectives are not established. Japan has logframes and outcome targets for projects, but it does not set out in measurable ways the broader development results it seeks to achieve or contribute to through a clearly articulated chain of expected results from inputs and outputs to outcomes and impact.2 JICA is renowned for its bottom-up approach that leads directly to tangible outcomes at a local level and in which all stakeholders participate. The link between policy, its implementation and the results on the ground could be established through strengthened dialogue between the bottom-up approach and institutional or system-wide reform in partner countries (Yoshida, 2018[1]). This is all the more important given the size of Japanese investments as the fourth largest DAC donor overall.

A more strategic approach to results is needed. While Country Development Cooperation Policies, or country strategies, build on partner countries’ development plan and the SDGs, they do not include indicators and are not used to define the results that Japan wishes to achieve or how it contributes to meeting the SDGs. As a result, third-party country assistance evaluations find it difficult to assess the actual impact, broad relevance or specific contribution of Japan’s development co-operation since these results or collective efforts with the partner country and stakeholders are not set out at the beginning. An overall results framework for each country could help bring together all of Japan’s investments and identify synergies among the various investments and instruments. Project and programme indicators are drawn from partner country national data sources where possible and align to national strategies (Chapter 5). However, they could more explicitly demonstrate how investments link to the SDGs or contribute to partner countries’ own development outcomes.

Statistical support to partners and joint quality assurance monitoring are good practice. Japan supports the statistical capacity of partner countries through in-kind technical co-operation and earmarked funding provided through the Association of Southeast Asian Nations (ASEAN), the United Nations Economic and Social Commission for Asia and the Pacific, and the World Bank to national statistical agencies and regional statistics institutes.3 Every six months, and sometimes more frequently depending on the stage of project implementation, JICA, together with local and national authorities, monitors progress for quality assurance and issues a report of social and environmental conditions, which helps to avoid parallel monitoring efforts.4 This participatory approach, including by piloting the problem-driven iterative adaptation in conflict-affected areas, is good practice.

Monitoring and results are used for accountability to track project and output delivery and to verify internal controls, as well as to inform new phases or future projects. Where possible, on-the-spot monitoring allows for the adjustment of some parameters, which is more easily done for loan projects. There is an opportunity for JICA to look beyond the immediate delivery of outputs of its own projects to consider how these are part of a broader outcome.

Building a culture of results and learning, backed by appropriate incentives, is a challenge for Japan as for other DAC members. Japan could draw on work by the OECD’s Development Co-operation Directorate to develop tools and guidance for implementing the Guiding Principles on Managing for Sustainable Development Results. This work could help Japan to learn from other DAC members’ successes and failures in establishing a coherent system-wide approach to results that is linked to the objectives it wants to achieve. A shift towards managing for results could alleviate the need for systematic evaluations in order to adopt a more strategic approach to evaluation, as recommended in the 2014 Peer Review (OECD, 2014[2]).

Evaluations are planned annually and the project cost is the determining factor for whether or not to conduct an external evaluation for JICA. The Ministry of Foreign Affairs (MOFA) has an annual evaluation plan drafted by the Evaluation Division and approved by the Management (OECD, 2016[3]). JICA does not have an evaluation plan, but it publishes requests for proposals for external evaluations via its procurement portal on an annual basis. MOFA conducts its own policy evaluations but hires third parties for ODA evaluations of country or regional assistance, themes, or aid modalities. External evaluators are commissioned for all JICA projects costing over USD 9 million (JPY 1 billion). JICA carries out internal ex-post evaluations for all projects costing over USD 1.8 million (JPY 200 million), and less than USD 9 million, Ex-post evaluations are budgeted separately from the initial project cost. The regional or sector department is responsible for appropriating the budget for ex-ante evaluations as part of the initial project cost and the evaluation department appropriates the budget for ex-post evaluations. In fiscal year 2017, there were 86 external ex-post evaluations of JICA projects (JICA, 2019[4]).

Japan evaluates investments from a diplomatic perspective. Japan assesses its investments relative to national interest, in line with the 2015 Development Cooperation Charter. These evaluations are guided by standardised questions.5 For example, the 2017 Country Assistance Evaluation of Cambodia highlights the strong economic co-operation between the two countries, and suggests that Japan should consider that there may be differences between the needs of the general Cambodian population and those identified or prioritised by the Cambodian Government (Waseda University, 2018[5]). This is a good example of how Japan incorporates the ‘coherence’ criteria recently added to the DAC evaluation criteria.

Japan has made some progress since the 2014 Peer Review. The last peer review suggested Japan might adopt a more strategic approach to the projects that are evaluated by using criteria such as risk or innovation (Annex A), and progress is being made in this regard. While ex-post evaluations are still standard practice for every project costing over USD 1.8 million, midterm evaluations are now no longer obligatory for JICA in order to allow flexibility in project monitoring. JICA has also adopted a pragmatic approach to conducting ex-ante and ex-post evaluations of multiple projects with the same purpose. For example, ex-post evaluations can integrate ODA loans and technical co-operation projects or grant aid and technical co-operation projects that share the same purpose (JICA, 2019, p. 7[4]). All evaluations are published online, including summaries in English.

Japan has an opportunity to make its evaluations more strategic. Both MOFA and JICA have a sound evaluation policy and guidelines (MFA, 2019[6]). These are aligned with the previous DAC evaluation criteria and will be adapted to the new DAC evaluation criteria.6 Japan should make most use of this adaptation process to match evaluation efforts with learning and accountability needs in order to develop more strategic evaluations that look beyond project delivery to more systemic and programme-based outcomes. JICA is currently working to build its evaluation methodology for private sector investment finance based on good practice standards that consider the particularities of this modality.

Annual evaluation reports compile the results of all evaluations. MOFA’s ODA evaluations, including the ratings and response measures to the recommendations, are compiled in annual reports and available online. JICA’s external evaluations ratings (A-D) based on subcomponents related to effectiveness and impact; relevance; efficiency; and sustainability equally feature in annual online reports. The reports analyse evaluation results and feature overviews of the year’s evaluations, including those of less successful interventions.7 This is an important learning practice, highlighting how things can be done differently in future interventions (JICA, 2019[4]). The JICA report draws practical lessons to inform future projects and policies, such as challenges and recommendations in external ex-post evaluations in Afghanistan (Chapter 7).

Japan’s continued emphasis on comprehensive and joint evaluations with partner countries is particularly noteworthy. It has invested in building the evaluation capacity of national authorities in Viet Nam and Myanmar and has conducted a number of joint evaluations with partner countries including the Philippines, Thailand and Viet Nam. Japan places high value and importance on conducting joint site visits and joint evaluations with official counterparts in ministries and implementing agencies and strengthening their capacity through its technical assistance; this was confirmed by government officials in Ghana and Cambodia. Evaluations play a key role in informing subsequent phases and future project design.

JICA has also been evaluated along with other development partners. In one recent example, the Government of the Philippines sought permission to conduct an evaluation of the various projects with different modalities, terms, and conditions provided by JICA, the World Bank, and the French Development Agency (AFD) in the aftermath of Typhoon Yolanda (JICA, 2019, p. 35[4]).

Evaluation findings are widely incorporated into future work. Although JICA does not provide official management responses to evaluations8, once an ex-post evaluation has been completed, monitoring determines the extent to which measures and actions have incorporated the evaluation recommendations. The findings of ex-post evaluations are available publicly on JICA’s official website. Evaluation results are fed back into strategies and thematic guidelines, reflected in new and ongoing projects, and fed back to the programmes and policies of partner countries. For example, in Ghana, a final evaluation of the community health project recommended involving the central government for reasons of sustainability, but also to ensure that central government “owned” the tools and systems to make the national scale-up more likely. This recommendation was taken on board by JICA in a follow-up project.

JICA’s public website on lessons learned from its evaluations9 must be consulted before a new project design is approved to ensure experience is fed into similar projects. In this way, it is able to scale-up good practices and results across its broad range of programming. In a few cases though, referring back to previous projects’ successes and learning from failures can stifle experimentation and contextualisation.10 Japan could also do more to systematically draw lessons from partner governments and other stakeholders.

JICA has a strong internal knowledge management system in place to share lessons and challenges across 19 communities of practice in specific sectors and areas. The system works well to share information and to connect Tokyo with country offices. In addition to JICA’s publications and seminars, JICA’s knowledge is widely shared through the country-based ODA Task Force meetings, consisting of Japanese embassies and JICA’s overseas offices, as well as overseas offices of the Japan External Trade Organisation (JETRO), the Japan Bank for International Cooperation (JBIC), and private companies in more than 130 partner countries and regions. MOFA’s International Co-operation Bureau is responsible for disseminating evaluations and lessons learned via the intranet and conducts regular trainings on ODA evaluations. For JICA, better knowledge management is seen as key to promoting innovation in development co-operation.


[4] JICA (2019), JICA Annual Evaluation Report 2018, Japan International Cooperation Agency, https://www.jica.go.jp/english/our_work/evaluation/reports/2018/c8h0vm0000f4nmle-att/full_2018_a4.pdf.

[6] MFA (2019), ODA Evaluation Guidelines, https://www.mofa.go.jp/policy/oda/evaluation/basic_documents/pdfs/guidelines12th.pdf (accessed on 25 February 2020).

[7] Nakano, Y. and E. Magezi (2019), The impact of microcredit on agricultural technology adoption and productivity: evidence from randomized control trial in Tanzania, https://www.jica.go.jp/jica-ri/publication/workingpaper/l75nbg000018sx0m-att/JICA_RI_WP_No.193.pdf (accessed on 25 February 2020).

[3] OECD (2016), Evaluation Systems in Development Co-operation: 2016 Review, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264262065-en.

[2] OECD (2014), OECD Development Co-operation Peer Reviews: Japan 2014, OECD Development Co-operation Peer Reviews, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264218161-en.

[9] Sawada, Y. et al. (2017), Individualized Self-learning Program to Improve Primary Education: Evidence from a Randomized Field Experiment in Bangladesh, https://www.jica.go.jp/jica-ri/publication/workingpaper/l75nbg00000g3kap-att/JICA-RI_WP_No.156.pdf (accessed on 25 February 2020).

[8] Tanaka, T. et al. (2019), Barriers to Public Pension Program Participation in a Developing Country, https://www.jica.go.jp/jica-ri/publication/workingpaper/l75nbg0000196kst-att/JICA_RI_WP_No199.pdf (accessed on 25 February 2020).

[5] Waseda University (2018), Country Assistance Evaluation of Cambodia: Third Party Evaluation, https://www.mofa.go.jp/policy/oda/evaluation/FY2017/pdfs/cambodia.pdf (accessed on 25 February 2020).

[1] Yoshida, K. (2018), “The policy-implementation-results linkage for education development and aid effectiveness in the “Education 2030” era”, Compare: A Journal of Comparative and International Education, Vol. 48/1, pp. 39-55, https://doi.org/10.1080/03057925.2017.1283583.


← 1. For a list of ex-ante evaluations, see www.jica.go.jp/english/our_work/evaluation/oda_loan/economic_cooperation/index.html.

← 2. While the Administrative Review and the Government Policy Evaluation Act reinforces its role, Japan does not have a structured results-based management system in place.

← 3. Total support to statistical capacity building was USD 7.7 million in 2017 and 2018.

← 4. An example of environment and social monitoring for the Olkaria V Geothermal Project in Kenya can be found here: www.jica.go.jp/english/our_work/social_environmental/id/africa/kenya/c8h0vm000090rho9-att/c8h0vm0000ez6qn7.pdf.

← 5. Questions for the ODA evaluation of diplomatic viewpoints include: (1) How is Japan’s ODA geopolitically important?; (2) How important is the bilateral relationship between Japan and the recipient country?; (3) How has Japan’s ODA contributed to strengthen bilateral relationship politically and economically?; (4) How has Japan’s ODA promoted the recipient country people’s understanding of Japan? Source: www.mofa.go.jp/mofaj/gaiko/oda/files/000337119.pdf.

← 6. For an explanation of the criteria, see www.oecd.org/dac/evaluation/revised-evaluation-criteria-dec-2019.pdf.

← 7. For example, a water supply and sewerage project in Peru improved services in provincial cities, but the volume of sewerage received exceeded the planned volume, pointing to the importance of accurate demand forecast.

← 8. MOFA provides official management responses to evaluations through feedback meetings.

← 9. Available at www.jica.go.jp/english/our_work/evaluation/lessons/index.html.

← 10. This was mentioned by some staff members in the field who sometimes felt limited by what they could do based on past projects.

Metadata, Legal and Rights

This document, as well as any data and map included herein, are without prejudice to the status of or sovereignty over any territory, to the delimitation of international frontiers and boundaries and to the name of any territory, city or area. Extracts from publications may be subject to additional disclaimers, which are set out in the complete version of the publication, available at the link provided.

© OECD 2020

The use of this work, whether digital or print, is governed by the Terms and Conditions to be found at http://www.oecd.org/termsandconditions.