Annex B. Methodology for composite indexes on green budgeting and gender budgeting

The narrowly defined composite indexes described here represent the best way of summarising discrete, qualitative information. “Composite indexes are much easier to interpret than trying to find a common trend in many separate indicators” (Nardo et al., 2005[1]). However, their development and use can be controversial. These indexes are easily and often misinterpreted by users due to a lack of transparency about how they are generated, which makes it difficult to truly unpack what they are actually measuring.

The OECD has taken several steps to avoid or address common problems associated with composite indexes. The composites presented in this publication were developed using the steps identified in the Handbook on Constructing Composite Indicators (OECD/European Union/EC-JRC, 2008[2]) that are necessary for the meaningful construction of composite or synthetic indexes.

Each composite index is based on a theoretical framework representing an agreed concept in the area it covers. The variables comprising the indexes are chosen based on their relevance to the concept. Each index is constructed in close collaboration with the relevant OECD expert groups, which advised on the variables and the weighting schemes to use for the composite.

A number of statistical analyses were also conducted to ensure the validity and reliability of the composite indexes. The survey questions used to create the indexes are the same across countries, to ensure indexes are comparable. In order to eliminate scale effects, all indicators and variables were normalised between “0” and “1” for comparability. To build the composites, all indicators were aggregated using a linear method. The index scores were determined by adding together the weighted scores of each indicator. Statistical tools (i.e. Cronbach’s alpha) were also employed to establish the degree of correlation among a set of variables comprised in each index and to check their internal reliability. This implies that all of the variables comprised in each index have intrinsic value but are also interlinked and capture the same underlying concept. Finally, sensitivity analysis using Monte Carlo simulations was carried out to establish the robustness of the index scores to different weighting options.

Green budgeting refers to integrating climate and environmental considerations into the budgetary process. It involves the use of special initiatives, processes and analytical tools with a view to promoting policies and investments that help achieve climate and environmental goals and commitments. The 2022 OECD Green Budgeting Index is designed around the four building blocks in the OECD Green Budgeting Framework: 1) institutional arrangements; 2) methods and tools; 3) accountability and transparency; and 4) the enabling environment in budgeting (OECD, 2020[3]). Each building block is weighted equally (25%).

Data used for the construction of the 2022 OECD Green Budgeting Index are derived from the 2022 OECD-IDB Survey on Green Budgeting. Survey respondents were predominantly budget officials within central budget authorities in 12 Latin America and Caribbean countries. The 2022 OECD Green budgeting Index applies for the 5 countries that implement green budgeting (Chile, Colombia, Dominican Republic, Honduras and Mexico). The variables and weights comprised in the index were selected based on their relevance to the concept by a group of experts within the OECD and in consultation with country delegates to the OECD Paris Collaborative on Green Budgeting. While the 2022 OECD Green Budgeting Index allows for cross-country comparison, it is not context specific, nor can it fully capture the complex realities of the quality, use and impact of green budgeting approaches. This comparison should hence not be seen as a measurement of quality or a ranking. It shows that countries have adopted multi-initiative approaches to green budgeting by using each of the four building blocks (OECD, forthcoming[4]).

The components used in the construction of this index, and the weights given to each, are indicated in the figure below.

Sensitivity analysis was carried out to establish the robustness of the indicators to different weighting options through Monte Carlo simulations. The results of the sensitivity analysis at building block level for the 2022 OECD Green Budgeting Index show that, for most of the countries analysed, total scores are not very sensitive to the choice of values given to the categories. Cronbach’s alpha coefficient is equal to 0.75, indicating that the building blocks are measuring the same underlying construct (OECD, forthcoming[4]).

Gender budgeting refers to the integration of a clear gender perspective within the overall context of the budgetary process. It involves the use of special processes and analytical tools with a view to promoting policies and investments that help achieve gender equality goals. To strengthen the implementation of gender budgeting, the OECD has recently updated its Framework on Gender Budgeting, now capturing five building blocks: 1) institutional and strategic arrangements; 2) enabling environment; 3) methods and tools; 4) accountability and transparency; and 5) impact (Gatt Rapa and Nicol, forthcoming[5]). The 2022 OECD Gender Budgeting Index is designed around these five building blocks. Each building block is weighted equally (20%).

Data used for the construction of the 2022 OECD Gender Budgeting Index are derived from the 2022 OECD-IDB Survey on Gender Budgeting. Survey respondents were predominantly senior budget officials within central budget authorities in 13 Latin America and Caribbean countries. The 2022 OECD Gender budgeting Index applies for the 9 countries that implement gender budgeting (Argentina, Brazil, Chile, Colombia, Guatemala, Honduras, El Salvador, Mexico and Paraguay). The variables and weights comprised in the index were selected based on their relevance to the building block by a group of experts within the OECD and in consultation with country delegates to the Senior Budget Officials (SBO) Network on Gender Budgeting. While the 2022 OECD Gender Budgeting Index allows for cross-country comparison, it is not context specific, nor can it fully capture the complex realities of the quality, use and impact of gender budgeting approaches. This comparison should hence not be seen as a measurement of quality or a ranking. It shows that countries have adopted multi-initiative approaches to gender budgeting by using each of the five building blocks (OECD, 2023[6]).

The components used in the construction of this index, and the weights given to each, are indicated in the figure below.

Sensitivity analysis was carried out to establish the robustness of the indicators to different weighting options through Monte Carlo simulations. The results from the sensitivity analysis at the building block level for the 2022 OECD Gender Budgeting Index show that, for the majority of countries analysed, total scores are not very sensitive to the choice of values given to the categories. Cronbach’s alpha coefficient is equal to 0.84, indicating that the building blocks are measuring the same underlying construct (OECD, 2023[6]).

References

[5] Gatt Rapa, K. and S. Nicol (forthcoming), OECD Framework for Gender Budgeting, OECD Publishing, Paris.

[1] Nardo, M. et al. (2005), Tools for Composite Indicators Building, European Commission, https://publications.jrc.ec.europa.eu/repository/handle/JRC31473.

[6] OECD (2023), Gender Budgeting in OECD Countries 2023, OECD Publishing, Paris, https://doi.org/10.1787/647d546b-en.

[3] OECD (2020), OECD Green Budgeting Framework: Highlights, OECD, Paris, https://www.oecd.org/environment/green-budgeting/OECD-Green-Budgeting-Framework-Highlights.pdf.

[4] OECD (forthcoming), Green budgeting in OECD Countries 2024, OECD Publishing, Paris.

[2] OECD/European Union/EC-JRC (2008), Handbook on Constructing Composite Indicators: Methodology and User Guide, OECD Publishing, Paris, https://doi.org/10.1787/9789264043466-en.

Legal and rights

This document, as well as any data and map included herein, are without prejudice to the status of or sovereignty over any territory, to the delimitation of international frontiers and boundaries and to the name of any territory, city or area. Extracts from publications may be subject to additional disclaimers, which are set out in the complete version of the publication, available at the link provided.

© OECD 2024

The use of this work, whether digital or print, is governed by the Terms and Conditions to be found at https://www.oecd.org/termsandconditions.