Annex A. Methodological annex

The OECD fragility framework considers fragility to be multidimensional, measurable on a spectrum of intensity and expressed in different ways across five dimensions. It uses robust quantitative approaches to measure the magnitude of fragility, and it compares and contrasts different types of fragility descriptively. This mixed approach allows the analysis to extract the best value from the quantitative methods while also addressing the limitations of these methods through qualitative descriptions.

The methodology is based on a two-stage process that first examines contexts in each of five dimensions and then aggregates this information to obtain an overall picture of fragility. For each dimension, principal components analysis (PCA) is used to combine the risk and coping capacity indicators into two statistically derived components (Figure A A.1). Deriving two measures per dimension has distinct advantages over creating one composite index. First, using two measures allows for greater understanding of the differences among contexts that would score equally when a single measure is used. Second, using the first two principal components allows contexts to be broadly grouped based on their similarities in all of the input variables. Third, each indicator is weighted by the amount of new information it brings to the data, rather than on a set of normative judgements on their relative importance. With the components of each dimension calculated, contexts are then grouped on similarities and classified descriptively. Thus this mix of both quantitative and qualitative methods offers a more flexible approach to describing the diversity of fragility.

Figure A A.1. The 2018 OECD fragility framework methodology
picture

Once contexts are classified into groups within each dimension, the second part of the methodology aggregates this information to arrive at an overall picture of fragile contexts. To do so, the components of each dimension provide inputs to a second aggregate PCA that is then used to produce the 58 fragile contexts in the fragility framework.

The methodology is ambitious in its objectives but has limitations. By using PCA, the range of indicators can be reduced to two core components, thereby explaining most of the variance in the original data. However, in doing so information invariably is lost. The second stage of PCA (PCA Stage 2) exacerbates this loss of information. In short, the results of this approach are a summary of the initial indicators that is then interpreted in terms of fragility. Despite these limitations, this summary is both more informative and less arbitrary than any composite index based on the initial indicators.

Aside from the technical limitations, there are also certain practical limitations to what can be captured in any quantitative approach. The unit of analysis for the OECD fragility framework is country level. As a consequence, the framework is unable to capture macro-level drivers of fragility – drivers that spill over borders – or micro-level drivers that lead to localised pockets of fragility within states. Going forward, it would be useful to find ways to draw on subnational data and to link up regional and global data. Further, while data on governance are widely available, data on informal institutions are less so. While every effort has been made to include indicators of both of these, at this point the lack of quality data is a limiting factor for the model. Finally, the calculations exclude 27 countries and territories where there was insufficient data to feed into the analysis (Box A.1).

Box A A.1. Countries and territories not included in the fragility framework

Data availability is a key issue in calculating the OECD fragility framework. As the unit of analysis is the state or territory, it is important to select indicators that are comparable across those states and territories. While statistical imputation methods can be used to fill data gaps, such an approach is best used sparingly; preference should always be given to real-world data, even if it means dropping indicators or countries and territories that otherwise would have been included. The fragility framework methodology aims to strike a balance between the number of indicators, the contexts covered and the amount of imputation that would be required to build a complete data set. A criterion for inclusion in the OECD framework was at least 70% of the required data had to be available for a country or context. As a result, only 172 contexts could be included in the calculations.

This does not mean that the excluded contexts are not fragile. Indeed, many of those excluded are small island developing states that face unique challenges. The final list also excludes two territories with UN peacekeeping missions (Kosovo and Western Sahara) and several Pacific Island countries whose high levels of interpersonal violence are well known.

Table A A.1. Countries and territories excluded in 2018 due to insufficient data

Anguilla

Kiribati

Niue

Tonga

Antigua and Barbuda

“Kosovo”

Palau

Turks and Caicos Islands

Belize

Malta

Saint Helena, Ascension and Tristan da Cunha

Tuvalu

Cook Islands

Marshall Islands

Saint Kitts and Nevis

Dominica

Mayotte

Wallis and Futuna

Federated States of Micronesia

Saint Lucia

Western Sahara

Grenada

Nauru

Tokelau

Montserrat

Saint Vincent and the Grenadines

Samoa

Indicator coverage and missing data

The choice of indicators has been driven by selection criteria in line with the OECD’s fragility concept of high risk and low coping capacity. Normal technical criteria for selecting good indicators were used, but with a particular and added emphasis on selecting indicators based on their relationship to fragility – that is, are they a cause of fragility or an outcome of fragility? Indicators that represent outcomes of fragile contexts do not offer clear guidance as to policies that can reduce fragility. For example, infant mortality is an indicator used in several fragility measures. However, infant mortality is arguably more of an outcome of a fragile health context than a cause of it. In selecting indicators, then, the following factors were considered, in keeping with OECD concept of fragility:

  • Risk. Do the indicators alter either likelihood or impact ex-ante?

  • Coping capacity. What indicators would stop the risk cascading ex-post if the risk occurred

Using these criteria does not eliminate the challenge of separating some indicators into the category either of a risk or a coping capacity. For example, the level of armed personnel can be considered a coping capacity for dealing with insurgencies. It can also be considered a contributor to the risk of violence. Methodological decisions have been made to account for this challenge to produce the best approximation, given the limitations.

Further, some coping capacity indicators have been used in more than one dimension, introducing an unintended issue in aggregating the dimensions to provide the final 58 contexts used in analysing flows by fragility. Using indicators more than once essentially weights these indicators more than others. Statistical measures have been used to minimise this effect, and this method ultimately was chosen as the preferred alternative to dividing some of these indicators into one, and only one, dimension. For example, government effectiveness and rule of law are important not only for the security dimension but also for the environmental dimension. Forcing them into a single dimension arbitrarily creates an incomplete picture of the interconnectedness of coping capacities. Therefore, the de facto weighting effect was considered justifiable, if not ideal, given cross-dimensional importance. Table A A.2 lists the indicators used in the 2018 fragility framework.

Table A A.2. Indicators of Fragility

Dimension

Type

Indicator name

Source

Description

Societal

Risk

Gini coefficient

V-DEM

Distribution of income expressed as a Gini coefficient

Risk

Gender inequality

UNDP/HDI

Measures gender inequalities in reproductive health (maternal mortality ratio and adolescent birth rates); empowerment (proportion of parliamentary seats occupied by females and proportion of adult females and males aged 25 years and older with at least some secondary education); and economic status (expressed as labour market participation and measured by labour force participation rate of female and male populations aged 15 years and older)

Risk

Horizontal inequality

V-DEM

Do all social groups (distinguished by language, ethnicity, religion, race, region or caste) enjoy the same level of civil liberties or are some groups generally in a more favourable position?

Risk

Uprooted people

INFORM

Combination of the number of refugees, returned refugees and internally displaced persons

Risk

Urbanisation growth

WB

Annual urban population growth

Coping

Core civil society index

V-DEM

Provides a measure of how robust a nation’s civil society is

Coping

Access to justice

V-DEM

Extent to which citizens enjoy secure and effective access to justice

Coping

Voice and accountability

WGI

Reflects perceptions of the extent to which a country’s citizens are able to participate in selecting their government, as well as freedom of expression, freedom of association and free media

Political

Risk

Regime persistence

Polity IV

Number of years polity has persisted (as measured by no change in any of the Polity IV measures)

Risk

Political terror

PTS

Levels of state-sanctioned or state-perpetrated violence (e.g. political violence such as assassinations of political challengers and police brutality)

Risk

Perception of corruption

TI

TI Transparency International (TI) Corruption Perceptions Index (CPI) ranks countries annually by their perceived levels of corruption, as determined by expert assessments and opinion surveys”. The CPI generally defines corruption as “the misuse of public power for private benefit”.

Coping

Decentralised elections

V-DEM

Are there elected regional governments and, if so, to what extent can they operate without interference from unelected bodies at the regional level?

Coping

Voice and accountability

WGI

Reflects perceptions of the extent to which a country’s citizens are able to participate in selecting their government, as well as freedom of expression, freedom of association and free media

Coping

Restricted gender physical integrity value

OECD

Measures prevalence of laws on rape and domestic violence and captures experience of violence

Coping

Judicial constraints on executive power

V-DEM

To what extent does the executive respect the constitution and comply with court rulings, and to what extent is the judiciary able to act in an independent fashion?

Coping

Legislative constraints on executive power

V-DEM

To what extent are the legislature and government agencies (e.g. comptroller general, general prosecutor or ombudsman) capable of questioning, investigating and exercising oversight over the executive?

Environmental

Risk

Natural hazard exposure

INFORM

Measures the exposure to disasters such as earthquake, tsunami, flood, tropical cyclone and drought

Risk

Environmental health

Yale

Measure of health impacts, quality of air, water and sanitation

Risk

Uprooted people

INFORM

Combination of the number of refugees, returned refugees and internally displaced persons

Risk

Prevalence of infectious disease

GBD and CSIS

Infectious disease deaths per 100 000 population

Risk

Socio-economic vulnerability

INFORM

Measures the (in)ability of individuals and households to afford safe and resilient livelihood conditions and well-being; combines indicators of development and deprivation, inequality and aid dependency

Coping

Rule of law

WGI

Reflects perceptions of the extent to which agents have confidence in and abide by the rules of society, in particular the quality of contract enforcement, property rights, police and courts, and the likelihood of crime and violence

Coping

Government effectiveness

WGI

Captures perceptions of the quality of public services; the quality of the civil service and the degree of its independence from political pressures; the quality of policy formulation and implementation; and the credibility of the government’s commitment to such policies

Coping

Core civil society index

V-DEM

How robust is civil society?

Coping

Food security

INFORM

Prevalence of undernourishment, average dietary supply adequacy, domestic food price index and domestic food price volatility

Economic

Risk

Resource rent dependence

WB

The sum of oil rents, natural gas rents, coal rents (hard and soft), mineral rents and forest rents

Risk

General government gross debt

IMF

General government debt as a percentage of GDP

Risk

NEET

ILO

Proportion of youth not in employment, education or training (NEET)

Risk

Aid dependency

INFORM

Combination of net ODA as a percentage of GNI, total ODA per capita in the last two years and total humanitarian aid per capita in last two years

Risk

Unemployment rate

WB

Unemployment rate

Risk

Socio-economic vulnerability

INFORM

Combination of indicators related to development and deprivation, inequality, and aid dependency

Risk

GDP growth rate

WB

Compound annual growth rate of GDP over the last five years

Coping

Women in labour force

UNDP/HDI

Percentage of female participation in the labour force

Coping

Males in labour force

UNDP/HDI

Percentage of male participation in the labour force

Coping

Education

HDI

Measured by the mean of years of schooling for adults age 25 years and over and expected years of schooling for children of school age entering school

Coping

Regulatory quality

WGI

Reflects perceptions of the ability of the government to formulate and implement sound policies and regulations that permit and promote private sector development

Coping

Remoteness

EVI

Trade-weighted average distance from world markets

Coping

Food security

INFORM

Prevalence of undernourishment, average dietary supply adequacy, domestic food price index and domestic food price volatility

Security

Risk

Violent conflict risk

INFORM

Statistical risk of violent conflict in the next 1-4 years based on 25 quantitative indicators from open sources.

Risk

Homicide rate

UNODC

Intentional homicide rate per 100 000 population

Risk

Level of violent criminal activity

IPD

Intensity of violent activities by underground political organisations: by criminal organisations (e.g. drug trafficking, arms trafficking, prostitution, etc.).

Risk

Deaths by non-state actors per capita

UCDP-NS

Total of one-sided and non-state actor datasets, average per capita rate, 2013-16.

Risk

Impact of terrorism

IEP/START

Global Terrorism Index score for a context in a given year accounts for the relative impact of incidents in the year. Four factors are counted: number of terrorist incidents; number of fatalities caused by terrorism; number of injuries caused by terrorism; and approximate level of total property damage from terrorist incidents in a given year. It is a five-year weighted average to capture lingering fear effects.

Risk

Battle-related deaths per capita (log)

UCDP-BD

Total of battle-related deaths per capita, transformed using the logarithm function

Coping

Police officers per 100 000

GPI

Police officers per 100 000 population

Coping

Armed security officers per 100 000

GPI

Armed security officers per 100 000 population

Coping

Rule of law

WGI

Reflects perceptions of the extent to which agents have confidence in and abide by the rules of society, in particular the quality of contract enforcement, property rights, police and courts, and the likelihood of crime and violence

Coping

Control over territory

V-DEM

Over what percentage of the territory does the state have effective control?

Coping

Government effectiveness

WGI

Captures perceptions of the quality of public services, the quality of the civil service and the degree of its independence from political pressures; the quality of policy formulation and implementation; and the credibility of the government's commitment to such policies.

Coping

Restricted gender physical integrity value

OECD

Measures prevalence of laws on rape and domestic violence and captures experience of violence

Coping

Formal alliances

COW

Formal alliance between at least two states that falls into the class of defence pact, neutrality or non-aggression treaty, or entente agreement

Age of data

The 43 indicators selected do not cover all contexts and imputation techniques have been used to fill in data gaps. Lack of data is the primary reason why a context may not be included. At least 70% of data for a context had to be available for it to be included in the OECD fragility framework. In 2018, this yields a list of 172 contexts. It is possible to assume that contexts missing from the dataset have a certain value for some indicators. For example, those missing from the datasets for battle deaths and deaths by non-state actors can be assumed to have a value of 0. Where no reasonable assumption could be made, data are imputed using k-nearest neighbour (KNN) imputation that uses statistical inference to fill in missing data from the k most similar contexts. In the OECD fragility framework, this has been done using the 15 most similar contexts for each missing data point (Table A A.3).

Table A A.3. Indicator coverage

Indicator

Minimum year of data used

Maximum year of data used

Number of countries with available data

Imputation technique used

Access to justice

2016

2016

169

KNN

Aid dependency

2016

2016

172

KNN

Armed security officers per 100 000

2017

2017

161

KNN

Battle-related deaths per capita (log)

2016

2016

172

Missing countries assigned 0

Control over territory

2012

2015

164

KNN

Core civil society index

2016

2016

169

KNN

Deaths by non-state actors per capita

2016

2016

172

Missing countries assigned 0

Decentralised elections

2016

2016

169

KNN

Education

2015

2015

170

KNN

Environmental health

2016

2016

169

KNN

Food security

2016

2016

172

KNN

Formal alliances

2012

2017

172

KNN

GDP growth rate

2015

2015

165

KNN

Gender inequality

1995

2015

154

KNN

General government gross debt

2008

2016

164

KNN

Gini coefficient

2012

2015

148

KNN

Government effectiveness

2016

2016

172

KNN

Homicide rate

2008

2015

172

KNN

Horizontal inequality

2016

2016

169

KNN

Impact of terrorism

2016

2016

172

KNN

Judicial constraints on executive power

2016

2016

169

KNN

Legislative constraints on executive power

2016

2016

169

KNN

Level of violent criminal activity

2016

2016

140

KNN

Males in labour force

2015

2015

171

KNN

Natural hazard exposure

2016

2016

172

KNN

NEET

1995

2016

140

Combined OECD data with World Bank and ILO datasets

Perception of corruption

2016

2016

166

KNN

Police officers per 100 000

2017

2017

161

KNN

Political terror

2015

2015

172

KNN

Prevalence of infectious disease (deaths per 100 000)

2016

2016

172

KNN

Regime persistence

2016

2016

164

KNN

Regulatory quality

2016

2016

172

KNN

Remoteness

2005

2015

155

OECD countries assigned 0, KNN imputation for the remainder

Resource rent dependence

2007

2015

171

KNN

Restricted gender physical integrity value

2014

2014

144

OECD countries given OECD average, KNN imputation for the remainder

Rule of law

2016

2016

172

KNN

Socio-economic vulnerability

2016

2016

172

KNN

Unemployment rate

2016

2016

170

KNN

Uprooted people

2016

2016

172

KNN

Urbanisation growth

2011

2016

172

KNN

Violent conflict risk

2016

2016

172

KNN

Voice and accountability

2016

2016

172

KNN

Women in labour force

2015

2015

171

KNN

Creating a time series

States of Fragility 2018 extends previous OECD work by creating a time series for deeper analysis of improvements and deteriorations in previously identified fragile contexts. To do this, all data have been imputed and extended to a period spanning 2016 and 2017. Using only 2016 or 2017 data, PCA models were generated for all five dimensions and aggregate scores. These models were applied across different years to create a time series comparable from year to year.

Cluster analysis

Fragile contexts can be grouped based on their similar characteristics. In order to identify frequently repeating patterns across these contexts and to group them based on their performances, a clustering algorithm was carried out using a hierarchical clustering procedure. Hierarchical clustering has emerged as a useful baselining tool in political science studies (Wolfson, Madjd-Sadjadi and James, 2004[1]). The clustering procedure provides two results. First, each context is grouped with other contexts that have the highest possible similarity with one another. Second, it defines the profile of an average context for each group. The profiles highlight the relevant attributes and distinct profile of each group, making it possible to quantitatively differentiate among groups. Six groups or clusters have been identified and named. The main attributes of each cluster are defined by their unique quantitative behaviour.

The OECD uses the clustering procedure as an indicative aid to assist in the qualitative assessment of different types of fragility. The severity of factors and/or combination of factors have been assessed by experts. Within each dimension, clusters have been ranked on a six level severity scale:

1 = Severe fragility

2 = High fragility

3 = Moderate fragility

4 = Low fragility

5 = Minor fragility

6 = Non-fragile

Once grouped, each cluster is compared to every other cluster to determine what characteristics best define its specific fragility. To do this, a Tukey ANOVA test is used (Hinton, 2014[2]). This method takes the means of all indicators within each cluster and conducts a difference in means test, which compares the mean of indicators in every other cluster. A statistical significance criterion has been developed to identify indicators whose levels are unique to any cluster when compared to the rest of the world. This criterion identifies indicators where the mean for any one cluster is significantly different at 95% confidence levels from at least four of the remaining clusters. Broadly speaking, this criterion can be interpreted as highlighting indicators in each cluster that are statistically different to at least 80% of the rest of the world.1

The next sections will show and describe the results of the cluster analyses for each of the five dimensions of the OECD’s 2018 fragility framework.

Economic dimension

The economic dimension aims to capture the vulnerability to risks stemming from the weaknesses in economic foundations and human capital including macroeconomic shocks, unequal growth, high youth unemployment, etc.

Figure A A.2 shows the economic dimension biplot with each cluster in a different colour. The results of the Tukey ANOVA test for significance suggest that the aqua-coloured cluster is distinguished by higher participation of men and women in the workforce. However, this cluster is weaker in regulatory quality, food security, education, socio-economic vulnerability and dependence on resource rent. The olive-coloured cluster shares the same weaknesses, with additional weaknesses in NEET and aid dependence.

Figure A A.2. Economic dimension typology
picture

Environmental dimension

The environmental dimension aims to capture the vulnerability to environmental climactic and health risks to citizens’ lives and livelihoods. This includes exposure to natural disasters, pollution and disease epidemics.

Figure A A.3 shows the biplot with each cluster in a different colour. The results of the Tukey ANOVA test for significance suggest that the aqua-coloured cluster is distinguished by low environmental health and high prevalence of disease and socio-economic vulnerability. The violet cluster is distinguished by high exposure to natural disaster. The grey cluster is strong in rule of law and government effectiveness and has low socio-economic vulnerability and lower levels of environmental health.

Figure A A.3. Environmental dimension typology
picture

Security dimension

The security dimension aims to capture the vulnerability of citizen security emanating from social and political violence. As such it includes indicators of citizen exposure to direct political and social violence.

Figure A A.4 shows the security dimension biplot with each cluster in a different colour. The results of the Tukey ANOVA test for significance suggest that the aqua-coloured cluster is distinguished by higher deaths by non-state actors, higher battle-related deaths and low control of territory. However, this cluster has stronger coping capacities than other clusters. The olive cluster is distinguished by higher levels of interpersonal and political violence. It is weak in coping capacities relating to rule of law and government effectiveness. The grey cluster is stronger in government effectiveness, rule of law and formal alliances, which lead to lower risk of conflict and violent crime.

Figure A A.4. Security dimension typology
picture

Political dimension

The political dimension aims to capture the vulnerability to risks inherent in political processes, events or decisions, to its political inclusiveness (incl. elites) and transparency (corruption) and to its ability to accommodate change and avoid oppression.

Figure A A.5 shows the biplot coloured by cluster. The results of the Tukey ANOVA test for significance suggest that the aqua-coloured cluster is distinguished by having higher political terror and perception of corruption. It is also weak in coping capacities relating to voice and accountability, gender physical integrity, and constraints on executive power. Conversely, the olive cluster is weak in all but one of the coping capacities but does not have the same presence of risk factors as can be seen in the aqua cluster. The brown and grey clusters have strong coping capacities coupled with low levels of risk factors.

Figure A A.5. Political dimension typology
picture

Societal dimension

The societal dimension aims to capture the vulnerability to risks affecting societal cohesion that stem from both vertical and horizontal inequalities (inequality among culturally defined [or constructed] groups), social cleavages, etc.

Figure A A.6 shows the biplot coloured by cluster. The results of the Tukey ANOVA test for significance suggest that the aqua-coloured cluster is distinguished by having lower coping capacities in voice and accountability and access to justice. It is also faced with high risks through urbanisation growth, uprooted people and both gender and income inequality. The olive cluster is distinguished by low levels of all coping capacities and high horizontal inequality. The grey cluster is strong in coping capacities and faces lower levels of risk.

Figure A A.6. Societal dimension typology
picture

Aggregate fragility

The second part of the OECD methodology aggregates all of the information to arrive at an overall picture of combinations of fragilities. This second tier aggregate analysis generated the group of the 58 most fragile contexts, which are classified as extremely fragile and fragile.

The second tier PCA generated six fragility clusters that, are differentiated not only by their extent of fragility but also in the dominant characteristics of that fragility. This is shown in Figure A A.7. The first dimension of the PCA represents coping capacities. The second dimension represents the types of fragility. To arrive at the 58 most fragile contexts, two cut-offs have been selected. In order for a context to be classified as extremely fragile, it has to score less than -2.5 on the first principal component of the aggregate PCA shown in Figure A A.7. In order to be classified as fragile, a context must score between -1.2 and -2.5 on the first principal component.

The biplot of the aggregate can be split by contexts above and below the x-axis. Those above the x-axis are dominated by economic factors and the contexts below the x-axis are dominated by fragilities in the political, societal and/or security dimensions. Fragility in the environmental dimension can be found in contexts above and below the x-axis.

Figure A A.7. Biplot for aggregate fragility
picture

References

[2] Hinton, P. (2014), Statistics explained, Routledge, https://www.routledge.com/Statistics-Explained-3rd-Edition/Hinton/p/book/9781848723122 (accessed on 26 June 2018).

[1] Wolfson, M., Z. Madjd-Sadjadi and P. James (2004), “Identifying National Types: A Cluster Analysis of Politics, Economics, and Conflict”, Journal of Peace Research, Vol. 41/5, pp. 607-623, http://dx.doi.org/10.1177/0022343304045975.

Note

← 1. For further information, also see (Lamb, 2017[145]), (Lamb, 2013[147]) and (Lamb and Mixon, 2014[146]).

This report includes 172 countries grouped into 6 clusters for each dimension. By conducting the Tukey ANOVA test at 95% confidence for all our indicators, we are comparing the indicators’ means of each cluster to the indicators’ means of the other clusters. If any mean is statistically different from the means of at least four of the remaining five clusters, it is considered a defining characteristic. Significance in this case can broadly be interpreted as indicators’ means for each cluster being statistically different to approximately four-fifths (80%) of the rest of the world.

End of the section – Back to iLibrary publication page