2. Assessment methodology and process

The publication series, Competitiveness in South East Europe: A Policy Outlook (Competitiveness Outlook), provides a comprehensive assessment of competitiveness reforms in the six Western Balkan (WB6) economies: Albania, Bosnia and Herzegovina, Kosovo, Montenegro, North Macedonia, and Serbia. The publication uses international and OECD good practice to design sustainable economic reform agendas that enable policy makers to achieve greater competitiveness for their economies.

As the third edition of the Competitiveness Outlook series, the Competitiveness Outlook 2021 provides an important benchmarking tool that monitors the development, implementation and performance of policies affecting a broad range of policy areas. Its methodology has been only marginally refined in comparison to the 2018 edition, thus making it possible to track progress on policy design and implementation over time. This integral feature supports the domestic policy cycle by equipping policy makers with a tool to measure progress, ensure stronger co-ordination and consistency between policies and identify policy priorities. Likewise, the Competitiveness Outlook’s collaborative assessment process is itself a vital exercise that fosters government capacity by integrating the perspectives of government and non-government stakeholders in order to identify the primary economic challenges in an economy and develop tailored policy reforms to address them. This collaborative approach seeks to provide policy makers with a robust source of evidence-based guidance on how to achieve greater economic competitiveness and prosperity.

For the first time in the Competitiveness Outlook series, economy-specific profiles for each of the six WB6 economies offer an in-depth analysis of the key policy areas for strengthening competitiveness post-COVID-19. The inclusion of economy-specific profiles enables this edition of the Competitiveness Outlook to provide individualised structural reform recommendations tailored to the specific challenges of each of the Western Balkan economies.

This assessment cycle of the Competitiveness Outlook took place during the COVID-19 pandemic that began in early 2020. The COVID-19 related developments in France and the Western Balkans had a significant impact on the project activities. Multiple lockdowns in Europe and the Western Balkan economies over the course of 2020 created new challenges associated with remote working and other atypical working conditions. The extraordinary challenges presented by the pandemic necessitated major modifications to the planned project activities. Accordingly, this assessment process used a flexible and innovative approach to successfully conduct data collection and analysis, as well as to engage in the extensive policy dialogue that this assessment requires (Box 2.2).

Although the term “competitiveness” is frequently used by academics, policy makers and the general public, it lacks a common definition and can have different meanings in different contexts, such as a firm, industry, regional, national or supranational context. Academics have attempted to find general definitions that recognise the multi-dimensional character of competitiveness and that can be more practical when applied at the macro level.

The Competitiveness Outlook uses the following three definitions of competitiveness, which all include the concept of sustainability and go beyond an exclusive focus on cost competitiveness and GDP:

  • “The ability of companies, industries, regions, nations or supranational regions to generate, while being and remaining exposed to international competition, relatively high factor income and factor employment levels on a sustainable basis” (Hatzichronoglou, 1996[1]).

  • “A measure of an economy’s ability to provide its population with high and rising standards of living and high rates of employment on a sustainable basis” (EC, 2012[2]).

  • “The ability of a country (region, location) to deliver the beyond-GDP goals for its citizens, today and tomorrow” (Aiginger, Bärenthaler-Sieber and Vogel, 2013[3]).

These holistic definitions imply that to increase an economy’s competitiveness, policy reforms should not be pursued in isolation. Instead, they should seek a comprehensive approach with co-ordinated actions across a variety of policy areas, enabling them to build on one other. In order to achieve this holistic policy-making process, the Competitiveness Outlook 2021 used a collaborative assessment approach encompassing 16 policy dimensions. These dimensions are grouped under four key pillars that are crucial for strengthening competitiveness (Table 2.1).

The Competitiveness Outlook’s methodology is designed to provide an evidence-based assessment of progress in the design and implementation of policies related to overall economic competitiveness. The assessment is based on a set of indicators that have been tailored to each of the 16 policy dimensions and evaluated using a highly participatory process that involves more than 700 local stakeholders.

Each policy dimension is divided into several sub-dimensions that highlight the key elements of that policy area. The sub-dimensions in turn are composed of qualitative and quantitative indicators (Figure 2.1), which capture the design, implementation and performance of policies, processes and institutions. To reflect the latest trends in OECD economic policy research and good practice developed since the previous Competitiveness Outlook (2018) (OECD, 2018[4]), new qualitative and quantitative indicators have been included across the 16 assessment frameworks. Moreover, the assessment has been designed to inform economies on their progress towards achieving the UN Sustainable Development Goals.

The indicators in the Competitiveness Outlook’s 2018 edition were taken as a basis for this publication and further refined in order to increase the focus on critical areas and to integrate additional OECD tools and instruments (Box 2.1). The set of indicators used for each of the 16 policy dimensions can be found in the “assessment framework” included at the beginning of each chapter.

Qualitative indicators assess whether competitiveness-enhancing policy settings, strategies, processes or institutions exist and, if so, the extent to which they have been adopted, implemented, monitored and updated. Each qualitative indicator is assigned a numerical score that reflects the level of policy development and implementation in order to facilitate the comparison of performance among the Western Balkan economies (Figure 2.2).

Table 2.2 illustrates how the general structure of the scoring scale used to measure qualitative information has been translated into level descriptors, using the environment policy dimension’s climate change mitigation and adaptation indicator as an example.

Policy dimension and sub-dimension average scores are generated by calculating a simple average across the relevant qualitative indicator scores. Indicators are not weighted because the relative importance of each indicator may be different to different stakeholders. Average scores should therefore be interpreted with caution and taken only as a rough estimate of overall policy development.

Unlike the other 15 policy dimensions assessed in this edition, where indicators are allocated a score from one to five, the assessment of the four qualitative policy areas and one quantitative policy area in the Competition policy dimension is based on yes/no (coded as 1/0) answers to 71 questions in a questionnaire (listed in the Annex to the Competition policy chapter). Where a response to a question is yes (coded as 1), this is referred to as an adopted criterion. Each of the five policy areas has a different number of possible criteria that can be stated as having been adopted. For more information, please see the Competition policy chapter.

The quantitative indicators are input and output factors pertinent to the assessment of policies, policy making, institutional conditions and policy results – e.g. public or private spending in the policy field in question; the share of actors engaging in a certain activity; or the volume of a certain output resulting from a policy or economic activity. They complement qualitative indicators by supplying quantifiable information on the performance of policy settings, processes and institutions.

The Assessment framework part in each of the 16 policy dimension chapters provides an overview of the qualitative and quantitative indicators used to assess this policy dimension.

The Competitiveness Outlook 2021’s assessment process consisted of four main phases, conducted between October 2019 and June 2021: 1) the design phase; 2) the evaluation phase; 3) the consolidation phase; and 4) the review and publication phase:

  1. 1. Design phase (October 2019 – March 2020)

In this phase, the OECD developed the qualitative and quantitative questionnaires to assess each of the 16 policy dimensions. This process constituted extensive research to ensure that the latest international and OECD good practice were incorporated into the assessment frameworks for each policy dimension. For each qualitative indicator assessed, the OECD compiles a table, which lists the necessary elements that need to be met to attain a certain score level. Respondents have to complete a set of yes/no, multiple-choice and open-ended questions. The answers to these questions and the listed criteria for each score level enable the respondents to self-evaluate the score for this indicator. For every answer, the OECD asks respondents to provide supporting evidence (e.g. adopted strategies, legislation, monitoring reports, etc.), which is subsequently verified and the score level either confirmed or revised. The quantitative questionnaires are tailored to each economy depending on the extent to which statistical data are already available to the OECD. Only quantitative indicators for which data are not available publicly are included in the quantitative questionnaire. The qualitative and quantitative questionnaires had to be digitalised and complemented with additional guidance documents following the impact of the COVID-19 pandemic (Box 2.2).

  1. 2. Evaluation phase (April – November 2020)

This phase consisted of the government self-assessment. It was led by a designated Government Co-ordinator (see Acknowledgements for a list of names), and co-ordinated with the help of policy dimension and statistical office contact points in each economy. Since the qualitative and quantitative questionnaires for each policy dimension always cover several sub-dimensions (Figure 2.1), their completion called for an inclusive approach which led to extensive collaboration among different ministries, public agencies and statistical offices in each WB6 economy, often underpinned by dedicated working groups. Completing the OECD questionnaires thus enabled WB6 policy makers of different institutions working in the same policy field to reflect together on the most pressing challenges, priorities and most suitable way forward for their policy field. This phase also enabled WB6 policy makers to reflect on the roles of their respective services in the policy dimension in question, forge closer ties across different institutions, and to improve future inter-ministerial collaboration for holistic policy making beyond this assessment. Remaining information and data gaps were filled through virtual fact-finding meetings with relevant government stakeholders as well as local non-government stakeholders.

  1. 3. Consolidation phase (December 2020 – March 2021)

The results of the government self-assessments were consolidated into draft economy profiles and draft regional policy chapters whose key findings and initial policy recommendations were presented at seven stakeholder meetings (one for each WB6 economy and one regional meeting). While these meetings were held in the six Western Balkan capitals in previous assessment cycles, COVID-19 restrictions meant that the roundtables had to be held virtually. The meetings brought together Western Balkan government officials as well as international donor representatives and ambassadors from a number of interested EU and OECD countries. The consolidated assessment and first policy recommendations for each policy area were discussed with the meeting participants and formed the basis for completing the draft economy profiles and regional chapters of the Competitiveness Outlook 2021 publication.

  1. 4. Review and publication phase (January – June 2021)

The 6 draft economy profiles and 16 regional chapters were shared with Western Balkan government officials and experts across the OECD, the European Commission, and regional expert organisations for comprehensive review. Their helpful feedback and comments helped to maximise the accuracy of the publication’s analysis as well as its salience for local policy makers. Subsequently, the OECD prepared the report for publication. The publication was released at the Berlin Process Summit on 5 July 2021.

The Competitiveness Outlook’s methodology is complemented by a participatory assessment process which has been designed to foster peer learning, to create consensus on reform priorities and to facilitate stakeholder co-ordination. The participatory basis of the publication is reflected in the high number and diversity of stakeholders included in the assessment and the Competitiveness Outlook meetings and roundtables (Table 2.3).

The involvement of these stakeholders strengthened the analysis in terms of data collection. In turn, the stakeholders engaged in gainful exchanges among themselves and with the OECD on concrete policy issues. This participatory approach is a particular strength of the Competitiveness Outlook, enabling the assessment process to:

  • catalyse positive organisational change through government self-assessment and roundtables which ask stakeholders to judge the success of current policies and resource allocations, while identifying possible directions for improvement

  • strengthen inter-ministerial consultation by asking Western Balkan government officials across various ministries and agencies to exchange and co-ordinate with one another as they assess the policy dimensions, as well as by bringing them together during economy-specific stakeholder meetings to reflect on the roles of their respective services in the policy dimension in question

  • encourage Western Balkan statistics offices and government bodies to produce new or more frequent statistics through the process’s demand for key indicators for assessing competitiveness

  • support regional integration and a regional perspective on policy issues by virtue of the regional virtual stakeholder meeting.

The Competitiveness Outlook 2021 possesses a number of strengths which make it a uniquely valuable tool for Western Balkan policy makers, citizens, researchers, as well as investors and international donor organisations. Table 2.4 lists the Competitiveness Outlook’s main strengths, as well as its limitations.


[3] Aiginger, K., S. Bärenthaler-Sieber and J. Vogel (2013), Competitiveness under New Perspectives, European Commission, Brussels, http://www.oecd.org/economy/Competitiveness-under-New-Perspectives.pdf.

[2] EC (2012), Operational Guidance for Assessing Impacts on Sectoral Competitiveness Within the Commission Impact Assessment Toolkit, a ’Competitiveness Proofing’ Toolkit for use in Impact Assessment, European Commission, Brussels, https://ec.europa.eu/transparency/regdoc/rep/2/2012/EN/2-2012-91-EN-F1-1.PDF.

[1] Hatzichronoglou, T. (1996), “Globalisation and Competitiveness: Relevant Indicators”, OECD Science, Technology and Industry Working Papers, No. 1996/5, OECD Publishing, Paris, https://dx.doi.org/10.1787/885511061376.

[4] OECD (2018), Competitiveness in South East Europe: A Policy Outlook 2018, Competitiveness and Private Sector Development, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264298576-en.

Metadata, Legal and Rights

This document, as well as any data and map included herein, are without prejudice to the status of or sovereignty over any territory, to the delimitation of international frontiers and boundaries and to the name of any territory, city or area. Extracts from publications may be subject to additional disclaimers, which are set out in the complete version of the publication, available at the link provided.

© OECD 2021

The use of this work, whether digital or print, is governed by the Terms and Conditions to be found at http://www.oecd.org/termsandconditions.