Annex A. Methodology for the 2019 Small Business Act assessment:

This section provides a detailed overview of the assessment methodology for the 2019 SME Policy Index.

Overview of the 2019 assessment framework and scoring

The process comprises two parallel assessments, a self-assessment by the government and an independent assessment by local consultants (see the Policy Framework and Assessment Process chapter).

The assessment grid is built upon the ten principles of the SBA, divided into 34 sub-dimensions. The sub-dimensions are usually also divided into thematic blocks, each with its own set of indicators. The thematic blocks are typically broken down into three components, representing different stages of the policy cycle: planning and design, implementation, and monitoring and evaluation (Figure A A.1). In few sub-dimensions where this approach is not applicable, for example in relation to the SME definition or the availability of some financial instruments within the access to finance dimension, thematic blocks may differ.

Figure A A.1. Dimension, sub-dimension and indicator level examples
Figure A A.1. Dimension, sub-dimension and indicator level examples

The approach of assigning scores to reflect different stages of the policy cycle, allows governments to identify and target the stages where they have particular strengths or weaknesses. Each policy dimension, and their constituting parts, is assigned a numerical score ranging from 1 to 5 according to the level of policy development and implementation, so that performance can be compared across economies and over time. Level 1 is the weakest level, whist the level 5 is the strongest, indicating a level of development commensurate with OECD good practices (Table A A.1). The levels are determined through a participatory and analytical process, which is conducted by means of a self-assessment by the government and an independent assessment by local consultants, supplemented by private-sector interviews.

Table A A.1. Description of score levels

Level 5

Level 4 plus results of monitoring and evaluation inform policy framework design and implementation

Level 4

Level 3 plus evidence of a concrete record of effective policy implementation.

Level 3

A solid framework, addressing the policy area concerned, is in place and officially adopted.

Level 2

A draft or pilot framework exists, with some signs of government activity to address the policy area concerned.

Level 1

No framework (e.g. law, institution) exists to address the policy topic concerned.

The assessment framework comprises qualitative and quantitative indicators, which are scored based on questions in the following forms:

  • Binary questions, allowing yes or no answers, such as “Does a legal definition of SMEs exist in your country?”

  • Multiple-choice questions, such as “Does a multi-year SME strategy exist?” or “Does an SME policy implementation body or equivalent exist?” These questions offer a drop-down list of answers, such as “strategy is in the process of development”, “draft strategy exists but not yet approved by the government”, “strategy exists, it has been approved by the government and is in the process of implementation”, and “there is no strategy in development”.

  • Open-ended questions, acquiring further descriptive evidence to add to the binary and multiple-choice questions. These questions are not themselves scored. Examples include “What is the budget for the SME implementation agency?”, “How many people work in the agency?” and “How many ministries are represented in the governance board?”

Whenever possible, economies are asked to support their answers with evidence, by providing source documents such as strategies and budget plans. Each questionnaire also includes definitions of some of the terms used in the questions, as well as references and examples to help respondents answer the questions.

Each of the core questions is scored equally within the thematic block. For binary questions, a “yes” is awarded full points and a “no” receives zero points. For multiple-choice questions, scores for the different options range between zero and full points, depending on the indicated level of policy development.

The scores for the core questions are then added up to provide a score for each component. Scores are initially derived as percentages (0-100) and then converted into the 1-5 scale. Scores for the thematic blocks are then aggregated to attain a score for the sub-dimension, with each thematic block being weighted based on expert consultation. In general, planning and design accounts for 30% of the score, implementation 50%, and monitoring and evaluation 20%, to emphasise the importance of policy implementation (Figure A A.2). Weights were applied at sub-dimension level and thematic block level in the same way.

Figure A A.2. Scoring breakdown per sub-dimension
Figure A A.2. Scoring breakdown per sub-dimension

Main changes to the assessment framework since the 2016 SBA assessment

Since 2016, the SBA assessment framework has been revised to provide a more in-depth analysis of SME polices by covering the cutting-edge aspects. By asking more detailed questions about some institutions and policy measures, the assessment allows the formulation of more concrete and targeted recommendations.

Five new sub-dimensions have been added to the assessment and others expanded and amended to include policy aspects that were not covered in the previous cycle. The policy dimensions and sub-dimension weightings have been adjusted to allow for these additions while maintaining as much comparability with the 2016 assessment as possible.

Table A A.2. Overview of changes to the SBA assessment

SBA principle

Related policy dimension

Key changes

1. Create an environment in which entrepreneurs and family businesses can thrive and entrepreneurship is rewarded

1. Entrepreneurial learning and women’s entrepreneurship

New questions were introduced relating to the cross-sectoral co-ordination of policies addressing women’s entrepreneurship, incentives for strengthening women’s participation in the formal sector, data availability and gender sensitivity of policies.

2. Ensure that honest entrepreneurs who have faced bankruptcy quickly get a second chance

2. Bankruptcy and second chance for SMEs

A new sub-dimension “Prevention measures” was introduced to better distinguish between pre- and post-bankruptcy procedures.

3. Design rules according to the “think small first” principle

3. Institutional and regulatory framework for SME policy-making

Holistic measures to fight against the informal economy with a view to supporting the formalisation of SMEs have been added to the assessment.

The assessment now pays more attention to the monitoring and evaluation part of legislative simplification and RIA.

4. Make public administration responsive to SMEs

4. Operational environment for SMEs

A new sub-dimension on tax compliance procedures for SMEs has been added to the assessment.

Monitoring and evaluation mechanism for company registration procedures are assessed.

5. Adapt public policy tools to SME needs

5a. Support services for SMEs

The questions were streamlined to acknowledge a more diverse set of answers. The assessment no longer evaluates whether economies’ strategy documents associated with BSS provision include both measurable targets and the expected impact of measures, but only if they include measurable targets.

5b. Public procurement

The assessment requests more information concerning general issues related to participation of SMEs in public procurement procedures.

6. Facilitate SME access to finance and develop a legal framework and business environment supportive of timely payments in commercial transactions

6. Access to finance for SMEs

Introduction of a more detailed analysis of the quality of leasing and factoring legal frameworks.

7. Help SMEs to benefit more from the opportunities offered by the Single Market

7. Standards and technical regulations

The previous sub-dimensions on Technical regulations, Standardisation, Accreditation, Metrology, Conformity assessment and Market surveillance have been consolidated into a single sub-dimension, Harmonisation with the EU acquis.

A new sub-dimension was introduced to examine SMEs’ access to standardisation.

8. Promote the upgrading of skills and all forms of innovation

8a. Enterprise skills

Questions on how SME skills are linked to the smart specialisation approach and the Economic Reform Programme were introduced. Different categories of training provision are now addressed: start-ups, women entrepreneurs, SME internationalisation and growth, family businesses, and training in green and digital skills.

8b. Innovation policy for SMEs

A new sub-dimension was introduced to recognise the importance of effective collaboration between SMEs and research institutions to enhance technology transfer and commercialisation of innovations.

9. Enable SMEs to turn environmental challenges into opportunities

9. SMEs in a green economy

An enhanced focus was put on examining whether the promotion of eco-efficiency and eco-innovation in the national strategic policy frameworks has been undertaken. The Incentives and instruments sub-dimension was updated to encompass a broader range of regulatory and other market-based instruments including performance standards, and environmental taxes and charges. Efforts on promotion and providing guidance carried more weight and were more thoroughly evaluated in this cycle of the assessment.

10. Encourage and support SMEs to benefit from growth markets

10. Internationalisation of SMEs

A new sub-dimension on e-commerce was added to assess whether programmes to promote the use of e-commerce and legal frameworks for e-payments and consumer protection are in place to secure the operational environment needed to support e-commerce.

Greater emphasis on the implementation of policies

In the previous assessment cycle, the assessed economies had made substantial progress in establishing policy frameworks across different areas. However, solid policy designs and institutional set-ups do not always translate into effective policy implementation. In order to better assess the effectiveness of SME policies, the assessment has been revised, to place more emphasis on their implementation.

The weightings have therefore been adjusted and the implementation stage of the policy cycle is now allocated a weight of 50%, compared to 45% in the previous assessment. While the weight assigned to the planning and design stage has fallen to 30%, the weight for monitoring and evaluation has remained the same.

Private-sector feedback has been included to gauge the outcome of policies

The new assessment cycle endeavoured to measure policy outcomes more effectively by paying more attention to the insights of representatives from the private sector and civil society on the effectiveness of SME policies and support measures. In addition to shedding light on the actual state of policy implementation, their feedback also helped to uncover any unintended effects of existing policies. Standardised interviews were used for SME owners and managers and representatives of chambers of commerce, while the views of civil society representatives were captured by the local consultants when they were filling out questionnaires.

The face-to-face interviews with SME owners and managers were in two parts. In the first part, the interviewees were asked screening questions to collect information about their firms’ characteristics. In the second part, interviewees were asked questions related to SME policies, covering the various SME Policy Index dimensions. Figure A A.3 illustrates the two-part structure of interviews with examples of questions for Dimension 2: Bankruptcy and second chance for SMEs. A detailed description of the interviews is presented in Annex C.

Figure A A.3. Structure of private sector interviews
Figure A A.3. Structure of private sector interviews

Statistical data have been collected to allow improved comparison of performance over time

As some economies have reached high levels of convergence with the SBA principles (Level 3 or above) and advanced levels of policy implementation, information about the performance of policies became more important. The revised approach has enhanced its use of several types of statistics, which complement the information acquired from the two assessments. The approach integrates national statistics, company-level data from other international organisations, or independent company surveys to measure the performance of policies on the ground. Table A A.3 provides an overview of the types of statistics and data collected, their purpose within the SME Policy Index, and their main sources.

The enhanced data collection also led to the greater involvement of national statistical offices in this cycle.

Table A A.3. Data types and sources

Type of data

Main purpose within the

SME Policy Index


Macroeconomic data and business statistics

Statistical information for the country reports

Support of the policy narrative written by the independent expert

National statistical offices

International databases and entrepreneurship indicator programmes (OECD, Eurostat, World Bank, etc.)

Statistics on policy outputs

Measurement of the policy intensity and the actual policy output

Measurement of policy convergence towards the EU average

Section on open questions in the SBA assessment questionnaires

Data from the SBA Factsheets

Survey data

Measurement of SME perceptions of the effectiveness and usefulness of certain SME policies

Measurement of policy convergence with the EU average

BEEPS Survey

Eurobarometer data from the SBA Factsheet

where available

End of the section – Back to iLibrary publication page