Chapter 2. Policy framework and assessment process
The ASEAN SME Policy Index framework aims to provide an independent and rigorous assessment of the policy environment for SMEs and to benchmark this assessment against the goals and actions identified in the ASEAN Strategic Action Plan for SME Development (SAP SMED) 2016-2025, as well as OECD good practice. It aims to provide guidance for policy reform and development on the basis of these findings.
While other indices and benchmarking exercises assess the business environment in Southeast Asia, the SME Policy Index adds value by going beyond the statistics to examine the policy environment for SMEs across a broad range of areas. In addition:
-
It focuses on the ASEAN region as it embarks upon further economic integration, enabling policy makers to identify challenges that may lie ahead.
-
It evaluates the SME policy environment comprehensively around the objectives of the SAP SMED 2016-2025, allowing for an assessment of progress.
-
It takes a participatory approach to evaluation and measurement through its involvement of policy makers, the private sector and partner organisations.
-
It provides guidance on how to improve the SME policy framework in each country through good practice examples and policy recommendations.
-
It incorporates relevant data by other organisations, such as the World Bank’s Doing Business report and Enterprise Surveys, as well as additional ERIA and OECD data, such as ERIA’s foreign investment liberalisation index scores and the OECD’s trade facilitation index scores.
-
It uses the country context and broader factors affecting SME development to complement the analysis that emerges from the scores.
However, the SME Policy Index methodology also has limitations:
-
The inputs and outputs of government policy can be hard to capture, particularly in countries where SME policies are mainly implemented by local government.
-
A lack of SME data in many Southeast Asian countries, as well as substantially diverse SME definitions, limit the comparability of data across economies.
-
The indicator weights are defined based on expert opinion, and therefore can be open to challenge.
-
Certain indicators may have been given special prominence due to their importance to ASEAN member states and the SAP SMED 2016-2025, rather than necessarily on the basis of expert opinion.
-
The assessment covers a hugely diverse group of countries that differ substantially in terms of income, geography, population, resource endowment and institutional capacity. The application of a single set of indicators and weights can therefore mask certain country-specific characteristics in the scoring. Attempts were made to address this constraint, such as graduated scoring and the exclusion of certain countries from indicators that were not applicable (for more information, see Annex A).
The 2018 assessment framework
The 2018 ASEAN SME Policy Index assessment framework maps and benchmarks SME policies across eight policy areas (“dimensions”), which are broken down into 25 components (“sub-dimensions”). These components were developed in reference to the SAP SMED 2016-2025 as well as the more generic SME Policy Index framework developed by the OECD. A repeated application of the framework, at least over the duration of the SAP SMED 2016-2025, should provide a dynamic picture of SME policy reforms and priorities over time.
The 2018 ASPI assessment framework differs significantly from the pilot framework used in 2014. While the pilot also featured eight dimensions, it touched upon different policy areas, some of which have become sub-dimensions in the new assessment. It also covered fewer and different indicators, and this limits the comparability of the two assessments.
A detailed description of the assessment framework, including changes from the 2014 version, is presented in Annex A.
The assessment process
The SME Policy Index is based on the results of two parallel assessments. A self-assessment is conducted by governments, led by a predesignated “national co-ordinator” (a government official assigned to lead the policy assessment). This self-assessment is informed by inputs collected from the various agencies and ministries involved in SME policy development and implementation. In addition, an independent assessment is conducted by the OECD and ERIA based on inputs from a team of local experts who collect data and information and conduct interviews with key stakeholders and private sector representatives, led by a predesignated “national consultant”. In the 2018 exercise, a number of national consultants collaborated with national co-ordinators on information gathering due to the difficulty of obtaining data in some countries. Preliminary scores were developed on the basis of these two assessments, enhanced by further desk research, missions and consultations with government representatives conducted by the OECD and ERIA. A series of stakeholder meetings were subsequently held in each economy to discuss and compare the two parallel assessments and to help reduce bias and misjudgements. These meetings were typically attended by 30-50 SME policy stakeholders, including representatives of ministries and government agencies, international donors, civil society, the academic community, NGOs and the private sector. At these meetings, discrepancies between the parallel assessments were discussed, information gaps filled and draft scores presented. Based on the information gathered, the OECD and ERIA determined the final scores for each country and presented them to the national co-ordinators and consultants at a regional meeting held at the end of the assessment process.
A detailed description of how the scoring was conducted, including innovations since the 2014 assessment, is presented in Annex A.
Timing of the 2018 assessment
The 2018 ASPI assessment was carried out between 1 July 2016 and 31 October 2017 in three phases:
-
Design phase (July-December 2016). The methodology and assessment framework were prepared in consultation with the ASEAN Co-ordinating Committee on Micro, Small and Medium Enterprises (ACCMSME), drawing reference from the OECD SME Policy Index methodology and the 2016-2025 SAP SMED. National co-ordinators and national consultants for each country were identified.
-
Data collection and evaluation phase (January-October 2017). ASEAN countries carried out a self-evaluation of their policy frameworks (through an assessment questionnaire and other materials). This was complemented by an independent assessment carried out by the OECD and ERIA with the support of local experts. Stocktaking missions and workshops were held in all ten countries to support the data collection exercise. Desk research and follow-up with relevant stakeholders were used to fill information gaps and resolve inconsistencies. Finally, a workshop was conducted in Jakarta in October 2017, followed by a presentation of the initial results at the fourth ACCMSME meeting, held in Manila in October 2017.
-
Consolidation and publication phase (November 2017-July 2018). Additional data were gathered, the draft report was prepared and the report was reviewed by participating countries, the ACCMSME and the ASEAN Secretariat. The draft report was presented at the fifth ACCMSME meeting, held in Luang Prabang in April 2018. Following the meeting, final comments were integrated into the report. The completed publication was presented at the Senior Economic Officials Meeting in July 2018 and formally launched at the ASEAN Economic Ministers’ Meeting in August 2018. A series of training workshops and national dissemination events were due to be conducted after the launch.
The 2016-2025 ASEAN Strategic Action Plan for SME Development
The ASEAN Strategic Action Plan for SME Development 2016-2025 was developed in the context of the ASEAN Economic Community (AEC), which was established in 2015. The commitment to co-operate on SME development falls under the AEC Blueprint’s third pillar, “equitable economic development”. It represents ASEAN’s recognition that SMEs may require additional support to take full advantage of the trade and investment opportunities that are expected to come from further economic integration, as well as to increase their competitiveness. This is regarded as key to narrowing the development gap across and within ASEAN member states.
Built around five strategic goals and twelve desired outcomes, SAPSMED 2016-2025 proposes a set of time-bound actions and action lines with key performance indicators (KPIs). Its main objectives are to: i) promote productivity, technology and innovation; ii) increase access to finance; iii) enhance market access and internationalisation; iv) enhance the policy and regulatory environment; and v) promote entrepreneurship and human capital development. It was launched at the ASEAN Business and Investment Summit in November 2015.