2. Evidence-based policy making and stakeholder engagement

Regulations, and the process of making them, are expected to reflect the needs and reality of society, but they also ought to adapt and react quickly to changes. This adaptation is more likely when the systems and practices for creating and improving regulations are fully embedded into the country’s decision-making processes, rather than viewed as a bureaucratic afterthought.

This chapter provides an overview of the trends and progress of OECD countries in the implementation of the 2012 OECD Recommendation of the Council on Regulatory Policy and Governance, with a particular focus on the use of regulatory management tools, particularly stakeholder engagement, regulatory impact assessment (RIA) and ex post evaluation of regulations. Sound regulatory management practices help to create an environment that fosters better regulations, which in turn can improve economic performance. In particular, this entails consideration of whether to regulate and of alternative options; assessment of regulations before their drafting; enactment or modification; evaluation of existing regulations to make sure that they are reaching the objectives for which they were created – especially when they are developed without a previous assessment, like in times of crisis –; and the constant involvement of stakeholders throughout these processes.

The chapter further describes how countries provide their citizens with mechanisms to challenge existing regulations. The right to challenge laws is a central democratic right of citizens. Avenues to challenge regulations should be available on both their creation and their legitimacy.

The chapter also summarises the arrangements put in place by jurisdictions to seek regulatory coherence across all levels of governments, and to promote and implement regulatory management practices at the subnational level. Lower levels of governments may establish additional layers of regulation, and/or can be responsible for implementing regulations issued at the national level. Hence, regulatory policies and tools should also be adopted by regional and local governments, and mechanisms should exist to aim for a frictionless regulatory framework across levels of governments.

The implementation of the remaining principles in the Recommendation related to oversight and performance evaluation, international regulatory co-operation, governance of regulators and risk are discussed in-depth in the subsequent chapters of the Outlook.

Most of the analysis of any trends and improvements is based on the results from the 2021 OECD Indicators of Regulatory Policy and Governance (iREG) survey, which covers 38 OECD members and the European Union. The survey was first conducted in 2014, then in 2017, and now for a third time in 2020-21. This allows for comparative analysis of the adoption and implementation of better regulatory practices not only across different countries, but also over time. Composite indicators summarising key information on stakeholder engagement and the use of evidence in the development and revision of regulations will be prepared (Box 2.1).

The OECD 2012 Recommendation of the Council on Regulatory Policy and Governance (OECD, 2012[1]) (Recommendation) advises governments on the effective use of regulation to achieve better social, environmental and economic outcomes. It provides practical measures to assess countries’ capacities to develop, implement, and review quality regulations in 12 principles.

No country has fully implemented the Recommendation. Findings from the Indicators of Regulatory Policy and Governance suggest that the current pace of implementation is too slow. Projections based on the survey results indicate that countries which are at the bottom of implementation would need more than 30 years to catch up to more advanced countries. Even those more advanced have a long way to go to fully implement the Recommendation. All countries therefore need to increase the speed of reform and invest more in solid regulatory policy to ensure their regulations are evidence-based and work in practice.

On average, countries have made small improvements in the adoption of regulatory management tools since 2017, with larger changes in ex post evaluation than for RIA and stakeholder engagement. This is unsurprising as ex post evaluation remains the least developed regulatory management tool overall, leaving it with the largest potential scope for reform.

It is worth noting that despite the overall slow pace of change, some countries have made more substantive changes in their regulatory management practices overall since 2017.

  • Chile has made important improvements to its regulatory management tools over the last years. In 2019, Chile adopted Presidential Instructive No. 3/2019, which broadens the requirement to conduct RIA, making it mandatory for all primary laws initiated by the executive and for all subordinate regulations. There is now a threshold for conducting RIAs, which will determine whether a standard or high impact RIA should be conducted. RIAs are now required to consider alternative non-regulatory options and a range of specific impacts. Public consultations are now required for major regulatory proposals for which a high impact RIA is to be conducted.

  • Greece has introduced Law 4622 in 2019, which further embeds regulatory management tools into the rule making process for primary laws. A list of laws to be prepared or modified is now published in advance. The guidance on regulatory impact assessment (RIA) for primary laws has been updated and now includes guidelines on how to conduct stakeholder engagement. Additional categories of regulatory costs shall be quantified, and regulators shall assess the regulatory impacts on a larger range of factors, including gender equality. Draft primary laws are now frequently posted on the consultation portal.

  • Latvia’s recent reforms cover all three regulatory management tools. Public consultations are now systematically conducted at a late-stage of policy development; written guidance is now available to assist officials on the preparation of RIA; and regulatory stock reviews are required for some subordinate regulations.

  • SMEs in the Netherlands are now engaged in the early stages of the development of a regulation as part of an SME Test. New guidelines have been issued on the impacts on borders regions, gender equality and developing countries and the Sustainable Development Goals. The regulatory oversight body is now responsible for reviewing the quality of ex post evaluations of regulations, and has developed a toolbox with guidance for officials conducting policy evaluations.

  • The Government of Portugal has recently undertaken a range of key reforms to implement and strengthen regulatory impact assessments. Regulatory alternatives as well as an increasingly broad range of impacts are now required to be analysed, and the scrutiny of quality of RIA for subordinate regulations has been reinforced. A new central consultation platform has been introduced for subordinate regulations, which is only used for late-stage consultation when there is a regulatory draft.

  • Spain has strengthened its RIAs through the creation of a dedicated body. The Regulatory Coordination and Quality Office is in charge of promoting the quality, co-ordination and coherence of regulatory management tools. The transparency of the consultation process is improving with a new centralised platform lists all ongoing consultations. The platform also provides access to the annual regulatory planning agenda.

Regulatory management tools remain mostly focused on laws initiated by the executive. In the majority of OECD members, there is no requirement to conduct neither consultation with the general public nor RIA to inform the development of primary laws initiated by parliament. This is not necessarily a problem in instances where the executive is responsible for initiating the vast majority of laws, but that is not the case for all OECD members. Differences in law-making procedures mean that initiating laws through parliaments could be considered as a route that bypasses regulatory requirements. The OECD has previously suggested that parliaments should be encouraged to set up their own procedures to guarantee the quality of legislation, such as consultation, RIA, and ex post evaluation (OECD, 2015[3]). Despite this, the number of OECD members with requirements for laws initiated by the parliament has changed little since 2014.

Countries improved their stakeholder engagement practices with respect to subordinate regulations to a greater extent than primary laws. Systematic adoption improved through new requirements to conduct stakeholder engagement, and through conducting late stage consultation more frequently. Improvements in the oversight and quality control of stakeholder engagement, by having oversight bodies in charge of promoting and scrutinising consultations, account for the improvement. For primary laws, there have been very slight improvements on the methodology of stakeholder engagement, which can be explained by the increased use of virtual meetings for consultations, and different documents being made available during consultations.

Countries that undertook substantive changes since 2017 include Chile, Colombia, Costa Rica, Greece, Iceland, Latvia, the Netherlands, Norway and Spain.

  • In Chile, following a new decree, public consultation is now required for the development of subordinate regulations for which a high impact RIA is to be conducted. Some ministries now publish annually a list of regulations that they plan to review, consolidate, modify or enact, providing the opportunity for the public to provide comments and feedbacks on the plans. A central website links to open consultations and reform plans of each ministry.

  • Colombia developed SUCOP, a digital platform that aims at centralising stakeholder engagement practices across all government entities. At the same time, ministries continue using their own websites for consultations.

  • Costa Rica has expanded stakeholder engagement practices, such as forward planning and a more intensive use of the SICOPRE, which is a centralised webpage that makes regulatory impact assessments (RIAs) and public consultations available. It also allows for comments by the public, to which regulators respond.

  • Greece has increased the frequency with which it posts draft primary laws on its consultation portal for the public to comment on and it now publishes a list of laws to be prepared or modified in advance. It has also developed written guidance on how to conduct stakeholder engagement for primary laws.

  • Iceland has significantly improved its consultation system over the past years. It launched a new centralised interactive consultation website where stakeholders can provide their comments both at the early and late stages of the consultation process. The website now also provides access to preliminary RIAs, draft regulations, and a summary explaining how comments impacted the proposal. Additionally, Iceland encourages the participation of the general public through social media for some consultations, and the public can register to receive e-mail alerts when new consultations are posted online.

  • Public consultations in Latvia are now systematically conducted at a late-stage of policy development and stakeholders benefit from having a broader range of supporting material to help focus their input into policy proposals.

  • Netherlands now offers written guidance to policy makers on how to conduct stakeholder engagement. In the past three years it began to carry meetings at an early-stage of policy development with SMEs, as part of their SME Test.

  • In Norway the regulatory oversight body has strengthened its capabilities to scrutinise regulatory proposals and provide comments on stakeholder engagement activities.

  • Spain now lists all ongoing consultations on its centralised online platform and allows citizens to engage before regulatory development starts and at the draft regulation stage.

Citizens can offer valuable inputs on the feasibility and practical implications of regulations. Meaningful stakeholder engagement can lead to higher compliance with regulations, in particular when stakeholders feel that their views have been considered. From a regulatory policy perspective, this entails granting members of the public sufficient opportunity to help shape, challenge, and reform the regulations that they encounter in their daily lives. Too often we witness examples where decisions are made without the involvement of those affected, much to the detriment of society (Chapter 1).

Everyone is affected by laws including citizens, businesses, consumers, and employees (as well as their representative organisations and associations), the public sector, non-governmental organisations, international trading partners, and other stakeholders that can also be disadvantaged, or less influential groups (OECD, 2012[1]). Policy makers should be aware that these groups have different means (e.g. in terms of resources and time) and should tailor engagement strategies accordingly to ensure that all voices have the opportunity to be heard (OECD, 2009[4]; OECD, 2015[5]).

The OECD 2012 Recommendation on Regulatory Policy and Governance provides three broad tenets of communication, consultation, and engagement that policy makers should adhere to (Box 2.2).

The following sections will refer to either early or late stage consultation. Early stage consultation is at a point in time where policy makers have identified that a public policy problem exists and are considering various ways to solve it. Late stage consultation is at a point in time where the decision to regulate has been made.

A clear overarching strategy that outlines how communication will take place, and what information will be communicated to stakeholders is important. The strategy helps to set community expectations about the channels and forms of communication that will be used. It can also be used to hold policy makers to account.

How governments communicate with affected parties is crucial to receiving input as part of the development of regulations. This looks to the various means of communications used by policy makers when engaging with stakeholders. The most appropriate means of communication will vary depending on the policy proposal at hand, the resources and capacities of the affected stakeholders, as well as the stage in policy development.

Continuing a larger digital government trend (OECD, 2014[6]), the majority of OECD members list consultations on a central platform, acting as a single entry point for stakeholders. Countries have taken a variety of approaches when designing central platforms (Box 2.3). Information on public consultation webpages can be tailored to better signal to stakeholders the types of consultations that might be of relevance to them. For instance in Spain, the consultation webpage clearly differentiates between proposals that seek input on the policy problem and potential solutions, and those where a preferred regulatory option has been identified and a draft regulation is available.

Governments can help to direct stakeholders’ attention to where it is needed most to support policy making. For instance, it may make sense at a more nascent stage to present a policy problem and direct stakeholders’ feedback to help determine potential solutions. Apart from potentially improving regulatory design, this also can improve regulatory “outputs” such as improved compliance rates, desired behavioural changes in market participants, and improved trust in government. Moreover, improving regulations has the potential to improve economic performance by fostering a more competitive and inclusive society (OECD, 2009[4]; OECD, 2017[7]).

Consultations may be better suited to focus on implementation issues at later stages of regulatory design. Generally at this point the decision to regulate has been made and there is limited scope to change the preferred regulatory path identified. Usually at this stage a draft regulation is made available for stakeholders to view and comment on. Stakeholders can still provide valuable input to improve the efficacy of regulations at this point, e.g. by highlighting competing or inconsistent objectives, and raising compliance and enforcement issues. In both cases, the way in which questions are put to stakeholders matters as they can affect respondents’ behaviour and answers (Box 2.4).

Consideration needs to be given to the modes of communication. The format of consultation may seem, in and of itself, relatively unimportant in the policy making process. However it plays a vital role in ensuring that stakeholders can understand the input sought. Resilience, creativity, agility and adaptation on the forms of consultations are needed for regulations to fully benefit from stakeholders’ feedback (OECD, 2020[11]). Policy makers need to tailor the forms of consultation to, for instance, reflect widespread industry standards (e.g. paper, online, or a variety of forms), cognisant that not all affected stakeholders universally utilise the same communication forms. Inappropriate or underutilised forms of communication runs the risk of excluding stakeholders from the policy making process, thereby undermining a sense of shared ownership, and potentially adversely affecting compliance and trust in eventual regulations (Lind and Arndt, 2016[12]).

The form of consultation adopted by OECD members differs depending on whether the consultations are at an early or late stage of policy development. At an early stage, consultations are more often undertaken on formal and informal bases with selected groups, as well as meetings with advisory groups or preparatory committees (Figure 2.3). The results suggest that consultation at this stage is more selective rather than open to the general public. Such an approach may be appropriate for technical or complex regulatory proposals where expert input is sought, and by its nature tends to be found only across limited groups. For proposals of a more general nature where policy makers are unsure about the magnitude or distribution of impacts it remains important to elicit feedback from a broad range of stakeholders, particularly as some may be “unknown” to policy makers at this stage since the complete range of impacts have yet to be ascertained.

It is most common for OECD members to conduct public consultation over the internet with invitation to comment, as well as other forms of open consultation, for late stage consultations (Figure 2.3). Similar to 2017, around 30% of countries systematically use at least one form of public and open consultation approaches to stakeholder engagement at a late stage for both primary laws and subordinate regulations.

Since 2017 there has been an increase in the use of virtual meetings to engage with stakeholders in some countries, at both early and late stage consultations (Figure 2.3). This reflects the current circumstances as a result of the COVID-19 pandemic which has made some more “traditional” forms of consultation impossible. It will be interesting to see if virtual meetings act as a complement to other methods or starts to replace them once the recovery from the pandemic is underway. While providing more conduits helps to bolster the inclusiveness of consultations, they need to be financially justified. On the other hand, if virtual meetings become the norm, it will remain important to consider whether certain groups of stakeholders still predominantly rely on alternative means, and to find solutions to ensure that they are not unduly excluded from consultation processes.

Generally consultations should be made available to all citizens (OECD, 2017[7]). Beyond that, specifically determining who to consult with effectively means deciding who should be excluded from the consultation process. There may be justified reasons for limiting consultations due to factors such as confidentiality, the subject nature of the proposal (e.g. if it is highly technical, or if expertise lies in only limited areas), and for genuine matters of expediency (although this should not be used as a default excuse to avoid consulting).

Consultation should not be limited too early in the policy development process. At an early stage, the potential impacts of proposals may not be known with certainty, and therefore all potentially affected stakeholders may not be known either. As policy development matures, consultations may be more focussed as alternative options are fully explored and the various impacts assessed. At a late stage of development the group of affected stakeholders may further diminish, for instance, because of the regulatory design itself (e.g. the draft regulation excludes SMEs), by the imposition some sort of threshold or filter (e.g. a regulatory proposal is focussed on only businesses in excess of a certain turnover, amount of pollution emitted etc), or by geographic or locational restrictions.

It is important to allow for sufficient checks and balances within a consultation process. For instance, there is a risk that consulting the “usual suspects” leads to the “usual answers”. Policy makers can be assisted in identifying vested interests (and thereby reducing risks of regulatory capture) by consulting broadly allowing other stakeholders to challenge positions put by the “usual suspects” (OECD, 2012[1]).

Stakeholders can themselves form groups to help present more unified and strengthened positions from the views expressed. The groups can help act as a conduit to collect comments from individual affected stakeholders. OECD members have reached out to these groups as well as formed groups of their own with specific stakeholders (Box 2.5).

Deciding when to consult is a central facet of decision making. Generally there are four distinct stages of consultation: to inform the community in advance; at the early and late stages during policy development; and on the revision and modification of existing laws (OECD, 2012[1]). Establishing when to consult can be of critical importance to the design of the resultant policy: too early and stakeholders may not be able to help identify potential solutions; too late and stakeholders may feel that consultation is an obligatory step for policy makers in order to progress their policies to the decision stage. The solution is to get consultation “just right”, but it does not follow that it is necessarily between the early and late stages. As noted above, early and late stage consultation are both important in their own right.

It is not necessarily appropriate to consult at each stage for every regulatory proposal. For instance, providing advanced notice about closing an identified regulatory failure may lead to worse social outcomes than consulting only at a later stage because of the socially undesirable behaviour it may incentivise. In a similar vein, where there are strong continued links between policy makers and stakeholders there may be less of a need to conduct more formal consultation during earlier stages of policy development as both parties are well informed. However it would generally be expected that consultations take place at a later stage of policy development.

OECD members do not yet systematically allow for the participation of stakeholders in the development of regulations throughout the policy cycle. Most OECD countries consult with stakeholders on draft proposals, but only a few consult systematically at an early stage (Table 2.1), a situation that has not improved in the last few years.

Announcing forthcoming consultations assists stakeholders to organise themselves in order to focus their efforts on the consultations that affect them the most. Receiving better organised information from stakeholders can help to improve policies from the outset. It may mean that potential alternative solutions can be ruled out earlier in the development process than they otherwise would be, thereby saving time, resources, and consultation energy with affected parties. More fundamentally it demonstrates a strong adherence to the principles of open government (OECD, 2017[7]).

Announcing in advance that consultations will take place remains an uncommon practice across the OECD. Only six OECD members and the EU do so for all consultations on primary laws, and even less do so for all consultations on subordinate regulations (Figure 2.4).

It is also uncommon that stakeholders are informed in advance that evaluations of existing regulations will take place (see Figure 2.4). Only six countries and the EU systematically inform the public in advance of ex post evaluations that will be carried out, while five countries do so only for some of their planned ex post evaluations. As an example, Canada publishes a regulatory review plan two years in advance, and ministries in Italy are required to also publish a two-year plan of their upcoming ex post evaluations.

OECD members adopt different communication approaches when informing the general public or particular stakeholders about forthcoming consultations. Some countries use websites to announce future consultations, while others publish a road map or other early-warning document. For instance, members of the public can sign-up on both the UK and European Commission’s websites to receive email alerts about upcoming public consultations. Similarly, Estonia uses an automatic alert system from a dedicated Information System for Legislative Drafts, while the Slovak Republic publishes a set of preliminary information regarding regulatory proposals, including contact information and the dates of expected public consultation. At a more general level, around about two-thirds of OECD members publish a list of primary laws that they plan to prepare or modify, while almost half do so for subordinate regulations (Figure 2.5). In some cases these plans are open for consultation, like in Costa Rica, where all executive entities are required to publish their plans for Better Regulation that present the planned administrative procedures to be modified annually. Even though this is not an advance notice of upcoming consultation, it alerts the public of upcoming amendments to regulations where the public can ask to participate in consultations if they are not made available.

Minimum consultation periods help to ensure that stakeholders have the opportunity to provide input on regulatory proposals. That said, some operational flexibility is required to ensure that the period given is appropriate for each proposal at hand. More than two-thirds of OECD members have formalised minimum consultation periods, and generally range from as little as eight days to as much as 12 weeks. For instance, in Belgium, some consultations are open from four to six weeks, and Sweden, Switzerland and the European Commission have a minimum period of 12 weeks. Minimum periods do not systematically apply to consultations for all regulations, as countries decide on the parameters. For instance, in Chile proposals where a high impact RIA is required, the period of consultation should be at least 10 days.

Engaging with stakeholders allows policy makers to question, consider, test, and revise different approaches to a policy problem, understand citizens’ and other stakeholders’ needs and improve trust in government (OECD, 2016[13]). Viewed in this way, an iterative approach to consultations is essential. Facilitating consultations through multiple rounds assists policy makers overcome stubborn resistance to change (e.g. from incumbent vis-à-vis potential firms), enable participants to see how they have influenced policy development, and build a shared sense of policy ownership, hence the use of the term “engagement” (OECD, 2017[7]; OECD, 2015[14]).

One of the starting points of engagement is to publish comments received during consultations. Publication helps to demonstrate an open and inclusive approach to policy making. It also assists in assessing the robustness of evidence and information presented by stakeholders (either by policy makers, other stakeholders, or the public at large).

A majority of OECD members publish participants’ views from consultation processes, either as the comments themselves, or as an online summary (Figure 2.6). Switzerland for example publishes a report summarising comments received, as well as every comment received (even those of individuals); regardless of the number. In some OECD countries, such as Iceland and the Netherlands, consultations are systematically conducted on interactive websites where participants can see “live” comments from other stakeholders and provide direct feedback. In addition to consultation websites, the UK and Canada have also relied on social media platforms (e.g. Twitter, Facebook, etc.) to conduct consultations, where comments received are publicly available.

Decision makers are usually made aware of consultation results. Only in a minority of OECD members are consultation results not provided to decision makers (18% for primary laws and 16% for subordinate regulations, respectively). In 2021, close to two-thirds of OECD members included the views expressed in the consultation process in the resulting RIA; a surprising result given that only one-third of countries are policy makers required to consider consultation comments when developing final proposals.

Responding to comments received is an integral aspect of strong stakeholder engagement (Lind and Arndt, 2016[12]). This includes explaining how they were taken into account for the revision of the draft regulation and, when relevant, explaining their exclusion. Policy makers’ treatment of stakeholders’ input can either encourage or discourage stakeholders’ decisions to participate in future consultations.

Responding to consultation comments remains rare across the OECD (Box 2.6). Currently, less than one-third of countries systematically require a public response to consultation comments; however, in some countries there are public responses to comments even in the absence of a requirement. In those OECD members that have invested in interactive websites, policy makers also provide reactions and comments to stakeholder views as part of an effective “live debate” on policy proposals. Such an approach has brought stakeholders (and citizens more broadly) closer to decision makers, helped boost transparency and accountability, and reduced the transactions costs of consultations. It may have also helped to boost the level of stakeholder engagement on future proposals.

The COVID-19 pandemic has made it virtually impossible to carry out physical consultations. The necessary speed of government action, and the rapid evolution of the situation, has shaped engagement with stakeholders and public consultations for many governments. The OECD surveyed countries’ initial responses to the COVID-19 pandemic up until mid-September 2020.

Several OECD members made use of exception clauses that were already part of their consultation requirements before the pandemic. Such exceptions give regulators certain flexibility in the event of emergency situations without the need to formally bypass requirements. The Netherlands and Norway reported it was not necessary to bypass consultation requirements due to the COVID-19 pandemic because exceptions already existed. As Norway explained, its stakeholder engagement guidance document states that circulation for consultation of draft laws and regulations may be omitted if measures require swift implementation to avert serious outcomes with regard to life, health and the environment.

Many OECD members that made use of exemption mechanisms also introduced some form of emergency legislative procedures for putting in place crisis responses, thereby leaving less time for scrutiny by stakeholders. Along those lines, Finland reported that open consultation has often been conducted before introducing COVID-19 response measures, but that a shorter time period for such stakeholder engagement applied. Several OECD members made use of similar built-in exemption mechanisms, among which are Korea, Luxembourg, New Zealand and Switzerland.

In contrast, there are OECD members that reported that they did not change any of their consultation practices or requirements during the pandemic. Israel reported a number of public consultations on regulatory crisis responses, which followed regular rules of procedure. According to the government, Israel has not shortened its formal requirement for a minimum period for consultations with the public, including citizens, business and civil society organisations. The European Commission actually extended its public consultation periods to allow stakeholders more time to organise their responses. Around two-thirds of the public consultations undertaken by the Commission between June and end of September 2020 have been extended by one to eight weeks.

Some consultation with stakeholders has intensified due to the pandemic, as some regulators have relied on input from regulated industries to design emergency regulations. For instance, the UK Office of Communication (Ofcom) has reported holding more roundtable meetings between the regulator, government and industry in the crisis period than in the preceding six months.

As seen in other public sectors, the need for social distancing had an enhancing effect on the digitalisation of administrations and government. Poland, like other countries, reported that for the first time during the pandemic, public consultations were held in the form of videoconferences.

On average, OECD members’ RIA practices improved slightly in relation to subordinate regulations. The biggest development for subordinate regulations stemmed from improvements in oversight and quality control as well as, to a lesser extent, from more transparent RIA procedures. Although on average there has been little improvement in relation to RIA conducted on primary laws, some OECD members have nevertheless undertaken recent reforms.

Countries which made substantive changes to their RIA system include: Chile, Greece, Israel, Latvia, Portugal, and Spain.

  • Chile adopted a new Presidential Instructive that made RIAs mandatory for all primary laws initiated in the executive and for subordinate regulations. It now requires that RIAs consider alternative non-regulatory options, the likely distributional effects of proposals and a range of factors, including impacts on competition, small businesses, trade, environment and gender equality.

  • Regulators in Greece are now required to assess and quantify the impacts of regulations on a large range of factors, including gender equality and social goals.

  • Israel established its Better Regulation Department, which is entrusted with overseeing RIAs.

  • Latvia’s recent substantive reforms include a requirement to assess a wider range of costs in RIAs, such as financial, budgetary, and administrative costs, as well as an expectation to prepare RIAs early in the policy-making process to later undergo public consultation with the draft law.

  • Portugal formally established the use of RIA and has since expanded it, particularly for subordinate laws. Portugal has also reinforced the scrutiny of quality of RIA for subordinate regulations.

  • Spain too has introduced bodies whose functions include watching over the legal quality of regulations initiated by the executive and providing feedback and recommendations on Impact Assessments to regulators.

Regulatory Impact Assessment (RIA) is a central aid to decision making, helping to provide as much as possible objective information about the likely benefits and costs of particular regulatory approaches, as well as critically assessing alternative options – including non-regulatory ones. A well-functioning RIA system can assist in promoting policy effectiveness, efficiency, and coherence by clearly illustrating the inherent trade-offs within regulatory proposals. It does this by showing the efficiency and distributional outcomes of regulation. RIA also has the ability to reduce regulatory failures: for example RIA can illustrate that reducing risks in one area may create risks for another, or that unintended consequences would exceed any of the purported benefits. RIA can also reduce regulatory failure by demonstrating where there is no case for regulating, as well as highlighting the failure to regulate when there is a clear need.

Building on the OECD 2012 Recommendation of Regulatory Policy and Governance (OECD, 2012[1]), the OECD has recently published Best Practice Principles on Regulatory Impact Assessment (OECD, 2020[15]). These principles provide more detailed information and guidance for member and non-member countries on the critical elements required to develop and sustain a well-functioning RIA system (Box 2.7).

OECD members have recognised the importance of RIA: even in 2014, only two members for primary laws and one for subordinate regulations did not have a formal requirement to conduct RIA in place. In 2021, all OECD members now have a requirement in place to conduct RIA on at least some laws, and there has also been a slight rebalancing as members move away from a blanket requirement to a more proportionate approach (see below for further details). A gap remains between a requirement to conduct RIA and what actually happens in practice, albeit that gap has slowly reduced since 2014 (Figure 2.9).

Forthcoming OECD (2021[16]) research will explore some of the behavioural barriers that may be limiting the use of RIAs (and the other regulatory management tools), as well as pose some possible behaviourally-informed solutions that may help close this gap. Barriers may include: limited ability to focus attention on using RIA effectively, beliefs or motivations that guide users in certain directions, require assistance in making the right choice regarding the type of RIA to conduct, and determination to continuing using the tool effectively over time. A number of behaviourally-informed solutions are identified based on well-established insights, including the use of reminders, default settings, committing to implementing certain actions, diversifying teams, and reframing perceptions around the use of the tool. Future research intends to explore these barriers and solutions in real-world regulatory governance settings to demonstrate what could work in practice.

Considering all feasible options when potentially embarking on regulating is crucial to ensure that the broadest possible range of alternatives are genuinely considered by policy makers. This was recognised in the OECD 2012 Recommendation of Regulatory Policy and Governance (OECD, 2012[1]): “Ex ante assessment policies should include a consideration of alternative ways of addressing the public policy objectives, including regulatory and non-regulatory alternatives to identify and select the most appropriate instrument, or mix of instruments to achieve policy goals. The no action option or baseline scenario should always be considered.”

The best practice principles complement the Recommendation by noting that RIA more generally is an iterative process (OECD, 2020[15]). This is certainly the case for the consideration of alternative options, as options are gradually ruled out as more information on their potential impacts becomes available; or where stakeholders identify that certain options proposed are not feasible – and also the possibility that stakeholders may raise alternative options not considered by policy makers. Developing RIA in co-operation with relevant stakeholders can help to improve policy design. In 2020 the Australian Government announced its intention to raise the cyber security and resilience of its critical infrastructure system. The RIA highlighted the critical infrastructure assets, possible positive security obligations for critical infrastructure owners and operators, and mandatory cyber incident reporting. Over 3 000 people shared their views and more than 350 submissions were received. As the RIA progressed, significant refinements were made to the policy as a result of the consultation, adding for example clarity to the definitions of critical infrastructure sectors, and extended timeframes for reporting cyber security incidents (Australia, 2020[17]).

Notwithstanding the Recommendation, today four OECD members do not require their regulators to consider the no action or baseline scenario, with an additional five that only require a baseline assessment for some subordinate regulations (Figure 2.10).

Failing to consider the no action or baseline scenario represents a significant weakness in the approach to RIA. First, it means that any alternatives cannot be meaningfully compared as the starting point (i.e. when the regulatory intervention began) is unknown or uncertain. Second, it creates little incentive to capture accurate data and changes in market participants’ behaviour because the starting point is unknown (both of which are crucial for monitoring and evaluation). Third, it means that evaluating government intervention is more difficult – policy makers will not know whether a regulation has succeeded or failed in achieving its goals as there is no counterfactual to compare against (see ex post evaluation section below). Fourth, it undermines one of the key benefits that the RIA process provides – an evidenced-based assessment of a complete range of feasible options presented to decision makers on a common basis.

Since 2014, around 85% of OECD members have had systematic requirements in place to identify and assess the impacts of the preferred regulatory option (Figure 2.11). The results have remained relatively stable and help to ensure that decision makers are aware and informed of the likely implications that will flow from the preferred regulatory approach.

Alternative regulatory options for primary laws are required to be systematically identified and assessed in around 80% of OECD members, with little change since 2014. This suggests that decision makers generally benefit from having information about alternative regulatory paths that could be taken to solve the policy problem at hand, albeit RIAs are slightly less likely to contain this information when compared with the preferred regulatory option.

Results from the Indicators of Regulatory Policy and Governance Survey suggest decision makers do not have the same quality of information at their disposal when making decisions about whether to regulate or whether a non-regulatory approach may be more appropriate. Alternative non-regulatory options are required to be identified and assessed substantially less than regulatory options. Around 70% of OECD members for primary laws – and just over half for subordinate regulations – have a requirement that proposals systematically identify and assess the impact of alternative non-regulatory options. Furthermore, where non-regulatory options are considered, results from the iREG survey indicate that generally there is only one non-regulatory option considered by policy makers (Figure 2.12).

The European Commission refined its consideration of baseline and regulatory options during the COVID-19 pandemic: Wherever the crisis might have had a significant bearing on the sector or policy area a regulatory initiative addresses, the Commission considered these effects in the baseline as well as in the preferred regulatory options. Any impact assessment would need to compare the impacts of a policy option against a regular baseline, not taking into account the pandemic, as well as a refined baseline, which considers the implications of the COVID-19 pandemic, such as economic slowdown or structural and behavioural change.

Overall the results suggest that decision makers are not always informed of the current situation (or baseline) before regulating, and that regulatory options are more likely to have been identified and assessed than non-regulatory ones. In turn, this risks prejudging that intervention is warranted and that it potentially be through regulatory means.

An increasing number of OECD members are introducing requirements for RIA to be proportionate to the significance of anticipated impacts (see Figure 2.13), thereby following the Best Practice Principles on Regulatory Impact Assessment (OECD, 2020[15]) (see Box 2.7). Whilst the majority of OECD members necessitate RIAs to be proportionate to the size of the anticipated impacts, these obligations have been introduced at a lower rate than between 2014 and 2017 and there currently remains approximately 15% of OECD members that do not have any proportionality requirement.

A fifth of OECD members still use a threshold test to determine whether a RIA is undertaken at all, demonstrating that an increasing number of regulatory proposals undergo at least some level of impact analysis. A common method across OECD members for establishing proportionate ex ante analysis is to introduce a threshold test, to determine whether a regulatory proposal warrants more in-depth RIA. In parallel with the proportionality requirement, the use of threshold test has increased across OECD members, though at a lower rate than before (see Figure 2.14). Whilst the Best Practice Principles on Regulatory Impact Assessment (OECD, 2020[15]) cite various alternatives for distinguishing which regulatory proposals should go through a certain level of analysis, the most common method used amongst OECD members is a threshold that is expressed both in quantitative and qualitative terms, with only one OECD member using a solely quantitative threshold test (Box 2.8).

Policy makers from OECD members are increasingly required to assess the impacts of regulatory proposals on a range of factors (see Figure 2.15). There continues to be a strong focus on analysing the economic impacts of regulatory proposals, with the effects on competition, public administration, and the budget being the most commonly required regulatory assessments. The regulatory impacts on micro, small and medium-sized companies are also commonly assessed amongst OECD members, although policy makers in OECD members have divergent approaches to implementing the SME Test (see Box 2.9).

Whilst they remain less developed than economic factors, policy makers from the OECD members are increasingly required to assess the social impacts of regulations. In particular, it is now mandatory to assess the impacts of regulations on poverty, on gender equality, and on the environment in 29, 32, and 32 OECD members, respectively. For example, the assessment of non-economic regulatory impacts – i.e. impact on gender equality, poverty, and on people with disabilities – is an integral and mandatory component of the RIA process in Portugal. There has also been a strong increase in the number of members reviewing the effects of regulation on specific regional areas, although the requirement to analyse regulatory impacts on foreign jurisdictions remains the lowest amongst all assessments. The magnitude of impacts on foreign jurisdictions will continue to grow in an ever increasingly interconnected world, potentially resulting in significant impacts (both positive and negative) being omitted from RIAs. By working together governments can better understand the potential extraterritorial impacts of their regulatory proposals. The topic of international regulatory co-operation is discussed in Chapter 4.

Results from this edition of the Regulatory Policy Outlook and from previous ones (OECD, 2015[3]; OECD, 2018[20]) have demonstrated that policy makers continue to be increasingly aware of the impacts that regulations can have. Of the impacts featured in Figure 2.15, the only area that is nearly universally covered by OECD members is the requirement to consider competition impacts. This reflects the central importance of competition to market economies (OECD, 2019[21]), however it does not necessarily relate to the concept of competitiveness (Box 2.10) – an area that will take on increased prominence in the years to come as the global economy begins to recover from the COVID-19 pandemic.

The result of OECD research (2021[22]) demonstrates that whilst many OECD countries assess some of the components of competitiveness as part of the RIA analysis, such as innovation and trade, these impacts are often assessed disjointedly and the second-order effects might not always be captured. For example, the direct regulatory costs to SMEs and the impact on SME’s cost competitiveness are often included in the SME test. RIAs, however, often neglect looking at the second-order effects, such as the way regulations may impact SMEs’ ability to access innovation or their capacity to enter international markets, which are both key drivers of competitiveness. This suggests that policy makers therefore do not assess the impacts on competitiveness in a holistic manner and therefore produce inaccurate cost-benefit analysis that underestimates the true cost of regulatory intervention. As a result, this artificially improves the results of the RIA and provides decision makers with inaccurate information.

The externalities of a regulatory proposal can be aggravated at a time of economic and sanitary crises, as demonstrated by the COVID-19 pandemic whose impacts have been uneven both within (impact on SMEs vs. larger firms) and across sectors (e.g. tourism vs. education), with spillovers affecting the economy as a whole. Policy makers may have to react quickly to address emerging issues and may be unable to fully identify the impacts of regulatory proposals. In recovering from the COVID-19 pandemic, countries across the globe might look beyond productivity and consider improving their competitiveness as a whole. Policy makers should attempt to assess how policy proposals affect the cost of the relevant factor (e.g. compliance, research and development, trade) as they currently do, but also how it affects the ability to attract economic activity as well as the capacity of firms to capture any competitive advantages vis-à-vis its current and potential competitors.

In emergency situations it may be necessary to adjust the level of impact assessment undertaken. This is a pragmatic realisation that collecting information, engaging with stakeholders, and assessing impacts takes time and resources – which may not be available or may detract from more important issues in an emergency. That said, some points of caution should be noted. Firstly, this should not be viewed as an opportunity to avoid considering impacts until the end of the policy making process – the events need to have been genuinely unforeseeable. Secondly, there may be opportunities to undertake some impact assessment, for example a focus (perhaps even only qualitatively) on the immediate anticipated effects of the policy. For instance Canada adjusted its RIA requirements for COVID-related proposals. Proposals could be developed using adjusted analytical requirements, including cost-benefit analysis and the small business lens analysis. These could be based on qualitative and quantitative data, but the requirement to monetise impacts was relaxed. In addition, proposals could be recommended for exclusion from the one-for-one rule. Any information that can be reasonably collected ex ante can firstly help to inform better decision making, and secondly can be used as a starting basis for later reviews of the policy (see ex post evaluation section below).

Nearly half of OECD members now have exceptions to conducting RIAs where regulations are introduced in response to an emergency (see Figure 2.16), and several members have used this mechanism to bypass their RIA requirements for some of the regulations introduced in response to the COVID-19 pandemic. A further four OECD members (Chile, Denmark, New Zealand, and Portugal) have introduced the exception to conducting RIA in case of emergency in their regulatory practices in the past three years, although in most cases this mechanism was introduced before the COVID-19 pandemic. A range of other emergency-based regulatory policy measures have been introduced by OECD members (Box 2.11).

The transparency surrounding the decisions to except proposals from conducting RIA remains blurred, with only a minority of OECD members currently publishing the decision that RIA will not be conducted where it ought to have been. There also remains nearly 60% of OECD members in which no body is responsible for reviewing the decision made by officials about whether a RIA is required. In effect, this suggests that a majority of OECD members can use the exception mechanisms to bypass RIA with little scrutiny on whether this decision is appropriate or proportionate to the regulatory proposal at hand.

Few countries have reported having a requirement to undertake a post-implementation review where a regulatory proposal was excepted from undertaking a RIA (Figure 2.17). Given the relatively low proportion of OECD members that have PIR requirements in place, it seems that countries are generally more likely to adopt an ad hoc approach to consequences for regulations that bypass ex ante impact assessment (and scrutiny). Irrespective of whether consequences are formalised, it is expected that there will be a significant amount of forthcoming policy reviews (see ex post evaluation section below).

OECD members have improved their ex post evaluations since 2017 for both primary and subordinate regulations. The largest improvements have been in transparency of ex post evaluations. Members have invested in dedicated websites for the public to make recommendations to modify and provide feedback on existing regulations, and in some countries stakeholders are actively engaged when ex post evaluations are conducted. The number of oversight bodies scrutinising ex post evaluations has increased since 2017. These bodies provide advice and guidance on conducting ex post evaluations, and assist officials to conduct them.

Countries which have had substantive changes to their ex post evaluation systems over the last years include Canada, Greece, Italy, Japan, Korea, Latvia, Lithuania, Mexico, Netherlands, Portugal and the European Union.

  • Canada recently updated its Cabinet Directive on Regulation; it now mandates government departments and agencies to conduct ex post evaluation on all subordinate regulations and it provides guidance and training to policy makers on how to carry them out.

  • Greece introduced Law 4622 in 2019. Amongst other topics, it made periodic ex post evaluations mandatory for all primary laws and for major subordinate regulations, and it now requires all ex post evaluations to contain an assessment of costs and benefits. Evaluation techniques and oversight functions related to ex post evaluations were also strengthened.

  • In Italy, new non-binding guidance on ex post evaluation was issued in 2018. Initial steps have been taken to plan ex post evaluations when preparing RIAs for major legislation. Ministries publish a two-year plan of regulations to be evaluated.

  • In Japan, the number of ex post evaluations has increased for both primary laws and subordinate regulations since 2017. Ex post evaluations are automatically triggered if a RIA was conducted during policy development.

  • In Korea, ex post evaluation is mandatory for all regulations developed by the executive and central ministries, which are required to outline the intended evaluation plan as part of each RIA. Packaged reviews of ex post evaluations are now subject to quality control.

  • As part of broader reforms in Latvia, ex post evaluations are now required for some subordinate regulations and an evaluation of all policy documents conforming to the SDGs was recently conducted.

  • Lithuania has introduced some general requirements to conduct monitoring and ex post reviews of existing primary laws and in 2020, it strengthened the regulatory oversight function and transparency of ex post evaluations.

  • Mexico introduced a new General Law for Better Regulation in 2018, which established new provisions to carry out ex post assessment of regulations that generate compliance costs. Mexico’s oversight body is now in charge of reviewing these ex post evaluations.

  • The Netherlands saw an improvement in oversight and quality control for periodic ex post evaluation of the effectiveness and efficiency of regulations. The Budget Inspectorate is now responsible for reviewing the quality of ex post evaluations and it has developed a toolbox with guidance for officials conducting these evaluations.

  • Portugal’s main regulatory oversight body was created in 2017 and has taken the role of co-ordinating ex post evaluations of subordinate regulations across the public administration and assisting officials in conducting them. Following the COVID-19 pandemic, Portugal introduced sunsetting clauses for some regulations.

  • The European Union’s ex post evaluation system combines systematic evaluations of individual regulations with comprehensive “fitness checks” of policy sectors, inviting comment on evaluation roadmaps. The EU’s regulatory oversight body provides summary ratings on evaluations that are made publicly available along with compliance statistics.

The stock of laws and regulations has grown rapidly in most countries. However not all regulations will have been rigorously assessed ex ante, and even where they have, not all effects can be known with certainty in advance. Regulations should be periodically reviewed; acknowledging that the original environment justifying the regulation may have changed, and as an opportunity to see how regulations have actually worked in practice.

Given the size of the regulatory stock, there are numerous opportunities to improve its functioning, thus increasing the benefits regulations provide, and at the same time ensuring that regulatory costs are kept to the minimum necessary. Evaluations of existing regulations can also produce important learnings about ways of improving the design and administration of new regulations – for example, to change behaviour more effectively (OECD, 2020[23]). In addition, the evaluation of regulatory impacts ex ante is often conducted under the hypothesis of a static economic equilibrium, whereas in practice, regulations interact together with more complex dynamics. In this way, ex post evaluations complete the regulatory cycle that begins with ex ante assessment of proposals and proceeds to implementation and administration.

The 2012 Recommendation calls on governments to “[c]onduct systematic programme reviews of the stock of significant regulation against clearly defined policy goals, including consideration of costs and benefits, to ensure that regulations remain up to date, cost justified, cost effective and consistent, and deliver the intended policy objectives.” The OECD publication Best Practice Principles on Reviewing the Regulatory Stock (OECD, 2020[24]), builds upon the 2012 Recommendation (OECD, 2012[1]) and helps elucidate key aspects of ex post evaluation (Box 2.12).

One clear message from the Best Practice Principles is that it is imperative to begin thinking about how regulations ought to be reviewed at the time when they are initially designed, particularly to identify the data that will need to be collected to assess the impacts. Previous findings established that data considerations are often neglected or not fully incorporated into the design of regulations (OECD, 2018[25]). In turn, this often makes undertaking ex post evaluations difficult as the lack of data mean that a counterfactual baseline is not necessarily possible to establish. Separately, a lack of adequate data considerations at the ex ante development stage means that the costs of regulations are not fully incorporated into regulatory proposals. Some OECD members have attempted to formalise processes to ensure that ex post evaluations are more fully embedded into the regulatory policy cycle (Box 2.13).

As the regulatory stock is far larger than the flow of new laws and regulations – ex post evaluations remain a missing piece in the OECD member better regulation toolkit. Only one-quarter of OECD members have systematic requirements in place to conduct ex post evaluations, with numbers essentially unchanged since 2014. For some of these members the scope of ex post evaluation requirements has changed, as was the case for Mexico since amendments to the General Law on Better Regulation were passed in 2018, which now extends beyond technical regulations as was the case previously. Furthermore, it is recognised that for a number of countries ex post evaluations is a relatively new area of regulatory management, and since 2014 four additional member countries have introduced some requirement to conduct ex post evaluations.

Consistent with previous Outlooks, OECD members have undertaken a range of ex post evaluations over the past five years. OECD members were most likely to have conducted principle-based reviews across a wide range of areas such as competition, administrative burdens, compliance with international instruments, overlaps between local, regional and federal regulations. Public regulatory stocktaking exercises were undertaken by nearly two-thirds of OECD members in the past five years, with slightly less in-depth reviews (Figure 2.20).

The requirement to review laws and regulations has been formalised by some OECD members through the use of automatic review clauses or sunset clauses, respectively. Despite the opportunity offered by such clauses, they are not always used to the fullest extent, even when governments regulate in areas subject to significant uncertainty at the time laws are made. They could act as an important discipline on lawmakers and at the same time transparently signal to stakeholders that there will be future opportunities to provide input on the retention, amendment, or removal of certain aspects of the regulation. The use of sunset clauses is slightly more prominent than automatic review clauses across OECD members, although both are usually implemented on an ad hoc basis. It should be noted that around half of OECD members do not currently utilise automatic review clauses and around 40% do not make use of sunsetting arrangements, with results virtually unchanged from 2014. Both of these clauses are important to ensure that the flow of regulations are subject to some sort of review in order to determine that they remain appropriate over time (OECD, 2020[24]).

As an improvement to the efficient review of regulations, OECD members can make use of flexible review arrangements such as deferring and packaging reviews in order to allow them to more holistically assess the impacts of laws on citizens and businesses. By bundling reviews of regulations in this way, governments can gain a better understanding of the broad system effects of regulation. At the same time, such reviews do offer stakeholders with a potentially superior opportunity to provide vital feedback on the interaction of regulations, while reducing consultation fatigue. In practice however, OECD members have generally undertaken packaged reviews on an ad hoc basis only, with more than 40% of members not undertaking packaged reviews at all.

Results from the iREG survey suggests that some OECD members require policy makers to identify a process to achieve a regulation’s goals at the time when the regulation is first created. However, when it comes to reviewing regulations via ex post evaluations, OECD members are less likely to have requirements in place to assess whether the underlying policy goals were in fact achieved or not (Table 2.2).

The Best Practice Principles note that it is important for ex post evaluations to contain recommendations for improvement as part of decisions around whether regulations remain fit for purpose in their current guise (OECD, 2020[24]). An important aspect of transparency to such ex post evaluations is how governments respond to such recommendations. However, in practice this is something that very few OECD members currently do (Box 2.14).

Having in-house capability in evaluation and review methods is essential, both in order to conduct reviews internally as well as to oversee those commissioned externally (OECD, 2020[24]). In order to assist governments to undertake ex post evaluations, a number of OECD members have provided guidance to evaluation teams. In 2014, around 40% of OECD members provided guidance on conducting ex post evaluations and now that is around 60%, reflecting the fact that more members are beginning to undertake ex post evaluations.

Governments have not materially increased the prevalence of various evaluation techniques and associated training programmes for ex post evaluations since 2014, and bespoke training programmes in ex post evaluation are only available in eight OECD members (Box 2.15).

Unlike ex ante impact assessment, ex post evaluations are able to observe actual impacts from regulations (OECD, 2020[24]). Assessing impacts in ex post evaluations should be done through a broad framework, assessing costs and benefits (OECD, 2020[24]). Since 2014, five more OECD members have begun requiring ex post evaluations to assess costs, and an additional six now require an assessment of benefits. This means that 60% of OECD members now have at least some requirement in place to assess both the costs and benefits when conducting ex post evaluations.

It is important to have methodologies in place for entities undertaking ex post evaluations so as to ensure that observed impacts can be assessed against the original anticipated impacts from the original regulatory development. This also highlights the importance of upfront data collection processes for the purposes of monitoring and evaluation at the ex ante policy development stage (OECD, 2018[25]). Since 2014, few changes were observed in this area, whether countries assess if actual impacts are in line with those anticipated, or whether they assess if regulations have had any unintended consequences. Since 2014, there has been a slight increase in OECD members undertaking RIA for some ex post evaluations, which has helped to provide a discipline and a focus to those ex post evaluations.

Japan is one of nine OECD members that currently systematically refer to the RIA that formed the basis for the original design of the regulation. This helps to ascertain whether impacts have eventuated as they were originally anticipated, and also allows those conducting ex post evaluations to establish whether the original rationale for policy intervention remains in the public interest.

Stakeholder engagement is vital throughout the entire regulatory lifecycle, including ex post evaluation, in order to garner feedback about the actual impacts of regulations “on the ground”, and to maintain both trust in and compliance with regulations in force (OECD, 2018[20]). Thirteen OECD members now systematically consult on ex post evaluations, thus recognising both its importance and value. Nevertheless, this is less systematically done than that undertaken during initial regulatory design (see stakeholder engagement section above). Informing affected stakeholders in advance of forthcoming ex post evaluations remains rare across the OECD membership.

The COVID-19 pandemic had a profound impact on many regulatory practices, as a number of actions had to be undertaken in an emergency context. From a recovery standpoint, ex post evaluation is one priority area, as undertaking a systematic process of review to assess what worked, what didn’t, and what could have worked better will be fundamental to improving future wellbeing.

Ex post evaluation presents an opportunity for governments to retain emergency rules that have had unintended positive consequences. Ex post evaluations take on an increased importance where ex ante impact assessment was limited or non-existent because of the genuine urgency to regulate as information gaps are potentially much larger. It also allows governments to retrospectively review whether other rules introduced during this time remain in the public interest. However, only four OECD members have formal consequences requiring ex post evaluations to be undertaken for regulatory proposals which bypass ex ante impact assessment processes during times of emergency (see RIA section above). Therefore for the majority of OECD members, requirements to conduct any form of ex post evaluation on these laws needed to be considered at the time when those laws were introduced (i.e. via either sunset or automatic review clauses), via an ad hoc decision given the magnitude of their impacts, or via the standard ex post evaluation requirements (to the extent that they exist).

OECD members self-reported a total of 190 specific regulations that were issued in response to the COVID-19 pandemic as of the iREG survey return date of 18 September 2020 (Table 2.3). Around half of these regulations included a sunset clause, while automatic evaluation requirements were much less frequently used. Reflecting the continued uncertainty and longevity of the pandemic, almost one-fifth of the original sunsetting arrangements needed to be extended beyond their original date. In some instances, calls to include review clauses were rejected. In Australia, for example, the Australian Parliament attempted to include a review clause for some COVID-related laws but this was rejected by the Government.

For some OECD members, the COVID-19 pandemic was the first time that general review provisions had ever been used. Finland introduced a regulatory measure to compensate companies’ costs during the pandemic, which included an automatic evaluation clause, which is a practice that Finland does not normally include for regulations issued in other contexts. Similarly, Israel reported five regulations that all included sunsetting clauses, whereas ordinarily such clauses are not utilised.

These results suggest that there will be a significant number of forthcoming reviews on regulations that would otherwise sunset. In turn this highlights the importance of having core competences to conduct good quality ex post evaluations amongst the civil service. Co-ordination between entities conducting ex post evaluations and the respective oversight bodies will need to improve to ensure both the timely and prioritised review of COVID-related regulations. Last, the forthcoming reviews highlight the need to ensure that both the oversight bodies and entities conducting ex post evaluations are adequately resourced.

Many governments’ immediate regulatory responses to deal with the COVID-19 pandemic were designed to have significant impacts on public health. The sheer volume of COVID-related laws passed will mean that there will be a significant flow of ex post evaluations in the future. The best practice principles note that priority should be given to reviewing those regulations that have both wide ranging and significant impacts on particular societal groups (OECD, 2020[24]). For a number of OECD countries, COVID-related responses took the form of omnibus Bills – targeting a range of areas from one piece of legislation. It may makes sense to review such laws in a package format so that all areas can be reviewed holistically (OECD, 2020[24]).

Decisions around the timing of ex post evaluations and PIRs emanating from the COVID-19 pandemic matter. Generally, ex post evaluations (and of post-implementation reviews – see RIA section above) ought to be undertaken at a point in time where sufficient data are collected to ascertain whether regulations have had their intended effect (OECD, 2020[24]). In Australia for example, generally PIRs are to be completed two years after the “implementation” of the policy is complete (five years for decisions which have substantial or widespread impacts on the Australian economy) (Australian Government, 2020[26]). By its nature, the implementation period is extensive for some policies; especially for significant policy decisions with a broad range of impacts. The result is that a substantial amount of time could pass before the decision is reviewed. While this helps to increase the data collection period – indeed the evaluation should be more comprehensive because of it – it may mean that the decision is embedded to such an extent that even if the evaluation found the policy to be flawed in some way, nothing could be done about it (or the costs of doing something about it would exceed the anticipated benefits from its removal or modification).

As noted in Box 2.14, ex post evaluations do provide opportunities to learn from previous mistakes, as well as highlight what has worked well. In order to best manage the next crisis, ex post evaluations from the COVID-19 pandemic could form a valuable repository of information in the future for policy makers. The OECD is currently conducting a meta-analysis of all evaluations conducted by countries in the context of the COVID-19 pandemic, including on regulatory policy.

In their day to day lives, there are moments when citizens and businesses need decisions by public authorities to be able to carry out their desired or needed activities. For instance, when they request a license to operate in a regulated sector or to drive a car. It is also the case when a regulator imposes a sanction to an individual for not complying with an existing regulation. When applying these regulations, policy makers are called to act within the scope of their legal powers, be reasonable and proportionate when deciding individual cases. Though in principle those decisions are expected to be impartial and lawful, it might be that the recipient of the decision does not agree with it, and has an argument against its legality, the fairness of the procedure, or even whether due process was respected when making the decision.

These businesses and citizens should have avenues to challenge or appeal individual decisions that those authorities in the application of existing regulations, as called for by the 2012 Recommendations. The existence and use by the public of these mechanisms prevents abuse of discretionary authority, and preserves the integrity of the regulatory system, which in turn improves the effectiveness of well thought out regulations.

Mechanisms to challenge decisions made in individual cases exist in all OECD members. In almost all countries, citizens and businesses can have those decisions reviewed by a court (in 38 countries) or by the body in charge of enforcing the regulation that was the base for the decision (in 34 countries). In some countries, there are other mechanisms available to challenge these decisions. For instance, it is possible to appeal an individual decision before an independent body (in 26 countries), have a ministry review the decision (in 25 countries) or have the decision reviewed by a specialised administrative jurisdiction (in 22 countries) (see Figure 2.21). In addition, 24 OECD members reported that it is possible for businesses and citizens to issue a petition to the agency that made the decision to reconsider the decision, for instance, when a licence to start a businesses is denied.

Being able to challenge the application of an individual regulation is essential, but it is also essential for the decision to that challenge to come in a timely manner. This allows businesses and citizens to plan accordingly, and estimate the risks and costs that a negative or positive response might entail in time. For this, governments are called to have standard time periods on which a business or individual can expect a decision to be made when appealing against a regulatory decision, to the extent possible (OECD, 2012[1]). Ten of the surveyed countries reported having a standard period within which parties can expect a decision to be made for at least one of the mechanisms for challenging individual decisions. However, most countries indicated that the existence of these periods depends on many factors, such as the mechanism itself, the legislation that may govern the form of review, the complexity of the case, the workload of the deciding body, etc. For instance, judicial reviews in Canada are not subject to deadlines as it could fetter the discretion of the courts, but there are sectorial administrative regulations, such as the Canada Transportation Act, that sets out specific timelines.

In addition businesses and, where relevant, citizens more broadly, should have access to mechanisms to challenge regulations themselves on which those decisions are based. This helps remove from the legal system any existing regulation that might have missed a legal or constitutional filter during its development and therefore does not fit the legal framework of the country, may also help eliminate at least some regulations that do not pass a proportionality test. This then stops those illegal or unconstitutional regulations from being used and applied to other citizens and businesses, helping to the consistency and effectiveness of the regulatory system.

Both citizens and businesses have a least one mechanism to challenge the legality of existing regulations in 27 of all surveyed countries. The most common mechanism that countries reported having is judicial challenge, where businesses and citizens can challenge the legality of an existing regulation before a court. Also, 15 of the surveyed countries provide their citizens the possibility to challenge the legality of an existing regulation before an administrative jurisdiction. A few countries reported that it is possible to challenge the legality of a regulation before regulatory bodies, where, for instance, it is possible to challenge the legality of the regulation issued by an economic regulator before said regulator. In addition, 26 of the surveyed countries reported having a mechanism for the public to challenge a regulation that is not in accordance with their respective constitutions (see Figure 2.22). As with being able to challenge the legality of a regulation, having mechanisms to remove a regulation from the legal system based on its lack of compatibility with the constitution, helps with the coherence of the regulatory framework, and the fair treatment of all citizens and businesses.

The 2012 Recommendation of the Council on Regulatory Policy and Governance invites countries to “promote regulatory coherence through co-ordination mechanisms between the supra national, the national and sub-national levels of government”. Furthermore, it advises countries to “Identify cross cutting regulatory issues at all levels of government, to promote coherence between regulatory approaches and avoid duplication or conflict of regulations”. Additionally, the Recommendation urges countries to “Foster the development of regulatory management capacity and performance at sub-national levels of government” (OECD, 2012[1]).

In this sub-section we report the results and findings of data collected across OECD jurisdictions to attempt to measure progress in implementing some key elements of these recommendations.

Sub-national governments have a key role in delivering public policy objectives through regulations. They may have the legal capacity to issue and enforce regulation within their own domains, as in federal-type jurisdictions. Or they may be tasked to implement and enforce regulations issued by upper levels of government, through the issuance of secondary legal instruments, such as by-laws, manuals or guidelines, coupled with actions to ensure the enforcement and compliance of the regulations, such as inspections. The latter activities are commonplace across both federal and unitary-type countries.

In this setting, the national or central levels of government should warrant that there are mechanisms in place to ensure that there is regulatory coherence to avoid voids, overlaps or conflicts in both the contents of the regulatory instruments and the enforcement approaches across levels of government. Additionally, sub-national governments are part of the public administration machinery, and as such, they should be able to follow principles and apply the appropriate tools to ensure that regulations they issue are of high-quality. Also, they should ensure that they are effective in enforcing and implementing the regulatory framework. To achieve this, national or central levels of government should step in to help generate the necessary capacities in sub-national governments.

For instance, a small business owner facing stricter regulatory requirements in health aspects due to the pandemic caused by the COVID-19 may need to comply with specific levels of hygiene in his restaurant’s kitchen, as set by a national law on health. The regional government has issued the guidelines and procedures to comply with this requirement, and the local government is tasked with carrying out inspections to verify the compliance with the procedure and the standard. In this example, the public policy objective is clearly to protect the health of customers when consuming in the restaurant. However, this objective can only be achieved effectively if the guidelines and procedures issued by the regional government are fully consistent with the national law on health. Moreover, the preparation and publication of this secondary regulation would benefit from applying tools on regulatory management to promote regulatory quality such as RIA and stakeholder engagement. This will only be possible if regional governments have the capacity to implement and use these tools. The same reasoning applies to local governments responsible for the enforcement and inspections of the regulatory framework. An argument can even be raised to establish co-ordination mechanism across levels of governments, so the results of the enforcement and inspection activities loop back into the ex-post assessment of the law, the procedures and guidelines, which will help ascertain whether the public policy objective is being achieved.

With some variations, similar examples to the one mentioned above can be found across OECD countries, regardless of whether they are considered federal or unitary-type countries. Hence, the relevance of attempting to measure the progress in implementing the 2012 Recommendation on the aspects of regulatory coherence across levels of governments, and on fostering of regulatory policy in sub-national governments. In general, the data collected shows that practices across OECD countries to pursue these objectives are not commonplace yet. This finding contrasts with the general uptake of tools such as RIA and stakeholder engagement. Hence, countries have ample opportunities to engage in actions with sub-national governments that will benefit the overall effectiveness of the regulatory framework in delivering policy objectives.

Figure 2.23 presents the results of the question “Are there one or several co-ordination mechanism(s) across national and sub-national governments or municipalities to promote regulatory coherence in regulatory approaches and avoid duplication or conflict of regulations?” and it identifies the type of mechanism when countries answer positively. 26 out of 38 respondents confirmed that they have a mechanism in place, where one of the most common types is a standing co-ordination mechanism. The National Federation Reform Council of Australia and the National Council on Better Regulation of Mexico provides examples of the standing co-ordination mechanism that seeks to promote regulatory coherence across levels of government (see Box 2.16 and Box 2.17).

In Figure 2.23 it stands out that all countries which could be considered federal have some type of mechanism which seeks to promote regulatory coherence. By the same token, the data shows that eighteen countries which can be categorised as unitary also have this type of mechanism. Hence, the data shows that two thirds of surveyed countries consider important to establish practices that advocate for a consistent regulatory system, irrespective of a federal or unitary status. This suggests that the rest of the OECD countries have the opportunity to implement systems to engage with sub-national governments to enhance the quality of the regulatory framework, regardless of the status of a unitary-type jurisdiction.

One of the ways to foster the development of regulatory management capacity and performance at sub-national levels of government is by promoting best practice across these governments, and between the sub-national and national levels. The sharing of lessons learned, successful cases, and the “dos and don’ts” in the design, implementation and evaluation of regulatory policy and its tools can be an effective way of promoting their adoption. Figure 2.24 shows that only 17 out of 38 countries have these sharing mechanisms in place, in which workshops, seminars or conferences is the most common type of practice, followed by reports on good practices and lessons learned. Therefore, this result suggests that OECD countries have ample room to establish mechanisms to share best practices in regulatory management, and this applies to both federal and unitary types of governments.

Mexico offers an example on reports that benchmark regulatory management practices at the subnational level through the reports prepared by the National Observatory on Better Regulation (see Box 2.17).

Figure 2.25 shows the results of data collected to probe further in other types of mechanisms to foster the development and performance of regulatory policy at sub-national levels of government, and the actual use of this policy. For instance, 15 countries answered that they actively support the implementation of regulatory policy in this level of government. The Primary Authority scheme of the UK offers a relevant example of how central government can support local governments in the delivery and enforcement of regulations (see Box 2.18).

One of the central elements to aspire to an effective design and implementation of regulatory management practices is to have a body that promotes the use of good regulatory practices and evidence-based policy making. Figure 2.25 shows that 15 countries have this type of body in all or some of their regional governments, and 12 at the municipal level.

Stakeholder engagement is another practice that is key for an effective system that promotes regulations based on evidence. Figure 2.25 indicates that only 14 countries have a specific mechanism between the national and sub-national governments to communicate the views of local firms and citizens to inform the development of regulations.

Finally, Figure 2.25 shows that there is an incipient use of actions oriented to facilitate variation and experimentation in regulatory approaches at sub-national levels of government. Only four countries reported engaging in these practices. Practices might include alternative approaches to regulation, use of ICT tools such as machine learning, and special regimes such as sandboxes.

Overall, Figure 2.25 shows there are several gaps that OECD countries could bridge in order to foster the development and performance of regulatory management capacity at sub-national levels of governments. This assertion is valid for both federal type and unitary countries. For instance, OECD countries could promote the establishment of a body at sub-national levels of government tasked with the oversight of regulatory policy. Also, there is room to foster the formation of procedures to collect and employ the feedback and opinions of local firms and citizens to strengthen regulatory policy at the sub-national and national level.

To conclude, actions taken by OECD countries to promote regulatory coherence across levels of government and to foster the development and performance of regulatory management capacity in sub-national governments are not widespread yet. As set out by the 2012 Recommendation on Regulatory Policy and Governance, OECD countries should consider including mechanisms to incorporate sub-national governments in the design and implementation of their regulatory quality policies. The 2020 iREG data demonstrates that several countries have advanced in this front, in both federal and unitary type jurisdiction, and this progress should be employed as inspiration for more action.


[32] Anglmayer, I. (2020), Better Regulation practices in national parliaments, https://doi.org/10.2861/06573.

[2] Arndt, C. et al. (2015), “2015 Indicators of Regulatory Policy and Governance: Design, Methodology and Key Results”, OECD Regulatory Policy Working Papers, No. 1, OECD Publishing, Paris, https://dx.doi.org/10.1787/5jrnwqm3zp43-en.

[26] Australian Government, D. (2020), Post-implementation Reviews: Guidance Note, https://www.pmc.gov.au/sites/default/files/publications/post-implementation-reviews.pdf.

[17] Australia, P. (2020), Security Legislation Amendment (Critical Infrastructure) Bill 2020, https://www.legislation.gov.au/Details/C2020B00201/Explanatory%20Memorandum/Text.

[31] BEIS (2017), Primary Authority Statutory Guidance, https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/707382/primary-authority-statutory-guidance-2017.pdf (accessed on 18 March 2021).

[28] CONAMER (2021), Consejo Nacional de Mejora Regulatoria [National Council on Better Regulation], https://conamer.gob.mx/cnmr/GruposTrabajo (accessed on 18 March 2021).

[29] Congreso de la Unión (2018), Ley General de Mejora Regulatoria, http://www.diputados.gob.mx/LeyesBiblio/ref/lgmr/LGMR_orig_18may18.pdf (accessed on 22 May 2018).

[22] Davidson, P., C. Kauffmann and M. de Liedekerke (2021), “How do laws and regulations affect competitiveness: The role for regulatory impact assessment”, OECD Regulatory Policy Working Papers, No. 15, OECD Publishing, Paris, https://dx.doi.org/10.1787/7c11f5d5-en.

[27] Government of Australia (2020), Department of the Prime Minister and Cabinet, https://www.pmc.gov.au/domestic-policy (accessed on 17 March 2021).

[12] Lind, E. and C. Arndt (2016), “Perceived Fairness and Regulatory Policy: A behavioural science perspective on government-citizen interactions”, OECD Regulatory Policy Working Papers, Vol. 6.

[30] Ministry of Economy (2021), Observatorio Nacional de Mejora Regulatoria [National Observatory on Better Regulation], http://onmr.org.mx/#divISMR (accessed on 18 March 2021).

[16] OECD (2021), “Behavioural insights and regulatory governance: Challenges and opportunities for improving regulatory oversight and management tools”, OECD working papers (forthcoming).

[18] OECD (2020), A closer look at proportionality and threshold tests for RIA: Annex to the OECD Best Practice Principles on Regulatory Impact Assessment, http://www.oecd.org/regreform/Proportionality-and-threshhold-tests-RIA.pdf.

[23] OECD (2020), Improving Governance with Policy Evaluation: Lessons From Country Experiences, OECD Public Governance Reviews, OECD Publishing, Paris, https://dx.doi.org/10.1787/89b1577d-en.

[11] OECD (2020), Innovative Citizen Participation and New Democratic Institutions: Catching the Deliberative Wave, OECD Publishing.

[15] OECD (2020), Regulatory Impact Assessment, OECD Best Practice Principles for Regulatory Policy, OECD Publishing, Paris, https://doi.org/10.1787/7a9638cb-en.

[24] OECD (2020), Reviewing the Stock of Regulation, OECD Best Practice Principles for Regulatory Policy, OECD Publishing, Paris, https://doi.org/10.1787/1a8f33bc-en.

[21] OECD (2019), Competition Assessment Toolkit: Volume 1. Principles, http://www.oecd.org/competition/toolkit.

[9] OECD (2019), Tools and Ethics for Applied Behavioural Insights: The BASIC Toolkit, OECD Publishing, Paris, https://dx.doi.org/10.1787/9ea76a8f-en.

[25] OECD (2018), 10th OECD Conference on Measuring Regulatory Performance: Key findings and conference proceedings, Measuring the implementation of regulation, OECD Publishing, Paris, http://www.oecd.org/gov/regulatory-policy/Proceeding-MRP-Mexico-June-2018.pdf.

[20] OECD (2018), OECD Regulatory Policy Outlook 2018, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264303072-en.

[8] OECD (2017), Behavioural Insights and Public Policy: Lessons from Around the World, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264270480-en.

[7] OECD (2017), Recommendation of the Council on Open Government, OECD/LEGAL/0438.

[13] OECD (2016), Open Government: The Global Context and the Way Forward, OECD Publishing.

[3] OECD (2015), OECD Regulatory Policy Outlook 2015, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264238770-en.

[5] OECD (2015), Policy Shaping and Policy Making: The Governance of Inclusive Growth, OECD Publishing, Paris, http://www.oecd.org/gov/governance-for-inclusive-growth.htm.

[14] OECD (2015), Stakeholder engagement in regulatory policy, OECD Publishing, Paris, https://doi.org/10.1787/9789264241800-en.

[6] OECD (2014), Recommendation of the Council on Digital Government Strategies.

[1] OECD (2012), Recommendation of the Council on Regulatory Policy and Governance.

[4] OECD (2009), Focus on Citizens: Public Engagement for Better Policy and Services, OECD Publishing, Paris, https://doi.org/10.1787/9789264048874-en.

[19] OECD (forthcoming), “The SME test: Taking SMEs into account when regulating. A comparative study of SME tests in OECD countries”.

[10] Tversky, A. and D. Kahneman (1974), “Judgment under uncertainty: Heuristics and biases”, Science, Vol. 185/4157, pp. 1124-1131, https://doi.org/10.1126/science.185.4157.1124.

Metadata, Legal and Rights

This document, as well as any data and map included herein, are without prejudice to the status of or sovereignty over any territory, to the delimitation of international frontiers and boundaries and to the name of any territory, city or area. Extracts from publications may be subject to additional disclaimers, which are set out in the complete version of the publication, available at the link provided.

© OECD 2021

The use of this work, whether digital or print, is governed by the Terms and Conditions to be found at http://www.oecd.org/termsandconditions.