3. Regulatory oversight

Robust oversight is a cornerstone of effective regulatory policy and practical implementation of the 2012 Recommendation of the Council on Regulatory Policy and Governance, which notably stresses the importance of establishing mechanisms and institutions to actively provide oversight of regulatory policy procedures and goals, support and implement regulatory policy, and thereby foster regulatory quality.

Regulatory oversight bodies (ROBs) need to incentivise civil servants to use regulatory management tools and follow due process to produce high-quality regulations, foster a whole-of-government perspective towards regulation and ensure a consistent approach to regulatory policy through appropriate co-ordination across the public administration. In light of emerging phenomena warranting concerted action globally, such as health threats, climate and environmental issues or technological change, ROBs also have a crucial role to play in promoting international regulatory cooperation as well as the adoption of innovative practices and forward-looking approaches to deal with uncertainty and enhance systemic resilience. The Recommendation of the Council for Agile Regulatory Governance to Harness Innovation and related practical guidance underscore the importance of regulatory oversight in that context (see also chapter on Regulatory policy 2.0).

Despite the critical role of regulatory oversight systems in enabling effective regulatory frameworks, available evidence points to significant room for improvement in this area. For example, among the four dimensions covered by the OECD composite indicators (see Chapter 2 on regulatory management tools for more details), oversight and quality control of regulatory management tools has remained comparatively underdeveloped over the 2014-2020 period. Challenges relating to the COVID 19 crisis raise, in turn, questions about the role of ROBs in emergency and high-uncertainty contexts. The present chapter builds upon previous OECD-led efforts to gain a better understanding of regulatory oversight systems. It provides updated information with a focus on “core” functions of regulatory oversight undertaken on a systematic basis: quality control of regulatory management tools; issuance or provision of relevant guidance on the use of regulatory management tools; co-ordination on regulatory policy and systematic evaluation of regulatory policy (see Box 3.1).

The main source of data and information is iREG 2020, which has fully integrated selected content from the 2017 OECD survey on regulatory oversight bodies for greater coherence and minimal burden for respondents. In addition, new data have been collected on oversight of ex post evaluation. Bodies outside the executive branch of government may be underrepresented in the sample given the strong focus on, and reporting by, government entities. In addition, respondents reported the smallest unit with responsibility for oversight function, which means several bodies may belong to same ministry or department. Data in this chapter are expressed using two basic complementary units: percentage (or number) of jurisdictions and percentage (or number) of all ROBs considered (across all jurisdictions). Unless otherwise stated, jurisdictions include OECD members and the European Union.

The chapter first analyses the institutional organisation of regulatory oversight as well as its evolution over time. It then discusses in more detail existing oversight and quality control mechanisms for regulatory management tools based on country-level data and relevant examples. The chapter concludes with a set of considerations on the meaning and determinants of well-performing regulatory oversight.

This section explores the institutional arrangements and organisation of regulatory oversight functions in OECD countries, including their evolution in comparison with the situation 2017, when data were last collected (and in 2014 whenever data are available).

OECD members remain invested in regulatory oversight. In 2020, as in 2017, all 39 reporting jurisdictions continued to have at least one dedicated body responsible for promoting regulatory policy as well as monitoring and reporting on regulatory reform and regulatory quality in the national administration from a whole-of-government perspective (this figure stood at 33 in 2014).1 This highlights the crucial importance of dedicated mechanisms and institutions to ensure decision-making is systematically grounded on the best available evidence.

In line with the 2012 Recommendation on of the Council on Regulatory Policy and Governance, a number of OECD members have continued to strengthen and institutionalise their existing oversight mechanisms. 2017 data showed that the mandates of the vast majority of ROBs were established in either law or statutory requirement or, alternatively, in a presidential or cabinet directive. According to 2020 data, ROBs continue to have a strong legal anchoring. As shown in Figure ‎3.1, since the beginning of 2018, five ROBs have had their mandate renewed and for seven of them it has become permanent. The latter include Denmark’s Government Economic Committee, the two bodies within Greece’s Secretariat General of Legal and Parliamentary Affairs, Latvia’s State Chancellery, Mexico’s CONAMER, Portugal’s Technical Unit for Legislative Impact Assessment, and Spain’s Regulatory Coordination and Quality Office. In addition, a number of ROBs have assumed new responsibilities, which may be a sign of governments’ willingness to embed these ROBs further in the wider regulatory policy environment.

Compared with the snapshot presented in the previous edition of the Outlook, which relied on a broader definition, the current institutional framework for regulatory oversight appears less fragmented - although there are generally several ROBs sharing certain functions. In total, 92 ROBs across all jurisdictions were reported as being in charge of performing at least one core regulatory oversight function on a systematic basis as of end 2020. This amounts to an average of nearly 2.4 ROBs per jurisdiction. By means of comparison, 163 ROBs (or more than four per jurisdiction on average) were considered for analytical purposes in the previous edition of the Outlook, which relied on a broader definition. Even so, with responsibilities for certain oversight functions often in the hands of more than one ROB, effective coordination on regulatory policy matters remains essential nevertheless. Many countries therefore choose to locate the main responsibility for that function as close to the centre of government as possible.

As shown in Figure 3.2, more than three-quarters of ROBs are located within government, half of which at the centre of government, i.e. in a body that provides direct support and advice to the Head of Government and the Council of Ministers, for example: Heads of Prime Minister's Offices, Cabinet Secretaries, or Secretaries-General of the Government (OECD, 2020[2]). In addition, most jurisdictions have more than one ROB within government. The latest data also confirm the prominent role of non-departmental bodies in regulatory oversight. In most cases, these are arm’s length bodies, which are not subject to the direction on individual decisions by executive government but may be supported by government officials (OECD, 2018[3]). In a context where analytical rigour and credibility are paramount, these bodies are valuable sources of advice and scrutiny. In most jurisdictions where this kind of bodies operate, they also evaluate regulatory policy and contribute to the policy debate by putting forward suggestions for reform. For example, RegWatchEurope, a network of independent bodies, has issued recommendations for developing regulatory oversight further at EU level (RegWatchEurope, 2020[4]).

Among core oversight functions, the most widespread remains quality control of RIA: about 75% of all ROBs, across jurisdictions, have it among their responsibilities. By contrast, only about 45% and 30% of all ROBs are responsible for quality control of stakeholder engagement activities and ex post evaluation of regulation respectively. The other two core functions considered, guidance on the use of regulatory management tools and systematic evaluation of regulatory policy are within the remit of about 70% and 55% of all ROBs respectively.

ROBs carrying out core regulatory oversight functions typically perform “non-core” functions that are also important. The most frequent among these functions relate to the systematic improvement of regulatory policy and advocacy across government (e.g. by proposing changes to the regulatory policy framework, promoting the use of good regulatory practices or ensuring institutional relations), in which about 75% of all ROBs are involved. About two-thirds of ROBs are in turn responsible for training and capacity building activities regarding the application of regulatory management tools, and nearly half of them for identifying areas of policy where regulation can be made more effective, e.g. by gathering opinions from stakeholders, preparing reviews of existing regulation or analysing the stock and/or flow of regulation. Scrutiny of the legal quality of regulation under development, in turn, is among the functions of nearly 40% of all ROBs.

Regarding the distribution of core functions across locations, the patterns observed in the OECD Regulatory Policy Outlook 2018 still seem to hold true to a large extent. ROBs located at the centre of government are entrusted with a relatively broad range of functions. As may be expected, they are by far the preferred location for functions where centrality is essential, such as coordination-related matters (e.g. promotion of joined up approaches to regulatory quality and the consistent application of relevant tools) and provision of guidance on the use of regulatory management tools. These functions are within the remit of about 80% and 75% of all ROBs at the centre of government respectively.

ROBs in other parts of government also have a diverse range of responsibilities. Those located in Ministries of Economy, Finance or Treasury tend to focus on quality control of regulatory management tools (about 80% of all ROBs in that location scrutinise RIAs) and are also involved in providing guidance and training as well as in identifying potential areas for improvement. ROBs located at Justice Ministries focus on reviewing the legal quality of proposals, although not exclusively: the vast majority of them issue guidance and more than 70% are involved in RIA scrutiny to some extent.

Non-departmental bodies have a clear focus on RIA scrutiny: all of them were reported to have it among their responsibilities. Approximately 45% and 35% of them scrutinise stakeholder engagement and ex post evaluations respectively. They are also heavily involved in the systematic evaluation of regulatory policy, as more than three quarters of them have this among their functions. ROBs external to government, in turn, focus on reviewing the quality of regulatory management tools, chiefly of RIA, as well as on the systematic evaluation of regulatory policy and the identification of areas for regulatory improvement. It should be noted that applying a revised definition has notably lowered the proportion of ROBs external to government (mainly in Parliament or part of the Judiciary) that have been considered for analytical purposes compared with 2017.

Institutionalisation of regulatory oversight is also developing beyond the OECD membership. For example, in 2019, Thailand took major steps forward in strengthening their capacity to conduct effective oversight (see Box 3.2).

This section focuses on quality control of regulatory management tools: RIA, stakeholder engagement and ex post evaluation. The analysis on oversight of ex post evaluation draws on newly collected variables.

Among the four dimensions covered by the OECD composite indicators, oversight and quality control of regulatory management tools, which accounts for the role and attributions of ROBs as well as for publicly available evaluations, appears to be comparatively underdeveloped when considering the 2014-2020 period: over time, this dimension has shown the lowest scores in each composite indicator, which highlights the need for stepping up efforts in this area. Despite recent improvements, oversight and quality control scores are particularly low for ex post evaluation. Indeed, quality control of both ex post evaluations and, to a lesser extent, stakeholder engagement, remains insufficient, as only slow progress has been achieved in recent years. Oversight focus continues to lie primarily with the scrutiny of RIA quality, with the vast majority of jurisdictions having a body in charge of this function.

Figure 3.4 provides an overview of the location of ROBs responsible for quality control of the different regulatory management tools. It reflects the overall distribution of ROBs across locations and confirms the pre-eminence of RIA in terms of coverage that was observed in previous analytical work. For ROBs in charge of quality control, RIA is their main focus (quality control of the other regulatory management tools being rarely dissociated from RIA’s), sometimes on an exclusive basis. In the case of ROBs at the centre of Government, quality control of RIA and stakeholder engagement practices related to the development of laws and regulations often go hand in hand.

Countries resort to a combination of approaches including support and advice (more widespread) and formal opinions that can in some cases be made public and/or coupled with more stringent sanctioning mechanisms to request that quality be improved.

Based on the updates received regarding the mandate of ROBs, the general features regarding the scope and prerogatives of those in charge of quality control of regulatory management tools that were identified in the 2017 survey can still be assumed to apply; i.e. RIA quality control typically focuses on the quality of evidence (and on costs and impacts on businesses more often than on benefits and impact on citizens2) and compliance with applicable rules and procedures, whereas quality control of stakeholder engagement and ex post evaluation aims primarily at ensuring that formal and methodological requirements are met.

As of end 2020, about 80% of jurisdictions declared to have a government body outside the ministry sponsoring the regulation that is responsible for reviewing the quality of RIA. This confirms the central role of RIA scrutiny in oversight systems and, as shown in Figure 3.5, it continues the upward trend observed since 2014, when more than one-third of jurisdictions did not have such a body. Coverage is roughly equivalent for primary laws and subordinate regulation. Approximately 75% of all reported ROBs have RIA quality control among their responsibilities, and about more than 70% of these are located within government. Non-departmental oversight bodies also focus strongly on this function. In addition, about one-quarter of jurisdictions reported having a specific parliamentary committee or other parliamentary body with responsibilities for reviewing the quality of the RIA system as a whole.

In just under 40% of jurisdictions (in similar proportions both primary laws and subordinate regulations), the oversight body (or bodies) in charge of RIA scrutiny has some sort of sanctioning power, i.e. it can return the RIA for revision if it deems it inadequate. Figure 3.6 shows the reasons that ROBs can invoke to return a RIA in that case. Cost assessment deficiencies are the most widespread criteria, whereas lack of effective consultation and assessment of alternative options seems to carry comparatively less weight (further details on RIA requirements can be found in the section on Regulatory Impact Assessment earlier in this report). In all but a few cases, this sanctioning power can however be overturned by means of an active decision; e.g. from cabinet, a minister or a high-ranking official.

As far as RIA scrutiny is concerned, the number of jurisdictions in which ROBs have sanctioning power has not increased over the period 2014-2020. These figures do not capture, however, instances where quality control takes place in more consensus-oriented settings (e.g. recommendations from the ROB will be adhered to even if there is no prospect of a formal sanction). In addition, the number of jurisdictions with a ROB in charge of RIA quality control whose mandate is grounded on a legally binding document, which may be considered a proxy for these bodies’ influence, has grown from 18 in 2014 to 29 in 2020.

The effectiveness of RIA systems can be undermined in the absence of a systematic and effective obligation for legislative proposals to undergo RIA. If legislative proposals are arbitrarily exempted from ex ante impact assessment, or if RIA obligations can be circumvented easily, regulatory quality is bound to suffer. Only a few countries report that RIA is always conducted without exception: Austria, Canada, Estonia, Finland, France, Germany, Korea, Lithuania and Spain (in all but three of them, this applies to both primary legislation and subordinate regulation proposals). Decisions to waive RIA should therefore be exceptional, transparent and subject to systematic scrutiny. This is however an area where there is still room for improvement. As discussed in the Regulatory Impact Assessment section and shown in Figure 3.7, only a minority of OECD members currently publish decisions that RIA will not be conducted where it ought to have been. There also remains nearly two-thirds of OECD members without a body responsible for reviewing the decision made by officials about whether a RIA is required. This means that, in practice, in a majority of OECD members, exception mechanisms can be used to bypass RIA with little scrutiny on whether this decision is appropriate or proportionate to the regulatory proposal at hand.

Despite their critical importance for regulatory quality, oversight of ex post evaluations remains underdeveloped compared to quality control of RIA. The share of jurisdictions with a body outside the unit conducting the evaluation in charge of reviewing the quality of ex post evaluations has grown compared with 2014 for both primary laws and subordinate regulation. However, in both cases, these still account for under one third of jurisdictions. In addition, only a fraction of jurisdictions with a body in charge of reviewing the quality of individual regulations’ ex post evaluations report to do so for all of them.

Quality control of ex post evaluations is generally performed by ROBs at the centre of government, which often share this responsibility with bodies external to government or interdepartmental bodies. In most jurisdictions with a body in charge of reviewing individual regulations’ ex post evaluations, ROBs provide feedback or advice during the preparation of ex post evaluations and also issue formal opinions on their quality. As shown in Figure 3.8, uptake of both these practices has increased in relative terms since 2014, although absolute figures remain low and few OECD members make ROBs’ formal opinions publicly available.

Although overall progress in the development of robust and systematic oversight of ex post evaluations of regulation remains slow, some OECD members offer promising examples. Germany is one of the countries that have stepped up efforts in this area. In November 2019, the country adopted a more holistic and systematic approach requiring independent quality assurance for all internal ex post evaluations and all ex post evaluations of legislative proposals exceeding EUR 5 million in annual compliance costs. In the same vein, in 2020 Lithuania institutionalised the ex post assessment of regulations and designated the Ministry of Justice as dedicated function for ex post evaluation coordination.

When considering the period 2014-2020, some progress can also be observed, in relative terms, in the number of jurisdictions with a dedicated body in charge of reviewing the quality of ad hoc reviews of the regulatory stock (e.g. administrative burden reviews or in-depth reviews). The same applies to those with a body responsible for reviewing the quality of ex post evaluations of packages of legislation. However, in both cases, and both for primary laws and subordinate regulations, these only represent a small minority of surveyed jurisdictions (see Figure 3.9). Further efforts will thus be required to develop oversight in these areas, which are crucial for ensuring full implementation of the 2012 Recommendation and, ultimately, regulatory quality.

As of end 2020, approximately 30% of jurisdictions declared to have a mechanism (e.g. a body, unit or network) to co-ordinate ex post evaluation efforts across the public administration from a whole-of-government perspective, thus signalling their awareness of the importance of joined-up approaches for ensuring laws and regulations remain relevant and fit for purpose.

In about 60% of jurisdictions, regulators are formally required to consider consultation comments when developing the final regulation (see section on stakeholder engagement earlier in this report for more details). This proportion applies to both primary laws and subordinate regulation. A common approach to ensuring accountability in this regard consists of requiring a review by a standing or central body to oversee that related practices are up to standard. About half of the jurisdictions with formal requirements on stakeholder consultation declared to have such a mechanism – again, in similar proportions for primary laws and subordinate regulation. Quality control of stakeholder engagement activities in this context is often carried out by ROBs at the centre of government (more than 60% of them have it in their mandate), sometimes with the involvement of bodies external to government or non-departmental bodies. In only a handful of cases are regulators held accountable by means of judicial reviews. For both approaches, uptake has remained stable compared with 2017.

The OECD Recommendation of the Council on Regulatory Policy and Governance stresses the importance of assessing the functioning of regulatory management tools as part of governments’ efforts to evaluate the performance of regulatory policy. Principle 6 of the Recommendation encourages members to publish regular reports on the performance of regulatory policy and reform programmes and the public authorities applying the regulations, including information on how regulatory tools such as RIA, public consultation practices and reviews of existing regulations are functioning in practice. In the same vein, the OECD has developed a Framework for Regulatory Policy Evaluation to assess the success of regulatory policy in achieving policy objectives in the most efficient manner and bringing about improvements in growth and societal welfare (OECD, 2014[6]). ROBs have a decisive role to play in this context. In addition to engaging in evaluative work in their own right, they can notably promote concerted approaches across the public administration to ensure that relevant information already produced by regulatory management systems is collected systematically as indicators, and that measurement and assessment efforts encompass all relevant domains of regulatory reform instead of focusing only on certain aspects such as the cost of complying with administrative obligations (Radaelli and Fritsch, 2012[7]).

OECD members still need to step up their efforts to develop comprehensive monitoring and evaluation mechanisms for regulatory management tools. Although RIA is a cornerstone of regulatory management in most countries, performance measurement of this tool is still not fully transparent or systematic. Approximately one third of jurisdictions still do not publish online reports on the performance of their RIA system and in many of them reporting is not regular but ad hoc (see Table 3.1). In the same vein, monitoring the appropriate functioning of RIA systems by means of performance indicators or opinion surveys is still very far from being commonplace.

The need for improved monitoring and evaluation is even stronger for the other two regulatory management tools. Assessing the effects of ex post evaluations of regulation are particularly valuable to identify areas for improvement and reform. Their usefulness is notably illustrated by the work carried out by the EU’s Regulatory Scrutiny Board to draw forward-looking conclusions from its scrutiny of ex post evaluations, both from a methodological (e.g. recurrent design flaws) and an institutional perspective (e.g. potential biases and conflicts of interest) (Regulatory Scrutiny Board, 2019[8]).

However, only a handful of jurisdictions assess the effectiveness of ex post evaluations in improving the regulatory stock. In 20203, less than one-fifth of them declared to have done so in the previous five years and made findings publicly available. In the same vein, there are still few instances of reports being published online on the performance of the ex post evaluation system (i.e. to understand how it functions in practice) and many of them are not systematic. Moreover, the vast majority of jurisdictions do not use indicators on the percentage of ex post evaluations that comply with formal requirements/guidelines or opinion surveys to monitor the usefulness or quality of ex post evaluations. Although these figures are partly explained by the fact that relatively few countries undertake ex post evaluation on a systematic basis, the current level of effort remains insufficient.

Monitoring and evaluation of stakeholder engagement has not progressed in recent years either and it remains underdeveloped. Approximately one third of jurisdictions publish reports on the performance of consultation practices on improving draft regulations, nearly always on an ad hoc basis. Only seven jurisdictions reported to collect indicators on the percentage of consultations that comply with formal requirements or guidelines, and only three collect indicators on the results of surveys on the usefulness or quality of stakeholder consultations. Moreover, only the European Union declared to evaluate consultation of foreign stakeholders.

Previous OECD work focusing on EU Member States (OECD, 2019[9]) highlights that, although members rarely review the performance of their consultation systems and how they work in practice, evaluations have demonstrated that they can be a powerful tool when performed. Indeed, these evaluations provide insights to improve the effectiveness and ultimately also the acceptance of consultation channels amongst stakeholders. The Netherlands, for example, have reviewed the extent to which their online consultation system was valued by citizens, companies, and departmental staff, as well as whether the objectives of the legislative process were being achieved. Review results have also served to identify weaknesses in the consultation system, such as the relative lack of visibility for citizens and businesses as to how their inputs are taken into account. The European Commission, in turn, reviewed its stakeholder consultation practices prior to updating its consultation system in 2015 and, subsequently, in 2018-2019 as part of a broad-based stocktaking exercise on its commitment to Better Regulation (European Commission, 2019[10]).

Well-functioning regulatory oversight is crucial to bridge the gap between political commitments and formal requirements on the one hand and effective implementation on the other hand. In addition, it is necessary to ensure a whole-of-government approach and thus the effective uptake of the 2012 Recommendation in its entirety. Under the current, rather exceptional circumstances, ROBs have an essential role to play in ensuring transparency in emergency rulemaking as well as due scrutiny. In addition, they can also contribute to a sustainable economic recovery by ensuring that regulatory decisions are forward-looking.

Previous sections in this chapter refer to the importance of seamless coordination and well-conceived institutional arrangements obeying to efficiency and complementarity criteria to avoid unnecessary overlaps and fragmentation. It is equally important that ROBs can adapt timely to emerging needs and challenges, such as those stemming from rapid technological change and global threats, in order to help address them. Survey data show that a number of governments are taking steps in this direction. Almost half of jurisdictions declared to have at least an oversight body focusing on innovation-friendly regulation, e.g. by helping ministries and regulators take into account impacts of regulation on innovation. Moreover, nine ROBs have expanded their scope of intervention to cover additional areas (e.g. regulation of new technologies) and seven have otherwise had their mandate adjusted (in most cases, to adapt to changed institutional settings or improve workflows). In the same vein, about 40% of jurisdictions reported having a body in charge of overseeing regulatory quality during a crisis, for instance in the context of emergency rulemaking (see Figure 3.10 for further details).

The growing complexity and uncertainty levels affecting decision-making mean that ROBs need sufficient capacity to anticipate risks and understand potential innovation pathways and outcomes. The European Commission, for example, has decided to embed strategic foresight into its working methods (including to inform the design of new initiatives and the review of existing ones), and the mandate of its ROB, the Regulatory Scrutiny Board (RSB), has been expanded to include foresight (European Commission, 2020[11]). ROBs can indeed play a decisive role in promoting innovative approaches to regulatory policy and contributing to bring about the necessary changes in terms of institutional culture and mind-sets. Several developments at the country level are worth highlighting in this regard (see Box 3.3).

Appropriate execution of regulatory oversight functions also requires appropriate resourcing, especially in light of the above-mentioned additional needs in terms of analytical capacity. Reporting in this respect is however neither comprehensive nor fully standardised.4 Therefore, currently available evidence does not allow determining the extent to which resources at the disposal of ROBs are commensurate to either needs or the challenges they face. According to 2020 data (see Figure 3.11), across all jurisdictions, 23 ROBs reported changes in either financing or staff. In real terms, nine ROBs had their budget increased over the reporting period and three indicated that it had decreased. Staff numbers, in turn, grew for twelve ROBs and diminished for ten of them.

In addition to the institutional design elements and related requirements discussed earlier in this chapter, there are other factors that, while generally harder to track and measure given the “intangible” nature of some of them, should not be overlooked if regulatory oversight is to be effective. These factors include ROBs’ legitimacy and credibility as well as their influence and ability to elicit buy-in from those involved in better regulation across government through upstream work and the provision of ongoing guidance, advice and support.

In Portugal, for example, the Technical Unit for Legislative Impact Assessment (UTAIL) has developed strong and productive relationships with government officials involved in RIA, from high-level officials to technical staff. UTAIL is also responsible for RIA capacity building across the public administration (including methodological guidance) and carries out important upstream work in that capacity. Its advice is valued by members of the executive, which contributes to the development of better regulation in Portugal. UTAIL’s advice and networking efforts are also paying off in terms of the RIA procedure itself: while having no formal power to alter assessments whose analytical quality is deemed insufficient, in practice it is able to use its good relations and expertise to find compromises with the analytical services.

Australia’s Office of Best Practice Regulation (OBPR), in turn, offers valuable examples of how ROBs can support a coordinated and comprehensive approach to analysis and work upstream to demonstrate the value of better regulation tools for ministries across government (see Box 3.4).

Maximising the value added of regulatory oversight processes and structures in the years to come will require a thorough understanding of their performance as well as its underlying factors. Despite promising examples and new opportunities offered by emerging analytical tools and methods, there is still much to be improved in that respect.

As of end 2020, approximately two-thirds of jurisdictions indicated that reports were prepared on the effectiveness of at least one ROB responsible for quality control of regulatory management tools; e.g. containing information on its activities, the fulfilment of its mission, or results of perception surveys on its performance and value added. The vast majority of them declared to publish those reports. In most cases, reporting takes place on an annual basis.

Generally, there continues to be relatively scant evidence on the impacts of regulatory oversight on regulatory improvement, let alone broader policy goals. Among other reasons, this owes to the fact that ROBs’ reporting activity focuses primarily on implementation (e.g. number of items scrutinised, turnaround times) and conformity with formal requirements (which tend to be easier to track and measure) rather than effectiveness and outcomes.

Certain jurisdictions are however showing the way by deploying considerable efforts to monitor and evaluate the results of their ROBs’ work. In its 2019 annual report, the Norwegian Better Regulation Council published performance indicators seeking to capture, among other aspects, the effect of the Council’s statements in which it had deemed RIAs not to be fit for purpose. This report also included and assessment of the general trends and developments regarding RIAs within the Council’s remit and any recurring problems, as well as an overview of the Council’s guidance and information activities to foster effective regulations. Mexico’s CONAMER, has in turn developed an "indicators for results" approach encompassing indicators to assess its contribution to reducing regulatory burden (Comisión Nacional de Mejora Regulatoria, 2019[15]). In Korea, white papers for Regulatory Reform are published on an annual basis including a regulatory reform satisfaction index, and the EU’s Regulatory Scrutiny Board publishes key performance indicators including on quality improvements subsequent to interactions with European Commission services in its oversight capacity.

Moreover, as discussed in OECD case study work (OECD, 2018[3]), a number of non-departmental bodies use surveys to collect feedback on the quality and impact of their work. Moreover, in some jurisdictions ROBs must undergo external evaluation on a periodic basis. Box 3.5 presents selected examples of both practices.

In addition, there are promising examples of how new technology-based solutions and analytical tools can help improve our understanding of ROBs’ performance and its determining factors in order to maximise the value added of regulatory oversight processes and structures. A recent study, for example, uses supervised machine learning algorithms to identify major change requests in RSB opinions and text similarity measures to identify changes between draft and final versions of impact assessment reports (see Box 3.6).

In the same vein, Australia’s OBPR are developing a bespoke IT system for RIA aimed at improving workload management related to RIA scrutiny (e.g. via automated processes, enhanced information management and system notifications for better work prioritisation) as well as the quality of impact analysis advice. This system is expected to provide a system-wide perspective by capturing a richer set of data and allowing for reporting and analysis of aggregate data (e.g. to identify recurrent issues in RIAs from a given ministry and tailor capacity building accordingly). Crucially, it is conceived to help OBPR understand what kind of feedback is more effective at each stage of the policy cycle and target their efforts accordingly. By doing so, the system can also contribute to improving impact analysis across Government.

It could be extremely valuable to exploit the full range of opportunities offered by emerging analytical tools and methods, such as those illustrated by the above-mentioned examples, to improve the better regulation community’s understanding of ROBs’ impact and, ultimately, enhance regulatory quality.

References

[16] ATR (2019), Dutch Advisory Board on Regulatory Burden. Annual report 2019, https://www.atr-regeldruk.nl/wp-content/uploads/2020/06/2019-ATR-annual-report.pdf (accessed on 5 March 2021).

[15] Comisión Nacional de Mejora Regulatoria (2019), Informe de avances en la implementación de la Estrategia Nacional de Mejora Regulatoria y de la Comisión, https://www.gob.mx/conamer/documentos/informe-de-avances-de-en-la-implementacion-de-la-estrategia-nacional-de-mejora-regulatoria-y-de-la-comision-nacional-de-mejora-regulatoria.

[12] Denmark’s Agency for Digitisation (2020), Secretariat for digital-ready legislation, https://en.digst.dk/policy-and-strategy/digital-ready-legislation/secretariat-for-digital-ready-legislation/.

[11] European Commission (2020), Strategic foresight, https://ec.europa.eu/info/strategy/priorities-2019-2024/new-push-european-democracy/strategic-foresight_en (accessed on 15 December 2020).

[10] European Commission (2019), Better regulation: taking stock and sustaining our commitment, https://ec.europa.eu/info/law/law-making-process/planning-and-proposing-law/better-regulation-why-and-how/better-regulation-taking-stock-and-sustaining-our-commitment_en (accessed on 5 March 2021).

[13] Government of Canada (2019), External Advisory Committee on Regulatory Competitiveness, https://www.canada.ca/en/government/system/laws/developing-improving-federal-regulations/modernizing-regulations/external-advisory-committee-regulatory-competitiveness.html.

[2] OECD (2020), Network of Senior Officials from Centres of Government (CoG), https://www.oecd.org/governance/cog.htm.

[5] OECD (2020), Regulatory Management and Oversight Reforms in Thailand: A Diagnostic Scan, OECD, Paris, https://www.oecd.org/regreform/RIA-Thailand.pdf.

[9] OECD (2019), Better Regulation Practices across the European Union, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264311732-en.

[3] OECD (2018), Case Studies of RegWatchEurope regulatory oversight bodies and of the European Union Regulatory Scrutiny Board, https://www.oecd.org/gov/regulatory-policy/regulatory-oversight-bodies-2018.htm.

[6] OECD (2014), OECD Framework for Regulatory Policy Evaluation, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264214453-en.

[7] Radaelli, C. and O. Fritsch (2012), Measuring Regulatory Performance – Evaluating Regulatory Management Tools and Programmes, http://www.oecd.org/gov/regulatory-policy/2_Radaelli%20web.pdf.

[8] Regulatory Scrutiny Board (2019), Annual Report 2019, https://ec.europa.eu/info/sites/info/files/rsb_report_2019_en.pdf (accessed on 5 March 2021).

[4] RegWatchEurope (2020), Further Development of Regulatory Oversight at EU-level. Joint Discussion Paper by RegWatchEurope, https://www.regwatcheurope.eu/?p=282.

[1] Renda, A. (Forthcoming), Defining and Contextualising Regulatory oversight and Co-ordination.

[17] Senninger, R. (2020), Meet the critics: Analyzing the EU Commission’s Regulatory Scrutiny Board through quantitative text analysis, http://dx.doi.org/doi:10.1111/rego.12312 53.

[14] UK Government (2019), Regulation for the Fourth Industrial Revolution: White Paper, https://www.gov.uk/government/publications/regulation-for-the-fourth-industrial-revolution.

Notes

← 1. Approximately one third of jurisdictions also reported the existence of networks of regulators that are involved in the exchange and dissemination of good regulatory practices. These networks are however not considered for analytical purposes here.

← 2. As discussed later in this section, cost assessment deficiencies are the most widespread criterion allowing oversight bodies to return RIAs for review. This can be considered a proxy for their relative importance in RIA oversight mechanisms.

← 3. Based on data from 38 OECD member countries and the European Union.

← 4. For example, in the 2017 OECD survey on regulatory oversight bodies, data on resources were only reported for about half of all ROBs in charge of quality control functions. In addition, comparison is not straightforward given that in some cases figures were reported for an entity as a whole rather than actual oversight functions.

Metadata, Legal and Rights

This document, as well as any data and map included herein, are without prejudice to the status of or sovereignty over any territory, to the delimitation of international frontiers and boundaries and to the name of any territory, city or area. Extracts from publications may be subject to additional disclaimers, which are set out in the complete version of the publication, available at the link provided.

© OECD 2021

The use of this work, whether digital or print, is governed by the Terms and Conditions to be found at http://www.oecd.org/termsandconditions.