Annex F. Methodology for the OECD Digital Government Index
The OECD Digital Government Index (DGI) monitors the implementation of the OECD Recommendation of the Council on Digital Government Strategies, adopted on 15 July 2014. The recommendation calls for a paradigm shift from e-government to digital government, bringing governments closer to citizens and businesses through the adoption of strategic approaches to the use of digital technology and data to spur more open, participatory and innovative governments (OECD, 2014[1]).
The DGI draws upon the long-standing work of the OECD advising governments to strategise with digital technologies and data for improved and joined-up public services and operations, as well as increased trust in public institutions, as outlined in the OECD Digital Government Policy Framework (OECD, 2020[2]). The framework is a policy instrument to help governments design and implement policies to become digitally competent, and it frames the methodology and survey for the DGI across the six dimensions for digital maturity in the public sector:
Digital by design: when a government governs and leverages digital technologies to rethink and re-engineer public processes, simplify procedures, and create new channels of communication and engagement with stakeholders.
Data-driven public sector: when a government values data as a strategic asset and establishes governance, access, sharing and re-use mechanisms for improved decision making and service design and delivery.
Government as a platform: when a government deploys a wide range of platforms, standards and tools to foster integration and coherence in the public sector as well as to help teams focus on user needs in public service design and delivery.
Open by default: when a government opens up the public government data and policy-making processes (including algorithms), within the limits of existing legislation and balancing the national and public interest.
User-driven: when a government accords a central role to people’s needs and convenience in the shaping of processes, services and policies; and by adopting inclusive mechanisms that enable this to happen.
Proactiveness: when a government anticipates people’s needs and respond to them rapidly, avoiding the need for cumbersome data and service delivery processes.
Based on the Policy Framework, the DGI is a composite index composed of these six indicators, each equally weighted (1/6 each). The DGI additionally includes four transversal facets for a qualitative analysis on the comprehensiveness of digital government reforms across participant countries: strategic approach, policy levers, implementation and monitoring. Data for the first and pilot edition of the DGI were collected through the OECD Survey on Digital Government 1.0, including answers from 33 countries (29 OECD countries and 4 key partner countries)1.
The statistical analyses confirmed that the 210 items in the 6 dimensions measure the underlying concepts. The results obtained from the statistical analyses justified discussing country differences with both the composite score and the dimensions scores.
Four types of statistical analyses were conducted to ensure the highest standards of reliability and validity of the DGI. Descriptive statistics were used to analyse the distribution of dimension scores, with no item whose average value was 0.0 or 1.0. The validity of all the items included in the composite scores has been confirmed. Correlation coefficients between item scores and dimension scores were calculated in order to check construct validity. Polyserial correlation2 was employed if the number of categories for an item was less than 4, otherwise Pearson’s correlation3 was employed. Items whose correlation coefficients were less than 0.1 were reallocated or eliminated (Ubaldi and Okubo, 2020[3]).
Cronbach’s alpha coefficients (α) – computed to verify the reliability of the dimension scores – confirmed the internal consistency for all dimensions (the coefficient ranged from 0.67 for the dimension of open by default to 0.91 for digital by design). In addition, the correlation between dimensions was analysed. The correlations ranged from 0.20 between open by default and proactiveness to 0.84 between user-driven and digital by design, implying that the dimensions measured related concepts. This confirmed the constructed validity of the Survey on Digital Government 1.0. Lastly, the correlation coefficients between the composite score and the dimension scores confirmed the dimensions measure similar aspects with the composite score, with correlations coefficients ranging from 0.65 for open by default to 0.93 for user-driven (Ubaldi and Okubo, 2020[3]).
The use of digital technologies and data in the public sector has also been of interest to other international and multilateral organisations, with a particular focus on assessing the progress of e-government readiness and the availability of digital public services. Three measurement efforts stand out: the United Nations E-Government Survey, and the E-Government Benchmark and the Digital Economy and Society Index (DESI) of the European Commission (EC).
The United Nations has developed the global and long-standing E-Government Survey, a quantitative composite index to assess the readiness and capacity of public sector organisations to deliver digital services based on website assessment, telecommunications infrastructure and human resource endowment.
The European Commission has advanced the measurement work on digital services through two instruments. First, the EU E-Government Benchmark based on the Tallinn Ministerial Declaration of 2017, the Digital Single Market Vision and broader EU2020 goals. It is a monitoring instrument used by the EC to provide insight into the use of information and communication technology (ICT) in the public sector. Among its components, it evaluates the maturity of public services in terms of user centricity (availability of online services), transparency (implementation of good transparent service procedures), cross-border services and use of key technological enablers.
Second, the EC measures the broader role of digital technologies and data in EU countries through the Digital Economy and Society Index. The DESI encompasses five dimensions to assess Europe’s digital performance, with a dedicated dimension for the availability of public services through digital channels – along with connectivity, human capital, use of Internet services and integration of digital technologies.
Compared to other measurement efforts, the DGI values the “digital by design” principle, where digital technologies are systematically applied to improve policies, services and processes, broadening the scope of citizens’ choices to interact with government, regardless their preferred channel (digital or not). The DGI and OECD vision on digital government and public sector data acknowledges the importance of shared tools and mechanisms to attain the full potential of digital technologies, as they enable integration across channels and organisations. In this sense, the DGI covers the implementation of cross-government digital and data standards, key enablers, and principles, as they have a major impact on whole-of-government approaches to a coherent design and provision of services, public sector operations, and decision-making processes.
References
[1] OECD (2014), OECD Recommendation on Digital Government Strategies, OECD, Paris, https://legalinstruments.oecd.org/en/instruments/OECD-LEGAL-0406.
[2] OECD (2020), “The OECD Digital Government Policy Framework: Six dimensions of a Digital Government”, OECD Public Governance Policy Papers, No. 02, OECD Publishing, Paris, https://doi.org/10.1787/f64fed2a-en.
[3] Ubaldi, B. and T. Okubo (2020), “OECD Digital Government Index (DGI): Methodology and 2019 results”, OECD Working Papers on Public Governance, No. 41, OECD Publishing, Paris, https://dx.doi.org/10.1787/b00142a4-en.
Notes
← 1. For detailed information on countries’ composite score and score per dimension, please consult Table F.1 (Digital government index dimension scores, 2019) [https://doi.org/10.1787/888934260054].
← 2. Correlation coefficient between a continuous variable and a discrete variable.
← 3. Correlation coefficient between two continuous variables.