1. Connecting cities’ innovation and data use to residents’ well-being
Based on previous research, literature and case studies, this chapter provides an overview of public sector innovation and data use in city government, and their potential to help cities improve residents’ well-being. The chapter proposes that, while anecdotal evidence suggests local public sector innovation and data use can improve residents’ well-being, such outcomes are not guaranteed. By developing certain innovation and data use capacities, cities can increase the likelihood that these activities will improve residents’ lives. However, despite growing interest in local public sector innovation and data use, a system to evaluate how they correlate with residents’ well-being does not yet exist. This chapter concludes by introducing an OECD methodology to connect the dots between local public sector innovation, data use and well-being, the findings of which are reviewed in later chapters.
Around the globe, cities are embracing innovation and data use to support how they run the city and address residents’ needs. The OECD/Bloomberg Philanthropies (2019[1]) report, Enhancing Innovation Capacity in City Government, focused on why local public sector innovation occurs in cities, what drives it and where innovation is expected to deliver better outcomes for residents.
The 2019 report also established an analytical framework for assessing public sector innovation capacity. Tenets of this framework included a strategic approach (including clear goals), organisational and staff structure, data management capacity, openness to partnerships and the importance of evaluating innovation outcomes. That framework laid the foundation for this report, which analyses the impact of cities’ innovation activity on residents’ well-being through five components: (1) strategy, goals and approaches; (2) organisational and staff structure; (3) funding; (4) data use; and (5) outcome evaluation (Box 1.1).
Following the next steps laid out in that report, this report incorporates results from the 2018–20 OECD/Bloomberg Survey on Innovation Capacity on Cities (Box 1.1) and the What Works Cities (WWC) Assessment (Box 1.2) to dive deeper into cities’ data use and its impact alongside innovation capacity on residents’ well-being, and provides recommendations for policy makers to leverage these approaches for improved outcomes. A dedicated website mapping innovation capacity in cities is also publicly accessible at https://cities-innovation-oecd.com/.
Overview and intent of the Survey
In 2018, the OECD and Bloomberg Philanthropies joined forces to assess how cities around the world develop their capacity to innovate. To support this research, a Survey on Innovation Capacity in Cities was developed and sent to over 100 cities worldwide. Survey results featured in an online innovation platform and were analysed in the 2019 report Enhancing Innovation Capacity in City Government.
Building on these successes, the OECD/Bloomberg team launched a second project phase in 2020 with an updated survey to deepen understanding of data use and innovation capacity in cities, and how they generate better well-being outcomes for residents.
Innovation components examined by the Survey
The 2020 OECD/Bloomberg Philanthropies Survey, like its 2018 predecessor, comprised five sections designed to capture why and how cities invest in and manage their innovation and data-use capacity:
Innovation definition, goals, and approaches: Examines how cities build and maintain innovation capacity; what innovation means to each city; and what are cities’ goals, strategies and approaches.
Organisational and staff structure: Examines how innovation work is organised and staffed within city administrations, e.g. whether an office or certain staff is dedicated to innovation.
Funding and resources: Examines funding and resources dedicated to developing and maintaining cities’ innovation capacity (as distinct from funding for programmes), e.g. to expand innovation staff.
Data use for innovation. Examines how each city generates, manages and/or shares data.
Outcomes monitoring and evaluation: Examines whether and how cities assess outcomes related to their innovation strategy and goals.
Changes and additions in the 2020 version of the survey
A combined 147 cities responded to the 2018 and 2020 versions of the survey. To gain deeper insights into each innovation component, 24 new questions were added to the 2020 version. Eighty-nine cities answered the 2018 survey, while 70 answered the 2020 version. Of the latter 70 cities, 12 had also responded to the 2018 survey version, while 58 were completely new respondents. In addition, another 13 cities that had responded to the 2018 survey version also responded to an abridged version of the 2020 survey, containing only the 24 new questions.
This hybrid approach was designed to facilitate participation in the updated 2020 survey, allowing for a wider analysis in this report by using responses to both versions. The list of respondents can be found in Annex 1.A.
Source: OECD/Bloomberg (2018-20), Philanthropies Survey on Innovation Capacity in Cities; OECD (2019[1]), Enhancing Innovation Capacity in City Government, OECD Publishing, Paris, https://doi.org/10.1787/f10c96e5-en.
Overview and intent of the WWC Performance Assessment
Launched by Bloomberg Philanthropies in 2015, What Works Cities (WWC) aims to enhance cities’ ability to use data and evidence to address challenges and improve residents’ lives. Eligible cities can receive free support through the programme, including an assessment of their data practices, access to strategic resources, connection to a network of cities, technical assistance, coaching, etc.
Practices examined by the WWC Performance Assessment
The WWC performance assessment framework uses 45 criteria, grouped under eight foundational practices, to measure the extent to which leaders incorporate data and evidence in city management and decision making. These include:
Data governance (5 criteria): Governance of data within city administration, e.g. keeping a comprehensive data inventory, documenting data governance responsibilities, etc.
Evaluations (5 criteria): Use of data to evaluate practices, programmes or policies.
General management (9 criteria): Support for data and evidence by city leadership, and the existence of staff dedicated to developing/implementing data-use programmes and strategies.
Open data (4 criteria): Local government’s commitment to sharing data publicly.
Performance and analytics (7 criteria): Use of data to monitor performance and advancement toward strategic goals.
Repurposing (4 criteria): Use of data by local government to efficiently allocate, re-allocate or discontinue programmes.
Results-driven contracting (7 criteria): Use of data by local government to inform and evaluate key procurements, contracts and/or grants.
Stakeholder engagement (4 criteria): Local government’s commitment to engage with data users (residents and partners) in the design and implementation of data policies and practices.
Source: What Works Cities (2018[2]), “What Works Cities Assessment Guide: Certification Criteria”, https://medium.com/what-works-cities-certification/what-works-cities-certification-assessment-guide-5c514f1dff1b (accessed on 14 January 2021); Bloomberg Philanthropies (2020[3]), “What Works Cities”, website, https://whatworkscities.bloomberg.org/ (accessed on 18 January 2021).
Facing myriad societal challenges, the public sector is embracing innovation to improve internal operations and service delivery in hopes of yielding better outcomes for residents. There is growing interest in the potential of public sector innovation to “improve the efficiency in how resources are used, the quality of public services, and address a diverse range of societal challenges, including climate change, demographic pressures, urban congestion and social and economic inequality” (Arundel, Bloch and Ferguson, 2019[4]). Public sector innovations, executed through a variety of avenues and actors “typically improve services, sometimes in an impressive manner” (Goldsmith and Kleiman, 2017[5]). However, while municipal-level public sector innovation “has begat some of the most inventive policy…precisely because the trash has to be picked up even when the budget is getting squeezed,” a “thorough literature review finds few texts that assess the current wave of local innovation” (Goldsmith and Kleiman, 2017[5]).
The public sector’s interest in data has increased as well. This interest accelerated thanks to socio-economic and technological trends, including the growing capacity for data generation and collection, the power of data analytics, and the paradigm shift in knowledge creation and decision making. Data is increasingly leveraged to increase productivity, anticipate future trends and risks, improve local service delivery and inform decision making more broadly. The OECD report Data-Driven Innovation (2015[6]) affirms that “better access to and use of public sector data can lead to important value creation from economic, social, and good governance perspectives.” Data analytics can also help city governments “predict the causes of problems rather than just responding after they occur” (Goldsmith and Kleiman, 2017[5]).
Such benefits of public sector innovation and data use might be valuable for cities, which find themselves “under pressure to provide high-quality services to residents in cost effective ways” in the face of “increasing citizen expectations, decreasing government budgets, and changing demographics” (What Works Cities, 2015[7]). In a demanding time that requires municipal governments to do more with less, one solution is to leverage innovative practices such as qualitative and quantitative data, human-centred design and strategic partnerships to determine the programmes and services that work best for residents. The need to make city operations more efficient and services more accessible became even more imperative in the context of the COVID-19 pandemic.
Properly deployed, public sector innovation and data use can help cities respond to these challenges by generating public value that improves residents’ well-being. This value may include economising resources through internal efficiency, targeting service delivery (including digitally), facilitating resident engagement and feedback, informing cities’ plans for future challenges and supporting government transparency and accountability. Chapters 2 and 3 explore the potential for innovation and data use to create this value.
Unique aspects and challenges to public sector innovation
A methodology to measure public sector innovation is essential to evaluating its impact on well-being, but first requires a definition of public sector innovation. Public sector innovation is about “finding new ways to improve society, government itself, and the relationship between government and the public” (Janssen et al., 2017[8]).
While the 2005 Oslo Manual (OECD/Eurostat, 2005[9]) presented a measurement framework for innovation, it only applied to the business sector, despite innovation’s growing presence in the public sector as well (OECD, 2015[10]). Meanwhile, increased attention paid to innovation occurring in the public sector “has attracted a growing body of empirical research, motivated in part by the increasing demand for benchmarking the efficiency and quality of public services as well as identifying the factors that contribute to desirable innovation outputs and outcomes” (OECD/Eurostat, 2018[11]).
In response to this developing body of research “adapting the guidelines in the [2005 edition of the Oslo Manual] to develop surveys of innovation in public administration organisations”, the 2018 Oslo Manual provided a “conceptual framework and general definition of innovation that is applicable to all sectors in the economy”, including government and the broader public sector (OECD/Eurostat, 2018[11]). The 2018 Oslo Manual thus defines innovation as “a new or improved product or process that differs significantly from the unit’s previous products or processes, and that has been made available to potential users or brought into use by the unit” (OECD/Eurostat, 2018[11]).
Compared to the private sector, policy interest in public sector innovation concerns “how innovation occurs in order to increase its use to solve problems and improve outcomes”, including for residents (2019[4]). Public sector managers express that “innovation must make something better or have a goal to deliver better outputs”. Thus, this report uses the same definition of public sector innovation (Box 1.3) as the OECD and Bloomberg Philanthropies’ (2019[1]) report, Enhancing Innovation Capacity in City Government, based on research by the OECD Observatory of Public Sector Innovation (OPSI) and the Oslo Manual of Innovation.
Consistent with the OECD and Bloomberg Philanthropies’ 2019 report, Enhancing Innovation Capacity in City Government, this report defines public sector innovation as any service or product that has the following characteristics:
Source: OECD/Eurostat (2018[11]); OECD (2015[10]); OECD (2019[1]).
As Chapters 2 and 3 will explore in depth, cities increasingly embrace the potential of innovation and data use to improve their residents’ lives. According to the 2018 OECD/Bloomberg Survey responses, a majority of cities dedicate staff and funding to innovation, and half of them already adopted formal innovation strategies. In addition, 81% of cities report that data plays an important role in their innovation efforts and decision making, while more than seven out of ten cities publish open data online and share city data on public procurement in the name of transparency.
Most surveyed cities report that public sector innovation helps them improve service delivery and internal operations, anticipate and manage future challenges, improve resident outcomes and generate other benefits. Surveyed cities report successes that manifest in multiple ways, thanks to their innovation and data endeavours. Local public sector innovation shows potential to improve residents’ well-being (Box 1.4), whether that means using human-centred design to inform legislation on new forms of urban mobility (Seattle, WA, United States), leveraging smartphone GPS data to improve road conditions (Dublin, Ireland), or establishing a forum for co-creation that directly engages the community (Leipzig, Germany). However, while these examples imply tangible benefits for residents, many cities struggle to develop capacity in areas related to public sector innovation and/or data use—including key components that could demonstrate impact.
Using innovative approaches to change the culture of the city
In Leipzig, Germany, the “Leipzig Thinking Ahead” initiative serves as a co-ordination vehicle for civic participation, bringing together residents, elected officials, policy makers, and experts to address sustainable urban development. Leipzig Thinking Ahead fosters a culture of resident engagement and co-creation through virtual events, a future-planning series, conducting surveys, and even providing the use of Legos for residents to construct their own vision for the city.
Seattle, WA, United States, uses human-centred design to engage rideshare drivers and better understand their needs and preferences in relation to potential legislation on minimum compensation. The city designed an ethnographic outreach and engagement strategy that included intercept interviews, focus groups, a telephone town hall, and an online survey. By including perspectives from rideshare drivers, Seattle ensured that innovation efforts put them at the centre.
Moscow, Russia, has used the Our City portal since 2011. The electronic platform allows Muscovites to influence the timeliness and quality of repair work, report on maintenance conditions of urban facilities, report violations and evaluate the work of institutions. Citizens also monitor the work of the metro, the cleanliness of yards and parks, conditions at construction sites and in public transport, the work of clinics and prices in pharmacies. Since the portal’s creation, more than four million urban problems reported were resolved.
Leveraging data and digitalisation to increase safety and inform decision making
In Godoy Cruz, Argentina, the Sistema de Alerta Comunitaria (Community Alert System) app allows residents to notify a 24-hour monitoring centre of safety or health dangers. These warnings are vetted and broadcast to other cell phones in the area, giving advance notice to neighbours and saving lives. The app is used in over 10 000 households and is being replicated by other cities across the country.
Dublin, Ireland, has engaged in several Small Business Innovation Research (SBIR) challenges, including one focused on improving the cycling experience in the city. This SBIR led to the development of a bike light using sensor technology that gathers data on road surface quality, frequently used routes, and near-miss incidents to improve Dubliners’ cycling experience and inform decisions on new infrastructure plans. The initial trial collected data from 200 participants, presented to the city as an easily readable dashboard.
Montgomery, AL, United States, created a programme to identify and prevent blighted and abandoned properties by combining and analysing data from utility partners and housing codes.
Source: OECD/Bloomberg (2018-20), Philanthropies Survey on Innovation Capacity in Cities.
Uptake of innovation and data use has accelerated, but evaluation lags
At least half of surveyed cities report having three of the five components integral to innovation: a formal strategy, dedicated staff, and funding. In addition, nearly half of surveyed cities report systematic assessment or evaluation of some, but not all, impacts of either their innovation strategy or its outcomes. (It is likely that other surveyed cities conduct ad hoc evaluations of innovation activity despite not having a formalised strategy.) However, consistent with responses to the 2018 OECD/Bloomberg Survey (2019[1]), only 16% of cities report a systematic assessment or evaluation of both the impact of their innovation strategy and outcomes—making it the least common of the five components among surveyed cities.
This relative dearth appears consistent with the broader research landscape on local-level public sector innovation, which “has been descriptive in nature, often detailing programme elements rather than critically assessing new innovations or placing them in a larger context” (Goldsmith and Kleiman, 2017[5]). It is also significant, considering that evaluation of innovation strategy and outcomes help cities refine innovation implementation for greater impact, determine whether programmes and policies are working, guide budget and policy decisions, and advocate for more funding. Evaluating innovation outcomes can ensure public accountability and foster trust in city leaders, allowing voters to assess the results of a given innovation investment and voice their opinion on whether it should be continued. Without a method to systematically assess innovation outcomes, city governments cannot build evidence that innovation activity improves residents’ lives. According to Demircloglu (2017[12]), “there is not much established or well-developed research, theories, and cases that inform us of the conditions, types, measurement, outputs, and outcomes of innovation in the public sector context.”
The five public sector innovation components examined by the Survey (e.g. strategy, staffing, funding, data use and evaluation) are interdependent: building robustness in each component increases the likelihood that a city’s innovation will have deep and lasting impact. For cities to develop a strong evaluation capacity, they might first need a strategy that sets measurable goals, staff to perform programme assessment, funding to support staff, and basic data skills to draw quantifiable conclusions beyond anecdotal case study-type observations. Cities may struggle to build a system of assessment or evaluation of innovation if these other components are poorly developed.
Obstacles to evaluating innovation
Although cities might argue that evaluation capacity lags due to underdevelopment of the other components, it more likely indicates the need for a cultural shift of the public sector at large toward quantitative assessment, beyond just innovation. According to the 2020 OECD report Innovation for Development Impact (OECD, 2020[13]), even in countries “where an evidence culture is strong, the role of monitoring, evaluation and learning in innovation is weak…there is not yet a culture of evidence-based innovation—evaluation and evidence are often absent.”
Another obstacle to developing evaluation capacity lies in the wide-ranging applicability of innovation. Because some of the most significant outcomes of municipal government activity are qualitative – such as increased resident engagement, a higher sense of safety, or improved satisfaction with community outreach – cities may struggle to quantify and measure impact despite having some data on hand (Figure 2.23). Evaluating intangible outcomes is a challenge across the public sector, including for innovation. The 2018 Oslo Manual elaborates that:
The absence of a market alters both the incentives for innovation and the methods for measuring innovation outcomes [in the public sector] compared to the business sector. Without data on the cost or price paid for government services, outcome measurement has relied on subjective, self-reported measures, such as an increase in efficiency or improved user satisfaction (Bloch and Bugge, 2013). High-quality outcome measures are generally only available for specific innovations. Examples include the cost and benefits of new treatments or protocols in hospitals or new educational methods in schools. (OECD/Eurostat, 2018[11])
Examples from surveyed cities in Chapter 2 (e.g. Box 2.1, Box 2.2, Box 2.5) provide some quantitative evidence of innovation outcomes, but may fail to capture the full scope of those innovations’ impacts on resident well-being.
Another difficulty measuring –and especially comparing–public sector innovation at the city level through a quantitative universal framework is that such activity is most often hyperlocal, place-based, and context-specific. For example, the impetus for innovation in various cities featured in Box 1.4 differ greatly: Leipzig, Germany responded to calls for sustainable urban development while Godoy Cruz, Argentina prioritised resident safety, and Seattle, United States addressed fallout from a shock to the local labour market. While cities frequently face common problems, the nuances around specific local issues can influence the approaches that cities take, making it difficult to compare innovation activity on any universal scale. Even when quantifiable metrics exist around outcomes, the contextual incongruence of local-level innovation may complicate efforts to glean meaningful insights.
Establishing causality also poses a challenge. As discussed later in this chapter (see Methodological considerations and caveats), while this report strives to correlate public sector innovation components and data use practices to resident well-being outcomes, causality cannot be assumed for several reasons. While the analysis in this report corrects for certain factors, others yet unobserved might contribute to innovation outcomes.
Progress toward evaluation of innovation outcomes
While “challenges related to the measurement of public sector innovation are multiple and non-trivial” (OECD, 2015[10]), cities appear to be working toward greater evaluation capacity. Certain signs exist, including the 2018 OECD/Bloomberg Survey results that suggest cities with older innovation teams evaluate innovation outcomes and/or strategy at a higher rate than cities with newer teams (Table 2.2). Likewise, cities with newer innovation teams report not evaluating innovation outcomes or having no formal innovation strategy whatsoever at a higher rate than cities with older teams. These survey results and others suggest that cities are in fact evaluating outcomes, and the more present certain factors are (e.g. the tenure of innovation teams, access to stable funding, ongoing staff training, and rigorous data practices), the more they may evaluate outcomes. (For more on these trends, see “Evaluating outcomes can create feedback loops that lead to greater impact” in Chapter 2.)
Another sign that surveyed cities are increasing their evaluation capacity is that half report an innovation strategy and/or formal innovation goals–both foundational to fostering a culture of outcome assessment. This is significant because the trends identified in cities’ current and future innovation funding plans imply that most first prioritise establishing an innovation strategy and team, then pivot to building data use and evaluation capacity afterward (Figure 2.11).
Surveyed cities also report measuring the outcomes of innovation efforts in at least ten distinct categories to varying degrees (Figure 2.22). Though none of these categories are measured by a majority of cities, these findings suggest progress.
Data use is recognised as a valuable tool, but its application remains elusive
In recent years, local governments increasingly recognise the critical role that data can play in improving well-being outcomes for residents. Data use could also play a role in evaluating public sector policy and activity: “incorporating more sophisticated analytic tools will assist cities in a shift from measuring narrower activities to more systemic issues” (Goldsmith and Kleiman, 2017[5]).
This growing interest is partly due to the potential for leveraging the swaths of available data for insights into the behaviours, preferences, needs and difficulties of residents, businesses and public services in the city. As part of their daily operations, local governments generate and assemble a massive quantity of data with diverse forms and characteristics. Meaningful insights derived from this data could contribute to the processes of knowledge creation, innovation and policy making.
Nevertheless, data use is the second scarcest innovation component after outcome evaluation among surveyed cities. Just 39% of cities report that data plays a significant role in their innovation efforts and decision making, and less than half report using data to align budget processes with strategic priorities. While data use has a strategic role in supporting innovation, developing data capacity and culture within the public sector could also have positive impacts in other areas of budget and policy decision making, staff and resource allocation, programme evaluation, future risks anticipation and management.
Despite cities’ increased awareness and interest in leveraging data to guide decision making and inform policy, the potential of a data-driven public sector still eludes most governments. Building a data-driven culture within public sector organisations can be an intensive process. At every stage along the government-data value cycle, municipalities face diverse challenges and risks at the strategic, organisational and technical levels. From the lack of strategic data leadership and vision to insufficient staff with data capacity, suboptimal deployment of data can have broader implications for cities’ governance beyond innovation. Surveyed cities describe various obstacles to optimising data use for innovation, including a lack of data compatibility across policy areas, a lack of staff capacity to collect data, and that data are not shared among agencies. In addition, while it can take time to see tangible results in many cases, interruptions to the process might be encountered due to inconsistent or insufficient funding, mayoral or administrative turnover that deprioritises data use, or unforeseen events like COVID-19 that trigger a shift in resource allocation.
Other barriers to using data to inform decision making in city government include: a lack of staff and financial resources; limited knowledge and expertise in data; a lack of trust in data generated by city systems; old and incompatible systems for data collection and analysis; and challenges obtaining buy-in from stakeholders (What Works Cities, 2015[7]). Cities can also face regulatory hurdles around data, such as the lack of an enabling framework for data sharing across organisations and/or with other levels of government. Thus, cities are often compelled to focus data efforts on a limited number of policy areas and/or one-off projects because of the wide-ranging measurement agenda, which encompasses vastly different needs, and calls for a variety of skills, capacities and strategies.
Encouragingly, many surveyed cities report concerted efforts to break down siloes, which are common vestiges of traditional public sector organisations. Also encouraging, several cities report possessing “sufficient” data to advance innovation work in 19 sectors, mostly transport/mobility, land use/zoning, law enforcement, water and sanitation, and economic development. Use of data-driven decision making like in Syracuse, New York (United States) demonstrates the potential of data use by cities (Box 1.5).
Syracuse is one of the snowiest cities in the United States, receiving an average of over 100 inches of snow per year. The city turned to its data on salt purchasing and historical snowfall patterns to inform its budgeting and purchasing forecasts. By predicting salt usage and purchasing when the commodity is cheaper, the city was able to reduce its salt budget by over USD 800 000.
Source: Bloomberg Philanthropies (2020[14]), “What Works Cities (WWC) Certification (database)”, Unpublished.
Towards a local government framework for data governance
A coherent data governance framework constitutes the first step in building a data-driven organisation: it can forge a strategic vision and facilitate municipalities’ technical capacity to leverage data for residents’ well-being. By considering every stage of the government data value cycle, from data collection and storage to data analysis and publication, a governance framework can facilitate data sharing within and beyond the organisation, minimise security and privacy risks, and maximise the public value derived from the use and re-use of data. Indeed, an effective framework not only focuses on technical aspects such as data interoperability and standards, but also creates an enabling environment for systematic use of data for problem solving and decision making.
At the national level, most OECD governments work to put in place a framework that maximises the potential of data (OECD, 2019[15]). Even though many OECD countries have implemented regulations, standards, and strategies for data management and digital governments, these frameworks tend to be fragmented, as different public sector organisations are responsible for different aspects of data. Such fragmentation of internal organisation, and thus governance, impedes the public sector’s ability to integrate and manage data.
Drawing on the OECD’s considerable experience in digital government and government data, as well as extensive literature review on data governance, the OECD publication The Path to Becoming a Data-Driven Public Sector (2019[15]) proposes a common framework for public sector data governance to help standardise the concept and promote its implementation across countries. The national framework organises (non-exclusive) data governance elements into six groups under three layers:
2. Tactical layer, including (b) Capacity for coherent implementation and (c) Legal and regulatory framework
3. Delivery layer, including (d) Integration of the data value cycle, (e) Data infrastructure and (f) Data architecture
While this framework was formulated for national government, it remains relevant for the sub-national level, where municipalities increasingly seek to develop a more comprehensive and coherent approach to data governance. This report proposes a tailored model for data governance in the local public sector. With a focus on the strategic and tactical layers, the tailored framework provides municipalities with a structured approach to target key data governance elements that simultaneously generate public value and transition toward a data-driven organisation.
City governments aim to ensure residents’ well-being through the policies they put in place, the services they provide, and the rules and norms they establish. Innovation and data are key tools that local governments can use to meet this objective and, as any tool, they need to be examined in relation to the objective they serve. This section studies the links between public sector innovation and data use by city governments (the tools) with respect to residents’ well-being outcomes (the objective).
Assessing links between well-being, public sector innovation and data use in cities requires defining and measuring all three aspects. To do so, this report leverages the OECD regional and local well-being framework (OECD, 2014[16]; OECD, 2019[17]), the OECD/Bloomberg Survey on Innovation Capacity in Cities carried out in 2018 (OECD, 2019[1]) and 2020 (updated version for this report), and the What Works Cities Standard (Bloomberg Philanthropies, 2020[3]) – including its WWC Certification database 2018-20 (Bloomberg Philanthropies, 2020[14]). The OECD regional and local well-being framework guides the choice of well-being outcomes used in the analysis, while the Survey on Innovation Capacity in Cities and the What Works Cities Assessment framework provide, respectively, information on public sector innovation capacity and data use practices likely to affect residents’ well-being.
Measuring well-being outcomes in cities
The OECD regional and local well-being framework facilitates a multi-dimensional view of how life is for people in the place they live (OECD, 2014[16]; OECD, 2019[17]). The framework identifies 11 dimensions to assess people’s well-being outcomes: Jobs, Income, Housing, Access to services, Education, Civic engagement, Health, Environment, Safety, Community, and Life satisfaction. These include objective indicators (e.g. life expectancy) and subjective indicators (e.g. self-reported health) that contribute to an understanding of well-being that goes beyond material conditions. The original framework uses 13 baseline indicators for large OECD regions, but can be adapted to the specificities of any country, region or city, or to different assessment frameworks. For example, this framework was used to assess well-being in Mexican states in 2015 (OECD, 2015[18]), in Danish cities in 2016 (OECD, 2016[19]), in urban agglomerations in Córdoba, Argentina in 2018 (OECD, 2019[17]), and more recently across local authorities in Wales, United Kingdom (OECD, 2020[20]).
Three features of the OECD regional and local well-being framework emerge as key to our analysis, notably that well-being is (1) multi-dimensional, (2) people-centred and (3) shaped by governance and policy tools such as public sector innovation and data use. Well-being indicators therefore focus on outcomes that directly reflect people’s well-being (e.g. exposure to air pollution or having access to tertiary education) rather than on inputs, which tend to be the means to improve outcomes (e.g. investments in public transport or number of public universities).
Measuring well-being at the local level helps understand how local conditions and institutions, such as public sector innovation and data use, affect people’s lives. The OECD regional and local well-being framework shows that well-being outcomes are not only the consequence of individual characteristics, but also of place-based characteristics, including local institutions, governance, and tools for policy making such as public sector innovation and data use (Figure 1.1).
Source: Adapted from OECD (2019[17]), How's Life in the Province of Córdoba, Argentina?, OECD Publishing, Paris, https://doi.org/10.1787/97f189b1-en.
To measure well-being outcomes in cities for this report, the OECD collected 30 indicators across 11 well-being dimensions for 200 cities in the United States and 19 European countries. The main sources of well-being data for cities in the United States were the American Community Survey (ACS), the City Health Dashboard and Gallup US Daily, the Reflective Democracy Campaign, Who Votes for Mayor and Ballotpedia. For European cities, the main sources of data were Eurostat and the European Quality of Life Survey (EQLS) (see Box 1.6). While more than 100 indicators were reviewed, only the 30 most relevant indicators with good coverage across cities were retained for the final assessment (Table 1.1).
Due to the challenge of collecting comparable city-level data across countries, the quantitative analysis in this report relies on a sub-sample of the cities that participated in either the 2018 OECD/Bloomberg Survey on Innovation Capacity in Cities or in the What Works Cities Assessment framework, and for which it was possible to retrieve relevant well-being indicators. The sample was also restricted to cities in the United States and Europe because the low number of cities in Latin America, Asia, Africa and Oceania participating in the frameworks and with sufficient well-being data did not allow for a meaningful representation of these areas. Of note, while 112 cities come from the 2018 OECD/Bloomberg Survey on Innovation Capacity in Cities and 145 come from the What Works Cities Assessment framework, the final sample consists of 200 cities because 57 belong to both (Annex Table 1.B.1).
American Community Survey
The American Community Survey (ACS) publishes detailed information on the economic, housing and demographic characteristics of cities and regions in the United States. This source informs objective well-being indicators like the percentage of households living in affordable housing or the employment rate. For most indicators, the ACS provides comprehensive coverage across 163 American cities that participated in either the What Works Cities Assessment or the 2018 OECD/Bloomberg Survey on Innovation Capacity in Cities.
City Health Dashboard
The City Health Dashboard is an initiative of NYU Langone Health and NYU Wagner that provides information on health and health-related factors affecting quality of life in cities across the United States. It includes, for example, indicators on access to preventive services, walkability and air pollution. The number of cities represented in this source ranges from 103 to 155, depending on the indicator.
Eurostat
Eurostat’s City statistics database contains objective well-being indicators on housing, health, labour markets, education, environment and transport in European cities. This source includes up to 52 cities participating in the 2018 OECD/Bloomberg Survey on Innovation Capacity in Cities or the What Works Cities Assessment.
European Quality of Life Survey
The European Quality of Life Survey (EQLS) provides information on people’s perceptions of quality of life and city administration. Indicators include life satisfaction, city satisfaction and satisfaction with public transport. Only 26 cities that took part in the 2018 OECD/Bloomberg Survey on Innovation Capacity in Cities were represented in the latest edition of the EQLS.
Gallup U.S. Daily
Gallup US Daily tracks people’s perceptions on economic and social issues at the local level. Individual-level data from Gallup US was aggregated at the city level using ZIP codes (more than 30 000 available) to create indicators on life satisfaction, city satisfaction, self-reported health and perceived material conditions for 160 US cities.
The Reflective Democracy Campaign
The Reflective Democracy Campaign publishes data on the demographics of candidates and elected officials in the United States. Their 2016-17 Demographics of Power Report gives information on gender and non-white representation gaps in the largest 200 US cities, of which 103 belong to either the What Works Cities Assessment or the 2018 OECD/Bloomberg Survey on Innovation Capacity in Cities.
Ballotpedia and Who Votes for Mayor
Information from Ballotpedia, a non-profit providing a digital encyclopaedia of American politics and elections, and from Who Votes for Mayor, a project by Portland State University studying voting at the local level, was used to create an indicator on voter turnout for mayoral elections in 71 US cities.
While all 30 indicators contribute to understanding various aspects of well-being, 13 headline indicators with the largest coverage were selected to provide an overview across well-being dimensions. In order to provide a first overview of how well-being outcomes differ across cities with different capacities for public sector innovation and data use, this report builds indexes by well-being dimension (from 0 to 100, see Annex 1.C). The report then explores the links between individual well-being indicators and different components of public sector innovation and the foundational areas of data use.
The final database covers a wide and diverse sample in terms of population size of 200 cities across the United States and Europe (see full sample of cities in Annex Table 1.B.1). While the average population of the 161 US cities is around 400 000, the average population of the 39 EU cities is above 1 million. Both the US and EU samples have around 30% of cities with between 200 000 and 500 000 inhabitants. Nevertheless, while more than half of the US cities have below 200 000 inhabitants, more than half of the EU cities in the analysis have more than 500 000 people (Table 1.2).
Assessing public sector innovation capacity
The assessment of public sector innovation capacity in cities relies on the joint 2018 OECD/Bloomberg Survey on Innovation Capacity in Cities. The survey investigated the main drivers and characteristics of public sector innovation capacity in cities across five components: Innovation strategy, Innovation staff, Funding for innovation, Data for innovation, and Innovation outcomes evaluation (Box 1.1). A selection of survey questions for each of the five was used to create a public sector innovation (PSI) score with total value from 0 to 10 (Table 1.3). Thus, the PSI score captures essential aspects of public sector innovation capacity in cities. Although 147 cities participated in the survey, only 112 (of which, 74 in the United States and 38 in Europe) responded to the questions for the PSI score and had sufficient data on relevant outcome indicators.
Assessing data use in cities
The measurement in this report of data use in city management and policy making relies on the What Works Cities Certification, a standard of excellence for well-managed, data-driven local government. The WWC Standard identifies 45 criteria of data use distributed across eight foundational areas: Data Governance, Evaluations, General Management, Open Data, Performance and Analytics, Repurposing, Results-Driven Contracting, and Stakeholder Engagement (see Box 1.2). The criteria are used to create a score from 0 to 45, where each point represents a data use practice implemented by the city administration and validated by a team of experts (see full list of practices in Annex Table 1.B.1). Overall, the score reflects cities’ commitment to using data for administration, policy design and evaluation. In total, the analysis covers 145 cities (141 in the United States, 4 in Europe) that participated in the WWC programme and for which there is sufficient information on well-being outcomes.
Methodological considerations and caveats
The analysis in this report (Chapters 2 and 3) brings novel and internationally comparable evidence on well-being outcomes for cities with various degrees of public sector innovation capacity and data use. However, for reasons explained below, the evidence should be neither interpreted as causal nor generalised to cities outside the sample – particularly to cities outside the United States and Europe.
First, the current assessment is limited to cities that took part in either the 2018 OECD/Bloomberg Survey on Innovation Capacity in Cities or in the What Works Cities Assessment. Further, due to the challenge of finding internationally comparable data on well-being outcomes at the city level, the sample is restricted to cities in the United States and Europe. As such, the sample used for quantitative analysis represents only a fraction of the universe of cities that might be pursuing innovation and data use activities worldwide.
Second, the self-selection of cities into the 2018 OECD/Bloomberg Survey on Innovation Capacity in Cities and the What Works Cities Assessment makes it difficult to draw causal conclusions about the effects of city innovation and data use on resident well-being. In an ideal setting, the cities would be drawn from a random sample representative of all types of cities in the countries of interest. In this case, the analysis is limited to cities that volunteered to participate in one or both of the programmes and engaged in the data collection process. Relative to non-participating cities, cities that responded to the 2018 OECD/Bloomberg Survey on Innovation Capacity may be more advanced or inclined to showcase high levels of commitment to innovation. Similarly, cities that undertook the What Works Cities Assessment are likely advanced or highly committed to data use and may be incentivised by the prospect of dedicated training and expertise.
As with any empirical exercise, the effect of public sector innovation and data use on well-being can be difficult to assess as there are often many factors and policies that simultaneously affect a particular outcome. Although the present analysis corrects for the effect of city size (taken as the population of the city) and economic development (measured as the percentage difference between the average city and national household income), there might be other factors that remain unobserved.
Lastly, the analysis does not capture virtuous effects of public sector innovation and data use that may manifest in the very long term or via indirect channels. In particular, public sector innovation and data use scores only cover one or two years, which limits the potential to exploit the time dimension at this stage.
References
[4] Arundel, A., C. Bloch and B. Ferguson (2019), “Advancing innovation in the public sector: Aligning innovation measurement with policy goals”, Research Policy, Vol. 48/3, pp. 789-798, http://dx.doi.org/10.1016/j.respol.2018.12.001.
[3] Bloomberg Philanthropies (2020), “What Works Cities”, website, https://whatworkscities.bloomberg.org/ (accessed on 18 January 2021).
[14] Bloomberg Philanthropies (2020), “WWC Certification (database)”, Unpublished.
[12] Demircioglu, M. (2017), “Reinventing the Wheel? Public Sector Innovation in the Age of Governance”, Public Administration Review, Vol. 77/5, pp. 800-805, http://dx.doi.org/10.1111/puar.12821.
[5] Goldsmith, S. and N. Kleiman (2017), A New City O/S: The Power of Open, Collaborative, and Distributed Governance, Brookings Institution Press, http://www.jstor.org/stable/10.7864/j.ctt1vjqnwd.
[8] Janssen, M. et al. (2017), Driving public sector innovation using big and open linked data (BOLD), Springer New York LLC, http://dx.doi.org/10.1007/s10796-017-9746-2.
[13] OECD (2020), Innovation for Development Impact: Lessons from the OECD Development Assistance Committee, The Development Dimension, OECD Publishing, Paris, https://dx.doi.org/10.1787/a9be77b3-en.
[20] OECD (2020), The Future of Regional Development and Public Investment in Wales, United Kingdom, OECD Multi-level Governance Studies, OECD Publishing, Paris, https://dx.doi.org/10.1787/e6f5201d-en.
[1] OECD (2019), Enhancing Innovation Capacity in City Government, OECD Publishing, Paris, https://dx.doi.org/10.1787/f10c96e5-en.
[17] OECD (2019), How’s Life in the Province of Córdoba, Argentina?, OECD Publishing, Paris, https://doi.org/10.1787/97f189b1-en.
[21] OECD (2019), How’s Life in the Province of Córdoba, Argentina?, OECD Publishing, Paris, https://dx.doi.org/10.1787/97f189b1-en.
[15] OECD (2019), The Path to Becoming a Data-Driven Public Sector, OECD Digital Government Studies, OECD Publishing, Paris, https://dx.doi.org/10.1787/059814a7-en.
[19] OECD (2016), Well-being in Danish Cities, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264265240-en.
[6] OECD (2015), Data-Driven Innovation: Big Data for Growth and Well-Being, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264229358-en.
[18] OECD (2015), Measuring Well-being in Mexican States, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264246072-en.
[10] OECD (2015), The Innovation Imperative: Contributing to Productivity, Growth and Well-Being, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264239814-en.
[16] OECD (2014), How’s Life in Your Region?: Measuring Regional and Local Well-being for Policy Making, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264217416-en.
[11] OECD/Eurostat (2018), Oslo Manual 2018: Guidelines for Collecting, Reporting and Using Data on Innovation, 4th Edition, The Measurement of Scientific, Technological and Innovation Activities, OECD Publishing, Paris/Eurostat, Luxembourg, https://dx.doi.org/10.1787/9789264304604-en.
[9] OECD/Eurostat (2005), Oslo Manual: Guidelines for Collecting and Interpreting Innovation Data, 3rd Edition, The Measurement of Scientific and Technological Activities, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264013100-en.
[2] What Works Cities (2018), “What Works Cities Assessment Guide: Certification Criteria”, What Works Cities, Medium.com, https://medium.com/what-works-cities-certification/what-works-cities-certification-assessment-guide-5c514f1dff1b (accessed on 14 January 2021).
[7] What Works Cities (2015), What Works Cities Brief: The City Hall Data Gap, What Works Cities/Bloomberg Philanthropies, https://www.bbhub.io/dotorg/sites/8/2016/03/WWC_Brief_c.pdf (accessed on 8 December 2020).
Well-being indicators use different units depending on the aspect they measure. For example, household income is typically expressed in USD PPP (US dollars using purchasing power parity), whereas life expectancy and representation of women in local government are expressed in years and as a percentage, respectively. To compare and aggregate well-being indicators using the same scale, the OECD well-being framework normalises them using the min-max method (OECD, 2019[21]). This statistical formula transforms the value of the indicator into a score from 0 to 100, where 100 is the highest score possible for a normalised indicator.
To transform the value of an indicator into a well-being score (0-100) three steps must be taken:
1. Identify the city minimum and the city maximum values for each well-being indicator.
2. Normalise the indicators by applying the min-max formula (see below).
3. Calculate the mean of the normalised indicators within the same well-being dimension.
Formula is used for indicators with a positive sense (e.g. employment, life satisfaction) and formula for indicators with a negative sense (e.g. unemployment, air pollution).
Finally, based on the third step, when a well-being dimension is measured by more than one indicator (e.g. “Jobs”, which comprises employment and unemployment rates, or “Health”, which combines life expectancy and perceived health), the score of the well-being dimension is defined by the simple mean of the normalised indicators in the same dimension.