copy the linklink copied!2. New ways of delivering care for better outcomes

Martin Wenzl

Health care delivery supported by information and communications technology (ICT) has great potential to make health systems more effective in improving health, more equitable and more efficient. ICT and data can be harnessed to redesign health services according to needs and to deliver services in an integrated and people-centred way. The increasing number of patients with complex needs in OECD countries stand to gain the most from new models of care delivery. ICT can help identify such patients, inform them about their own health and care, improve communication and coordination between them and their providers, increase the accuracy of diagnoses and clinical decision-making, and help monitor their health remotely and deliver appropriate services across geographical distances. However, without an overarching architecture that ensures that new tools are interoperable and can be integrated with existing information systems, ICT may entrench and even exacerbate fragmentation and inequity. Many OECD countries still appear to be far from realising this potential for transforming care delivery.

    

copy the linklink copied!2.1. Introduction

The prevailing models of health care delivery are inflexible, fragmented and specialty-based. In the context of changing disease patterns, financial pressure and emerging technologies, such models cannot be relied on to serve population health needs sustainably. Future services must be tailored to patient needs and delivered in an integrated manner, targeting the right patients at the right time, while delivering a positive experience for patients as well as providers.

This chapter addresses the central issue of how to use information and communication technology (ICT) to improve health care delivery for achieving better care quality and health outcomes.1 Focusing on people with complex health needs, it outlines the opportunities and risks as well as potential costs of increased use of ICT for improving care delivery.

ICT improves the capacity for using data to generate, manage and share valuable information much more effectively and at a lower cost. If used appropriately, ICT can help health systems achieve their overarching goals by adopting better and data-driven ways of delivering care that provide the right and high-quality health services to the right people at the right time.

However, there are no guarantees that greater adoption of ICT will automatically meet policy objectives. Implementing ICT across entire health systems in a way that improves effectiveness, reduces health disparities and achieves both sustainably and efficiently is a major policy challenge. Nor is greater penetration of ICT itself sufficient to improve the performance of health systems. Ultimately, digital technologies should not merely be used to digitise analogue processes and services, but as an opportunity to fundamentally rethink and reorganise processes, workflows and services in a way that addresses peoples’ health needs and improves outcomes sustainably (OECD, 2019[1]). This requires a concerted effort led by policy makers, health system managers and health professionals and that is supported by all stakeholders, including patients.

For the purpose of this Chapter, care delivery refers to the complete set of modalities in which health care is delivered to patients, including the physical setting, the health care providers involved, the mode of interaction between patients and professionals as well as among the various professionals and provider institutions involved in care, attendant remuneration mechanisms for provider institutions and professionals, and any supporting tools used, in particular ICT.

Care delivery can be described on varying scales. Distinct models of care delivery can be found in small-scale and local pilot initiatives or can, once matured and broadly implemented, be the prevailing way of providing care in a health system. An entire level of care (such as primary care) can be organised according to a given model of care delivery, as can be care for a specific disease (such as disease management for diabetes or other chronic conditions) or care for a specific patient group (such as case management for elderly patients with complex needs).

For example, in many health systems solo-practice by general practitioners (GPs) has traditionally been a prevailing model of delivering primary care. In this model, patients would make appointments when they felt ill and would see their GP on an episodic basis to receive treatments and individual referrals to other providers of care. GPs would mainly work independently from other professionals, be paid on a fee-for-service basis and provide services in a reactive manner.

More recently, the primary care has started relying on digital technology, replacing or complementing face-to-face consultations with automated triage and tele-medicine, allowing for remote contacts between patients and physicians. In some health systems, primary care is increasingly provided by multi-disciplinary group practices. These are newer models of primary care delivery.

The increasing prevalence of chronic diseases has made disease and case management more common as distinct models of care delivery for patients with a single or multiple chronic diseases. In such models, the provision of care involves not only episodic consultations but also proactive identification of patients, for example through screening, broad assessments of their health status and their care needs, and their continued and proactive monitoring, often by a dedicated case manager, to respond to changes in need. Care delivery is often supported by ICT systems to exchange information among professionals and with patients and fee-for-service payments are often replaced with capitated or bundled payments.

This chapter comprises three main sections. Section 2.2 shows that people with complex health needs are the greatest potential beneficiaries of harnessing ICT in care delivery and shows how data can help design and target needs-based health services. Section 2.3 discusses how ICT, and the use of data generated by such technology, can improve the process of health care delivery to complex patients. Section 2.4 examines why successful new ways of delivering care are not often scaled and sustained, and how this can be addressed. Content of this Chapter is based on the published literature but also relies heavily on examples and case studies of care delivery in health systems of OECD countries.

copy the linklink copied!2.2. Using data to design better health services and target them more accurately

Patients with complex needs, such as those with multiple chronic diseases, stand to gain most from harnessing ICT and in the delivery of care. Secondary use of data is the key to remodelling services around patient needs. But integrating care for complex patients can also be resource intensive and costly. Integrated care therefore needs to be personalised and targeted accurately at those people who can benefit most.

2.2.1. Data present an opportunity to sustainably improve care for the growing number of patients with complex health needs

Chronic diseases are now the main causes of mortality and morbidity in OECD countries. Increasing shares of populations are affected by multi-morbidity, the presence of several concomitant chronic illnesses (physical and mental) in the same person. Overall estimates of the prevalence of multi-morbidity across OECD countries are not available. However, country-specific surveys and epidemiological studies suggest that prevalence is high and increasing. In Ontario, for example, the prevalence of multi-morbidity increased by 40% between 2003 and 2009 (Koné Pefoyo et al., 2015[2]). Estimates suggest that multi-morbidity may now affect approximately one in three adults in Ontario, one in four adults in Australia and one in five adults in Denmark (Mondor et al., 2018[3]; Schiøtz et al., 2017[4]; AIHW, 2018[5]). In a sample of ten European countries,2 the prevalence of multi-morbidity among people aged 50 and above has been estimated to have increased from 38% in 2007 to 42% in 2015 (Palladino et al., 2019[6]).

Current ways of delivering care are ill-suited to emerging health needs

Health care is often provided inflexibly in a fragmented and specialty-based way. This is particularly ill-suited for serving the increasing number of people with multi-morbidity and complex needs in OECD countries.

People with multi-morbidity manage a high volume of information, interactions with various providers, and self-care tasks; they need to coordinate, synthesise, and reconcile health information from multiple sources and about different diseases; and their position at the intersection of multiple health issues requires self-advocacy and expertise. Multi-morbidity also often occurs among the elderly, the disabled or people in lower socio-economic classes, who might find it particularly difficult to navigate current health systems and accomplish all of these tasks related to managing their health. In Denmark, for example, multi-morbidity affects every other person older than 65 (Schiøtz et al., 2017[4]). Among people with lower secondary education only, the prevalence is double that among people with postgraduate education (ibid.). The co-occurrence of multi-morbidity with difficult socio-economic circumstances make the care needs of such population groups particularly complex.

ICT and data present an opportunity to make care more effective and efficient

People with complex needs account for a disproportionate share of total health care utilisation and expenditure in OECD countries. In United States, for example, health care costs of people with three or more chronic conditions are almost twice as high as in the average adult population and costs of people with three or more chronic conditions and disability are more than four times higher (Hayes et al., 2016[7]).

A complexity-based stratification of the population of the Spanish region of Catalonia (see Chapter 8 on system governance, stewardship and resource allocation for further details) found that, compared to people below the median on the complexity scale, people in the top percentile of the complexity score were 27 times more likely to have an emergency hospital admission (31.9% vs. 1.2%), had 15 times the number of primary care consultations (31 consultations on average per person and year vs. 2.1) and 61 times the pharmaceutical expenditure (EUR 1 394 on average per person and year vs. EUR 23) (Monterde, Vela and Clèries, 2016[8]). A recent cost-of illness study in New Zealand found that co-morbidity resulted in greater health care expenditure than the expected sum of the present conditions in isolation, with 24% of all health care expenditure attributed to this super-additive feature of complexity (Blakely et al., 2019[9]).

The enhanced use of data and deployment of the knowledge that can be generated from them present a great opportunity for improving the care for people with complex health needs. Through its ability to generate and analyse vast amounts of data, ICT can help improve care for these people in multiple ways, as described below. Effective uses of ICT and data for improving care for people with complex needs are necessary to promote financial sustainability of health systems through deploying services where they generate the most benefit, which drives system efficiency. They also increase efficiency by preventing more costly future service use, such as emergency room visits and hospital admissions.

However, needs-based care delivery does not necessarily imply cost savings; it can also result in increased secondary and tertiary care utilisation and higher aggregate costs, in particular in the short and medium term. This was the case, for example, in the Personalised Integrated Care Programme in the United Kingdom, a pilot project of personalised support to older people who are at risk of a future emergency admission. While the scheme delivered positive outcomes as reported by patients and staff, it resulted in a rise in hospital activity (and costs) for enrolled participants in the 16 months following implementation. The most likely reason for this increase is that participation led to previously unidentified health needs being addressed (Nuffield Trust, 2019[10]). It is possible, however, that this increased hospital activity prevented more serious morbidity in the future and, in some cases, perhaps premature death (but longer-term effects were not examined). It may therefore translate to lower future expenditure, efficiency gains and value for money. This serves as a reminder that expenditure should ultimately be viewed in the context of long-term health outcomes and across budget silos.

2.2.2. Personalising care with better information

Secondary use of large datasets can also be a key source of information for redesigning services and tailoring distinct interventions to individual patient needs. Personalised care concerns tailoring care pathways to individual needs and preferences and can also only be achieved through effective use of routine data.3 Integrated care for patients with complex needs can benefit particularly from data analytics for personalisation.

For example, electronic data from a range of sources can enable modelling of complex care pathways and developing treatment guidelines that take into account interactions between co-morbidities, complicating factors and distinct treatments. Considering diseases in isolation is a widely recognised shortcoming of existing paradigms in medical research and the resulting management of chronic diseases (Tillmann et al., 2015[11]). Many randomised controlled trials (RCTs), which are the current mainstay of evidence generation in medicine, explicitly exclude complex patients to increase the likelihood of isolating the effect of the intervention under investigation despite the fact that people in the general population are more likely to be affected by multi-morbidity than by any single chronic disease. Deriving medical knowledge only from prospective studies with limited sample sizes, including RCTs, necessarily misses an opportunity to generate knowledge from the data on the majority of the population that is treated in routine outside of prospective studies (also see Chapter 7 on biomedical technologies).

Ushering in ‘System Medicine’ with modern data analytics

The term system medicine has been proposed to describe a new paradigm in which the development and selection of treatment strategies for patients with complex diseases is based on data-driven analysis of the human body as a complex system of interacting biological process that determine an individual’s level of health (and disease) (Gietzelt et al., 2016[12]; Tillmann et al., 2015[11]).

ICT allows for combining data from various sources and for analysing large amounts of data to model real-life disease trajectories, disease interactions and effects of medical interventions. Only once patient complexity is better understood can this knowledge be applied in the routine delivery of care for complex patients. For example, clinical decision aids can only provide appropriate and personalised treatment guidelines for patients with multi-morbidity once the interactions of diseases present in the same patient and interactions of corresponding treatments have been studied and are thoroughly understood.

Even if medical research may still be a long way from a true system medicine approach, early examples of how large volumes of electronic data can be used to generate knowledge of disease complexity are already available. The Spanish GMA system (see Chapter 8 on system governance, stewardship and resource allocation) has recently also been used in Catalonia for a number of epidemiologic studies that assessed, for example, the co-morbidity burden, complexity and resource use of patient populations with specific index diseases.4 Such studies help identify patient subgroups that require specific interventions and inform the improvement of their care.

With increased computational power, new techniques can be used to analyse large routine datasets. Data mining, for example, allows medical research to not only take a hypothesis-driven but also a data-driven approach Phinney et al. (2017[13]), for example, show that data mining techniques can identify patient characteristics associated with a high risk of health deterioration simply by recognition of patterns in the data. Results from such analyses can support the delivery of health care but also encourage additional hypothesis-driven research: while algorithmic data mining can, in this example, identify who is at risk of health deterioration, only more traditional hypothesis-driven research can go further and answer the question of why a set of patient characteristics are associated with health deterioration (ibid.). Artificial intelligence can be deployed for making computers more accurate in predicting outcomes, such as hospital readmission, the occurrence of complications or death by feeding them data, and allow for the corresponding adaptation of interventions and care delivery (Topol, 2019[14]; JASON, 2018[15]).

Making the necessary data available to unlock their knowledge potential

Using data analytics for greater personalisation and needs-based redesign of services requires that the necessary data are accessible and available for such purposes. Because of the properties of electronic data, which allow for their duplication and sharing across geographic distances at very low marginal costs, databases to support research can be created relatively cheaply through extraction and linkage of data from routine sources (Kannan et al., 2017[16]). While data extraction, and especially cleaning and curating them for analysis, can be costly, such secondary use is likely much cheaper than original data collection for each research purpose. Registries, for example, can be developed virtually by pooling data from other, existing sources such as health records, prescriptions hospital admissions data. New Zealand has created such a ‘virtual’ national registry of diabetic patients as a useful resource for policy makers, providers and patients (see Chapter 8 on system governance, stewardship and resource allocation).

The combination of data generated in routine health care with other datasets, in particular genomic data, allows for unlocking new knowledge that can help personalise treatments and make them more effective. Agarwala et al. (2018[17]), for example, show how gaps in knowledge underlying the selection of treatments for cancer can be filled by analysis of a combination of longitudinal EHR data from cancer centres with genomic datasets. Using the treatment response and health outcomes achieved in large samples of patients together with information on genetic characteristics of tumour mutations significantly increases the likelihood that the most appropriate treatment combination is selected for any given patient from the plethora of options available (ibid.).

In conclusion, Agarwala et al. (2018[17]) also highlight, however, that accessing dispersed datasets and linking them in accordance with data privacy requirements is very challenging. More integrated information infrastructure and continuous data sharing among providers and laboratories would enable the unlocking of information contained in such data (ibid.). This requires data governance and policy frameworks that manage privacy risk while permitting secondary uses of personal health data for public benefit.

Integration of genomic and other –omics data into care pathway design is an emerging frontier. Geisinger, a private health insurer and integrated provider network in Pennsylvania and New Jersey in the United States, has used data-based patient pathways in the past to reduce unexplained variation in clinical practice and improve the quality of care. Since 2014, Geisinger integrates genomic information into the EHRs of patients who consented to having their DNA sequenced (see Box 2.1). This information is used to personalise care. For patients that are insured with Geisinger Health Plan, clinical exome sequencing is included in benefit packages at no additional cost and any care recommended as a result of a pathogenic variant found in analysis of the sequence is considered medically necessary, and therefore covered per the terms of the individual’s specific benefit package (Williams et al., 2018[18]).

copy the linklink copied!
Box 2.1. Integrating genomic information into clinical care at Geisinger in the United States

Geisinger is a private health insurer and integrated provider network in Pennsylvania and New Jersey, which serves a population of more than 4 million people with about 1.5 million patient visits annually. About one-third of patients are also insured with Geisinger.

To improve care and health outcomes through earlier diagnoses and personalisation of care, Geisinger launched the MyCode biorepository of genomic patient data in 2007. Data was initially collected for research purposes and later to be included in EHRs and used in the delivery of health care. Since 2014 MyCode conducts whole exome sequencing and genotyping on collected samples to capture the part of the genome containing the most clinically relevant information. Geisinger patients are enrolled in MyCode irrespective of health status. So far, about 200 000 patients consented to enrolment, representing about 90% of patients who have been offered enrolment.

Patient DNA is sequenced in a laboratory using blood samples. Results are compared with a reference DNA to identify high-confidence, likely or known pathogenic variants. Predictor snippets re-sequenced. Results are reported to clinicians, patient and family and placed in the EHR. For positive results, EHRs are reviewed to check if an illness has already been identified. Professionals are informed first to familiarise themselves with the results and the implications for care, after which patients are notified and given the opportunity to discuss implications of the result for their health care. Information on first-degree relatives is also communicated in a family letter. Results are also deposited in publicly accessible databases.

A variety of follow-up options with the team of health care professionals is available to patients, depending on whether or not they are insured with Geisinger. The genetic information can lead to a number of follow-up actions in clinical care, such as conducting additional diagnostic tests, recommending additional interventions or closer monitoring of medication adherence or life style. The effects of using genetic information in care are monitored according to an outcomes framework and using EHR data. The framework contains metrics related to process, health status, costs, behavioural factors and patient-reported measures.

Approximately 3-4% of sequences identify clinically actionable information, about 50% of which is new information, in particular about family members. Evidence of the effects of integrating genetic information in care, whether in terms of process, health outcomes, or cost, is not yet publicly available.

A challenge in integrating genomic information into clinical practice is to choose the variants that should trigger follow-up action. Therefore, only variants with high certainty about predicting disease are reported (currently 76 variants) while information on low-certainty predictors is retained for subsequent analysis. To move towards a learning health system, clinical data are fed back into the sequence to improve variant annotation and the understanding of the effect of genetic variants on the risk of disease. The initiative is also quite resource intensive, requiring a good data infrastructure and analytical capacity to compare sequences to reference DNAs as well as a sufficient number of genetic counsellors to interpret and communicate implications. The interpretation of results is generally more difficult and fuzzy than for diagnostic tests, not least because patients are enrolled regardless of disease status so that the probability of a given patient’s having a condition associated with a predictor is low and the risk of false positives is high.

Source: Geisinger, personal communications; Williams et al (2018[18]), “Patient-Centered Precision Health In A Learning Health Care System: Geisinger’s Genomic Medicine Experience”, http://dx.doi.org/10.1377/hlthaff.2017.1557.

copy the linklink copied!2.3. Enhancing care delivery with digital technology

Providing high quality care to patients with complex needs requires that service delivery across different settings is seamless. Everyone involved in providing care – patients, doctors, pharmacists, nurses, dieticians, other allied health professionals, social care providers and so on need to communicate effectively with each other, have relevant and timely information and coordinate their activities.

Shared information systems that enhance communication and information flow across the continuum of care have been recognised for some time as one of the key prerequisites for integrating activities of distinct health care providers (Suter et al., 2009[19]; Gray Steele et al., 2016[20]). The needs of complex patients can best be supported by systems that allow for person-centred and bi-directional information sharing between patients and providers as well as among individual providers, for example through EHRs, tele-monitoring systems or web-based applications (Gray Steele et al., 2016[20]). If ICT-supported care delivery, and improved coordination, can help attract younger and healthier patients to automated and less costly services, capacity could be freed up to focus for more resource-intensive services on more complex patients, leading to efficiency gains.

Despite the various ways that ICT can improve care delivery, the evidence on the effectiveness of new ICT-enabled ways of delivering care to improve health outcomes of complex patients is still weak and not yet conclusive. Similarly, the evidence on the effects of novel ICT that supports patient-centred and integrated health service delivery on patient outcomes is sparse (Demiris and Kneale, 2015[21]). Recent analyses of integrated care projects that use ICT for people with multi-morbidity in Europe, for example, found little evidence that such models of care delivery are effective (Melchiorre et al., 2018[22]; Barbabella et al., 2017[23]).

Interventions for people with complex needs as well as services that rely on ICT are often customised locally and may have multiple and interacting components so that their success or failure depends as much on their implementation in local work flows as on their design. ICT is an enabler of better delivery of care and should be seen as an essential part of an intervention, not an intervention in its own right (Gray Steele et al., 2016[20]; Melchiorre et al., 2018[22]). It is therefore difficult to generate, synthesise, interpret and generalise evidence of the ultimate effectiveness of technology in isolation.

Nonetheless, there are many and often relatively small-scale initiatives across OECD countries that demonstrate how ICT can be used effectively to improve care delivery, and some of these show promising initial results. A recent study from Australia (Shaw, Hines and Kielly-Carroll, 2018[24]) concluded that many of the ICT tools for health described below, including patient portals, mobile technologies that deliver information such as patient reminders, electronic discharge summaries and clinical decision aids, can improve patient outcomes. The authors also cautioned, on the other hand, that such tools can also have negative effects on practice, user experience and outcomes if not designed or implemented appropriately (ibid.).

This section aims to answer the question of how ICT can be used to enhance service delivery, in particular for patients with complex health needs. While ICT supports the formulation of guidelines and personalisation of care pathways as well as needs-based health service planning and resource allocation (see Chapter 8 on system governance, stewardship and resource allocation), digital technology also allows for the set of services to be delivered to patients more effectively and efficiently, while making them more responsive to needs as patients move through the health system.

A number of avenues show promise in employing ICT for improving outcomes and making care for complex patients more efficient. This section identifies four such avenues: giving patients access to their own data and facilitating patient-provider communication; enhancing communication and coordination among providers; using data to improve decision making in clinical practice; and tele-medicine. This section relies mainly on case studies from OECD countries to show different ways in which ICT can be used effectively in care delivery. Case studies are instructive through illustrating innovative ways in which ICT can be used to tackle challenges identified locally. Rather than prescribing off-the-shelf solutions, this can help decision makers learn about experience elsewhere and apply the lessons to their local context.

2.3.1. Giving patients access to their own data and facilitating patient-provider communication

Easily accessible and understandable information on health and health care can empower patients, improve their health behaviours and self-care and enhance support by informal caregivers. At the same time, tools that provide for two-way data exchange can also enhance communication between patients and providers. Better provider-patient communication can improve the responsiveness of health services and, ultimately, improve outcomes and increase patient satisfaction. While having access to data can increase self-management capacity, self-management cannot replace professional care. Rather, it can enable patients not only to improve their own health behaviour but also to reach the health care they need (Morton et al., 2017[25]).

ICT, such as web-based portals and mobile apps that are integrated with information systems of health care providers, can make personalised information available to patients at low cost and encourage information exchange between patients and providers. Tools that make personal health information accessible to patients, by tapping into existing information systems such as but not only electronic medical and health records (EMR and EHRs), are often referred to as patient portals. They can increase patient awareness and help them make decisions, giving them more confidence in their care and reducing anxiety, fear and uncertainty (Roberts et al., 2017[26]; Morton et al., 2017[25]). Studies also found that patients appreciated the ability of technology to share information with their families (Roberts et al., 2017[26]).

Patients with complex needs benefit from empowerment through information

For patients with complex needs, patient portals are best integrated with a range of tools that help them manage their health and facilitate patient-provider interactions. Because of the co-occurrence of several health problems and the breadth of services they receive, such patients can benefit particularly from more accessible information about their health and health care. A recent literature review from Australia found that successful patient portals are integrated with provider information systems, such as EHRs, and with clinical decision support tools, and provide functions for secure messaging, patient reminders and prescription refill orders (Shaw, Hines and Kielly-Carroll, 2018[24]).

While enhanced information in itself can support self-management through increasing awareness, patient portals are especially effective for complex patients when integrated with self-management applications. They can also be integrated with remote monitoring tools that feed information systems with patient data while patients are not in contact with their health care providers (see Section 2.3.4). Importantly, all these ICT solutions are more effective when part of broader strategies that make care more patient-centred, such as case management (Goldzweig et al., 2013[27]).

Patients who use self-management applications (‘apps’) have been found to perceive greater awareness of their condition, to be better able to make health-related decisions and feel more equal to professionals allowing them to engage in meaningful discussions (Morton et al., 2017[25]). Apps that allow patients to learn interactively, especially through self-assessment and feedback features, increase patient participation in their care (Roberts et al., 2017[26]).

Self-monitoring of data can motivate patients to engage in behaviours that help improve their health outcomes, even when using applications that do not support behaviour change explicitly (Morton et al., 2017[25]). For example, perceiving an interaction between certain activities and physiological data, such as reducing blood pressure by adhering to medication, to better manage diabetes through physical activity and diet, or to control COPD by engaging in more physical exercise, not only encourages further self-monitoring but also motivates to engage in self-management in order to see an improvement in the data (ibid.) This motivation to change behaviour based on physiological data was found even among patients using standalone monitoring systems with no explicit support for behaviour change or educational functions (ibid.).

The Swedish ePATH (electronic Patient Activation in Treatment at Home) project applied a user-centred design process to incorporate a number of ways of enhancing the self-care capacity of patients with chronic or complex diseases (Schildmeijer et al., 2018[28]). In addition to informing patients through functions for planning self-care activities, medication management, health and symptom tracking, and two-way communication with health care providers, the application used various psychological tools to motivate patients to engage in self-care. Through recording self-care activities, health care providers could get a better understanding of symptom development and medication adherence (ibid.).

Technologies that help engage patients in their care are underused

The benefits of giving people access to their own data are many. But recent studies suggest that systems that share and actively provide health data to patients to support self-management are still under-used, in particular for complex patients. A recent analysis of integrated care projects that use ICT for people with multi-morbidity in Europe, for example, found that ICT that supports patient self-management is among the tools least used in these projects (Melchiorre et al., 2018[22]). Barbabella et al. (2017[23]) found that nine in ten hospitals in Europe (90%) do not permit patients to access their own health data. Similarly, in programmes to improve care for multi-morbid patients, tools for sharing of information mainly focus on interactions between professionals and provider organisations, not on making information available to patients (Melchiorre et al., 2018[22]).

For the potential of personalised health information to be realised, patients also have to actively access and used the information that is made available. However, people have sometimes been slow in the uptake of tools such as patient portals (see, for example, NHS Digital (2019[29]) and Adler-Milstein and Longhurst (2019[30])). This underlines the need not only for user-centred design of such solutions but also for supporting people in accessing and making use of the information that is made available.

Although their own data are not yet actively provided to patients in many new models of care delivery, many OECD countries are currently investing in patient portals that make enable people to access their health information and increasingly integrate these systems with the wider health information architecture. Of 15 countries who responded to the survey conducted during research for this report, 12 (80%) reported that they already have or are in the process of implementing an ICT system that gives people access to their own health data.

In Australia, for example, the My Health Record system provides a secure online summary of key personal health information and is available to all residents. Per November 2018, approximately 25% of the Australian population (more than 6 million people) were registered in the system and more than 14 000 health care provider organisation were registered to contribute data, including primary care practices, hospitals, pharmacies, diagnostic imaging labs and pathology practices. In 2019, the Australian government moved towards an opt-out principle to improve uptake of the My Health Record system, so that all residents will have a record by default unless they choose not to have one.

In Canada, the provincial government of Nova Scotia offers its residents a patient portal called MyHealthNS. The portal allows patients and doctors to share information, including routine test results. Once patients have created their secure online health record, they can receive and store test results and specialist reports electronically. They can also log health information, such as blood pressure readings, immunisations, allergies and medications.

All Estonian citizens have access to their electronic health record through a national patient portal using their personal identification number and the relevant security measures tied to it. The portal not only provides access to data but also has a number of basic interactive functions (see Box 2.2).

copy the linklink copied!
Box 2.2. The Estonian patient portal

In Estonia, all citizens who are insured by the Estonian Health Insurance Fund have access to their health data through a web-based patient portal. The portal provides access to the national health database, which unifies data from various health care providers in electronic health records (EHR). People can view their medical data, including data entered by health care providers on diagnoses, test results and their interpretations, and treatments received as well as data on medicines prescribed and dispensed.

In addition to providing access to data stored in their EHR, the portal allows people to create summary documents (such as case summaries and dental care charts), set up reminders, book appointments, inform all medical institutions simultaneously about changes in their contact details, make declarations of intent (such as registering for organ donation) and initiate administrative processes. For example, instead of seeing a health care provider for such purposes, they can apply for health certificates through virtual medical checks that use existing medical data in their EHR and make such documents available for administrative purposes, for instance for getting a driving license.

By default, all citizens can access their own data and health care providers can access data of their patients. Parents also have access to data of their underage children. However, users are their own access administrators and can restrict data access selectively or opt out of the system entirely at any time. Adult users can authorise other persons to access their data and appoint representatives for the performance of certain activities (for instance for buying prescription medicines) so that, for example, people can support the care of their parents or grandparents. A function to give consent for use of data for research purposes is currently under development.

For data security, the system relies on digital authentication for access, digital signing of all data, encryption and decentralised data storage, and logging of all activity backed by blockchain technology. People access the portal using their digital identity card tied to a citizen ID, which is identical for all public services, including health care. Every data query results in an unalterable log so that any potential abuse remains fully traceable. Data access logs are monitored centrally and by users themselves, who can check by whom and when data were viewed. In the past, health care providers who accessed data without appropriate authorisation already faced severe disciplinary measures, including loss of their license to practice.

As per 2018, the portal has been actively used by approximately 480 000 people, representing 37% of the Estonian population. Just under 700 people have opted out of the system, which represents less than 1% of users.

Source: Based on Estonian Ministry of Health and Estonian Health Insurance Fund, personal communications; https://www.sm.ee/en/patients-portal-and-health-information-system.

In Finland, the city of Oulu has opened the Self-Care platform to all of its citizens since 2010. Self-care is a web-based communication platform for patients and professionals that makes available information to encourage healthier life styles and disease prevention and provides support for managing chronic diseases (Lupiañez-Villanueva, Sachinopoulou and Theben, 2015[31]). It is integrated with people’s EHR and provides a wide range of functions, including online booking of appointments and sharing of test results; e-prescriptions; an information portal on treatment of chronic illnesses and health promotion as well as nutrition diaries and weight control tools; an advice service through which people can log inquiries that are answered by health professionals; data governance functions for citizens to authorise data transfers between providers; and a tool for providers to monitor the health status of their patients (ibid.).

While Self-Care is available to all people regardless of their health status, it has been recognised as a key ICT enabler of the chronic care model also implemented by the city, and supports the shared use of data, not only among health care but also health and social care providers. As per 2017, there were approximately 60 000 registered users among a total population of about 200 000 in the city (Oulu Healthcare and Social Welfare, 2018[32]). The goal is to scale the system to the entire region with a population of about 400 000.

Widening health disparities must be actively avoided

Many people with multi-morbidity are likely to adopt and use technologies that allow access to their health information (Yamin et al., 2011[33]). But evidence also suggests that there are disparities in the use of patient portals between patients with different socio-economic backgrounds (Shaw, Hines and Kielly-Carroll, 2018[24]; Goldzweig et al., 2013[27]). These reflect the digital divide and lower digital literacy among disadvantaged population groups (ibid.).

It is therefore important that implementations of patient portals and other ICT that facilitates sharing of data with patients not only make the electronic tools available but also support adoption by people who can benefit most from their use. In Estonia, for example, training courses and tutorials on digital tools are made available to patients and professionals with lower digital literacy. Human centricity and patient empowerment is also among the five pillars of the Estonian e-health strategy (2015-2020), which aims to develop the abilities of people to self-manage and self-educate using apps and online solutions.

Another way of encouraging adoption by patient populations that can benefit from enhanced access to their data is making tools available in provider settings and having professionals demonstrate and support their use (Shaw, Hines and Kielly-Carroll, 2018[24]). In the United States, for example, a mobile device-based patient portal is currently under evaluation that engages patients with multiple chronic conditions during a hospitalisation when one of their diseases deteriorates. This approach is taken because a hospital episode is expected to make the health problem more salient to patients and increase engagement (McAlearney et al., 2016[34]). Using the hospital episode as an entry point, the application then aims to increase patient self-management following discharge through various functions. It provides health summaries, medication listings, daily care plans, health education videos and other materials, advice on prevention, secure messaging with providers and appointment tracking and a patient interface for health data entry. The solution is integrated with EHRs maintained by providers.5

2.3.2. Communication and coordination among providers is key to improving care and health outcomes

Coordination of activities between the wide range of different providers involved in care of complex patients is key to improving outcomes and avoiding harmful treatment interaction and waste. By definition, ICT can play an enabling role in improving communication and coordination across all settings and professions involved in the delivery of care, including transitions between hospitals and home- and community-based care and transitions between health and social care.

In a survey in Scotland, for example, GPs reported that they believed that sharing of data through an EHR system enhances patient safety, improves clinical management, reduces hospital admissions, empowers clinicians, aids communication across services and enables decisions to be responsive to patients’ wishes (Craig et al., 2015[35]). Doctors also believed that patients with multiple and complex health problems benefit particularly from information sharing (ibid.).

A recent review of care delivery models that use ICT and aim to improve care for elderly people with multi-morbidity found that tools that improve communication and coordination among providers, in particular shared EHRs, are one of the most common ICT components of such models (Melchiorre et al., 2018[22]). Managers of these care models reported that ICT-supported care coordination led to improvements in the quality of care, quality of life of patients and the efficiency of care (ibid.). In another example, adoption of EHRs in hospitals in the United States was found to be associated with reductions in mortality (Lin, Jha and Adler-Milstein, 2018[36]).

Integrated health record systems are an important foundation

Interoperability and shared data standards or integrated information systems play an important foundational role in enabling this communication and coordination among service providers. Many countries are making good progress in implementing a single, integrated EHR system. In Lithuania, for example, the central e-health system (ESPBI IS) stores patient information from various providers in a single and shared repository, following the principle of ‘One Resident – One EHR’. The system also provides electronic workflows for appointments, referrals and e-prescriptions that save time and reduce errors in transmitting information, making provider interactions more efficient. At the same time, patients can securely access their data online, through a patient portal (a similar patient portal that provides access to EHR data in Estonia is described in Section 2.3.1). Nearly 95% of the Lithuanian population have an EHR and, by mid-2018, more than 70% of providers were connected to the central e-health system.

In the NHS Scotland projects are underway to make electronic records interoperable between the health and social care system, which have historically relied on separate record systems (Gray Steele et al., 2016[20]). This is particularly important for patients with complex needs, who often require health and social care. Time will tell if initiatives such as the ones described here result in better care outcomes and efficiency. Evaluation of initiatives such as these, while challenging, is very important (see Section 2.4.3).

Of course, enabling a range of providers’ access to personal health data introduces risks. Authorisation of access and any alterations to the information must be tracked. This can be enabled by ancillary digital technologies such as a blockchain, which does not hold any health or clinical data but can provide a record of authorisation and access to the data. Estonia, for example, tracks all changes to information on EHRs – including when, where and by whom the entry was made – and keeps a record of all amendments in separate places including on a blockchain. This provides an immutable log should an unauthorised access and manipulation of data occur.

Other ICT functionalities can also contribute

While a shared and interoperable EHR system is a linchpin of care coordination, a wide range of data-driven modalities are available to share information effectively and ultimately improve the people-centeredness and integration of care. A recent review by the RAND Corporation identified five key ICT functionalities that are widely used already or being piloted for care coordination: dashboards, patient relationship management, event alerts, referral tracking and care plans (Rudin et al., 2017[37]). These functions are most effective when integrated with each other and with existing information systems.

Shared electronic care plans, for example, can provide personalised care pathways defined by patient need and outline optimal treatments to both providers and patients. In addition to including shared patient background information, they can include care team member designations that help professionals understand their responsibilities, and task management functionalities that improve treatment adherence (Rotenstein et al., 2016[38]). Similarly, electronic hospital discharge summaries (EDS) can be simple and effective tools to improve coordination of care between hospitals and community-based providers. EDS can be populated and sent automatically from hospital EMR systems and be integrated with reminders for health professionals responsible for post-discharge care.

ICT that allows for remote delivery of services (see Section 2.3.4) can also support interactions among professionals, saving time and making care more efficient. Various ICT-based solutions have been implemented in OECD countries to improve information exchange between professionals to bring medical expertise closer to patients rather than moving patients physically to where expertise is located, especially by linking local providers with specialists based far away.

In Estonia, for example, an e-consultation service has been implemented that allows GPs to consult with specialists on difficult cases online. GPs then either implement specialist advice themselves or refer patients to further services that are necessary. Uptake of the service is incentivised by the Estonian Health Insurance Fund, which pays specialists the same rate for e-consultations as for face-to-face patient contacts. In England, some NHS Clinical Commissioning Groups have implemented virtual services for GPs to send questions to specialists for a quick reply, eliminating the need for specialist appointments. Such services enhance the role of GPs in providing care and frees up specialist time, through avoiding unnecessary referrals. Similar remote consultation services are available to primary care professionals in Canada.

In Poland, a “telestroke” system is being established to increase the speed and therefore effectiveness of treating stroke. The telestroke system uses ICT for remote consultations between specialised stroke centres and local providers where specialists are not available. Project ECHO is an initiative based in the United States developing ICT-based services to support community-based health professionals with remote specialist advice covering a wide range of medical specialties.6 It also allows specialists to learn from cases located far away.

Challenges to deploy ICT relate to engagement and workflow redesign

However, significant barriers remain to greater use of ICT for care coordination. These include, for example, limited engagement of professionals in development and implementation of tools, and attendant challenges with their integration into existing workflows, slow adoption and sub-optimal use, and a lack of standard definitions of the purposes and functions of tools as well as the roles of users (Rudin et al., 2016[39]). Greater user involvement in designing tools is one way to reduce barriers to their adoption (see Section 2.4).

As such, ICT tools that encourage better coordination and integration of care for complex patients are not yet ubiquitous across health systems of OECD countries. Of the 15 countries that responded to the survey conducted in the research for this report, 9 (60%) mentioned initiatives to allow for and/or promote electronic exchange of data between providers. Four countries (27%) reported that patients with multi-morbidity or other types of complexity have been identified as a specific target group for new ways of care delivery and only 2 countries (13%) reported that they are implementing integrated care for such patients. The latter does not necessarily mean that such care delivery models do not exist – but they may not have gone beyond local pilot initiatives yet and may not be a system-level priority.

However, in many OECD countries there are examples of integrated care delivery models that are supported by ICT and the use of data. Many Spanish regions actively enrol complex patients into integrated care. An example of such a model from the Basque Country is in Box 2.3.

In Australia, the Health Care Home (HCH), currently being trialled, aims to provide coordinated and team-based care for patients with chronic and complex conditions, supported by ICT. All patients enrolled in the HCH have an electronic care plan, defined and overseen by a nominated clinician who takes overall responsibility for the care of an individual patient. This electronic plan is shared with patients and all professionals involved in their care. The shared care plan aims not only to increase coordination of the services but also patients’ own participation in their care, both inside and outside of the HCH. Providers are also expected to share patient data and use such data to monitor and track patient health indicators and outcomes (Health Policy Analysis, 2017[40]). A 2-year evaluation of the effects of HCH in terms of quality of care and patient experience, provider experience, health service use and costs is due to be completed by the end of 2019 (ibid.).

copy the linklink copied!
Box 2.3. Integrated care in the Basque Country (Spain)

The Department of Health of the Basque Government has implemented integrated care for frail elderly adults and patients with multi-morbidity. This is part of an overall ‘Chronicity Strategy’ adopted in 2010, which includes risk stratification of the entire Basque population, and is supported by a broad e-health Strategy. Based on the stratification, the Basque health authority provides population-level prevention, disease management, or integrated case management for the most complex patients with multi-morbidity. Integrated care aims to improve continuity of care, adherence to therapy and, ultimately, patient experience and health outcomes. By October 2017, more than 4 000 patients were enrolled in integrated care. The target for 2019 is to enrol 16 000 patients.

At the core of the care delivery process are “Integrated Care Organisations” (ICO) that oversee primary and hospital care for a defined population catchment area and provide preventive interventions and personalised care. Care relies on three provider pillars: hospital-based professionals overseen by reference internists; primary care teams; and a 24/7 nurse-led call centre. New roles have been defined for nurses who act as liaison officers and case managers. The model aims to improve the management of polypharmacy, patient empowerment and self-management capacity and coordinate health and social care.

An e-health Strategy and various ICT tools support care delivery. These include a patient portal, a shared electronic health record (EHR), an electronic prescription system and tele-monitoring. A custom version of the Adjusted Clinical Groups Predictive Model (ACG-PM) is used for risk stratification and case finding, unifying various data sources (e.g. including demographics, ambulatory and hospital diagnoses, prescriptions and service utilisation). Risk stratification is not only used to identify the right patients to enrol into integrated care but also to support the formulation of needs-based care plans. The same data are also used to feed business intelligence (BI) software that generates scorecards for managers to monitor care delivery.

Monitoring is based on a range of process- and outcome-related indicators, defined across nine domains including effectiveness, efficiency and equity. Indicators include, for example, rates of hospital admissions, readmissions and mortality (to gauge effectiveness); costs of primary care consultations, emergency room visits and hospitalisations (to gauge efficiency); and breakdown of the patient population enrolled by sex and income (to gauge equity).

Results of rigorous studies that evaluate the effectiveness of the integrated care model are not yet available. Evaluation of the pilot project (2015-16) found improved care coordination, lower numbers of hospital admissions and visits to the emergency room, higher numbers of GP consultations and increased patient, family and caregiver satisfaction. Before-and-after comparisons found a 12% reduction in hospital days for multi-morbid patients and decrease in readmission rates of nearly 17%. The model has been deemed cost-effective overall, mainly by improving outcomes while remaining cost-neutral.

Source: Based on Basque Regional Health Authority, personal communications; Scirocco Project (2017[41]), “Overview of Scirocco Good Practices: Basque Country - Care Plan for Elderly”, https://www.scirocco-project.eu/wp-content/uploads/2017/04/SciroccoGP-Basque-6-Care-Plan-for-Elderly.pdf.

Implementing ICT must be part of a broader change and improvement strategy

To truly achieve patient-centred and integrated delivery of health care, however, adoption of ICT that can enhance communication and coordination of care needs to be part of a much broader effort to establish teamwork and collaboration among professionals as a the standard way of operating. This requires not only the right policy framework that encourages cooperation and greater care integration, through institutional structures and incentives, but perhaps nothing short of a fundamental cultural change in the way health professionals are educated and work (Mulvale, Embrett and Razavi, 2016[42]). The workforce considerations of implementing ICT are explored in more detail in Chapter 4.

Examples of projects in OECD countries that aim to improve communication and coordination between providers show that ICT, data and better information can be a key enabler of better collaboration between providers. But instituting the necessary behavioural changes and ensuring that ICT and knowledge are used effectively remains the biggest challenge.

To make care delivery more person-centred and improve the management of chronic disease and multi-morbidity, the Veneto region of Italy introduced Integrated Medical Groups (IMGs) in 2016 as a new model of delivering primary care (Ghiotto et al., 2018[43]). These groups comprise at least 4 general practitioners as well as nurses and other health professionals are embedded in local health units, which promote the integration of health and social care and between hospitals and other medical services, share electronic medical records and provide care in accordance with pre-defined diagnostic-therapeutic pathways (ibid.). These pathways define the respective roles and responsibilities of professionals and how they cooperate among each other; they also aim to promote patient engagement. The extent to which professionals enter and share structured information is monitored.

Germany has historically relied on professional autonomy and office-based physicians in solo-practice as a predominant way of providing outpatient care. Statutory health insurance provides patients with free choice of GPs and specialists and access to care without cost at the point of service. These characteristics pose a challenge in care for chronic diseases. The Joint Federal Committee (G-BA) currently funds the Accountable Care (ACD) project, which uses routine data from sickness funds to identify patients with chronic diseases who see multiple office-based physicians and then aims to improve care coordination among them through moderated working groups (see Box 2.4).

copy the linklink copied!
Box 2.4. Accountable Care (ACD) in Germany

The Accountable Care (ACD) project was launched in 2017 with the goal of improving cooperation and regular feedback among office-based physicians to improve the quality of health care by reducing avoidable hospitalisations and improving patient outcomes. Improving job satisfaction of physicians was another goal. It targets patients with one or more of 14 diseases with a high proportion of avoidable hospitalisations. Most diseases are chronic, including hypertension, diabetes, COPD or chronic back pain.

The absence of formal gatekeeping by general practitioners, no shared electronic health record, free provider choice by patients and the attendant lack of coordination among providers, in particular office-based physicians who often work in solo-practice, have been identified as challenges in care for chronic diseases. Estimates suggest that 60% to 90% of hospitalisations for chronic diseases could be avoidable. The German Joint Federal Committee (G-BA) funds this 3-year project using routine data to improve cooperation among office-based physicians.

Pseudonymous routine data from four German regions are analysed to identify patients who are seen by several office-based physicians (referred to as “shared patients”) and corresponding de-facto physician networks. Physician networks receive information on their existing networks, including typical patient pathways, and are asked to improve cooperation by defining communication channels, action plans and care pathways. Providing physicians with this information on how they are connected with their colleagues and on the outcomes of care delivered within their informal networks can help them make improvements and strengthen their awareness of possible discontinuities in care. Trained moderators lead “quality circle” meetings every six months to provide structured dialogue. Patient outcomes and medical guideline adherence is monitored and fed back to physician networks quarterly. Patient-reported indicators are aggregated and reported at the network-level. In addition, all participating physician practices receive analyses of routine data pertaining to all other patients they treat.

Following an application to the relevant regulatory authorities in accordance with the German Code of Social Law, data from the associations of statutory health insurance physicians and data from sickness funds were linked within a Trust Centre at LMU Munich. These data encompass all patient contacts with the ambulatory care sector and hospitals stays that are billed to the sickness funds, including information on diagnoses, procedures and prescribed medication. Linking the routine data from the sickness funds with the data from physician associations allows for visualising actual patient care pathways.

However, some information on cross-sectoral services and services purchased through selective contracting are not available in the dataset. The linking of routine data with more comprehensive and meaningful data on clinical parameters could further improve the quality of feedback to physicians. Another barrier to more effective and efficient sharing of data is a lag of 10 month in data availability.

The care delivery model is currently evaluated in a cluster-randomised controlled trial (cRCT), with some physician networks engaging in the quality circles and performance monitoring and some assigned to a control group. Evaluation is due to be completed in 2020. The 3-year project led by Ludwig Maximilian University (LMU) Munich received EUR 3.8 million in funding from the G-BA Innovation Fund.

Source: Based on German Joint Federal Committee (G-BA) (2019[44]), “ACD – Accountable Care in Deutschland - Verbesserung der Patientenversorgung durch Vernetzung von Leistungserbringern und informierten Dialog”, https://innovationsfonds.g-ba.de/projekte/versorgungsforschung/acd-accountable-care-in-deutschland-verbesserung-der-patientenversorgung-durch-vernetzung-von-leistungserbringern-und-informierten-dialog.45.

2.3.3. Data-driven decision aids in clinical practice enhance diagnosis accuracy and appropriateness of treatment

Computers far exceed the abilities of the human brain to process large amounts of data. Clinical decision-support systems can match the characteristics of individual patients to large volumes of data and use algorithms to create personalised predictions of disease status, diagnoses, appropriate treatment and help make other clinical decisions (Shaw, Hines and Kielly-Carroll, 2018[24]). Decision aids can thus improve the accuracy of diagnosis and treatment decisions made by professionals.

Decision aids can be used in variety of health care settings. For example, algorithms have now been used successfully for some time in interpreting diagnostic images and have been shown to outperform humans in certain tasks related to diagnoses and prognoses (Litjens et al., 2017[45]; Dimitriou et al., 2018[46]; Topol, 2019[14]). Algorithms can also be faster than humans in interpreting diagnostic images, which can have positive effects on treatment outcomes if delays in making decisions puts patients at risk (Arbabshirani et al., 2018[47]; Topol, 2019[14]). In the emergency room, computerised clinical decision support systems have been found to improve care in terms of process-related measures (Bennett and Hardike (2016[48]), also see Chapter 4 on the health workforce).

Only 2 of 15 countries that responded to the survey conducted in research for this report reported projects to implement clinical decision support systems. There are, however, some examples of innovative care delivery models that feature digital decision support systems. The Finnish POTKU model, for example, provides GPs with the locally-developed Evidence-Based Medicine electronic Decision Support (EBMeDS) system. The system matches evidence-based treatment guidelines and recommendations with patient records and provides personalised care guidance (Hujala et al., 2016[49]). The system also generates automated reminders and warnings (ibid.).

Decision support can be especially useful in complex patients

Patients with complex needs can benefit particularly from decision support algorithms because they often need treatments for several diseases at the same time, which creates complex combinations and risks of adverse interactions. Polypharmacy is often the consequence of having several chronic diseases so that polypharmacy is highly prevalent among older population groups and people with complex health needs. For example, in Sweden a population-based study found that, in 2010, an average adult aged 65 years or above was exposed to 4.6 medicines at the same time (Morin et al., 2018[50]). In this elderly population group, 44% of people were exposed to five or more drugs at the same time, and 12% to ten drugs or more (ibid.)

As described in Section 2.2, treatment guidelines can be contradictory for patients with multi-morbidity because medical research continues to focus on single diseases. Algorithms that match all diagnoses and characteristics of a patient to recommended treatments can help identify possible adverse treatment interactions if clinical guidelines conflict and provide mitigation strategies. To manage complex drug interactions in patients with polypharmacy, algorithms can use electronic data on prescriptions to generate automated warnings of high-risk drug combinations or drug-induced complications to prescribing physicians, pharmacists and patients (Molokhia and Majeed, 2017[51]). Sharing of electronic prescription data can avoid prescribing errors that happen simply because prescribers are not aware of all the drugs taken by their patients (Lavan, Gallagher and O’Mahony, 2016[52]).

Electronic drug monitoring tools can also make health care more efficient by freeing up time spent by professionals on tasks that can be automated, without compromising outcomes. A study of the work of nurses in an Australian hospitals, for example, found that nurses spent significantly less time on medication monitoring tasks and more on other patient care-related tasks following the implementation of an electronic monitoring tool for rheumatology patients (Callen et al., 2013[53]).

Ensuring patient involvement in decision making

Patient involvement in decision-making is an important part of patient-centred care. Algorithms can also be designed to take into account patient preferences (Wilk et al., 2017[54]; Zamborlini et al., 2016[55]). However, patient involvement is still the exception rather than the rule. A literature review published in 2015 found that patient participation in making decisions is still limited and that few tools are developed to actively and directly involve patients in the decision process (Sacchi et al., 2015[56]). Similarly, a review that focused on decision aids in the care for patients with multi-morbidity also found that patients were often not actively involved in the decision-making processes (Fraccaro et al., 2015[57]).

Integration and interoperability are of vital importance

Interoperability of information systems and data quality are key to realising the full potential of data-based decision aids. Algorithms can only produce accurate and personalised care recommendations if underlying data are accurate and contain the necessary variables to provide complete personal profiles of patients. This requires combining personal data from disparate sources. However, many decision aids currently used do not yet take a system medicine approach to integrate data from various sources. For example, Gietzelt et al. (2016[12]) found that many decision models only use a single type or source of data and few combine more than two types or sources of data. Decision models that do combine at least two types of data most commonly use genomics and molecular data combined with clinical data extracted from electronic medical records (ibid.).

Data from EHRs are also used for predictive modelling to improve the appropriateness of care as patients receive treatment. For example, EHRs have been used at Kaiser Permanente in the United States to predict deterioration of hospital inpatients and unplanned transfer to intensive care units (Kipnis et al., 2016[58]). In another study using EHR data from Canadian and United States university hospitals, deep learning algorithms achieved high accuracy in predicting disease onset, hospital mortality, unplanned readmissions, prolonged length of stay and the final discharge diagnoses of patients (Rajkomar et al., 2018[59]; Miotto et al., 2016[60]).

Decision support systems are best integrated with other ICT tools that support the delivery of care and embedded into clinical workflows. In the United States, emergency room software that unifies all relevant patient information and integrates it with checklists and decision-support was found to reduce mortality and length of stay, resulting in cost savings (Olchanski et al., 2017[61]). A recent review from Australia found that decision-support systems are most successful when implemented in combination with additional software components and that their adoption, and ultimate effect on the quality of care, can be improved by ensuring interoperability with existing ICT systems and focus on a local minimum set of indicators (Shaw, Hines and Kielly-Carroll, 2018[24]). Integration with other ICT systems is also key to understand the entire range of information various systems deliver to health professionals and patients, to help ensure that people are not overloaded with information and risk ignoring the most important alerts that decision support systems deliver.

More research is needed to establish the effectiveness of decision aids

Although the utility of decision aids may be more straightforward to evaluate than other ICT tools (through independent validation of the appropriateness of recommendations the systems generate) evidence of decision aids’ effectiveness remains patchy. A review by Bennett and Hardiker (2016[48]) concluded that there was mainly low-quality evidence of the effectiveness of decision aids used in emergency care. Looking at decision aids in the care for patients with multi-morbidity, a review by Fraccaro et al. (2015[57]) found that there were no rigorous evaluations of usability or effectiveness of the tools used.

2.3.4. Tele-medicine can make care more appropriate and efficient

One of the most common applications of ICT in the health sector is tele-medicine, which can be defined as the use of ICT to deliver health care at a distance (Cravo Oliveira Hashiguchi, forthcoming[62]).7 The scope of tele-medicine is broad. It includes tele-monitoring, whereby health care professionals can monitor vital data of patients as well as disease symptoms, signs and signals remotely through the use of ICT, and interactive tele-medicine, whereby ICT is used to bridge geographical distance between patients and providers to for patient/provider interaction and for remote delivery of medical services, such as tele-consultations (ibid.).

In Israel, for example, the national digital health program launched by the government in 2015 includes various initiatives related to remote patient monitoring and remote service delivery. The Ministry of Health has established a tele-medicine platform that can be used by all health care providers in the country for the provision of remote services to patients and to enhance information exchange between providers (See Box 2.5). The four statutory health insurance funds that operate their own provider networks and also function as health maintenance organisations (HMOs) provide tele-consultation services with general practitioners, paediatricians and dermatologists. One HMO offers an interdisciplinary remote consultation service to provide online support and treatment for patients with chronic diseases.

copy the linklink copied!
Box 2.5. The national tele-medicine platform in Israel

As part of the national digital health plan launched in 2015, the Israeli Ministry of Health funded and developed a national tele-medicine platform, which provides tele-monitoring functions, allows for remote service delivery to patients as well as improved communication among providers.

The platform aims to make services more accessible for patients with limited mobility and those living in peripheral areas that are poorly served by existing provider infrastructure and to reduce the reliance on private providers by broadening availability of public services. It also allows for urgent after-hour consultations. Providers that use the tele-medicine platform have full access to patients’ medical files and can share information with other providers electronically. Four distinct services based on the platform are currently being piloted.

Chronic Disease Management provides in-home monitoring devices to patients with chronic disease. The monitoring system is connected to a central medical call centre that can dispatch appropriate services based on the data received. The service aims to improve adherence to treatment plans, reducing unplanned hospitalisations and other avoidable service use.

Health Data Management automatically manages and analyses patient-generated health data, such as vital signs and medical history. This solution provides proactive alerts to providers when a patient’s condition deteriorates.

Tele-Consultation provides patients the possibility to consult with specialists remotely. The service aims to bridge geographic distance between patients and the specialist services required by their condition through teleconferencing technology. The solution is designed for sessions between a patient and a single physician and between a patient and several physicians. It aims to decrease waiting times, which can have positive effects on health outcomes through earlier disease detection and subsequent interventions, and decrease costs through avoiding face-to-face consultations.

Tele-Rehabilitation provides post-acute rehabilitation services remotely, allowing patients who cannot access such services physically to benefit from rehabilitation support. The service aims to improve quality of care for patients.

Source: Based on OECD survey and personal communications with the Israeli Ministry of Health.

Tele-medicine has a range of advantages and can make care more appropriate and efficient. However, it needs to be deployed carefully for these potentials to be realised. Similarly to other uses of ICT, tele-medicine is a mere tool for facilitating interactions among providers and patients; it is not a medical intervention in its own right. Just like face-to-face consultations, services delivered via tele-medicine can be appropriate for patient needs or inappropriate. If used inappropriately, tele-medicine can exacerbate inequity through favouring access to services by younger and healthier people, create frivolous demand and overburden health care providers by unrealistic expectations of continuous monitoring. Electronic transmission of patient data can also represent risks to privacy.

This section discusses important aspects of tele-monitoring and remote delivery of services. A more comprehensive review of current uses of tele-medicine in OECD countries, evidence on its effect on health system performance and lessons for its appropriate use are available in Cravo Oliveira Hashiguchi (forthcoming[62]).

Tele-monitoring can make care more responsive and appropriate, leading to increased effectiveness and efficiency

Tele-monitoring can increase the awareness of patients’ conditions by health professionals and enable earlier and more accurate identification of clinically relevant symptoms, signs and signals through electronic transmission of patient data – in most cases passively (without the patient having to manually input data). This can make health care more responsive and appropriate, improve therapy and medication adherence, avoiding costly interventions later on. Providing professionals with real-time data on their patients may help make face-to-face interactions more timely and focused, and increase patient adherence to treatments, ultimately making treatments more effective (Morton et al., 2017[25]; Noah et al., 2018[63]). By focusing the time of professionals on the most important tasks, tele-monitoring can also increase productivity (Noah et al., 2018[63]).

All of this can particularly benefit patients with chronic diseases, who need treatments over prolonged periods of time and specific acute-care interventions when their conditions deteriorate, and people who live in remote and underserved areas. At the same time, the abundance of data generated by tele-monitoring tools can pose privacy risks, may cause information overload and may lead to unrealistic expectations of patients vis-à-vis health professionals.

A trial in psychiatric care in the United Kingdom, for example, found that providing regular feedback to therapists on patient outcomes allowed therapists to focus their attention on patients who were not on track and to identify and resolve obstacles to clinical improvement, which ultimately alleviated depression and anxiety (Delgadillo et al., 2018[64]). Outcome feedback in this context refers to routine monitoring of the patient’s condition and comparing the patient’s symptoms with those observed in similar cases (Delgadillo et al., 2018[64]; Glazebrook and Davies, 2018[65]).

While many individual examples of the use of tele-monitoring exist in OECD countries, few countries use them on a large scale to improve care. Of the 15 countries that responded to the survey conducted in research for this report, only 4 countries (Canada, Israel, Norway and Poland) mentioned projects to implement patient tele-monitoring systems. Analysis of the WHO Third Global Survey on eHealth (conducted in 2015) found that only Canada, Japan and Spain already had well established and relatively large-scale tele-monitoring systems (Cravo Oliveira Hashiguchi, forthcoming[62]). In Canada, for example, tele-monitoring has been promoted as a tool for patients with complex chronic diseases to improve self-management of their illnesses. Canada Health Infoway, the national funding entity that promotes health-related ICT, has made tele-monitoring a priority and funded a number of projects across provinces, including the Ontario Telehomecare project (see Box 2.6). In many other countries, tele-monitoring is often only used in small-scale pilot projects (Cravo Oliveira Hashiguchi, forthcoming[62]).

copy the linklink copied!
Box 2.6. Telehomecare in Ontario, Canada

The Ontario Telehomecare project provides coordinated support from primary care to people with complex chronic diseases in their own homes. Goals are to maintain people’s independence in their own community, providing them access to appropriate care when needed and decrease the need for emergency department visits and acute hospital admissions, thereby saving costs.

Since 2013, over 9 000 patients have been enrolled in the program. The initial focus was on people with congestive heart failure (CHF) and/or chronic obstructive pulmonary disease (COPD). The program was subsequently expanded to people with diabetes as comorbidity, patients living in supported living environments, patients transitioning from hospital to home and patients requiring remote monitoring, in a shared post-acute care model.

Nurses remotely monitor the health status of patients and provide assistance and coaching for self-care. Patients are provided with a touchscreen device to transmit data as well as a blood pressure cuff, pulse oximeter and weight scale. They also receive training in use of the devices. Patients submit data daily during weekdays to nurses, who review results and contact patients if changes in health status need further investigation. Nurses also get in touch with patients weekly by phone to help develop skills and confidence needed to manage symptoms, medications and lifestyle changes. Physicians can receive regular progress reports about their patients enrolled in telehomecare.

Evaluations of the project found that patients with CHF and/or COPD reported increased confidence in self-managing symptoms, leading to reduction in hospital emergency department visits and hospital admissions. Patients enrolled in the program were also found to have reduced levels of blood pressure while evidence of effects in terms of other health outcomes is not available. However, these studies did not compare the people enrolled in against a control group.

A qualitative study, based on semi-structured interviews, document review, and observation of 39 patients and their informal caregivers and 23 professionals involved in telehomecare, identified a number of facilitators and barriers of implementation. Facilitators included user-friendly technology; patient motivation to participate and increase self-care capacity; the integration of the telehomecare into broader health service provision; and comprehensive program evaluation. The main barriers included issues related to using the technology, such as poor memory as to when to take readings or physical difficulties in using technology for people with functional limitations; time constraints for professionals limitations, gaps in provision of care needed by patients; and barriers to patient participation related to geography and social location.

Source: Based on Canada Health Infoway (2016[66]), “Ontario Telemedicine Network Telehomecare Deployment Project: Phase 2-Remote Patient Monitoring”, https://www.infoway-inforoute.ca/en/plan/3176-ontario-telemedicine-network-telehomecare-deployment-project-phase-2-remote-patient-monitoring/download; Hunting et al. (2015[67]), “A multi-level qualitative analysis of Telehomecare in Ontario: challenges and opportunities”, http://dx.doi.org/10.1186/s12913-015-1196-2; Ontario College of Family Physicians (2016[68]), “Telehomecare supports chronic disease management”, https://ocfp.on.ca/communications/telehomecare-supports-chronic-disease-management; Sahakyan et al. (2018[69]), “Changes in blood pressure among patients in the Ontario Telehomecare programme: An observational longitudinal cohort study”, http://dx.doi.org/10.1177/1357633X17706286; Stanimirovic et al. (2018[70]), “Impact of telehomecare on health system utilization in patients with heart failure”, http://dx.doi.org/10.1016/j.cjca.2018.07.143.

For complex patients, tele-monitoring can be integrated with ICT tools that support patient-self management. Giving patients access to their own data has a number of advantages in itself and the data captured by tele-monitoring tools can also drive personalised supports for self-management, such as reminders, goal-setting or personalised life style advice (see Section 2.3.1). Tele-monitoring tools are powerful tools to improve medication adherence, by enhancing, for example, patient education and patient awareness of their own medication-taking patterns (Vrijens, Urquhart and White, 2014[71]). A randomised pilot study of a tele-medication monitoring system in the United States, for example, showed that such a system can reduce the number of hospitalisations and length of inpatient stays of patients with chronic heart failure (Hale et al., 2016[72]).

Potential risks of tele-monitoring must be recognised and managed

There are a number of pitfalls to avoid when implementing patient tele-monitoring tools. Most importantly, people need to be comfortable with sharing their data with professionals through ICT tools. A poll in the United Kingdom found that 57% of respondents were willing to share data with the National Health Service via a lifestyle app or fitness tracker (Castle-Clarke, 2018[73]). Adoption of tele-monitoring tools may be more difficult to achieve among elderly patients, who are more likely to have complex health needs. Results from the poll in the United Kingdom indicated a clear generational gradient of respondents: while approximately 70% of 15-24 year olds were “very or fairly willing” to share such data and only 25% in that age group were “very or fairly unwilling” to do so, just under half of over 65 year-olds were willing to share data with the other half unwilling to do so (ibid.).

To build trust and acceptance of tele-monitoring, ensuring data privacy is paramount. While data privacy and security are broader concerns related to all technologies that transmit personal health data electronically, not all technologies currently available are secure. Especially applications that can be downloaded by patients themselves and are not subject to regulatory oversight may bear risks. A review by Dehling et al. (2015[74]) of more than 24 000 mobile health apps available for Apple iOS and Android, for example, found that more than 90% of apps available posed at least some risk of damage through information security and privacy infringements while some 12% of apps were classified in the highest risk category.

Professionals can be burdened by unrealistic expectations of continuous monitoring (Morton et al., 2017[25]). While the feeling of being monitored, in particular when patients are contacted by professionals when the monitored parameters are out of range, can reduce anxiety by patients, it can also induce feelings of over-reliance on professionals. Where the level of patient autonomy permits, one solution to these problems is to make patients responsible for contacting professionals when their data were not within an expected range, which can improve both, patient empowerment and the quality of care by making interactions more effective. The style of feedback has an important influence on how much responsibility the patient adopts for self-management (ibid.).

There may also be emotional barriers to adoption of remote- and self-monitoring tools. A 2015 study from the United States, for example, investigated the perceptions of electronic health monitoring tools by multi-morbid patients (Ancker et al., 2015[75]). It found that patients sometimes perceived monitoring data as an additional burden, that making data more salient to patients can provoke strong emotional reactions and that patients often notice that physicians have more trust in data that is measured by technology than in self-reported information (ibid.).

Evidence is encouraging but difficult to generalise

As with other ICT solutions that have the potential to improve care, rigorous evidence of the effectiveness of tele-monitoring in terms of health care process measures and health outcomes is only just emerging. Because tele-monitoring can be used in many different ways, studies are often context-specific and their findings cannot be generalised easily. Also, studies of effectiveness in terms of health outcomes often only look at patients with specific diseases, and not necessarily the most complex patients with multiple health problems.

A recent OECD working paper found that tele-monitoring improves patient satisfaction, empowerment and reassurance by providing a greater sense of security while away from health care providers. Tele-monitoring has also been found to reduce emergency room visits and unplanned and avoidable admissions to hospitals by following patients more closely in their own homes while it appears to either have no effect or increase the use of face-to-face primary care (Cravo Oliveira Hashiguchi, forthcoming[62]). Increased use of face-to-face care, whether appropriate or not, is often a result of greater patient awareness of medical needs (ibid.). Where this leads to more appropriate care, tele-monitoring can also improve health outcomes. It has been shown, for instance, to reduce mortality of patients with heart failure (HF). A recent literature review by Noah et al (2018[63]) found that remote monitoring tools had positive early effects in terms of clinical outcomes in the management of some chronic diseases, including COPD, Parkinson’s, hypertension and lower back pain. On the other hand, integrated self-management tools were not always effective. The review focused on non-invasive, wearable devices that automatically transmit data to a web portal or mobile application for the purposes of self-monitoring or monitoring by a health professional.

Remote delivery of services can improve access and make care more efficient

Delivery of services across geographic distances using ICT can save time for health care providers and patients and improve access to services in remote areas and for isolated sub-populations. Remote service delivery can thus make health care more efficient and more equitable. However, equitable access to enabling technologies and special support for people with lower digital and health literary are prerequisites for achieving the goal of equity. Policy also needs to ensure that the ease of accessing remote services does not lead to additional demand by population groups with better digital literacy, who also tend to be younger and healthier, at the expense of those in need.

Similar to tele-monitoring, the growing evidence base on remote service delivery needs to be interpreted carefully as ICT can facilitate the delivery of appropriate and inappropriate services alike. Evidence suggests that services delivered remotely result in health outcomes that are comparable to outcomes of care delivered face-to-face, while there are a number of non-clinical benefits to patients, in particular ease of access but also increased timeliness, coordination and continuity of care and promotion of knowledge sharing and continuous learning among professionals and patients (Cravo Oliveira Hashiguchi, forthcoming[62]).

Tele-medicine often complements, rather than replaces, face-to-face care

While remote delivery of services cannot always replace face-to-face consultations, remote interactions with professionals can nevertheless serve as an efficient entry-level contact with the health system, and improve the patient-centeredness, appropriateness and ultimately the effectiveness of subsequent face-to-face services (Pecina and North, 2016[76]). For people with multiple chronic diseases who require care over prolonged periods of time, remote delivery of care can greatly enhance access to appropriate services, in particular for people with limited ability and those living in areas that are remote or have poor provider infrastructure. Remote service delivery is best integrated with remote monitoring solutions described above.

In Australia, for example, Head to Health is a digital mental health gateway that aims to improve access to mental health services most suited to peoples’ needs through a stepped-care approach supported by ICT. Services can be accessed through a single webpage,8 which either makes electronic services available directly, allows people to access remote telephone and online crisis counselling and to schedule face-to-face consultations with professionals. Digital services generally focus on highly prevalent conditions, such as anxiety and depression, and are delivered via desktop computers and mobile apps.

For people with multiple chronic diseases who require care over prolonged periods of time, remote delivery of care can greatly enhance access to appropriate services, in particular for people with limited mobility and those who live in areas that are geographically remote or have poor provider infrastructure. This requires that remote monitoring solutions described above are integrated with remote service delivery and that information generated by remote monitoring leads to appropriate patient/provider interactions. The review by Melchiorre et al. (2018[22]), however, found that electronic tools for remote monitoring and patient/provider interactions are not yet widely used in care delivery for patients with multi-morbidity. Impacts on costs and efficiency need to be monitored and evaluated carefully.

The effects of remote service delivery on total costs and efficiency are not easy to predict. Where remote consultations replace and avoid unnecessary, face-to-face contacts and help avoid unnecessary face-to-face consultations, they can lead to cost savings or efficiency gains. By providing an easy first point of contact with the health system and making services more accessible, however, they can also increase demand for both, remote and face-to-face provider consultations and increase costs (Castle-Clarke, 2018[73]). Evidence on cost-effectiveness of care delivered through tele-medicine is context-specific and cannot be easily generalised with the data that are reported (Cravo Oliveira Hashiguchi, forthcoming[62]). As illustrated in the example of the UK Personalised Integrated Care Programme (Section 2.2.1), additional demand may also represent previously undiscovered health need, leading to better outcomes (and perhaps reduced long-term costs).

In another example, an initial evaluation of the “GP at Hand” primary care practice in London found that patient registration with the service coincided with declines in the use of emergency services, suggesting that there may be some efficiency gains from replacing costly hospital services. At the same time, the rapid uptake of services by people who are relatively young and healthy suggests issues with financial sustainability if the care delivery model were to be scaled. The initial evaluation of GP at Hand also indicated that people preferred remote consultations with physicians over automated services (Ipsos MORI et al., 2019[77]), suggesting that efficiency gains by substituting human resources may be difficult to achieve. Further information on GP at Hand is in Box 2.7.

copy the linklink copied!
Box 2.7. Babylon Health “GP at Hand” in London, England

GP at Hand is a primary care practice in central London that that provides remote consultations as first point of contact between patients and primary care professionals since July 2017. The practice is privately owned by Babylon Health and funded by the National Health Service (NHS) through the local Clinical Commissioning Group (CCG).

The CCG had a patient population of 231 000 people as per 1 January 2018. The number of people registering with GP at Hand increased rapidly after its introduction, reaching 49 000 by April 2019. Registered patients are younger, more educated and affluent and healthier than on average in London and England. For example, 81% of people registered are aged 20-39 years vs. 35% in London. Except for asthma, age- and sex-adjusted prevalence of common chronic diseases are 30% to 55% below the national averages. Older people and people with complex health needs and are less likely to register.

An online application, which includes a so-called symptom-checker linked to a triage system that recommends a course of action (e.g. to book an appointment, to go to A&E) but no diagnoses, provides the first point of contact for patients. Patients can also opt to book remote consultations without using the symptom checker. Remote consultations by phone or video are available around the clock, usually within two hours. Face-to-face appointments are offered at five clinics across London but, with exceptions for some services, patients are generally required to book a remote consultation first, which may then result in a face-to-face consultation. A multidisciplinary care team, led by a full-time care coordinator, is available to complex patients but only actively managed 51 patients as per early 2019.

Physicians can conduct remote consultations from home or from a physical hub in London. GP at Hand is funded in the same way as traditional primary care practices in England, through risk-adjusted capitation, and Babylon Health receive a portion of the funding for providing the digital infrastructure. Patients that register with GP at Hand are automatically deregistered from their prior practice. GP at Hand has a larger catchment area than traditional practices that only serve the population in their immediate vicinity; anybody who can access one of the five clinics within 40 minutes’ travel time can register, effectively extending the catchment area to much of greater London. In February 2019, NHS approved a request for the service to be extended to the city of Birmingham.

An evaluation of the first two years of operation was published in May 2019, based on a patient experience survey, qualitative case studies of service delivery, and analyses of service utilisation and economic impacts using routine data. Effectiveness in terms of health outcomes and cost-effectiveness were not evaluated.

The evaluation found that remote consultations with physicians were the most popular services, especially shortly after registering with the practice and by phone rather than video, followed by use of the symptom checker and face-to-face physician consultations. People are attracted to the practice by the ease of accessing services – approximately 40% of remote consultations occur outside of regular business hours and patients report appreciating quick responses, not needing to take time off work and relatively short travel times to face-to-face consultations. Patients were found to be satisfied with the quality of services, with 90% stating that it was ‘good’ and 60% that it was ‘very good’, which exceeded satisfaction in a matched control group. Approximately 70% reported that quality of care was better than at their previous primary care practice. GP at Hand patients are more intensive users of emergency services and NHS telephone support than the national average, but use of such services was found to decrease after registration, suggesting that the primary care service may substitute some emergency services. However, because of the lack of a control group, it is not clear if such use may also have decreased with registration at a traditional primary care practice.

The main concerns raised by professionals and patients surveyed were related to the suitability of the service for patient with complex needs and potential loss of continuity of care. Other concerns include the appropriateness of the funding through the traditional risk-adjusted capitation formula, and the overall financial impact on the NHS given that a large physician workforce provides around-the-clock services to a relatively young and healthy patient population.

Source: Based on Ipsos MORI et al. (2019[77]) “Evaluation of Babylon GP at hand: Final evaluation report”, https://www.hammersmithfulhamccg.nhs.uk/media/156123/Evaluation-of-Babylon-GP-at-Hand-Final-Report.pdf.

Ensuring equity is a challenge

Equitable access to enabling technologies and special support for people with lower digital and health literary are key prerequisites for tele-medicine to achieve its goals of increasing equity and efficiency. In addition, patients need to trust that the data transmitted by digital technologies are safe. These prerequisites are not always met in existing examples of care delivery models that feature tele-health technologies.

In many OECD countries, broadband internet access is more common in households in urban areas than in rural areas and in households with higher incomes (OECD, 2019[78]) (also see Section 2.4.2). The Gesundes Kinzigtal program in Germany (see Chapter 8 on system governance, stewardship and resource allocation for details), for example, struggled with insufficient IT infrastructure in remote areas and a large proportion of the population targeted, especially elderly people, were reluctant to adopt technologies, also due to lack of trust in data security (Melchiorre et al., 2018[22]). The review by Castle-Clarke (2018[73]) of ICT used in health care in the United Kingdom, including technology for remote delivery of services, found that people have limited knowledge of how data collected through ICT are used by the NHS and other organisations, which may be a cause of mistrust.

Unequal access to and use of ICT can lead to services be taken up by people with lower need, causing a misalignment between need and resource allocation. In England, for example, providers increasingly offer remote primary care services. GP at Hand (see Box 2.7) is mainly used by young and well-educated professionals but few patients with complex needs. Critics argue that use by mainly healthy and low-risk populations may divert resources away from people with higher need (DigitalHealth, 2018[79]; Oliver, 2019[80]; Iacobucci, 2018[81]), causing negative effects on efficiency and equity.

copy the linklink copied!2.4. A strategic approach is needed to planning scalability and sustainability of new was of delivering care

Innovative ways of delivering health care and supportive ICT are most often tested in pilot or research projects, with project-specific funding. But many fail to be scaled beyond the initial project phase even if they are promising or prove to be successful. There are a number of common challenges to the broad implementation of new ways of delivering care including financial, technological and cultural factors as well as change management more broadly. Overcoming these requires a holistic approach to design, planning, evaluation and implementation of projects, with an ultimate goal of scaling up successful projects and discontinuing unsuccessful ones.

Data quality is a cross-cutting concern that affects all types of secondary use of data and ICT-supported care delivery because data-driven health care and decision making are only as good as the data they are based on. Effective and efficient delivery of care requires reliable, accurate and timely information as well as effective use of ICT to produce knowledge and action. An OECD survey on the readiness of EHR data for secondary use showed that data quality remains a key concern and suggested a number of mechanisms countries can use to improve quality, including legal requirements, auditing and financial incentives (Oderkirk, 2017[82]). A report by the RAND Corporation identified poor data quality as well as a lack of data related to social determinants of health as particular barriers to progress in using data for the coordination of care for complex patients (Rudin et al., 2017[37]). Continued investments in data infrastructure, governance and quality therefore need to accompany new ways of delivering care.

2.4.1. An overall ICT strategy can guide design of individual projects and facilitate their scale-up

Countries that lead the way in adopting ICT to improve care delivery typically have instituted an overarching national or system-level digital strategy to guide individual projects (also see Chapter 8 on system governance, stewardship and resource allocation). Strategies often comprise mechanisms to select and fund innovative projects, to pilot new ways of care delivery and evaluate their effects and costs, and to scale-up successful projects. Strategies can catalyse the adoption and integration of innovative ways to deliver care without excessive disruption (Gray Steele et al., 2016[20]). They can also facilitate cooperation between providers, payers and the technology industry to encourage the development and implementation of ICT tools that meet patient and provider needs. The latter is key to implementing integrated care for complex patients (Melchiorre et al., 2018[22]).

As part of its digital health strategy, the Israeli Ministry of Health opens so-called challenge tenders to fund, implement and evaluate innovative ICT solutions in health care. In contrast to classical tendering in public procurement, in which specifications of the features of a solution are defined upfront and the most advantageous bid that meets specifications is then selected, challenge tenders do not prescribe an approach to tackling an issue. Rather, they broadly call for creative thinking in proposing technological solutions that may address a small or large part of an issue. For each of the challenges for which the Ministry opens a tender, it also leads efforts beyond the digital realm to tackle the problem systemically. Challenge tenders seek digital solutions that support the overall effort. Further information on the tendering process is in Box 2.8.

copy the linklink copied!
Box 2.8. Challenge tenders in the Israeli health system

The Israeli Ministry of Health launched a new funding and evaluation mechanism in 2016 for innovative ICT solutions to ‘challenges’ identified in the health system, referred to as challenge tenders.

Challenges are identified by the Ministry of Health through public consultations and interviews with stakeholders in the health system (for example, senior staff of the Ministry, HMOs and provider organisations). Challenges that are considered particularly amenable to ICT are prioritised by the Ministry. Tenders then involve two main stages.

In the first stage, the Ministry of Health only specifies the problem to be solved and private firms are asked to propose possible solutions. This gives the Ministry of Health visibility of the technologies that are available to help solve the problem at hand. The Ministry chooses possible solutions from these proposals.

Solutions selected in the first stage are presented to possible piloting organisations in the second stage to match a solution with a health care organisation and launch a pilot project. Pilots can be run at various levels of the health system, for example at an HMO or an individual hospital. More detailed specifications are defined at this stage in collaboration with the organisation that will host the pilot, including an update of key performance indicators (KPIs) for the solution.

To widen the range of potential solutions and encourage bids by innovative start-ups and other firms that are inexperienced with public procurement, the administrative and legal frameworks have been softened for challenge tenders. There are currently more than 500 health IT firms in Israel, many of which are small. While tenders are currently open to Israeli firms only, the Ministry plans to open them internationally in the future.

The Ministry of Health funds licensing, development, integration, project management and deployment of the solution at the piloting site. The piloting site funds local hardware, development and integration that is specific and may be needed on their side. The Ministry and piloting sites collaborate in evaluation and measuring the effectiveness of the solution based on KPIs defined in the tender.

Among other areas, solutions selected through challenge tenders are currently piloted for preventing medical errors that result from errors in patient identification and to prevent falls in the elderly population. Tenders may identify solutions from other sectors of the economy and result in a pilot of their applicability to health care. For example, in addition to a mobile application using technology similar to barcodes, biometric facial recognition algorithms used in the banking sector are implemented for identification of patients in hospitals. Digital solutions for preventing falls include portable sensors that people wear at home for real-time monitoring and analysis of the risk of falls and a range of tools, such as ‘smart’ treadmills, that help train people to improve their balance and stability.

Scaling to the national level is intended for successful solutions. Solutions are selected for scaling and long-term engagement with the Ministry only upon successful completion of a pilot, based on KPIs and projections of need for the solution and its costs. This stage has not been reached yet.

Source: Israeli Ministry of Health, personal communications.

2.4.2. Financial, technological and cultural barriers commonly impede broader adoption of new ways of delivering care

New models of care delivery also require new payment mechanisms

New ways of delivering care, by definition, require new processes and workflows. A recurrent barrier to scaling these innovations is insufficient funding or financial disincentives for adopting the new way of working. To overcome this, provider payment mechanisms need to be aligned to encourage care coordination and the use of supporting ICT. This requires a move away from fee-for-service (FFS) payments.

Alternative provider payment mechanisms, such as bundling, capitation and pay-for-performance, can play an important role in facilitating the adoption of new ways of delivering care (see Chapter 8 on system governance, stewardship and resource allocation for further discussion on provider payment). In Australia, for example, the Health Care Home (HCH) for patients with chronic and complex conditions described above deliberately deploys bundled payments instead of FFS (the conventional provider remuneration mechanism in Australia). Participating primary care practices that take overall responsibility of a patient’s care receive a monthly payment per patient to cover all care related to the chronic condition(s), including planning and review, and coordination of care (Health Policy Analysis, 2017[40]).

However, funding also needs to recognise up-front costs of designing and implementing ICT, and provide incentives or direct investment for implementing ICT tools that can increase effectiveness and efficiency of services at the margin. Up-front costs for designing and implementing ICT tools that support innovative care delivery are typically high while marginal costs of using them are usually low (sometimes approaching zero). For example, purchasing software might incur a one-off cost or annual license fee irrespective of whether it is used with 1, 10 or 1 000 patients. Additional funding may be necessary to cover initial implementation costs and relieve providers of some financial risk related to such investments. Sufficient upfront funding for new ICT tools can encourage innovation and finance necessary training and support for professionals and patients to facilitate implementation (Melchiorre et al., 2018[22]).

In scaling the GMA tool in Spain, the national Ministry of Health, Consumer Affairs and Social Wellbeing (MSCBS) funded the initial implementation of the tool in the various Spanish regions that manage their health systems autonomously. Ongoing costs of operating and using the tool are borne by regional health authorities.

In Canada, new care delivery models and services that rely on ICT are overseen and funded by Canada Health Infoway (also referred to as Infoway), an independent, not-for-profit organisation created and funded by the federal government. Infoway acts as a strategic investor, funding projects with provincial and territorial governments on a shared-cost basis, typically on a 75:25 ratio. Requests for proposals by provincial and territorial health departments adhere to defined criteria and milestones and all projects that receive Infoway funding are subject to an independent evaluation. Infoway has also directly designed, developed and implemented new ICT services, with 100% federal funding (such as PrescribeIT, an e-prescription system).

Interoperability and shared infrastructure enable scale-up of new ways of delivering care

Inadequate ICT infrastructures and limited interoperability of various tools are a common barriers to better integration of existing health care services (Melchiorre et al., 2018[22]). For example, among 101 innovative models of care for multi-morbid patients in Europe that use at least one ICT tool reviewed by Barbabella et al. (2017[23]), the scale of more than three-fourths remained local or regional and only about half were integrated into the wider health system.

Locally developed ways of care delivery and supporting ICT tools that are not interoperable with existing ICT infrastructures risk fragmenting care further rather than help integrate it and will also hamper scale-up of new ways of working beyond the local context. ICT tools that are developed in isolation for individual diseases also pose similar risks of further fragmentation of care for multi-morbid patients. Regulation and project funding mechanisms can set requirements for new ICT tools with respect to data standardisation, interoperability with other tools and their suitability for existing ICT infrastructures. These levers are typically best embedded in a national data governance and policy framework.

Skills need to evolve and cultural change be managed

Another significant issue pointed out in previous studies of ICT-supported care delivery is the lack of ICT skills among patients and professionals (Melchiorre et al., 2018[22]). Cultural factors, such as general resistance to change or professional autonomy, can also hamper adoption of new tools and new ways of working. User involvement and designing new care pathways and tools that do not add to the workload of professionals and self-care burden of patients are one way of reducing cultural resistance. Their involvement can also help create a sense of ownership and ultimately encourage uptake of the solutions.

In addition to a data governance framework that ensures data privacy, new tools also need to be accompanied by appropriate training programs, technical support and change management processes, in particular for health care professionals. While skilled professionals can lead the way in making patients more comfortable with new processes and technology, a lack of skills may reinforce cultural resistance to using ICT tools as a routine way of working (ibid., also see Chapter 4 on the health workforce).

Policy should tackle persistent disparities in digital and health literacy

With nearly ubiquitous availability of mobile devices and internet connectivity, the digital divide may be ostensibly narrowing, but inequalities in internet access persist. On average across OECD countries, the number of mobile broadband subscriptions is close to the number of people in the population (OECD, 2019[83]). However, especially in OECD countries with lower incomes but also in some high-income countries such as Belgium, Germany and Luxembourg, there are fewer subscriptions than people in the population (ibid.). Broadband access is more common among households in urban areas and with higher incomes (OECD, 2019[78]).

Significant inequalities in digital skills are also observed. As internet connectivity improves, related factors that inhibit adoption of ICT among high-need populations may gain in importance. Even if they are connected, population groups with high health need may still be disadvantaged in terms of their capacity to use ICT. People aged 55-74 were less likely than those aged 16 to 24 to use the internet in every OECD country for which data were available in 2016 (OECD, 2019[78]). Data on adult competences suggest that on average, 32% of those aged 55-65 have no computer experience or have failed core ICT tests, compared with just 5% of 16-24 year-olds (OECD, 2017[84]). The King’s Fund reports that about one-fifth of the United Kingdom population lack basic digital skills, in particular people in lower socio-economic groups (Castle-Clarke, 2018[73]). Figure 2.1 shows inequalities in the diffusion of online activities between people with high and low education levels in the population and between OECD countries.

copy the linklink copied!
Figure 2.1. Socio-economic disparities in online activities
Diffusion of selected online activities among individuals aged 16-74 in OECD countries, 2018.
Figure 2.1. Socio-economic disparities in online activities

Source: OECD ICT Access and Usage by Households and Individuals database (OECD, 2019[78]).

Poor digital literacy among high-need populations is one of the factors that impede the effective use of health-related ICT. A recent literature review on adoption of health-related applications in typically underserved populations,9 for example, found that the main barriers to adoption were low health literacy and lack of experience with using ICT; difficulties in accepting the presented information, for example, because it was considered not useful, confusing or contradictory to users’ own experience; and user-unfriendly and poorly designed interfaces (Huh et al., 2018[85]).

Foundational skills related to digital technology but also health literacy in all population groups, and in particular among the most vulnerable, are a key prerequisite for ICT-enabled care delivery to meet its goals. This is particularly true for care delivery supported by patient-interactive ICT, such as patient portals, remote monitoring or self-care devices. More broadly, further investments are needed to develop skills related to digital technologies and health literacy. These include offering incentives for and easing access to adult learning and improving the recognition of skills acquired after initial education so that everyone can participate in a digital society. Of course, more ‘upstream’ interventions such as social policies that support mobility and redistribution can also reduce digital divides.

Digital literacy is promoted by broader policies that aim to help people benefit equitably from an increasingly digitised economy. The Framework for Policy Action on Inclusive Growth (OECD, 2018[86]), which is part of the OECD Inclusive Growth Initiative, aims to help governments ensure a more equitable distribution of the benefits from economic growth along three major axes:

  • Investing in people and places that have been left behind, which highlights the promotion of life-long learning and the acquisition of skills, increasing social mobility, improving health and enhancing access to affordable housing, promoting regional catch-up and investing in community well-being. This requires ongoing financial commitment.

  • Supporting business dynamism and inclusive labour markets, which underscores the need to improve technology diffusion, innovation and entrepreneurship, as well as resilient labour markets and good jobs for all.

  • Building efficient and responsive governments, which advocates for good governance and people-centred digital government strategies, as well as a whole-of-government approach to policy development and implementation.

2.4.3. Pilot projects need to be evaluated rigorously to select successful ones for scale-up

Another barrier to successful scaling of new ways of delivering care is the lack of rigorous evidence of their effects in terms of health outcomes and costs. Especially evidence of the effects of new ICT, which is an enabler of new models of care, is often lacking (Safavi et al., 2019[87]).

In order to make investment decisions that improve health system performance and make care more efficient, pilot projects need to be evaluated rigorously and only successful ones should be sustained and scaled up. Evaluation should be an integral part of project implementation. In Germany, for example, the Innovation Fund of the Joint Federal Committee (G-BA), the highest decision-making body of the self-governing associations of health professionals, hospitals and social health insurers, finances projects that pilot innovative ways of delivering care. There is a legal requirement that projects that receive funding be evaluated and that successful ones be scaled up nationally. Projects are currently still in the pilot phase.

Routine health data should be deployed to evaluate care delivery

Rigorous methods, such as cluster-randomised controlled trials or case-control studies that rely on routine health care data, can be used to evaluate new ways of delivering care and supporting ICT tools in terms of their effects on process-related or health outcome measures. Where possible, evaluations should be conducted by independent parties who do not have a vested interest in the success of a new care delivery model or ICT tool. This can help avoid bias, and reduce the risk of non-publication of negative findings.

Evidence on ICT-enabled care delivery is currently building. But the evidence base in terms of health outcomes is still limited in several fields of application of ICT and cannot be generalised easily, for instance in remote patient monitoring (see Section 2.3.4 and Noah et al. (2018[63])). Evaluation methods are not always rigorous and pre-post studies without control groups are common. Rigorous evidence is also scarce on projects that make secondary use of clinical data to generate knowledge for improving health care. A recent literature review concluded that many studies report how secondary use of data should impact care processes, health outcomes, productivity and costs rather than actual effects (Meystre et al., 2017[88]). Pilot studies of health-related ICT tools have often yielded little evidence to guide further implementation and scale-up of these technologies (Wilson et al., 2018[89]). Where evidence is available, it can be difficult to interpret and to use for decision-making because of varying terminology, design of interventions and rapidly evolving technology (Shaw, Hines and Kielly-Carroll, 2018[24]).

Qualitative process evaluation can complement quantitative studies of effectiveness. Process evaluations can help, for example, distinguish reasons for failure of achievement of desirable outcomes between: (1) implementation failure or (2) the failure of the intervention itself (Maar et al., 2017[90]), which is particularly helpful in learning from failures of complex interventions that may fail for a variety of reasons. Identifying the main factors that caused success or failure can also help make adjustments to care delivery and ICT tools. This is particularly important in the development of ICT tools that are user friendly and support person-centred health care, which requires iterative approaches.

It is therefore key that health systems continue to evaluate new solutions and that evaluations be embedded in project implementations. Where possible, evaluations should make secondary use of existing data to produce results quickly and cheaply (also see Chapter 7 on biomedical technology). They should combine rigorous quantitative methods to assess effectiveness with qualitative research to explore the reasons for the results observed and help make adjustments to new ways of working and supporting ICT tools. Doing this well requires building (and investing in) the necessary technical and policy capacity.

A fit-for-purpose approval and regulation model may be needed

For health-related ICT tools, which can be developed in iterative processes that allow for changes and improvements to be made as soon as deficiencies become apparent, different evaluation methodologies might be needed than for medicines or medical devices. An agile and user-centred research and development cycles have been proposed to adapt the current 4-phase regulatory approval process for medicines and also allow for continuous and iterative development and incorporates development and testing processes typically used for ICT (Mathews et al., 2019[91]; Wilson et al., 2018[89]). Importantly, this approach would allow for limited market releases of ICT tools that are proven safe to test their effectiveness while allowing for replacement of existing versions by subsequent iterations as these become available (ibid.).

The US Food and Drug Administration (FDA) is currently implementing its Digital Health Innovation Action Plan. Under the plan, the regulatory agency is formulating new standards for mobile medical apps, tele-medicine and software as a medical device (SaMD), among other digital health technologies, to reduce delays in access to lower-risk technologies while ensuring safety and effectiveness of regulated technology.

2.4.4. Design of ICT tools should involve end users

As end users of new ICT tools, both patients and providers need to be involvement in their design because such tools have an indirect effect on care through altering workflows between professionals and in patient-provider interactions (Shaw, Hines and Kielly-Carroll, 2018[24]). Patients, in particular, may have different priorities from ICT firms, providers and payers, and therefore need to be represented in the development and implementation processes of ICT tools that support their care (Cohen et al., 2014[92]). Organisations need to establish learning mechanisms that allow patients and providers to identify incremental, progressive adjustments and feed those back to developers for improving ICT solutions (Shaw, Hines and Kielly-Carroll, 2018[24]).

Harnessing the collective wisdom of users in the design and implementation of tools is likely to make them more successful (Shaw, Hines and Kielly-Carroll, 2018[24]). Equally, novel technologies are likely to be successful only if they clearly reduce patient inconvenience and burden, helping them to accomplish their “illness work” more efficiently and effectively (Ancker et al., 2015[75]). For instance, a lack of harmonisation of digital health interventions with clinical pathways and existing systems may disrupt workflow. This in turn could lead to adverse effects on usability, accentuated implementation complexity and reduced patient safety. Secure messaging between consumers and clinicians has the potential to improve patient safety and quality, but may concurrently increase clinicians’ workload considerably, and impede their ability to respond to messages on time. Such effects on workflows need to be considered in the design of tools, to integrate them into workflows to support uptake and, ultimately, achieve positive outcomes.

Operational problems with many EHRs platforms are well documented. In some cases EHRs are so user-unfriendly that some physician practices employ scribes to enter information into the record while the provider interacts with the patient (Coiera et al., 2018[93]). The lack of practical functionality has several causes, including lack of user engagement in the purpose and design of the electronic platform and its interface. In the United States, the Department of Health and Human Services (HHS) is currently establishing a strategy to encourage the electronic exchange of health information by reducing the administrative burden of using EHRs and other health-related ICT. The strategy revolves around three overarching goals: reducing the effort and time required to record health information; reducing the effort and time required to meet regulatory reporting requirements; and improving the functionality and intuitiveness of EHRs (ONC, 2018[94]).

The Electronic Patient-Reported Outcomes Tool (ePRO) in Canada, for example, was developed in an iterative approach involving user groups of patients and primary care physicians (Steele Gray et al., 2016[95]). This is an innovative approach to designing mobile health technologies that meet patient needs and can be integrated into the care process. The tool allows providers and patients with complex chronic health needs to define and monitor patient-care plans to improve patient self-management and supports information sharing as well as shared decision-making between primary care physicians and other providers (ibid.). It is currently undergoing evaluation by randomised controlled trial to be completed in November 2019 (Steele Gray et al., 2016[96]).

copy the linklink copied!2.5. Conclusion

This chapter demonstrates that new ways of delivering health care supported by ICT have great potential to transform health care, making health systems more effective in improving population health, more equitable and more efficient in their use of resources. These goals can be achieved through harnessing ICT and electronic data to (a) redesign health services according to health needs of individuals and groups and (b) deliver these services in an integrated and patient-centred way.

The increasing number of patients with complex needs in OECD countries, who need health and social care services from several providers and over prolonged periods of time, stand to gain the most from new models of care delivery that use ICT. Digital technologies can help identify such patients, inform them about their own health and care, improve communication and coordination between them and their providers, increase the accuracy of diagnoses and clinical decision making, and help monitor their health remotely and deliver appropriate services across geographical distances.

However, a number of pitfalls need to be avoided if innovative and ICT-supported care delivery is to be effective, especially at scale. Importantly, ICT tools should not be seen as interventions in their own right – they are enabling tools that can alter and improve workflows in care delivery and need to be designed and implemented accordingly. Without an overarching ICT architecture that ensures that new tools are interoperable and can be integrated with existing information systems and stand-alone solutions for specific diseases, ICT can entrench and even exacerbate fragmentation of care rather than facilitate better integration. Policy also needs to ensure that ICT does not exacerbate inequity by favouring access to services by low-risk population groups with higher health and ICT literacy.

Secondary use of data is generally cheap, so greater use of existing data to generate knowledge and improve services can often be a highly cost-effective way of improving health outcomes. But this does not necessarily imply cost savings. The use of data and ICT can, for example, uncover unmet need and make new models of care delivery and digital services more accessible, increasing demand. This can lead to increases in aggregate expenditure. However, such cost implications should be seen in the context of (a) potentially improved health outcomes in vulnerable populations, and (b) reduced utilisation costs over the longer term. As with all technologies that generate and/or use personal health data for, ancillary and ethical concerns need to be managed.

Many OECD countries still appear to be far from realising the potential of ICT in transforming care delivery. The systematic identification of complex patients, system-wide efforts to integrate information systems to support integrated care delivery, making digital data and information accessible to patients and professionals, and truly integrated and knowledge-based care delivery models are still the exception rather than the rule. Only few countries report that the development of new and ICT-supported ways to deliver care target complex patients. Many ICT tools and models of care delivery described in this Chapter are local pilot projects, which, while promising, are yet unproven. In particular, rigorous evidence of the effectiveness and efficiency is still sparse. Countries with most success in this area have an overarching digital strategy and an integrated information infrastructure with a strong focus on interoperability through strong data governance.

It remains a challenge for health systems in OECD countries to evaluate innovative ways of working, discontinuing those that are ineffective or provide poor value for money and ensure that those, and only those, that prove successful are scaled up.

References

[30] Adler-Milstein, J. and C. Longhurst (2019), “Assessment of Patient Use of a New Approach to Access Health Record Data Among 12 US Health Systems”, JAMA Network Open, Vol. 2/8, p. e199544, http://dx.doi.org/10.1001/jamanetworkopen.2019.9544.

[17] Agarwala, V. et al. (2018), “Real-World Evidence In Support Of Precision Medicine: Clinico-Genomic Cancer Data As A Case Study”, Health Affairs, Vol. 37/5, pp. 765-772, http://dx.doi.org/10.1377/hlthaff.2017.1579.

[5] AIHW (2018), Australia’s Health 2018, Australian Institute of Health and Welfare, Canberra, https://www.aihw.gov.au/getmedia/7c42913d-295f-4bc9-9c24-4e44eff4a04a/aihw-aus-221.pdf.aspx?inline=true (accessed on 23 January 2019).

[75] Ancker, J. et al. (2015), ““You Get Reminded You’re a Sick Person”: Personal Data Tracking and Patients With Multiple Chronic Conditions.”, Journal of medical Internet research, Vol. 17/8, p. e202, http://dx.doi.org/10.2196/jmir.4209.

[47] Arbabshirani, M. et al. (2018), “Advanced machine learning in action: identification of intracranial hemorrhage on computed tomography scans of the head with clinical workflow integration”, npj Digital Medicine, Vol. 1, p. 9, http://dx.doi.org/10.1038/s41746-017-0015-z.

[23] Barbabella, F. et al. (2017), How can eHealth improve care for people with multimorbidity in Europe?, NIVEL and TU Berlin, Utrecht, Netherlands, http://www.icare4eu.org/pdf/PB_25.pdf.

[48] Bennett, P. and N. Hardiker (2016), “The use of computerized clinical decision support systems in emergency care: a substantive review of the literature”, Journal of the American Medical Informatics Association, Vol. 24/3, p. ocw151, http://dx.doi.org/10.1093/jamia/ocw151.

[98] Cainzos-Achirica, M. et al. (2018), “Individual income, mortality and healthcare resource use in patients with chronic heart failure living in a universal healthcare system: A population-based study in Catalonia, Spain”, International Journal of Cardiology, http://dx.doi.org/10.1016/j.ijcard.2018.10.099.

[53] Callen, J. et al. (2013), “Can technology change the work of nurses? Evaluation of a drug monitoring system for ambulatory chronic disease patients.”, International journal of medical informatics, Vol. 82/3, pp. 159-167, http://dx.doi.org/10.1016/j.ijmedinf.2012.11.009.

[66] Canada Health Infoway (2016), Ontario Telemedicine Network Telehomecare Deployment Project: Phase 2-Remote Patient Monitoring, Canada Health Infoway, Ottawa, https://www.infoway-inforoute.ca/en/plan/3176-ontario-telemedicine-network-telehomecare-deployment-project-phase-2-remote-patient-monitoring/download (accessed on 25 January 2019).

[99] Cancio, J. et al. (2018), “Influence of demographic and clinical characteristics of elderly patients with a hip fracture on mortality: A retrospective, total cohort study in North-East Spain”, Bone, Vol. 117, pp. 123-29, http://dx.doi.org/10.1016/j.bone.2018.09.002.

[73] Castle-Clarke, S. (2018), What will new technology mean for the NHS and its patients ? Four big technological trends, The King’s Fund, London, https://www.kingsfund.org.uk/sites/default/files/2018-06/NHS_at_70_what_will_new_technology_mean_for_the_NHS_0.pdf.

[92] Cohen, I. et al. (2014), “The legal and ethical concerns that arise from using complex predictive analytics in health care.”, Health affairs (Project Hope), Vol. 33/7, pp. 1139-1147, http://dx.doi.org/10.1377/hlthaff.2014.0048.

[93] Coiera, E. et al. (2018), “The digital scribe”, npj Digital Medicine, Vol. 1/1, http://dx.doi.org/10.1038/s41746-018-0066-9.

[35] Craig, J. et al. (2015), “An evaluation of the impact of the key information summary on GPs and out-of-hours clinicians in NHS Scotland”, Scottish Medical Journal, Vol. 60/3, pp. 126-131, http://dx.doi.org/10.1177/0036933015577540.

[62] Cravo Oliveira Hashiguchi, T. (forthcoming), “Bringing health care to the patient. An overview of telemedicine in OECD countries”, OECD Health Working Papers, OECD Publishing, Paris.

[74] Dehling, T. et al. (2015), “Exploring the Far Side of Mobile Health: Information Security and Privacy of Mobile Health Apps on iOS and Android.”, JMIR mHealth and uHealth, Vol. 3/1, p. e8, http://dx.doi.org/10.2196/mhealth.3672.

[64] Delgadillo, J. et al. (2018), “Feedback-informed treatment versus usual psychological treatment for depression and anxiety: a multisite, open-label, cluster randomised controlled trial”, The Lancet Psychiatry, Vol. 5/7, pp. 564-572, http://dx.doi.org/10.1016/S2215-0366(18)30162-7.

[21] Demiris, G. and L. Kneale (2015), “Informatics Systems and Tools to Facilitate Patient-centered Care Coordination.”, Yearbook of medical informatics, Vol. 10/1, pp. 15-21, http://dx.doi.org/10.15265/IY-2015-003.

[79] DigitalHealth (2018), NHS England puts out £250k tender for GP at Hand evaluation, DigitalHealth, https://www.digitalhealth.net/2018/03/nhs-england-250k-independent-evaluation-gp-at-hand/ (accessed on 21 January 2019).

[46] Dimitriou, N. et al. (2018), “A principled machine learning framework improves accuracy of stage II colorectal cancer prognosis”, npj Digital Medicine, Vol. 1/1, p. 52, http://dx.doi.org/10.1038/s41746-018-0057-x.

[57] Fraccaro, P. et al. (2015), “Adoption of clinical decision support in multimorbidity: a systematic review.”, JMIR medical informatics, Vol. 3/1, p. e4, http://dx.doi.org/10.2196/medinform.3503.

[43] Ghiotto, M. et al. (2018), “Strengthening primary care: The Veneto Region’s model of the Integrated Medical Group”, Health Policy, Vol. 122/11, pp. 1149-1154, http://dx.doi.org/10.1016/J.HEALTHPOL.2018.08.008.

[12] Gietzelt, M. et al. (2016), “Models and Data Sources Used in Systems Medicine. A Systematic Literature Review.”, Methods of information in medicine, Vol. 55/2, pp. 107-13, http://dx.doi.org/10.3414/ME15-01-0151.

[65] Glazebrook, C. and E. Davies (2018), “Outcome feedback technology helps therapists to tailor care”, The Lancet Psychiatry, Vol. 5/7, pp. 529-531, http://dx.doi.org/10.1016/S2215-0366(18)30212-8.

[27] Goldzweig, C. et al. (2013), “Electronic Patient Portals: Evidence on Health Outcomes, Satisfaction, Efficiency, and Attitudes”, Annals of Internal Medicine, Vol. 159/10, p. 677, http://dx.doi.org/10.7326/0003-4819-159-10-201311190-00006.

[20] Gray Steele, C. et al. (2016), “eHealth Advances in Support of People with Complex Care Needs: Case Examples from Canada, Scotland and the US.”, Healthcare quarterly (Toronto, Ont.), Vol. 19/2, pp. 29-37.

[72] Hale, T. et al. (2016), “A Remote Medication Monitoring System for Chronic Heart Failure Patients to Reduce Readmissions: A Two-Arm Randomized Pilot Study.”, Journal of medical Internet research, Vol. 18/5, p. e91, http://dx.doi.org/10.2196/jmir.5256.

[7] Hayes, S. et al. (2016), High-Need, High-Cost Patients: Who Are They and How Do They Use Health Care? A Population-Based Comparison of Demographics, Health Care Use, and Expenditures, Commonwealth Fund, New York City, NY, https://www.commonwealthfund.org/sites/default/files/documents/___media_files_publications_issue_brief_2016_aug_1897_hayes_who_are_high_need_high_cost_patients_v2.pdf (accessed on 10 December 2018).

[40] Health Policy Analysis (2017), Evaluation of the Health Care Homes program: Evaluation plan, Australian Department of Health, Canberra, https://www.hchevaluation.com/images/Project_documents/Health_Care_Homes_Evaluation_plan.pdf (accessed on 23 January 2019).

[85] Huh, J. et al. (2018), “Consumer Health Informatics Adoption among Underserved Populations: Thinking beyond the Digital Divide”, http://dx.doi.org/10.1055/s-0038-1641217.

[49] Hujala, A. et al. (2016), The POTKU project (Potilas kuljettajan paikalle, Putting the Patient in the Driver’s Seat), Finland, http://www.icare4eu.org/pdf/POTKU_Case_report.pdf (accessed on 18 December 2018).

[67] Hunting, G. et al. (2015), “A multi-level qualitative analysis of Telehomecare in Ontario: challenges and opportunities”, BMC Health Services Research, Vol. 15/1, p. 544, http://dx.doi.org/10.1186/s12913-015-1196-2.

[81] Iacobucci, G. (2018), “GP at Hand: Where have the new patients come from?”, BMJ, p. k2227, http://dx.doi.org/10.1136/bmj.k2227.

[77] Ipsos MORI et al. (2019), Evaluation of Babylon GP at hand. Final evaluation report, https://www.hammersmithfulhamccg.nhs.uk/media/156123/Evaluation-of-Babylon-GP-at-Hand-Final-Report.pdf (accessed on 17 June 2019).

[15] JASON (2018), Artificial Intelligence for Health and Health Care, https://www.healthit.gov/sites/default/files/jsr-17-task-002_aiforhealthandhealthcare12122017.pdf.

[44] Joint Federal Committee (G-BA) (2019), ACD – Accountable Care in Deutschland - Verbesserung der Patientenversorgung durch Vernetzung von Leistungserbringern und informierten Dialog, https://innovationsfonds.g-ba.de/projekte/versorgungsforschung/acd-accountable-care-in-deutschland-verbesserung-der-patientenversorgung-durch-vernetzung-von-leistungserbringern-und-informierten-dialog.45.

[16] Kannan, V. et al. (2017), “Rapid Development of Specialty Population Registries and Quality Measures from Electronic Health Record Data: An Agile Framework”, Methods of Information in Medicine, Vol. 56/99, pp. e74-83, http://dx.doi.org/10.3414/ME16-02-0031.

[58] Kipnis, P. et al. (2016), “Development and validation of an electronic medical record-based alert score for detection of inpatient deterioration outside the ICU.”, Journal of biomedical informatics, Vol. 64, pp. 10-19, http://dx.doi.org/10.1016/j.jbi.2016.09.013.

[2] Koné Pefoyo, A. et al. (2015), “The increasing burden and complexity of multimorbidity”, BMC Public Health, Vol. 15, http://dx.doi.org/10.1186/s12889-015-1733-2.

[52] Lavan, A., P. Gallagher and D. O’Mahony (2016), “Methods to reduce prescribing errors in elderly patients with multimorbidity”, Clinical Interventions in Aging, Vol. 11, p. 857, http://dx.doi.org/10.2147/CIA.S80280.

[36] Lin, S., A. Jha and J. Adler-Milstein (2018), “Electronic Health Records Associated With Lower Hospital Mortality After Systems Have Time To Mature”, Health Affairs, Vol. 37/7, pp. 1128-1135, http://dx.doi.org/10.1377/hlthaff.2017.1658.

[45] Litjens, G. et al. (2017), “A survey on deep learning in medical image analysis”, Medical Image Analysis, Vol. 42, pp. 60-88, http://dx.doi.org/10.1016/j.media.2017.07.005.

[31] Lupiañez-Villanueva, F., A. Sachinopoulou and A. Theben (2015), Oulu Self-Care (Finland) Case Study Report, European Commission Joint Research Centre, Luxembourg, http://dx.doi.org/10.2791/692203.

[90] Maar, M. et al. (2017), “A Framework for the Study of Complex mHealth Interventions in Diverse Cultural Settings.”, JMIR mHealth and uHealth, Vol. 5/4, p. e47, http://dx.doi.org/10.2196/mhealth.7044.

[91] Mathews, S. et al. (2019), “Digital health: a path to validation”, npj Digital Medicine, Vol. 2/1, http://dx.doi.org/10.1038/s41746-019-0111-3.

[34] McAlearney, A. et al. (2016), “High Touch and High Tech (HT2) Proposal: Transforming Patient Engagement Throughout the Continuum of Care by Engaging Patients with Portal Technology at the Bedside.”, JMIR research protocols, Vol. 5/4, p. e221, http://dx.doi.org/10.2196/resprot.6355.

[22] Melchiorre, M. et al. (2018), “eHealth in integrated care programs for people with multimorbidity in Europe: Insights from the ICARE4EU project”, Health Policy, Vol. 122/1, pp. 53-63, http://dx.doi.org/10.1016/j.healthpol.2017.08.006.

[88] Meystre, S. et al. (2017), “Clinical Data Reuse or Secondary Use: Current Status and Potential Future Progress I Introduction”, Yearbook of Medical Informatics, pp. 38-52, http://dx.doi.org/10.15265/IY-2017-007.

[60] Miotto, R. et al. (2016), “Deep Patient: An Unsupervised Representation to Predict the Future of Patients from the Electronic Health Records OPEN”, Nature Scientific Reports, Vol. 6, p. 26094, http://dx.doi.org/10.1038/srep26094.

[101] Miquel, M. et al. (2018), “Economic burden of cirrhosis in Catalonia: A population-based analysis”, BMJ Open, Vol. 8/e018012, http://dx.doi.org/10.1136/bmjopen-2017-018012.

[51] Molokhia, M. and A. Majeed (2017), “Current and future perspectives on the management of polypharmacy.”, BMC family practice, Vol. 18/1, p. 70, http://dx.doi.org/10.1186/s12875-017-0642-0.

[3] Mondor, L. et al. (2018), “Income inequalities in multimorbidity prevalence in Ontario, Canada: a decomposition analysis of linked survey and health administrative data”, International Journal for Equity in Health, Vol. 17/1, p. 90, http://dx.doi.org/10.1186/s12939-018-0800-6.

[8] Monterde, D., E. Vela and M. Clèries (2016), “Los grupos de morbilidad ajustados: nuevo agrupador de morbilidad poblacional de utilidad en el ámbito de la atención primaria”, Atención Primaria, Vol. 48/10, pp. 674-682, http://dx.doi.org/10.1016/J.APRIM.2016.06.003.

[50] Morin, L. et al. (2018), “The epidemiology of polypharmacy in older adults: register-based prospective cohort study”, Clinical Epidemiology, Vol. 10, pp. 289-98, http://dx.doi.org/10.2147/CLEP.S153458.

[25] Morton, K. et al. (2017), “Using digital interventions for self-management of chronic physical health conditions: A meta-ethnography review of published studies.”, Patient education and counseling, Vol. 100/4, pp. 616-635, http://dx.doi.org/10.1016/j.pec.2016.10.019.

[42] Mulvale, G., M. Embrett and S. Razavi (2016), “‘Gearing Up’ to improve interprofessional collaboration in primary care: a systematic review and conceptual framework”, BMC Family Practice, Vol. 17/1, p. 83, http://dx.doi.org/10.1186/s12875-016-0492-1.

[29] NHS Digital (2019), Patient Online Management Information (POMI), https://digital.nhs.uk/data-and-information/data-collections-and-data-sets/data-collections/pomi.

[63] Noah, B. et al. (2018), “Impact of remote patient monitoring on clinical outcomes: an updated meta-analysis of randomized controlled trials”, npj Digital Medicine, Vol. 1, http://dx.doi.org/10.1038/s41746-017-0002-4.

[10] Nuffield Trust (2019), Age UK’s Personalised Integrated Care Programme: Evaluation of impact on hospital activity, https://www.nuffieldtrust.org.uk/research/age-uk-s-personalised-integrated-care-programme-evaluation-of-impact-on-hospital-activity.

[82] Oderkirk, J. (2017), “Readiness of electronic health record systems to contribute to national health information and research”, OECD Health Working Papers, No. 99, OECD Publishing, Paris, https://dx.doi.org/10.1787/9e296bf3-en.

[1] OECD (2019), Going Digital: Shaping Policies, Improving Lives, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264312012-en.

[78] OECD (2019), “ICT Access and Usage by Households and Individuals”, OECD Telecommunications and Internet Statistics (database), https://dx.doi.org/10.1787/b9823565-en (accessed on 4 November 2019).

[83] OECD (2019), Mobile broadband subscriptions (indicator), https://dx.doi.org/10.1787/1277ddc6-en (accessed on 17 June 2019).

[86] OECD (2018), Opportunities for All: A Framework for Policy Action on Inclusive Growth, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264301665-en.

[97] OECD (2017), New Health Technologies: Managing Access, Value and Sustainability, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264266438-en.

[84] OECD (2017), Preventing Ageing Unequally, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264279087-en.

[61] Olchanski, N. et al. (2017), “Can a Novel ICU Data Display Positively Affect Patient Outcomes and Save Lives?”, Journal of Medical Systems, Vol. 41/171, http://dx.doi.org/10.1007/s10916-017-0810-8.

[80] Oliver, D. (2019), “David Oliver: Lessons from the Babylon Health saga”, BMJ, p. l2387, http://dx.doi.org/10.1136/bmj.l2387.

[94] ONC (2018), DRAFT: Strategy on Reducing Regulatory and Administrative Burden Relating to the Use of Health IT and EHRs, https://www.healthit.gov/sites/default/files/page/2018-11/Draft%20Strategy%20on%20Reducing%20Regulatory%20and%20Administrative%20Burden%20Relating.pdf.

[68] Ontario College of Family Physicians (2016), Telehomecare supports chronic disease management, https://ocfp.on.ca/communications/telehomecare-supports-chronic-disease-management (accessed on 25 January 2019).

[32] Oulu Healthcare and Social Welfare (2018), Oulu Self Care, European Commission, https://ec.europa.eu/health/sites/health/files/non_communicable_diseases/docs/ev_20181212_co04_en.pdf (accessed on 14 January 2019).

[6] Palladino, R. et al. (2019), “Multimorbidity And Health Outcomes In Older Adults In Ten European Health Systems, 2006–15”, Health Affairs, Vol. 38/4, pp. 613-623, http://dx.doi.org/10.1377/hlthaff.2018.05273.

[76] Pecina, J. and F. North (2016), “Early e-consultation face-to-face conversions”, Journal of Telemedicine and Telecare, Vol. 22/5, pp. 269-276, http://dx.doi.org/10.1177/1357633X15602634.

[13] Phinney, M. et al. (2017), “Contrast Mining for Pattern Discovery and Descriptive Analytics to Tailor Sub-Groups of Patients Using Big Data Solutions.”, Studies in health technology and informatics, Vol. 245, pp. 544-548.

[59] Rajkomar, A. et al. (2018), “Scalable and accurate deep learning with electronic health records”, npj Digital Medicine, Vol. 1/1, p. 18, http://dx.doi.org/10.1038/s41746-018-0029-1.

[26] Roberts, S. et al. (2017), “Using technology to engage hospitalised patients in their care: a realist review.”, BMC health services research, Vol. 17/1, p. 388, http://dx.doi.org/10.1186/s12913-017-2314-0.

[38] Rotenstein, L. et al. (2016), “The critical components of an electronic care plan tool for primary care: an exploratory qualitative study.”, Journal of innovation in health informatics, Vol. 23/2, p. 836.

[37] Rudin, R. et al. (2017), “Identifying and Coordinating Care for Complex Patients: Findings from the Leading Edge of Analytics and Health Information Technology.”, Rand health quarterly, Vol. 6/3, p. 2.

[39] Rudin, R. et al. (2016), “Knowledge gaps inhibit health IT development for coordinating complex patients’ care.”, The American journal of managed care, Vol. 22/9, pp. e317-22.

[56] Sacchi, L. et al. (2015), “Personalization and Patient Involvement in Decision Support Systems: Current Trends.”, Yearbook of medical informatics, Vol. 10/1, pp. 106-18, http://dx.doi.org/10.15265/IY-2015-015.

[87] Safavi, K. et al. (2019), “Top-Funded Digital Health Companies And Their Impact On High-Burden, High-Cost Conditions”, Health Affairs, Vol. 38/1, pp. 115-23, http://dx.doi.org/10.1377/hlthaff.2018.05081.

[69] Sahakyan, Y. et al. (2018), “Changes in blood pressure among patients in the Ontario Telehomecare programme: An observational longitudinal cohort study”, Journal of Telemedicine and Telecare, Vol. 24/6, pp. 420-427, http://dx.doi.org/10.1177/1357633X17706286.

[28] Schildmeijer, K. et al. (2018), “Developing an eHealth Tool to Support Patient Empowerment at Home.”, Studies in health technology and informatics, Vol. 247, pp. 925-929.

[4] Schiøtz, M. et al. (2017), “Social disparities in the prevalence of multimorbidity – A register-based population study”, BMC Public Health, Vol. 17/1, p. 422, http://dx.doi.org/10.1186/s12889-017-4314-8.

[41] Scirocco Project (2017), Overview of Scirocco Good Practices: Basque Country - Care Plan for Elderly, https://www.scirocco-project.eu/wp-content/uploads/2017/04/SciroccoGP-Basque-6-Care-Plan-for-Elderly.pdf (accessed on 15 March 2019).

[24] Shaw, T., M. Hines and C. Kielly-Carroll (2018), Impact of Digital Health on the Safety and Quality of Health Care, Australian Commission on Safety and Quality in Health Care, Sydney, https://www.safetyandquality.gov.au/wp-content/uploads/2018/02/Report-The-Impact-of-Digital-Health-on-Safety-and-Quality-of-Healthcar....pdf (accessed on 31 August 2018).

[9] Sheikh, A. (ed.) (2019), “Health system costs for individual and comorbid noncommunicable diseases: An analysis of publicly funded health events from New Zealand”, PLOS Medicine, Vol. 16/1, p. e1002716, http://dx.doi.org/10.1371/journal.pmed.1002716.

[70] Stanimirovic, A. et al. (2018), “IMPACT OF TELEHOMECARE ON HEALTH SYSTEM UTILIZATION IN PATIENTS WITH HEART FAILURE”, Canadian Journal of Cardiology, Vol. 34/10, p. S179, http://dx.doi.org/10.1016/j.cjca.2018.07.143.

[95] Steele Gray, C. et al. (2016), “Improving Patient Experience and Primary Care Quality for Patients With Complex Chronic Disease Using the Electronic Patient-Reported Outcomes Tool: Adopting Qualitative Methods Into a User-Centered Design Approach.”, JMIR research protocols, Vol. 5/1, p. e28, http://dx.doi.org/10.2196/resprot.5204.

[96] Steele Gray, C. et al. (2016), “Supporting Goal-Oriented Primary Health Care for Seniors with Complex Care Needs Using Mobile Technology: Evaluation and Implementation of the Health System Performance Research Network, Bridgepoint Electronic Patient Reported Outcome Tool.”, JMIR research protocols, Vol. 5/2, p. e126, http://dx.doi.org/10.2196/resprot.5756.

[19] Suter, E. et al. (2009), “Ten Key Principles for Successful Health Systems Integration”, Healthcare Quarterly, Vol. 13/sp, pp. 16-23, http://dx.doi.org/10.12927/hcq.2009.21092.

[11] Tillmann, T. et al. (2015), “Systems Medicine 2.0: potential benefits of combining electronic health care records with systems science models.”, Journal of Medical Internet Research, Vol. 17/3, p. e64, http://dx.doi.org/10.2196/jmir.3082.

[14] Topol, E. (2019), “High-performance medicine: the convergence of human and artificial intelligence”, Nature Medicine, Vol. 25/1, pp. 44-56, http://dx.doi.org/10.1038/s41591-018-0300-7.

[100] Vela, E. et al. (2018), “Population-based analysis of patients with COPD in Catalonia: A cohort study with implications for clinical management”, BMJ Open, Vol. 8/e017283, http://dx.doi.org/10.1136/bmjopen-2017-017283.

[71] Vrijens, B., J. Urquhart and D. White (2014), “Electronically monitored dosing histories can be used to develop a medication-taking habit and manage patient adherence.”, Expert review of clinical pharmacology, Vol. 7/5, pp. 633-644, http://dx.doi.org/10.1586/17512433.2014.940896.

[54] Wilk, S. et al. (2017), “Comprehensive mitigation framework for concurrent application of multiple clinical practice guidelines”, Journal of Biomedical Informatics, Vol. 66, pp. 52-71, http://dx.doi.org/10.1016/J.JBI.2016.12.002.

[18] Williams, M. et al. (2018), “Patient-Centered Precision Health In A Learning Health Care System: Geisinger’s Genomic Medicine Experience”, Health Affairs, Vol. 37/5, pp. 757-764, http://dx.doi.org/10.1377/hlthaff.2017.1557.

[89] Wilson, K. et al. (2018), “Agile research to complement agile development: a proposal for an mHealth research lifecycle”, npj Digital Medicine, Vol. 1, p. 46, http://dx.doi.org/10.1038/s41746-018-0053-1.

[33] Yamin, C. et al. (2011), “The Digital Divide in Adoption and Use of a Personal Health Record”, Archives of Internal Medicine, Vol. 171/6, http://dx.doi.org/10.1001/archinternmed.2011.34.

[55] Zamborlini, V. et al. (2016), “Inferring Recommendation Interactions in Clinical Guidelines”, Semantic Web, Vol. 7/4, http://www.semantic-web-journal.net/system/files/swj891.pdf (accessed on 8 November 2018).

Notes

← 1. The terms ICT and digital technology are used interchangeably in this Chapter and throughout this Report.

← 2. Austria, Belgium, the Czech Republic, Denmark, France, Germany, Italy, Spain, Sweden, and Switzerland.

← 3. Whereas precision medicine is defined as refining the understanding of disease prediction and risk, onset and progression in patients, to inform better selection and development of evidence-based and targeted therapies and associated diagnostics. This is achieved by taking into account the patient’s genomic and other biological characteristics, as well as health status, medications patients are already prescribed and environmental and lifestyle factors (OECD, 2017[97]). Both, precision medicine and personalised care are heavily reliant on evidence derived from secondary use of real-world or routine data.

← 4. See, for example, Cainzos-Achirica et al. (2018[98]), Cancio et al. (2018[99]), Miquel et al. (2018[101]) or Vela et al. (2018[100]).

← 5. For further discussion regarding the opportunities and risks of engaging patients with data, see Chapter 3 on The informed patient.

← 6. See https://echo.unm.edu.

← 7. Although the terms tele-medicine and tele-health are often used interchangeably, tele-health is broader than tele-medicine and encompasses any use of ICT to promote health, including non-clinical services. See Cravo Oliveira Hashiguchi (forthcoming[62]) for definitions and an overview of the broader e-health ecosystem that includes tele-medicine and tele-health.

← 8. See www.headtohealth.gov.au.

← 9. Including people among racial/ethnic minorities in the study context of the country; with lower educational attainment and literacy; facing economic barriers to accessing health care, e.g. as a result of employment status, poverty or insurance status; and people living in geographically isolated areas.

Metadata, Legal and Rights

This document, as well as any data and map included herein, are without prejudice to the status of or sovereignty over any territory, to the delimitation of international frontiers and boundaries and to the name of any territory, city or area. Extracts from publications may be subject to additional disclaimers, which are set out in the complete version of the publication, available at the link provided.

https://doi.org/10.1787/e3b23f8e-en

© OECD 2019

The use of this work, whether digital or print, is governed by the Terms and Conditions to be found at http://www.oecd.org/termsandconditions.