Lessons from shortcomings in machine learning for medical imaging

Large datasets are not a panacea

There is a tendency to expect that a clinical task can be “solved” if the dataset is large enough. After all, from prior research, large and diverse datasets help an algorithm generalise better to previously unseen data. There are several problems here. First, not all clinical tasks translate neatly into ML tasks. Second, creating larger and larger datasets often relies on automatic methods that may introduce errors and bias into the data (Oakden-Rayner, 2020). For example, a machine might label x-rays as showing the presence or non-presence of pneumonia based on words appearing in the associated radiology reports. In such a case, a phrase like “no history of pneumonia” might result in an x-ray wrongly labelled as showing the presence of pneumonia.

Finally, while large datasets improve algorithm training and generalisation, they also allow for better evaluation of algorithms. This is because more data are available for creating an estimate of performance on previously unseen future data. Analysis of predictions of Alzheimer’s disease across six surveys and more than 500 publications in Figure 1 shows that studies with larger sample sizes tend to report worse prediction accuracy. This is worrying since these studies are closer to real-life settings.

Figure 1. Larger brain-imaging datasets do not yield better machine-learning diagnosis of Alzheimer's disease

Note: From a clinical standpoint, two important medical imaging problems using machine learning are to 1) distinguish Alzheimer's disease (AD) both from a healthy control (HC), and from mild cognitive impairment (MCI), which can signal the onset of AD; and 2) distinguish progressive mild cognitive impairment (pMCI) from stable mild cognitive impairment (sMCI).

Source: Varoquaux and Cheplygina (2022).

Algorithm research may hit diminishing returns

A lot of research within medical imaging focuses on algorithm development, but the practical benefits of the reported accuracy gains are not always clear. For this essay, the authors studied eight medical imaging competitions on Kaggle, a platform where algorithm developers can compete to solve classification tasks, and where winning can involve significant incentives. Indeed, the most famous competition on lung cancer prediction had a prize of USD 1 million. The analysis compared two quantities: 1) the gap between the performance of the top algorithms; and 2) the expected variability in performance if a different subset of the data was used for evaluation. In other words, it tried to quantify the meaning of the final ranking. Would ranking of the winners alter if other images were used from the same or a different subset of the data? In most cases, the performance of the top algorithms is within the expected variability, and algorithms are thus not practically better or worse than one another (Varoquaux and Cheplygina, 2022).

Lack of representation from underdeveloped regions

Deep-learning studies are computationally intensive, and several ML studies have noted how this affects who gets to do research. A method may win just because more computational resources were available (Hooker, 2020). Meanwhile, the representation of prestigious labs and tech companies at conferences is increasing (Ahmed and Wahed, 2020). At a large medical imaging conference –MICCAI 2020 – only 2% of accepted papers were from underrepresented regions (Africa, Latin America, South/South-East Asia and the Middle East) (MICCAI Society, 2021). However, the need for medical AI might be even greater in these regions.

Build awareness of data limitations

It may not always be feasible to collect more data. However, it is important to understand the limitations of the data that are available, such as the sample size and characteristics of different patient groups. On this note, datasets should include a report of the data characteristics, as well as the potential implications for models trained on the data. Such a practice would be similar to providing “model cards”, a short document that accompanies a trained ML model and details benchmarked model performance under different conditions (Mitchell et al., 2019).

Reinvent benchmarking

Benchmarking the performance of algorithms alone is not sufficient to advance the field. Papers focusing on understanding, replication of earlier results and so forth are also valuable. If benchmarking the performance of algorithms is deemed essential in a publication, comparisons need to include both recent-and-competitive and traditional-yet-effective methods.

Furthermore, comparisons need to consider the range (rather than a single estimate) of each method’s performance. Ideally, they should use multiple, well-motivated metrics and statistical procedures (Bouthillier et al., 2021). More real-life effects of an algorithm might also be considered. This might include, for example, its carbon footprint, or how it affects the people it was designed to help (Thomas and Uminsky, 2020).

Improve publication norms

Many want to believe that publishing a novel algorithm with state-of-the-art results is the only way to create impact, but such results may be overly optimistic. In the practice of psychology, registered reports are prepared. In this approach, a planned study is reviewed and published before any experiments are done. More widespread adoption of this practice could reduce publication bias since “negative results” would also be published. From an institutional perspective, one could support different types of papers focusing on different forms of insight. These could include replications or retrospective analyses of methods, incentivising and rewarding (e.g. through research funding, hiring decisions) such practices.

Quality rather than quantity

Several of the current problems stem from the way researchers are evaluated when applying for academic positions or for research funding. The focus on metrics like the h-index needs to be reduced in favour of other practices, such as, for example, an evaluation of five selected publications. Such a shift could reduce the pressures that lead to publication of research with diminishing returns. The need for new approaches for evaluating research also holds when evaluating researchers based on previously acquired funding, which can entail the propagation of existing biases.

Funding for rigorous evaluation

Funding should focus less on perceived novelty, and more on rigorous evaluation practices. Such practices could include evaluation of existing algorithms, replication of existing studies and prospective studies. This would provide more realistic evaluations of how algorithms might perform in practice. Ideally, such funding schemes should be accessible to early career researchers, for example, by not requiring a permanent position at application.

Better recognition for open data and software

It should be more attractive to work on curated datasets and open-source software that everybody can use. It is difficult to acquire funding, and often to publish, when working on such projects. Many team members are therefore volunteers. This creates biases against groups that are already underrepresented but that might have innovative ideas that would be vital for the field. Such groups could include, for example, women who take on a greater share of household responsibilities and lower-income countries who cannot afford to take on unpaid jobs. More regular funding and consequently more secure positions would help to improve on the status quo.