Common Pitfalls and Recommendations for Using Machine Learning to Detect and Prognosticate for COVID-19 Using Chest Radiographs and CT Scans

Roberts, Michael and Driggs, Derek and Thorpe, Matthew and Gilbey, Julian and Yeung, Michael and Ursprung, Stephan and Aviles-Rivero, Angelica I. and Etmann, Christian and McCague, Cathal and Beer, Lucian and Weir-McCall, Jonathan R. and Teng, Zhongzhao and Gkrania-Klotsas, Effrossyni and Ruggiero, Alessandro and Korhonen, Anna and Jefferson, Emily and Ako, Emmanuel and Langs, Georg and Gozaliasl, Ghassem and Yang, Guang and Prosch, Helmut and Preller, Jacobus and Stanczuk, Jan and Tang, Jing and Hofmanninger, Johannes and Babar, Judith and Sánchez, Lorena Escudero and Thillai, Muhunthan and Gonzalez, Paula Martin and Teare, Philip and Zhu, Xiao Xiang and Patel, Mishal and Cafolla, Conor and Azadbakht, Hojjat and Jacob, Joseph and Lowe, Josh and Zhang, Kang and Bradley, Kyle and Wassin, Marcel and Holzer, Markus and Ji, Kangyu and Ortet, Maria Delgado and Ai, Tao and Walton, Nicholas and Lio, Pietro and Stranks, Samuel and Shadbahr, Tolou and Lin, Weizhe and Zha, Yunfei and Niu, Zhangming and Rudd, James H. F. and Sala, Evis and Schönlieb, Carola-Bibiane (2021) Common Pitfalls and Recommendations for Using Machine Learning to Detect and Prognosticate for COVID-19 Using Chest Radiographs and CT Scans. Nature Machine Intelligence, 3, pp. 199-217. Springer Nature. doi: 10.1038/s42256-021-00307-0. ISSN 2522-5839.

PDF - Published version
1MB

Official URL: https://doi.org/10.1038/s42256-021-00307-0

Abstract

Machine learning methods offer great promise for fast and accurate detection and prognostication of coronavirus disease 2019 (COVID-19) from standard-of-care chest radiographs (CXR) and chest computed tomography (CT) images. Many articles have been published in 2020 describing new machine learning-based models for both of these tasks, but it is unclear which are of potential clinical utility. In this systematic review, we consider all published papers and preprints, for the period from 1 January 2020 to 3 October 2020, which describe new machine learning models for the diagnosis or prognosis of COVID-19 from CXR or CT images. All manuscripts uploaded to bioRxiv, medRxiv and arXiv along with all entries in EMBASE and MEDLINE in this timeframe are considered. Our search identified 2,212 studies, of which 415 were included after initial screening and, after quality screening, 62 studies were included in this systematic review. Our review finds that none of the models identified are of potential clinical use due to methodological flaws and/or underlying biases. This is a major weakness, given the urgency with which validated COVID-19 models are needed. To address this, we give many recommendations which, if followed, will solve these issues and lead to higher-quality model development and well-documented manuscripts.

Item URL in elib:

https://elib.dlr.de/146203/

Document Type:

Article

Title:

Common Pitfalls and Recommendations for Using Machine Learning to Detect and Prognosticate for COVID-19 Using Chest Radiographs and CT Scans

Authors:

Authors	Institution or Email of Authors	Author's ORCID iD	ORCID Put Code
Roberts, Michael	Department of Applied Mathematics and Theoretical Physics, University of Cambridge	UNSPECIFIED	UNSPECIFIED
Driggs, Derek	Department of Applied Mathematics and Theoretical Physics, University of Cambridge	UNSPECIFIED	UNSPECIFIED
Thorpe, Matthew	Department of Mathematics, University of Manchester	UNSPECIFIED	UNSPECIFIED
Gilbey, Julian	Department of Applied Mathematics and Theoretical Physics, University of Cambridge	UNSPECIFIED	UNSPECIFIED
Yeung, Michael	Department of Radiology, University of Cambridge	UNSPECIFIED	UNSPECIFIED
Ursprung, Stephan	Department of Radiology, University of Cambridge	UNSPECIFIED	UNSPECIFIED
Aviles-Rivero, Angelica I.	Department of Applied Mathematics and Theoretical Physics, University of Cambridge	UNSPECIFIED	UNSPECIFIED
Etmann, Christian	Department of Applied Mathematics and Theoretical Physics, University of Cambridge	UNSPECIFIED	UNSPECIFIED
McCague, Cathal	Department of Radiology, University of Cambridge	UNSPECIFIED	UNSPECIFIED
Beer, Lucian	Department of Radiology, University of Cambridge	UNSPECIFIED	UNSPECIFIED
Weir-McCall, Jonathan R.	Department of Radiology, University of Cambridge	UNSPECIFIED	UNSPECIFIED
Teng, Zhongzhao	Department of Radiology, University of Cambridge	UNSPECIFIED	UNSPECIFIED
Gkrania-Klotsas, Effrossyni	Department of Infectious Diseases, Cambridge University Hospitals NHS Trust	UNSPECIFIED	UNSPECIFIED
Ruggiero, Alessandro	Royal Papworth Hospital, Cambridge	UNSPECIFIED	UNSPECIFIED
Korhonen, Anna	Language Technology Laboratory, University of Cambridge	UNSPECIFIED	UNSPECIFIED
Jefferson, Emily	Population Health and Genomics, School of Medicine, University of Dundee	UNSPECIFIED	UNSPECIFIED
Ako, Emmanuel	Chelsea and Westminster NHS Trust and Royal Brompton NHS Hospital	UNSPECIFIED	UNSPECIFIED
Langs, Georg	Department of Biomedical Imaging and Image-guided Therapy, Computational Imaging Research Lab Medical University of Vienna	UNSPECIFIED	UNSPECIFIED
Gozaliasl, Ghassem	Department of Physics, University of Helsinki	UNSPECIFIED	UNSPECIFIED
Yang, Guang	National Heart and Lung Institute, Imperial College London	UNSPECIFIED	UNSPECIFIED
Prosch, Helmut	Department of Biomedical Imaging and Image-guided Therapy, Computational Imaging Research Lab Medical University of Vienna	UNSPECIFIED	UNSPECIFIED
Preller, Jacobus	Addenbrooke’s Hospital, Cambridge University Hospitals NHS Trust	UNSPECIFIED	UNSPECIFIED
Stanczuk, Jan	Department of Applied Mathematics and Theoretical Physics, University of Cambridge	UNSPECIFIED	UNSPECIFIED
Tang, Jing	Research Program in System Oncology, Faculty of Medicine, University of Helsinki	UNSPECIFIED	UNSPECIFIED
Hofmanninger, Johannes	Department of Biomedical Imaging and Image-guided Therapy, Computational Imaging Research Lab Medical University of Vienna	UNSPECIFIED	UNSPECIFIED
Babar, Judith	Addenbrooke’s Hospital, Cambridge University Hospitals NHS Trust	UNSPECIFIED	UNSPECIFIED
Sánchez, Lorena Escudero	Department of Radiology, University of Cambridge	UNSPECIFIED	UNSPECIFIED
Thillai, Muhunthan	Department of Medicine, University of Cambridge	UNSPECIFIED	UNSPECIFIED
Gonzalez, Paula Martin	Cancer Research UK Cambridge Centre, University of Cambridge	UNSPECIFIED	UNSPECIFIED
Teare, Philip	Biopharmaceuticals R&D, AstraZeneca	UNSPECIFIED	UNSPECIFIED
Zhu, Xiao Xiang	UNSPECIFIED	https://orcid.org/0000-0001-5530-3613	UNSPECIFIED
Patel, Mishal	Biopharmaceuticals R&D, AstraZeneca	UNSPECIFIED	UNSPECIFIED
Cafolla, Conor	Department of Chemistry, University of Cambridge	UNSPECIFIED	UNSPECIFIED
Azadbakht, Hojjat	AINOSTICS Ltd	UNSPECIFIED	UNSPECIFIED
Jacob, Joseph	Centre for Medical Image Computing, University College London	UNSPECIFIED	UNSPECIFIED
Lowe, Josh	SparkBeyond UK Ltd	UNSPECIFIED	UNSPECIFIED
Zhang, Kang	Center for Biomedicine and Innovations at Faculty of Medicine, Macau	UNSPECIFIED	UNSPECIFIED
Bradley, Kyle	SparkBeyond UK Ltd	UNSPECIFIED	UNSPECIFIED
Wassin, Marcel	contextflow GmbH	UNSPECIFIED	UNSPECIFIED
Holzer, Markus	contextflow GmbH	UNSPECIFIED	UNSPECIFIED
Ji, Kangyu	Cavendish Laboratory, University of Cambridge	UNSPECIFIED	UNSPECIFIED
Ortet, Maria Delgado	Department of Radiology, University of Cambridge	UNSPECIFIED	UNSPECIFIED
Ai, Tao	Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology	UNSPECIFIED	UNSPECIFIED
Walton, Nicholas	Institute of Astronomy, University of Cambridge	UNSPECIFIED	UNSPECIFIED
Lio, Pietro	Department of Computer Science and Technology, University of Cambridge	UNSPECIFIED	UNSPECIFIED
Stranks, Samuel	Department of Chemical Engineering and Biotechnology, University of Cambridge	UNSPECIFIED	UNSPECIFIED
Shadbahr, Tolou	Research Program in System Oncology, Faculty of Medicine, University of Helsinki	UNSPECIFIED	UNSPECIFIED
Lin, Weizhe	Department of Engineering, University of Cambridge	UNSPECIFIED	UNSPECIFIED
Zha, Yunfei	Department of Radiology, Renmin Hospital of Wuhan University	UNSPECIFIED	UNSPECIFIED
Niu, Zhangming	Aladdin Healthcare Technologies Ltd	UNSPECIFIED	UNSPECIFIED
Rudd, James H. F.	Department of Medicine, University of Cambridge	UNSPECIFIED	UNSPECIFIED
Sala, Evis	Department of Radiology, University of Cambridge	UNSPECIFIED	UNSPECIFIED
Schönlieb, Carola-Bibiane	Department of Applied Mathematics and Theoretical Physics, University of Cambridge	UNSPECIFIED	UNSPECIFIED

Date:

15 March 2021

Journal or Publication Title:

Nature Machine Intelligence

Refereed publication:

Yes

Open Access:

Yes

Gold Open Access:

In SCOPUS:

Yes

In ISI Web of Science:

Yes

Volume:

DOI:

10.1038/s42256-021-00307-0

Page Range:

pp. 199-217

Publisher:

Springer Nature

ISSN:

2522-5839

Status:

Published

Keywords:

Computational science Diagnostic markers Prognostic markers SARS-CoV-2

HGF - Research field:

Aeronautics, Space and Transport

HGF - Program:

Space

HGF - Program Themes:

Earth Observation

DLR - Research area:

Raumfahrt

DLR - Program:

R EO - Earth Observation

DLR - Research theme (Project):

R - Artificial Intelligence

Location:

Oberpfaffenhofen

Institutes and Institutions:

Remote Sensing Technology Institute > EO Data Science

Deposited By:

Rösel, Dr. Anja

Deposited On:

26 Nov 2021 09:25

Last Modified:

05 Dec 2023 07:45

Repository Staff Only: item control page