The importance of multi-modal imaging and clinical information for humans and AI-based algorithms to classify breast masses (INSPiRED 003): an international, multicenter analysis
Title
The importance of multi-modal imaging and clinical information for humans and AI-based algorithms to classify breast masses (INSPiRED 003): an international, multicenter analysis
Creator
André Pfob
Chris Sidey-Gibbons
Richard G Barr
Volker Duda
Zaher Alwafai
Corinne Balleyguier
Dirk-André Clevert
Sarah Fastner
Christina Gomez
Manuela Goncalo
Ines Gruber
Markus Hahn
André Hennigs
Panagiotis Kapetas
Sheng-Chieh Lu
Juliane Nees
Ralf Ohlinger
Fabian Riedel
Matthieu Rutten
Benedikt Schaefgen
Maximilian Schuessler
Anne Stieber
Riku Togawa
Mitsuhiro Tozaki
Sebastian Wojcinski
Cai Xu
Geraldine Rauch
Joerg Heil
Michael Golatta
Date
2022
Description
Objectives: AI-based algorithms for medical image analysis showed comparable performance to human image readers. However, in practice, diagnoses are made using multiple imaging modalities alongside other data sources. We determined the importance of this multi-modal information and compared the diagnostic performance of routine breast cancer diagnosis to breast ultrasound interpretations by humans or AI-based algorithms.
Methods: Patients were recruited as part of a multicenter trial (NCT02638935). The trial enrolled 1288 women undergoing routine breast cancer diagnosis (multi-modal imaging, demographic, and clinical information). Three physicians specialized in ultrasound diagnosis performed a second read of all ultrasound images. We used data from 11 of 12 study sites to develop two machine learning (ML) algorithms using unimodal information (ultrasound features generated by the ultrasound experts) to classify breast masses which were validated on the remaining study site. The same ML algorithms were subsequently developed and validated on multi-modal information (clinical and demographic information plus ultrasound features). We assessed performance using area under the curve (AUC).
Results: Of 1288 breast masses, 368 (28.6%) were histopathologically malignant. In the external validation set (n = 373), the performance of the two unimodal ultrasound ML algorithms (AUC 0.83 and 0.82) was commensurate with performance of the human ultrasound experts (AUC 0.82 to 0.84; p for all comparisons > 0.05). The multi-modal ultrasound ML algorithms performed significantly better (AUC 0.90 and 0.89) but were statistically inferior to routine breast cancer diagnosis (AUC 0.95, p for all comparisons ≤ 0.05).
Conclusions: The performance of humans and AI-based algorithms improves with multi-modal information.
Key points: • The performance of humans and AI-based algorithms improves with multi-modal information. • Multimodal AI-based algorithms do not necessarily outperform expert humans. • Unimodal AI-based algorithms do not represent optimal performance to classify breast masses.
Methods: Patients were recruited as part of a multicenter trial (NCT02638935). The trial enrolled 1288 women undergoing routine breast cancer diagnosis (multi-modal imaging, demographic, and clinical information). Three physicians specialized in ultrasound diagnosis performed a second read of all ultrasound images. We used data from 11 of 12 study sites to develop two machine learning (ML) algorithms using unimodal information (ultrasound features generated by the ultrasound experts) to classify breast masses which were validated on the remaining study site. The same ML algorithms were subsequently developed and validated on multi-modal information (clinical and demographic information plus ultrasound features). We assessed performance using area under the curve (AUC).
Results: Of 1288 breast masses, 368 (28.6%) were histopathologically malignant. In the external validation set (n = 373), the performance of the two unimodal ultrasound ML algorithms (AUC 0.83 and 0.82) was commensurate with performance of the human ultrasound experts (AUC 0.82 to 0.84; p for all comparisons > 0.05). The multi-modal ultrasound ML algorithms performed significantly better (AUC 0.90 and 0.89) but were statistically inferior to routine breast cancer diagnosis (AUC 0.95, p for all comparisons ≤ 0.05).
Conclusions: The performance of humans and AI-based algorithms improves with multi-modal information.
Key points: • The performance of humans and AI-based algorithms improves with multi-modal information. • Multimodal AI-based algorithms do not necessarily outperform expert humans. • Unimodal AI-based algorithms do not represent optimal performance to classify breast masses.
Source
Eur Radiol
. 2022 Jun;32(6):4101-4115. doi: 10.1007/s00330-021-08519-z. Epub 2022 Feb 17.
. 2022 Jun;32(6):4101-4115. doi: 10.1007/s00330-021-08519-z. Epub 2022 Feb 17.
Language
English
URL
https://doi.org/10.1007/s00330-021-08519-z
Citation
André Pfob et al., “The importance of multi-modal imaging and clinical information for humans and AI-based algorithms to classify breast masses (INSPiRED 003): an international, multicenter analysis,” NEOMED Bibliography Database, accessed April 29, 2024, https://neomed.omeka.net/items/show/12199.