A Radiologist's Perspective of Medical Annotations for AI Programs: The Entire Journey from Its Planning to Execution, Challenges Faced

 

 Permissions and Reprints

Abstract

Artificial intelligence (AI) in radiology and medical science is finding increasing applications with annotations being an integral part of AI development. While annotation may be perceived as passive work of labeling a certain anatomy, the radiologist plays a more important role in this task apart from marking the structures needed. Apart from annotation, more important aspect of their role is planning the anatomies/pathologies needed, type of annotations to be done, choice of the annotation tool, training the annotators, planning the duration of annotation, etc. A close interaction with the technical team is a key factor in the success of the annotations. The quality check of both the internally and externally annotated data, creating a team of good annotators, training them, and periodically reviewing the quality of data become an integral part of their work. Documentation related to the annotation work is another important area where the clinician plays an integral role to comply with the Food and Drug Administration requirements, focused on a clinically explainable and validated AI algorithms. Thus, the clinician becomes an integral part in the ideation, design, implementation/execution of annotations, and its quality control. This article summarizes the experiences gained during planning and executing the annotations for multiple annotation projects involving various imaging modalities with different pathologies.

Publication History

Article published online:
11 December 2024

© 2024. Indian Radiological Association. This is an open access article published by Thieme under the terms of the Creative Commons Attribution-NonDerivative-NonCommercial License, permitting copying and reproduction so long as the original work is given appropriate credit. Contents may not be used for commercial purposes, or adapted, remixed, transformed or built upon. (https://creativecommons.org/licenses/by-nc-nd/4.0/)

Thieme Medical and Scientific Publishers Pvt. Ltd.
A-12, 2nd Floor, Sector 2, Noida-201301 UP, India

• References

• 1 Willemink MJ, Koszek WA, Hardell C. et al. Preparing medical imaging data for machine learning. Radiology 2020; 295 (01) 4-15
  CrossrefPubMedSearch in Google Scholar
• 2 Aljabri M, AlAmir M, AlGhamdi M, Abdel-Mottaleb M, Collado-Mesa F. Towards a better understanding of annotation tools for medical imaging: a survey. Multimedia Tools Appl 2022; 81 (18) 25877-25911
  CrossrefPubMedSearch in Google Scholar
• 3 Larson DB. Openness and transparency in the evaluation of bias in artificial intelligence. Radiology 2023; 306 (02) e222263
  CrossrefPubMedSearch in Google Scholar
• 4 Rouzrokh P, Khosravi B, Faghani S. et al. Mitigating bias in radiology machine learning: 1. Data handling. Radiol Artif Intell 2022; 4 (05) e210290
  CrossrefPubMedSearch in Google Scholar
• 5 European Society of Radiology (ESR). What the radiologist should know about artificial intelligence: an ESR white paper. Insights Imaging 2019; 10 (01) 44
  CrossrefPubMedSearch in Google Scholar
• 6 Mongan J, Halabi SS. On the centrality of data: data resources in radiologic artificial intelligence. Radiol Artif Intell 2023; 5 (05) e230231
  CrossrefPubMedSearch in Google Scholar
• 7 Galbusera F, Cina A. Image annotation and curation in radiology: an overview for machine learning practitioners. Eur Radiol Exp 2024; 8 (01) 1
  CrossrefPubMedSearch in Google Scholar
• 8 FDA. Clinical Data for Premarket Submissions. Accessed November 19, 2024 at: https://www.fda.gov/medical-devices/premarket-submissions-selecting-and-preparing-correct-submission/clinical-data-premarket-submissions
• 9 Yang Y, Li R, Xiang Y. et al. Expert recommendation on collection, storage, annotation, and management of data related to medical artificial intelligence. Intell Med 2023; 3 (02) 144-149
  CrossrefSearch in Google Scholar
• 10 Wang S, Li C, Wang R. et al. Annotation-efficient deep learning for automatic medical image segmentation. Nat Commun 2021; 12 (01) 5915
  CrossrefPubMedSearch in Google Scholar
• 11 Ma J, He Y, Li F, Han L, You C, Wang B. Segment anything in medical images. Nat Commun 2024; 15 (01) 654
  CrossrefPubMedSearch in Google Scholar
• 12 Renard F, Guedria S, Palma N, Vuillerme N. Variability and reproducibility in deep learning for medical image segmentation. Sci Rep 2020; 10 (01) 13724
  CrossrefPubMedSearch in Google Scholar