TY - JOUR
T1 - Explainable multiple abnormality classification of chest CT volumes
AU - Draelos, Rachel Lea
AU - Carin, Lawrence
N1 - KAUST Repository Item: Exported on 2022-09-09
Acknowledgements: The authors would like to thank the Duke Protected Analytics Computing Environment (PACE), particularly Mike Newton and Charley Kneifel, Ph.D. for providing the computing resources and GPUs needed to complete this work. The authors also thank Paidamoyo Chapfuwa, Ph.D. for thoughtful comments on a previous version of the manuscript, David Dov, Ph.D. for discussion of multiple instance learning, and Geoffrey D. Rubin, MD, FACR, for helpful remarks on the explanations. This work was supported in part by the National Institutes of Health (NIH) Duke Medical Scientist Training Program Training Grant, United States of America (GM-007171).
PY - 2022/8/24
Y1 - 2022/8/24
N2 - Understanding model predictions is critical in healthcare, to facilitate rapid verification of model correctness and to guard against use of models that exploit confounding variables. We introduce the challenging new task of explainable multiple abnormality classification in volumetric medical images, in which a model must indicate the regions used to predict each abnormality. To solve this task, we propose a multiple instance learning convolutional neural network, AxialNet, that allows identification of top slices for each abnormality. Next we incorporate HiResCAM, an attention mechanism, to identify sub-slice regions. We prove that for AxialNet, HiResCAM explanations are guaranteed to reflect the locations the model used, unlike Grad-CAM which sometimes highlights irrelevant locations. Armed with a model that produces faithful explanations, we then aim to improve the model's learning through a novel mask loss that leverages HiResCAM and 3D allowed regions to encourage the model to predict abnormalities based only on the organs in which those abnormalities appear. The 3D allowed regions are obtained automatically through a new approach, PARTITION, that combines location information extracted from radiology reports with organ segmentation maps obtained through morphological image processing. Overall, we propose the first model for explainable multi-abnormality prediction in volumetric medical images, and then use the mask loss to achieve a 33% improvement in organ localization of multiple abnormalities in the RAD-ChestCT dataset of 36,316 scans, representing the state of the art. This work advances the clinical applicability of multiple abnormality modeling in chest CT volumes.
AB - Understanding model predictions is critical in healthcare, to facilitate rapid verification of model correctness and to guard against use of models that exploit confounding variables. We introduce the challenging new task of explainable multiple abnormality classification in volumetric medical images, in which a model must indicate the regions used to predict each abnormality. To solve this task, we propose a multiple instance learning convolutional neural network, AxialNet, that allows identification of top slices for each abnormality. Next we incorporate HiResCAM, an attention mechanism, to identify sub-slice regions. We prove that for AxialNet, HiResCAM explanations are guaranteed to reflect the locations the model used, unlike Grad-CAM which sometimes highlights irrelevant locations. Armed with a model that produces faithful explanations, we then aim to improve the model's learning through a novel mask loss that leverages HiResCAM and 3D allowed regions to encourage the model to predict abnormalities based only on the organs in which those abnormalities appear. The 3D allowed regions are obtained automatically through a new approach, PARTITION, that combines location information extracted from radiology reports with organ segmentation maps obtained through morphological image processing. Overall, we propose the first model for explainable multi-abnormality prediction in volumetric medical images, and then use the mask loss to achieve a 33% improvement in organ localization of multiple abnormalities in the RAD-ChestCT dataset of 36,316 scans, representing the state of the art. This work advances the clinical applicability of multiple abnormality modeling in chest CT volumes.
UR - http://hdl.handle.net/10754/673946
UR - https://linkinghub.elsevier.com/retrieve/pii/S0933365722001312
UR - http://www.scopus.com/inward/record.url?scp=85136507756&partnerID=8YFLogxK
U2 - 10.1016/j.artmed.2022.102372
DO - 10.1016/j.artmed.2022.102372
M3 - Article
C2 - 36207074
SN - 0933-3657
VL - 132
SP - 102372
JO - Artificial Intelligence in Medicine
JF - Artificial Intelligence in Medicine
ER -