Explainable multiple abnormality classification of chest CT volumes

Rachel Lea Draelos, Lawrence Carin

Research output: Contribution to journalArticlepeer-review

10 Scopus citations


Understanding model predictions is critical in healthcare, to facilitate rapid verification of model correctness and to guard against use of models that exploit confounding variables. We introduce the challenging new task of explainable multiple abnormality classification in volumetric medical images, in which a model must indicate the regions used to predict each abnormality. To solve this task, we propose a multiple instance learning convolutional neural network, AxialNet, that allows identification of top slices for each abnormality. Next we incorporate HiResCAM, an attention mechanism, to identify sub-slice regions. We prove that for AxialNet, HiResCAM explanations are guaranteed to reflect the locations the model used, unlike Grad-CAM which sometimes highlights irrelevant locations. Armed with a model that produces faithful explanations, we then aim to improve the model's learning through a novel mask loss that leverages HiResCAM and 3D allowed regions to encourage the model to predict abnormalities based only on the organs in which those abnormalities appear. The 3D allowed regions are obtained automatically through a new approach, PARTITION, that combines location information extracted from radiology reports with organ segmentation maps obtained through morphological image processing. Overall, we propose the first model for explainable multi-abnormality prediction in volumetric medical images, and then use the mask loss to achieve a 33% improvement in organ localization of multiple abnormalities in the RAD-ChestCT dataset of 36,316 scans, representing the state of the art. This work advances the clinical applicability of multiple abnormality modeling in chest CT volumes.
Original languageEnglish (US)
Pages (from-to)102372
JournalArtificial Intelligence in Medicine
StatePublished - Aug 24 2022
Externally publishedYes

ASJC Scopus subject areas

  • Artificial Intelligence
  • Medicine (miscellaneous)


Dive into the research topics of 'Explainable multiple abnormality classification of chest CT volumes'. Together they form a unique fingerprint.

Cite this