TY - GEN
T1 - Deep Context-Encoding Network For Retinal Image Captioning
AU - Huang, Jia-Hong
AU - Wu, Ting-Wei
AU - Yang, Chao-Han Huck
AU - Worring, Marcel
N1 - KAUST Repository Item: Exported on 2021-08-28
Acknowledgements: This work is supported by competitive research funding from King Abdullah University of Science and Technology (KAUST) and University of Amsterdam.
This publication acknowledges KAUST support, but has no KAUST affiliated authors.
PY - 2021/8/23
Y1 - 2021/8/23
N2 - Automatically generating medical reports for retinal images is one of the promising ways to help ophthalmologists reduce
their workload and improve work efficiency. In this work, we propose a new context-driven encoding network to automatically generate medical reports for retinal images. The proposed model is mainly composed of a multi-modal input encoder and a fused-feature decoder. Our experimental results show that our proposed method is capable of effectively leveraging the interactive information between the input image and context, i.e., keywords in our case. The proposed method creates more accurate and meaningful reports for retinal images than baseline models and achieves state-of-the-art performance. This performance is shown in several commonly used metrics for the medical report generation task: BLEUavg (+16%), CIDEr (+10.2%), and ROUGE (+8.6%).
AB - Automatically generating medical reports for retinal images is one of the promising ways to help ophthalmologists reduce
their workload and improve work efficiency. In this work, we propose a new context-driven encoding network to automatically generate medical reports for retinal images. The proposed model is mainly composed of a multi-modal input encoder and a fused-feature decoder. Our experimental results show that our proposed method is capable of effectively leveraging the interactive information between the input image and context, i.e., keywords in our case. The proposed method creates more accurate and meaningful reports for retinal images than baseline models and achieves state-of-the-art performance. This performance is shown in several commonly used metrics for the medical report generation task: BLEUavg (+16%), CIDEr (+10.2%), and ROUGE (+8.6%).
UR - http://hdl.handle.net/10754/670747
UR - https://ieeexplore.ieee.org/document/9506803/
U2 - 10.1109/icip42928.2021.9506803
DO - 10.1109/icip42928.2021.9506803
M3 - Conference contribution
BT - 2021 IEEE International Conference on Image Processing (ICIP)
PB - IEEE
ER -