TY - GEN
T1 - Rethinking Learning-based Demosaicing, Denoising, and Super-Resolution Pipeline
AU - Qian, Guocheng
AU - Wang, Yuanhao
AU - Gu, Jinjin
AU - Dong, Chao
AU - Heidrich, Wolfgang
AU - Ghanem, Bernard
AU - Ren, Jimmy S.
N1 - KAUST Repository Item: Exported on 2022-12-14
Acknowledgements: The authors thank the reviewers of ICCP 2022 for valuable suggestions and Dr. Silvio Giancola for proofreading the rebuttals. This work was supported by the KAUST Office of Sponsored Research (OSR) through the Visual Computing Center (VCC) funding.
PY - 2022/9/26
Y1 - 2022/9/26
N2 - Imaging is usually a mixture problem of incomplete color sampling, noise degradation, and limited resolution. This mixture problem is typically solved by a sequential solution that applies demosaicing (DM), denoising (DN), and super-resolution (SR) sequentially in a fixed and predefined pipeline (execution order of tasks), DM→DN→SR. The most recent work on image processing focuses on developing more sophisticated architectures to achieve higher image quality. Little attention has been paid to the design of the pipeline, and it is still not clear how significant the pipeline is to image quality. In this work, we comprehensively study the effects of pipelines on the mixture problem of learning-based DN, DM, and SR, in both sequential and joint solutions. On the one hand, in sequential solutions, we find that the pipeline has a non-trivial effect on the resulted image quality. Our suggested pipeline DN→SR→DM yields consistently better performance than other sequential pipelines in various experimental settings and benchmarks. On the other hand, in joint solutions, we propose an end-to-end Trinity Pixel Enhancement NETwork (TENet) that achieves the state-of-the-art performance for the mixture problem. We further present a novel and simple method that can integrate a certain pipeline into a given end-to-end network by providing intermediate supervision using a detachable head. Extensive experiments show that an end-to-end network with the proposed pipeline can attain only a consistent but insignificant improvement. Our work indicates that the investigation of pipelines is applicable in sequential solutions, but is not very necessary in end-to-end networks.
AB - Imaging is usually a mixture problem of incomplete color sampling, noise degradation, and limited resolution. This mixture problem is typically solved by a sequential solution that applies demosaicing (DM), denoising (DN), and super-resolution (SR) sequentially in a fixed and predefined pipeline (execution order of tasks), DM→DN→SR. The most recent work on image processing focuses on developing more sophisticated architectures to achieve higher image quality. Little attention has been paid to the design of the pipeline, and it is still not clear how significant the pipeline is to image quality. In this work, we comprehensively study the effects of pipelines on the mixture problem of learning-based DN, DM, and SR, in both sequential and joint solutions. On the one hand, in sequential solutions, we find that the pipeline has a non-trivial effect on the resulted image quality. Our suggested pipeline DN→SR→DM yields consistently better performance than other sequential pipelines in various experimental settings and benchmarks. On the other hand, in joint solutions, we propose an end-to-end Trinity Pixel Enhancement NETwork (TENet) that achieves the state-of-the-art performance for the mixture problem. We further present a novel and simple method that can integrate a certain pipeline into a given end-to-end network by providing intermediate supervision using a detachable head. Extensive experiments show that an end-to-end network with the proposed pipeline can attain only a consistent but insignificant improvement. Our work indicates that the investigation of pipelines is applicable in sequential solutions, but is not very necessary in end-to-end networks.
UR - http://hdl.handle.net/10754/681702
UR - https://ieeexplore.ieee.org/document/9887682/
UR - http://www.scopus.com/inward/record.url?scp=85141054966&partnerID=8YFLogxK
U2 - 10.1109/ICCP54855.2022.9887682
DO - 10.1109/ICCP54855.2022.9887682
M3 - Conference contribution
SN - 978-1-6654-5852-8
BT - 2022 IEEE International Conference on Computational Photography (ICCP)
PB - IEEE
ER -