Finding Nano-Ötzi: Cryo-Electron Tomography Visualization Guided by Learned Segmentation

Ngan Nguyen, Ciril Bohak, Dominik Engel, Peter Mindek, Ondrej Strnad, Peter Wonka, Sai Li, Timo Ropinski, Ivan Viola

Research output: Contribution to journalArticlepeer-review

4 Scopus citations

Abstract

Cryo-electron tomography (cryo-ET) is a new 3D imaging technique with unprecedented potential for resolving submicron structural details. Existing volume visualization methods, however, are not able to reveal details of interest due to low signal-to-noise ratio. In order to design more powerful transfer functions, we propose leveraging soft segmentation as an explicit component of visualization for noisy volumes. Our technical realization is based on semi-supervised learning, where we combine the advantages of two segmentation algorithms. First, the weak segmentation algorithm provides good results for propagating sparse user-provided labels to other voxels in the same volume and is used to generate dense pseudo-labels. Second, the powerful deep-learning-based segmentation algorithm learns from these pseudo-labels to generalize the segmentation to other unseen volumes, a task that the weak segmentation algorithm fails at completely. The proposed volume visualization uses deep-learning-based segmentation as a component for segmentation-aware transfer function design. Appropriate ramp parameters can be suggested automatically through frequency distribution analysis. Furthermore, our visualization uses gradient-free ambient occlusion shading to further suppress the visual presence of noise, and to give structural detail the desired prominence. The cryo-ET data studied in our technical experiments are based on the highest-quality tilted series of intact SARS-CoV-2 virions. Our technique shows the high impact in target sciences for visual data analysis of very noisy volumes that cannot be visualized with existing techniques.

Original languageEnglish (US)
Pages (from-to)1-18
Number of pages18
JournalIEEE Transactions on Visualization and Computer Graphics
Volume29
Issue number10
DOIs
StateAccepted/In press - 2022

Keywords

  • Computer Graphics Techniques
  • Data visualization
  • Image segmentation
  • Life Sciences
  • Machine Learning Techniques
  • Noise measurement
  • Scalar Field Data
  • Signal to noise ratio
  • Task analysis
  • Three-dimensional displays
  • Visualization
  • Volume Rendering

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Computer Vision and Pattern Recognition
  • Computer Graphics and Computer-Aided Design

Fingerprint

Dive into the research topics of 'Finding Nano-Ötzi: Cryo-Electron Tomography Visualization Guided by Learned Segmentation'. Together they form a unique fingerprint.

Cite this