GPR-Net: Multi-view Layout Estimation via a Geometry-aware Panorama Registration Network

Jheng Wei Su*, Chi Han Peng, Peter Wonka, Hung Kuo Chu

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

1 Scopus citations

Abstract

We present a room layout estimation framework that jointly learns wide baseline panorama registration and layout estimation given a pair of 360° panoramas. To effectively tackle the wide baseline registration problem, we introduce a novel end-to-end supervised Geometry-aware Panorama Registration Network or GPR-Net that exploits the layout geometry and computes fine-grained correspondences on the layout boundary, instead of the global pixel-space. GPR-Net consists of two main parts. The geometry transformer learns a set of 1D horizon features sampled on the panorama. These 1D feature maps encode geometric cues describing the ceiling-wall and floor-wall layout boundaries, and the correspondence and co-visibility between layout boundaries. These learned geometric cues are further used for direct regression of relative pose (translation and rotation) with a pose transformer. The final layout is then obtained by registering the two layouts using the estimated pose and taking the union of the two individual layouts derived from the estimated layout boundary maps. Experimental results indicate that our method achieves state-of-the-art performance in both panorama registration and layout estimation on a large-scale indoor panorama dataset ZInD [3]. Our code is available online1.

Original languageEnglish (US)
Title of host publicationProceedings - 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2023
PublisherIEEE Computer Society
Pages6469-6478
Number of pages10
ISBN (Electronic)9798350302493
DOIs
StatePublished - 2023
Event2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2023 - Vancouver, Canada
Duration: Jun 18 2023Jun 22 2023

Publication series

NameIEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops
Volume2023-June
ISSN (Print)2160-7508
ISSN (Electronic)2160-7516

Conference

Conference2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2023
Country/TerritoryCanada
CityVancouver
Period06/18/2306/22/23

ASJC Scopus subject areas

  • Computer Vision and Pattern Recognition
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'GPR-Net: Multi-view Layout Estimation via a Geometry-aware Panorama Registration Network'. Together they form a unique fingerprint.

Cite this