Pixels to Graphs by Associative Embedding

Alejandro Newell; Jia Deng

Pixels to Graphs by Associative Embedding

Alejandro Newell, Jia Deng

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Graphs are a useful abstraction of image content. Not only can graphs represent details about individual objects in a scene but they can capture the interactions between pairs of objects. We present a method for training a convolutional neural network such that it takes in an input image and produces a full graph definition. This is done end-to-end in a single stage with the use of associative embeddings. The network learns to simultaneously identify all of the elements that make up a graph and piece them together. We benchmark on the Visual Genome dataset, and demonstrate state-of-the-art performance on the challenging task of scene graph generation.

Original language	English (US)
Title of host publication	31st Annual Conference on Neural Information Processing Systems (NIPS)
Publisher	NEURAL INFORMATION PROCESSING SYSTEMS (NIPS)
State	Published - 2017
Externally published	Yes

Cite this

@inproceedings{6634c83fbdb3495581c83595dc0642fb,

title = "Pixels to Graphs by Associative Embedding",

abstract = "Graphs are a useful abstraction of image content. Not only can graphs represent details about individual objects in a scene but they can capture the interactions between pairs of objects. We present a method for training a convolutional neural network such that it takes in an input image and produces a full graph definition. This is done end-to-end in a single stage with the use of associative embeddings. The network learns to simultaneously identify all of the elements that make up a graph and piece them together. We benchmark on the Visual Genome dataset, and demonstrate state-of-the-art performance on the challenging task of scene graph generation.",

author = "Alejandro Newell and Jia Deng",

note = "KAUST Repository Item: Exported on 2022-06-27 Acknowledged KAUST grant number(s): OSR-2015-CRG4-2639 Acknowledgements: This publication is based upon work supported by the King Abdullah University of Science and Technology (KAUST) Office of Sponsored Research (OSR) under Award No. OSR-2015-CRG4-2639. This publication acknowledges KAUST support, but has no KAUST affiliated authors.",

year = "2017",

language = "English (US)",

booktitle = "31st Annual Conference on Neural Information Processing Systems (NIPS)",

publisher = "NEURAL INFORMATION PROCESSING SYSTEMS (NIPS)",

}

TY - GEN

T1 - Pixels to Graphs by Associative Embedding

AU - Newell, Alejandro

AU - Deng, Jia

N1 - KAUST Repository Item: Exported on 2022-06-27 Acknowledged KAUST grant number(s): OSR-2015-CRG4-2639 Acknowledgements: This publication is based upon work supported by the King Abdullah University of Science and Technology (KAUST) Office of Sponsored Research (OSR) under Award No. OSR-2015-CRG4-2639. This publication acknowledges KAUST support, but has no KAUST affiliated authors.

PY - 2017

Y1 - 2017

N2 - Graphs are a useful abstraction of image content. Not only can graphs represent details about individual objects in a scene but they can capture the interactions between pairs of objects. We present a method for training a convolutional neural network such that it takes in an input image and produces a full graph definition. This is done end-to-end in a single stage with the use of associative embeddings. The network learns to simultaneously identify all of the elements that make up a graph and piece them together. We benchmark on the Visual Genome dataset, and demonstrate state-of-the-art performance on the challenging task of scene graph generation.

AB - Graphs are a useful abstraction of image content. Not only can graphs represent details about individual objects in a scene but they can capture the interactions between pairs of objects. We present a method for training a convolutional neural network such that it takes in an input image and produces a full graph definition. This is done end-to-end in a single stage with the use of associative embeddings. The network learns to simultaneously identify all of the elements that make up a graph and piece them together. We benchmark on the Visual Genome dataset, and demonstrate state-of-the-art performance on the challenging task of scene graph generation.

UR - http://hdl.handle.net/10754/626720

UR - http://arxiv.org/abs/1706.07365v1

M3 - Conference contribution

BT - 31st Annual Conference on Neural Information Processing Systems (NIPS)

PB - NEURAL INFORMATION PROCESSING SYSTEMS (NIPS)

ER -

Pixels to Graphs by Associative Embedding

Abstract

Other files and links

Fingerprint

Cite this