When NAS Meets Trees: An Efficient Algorithm for Neural Architecture Search

Guocheng Qian; Xuanyang Zhang; Guohao Li; Chen Zhao; Yukang Chen; Xiangyu Zhang; Bernard Ghanem; Jian Sun

doi:10.1109/CVPRW56347.2022.00314

When NAS Meets Trees: An Efficient Algorithm for Neural Architecture Search

Guocheng Qian, Xuanyang Zhang, Guohao Li, Chen Zhao, Yukang Chen, Xiangyu Zhang, Bernard Ghanem, Jian Sun

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

2 Scopus citations

Abstract

The key challenge in neural architecture search (NAS) is designing how to explore wisely in the huge search space. We propose a new NAS method called TNAS (NAS with trees), which improves search efficiency by exploring only a small number of architectures while also achieving a higher search accuracy. TNAS introduces an architecture tree and a binary operation tree, to factorize the search space and substantially reduce the exploration size. TNAS performs a modified bi-level Breadth-First Search in the proposed trees to discover a high-performance architecture. Impressively, TNAS finds the global optimal architecture on CIFAR-10 with test accuracy of 94.37% in four GPU hours in NAS-Bench-201. The average test accuracy is 94.35%, which outperforms the state-of-the-art.

Original language	English (US)
Title of host publication	2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)
Publisher	IEEE
ISBN (Print)	978-1-6654-8740-5
DOIs	https://doi.org/10.1109/CVPRW56347.2022.00314
State	Published - Aug 23 2022

Access to Document

10.1109/CVPRW56347.2022.00314

https://repository.kaust.edu.sa/bitstream/10754/676640/5/CVPR2022W_TNAS.pdf

Cite this

@inproceedings{981d23937eff4f6cb7f46dc9cd662295,

title = "When NAS Meets Trees: An Efficient Algorithm for Neural Architecture Search",

abstract = "The key challenge in neural architecture search (NAS) is designing how to explore wisely in the huge search space. We propose a new NAS method called TNAS (NAS with trees), which improves search efficiency by exploring only a small number of architectures while also achieving a higher search accuracy. TNAS introduces an architecture tree and a binary operation tree, to factorize the search space and substantially reduce the exploration size. TNAS performs a modified bi-level Breadth-First Search in the proposed trees to discover a high-performance architecture. Impressively, TNAS finds the global optimal architecture on CIFAR-10 with test accuracy of 94.37% in four GPU hours in NAS-Bench-201. The average test accuracy is 94.35%, which outperforms the state-of-the-art.",

author = "Guocheng Qian and Xuanyang Zhang and Guohao Li and Chen Zhao and Yukang Chen and Xiangyu Zhang and Bernard Ghanem and Jian Sun",

note = "KAUST Repository Item: Exported on 2022-09-14 Acknowledgements: This work was done when Guocheng was remotely interned at Megvii technology. This work was also supported by the KAUST Office of Sponsored Research (OSR) through VCC funding.",

year = "2022",

month = aug,

day = "23",

doi = "10.1109/CVPRW56347.2022.00314",

language = "English (US)",

isbn = "978-1-6654-8740-5",

booktitle = "2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)",

publisher = "IEEE",

}

TY - GEN

T1 - When NAS Meets Trees: An Efficient Algorithm for Neural Architecture Search

AU - Qian, Guocheng

AU - Zhang, Xuanyang

AU - Li, Guohao

AU - Zhao, Chen

AU - Chen, Yukang

AU - Zhang, Xiangyu

AU - Ghanem, Bernard

AU - Sun, Jian

N1 - KAUST Repository Item: Exported on 2022-09-14 Acknowledgements: This work was done when Guocheng was remotely interned at Megvii technology. This work was also supported by the KAUST Office of Sponsored Research (OSR) through VCC funding.

PY - 2022/8/23

Y1 - 2022/8/23

N2 - The key challenge in neural architecture search (NAS) is designing how to explore wisely in the huge search space. We propose a new NAS method called TNAS (NAS with trees), which improves search efficiency by exploring only a small number of architectures while also achieving a higher search accuracy. TNAS introduces an architecture tree and a binary operation tree, to factorize the search space and substantially reduce the exploration size. TNAS performs a modified bi-level Breadth-First Search in the proposed trees to discover a high-performance architecture. Impressively, TNAS finds the global optimal architecture on CIFAR-10 with test accuracy of 94.37% in four GPU hours in NAS-Bench-201. The average test accuracy is 94.35%, which outperforms the state-of-the-art.

AB - The key challenge in neural architecture search (NAS) is designing how to explore wisely in the huge search space. We propose a new NAS method called TNAS (NAS with trees), which improves search efficiency by exploring only a small number of architectures while also achieving a higher search accuracy. TNAS introduces an architecture tree and a binary operation tree, to factorize the search space and substantially reduce the exploration size. TNAS performs a modified bi-level Breadth-First Search in the proposed trees to discover a high-performance architecture. Impressively, TNAS finds the global optimal architecture on CIFAR-10 with test accuracy of 94.37% in four GPU hours in NAS-Bench-201. The average test accuracy is 94.35%, which outperforms the state-of-the-art.

UR - http://hdl.handle.net/10754/676640

UR - https://ieeexplore.ieee.org/document/9857078/

U2 - 10.1109/CVPRW56347.2022.00314

DO - 10.1109/CVPRW56347.2022.00314

M3 - Conference contribution

SN - 978-1-6654-8740-5

BT - 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)

PB - IEEE

ER -

When NAS Meets Trees: An Efficient Algorithm for Neural Architecture Search

Abstract

Access to Document

Other files and links

Fingerprint

Cite this