Pushing the Efficiency Limit Using Structured Sparse Convolutions

Vinay Kumar Verma, Nikhil Mehta, Shijing Si, Ricardo Henao, Lawrence Carin

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Scopus citations

Abstract

Weight pruning is among the most popular approaches for compressing deep convolutional neural networks. Recent work suggests that in a randomly initialized deep neural network, there exist sparse subnetworks that achieve performance comparable to the original network. Unfortunately, finding these subnetworks involves iterative stages of training and pruning, which can be computationally expensive. We propose Structured Sparse Convolution (SSC), that leverages the inherent structure in images to reduce the parameters in the convolutional filter. This leads to improved efficiency of convolutional architectures compared to existing methods that perform pruning at initialization. We show that SSC is a generalization of commonly used layers (depthwise, groupwise and pointwise convolution) in "efficient architectures."Extensive experiments on well-known CNN models and datasets show the effectiveness of the proposed method. Architectures based on SSC achieve state-of-the-art performance compared to baselines on CIFAR10, CIFAR-100, Tiny-ImageNet, and ImageNet classification benchmarks.
Original languageEnglish (US)
Title of host publication2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)
PublisherIEEE
Pages6492-6502
Number of pages11
ISBN (Print)9781665493468
DOIs
StatePublished - Feb 6 2023

Fingerprint

Dive into the research topics of 'Pushing the Efficiency Limit Using Structured Sparse Convolutions'. Together they form a unique fingerprint.

Cite this