NeuLens: Spatial-based Dynamic Acceleration of Convolutional Neural Networks on Edge
Document Type
Conference Proceeding
Publication Date
10-14-2022
Abstract
Convolutional neural networks (CNNs) play an important role in today's mobile and edge computing systems for vision-based tasks like object classification and detection. However, state-of-The-Art methods on CNN acceleration are trapped in either limited practical latency speed-up on general computing platforms or latency speed-up with severe accuracy loss. In this paper, we propose a spatial-based dynamic CNN acceleration framework, NeuLens, for mobile and edge platforms. Specially, we design a novel dynamic inference mechanism, assemble region-Aware convolution (ARAC) supernet, that peels off redundant operations inside CNN models as many as possible based on spatial redundancy and channel slicing. In ARAC supernet, the CNN inference flow is split into multiple independent micro-flows, and the computational cost of each can be autonomously adjusted based on its tiled-input content and application requirements. These micro-flows can be loaded into hardware like GPUs as single models. Consequently, its operation reduction can be well translated into latency speed-up and is compatible with hardware-level accelerations. Moreover, the inference accuracy can be well preserved by identifying critical regions on images and processing them in the original resolution with large micro-flow. Based on our evaluation, NeuLens outperforms baseline methods by up to 58% latency reduction with the same accuracy and by up to 67.9% accuracy improvement under the same latency/memory constraints.
Identifier
85140893455 (Scopus)
ISBN
[9781450391818]
Publication Title
Proceedings of the Annual International Conference on Mobile Computing and Networking MOBICOM
External Full Text Location
https://doi.org/10.1145/3495243.3560528
ISSN
15435679
First Page
186
Last Page
199
Grant
2047655
Fund Ref
National Science Foundation
Recommended Citation
Hou, Xueyu; Guan, Yongjie; and Han, Tao, "NeuLens: Spatial-based Dynamic Acceleration of Convolutional Neural Networks on Edge" (2022). Faculty Publications. 2592.
https://digitalcommons.njit.edu/fac_pubs/2592
