research-article

Public Access

Modular Neural Networks for Low-Power Image Classification on Embedded Devices

Authors:

Sara Aghajanzadeh,

George K. Thiruvathukal,

Yung-Hsiang LuAuthors Info & Claims

ACM Transactions on Design Automation of Electronic Systems (TODAES), Volume 26, Issue 1

Article No.: 1, Pages 1 - 35

https://doi.org/10.1145/3408062

Published: 15 October 2020 Publication History

All formats PDF

Abstract

Embedded devices are generally small, battery-powered computers with limited hardware resources. It is difficult to run deep neural networks (DNNs) on these devices, because DNNs perform millions of operations and consume significant amounts of energy. Prior research has shown that a considerable number of a DNN’s memory accesses and computation are redundant when performing tasks like image classification. To reduce this redundancy and thereby reduce the energy consumption of DNNs, we introduce the Modular Neural Network Tree architecture. Instead of using one large DNN for the classifier, this architecture uses multiple smaller DNNs (called modules) to progressively classify images into groups of categories based on a novel visual similarity metric. Once a group of categories is selected by a module, another module then continues to distinguish among the similar categories within the selected group. This process is repeated over multiple modules until we are left with a single category. The computation needed to distinguish dissimilar groups is avoided, thus reducing redundant operations, memory accesses, and energy. Experimental results using several image datasets reveal the effectiveness of our proposed solution to reduce memory requirements by 50% to 99%, inference time by 55% to 95%, energy consumption by 52% to 94%, and the number of operations by 15% to 99% when compared with existing DNN architectures, running on two different embedded systems: Raspberry Pi 3 and Raspberry Pi Zero.

References

[1]

A. Mohan et al. 2017. Internet of Video Things in 2030: A world with many cameras. In Proceedings of IEEE ISCAS 2017. 1--4.

[2]

S. Han et al. 2015. Deep compression: Compressing deep neural networks with pruning, trained quantization and Huffman coding. ArXiv:1510.00149 [cs].

[3]

H. Zhao et al. 2018. Thermal-sensor-based occupancy detection for smart buildings using machine-learning methods. ACM Transactions on Design Automation of Electronic Systems 23, 4 (2018), Article 54, 21 pages.

Digital Library

[4]

H. Huang et al. 2018. Distributed machine learning on smart-gateway network toward real-time smart-grid energy management with behavior cognition. ACM Transactions on Design Automation of Electronic Systems 23, 5 (2018), 1--26.

Digital Library

[5]

S. Aghajanzadeh et al. 2020. Camera placement meeting restrictions of computer vision. In Proceedings of IEEE ICIP 2020.

[6]

Y. Lu. 2019. Low-power image recognition. Nature Machine Intelligence 1, 4 (2019), 199.

[7]

K. He et al. 2016. Deep residual learning for image recognition. In Proceedings of IEEE CVPR 2016. 770--778.

[8]

M. Amir et al. 2018. Switching predictive control using reconfigurable state-based model. ACM Transactions on Design Automation of Electronic Systems 24, 1 (2018), Article 2, 21 pages.

[9]

R. Fallahzadeh et al. 2018. Trading off power consumption and prediction performance in wearable motion sensors: An optimal and real-time approach. ACM Transactions on Design Automation of Electronic Systems 23, 5 (2018), Article 67, 23 pages.

Digital Library

[10]

S. Anup et al. 2017. Visual positioning system for automated indoor/outdoor navigation. In Proceedings of IEEE TENCON 2017.

[11]

S. Alyamkin et al. 2019. Low-power computer vision: Status, challenges, and opportunities. IEEE Journal on Emerging and Selected Topics in Circuits and Systems 9, 2 (2019), 411--421.

[12]

K. Gauen et al. 2018. Three years of low-power image recognition challenge. In Proceedings of IEEE DATE 2018.

[13]

K. Simonyan et al. 2014. Very deep convolutional networks for large-scale image recognition. ArXiv:1409.1556 [cs].

[14]

Y. Cheng et al. 2015. An exploration of parameter redundancy in deep networks with circulant projections. In Proceedings of IEEE ICCV 2015.

Digital Library

[15]

A. Goel. 2019. Modular Neural Networks. Retrieved August 8, 2020 from https://github.com/abhinavgoel95/Modular_Neural_Networks.

[16]

I. Ghodgaonkar et al. 2020. Observing responses to the COVID-19 pandemic using worldwide network cameras. ArXiv:2005.09091.

[17]

C. Szegedy et al. 2013. Deep neural networks for object detection. In Proceedings of Advances in NeurIPS 2013. 2553--2561.

[18]

J. R. Quinlan. 1986. Induction of decision trees. Machine Learning 1, 1 (1986), 81--106.

[19]

T. Cover et al. 1967. Nearest neighbor pattern classification. IEEE Transactions on Information Theory 13, 1 (1967), 21--27.

Digital Library

[20]

N. Friedman et al. 1997. Bayesian network classifiers. Machine Learning 29 (1997), 131--163.

Digital Library

[21]

S. Kaski. 1998. Dimensionality reduction by random mapping: Fast similarity computation for clustering. In Proceedings of IEEE IJCNN 1998 and IEEE WCCI1998, Vol. 1. 413--418.

[22]

G. Huang et al. 2018. CondenseNet: An efficient DenseNet using learned group convolutions. In Proceedings of IEEE CVPR 2018.

[23]

S. Bianco et al. 2018. Benchmark analysis of representative deep neural network architectures. IEEE Access 6 (2018), 64270--64277.

[24]

A. Goel et al. 2020. A survey of methods for low-power deep learning and computer vision. In Proceedings of IEEE WF-IoT 2020.

[25]

M. Rastegari et al. 2016. XNOR-Net: ImageNet classification using binary convolutional neural networks. In Proceedings of ECCV 2016. 525--542.

[26]

H. Albalawi et al. 2017. Training fixed-point classifiers for on-chip low-power implementation. ACM Transactions on Design Automation of Electronic Systems 22, 4 (2017), 69:1--69:18.

Digital Library

[27]

H. Li et al. 2016. Pruning filters for efficient ConvNets. ArXiv:1608.08710 [cs].

[28]

A. Goel et al. 2018. CompactNet: High accuracy deep neural network optimized for on-chip implementation. In Proceedings of IEEE Big Data2018.

[29]

L. Jiang et al. 2019. Energy-efficient and quality-assured approximate computing framework using a co-training method. ACM Transactions on Design Automation of Electronic Systems 24, 6 (2019), Article 59, 25 pages.

Digital Library

[30]

F. N. Iandola et al. 2016. SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and < 0.5 MB model size. ArXiv:1602.07360 [cs].

[31]

M. Sandler et al. 2018. MobileNetV2: Inverted residuals and linear bottlenecks. In Proceedings of IEEE CVPR 2018. 4510--4520.

[32]

C. Szegedy et al. 2017. Inception-v4, Inception-ResNet and the impact of residual connections on learning. In Proceedings of ACM AAAI 2017. 4278--4284.

[33]

A. Sironi et al. 2015. Learning separable filters. IEEE Transactions on Pattern Analysis and Machine Intelligence 37 (2015), 94--106.

[34]

E. Denton et al. 2014. Exploiting linear structure within convolutional networks for efficient evaluation. In Proceedings of Advances in NeurIPS 2014. 1269--1277.

[35]

M. Jaderberg et al. 2014. Speeding up convolutional neural networks with low rank expansions. ArXiv:1405.3866.

[36]

G. Zhong et al. 2019. Synergy: An HW/SW framework for high throughput CNNs on embedded heterogeneous SoC. ACM Transactions on Embedded Computing Systems 18, 2 (2019), Article 13, 23 pages.

Digital Library

[37]

J. Li et al. 2018. SynergyFlow: An elastic accelerator architecture supporting batch processing of large-scale deep neural networks. ACM Transactions on Design Automation of Electronic Systems 24, 1 (2018), 1--27.

Digital Library

[38]

Y. Cheng et al. 2017. A survey of model compression and acceleration for deep neural networks. ArXiv:1710.09282 [cs].

[39]

G. Hinton et al. 2015. Distilling the knowledge in a neural network. ArXiv:1503.02531 [cs, stat].

[40]

J. Ba et al. 2014. Do deep nets really need to be deep? In Proceedings of Advances in NeurIPS 2014. 2654--2662.

[41]

J. Guérin et al. 2017. CNN features are also great at unsupervised classification. ArXiv:1707.01700 [cs].

[42]

V. Di Gesú et al. 1999. Distance-based functions for image comparison. Pattern Recognition Letters 20 (1999), 207--214.

Digital Library

[43]

R. Zhang et al. 2015. Bit-scalable deep hashing with regularized similarity learning for image retrieval and person re-identification. IEEE Transactions on Image Processing 24, 12 (2015), 4766--4779.

Digital Library

[44]

H. Zhu et al. 2016. Deep hashing network for efficient similarity retrieval. In Proceedings of AAAI 2016. 2415--2421.

[45]

G. Griffin et al. 2008. Learning and using taxonomies for fast visual categorization. In Proceedings of IEEE CVPR 2008. 1--8.

[46]

J. Deng et al. 2011. Fast and balanced: Efficient label tree learning for large scale object recognition. In Proceedings of Advances in NeurIPS 2011.

[47]

A. Beygelzimer et al. 2009. Conditional probability tree estimation analysis and algorithms. In Proceedings of ACM UAI 2009. 51--58.

[48]

X. Yuan et al. 2006. Automatic video genre categorization using hierarchical SVM. In Proceedings of ICIP 2006. 2905--2908.

[49]

M. Rastegari et al. 2012. Attribute discovery via predictable discriminative binary codes. In Proceedings of ECCV 2012. 876--889.

Digital Library

[50]

P. Panda et al. 2017. FALCON: Feature driven selective classification for energy-efficient image recognition. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 36, 12 (2017), 2017--2029.

[51]

A. Torralba et al. 2008. 80 million tiny images: A large data set for nonparametric object and scene recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 30 (2008), 1958--1970.

Digital Library

[52]

J. Redmon et al. 2016. 2016. YOLO9000: Better, faster, stronger. ArXiv:1612.08242 [cs].

[53]

A. Zweig et al. 2007. Exploiting object hierarchy: combining models from different category levels. In Proceedings of IEEE ICCV 2007.

[54]

V. Peluso et al. 2018. Scalable-effort ConvNets for multilevel classification. In Proceedings of IEEE/ACM ICCAD 2018. 1--8.

Digital Library

[55]

S. Chen et al. 2015. Discriminative hierarchical k-means tree for large-scale image classification. IEEE Transactions on Neural Networks and Learning Systems 26, 9 (2015), 2200--2205.

[56]

M. Marszalek et al. 2008. Constructing category hierarchies for visual recognition. In Proceedings of ECCV 2008. Vol. 5305. 479--491.

Digital Library

[57]

Y. Lukic et al. 2016. Speaker identification and clustering using convolutional neural networks. In Proceedings of IEEE MLSP 2016. IEEE, Los Alamitos, CA, 1--6.

[58]

A. K. Jain et al. 1997. Object detection using Gabor filters. Pattern Recognition 30 (1997), 295--309.

[59]

H. Levkowitz et al. 1993. GLHS: A generalized lightness, hue, and saturation color model. CVGIP: Graphical Models and Image Processing 55 (1993), 271--285.

Digital Library

[60]

Y. Qu et al. 2017. Joint hierarchical category structure learning and large-scale image classification. IEEE Transactions on Image Processing 26, 9 (2017), 4331--4346.

Digital Library

[61]

G. A. Miller. 1995. WordNet: A lexical database for English. Communications of the ACM 38 (1995), 39--41.

Digital Library

[62]

R. Xia et al. 2014. Supervised hashing for image retrieval via image representation learning. In Proceedings of AAAI 2014. 2156--2162.

[63]

F. Shen et al. 2018. Unsupervised deep hashing with similarity-adaptive and discrete optimization. IEEE Transactions on Pattern Analysis and Machine Intelligence 40, 12 (2018), 3034--3044.

Digital Library

[64]

J. Wang et al. 2010. Semi-supervised hashing for scalable image retrieval. In Proceedings of IEEE CVPR 2010. 3424--3431.

[65]

Z. Wang et al. 2018. Learning fine-grained features via a CNN tree for large-scale classification. Neurocomputing 275 (2018), 1231--1240.

[66]

D. Roy et al. 2018. Tree-CNN: A hierarchical deep convolutional neural network for incremental learning. ArXiv:1802.05800.

[67]

M. Sun et al. 2013. Find the best path: An efficient and accurate classifier for image hierarchies. In Proceedings of IEEE ICCV 2013. 265--272.

Digital Library

[68]

Y. Guo et al. 2016. Dynamic network surgery for efficient DNNs. In Proceedings of Advances in of NeurIPS 2016. 1379--1387.

[69]

L. A. Zadeh. 1965. Fuzzy sets. Information and Control 8, 3 (1965), 338--353.

[70]

N. D. Singpurwalla et al. 2004. Membership functions and probability measures of fuzzy sets. Journal of the American Statistical Association 99 (2004), 867--889.

[71]

M. Tan et al. EfficientNet: Rethinking model scaling for convolutional neural networks. In Proceedings of ICML 2019. 6105--6114.

[72]

P. Kontschieder et al. 2015. Deep neural decision forests. In Proceedings of ICCV 2015. 1467--1475.

Digital Library

[73]

A. Roy et al. 2016. Monocular depth estimation using neural regression forest. In Proceedings of IEEE CVPR 2016. 5506--5514.

[74]

G. Huang et al. 2017. Densely connected convolutional networks. In Proceedings of IEEE CVPR 2017.

[75]

A. Krizhevsky et al. 2009. Learning Multiple Layers of Features from Tiny Images. Technical Report TR-2009. University of Toronto.

[76]

Y. Netzer et al. 2011. Reading digits in natural images with unsupervised feature learning. In Proceedings of the NeurIPS 2011 Workshop on Deep Learning and Unsupervised Feature Learning.

[77]

G. Cohen et al. 2017. EMNIST: An extension of MNIST to handwritten letters. ArXiv:1702.05373 [cs].

[78]

J. Deng et al. 2009. ImageNet: A large-scale hierarchical image database. In Proceedings of IEEE CVPR 2009. 248--255.

[79]

G. Griffin et al. 2007. Caltech-256 Object Category Dataset. Technical Report. Available at http://authors.library.caltech.edu/7694.

[80]

Yokogawa. 2017. WT310E/TW310EH/WT332E/WT333E Digital Power Meter: User’s Manual. Retrieved August 9, 2020 from https://cdn.tmi.yokogawa.com/IMWT310E-01EN.pdf.

[81]

PyTorch. 2019. Torch.utils.data. Retrieved August 9, 2020 from https://pytorch.org/docs/stable/data.html.

[82]

A. Canziani et al. 2016. An analysis of deep neural network models for practical applications. ArXiv:1605.07678

[83]

B. Chen et al. 2018. Introducing the CVPR 2018 On-Device Visual Intelligence Challenge. Google AI Blog. Retrieved August 9, 2020 from http://ai.googleblog.com/2018/04/introducing-cvpr-2018-on-device-visual.html.

[84]

A. Mordvintsev et al. 2013. Getting Started with Videos: OpenCV-Python Documentation. Retrieved August 9, 2020 from https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_gui/py_video_display/py_video_display.html.

[85]

D. P. Kingma et al. 2014. Adam: A method for stochastic optimization. ArXiv:1412.6980 [cs].

[86]

A. Goel. 2019. Image Classification with the Modular Neural Network Tree. Retrieved August 9, 2020 from https://youtu.be/gdae-v-ZyVs.

[87]

S. Zagoruyko et al. 2016. Wide residual networks. ArXiv:1605.07146 [cs].

[88]

E. Dufourq et al. 2017. EDEN: Evolutionary deep networks for efficient machine learning. In Proceedings of PRASA-RobMech 2017.

[89]

T. Hastie et al. 2001. The Elements of Statistical Learning. Springer.

[90]

J. J. Jiang et al. 1997. Semantic similarity based on corpus statistics and lexical taxonomy. In Proceedings of ROCLING 1997. 19--33.

[91]

R. R. Selvaraju et al. 2016. Grad-CAM: Visual explanations from deep networks via gradient-based localization. ArXiv:1610.02391 [cs].

[92]

A. Krizhevsky et al. 2012. ImageNet classification with deep convolutional neural networks. In Proceedings of NeurIPS 2012.

[93]

A. Gondimalla et al. 2019. SparTen: A sparse tensor accelerator for convolutional neural networks. In Proceedings of ACM/IEEE MICRO 2019. 151--165.

Digital Library

[94]

J. Albericio et al. 2016. Cnvlutin: Ineffectual-neuron-free deep neural network computing. In Proceedings of ACM/IEEE ISCA 2016.

Digital Library

[95]

A. Parashar et al. 2017. SCNN: An accelerator for compressed-sparse convolutional neural networks. In Proceedings of ACM ISCA 2017.

Digital Library

Cited By

Liang YHsu YChung C(2024)A Low-Power Hierarchical CNN Hardware Accelerator for Bearing Fault DiagnosisIEEE Transactions on Instrumentation and Measurement10.1109/TIM.2024.335122973(1-11)Online publication date: 2024
https://doi.org/10.1109/TIM.2024.3351229
Chen YWang HBi ALiu YZhuang CLv YLiu T(2024)Embedded Automation Device Based on Trusted Computing2024 International Conference on Integrated Circuits and Communication Systems (ICICACS)10.1109/ICICACS60521.2024.10498242(1-9)Online publication date: 23-Feb-2024
https://doi.org/10.1109/ICICACS60521.2024.10498242
Goel ATung CEliopoulos NThiruvathukal GWang ALu YDavis J(2023)Tree-Based Unidirectional Neural Networks for Low-Power Computer VisionIEEE Design & Test10.1109/MDAT.2022.321701640:3(53-61)Online publication date: Jul-2023
https://doi.org/10.1109/MDAT.2022.3217016
Show More Cited By

Index Terms

Modular Neural Networks for Low-Power Image Classification on Embedded Devices

Recommendations

A dyadic multi-resolution deep convolutional neural wavelet network for image classification

For almost the past four decades, image classification has gained a lot of attention in the field of pattern recognition due to its application in various fields. Given its importance, several approaches have been proposed up to now. In this paper, we ...
Bi-branch deconvolution-based convolutional neural network for image classification

With the rise of deep neural network, convolutional neural networks show superior performances on many different computer vision recognition tasks. The convolution is used as one of the most efficient ways for extracting the details features of an image,...
Capsule Embedded ResNet for Image Classification
CSAI '21: Proceedings of the 2021 5th International Conference on Computer Science and Artificial Intelligence

Various neural network models have been proposed in the past decade. Among them, the residual neural network (ResNet) is one of the most successful models of convolutional neural networks (CNNs), while the capsule neural network (CapsNet) is more robust ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Design Automation of Electronic Systems

ACM Transactions on Design Automation of Electronic Systems Volume 26, Issue 1

January 2021

234 pages

ISSN:1084-4309

EISSN:1557-7309

DOI:10.1145/3422280

Editor:
X. Sharon Hu
University of Notre Dame, USA

Issue’s Table of Contents

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Journal Family

ACM Journals for the Design of Smart and Connected Systems

Publication History

Published: 15 October 2020

Accepted: 01 June 2020

Revised: 01 April 2020

Received: 01 November 2019

Published in TODAES Volume 26, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

National Science Foundation

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
1,901
Total Downloads

Downloads (Last 12 months)733
Downloads (Last 6 weeks)76

Reflects downloads up to 26 Jul 2024

Other Metrics

View Author Metrics

Citations

Cited By

Liang YHsu YChung C(2024)A Low-Power Hierarchical CNN Hardware Accelerator for Bearing Fault DiagnosisIEEE Transactions on Instrumentation and Measurement10.1109/TIM.2024.335122973(1-11)Online publication date: 2024
https://doi.org/10.1109/TIM.2024.3351229
Chen YWang HBi ALiu YZhuang CLv YLiu T(2024)Embedded Automation Device Based on Trusted Computing2024 International Conference on Integrated Circuits and Communication Systems (ICICACS)10.1109/ICICACS60521.2024.10498242(1-9)Online publication date: 23-Feb-2024
https://doi.org/10.1109/ICICACS60521.2024.10498242
Goel ATung CEliopoulos NThiruvathukal GWang ALu YDavis J(2023)Tree-Based Unidirectional Neural Networks for Low-Power Computer VisionIEEE Design & Test10.1109/MDAT.2022.321701640:3(53-61)Online publication date: Jul-2023
https://doi.org/10.1109/MDAT.2022.3217016
Kaur IJadhav A(2023)On the fly Deep Neural Network Optimization Control for Low-Power Computer Vision2023 IEEE International Performance, Computing, and Communications Conference (IPCCC)10.1109/IPCCC59175.2023.10253878(465-469)Online publication date: 17-Nov-2023
https://doi.org/10.1109/IPCCC59175.2023.10253878
Md Ramli SNizam Ibrahim MMohamad ADaud KSaidina Omar ADarina Ahmad N(2023)Review of Artificial Neural Network Approaches for Predicting Building Energy Consumption2023 IEEE 3rd International Conference in Power Engineering Applications (ICPEA)10.1109/ICPEA56918.2023.10093183(328-333)Online publication date: 6-Mar-2023
https://doi.org/10.1109/ICPEA56918.2023.10093183
Li YSui NGuo CGuo Z(2023)Real-Time Deep Learning Framework for Dermatology Image Classification on Low-Power Embedded Devices2023 13th International Conference on Information Science and Technology (ICIST)10.1109/ICIST59754.2023.10367148(496-503)Online publication date: 8-Dec-2023
https://doi.org/10.1109/ICIST59754.2023.10367148
Eliopoulos NLu Y(2023)Lightning Talk 6: Bringing Together Foundation Models and Edge Devices2023 60th ACM/IEEE Design Automation Conference (DAC)10.1109/DAC56929.2023.10247694(1-2)Online publication date: 9-Jul-2023
https://doi.org/10.1109/DAC56929.2023.10247694
Sitepu ALiu C(2023)Efficient Computer Vision Inference using Modular Neural Network Techniques2023 VTS Asia Pacific Wireless Communications Symposium (APWCS)10.1109/APWCS60142.2023.10234066(1-2)Online publication date: Aug-2023
https://doi.org/10.1109/APWCS60142.2023.10234066
Wong SChuah JYap HTan C(2023)Dissociation artificial neural network for tool wear estimation in CNC millingThe International Journal of Advanced Manufacturing Technology10.1007/s00170-022-10737-8125:1-2(887-901)Online publication date: 3-Jan-2023
https://doi.org/10.1007/s00170-022-10737-8
Joseph TBindiya T(2023)Performance-Driven LSTM Accelerator Hardware Using Split-Matrix-Based MVMCircuits, Systems, and Signal Processing10.1007/s00034-023-02412-442:11(6660-6683)Online publication date: 8-Jun-2023
https://dl.acm.org/doi/10.1007/s00034-023-02412-4
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Media

Figures

Other

Tables

View Issue’s Table of Contents