research-article

Open access

Confidential Execution of Deep Learning Inference at the Untrusted Edge with ARM TrustZone

Authors:

Md Shihabul Islam,

Mahmoud Zamani,

Chung Hwan Kim,

Kevin W. HamlenAuthors Info & Claims

CODASPY '23: Proceedings of the Thirteenth ACM Conference on Data and Application Security and Privacy

Pages 153 - 164

https://doi.org/10.1145/3577923.3583648

Published: 24 April 2023 Publication History

Abstract

This paper proposes a new confidential deep learning (DL) inference system with ARM TrustZone to provide confidentiality and integrity of DL models and data in an untrusted edge device with limited memory. Although ARM TrustZone supplies a strong, hardware-supported trusted execution environment for protecting sensitive code and data in an edge device against adversaries, resource limitations in typical edge devices have raised significant challenges for protecting on-device DL requiring large memory consumption without sacrificing the security and accuracy of the model. The proposed solution addresses this challenge without modifying the protected DL model, thereby preserving the original prediction accuracy. Comprehensive experiments using different DL architectures and datasets demonstrate that inference services for large and complex DL models can be deployed in edge devices with TrustZone with limited trusted memory, ensuring data confidentiality and preserving the original model's prediction exactness.

References

[1]

Julien Amacher and Valerio Schiavoni. 2019. On the Performance of ARM TrustZone. In Proc. cnum19th IFIP Int. Conf. Distributed Applications and Interoperable Systems nymDAIS. 133--151.

Digital Library

[2]

Andrew Anderson, Aravind Vasudevan, Cormac Keane, and David Gregg. 2017. Low-memory Gemm-based Convolution Algorithms for Deep Neural Networks. arXiv Preprint 1709.03395 (2017).

[3]

ARM. 2009. ARM Security Technology: Building a Secure System using TrustZone Technology. White paper PRD29-GENC-009492C. ARM.

[4]

Mamoun A Awad and Latifur R Khan. 2007. Web navigation prediction using multiple evidence combination and domain knowledge. IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, Vol. 37, 6 (2007), 1054--1062.

Digital Library

[5]

Keith Bonawitz, Hubert Eichner, Wolfgang Grieskamp, Dzmitry Huba, Alex Ingerman, Vladimir Ivanov, Chloé Kiddon, Jakub Konevc nỳ, Stefano Mazzocchi, Brendan McMahan, Timon Van Overveldt, David Petrou, Daniel Ramage, and Jason Roselander. 2019. Towards Federated Learning at Scale: System Design. In Proc. Machine Learning and Systems nymMLSys.

[6]

Victor Costan and Srinivas Devadas. 2016. Intel SGX Explained. IACR Cryptology ePrint Archive, Vol. 2016, 086 (2016).

[7]

Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. ImageNet: A Large-scale Hierarchical Image Database. In Proc. IEEE Conf. Computer Vision and Pattern Recognition nymCVPR.

[8]

Alexey Dosovitskiy and Thomas Brox. 2016. Inverting Visual Representations with Convolutional Networks. In Proc. IEEE Conf. Computer Vision and Pattern Recognition nymCVPR. 4829--4837.

[9]

Marat Dukhan. 2019. The Indirect Convolution Algorithm. arXiv Preprint 1907.02129 (2019).

[10]

Vitaly Feldman, Konstantin Kakaes, Katrina Ligett, Kobbi Nissim, Aleksandra Slavkovic, and Adam Smith. 2020. Differential Privacy: Issues for Policymakers. White paper. Simons Institute Theory of Computing, University of California at Berkeley.

[11]

Matt Fredrikson, Somesh Jha, and Thomas Ristenpart. 2015. Model Inversion Attacks That Exploit Confidence Information and Basic Countermeasures. In Proc. cnum22nd ACM Conf. Computer and Communications Security nymCCS. 1322--1333.

Digital Library

[12]

Zhongshu Gu, Heqing Huang, Jialong Zhang, Dong Su, Hani Jamjoom, Ankita Lamba, Dimitrios Pendarakis, and Ian Molloy. 2018. Confidential Inference Via Ternary Model Partitioning. arXiv Preprint 1807.00969 (2018).

[13]

Song Han, Huizi Mao, and William J. Dally. 2016. Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding. In Proc. cnum4th Int. Conf. Learning Representations nymICLR.

[14]

IBM X-Forcetextsuperscript® Research. 2017. The Weaponization of IoT Devices. www.ibm.com/downloads/cas/6MLEALKV.

[15]

Md Shihabul Islam, Mustafa Safa Ozdayi, Latifur Khan, and Murat Kantarcioglu. 2020. Secure IoT data analytics in cloud via Intel SGX. In 2020 IEEE 13th International Conference on Cloud Computing (CLOUD). IEEE, 43--52.

[16]

Md Shihabul Islam, Harsh Verma, Latifur Khan, and Murat Kantarcioglu. 2019. Secure real-time heterogeneous iot data management system. In 2019 first IEEE international conference on trust, privacy and security in intelligent systems and applications (TPS-ISA). IEEE, 228--235.

[17]

Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew Howard, Hartwig Adam, and Dmitry Kalenichenko. 2018. Quantization and Training of Neural Networks for Efficient Integer-arithmetic-only Inference. In Proc. IEEE Conf. Computer Vision and Pattern Recognition nymCVPR. 2704--2713.

[18]

Kyungtae Kim, Chung Hwan Kim, Junghwan “John” Rhee, Xiao Yu, Haifeng Chen, Dave Tian, and Byoungyoung Lee. 2020. Vessels: Efficient and Scalable Deep Learning Prediction on Trusted Processors. In Proc. cnum11th ACM Sym. Cloud Computing nymSoCC. 462--476.

[19]

cC etin Kaya Kocc. 2020. Formidable Challenges in Hardware Implementations of Fully Homomorphic Encryption Functions for Applications in Machine Learning. In Proc. cnum4th ACM Workshop Attacks and Solutions in Hardware Security nymASHES.

Digital Library

[20]

Alex Krizhevsky. 2014. One Weird Trick for Parallelizing Convolutional Neural Networks. arXiv Preprint 1404.5997 (2014).

[21]

Alex Krizhevsky and Geoffrey Hinton. 2009. Learning Multiple Layers of Features from Tiny Images. Technical Report. University of Toronto.

[22]

Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. 1998. Gradient-based Learning Applied to Document Recognition. Proc. IEEE, Vol. 86, 11 (1998), 2278--2324.

[23]

Taegyeong Lee, Zhiqi Lin, Saumay Pushp, Caihua Li, Yunxin Liu, Youngki Lee, Fengyuan Xu, Chenren Xu, Lintao Zhang, and Junehwa Song. 2019. Occlumency: Privacy-preserving Remote Deep-learning Inference Using SGX. In Proc. cnum25th Annual Int. Conf. Mobile Computing and Networking nymMobiCom.

Digital Library

[24]

Linaro. 2022a. OP-TEE Architecture. https://optee.readthedocs.io/en/latest/architecture/index.html.

[25]

Linaro. 2022b. Open Portable Trusted Execution Environment. www.op-tee.org.

[26]

Moritz Lipp, Daniel Gruss, Raphael Spreitzer, Clémentine Maurice, and Stefan Mangard. 2016. Armageddon: Cache Attacks on Mobile Devices. In Proc. cnum25th USENIX Security Sym. 549--564.

[27]

Aravindh Mahendran and Andrea Vedaldi. 2015. Understanding Deep Image Representations By Inverting Them. In Proc. IEEE Conf. Computer Vision and Pattern Recognition nymCVPR. 5188--5196.

[28]

Fan Mo, Ali Shahin Shamsabadi, Kleomenis Katevas, Soteris Demetriou, Ilias Leontiadis, Andrea Cavallaro, and Hamed Haddadi. 2020. DarkneTZ: Towards Model Privacy at the Edge Using Trusted Execution Environments. In Proc. cnum18th Int. Conf. Mobile Systems, Applications, and Services nymMobiSys. 161--174.

Digital Library

[29]

Olga Ohrimenko, Felix Schuster, Cédric Fournet, Aastha Mehta, Sebastian Nowozin, Kapil Vaswani, and Manuel Costa. 2016. Oblivious Multi-party Machine Learning on Trusted Processors. In Proc. cnum25th USENIX Security Sym. 619--636.

[30]

Heejin Park, Shuang Zhai, Long Lu, and Felix Xiaozhu Lin. 2019. StreamBox-TZ: Secure Stream Analytics at the Edge with TrustZone. In Proc. USENIX Annual Technical Conf. nymATC. 537--554.

[31]

Le Trieu Phong, Yoshinori Aono, Takuya Hayashi, Lihua Wang, and Shiho Moriai. 2017. Privacy-preserving Deep Learning Via Additively Homomorphic Encryption. IEEE Trans. Information Forensics and Security nymTIFS, Vol. 13, 5 (2017), 1333--1345.

[32]

Sandro Pinto and Nuno Santos. 2019. Demystifying ARM TrustZone: A Comprehensive Survey. ACM Computing Surveys nymCSUR, Vol. 51, 6 (2019), 1--36.

Digital Library

[33]

Qualcomm. 2020. We Are Making AI Ubiquitous. www.qualcomm.com/news/onq/2020/06/we-are-making-ai-ubiquitous.

[34]

Raspberry Pi. 2022. Raspberry Pi 3 Model B. https://www.raspberrypi.com/products/raspberry-pi-3-model-b/.

[35]

Joseph Redmon. 2013--2016. Darknet: Open Source Neural Networks in C. pjreddie.com/darknet.

[36]

Joseph Redmon. 2022. Tiny Darknet. pjreddie.com/darknet/tiny-darknet.

[37]

Joseph Redmon and Ali Farhadi. 2018. YOLOv3: An Incremental Improvement. arXiv Preprint 1804.02767 (2018).

[38]

Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. 2017. Membership inference attacks against machine learning models. In 2017 IEEE symposium on security and privacy (SP). IEEE, 3--18.

[39]

Karen Simonyan and Andrew Zisserman. 2015. Very Deep Convolutional Networks for Large-scale Image Recognition. In Proc. cnum3rd Int. Conf. Learning Representations nymICLR.

[40]

ST Microelectronics. 2022. Discovery Kit with STM32MP157C MPU. www.st.com/en/evaluation-tools/stm32mp157c-dk2.html.

[41]

Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. 2015. Going Deeper with Convolutions. In Proc. IEEE Conf. Computer Vision and Pattern Recognition nymCVPR. 1--9.

[42]

Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. 2016. Rethinking the Inception Architecture for Computer Vision. In Proc. IEEE Conf. Computer Vision and Pattern Recognition nymCVPR. 2818--2826.

[43]

TensorFlow. 2022. TensorFlow Lite. www.tensorflow.org/lite.

[44]

Florian Tramer and Dan Boneh. 2018. Slalom: Fast, verifiable and private execution of neural networks in trusted hardware. arXiv preprint arXiv:1806.03287 (2018).

[45]

Peter M. VanNostrand, Ioannis Kyriazis, Michelle Cheng, Tian Guo, and Robert J. Walls. 2019. Confidential Deep Learning: Executing Proprietary Models on Untrusted Devices. arXiv Preprint 1908.10730 (2019).

[46]

Johannes Winter. 2008. Trusted Computing Building Blocks for Embedded Linux-based ARM TrustZone Platforms. In Proc. cnum3rd ACM Workshop Scalable Trusted Computing nymSTC. 21--30.

Digital Library

[47]

Jiaxiang Wu, Cong Leng, Yuhang Wang, Qinghao Hu, and Jian Cheng. 2016. Quantized Convolutional Neural Networks for Mobile Devices. In Proc. IEEE Conf. Computer Vision and Pattern Recognition nymCVPR. 4820--4828.

[48]

I-Ling Yen, Jayabharath Goluguri, Farokh Bastani, Latifur Khan, and John Linn. 2002. A component-based approach for embedded software development. In Proceedings Fifth IEEE International Symposium on Object-Oriented Real-Time Distributed Computing. ISIRC 2002. IEEE, 402--410.

[49]

Lei Yu, Ling Liu, Calton Pu, Mehmet Emre Gursoy, and Stacey Truex. 2019. Differentially Private Model Publishing for Deep Learning. In Proc. cnum40th IEEE Sym. Security & Privacy nymS&P. 332--349.

[50]

Ning Zhang, Kun Sun, Deborah Shands, Wenjing Lou, and Y. Thomas Hou. 2016. TruSpy: Cache Side-channel Information Leakage From the Secure World on ARM Devices. IACR Cryptology ePrint Archive, Vol. 2016 (2016).

[51]

Shijun Zhao, Qianying Zhang, Yu Qin, Wei Feng, and Dengguo Feng. 2019a. Minimal Kernel: An Operating System Architecture for TEE to Resist Board Level Physical Attacks. In Proc. cnum22nd Int. Sym. Recent Advances in Intrusion Detection nymRAID. 105--120.

[52]

Shijun Zhao, Qianying Zhang, Yu Qin, Wei Feng, and Dengguo Feng. 2019b. SecTEE: A Software-based Approach to Secure Enclave Architecture Using TEE. In Proc. cnum26th ACM Conf. Computer and Communications Security nymCCS. 1723--1740.

Digital Library

[53]

Li Zhou, Hao Wen, Radu Teodorescu, and David HC Du. 2019. Distributing Deep Neural Networks with Containerized Partitions at the Edge. In Proc. cnum2nd USENIX Workshop Hot Topics in Edge Computing nymHotEdge. io

Cited By

Mo FTarkhani ZHaddadi H(2024)Machine Learning with Confidential Computing: A Systematization of KnowledgeACM Computing Surveys10.1145/367000756:11(1-40)Online publication date: 29-Jun-2024
https://dl.acm.org/doi/10.1145/3670007
Islam MAlouani IKhasawneh K(2024)Hardware Support for Trustworthy Machine Learning: A Survey2024 25th International Symposium on Quality Electronic Design (ISQED)10.1109/ISQED60706.2024.10528373(1-6)Online publication date: 3-Apr-2024
https://doi.org/10.1109/ISQED60706.2024.10528373
Li JLuo XLei HCheng J(2024)TEEm: Supporting Large Memory for Trusted Applications in ARM TrustZoneIEEE Access10.1109/ACCESS.2024.343123112(108584-108596)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3431231
Show More Cited By

Index Terms

Confidential Execution of Deep Learning Inference at the Untrusted Edge with ARM TrustZone
1. Computer systems organization
  1. Embedded and cyber-physical systems
2. Security and privacy
  1. Software and application security

Recommendations

Demystifying ARM TrustZone TEE Client API using OP-TEE
SMA 2020: The 9th International Conference on Smart Media and Applications

Recently, sensitive information such as financial data and electronic payment systems have been stored in mobile devices. To protect important data, TEE technology has emerged, a trusty and safe execution environment. In particular, ARM TrustZone ...
On the Performance of ARM TrustZone: (Practical Experience Report)
Distributed Applications and Interoperable Systems
Abstract
The TrustZone technology, available in the vast majority of recent Arm processors, allows the execution of code inside a so-called secure world. It effectively provides hardware-isolated areas of the processor for sensitive data and code, i.e., a ...
DarKnight: An Accelerated Framework for Privacy and Integrity Preserving Deep Learning Using Trusted Hardware
MICRO '21: MICRO-54: 54th Annual IEEE/ACM International Symposium on Microarchitecture

Privacy and security-related concerns are growing as machine learning reaches diverse application domains. The data holders want to train or infer with private data while exploiting accelerators, such as GPUs, that are hosted in the cloud. Cloud ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CODASPY '23: Proceedings of the Thirteenth ACM Conference on Data and Application Security and Privacy

April 2023

304 pages

ISBN:9798400700675

DOI:10.1145/3577923

General Chair:
Mohamed Shehab
University of North Carolina at Charlotte, USA
,
Program Chairs:
Maribel Fernandez
King's College London, UK
,
Ninghui Li
Purdue University, USA

Copyright © 2023 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

SIGSAC: ACM Special Interest Group on Security, Audit, and Control

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 April 2023

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

ONR
DARPA
NSF
ARO

Conference

CODASPY '23

Sponsor:

SIGSAC

CODASPY '23: Thirteenth ACM Conference on Data and Application Security and Privacy

April 24 - 26, 2023

NC, Charlotte, USA

Acceptance Rates

Overall Acceptance Rate 149 of 789 submissions, 19%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
648
Total Downloads

Downloads (Last 12 months)440
Downloads (Last 6 weeks)68

Reflects downloads up to 09 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Mo FTarkhani ZHaddadi H(2024)Machine Learning with Confidential Computing: A Systematization of KnowledgeACM Computing Surveys10.1145/367000756:11(1-40)Online publication date: 29-Jun-2024
https://dl.acm.org/doi/10.1145/3670007
Islam MAlouani IKhasawneh K(2024)Hardware Support for Trustworthy Machine Learning: A Survey2024 25th International Symposium on Quality Electronic Design (ISQED)10.1109/ISQED60706.2024.10528373(1-6)Online publication date: 3-Apr-2024
https://doi.org/10.1109/ISQED60706.2024.10528373
Li JLuo XLei HCheng J(2024)TEEm: Supporting Large Memory for Trusted Applications in ARM TrustZoneIEEE Access10.1109/ACCESS.2024.343123112(108584-108596)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3431231
Shabir MTorta GDamiani F(2024)Edge AI on Constrained IoT Devices: Quantization Strategies for Model OptimizationIntelligent Systems and Applications10.1007/978-3-031-66428-1_35(556-574)Online publication date: 31-Jul-2024
https://doi.org/10.1007/978-3-031-66428-1_35
Babar MHasan M(2023)Trusted Deep Neural Execution—A SurveyIEEE Access10.1109/ACCESS.2023.327419011(45736-45748)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3274190

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents