research-article

Open access

Security Enhancements for Approximate Machine Learning

Authors:

Jiliang ZhangAuthors Info & Claims

GLSVLSI '21: Proceedings of the 2021 Great Lakes Symposium on VLSI

Pages 461 - 466

https://doi.org/10.1145/3453688.3461753

Published: 22 June 2021 Publication History

Abstract

Approximate computing techniques for error-tolerant machine learning applications are gaining interest, promising a energy-accuracy balance for modern digital computing systems. As an ubiquitous step in machine learning, iterative solvers have been widely used for training neural networks and accelerating feedforward computations. In this paper, we provide three novel information hiding techniques that use properties of redundant number systems, most-significant digit-first arithmetic and the forward error analysis of stationary iterative methods, to secure approximate computing systems. We demonstrate that three different security signatures are encoded through redundant representation, function-equivalence arithmetic replacement and algorithmic optimisation. Our illustrative security enhancement countermeasures can be used to prevent potential attacks, such as privacy leakage, out-of-control systematic error and error injection in approximate computing.

Supplemental Material

MP4 File

GLSVLSI talk

Download
11.50 MB

References

[1]

Ameer MS Abdelhadi and Lesley Shannon. 2019. Revisiting deep learning parallelism: Fine-grained inference engine utilizing online arithmetic. In IEEE International Conference on Field-Programmable Technology. 383--386.

[2]

Achilles. [n.d.]. https://blog.checkpoint.com/2020/08/06/achilles-small-chip-bigperil/.

[3]

Pavan Adharapurapu and Milo? Ercegovac. 2005. A linear-system operator based scheme for evaluation of multinomials. In IEEE Symposium on Computer Arithmetic. 249--256.

Digital Library

[4]

Jorge Albericio, Alberto Delmás, Patrick Judd, Sayeh Sharify, Gerard O'Leary, Roman Genov, and Andreas Moshovos. 2017. Bit-pragmatic deep neural network computing. In IEEE/ACM International Symposium on Microarchitecture. 382--394.

Digital Library

[5]

Algirdas Avizienis. 1961. Signed-digit number representations for fast parallel arithmetic. IRE Transactions on electronic computers EC-10, 3 (1961), 389--400.

[6]

Pouya Dormiani, David Omoto, Pavan Adharapurapu, and Milos D Ercegovac. 2005. A design of online scheme for evaluation of multinomials. In Advanced Signal Processing Algorithms, Architectures, and Implementations XV. 59100S.

[7]

Charles Eckert, Xiaowei Wang, Jingcheng Wang, Arun Subramaniyan, Ravi Iyer, Dennis Sylvester, David Blaaauw, and Reetuparna Das. 2018. Neural cache: Bitserial in-cache acceleration of deep neural networks. In ACM/IEEE International Symposium on Computer Architecture). 383--396.

Digital Library

[8]

M.D. Ercegovac and T. Lang. 1988. On-line scheme for computing rotation factors. Journal Parallel and Distributed Computing 5, 6 (1988), 209--227.

Digital Library

[9]

Milos D Ercegovac and Tomas Lang. 2004. Digital arithmetic. Elsevier.

[10]

Milo? D Ercegovac and Robert McIlhenny. 2010. Design and FPGA implementation of radix-10 combined division/square root algorithm with limited precision primitives. In IEEE Asilomar Conference on Signals, Systems and Computers. 87--91.

[11]

Mingze Gao, Qian Wang, Akshaya S Kankanhalli Nagendra, and Gang Qu. 2017. A novel data format for approximate arithmetic computing. In 2017 22nd Asia and South Pacific Design Automation Conference (ASP-DAC). 390--395.

Digital Library

[12]

B. Girau and A. Tisserand. 1996. On-Line arithmetic-based reprogrammable hardware implementation of multilayer perceptron back-propagation. In IEEE International Conference on Microelectronics for Neural Networks. 168--175.

[13]

A. Gorji-Sinaki and M.D. Ercegovac. 1981. Design of a digit-slice on-line arithmetic unit. In IEEE Symposium on Computer Arithmetic. 72--80.

[14]

Robert Hamill, John V McCanny, and Richard L Walke. 2000. Online CORDIC algorithm and VLSI architecture for implementing QR-array processors. IEEE Transactions on Signal Processing 48, 2 (2000), 592--598.

Digital Library

[15]

Abdus Sami Hassan, Tooba Arifeen, and Jeong-A Lee. 2020. Data footprint reduction in DNN inference by sensitivity-controlled approximations with online arithmetic. In IEEE International Conference on Digital System Design. 534--541.

[16]

Nicholas J Higham. 2002. Accuracy and Stability of Numerical Algorithms. SIAM.

[17]

Zhijun Huang and Milos D Ercegovac. 2001. FPGA implementation of pipelined on-line scheme for 3-D vector normalization. In IEEE Symposium on Field- Programmable Custom Computing Machines. 61--70.

Digital Library

[18]

Mary Jane Irwin. 1977. An arithmetic unit for on-line computation. PhD Dissertation. University of Illinois at Urbana-Champaign.

[19]

Honglan Jiang, Francisco Javier Hernandez Santiago, Hai Mo, Leibo Liu, and Jie Han. 2020. Approximate arithmetic circuits: A survey, characterization, and recent applications. Proc. IEEE (2020).

[20]

Georgina Binoy Joseph and R Devanathan. 2016. Design and analysis of online arithmetic operators for streaming data in FPGAs. International Journal of Applied Engineering Research 11, 3 (2016), 375--390.

[21]

Georgina Binoy Joseph and R Devanathan. 2018. Algorithms for multiplierless multiple constant multiplication in online arithmetic. Circuits, Systems, and Signal Processing 37, 11 (2018), 5127--5142.

Digital Library

[22]

Georgina Binoy Joseph and R Devanathan. 2018. Computation of design metrics in online arithmetic networks. In IEEE International Conference on Emerging Trends and Innovations In Engineering And Technological Research. 1--4.

[23]

Patrick Judd, Jorge Albericio, Tayler Hetherington, Tor M Aamodt, and Andreas Moshovos. 2016. Stripes: Bit-serial deep neural network computing. In IEEE/ACM International Symposium on Microarchitecture. 1--12.

[24]

He Li, Ameer Abdelhadi, Runbin Shi, Jiliang Zhang, and Qiang Liu. 2021. Adversarial Hardware with Functional and Topological Camouflage. IEEE Transactions on Circuits and Systems-II: Express Briefs (2021).

[25]

He Li, James J Davis, John Wickerson, and George A Constantinides. 2018. Digit elision for arbitrary-accuracy iterative computation. In IEEE Symposium on Computer Arithmetic. 107--114.

[26]

H. Li, J. J. Davis, J. Wickerson, and G. A. Constantinides. 2019. ARCHITECT: Arbitrary-precision hardware with digit elision for efficient iterative compute. IEEE Transactions on Very Large Scale Integration (VLSI) Systems 28, 2 (2019), 516--529.

[27]

He Li, Ian McInerney, James J. Davis, and George A. Constantinides. 2020. Digit stability inference for iterative methods using redundant number representation. IEEE Trans. Comput. (2020).

[28]

Qiang Liu, Jia Liu, Ruoyu Sang, Jiajun Li, Tao Zhang, and Qijun Zhang. 2018. Fast neural network training on FPGA using quasi-newton optimization method. IEEE Transactions on Very Large Scale Integration (VLSI) Systems 26, 8 (2018), 1575--1579.

[29]

Weiqiang Liu, Chongyan Gu, Máire O'Neill, Gang Qu, Paolo Montuschi, and Fabrizio Lombardi. 2020. Security in Approximate Computing and Approximate Computing for Security: Challenges and Opportunities. Proc. IEEE 108, 12 (2020), 2214--2231.

[30]

Robert McIlhenny. 2002. Complex number on-line arithmetic for reconfigurable hardware: Algorithms, implementations, and applications. PhD Dissertation. University of California Los Angeles.

[31]

J-M Muller. 1994. Some characterizations of functions computable in on-line arithmetic. IEEE Trans. Comput. 43, 6 (1994), 752--755.

Digital Library

[32]

James M Ortega and Werner C Rheinboldt. 2000. Iterative solution of nonlinear equations in several variables. SIAM.

[33]

J-A Pineiro, Milos D Ercegovac, and Javier D Bruguera. 2004. Algorithm and architecture for logarithm, exponential, and powering computation. IEEE Trans. Comput. 53, 9 (2004), 1085--1096.

Digital Library

[34]

Sayeh Sharify, Alberto Delmas Lascorz, Kevin Siu, Patrick Judd, and Andreas Moshovos. 2018. Loom: Exploiting weight and activation precisions to accelerate convolutional neural networks. In ACM/ESDA/IEEE Design Automation Conference. 1--6.

Digital Library

[35]

K. Shi, D. Boland, and G. A. Constantinides. 2014. Efficient FPGA implementation of digit parallel online arithmetic operators. In IEEE International Conference on Field Programmable Technology. 115--122.

[36]

Kan Shi, David Boland, Edward Stott, Samuel Bayliss, and George A Constantinides. 2014. Datapath synthesis for overclocking: Online arithmetic for latencyaccuracy trade-offs. In ACM/EDAC/IEEE Design Automation Conference. 1--6.

Digital Library

[37]

Yang Song, Chenlin Meng, Renjie Liao, and Stefano Ermon. 2020. Nonlinear equation solving: A faster alternative to feedforward computation. arXiv preprint arXiv:2002.03629 (2020).

[38]

Milena Svobodova, Edita Pelantova, Marta Pavelka, and Christiane Frougny. 2019. On-line algorithms for multiplication and division in real and complex numeration systems. Discrete Mathematics & Theoretical Computer Science 21, 3 (2019).

[39]

Naofumi Takagi, Tohru Asada, and Shuzo Yajima. 1991. Redundant CORDIC methods with a constant scale factor for sine and cosine computation. IEEE Trans. Comput. 40, 9 (1991), 989--995.

Digital Library

[40]

AF Tenca and MD Ercegovac. 1997. A high-radix multiplier design for variable long-precision computations. In IEEE Asilomar Conference on Signals, Systems and Computers. 1173--1177.

[41]

Alexandre F Tenca, Song Park, and Lo'ai A Tawalbeh. 2006. Carry-save representation is shift-unsafe: The problem and its solution. IEEE Trans. Comput. 55, 5 (2006), 630--635.

Digital Library

[42]

P.K.-G. Tu. 1990. On-line arithmetic algorithms for efficient implementation. PhD Dissertation. University of California Los Angeles.

[43]

P.L.-G. Tu and M.D. Ercegovac. 1991. Gate array implementation of on-line algorithms for floating-point operations. Journal of VLSI Signal Processing 3, 4 (1991), 307--317.

Digital Library

[44]

D. Tullsen and M.D. Ercegovac. 1986. Design and implementation of an on-line algorithm. In SPIE Conference on Real-Time Signal Processing. 92--99.

[45]

Julio Villalba, Tomas Lang, and Javier Hormigo. 2011. Radix-2 multioperand and multiformat streaming online addition. IEEE Trans. Comput. 61, 6 (2011), 790--803.

Digital Library

[46]

Jingcheng Wang, Xiaowei Wang, Charles Eckert, Arun Subramaniyan, Reetuparna Das, David Blaauw, and Dennis Sylvester. 2019. A 28-nm compute SRAM with bit-serial logic/arithmetic operations for programmable in-memory vector computing. IEEE Journal of Solid-State Circuits 55, 1 (2019), 76--86.

[47]

YeWang, Jian Dong, Qian Xu, Zhaojun Lu, and Gang Qu. 2020. Is It Approximate Computing or Malicious Computing?. In ACM International conferences on Great Lakes Symposium on VLSI. 333--338.

[48]

Ye Wang, Qian Xu, Gang Qu, and Jian Dong. 2019. Information hiding behind approximate computation. In Proceedings of the 2019 on Great Lakes Symposium on VLSI. 405--410.

Digital Library

[49]

OsaakiWatanuki and Milos D Ercegovac. 1981. Floating-point on-line arithmetic: Algorithms. In IEEE Symposium on Computer Arithmetic. 81--86.

[50]

OsaakiWatanuki and Milos D. Ercegovac. 1983. Error analysis of certain floatingpoint on-line algorithms. IEEE Trans. Comput. C-32, 4 (1983), 352--358.

[51]

Valerii Zhabin and Valentina Zhabina. 2020. Methods of on-line computation acceleration in systems with direct connection between units. In IEEE International Conference on Dependable Systems, Services and Technologies. 356--362.

Cited By

Shakibhamedan SAminifar AVassallo LTaheriNejad N(2024)Harnessing Approximate Computing for Machine Learning2024 IEEE Computer Society Annual Symposium on VLSI (ISVLSI)10.1109/ISVLSI61997.2024.00110(585-591)Online publication date: 1-Jul-2024
https://doi.org/10.1109/ISVLSI61997.2024.00110
Que HJin YWang TLiu MYang XQiao F(2023)A Survey of Approximate Computing: From Arithmetic Units Design to High-Level ApplicationsJournal of Computer Science and Technology10.1007/s11390-023-2537-y38:2(251-272)Online publication date: 30-Mar-2023
https://dl.acm.org/doi/10.1007/s11390-023-2537-y

Index Terms

Security Enhancements for Approximate Machine Learning
1. Mathematics of computing
  1. Mathematical analysis
    1. Numerical analysis
2. Security and privacy
  1. Security in hardware

Recommendations

Machine Learning: The State of the Art

The two fundamental problems in machine learning (ML) are statistical analysis and algorithm design. The former tells us the principles of the mathematical models that we establish from the observation data. The latter defines the conditions on which ...
Machine Learning for Hardware Security: Opportunities and Risks

Recently, machine learning algorithms have been utilized by system defenders and attackers to secure and attack hardware, respectively. In this work, we investigate the impact of machine learning on hardware security. We explore the defense and attack ...
Complete Interval Arithmetic and Its Implementation on the Computer
Numerical Validation in Current Hardware Architectures

Let $I\textit{I \kern-.55em R}$ be the set of closed and bounded intervals of real numbers. Arithmetic in $I\textit{I \kern-.55em R}$ can be defined via the power set $\textit{I \kern-.54em P}\textit{I \kern-.55em R}$ of real numbers. If divisors ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

GLSVLSI '21: Proceedings of the 2021 Great Lakes Symposium on VLSI

June 2021

504 pages

ISBN:9781450383936

DOI:10.1145/3453688

General Chairs:
Yiran Chen
Duke University, USA
,
Victor Zhirnov
Semiconductor Research Corporation, USA
,
Program Chairs:
Avesta Sasan
George Mason University, USA
,
Ioannis Savidis
Drexel University, USA

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGDA: ACM Special Interest Group on Design Automation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 June 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Data Availability

GLSVLSI talk https://dl.acm.org/doi/10.1145/3453688.3461753#GLSVLSI21-vlsi44s.mp4

Funding Sources

Hunan Natural Science Foundation for Distinguished Young Scholars
National Natural Science Foundation of China

Conference

GLSVLSI '21

Sponsor:

SIGDA

GLSVLSI '21: Great Lakes Symposium on VLSI 2021

June 22 - 25, 2021

Virtual Event, USA

Acceptance Rates

Overall Acceptance Rate 312 of 1,156 submissions, 27%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
287
Total Downloads

Downloads (Last 12 months)77
Downloads (Last 6 weeks)11

Reflects downloads up to 15 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Shakibhamedan SAminifar AVassallo LTaheriNejad N(2024)Harnessing Approximate Computing for Machine Learning2024 IEEE Computer Society Annual Symposium on VLSI (ISVLSI)10.1109/ISVLSI61997.2024.00110(585-591)Online publication date: 1-Jul-2024
https://doi.org/10.1109/ISVLSI61997.2024.00110
Que HJin YWang TLiu MYang XQiao F(2023)A Survey of Approximate Computing: From Arithmetic Units Design to High-Level ApplicationsJournal of Computer Science and Technology10.1007/s11390-023-2537-y38:2(251-272)Online publication date: 30-Mar-2023
https://dl.acm.org/doi/10.1007/s11390-023-2537-y

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents