research-article

Open access

Low-precision Logarithmic Number Systems: Beyond Base-2

Authors:

Syed Asad Alam,

David GreggAuthors Info & Claims

ACM Transactions on Architecture and Code Optimization (TACO), Volume 18, Issue 4

Article No.: 47, Pages 1 - 25

https://doi.org/10.1145/3461699

Published: 17 July 2021 Publication History

All formats PDF

Abstract

Logarithmic number systems (LNS) are used to represent real numbers in many applications using a constant base raised to a fixed-point exponent making its distribution exponential. This greatly simplifies hardware multiply, divide, and square root. LNS with base-2 is most common, but in this article, we show that for low-precision LNS the choice of base has a significant impact.

We make four main contributions. First, LNS is not closed under addition and subtraction, so the result is approximate. We show that choosing a suitable base can manipulate the distribution to reduce the average error. Second, we show that low-precision LNS addition and subtraction can be implemented efficiently in logic rather than commonly used ROM lookup tables, the complexity of which can be reduced by an appropriate choice of base. A similar effect is shown where the result of arithmetic has greater precision than the input. Third, where input data from external sources is not expected to be in LNS, we can reduce the conversion error by selecting a LNS base to match the expected distribution of the input. Thus, there is no one base that gives the global optimum, and base selection is a trade-off between different factors. Fourth, we show that circuits realized in LNS require lower area and power consumption for short word lengths.

References

[1]

S. A. Alam and D. Gregg. 2020. Beyond base-2 logarithmic number systems (WiP Paper). In Proceedings of the ACM SIGPLAN/SIGBED International Conference on Language, Compilers, and Tools for Embedded Systems. ACM.

[2]

S. A. Alam and O. Gustafsson. 2014. Design of finite word length linear-phase FIR filters in the logarithmic number system domain. VLSI Des. 2014 (2014), 14.

Digital Library

[3]

M. Arnold, E. Chester, and C. Johnson. 2020. Training neural nets using only an approximate tableless LNS ALU. In Proceedings of the IEEE International Application-Specific System Architecture Processors Conference. 69–72.

[4]

M. G. Arnold. 2002. Reduced power consumption for MPEG decoding with LNS. In Proceedings of the IEEE International Application-Specific System Architecture Processors Conference.65–67.

Digital Library

[5]

M. G. Arnold, T. A. Bailey, J. R. Cowles, and M. D. Winkel. 1998. Arithmetic co-transformations in the real and complex logarithmic number systems. IEEE Trans. Comput. 47, 7 (1998), 777–786.

Digital Library

[6]

C. Basetas, I. Kouretas, and V. Paliouras. 2007. Low-power digital filtering based on the logarithmic number system. In Proceedings of the International Workshop on Power Timing Modeling Optimization Simulation. Vol. 4644. 546–555.

[7]

BLIF 1992. Berkeley logic interchange format (BLIF). University of California, Berkeley. Retrieved from http://www.cs.columbia.edu/ cs6861/sis/blif/index.html.

[8]

N. Bruschi, A. Garofalo, F. Conti, G. Tagliavini, and D. Rossi. 2020. Enabling mixed-precision quantized neural networks in extreme-edge devices. In Proceedings of the ACM International Conference on Computing Frontiers. 217–220.

[9]

A. Capotondi, M. Rusci, M. Fariselli, and L. Benini. 2020. CMix-NN: Mixed low-precision CNN library for memory-constrained edge devices. IEEE Trans. Circ. Syst. II: Express Briefs 67, 5 (2020), 871–875.

[10]

D. V. Satish Chandra. 1998. Error analysis of FIR filters implemented using logarithmic arithmetic. IEEE Trans. Circ. Syst. II 45, 6 (June 1998), 744–747.

[11]

D. V. S. Chandra. 1998. Error analysis of FIR filters implemented using logarithmic arithmetic. IEEE Trans. Circ. Syst. II 45, 6 (1998), 744–747.

[12]

J. Chen and X. Ran. 2019. Deep learning with edge computing: A review. Proc. IEEE 107 (2019), 1655–1674. Issue 8.

[13]

Wai-Kai Chen (Ed.). 2003. Memory, Microprocessor, and ASIC. CRC Press, Boca Raton, FL.

[14]

E. Chung and J. Fowers et al.2018. Serving DNNs in real time at datacenter scale with project brainwave. IEEE Micro 38, 2 (Mar. 2018), 8–20.

[15]

F. de Dinechin and A. Tisserand. 2001. Some improvements on multipartite table methods. In Proceedings of the IEEE Symposium on Computer Arithmetic. 128–135.

[16]

V. S. Dimitrov, G. A. Jullien, and W. C. Miller. 1999. Theory and applications of the double-base number system. IEEE Trans. Comput. 48, 10 (1999), 1098–1106.

Digital Library

[17]

Z. G. Feng and K. F. C. Yiu. 2010. A new formulation for the design of FIR filters with reduced hardware implementation complexity. In Proceedings of the IEEE International Conference on Green Circuits Systems.242–246.

[18]

A. Garofalo, M. Rusci, F. Conti, D. Rossi, and L. Benini. 2019. PULP-NN: accelerating quantized neural networks on parallel ultra-low-power RISC-V processors. Philos. Trans. Royal Soc. A 378, 2164 (Dec. 2019).

[19]

I. Hubara, M. Courbariaux, D. Soudry, R. El-Yaniv, and Y. Bengio. 2017. Quantized neural networks: Training neural networks with low-precision weights and activations. ACM J. Mach. Learn. Res. 18, 1 (Jan. 2017), 6869–6898.

[20]

K. Johansson, O. Gustafsson, and L. Wanhammar. 2008. Implementation of elementary functions for logarithmic number systems. IET Comput. Digit. Tech. 2, 4 (June 2008), 295–304.

[21]

N. G. Kingsbury and P. J. W. Rayner. 1971. Digital filtering using logarithmic arithmetic. Electron. Lett. 7, 2 (Jan. 1971), 56–58.

[22]

I. Kouretas, Ch. Basetas, and V. Paliouras. 2013. Low-power logarithmic number system addition/subtraction and their impact on digital filters. IEEE Trans. Comput. 62, 11 (2013), 2196–2209.

Digital Library

[23]

I. Kouretas and V. Paliouras. 2018. Logarithmic number system for deep learning. In Proceedings of the International Conference on Modern Circuits and Systems Technologies. 1–4.

[24]

V. B. Lawrence and A. C. Salazar. 1980. Finite precision design of linear-phase FIR filters. Bell Syst. Tech. J. 59, 9 (Nov. 1980), 1575–1598.

[25]

E. H. Lee, D. Miyashita, E. Chai, B. Murmann, and S. S. Wong. 2017. LogNet: Energy-efficient neural networks using logarithmic computation. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing.5900–5904.

[26]

A. M. Mansour, A. M. El-Sawy, M. S. Aziz, and A. T. Sayed. 2015. A new hardware implementation of base 2 logarithm for FPGA. Int. J. Signal Proc. Syst. 3, 2 (2015), 171–181.

[27]

Alan Mishchenko. 2005. ABC: A System for Sequential Synthesis and Verification. Retrieved from https://people.eecs.berkeley.edu/ alanmi/abc/.

[28]

J. N. Mitchell. 1962. Computer multiplication and division using binary logarithms. IRE Trans. Electron. Comp. EC-11, 4 (1962), 512–517.

[29]

Sanjit K. Mitra. 2006. Digital Signal Processing. TATA McGraw-Hill, University of California, Santa Barbara, CA.

[30]

D. Miyashita, E. H. Lee, and B. Murmann. 2016. Convolutional neural networks using logarithmic data representation. Retrieved from https://arXiv:cs.NE/1603.01025v2.

[31]

B. Nam, H. Kim, and H. Yoo. 2008. Power and area-efficient unified computation of vector and elementary functions for handheld 3D graphics systems. IEEE Trans. Comput. 57, 4 (2008), 490–504.

Digital Library

[32]

V. Paliouras and T. Stouraitis. 2000. Logarithmic number system for low-power arithmetic. In Proceedings of the International Workshop on Power Timing Modeling Optimization Simulation, Vol. 1918. 285–294.

[33]

M. Rastegari, V. Ordonez, J. Redmon, and A. Farhadi. 2016. XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks. Retrieved from https://arXiv:cs.CV/1603.05279v4.

[34]

M. Satyanarayanan. 2017. The emergence of edge computing. Computer 50, 1 (Jan. 2017), 30–39.

[35]

A. S. Sedra, K. C. Smith, T. C. Carusone, and V. Gaudet. 2019. Microelectronic Circuits (8th ed.). Oxford University Press.

[36]

W. Shi, J. Cao, Q. Zhang, Y. Li, and L. Xu. 2016. Edge computing: Vision and challenges. IEEE Internet Things J. 3 (2016), 637–646. Issue 5.

[37]

G. Sicuranza. 1982. On the accuracy of 2-D digital filter realizations using logarithmic number systems. In Proc. IEEE Int. Conf. Acoust. Speech Signal Process., Vol. 7. 48–51.

[38]

T. Stouraitis and V. Paliouras. 2002. Considering the alternatives in low-power design. IEEE Circ. Syst. Mag. 17, 4 (Aug. 2002), 22–29.

[39]

X. Sun, J. Choi, C. Y Chen, N. Wang, S. Venkataramani, V. Srinivasan, X. Cui, W. Zhang, and K. Gopalakrishnan. 2019. Hybrid 8-bit floating point (HFP8) training and inference for deep neural networks. In Proceedings of the Conference on Advances in Neural Information Processing Systems.4901–4910.

[40]

E.E. Swartzlander Jr. and A.G. Alexopoulos. 1975. The sign/logarithm number system. IEEE Trans. Comput. C-24, 12 (Dec. 1975), 1238–1242.

Digital Library

[41]

J. N. Coleman et al.2008. The european logarithmic microprocesor. IEEE Trans. Comput. 57, 4 (Apr. 2008), 532–546.

Digital Library

[42]

Sebastian Vogel, Mengyu Liang, Andre Guntoro, Walter Stechele, and Gerd Ascheid. 2018. Efficient hardware acceleration of CNNs using logarithmic data representation with arbitrary log-base. In Proceedings of the IEEE/ACM Conference on Computer-Aided Design (ICCAD’18). ACM, New York, NY, Article 9, 8 pages.

Digital Library

[43]

N. Wang, J. Choi, D. Brand, C.-Y. Chen, and K. Gopalakrishnan. 2018. Training deep neural networks with 8-bit floating point numbers. In Proceedings of the Conference on Advances in Neural Information Processing Systems.7675–7684.

[44]

Y. Wang, J. Lin, and Z. Wang. 2018. An energy-efficient architecture for binary weight convolutional neural networks. IEEE Trans. VLSI Syst. 26, 2 (Feb. 2018), 280–293.

Digital Library

[45]

B-.D. Yang and L.-S. Kim. 2003. A low-power charge-recycling ROM architecture. IEEE Trans. VLSI Syst. 4, 4 (2003), 590–600.

Digital Library

[46]

M. Zhu, Y. Ha, C. Gu, and L. Gao. 2016. An optimized logarithmic converter with equal distribution of relative errors. IEEE Trans. Circ. Syst. II: Express Briefs 63, 9 (2016), 848–852.

Cited By

Keilbart CGao YChua MMatthews EWilton SShannon L(2024)Designing an IEEE-Compliant FPU that Supports Configurable Precision for Soft ProcessorsACM Transactions on Reconfigurable Technology and Systems10.1145/365003617:2(1-32)Online publication date: 30-Apr-2024
https://dl.acm.org/doi/10.1145/3650036
Dimitrov VFord RImbert LMadanayake AUdayanga NWray W(2024)Multiple-base Logarithmic Quantization and Application in Reduced Precision AI Computations2024 IEEE 31st Symposium on Computer Arithmetic (ARITH)10.1109/ARITH61463.2024.00017(48-51)Online publication date: 10-Jun-2024
https://doi.org/10.1109/ARITH61463.2024.00017
Karim MFall C(2023)Unconventional Arithmetic CircuitsEncyclopedia of Materials: Electronics10.1016/B978-0-12-819728-8.00063-2(519-528)Online publication date: 2023
https://doi.org/10.1016/B978-0-12-819728-8.00063-2
Show More Cited By

Index Terms

Low-precision Logarithmic Number Systems: Beyond Base-2

Recommendations

Beyond Base-2 Logarithmic Number Systems (WiP Paper)
LCTES '20: The 21st ACM SIGPLAN/SIGBED Conference on Languages, Compilers, and Tools for Embedded Systems

Logarithmic number systems (LNS) reduce hardware complexity for multiplication and division in embedded systems, at the cost of more complicated addition and subtraction. Existing LNS typically use base-2, meaning that representable numbers are some (...
A complex-number multiplier using radix-4 digits
ARITH '95: Proceedings of the 12th Symposium on Computer Arithmetic

This paper describes the design of a 16/spl times/16 complex-number multiplier developed as part of the arithmetic datapath of a complex-number digital signal processor. The complex-number multiplier internally uses binary signed digits for fast ...
A 20 Bit Logarithmic Number System Processor

The architecture and performance of a 20-bit arithmetic processor based on the logarithmic number system (LNS) is described. The processor performed LNS multiplication and division rapidly and with a low hardware complexity. Addition and subtraction in ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Architecture and Code Optimization

ACM Transactions on Architecture and Code Optimization Volume 18, Issue 4

December 2021

497 pages

ISSN:1544-3566

EISSN:1544-3973

DOI:10.1145/3476575

Editor:
David Kaeli
Northeastern University, USA

Issue’s Table of Contents

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 July 2021

Accepted: 01 April 2021

Revised: 01 April 2021

Received: 01 September 2020

Published in TACO Volume 18, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
1,499
Total Downloads

Downloads (Last 12 months)978
Downloads (Last 6 weeks)71

Reflects downloads up to 01 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Keilbart CGao YChua MMatthews EWilton SShannon L(2024)Designing an IEEE-Compliant FPU that Supports Configurable Precision for Soft ProcessorsACM Transactions on Reconfigurable Technology and Systems10.1145/365003617:2(1-32)Online publication date: 30-Apr-2024
https://dl.acm.org/doi/10.1145/3650036
Dimitrov VFord RImbert LMadanayake AUdayanga NWray W(2024)Multiple-base Logarithmic Quantization and Application in Reduced Precision AI Computations2024 IEEE 31st Symposium on Computer Arithmetic (ARITH)10.1109/ARITH61463.2024.00017(48-51)Online publication date: 10-Jun-2024
https://doi.org/10.1109/ARITH61463.2024.00017
Karim MFall C(2023)Unconventional Arithmetic CircuitsEncyclopedia of Materials: Electronics10.1016/B978-0-12-819728-8.00063-2(519-528)Online publication date: 2023
https://doi.org/10.1016/B978-0-12-819728-8.00063-2
de Dinechin FKumm Mde Dinechin FKumm M(2023)Arithmetic for Deep LearningApplication-Specific Arithmetic10.1007/978-3-031-42808-1_24(707-759)Online publication date: 23-Aug-2023
https://doi.org/10.1007/978-3-031-42808-1_24
de Dinechin FKumm Mde Dinechin FKumm M(2023)Number FormatsApplication-Specific Arithmetic10.1007/978-3-031-42808-1_2(33-64)Online publication date: 23-Aug-2023
https://doi.org/10.1007/978-3-031-42808-1_2
Alsuhli GSakellariou VSaleh HAl-Qutayri MMohammad BStouraitis TAlsuhli GSakellariou VSaleh HAl-Qutayri MMohammad BStouraitis T(2023)LNS for DNN ArchitecturesNumber Systems for Deep Neural Network Architectures10.1007/978-3-031-38133-1_4(27-43)Online publication date: 2-Sep-2023
https://doi.org/10.1007/978-3-031-38133-1_4
Christ MDe Dinechin FPetrot F(2022)Low-precision logarithmic arithmetic for neural network accelerators2022 IEEE 33rd International Conference on Application-specific Systems, Architectures and Processors (ASAP)10.1109/ASAP54787.2022.00021(72-79)Online publication date: Jul-2022
https://doi.org/10.1109/ASAP54787.2022.00021
Jahanshahi SMolahosseini AZarandi A(2022)uLog: a software-based approximate logarithmic number system for computations on SIMD processorsThe Journal of Supercomputing10.1007/s11227-022-04713-y79:2(1750-1783)Online publication date: 2-Aug-2022
https://dl.acm.org/doi/10.1007/s11227-022-04713-y

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Media

Figures

Other

Tables

View Issue’s Table of Contents