Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
Springer Nature Link
Log in

Research of ReLU output device in ternary optical computer based on parallel fully connected layer

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

When implementing fully connected layer computation on electronic computer (EC), slow computing speed and high power consumption result in the inefficiency of the overall process. The ternary optical computer (TOC) platform with enormous data bits and reconfigurable processor can solve the inefficiency of full-connection computation on EC. In this paper, we design the parallel scheme of fully connected layer operations based on TOC and the rectified linear unit (ReLU) output device to achieve the computation of nonlinear fully connected layer. Furthermore, we also use the ReLU output device and the positive and negative discriminator of modified signed digit (MSD) data to design the ReLU judgment output device, which can realize the operation of the ReLU activation function in one step. The operations of the nonlinear fully connected layer with ReLU function on TOC can reduce the amount of computation by one magnitude and have lower power consumption by experimentally verifying the accuracy of operations and analyzing hardware resources and clock cycles. Through comparison find that TOC consumes fewer data bits, while improving the calculation speed by approximately 10%.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Availability of data and materials

Not applicable.

References

  1. An X, Zhu X, Xiao Y, et al (2021) Partial fc: training 10 million identities on a single machine. 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW) pp 1445–1449

  2. Bao J, Jin Y, Cai C (2007) An experiment for ternary optical computer hundred-bit encoder. Comput Technol Dev 17(2):19–22

    Google Scholar 

  3. Basha SHS, Dubey SR, Pulabaigari V et al (2020) Impact of fully connected layers on performance of convolutional neural networks for image classification. Neurocomputing 378:112–119. https://doi.org/10.1016/j.neucom.2019.10.008

    Article  Google Scholar 

  4. Coulthard B, Taylor BJ (2022) Natural language processing to identify case factors in child protection court proceedings. Methodol Innov 15(3):222–235. https://doi.org/10.1177/20597991221115967

    Article  Google Scholar 

  5. Hu X, Jin Y, Shan O (2014) A 40-bit multiplication routine of ternary optical computer. J Shanghai Univ 20(5):645–657

    Google Scholar 

  6. Huan Z, Qing Z, Jiyan Y (2021) Overview of the development of activation function and its nature analysis. J Xihua Univ 40(4):1–10. https://doi.org/10.12198/j.issn.1673-159X.3761

    Article  Google Scholar 

  7. Jin Y (2011) Draw near optical computer. J Shanghai Univ 17(4):401–411

    Google Scholar 

  8. Jin Y, Shen Y, Peng J et al (2010) Principles and construction of MSD adder in ternary optical computer. Sci China Inf Sci 53(11):2159–2168. https://doi.org/10.1007/s11432-010-4091-9

    Article  Google Scholar 

  9. Jin Y, Shen Y, Peng J et al (2013) Vector matrix multiplication in ternary optical computer. Int J Numer Anal Model 9(2):401–409

    Google Scholar 

  10. Jin Y, Wang Z, Liu Y et al (2019) Ternary optical computer. Chin J Nat 41(3):207–218

    Google Scholar 

  11. Junjie P, Youyi F, Xiaofeng Z et al (2018) Implementation of DFT application on ternary optical computer. Opt Commun 410:424–430. https://doi.org/10.1016/j.optcom.2017.10.033

    Article  Google Scholar 

  12. Kai S, LiPing Y (2016) The symmetric MSD encoder for one-step adder of ternary optical computer. Opt Commun 372:221–228. https://doi.org/10.1016/j.optcom.2016.04.034

    Article  Google Scholar 

  13. Kai S, Liping Y (2017) Control mechanism of double-rotator-structure ternary optical computer. Opt Commun 387:338–349

    Article  Google Scholar 

  14. Kai S, Yi J, Shan O et al (2012) Reconfigurable ternary optical processor with double rotator structure. Opt Precision Eng 20(9):1890–1898

    Article  Google Scholar 

  15. Li P, Wang B, Zhang L (2021) Virtual fully-connected layer: Training a large-scale face recognition dataset with limited computational resources. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp 13310–13319, https://doi.org/10.1109/CVPR46437.2021.01311

  16. Lu CT, Wang LL, Shen JH et al (2021) Image enhancement using deep-learning fully connected neural network mean filter. J Supercomput 77(3):3144–3164. https://doi.org/10.1007/s11227-020-03389-6

    Article  Google Scholar 

  17. Ma P, Petridis S, Pantic M (2022) Visual speech recognition for multiple languages in the wild. Nat Mach Intell 4(11):930–939. https://doi.org/10.1038/s42256-022-00550-z

    Article  Google Scholar 

  18. Mei L (2015) Multi-digit MSD multiplication algorithm on ternary optical computer and operation analysis. J Xi’an Technol Univ 35(12):1020–1025

    Google Scholar 

  19. Mei L (2016) Design and realization of ternary logic optical computer lab system. Comput Technol Dev 26(10):192–195

    Google Scholar 

  20. Schmidt-Hieber J (2017) Nonparametric regression using deep neural networks with relu activation function. Ann Stat 48(4):1875–1897

    MathSciNet  Google Scholar 

  21. Shen Y, Lei P, Yi J et al (2012) One-step binary MSD adder for ternary optical computer. Sci Sin 42(7):869–881

    Google Scholar 

  22. Song K (2016) Design of one-step carry-free adder with three-rotator structure. Opt Precision Eng 24(2):438–447

    Article  MathSciNet  Google Scholar 

  23. Song K (2018) Drstop control information generation method based on row operation unit. Tien Tzu Hsueh Pao/Acta Electronica Sinica 46(5):1133–1138

    MathSciNet  Google Scholar 

  24. Song K, Chen G, Jin Q et al (2019) Design of MSD multiplier for ternary optical computer processor based on minimum module. Opt Commun 448:33–42

    Article  Google Scholar 

  25. Song K, Zhang Y, Yan L et al (2020) Research on fully parallel matrix algorithm of ternary optical computer for shortest path problem. Appl Opt 59(16):4953–4963

    Article  Google Scholar 

  26. Valiant L (1990) A bridging model for parallel computation. Commun ACM 33(8):103–111

    Article  Google Scholar 

  27. Varshney M, Singh P (2021) Optimizing nonlinear activation function for convolutional neural networks. Signal Image Video Process 15(6):1323–1330. https://doi.org/10.1007/s11760-021-01863-z

    Article  Google Scholar 

  28. Wang SH, Muhammad K, Hong J et al (2020) Alcoholism identification via convolutional neural network based on parametric relu, dropout, and batch normalization. Neural Comput Appl 32(3):665–680. https://doi.org/10.1007/s00521-018-3924-0

    Article  Google Scholar 

  29. Wang Z, Jin Y, Shan O (2015) Implementation and improvement of sd11 decoder. J Shanghai Univ 21(2):109–120

    Article  Google Scholar 

  30. Wang Z, Shen Y, Li S et al (2022) A fine-grained fast parallel genetic algorithm based on a ternary optical computer for solving traveling salesman problem. J Supercomput 79(5):4760–4790. https://doi.org/10.1007/s11227-022-04813-9

    Article  Google Scholar 

  31. Wang Z, Shen Y, Li S et al (2022) Gradient rectified parameter unit of the fully connected layer in convolutional neural networks. Knowl Based Syst 248:108797.1-108797.12. https://doi.org/10.1016/j.knosys.2022.108797

    Article  Google Scholar 

  32. Yan J, Jin Y, Sun H (2004) Study on the feasibility of coding and decoding multi-bit ternary optical signal used in ternary optical computer. Comput Eng 30(14):175–177

    Google Scholar 

  33. Yan J, Jin Y, Zuo K (2008) Decrease-radix design principle for carrying/borrowing free multi-valued and application in ternary optical computer. Sci China Ser F Inf Sci 51(10):1415–1426

    Article  MathSciNet  Google Scholar 

  34. Ye C, Peng J, Kong S (2021) Implementation of wavelet transform on optical computer. Opt Commun 486:126761–126770. https://doi.org/10.1016/j.optcom.2021.126761

    Article  Google Scholar 

  35. Yue Z, Gao F, Xiong Q et al (2021) A novel semi-supervised convolutional neural network method for synthetic aperture radar image recognition. Cogn Comput 13(4):795–806. https://doi.org/10.1007/s12559-019-09639-x

    Article  Google Scholar 

  36. Zhang H, Zhou J, Zhang S et al (2017) Design and implementation of positive and negative discriminator of MSD data for ternary optical processor. Comput Res Dev 54(6):1391–1404. https://doi.org/10.7544/issn1000-1239.2017.20170093

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgments

The authors would like to express their sincere gratefulness to the TOC team, School of Computer Engineering and Science, Shanghai University, for providing the optical platform and giving many inspired ideas to the paper.

Funding

This work was supported by the National Natural Science Foundation of China (NSFC) (62262022, 62002117), and the Natural Science Foundation of Jiangxi Province (20232BAB202026, 20224BAB202021).

Author information

Authors and Affiliations

  1. School of Information Engineering, East China Jiao tong University, Shuanggang Road, Nanchang, 330013, China

    Kai Song, Huaqiong Ma & Haiming Zhang

  2. Software School, East China Jiao tong University, Shuanggang Road, Nanchang, 330013, China

    Liping Yan

Authors
  1. Kai Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Huaqiong Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Haiming Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Liping Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Kai Song and Huaqiong Ma proposed innovative idea, analyzed the feasibility, designed research methods, verified the correctness of the experiment and wrote original draft. Haiming Zhang and Liping Yan assisted in experimental operation, data analysis and the improvement of English quality.

Corresponding author

Correspondence to Kai Song.

Ethics declarations

Conflict of interest

The authors declare that there no competing interest regarding the publication of this paper.

Ethical approval

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Song, K., Ma, H., Zhang, H. et al. Research of ReLU output device in ternary optical computer based on parallel fully connected layer. J Supercomput 80, 7269–7292 (2024). https://doi.org/10.1007/s11227-023-05737-8

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-023-05737-8

Keywords

Search

Navigation