Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
SpringerLink
Log in

Parallel implementations of randomized vector algorithm for solving large systems of linear equations

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

The results of a parallel implementation of a randomized vector algorithm for solving systems of linear equations are presented in the paper. The solution is represented in the form of a Neumann series. The stochastic method computes this series by sampling only random columns, avoiding multiplication of matrix by matrix and matrix by vector. We consider the case when the matrix is too large to fit in random-access memory (RAM). We use two approaches to solve this problem. In the first approach, the matrix is divided into parts that are distributed among MPI processes and stored in the available RAM of the cluster nodes. In the second approach, the entire matrix is stored on each node’s hard drive, loaded into RAM, and processed in parts. Independent Monte Carlo experiments for random column indices are distributed among MPI processes or OpenMP threads for both approaches to matrix storage. The efficiency of parallel implementations is analyzed. Results are given for a system governed by dense matrices of size \(10^4\) and \(10^5\).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Data availability

All data generated during and/or analysed during the current study are available from the corresponding author on reasonable request.

Code availability

Code is available from the corresponding author on reasonable request.

References

  1. Tondi R, Cavazzoni C, Danecek P, Morelli A (2012) Parallel, large, dense matrix problems: application to 3D sequential integrated inversion of seismological and gravity data. Comput Geosci 48:143–156. https://doi.org/10.1016/j.cageo.2012.05.026

    Article  Google Scholar 

  2. Rostami MW, Olson SD (2019) Fast algorithms for large dense matrices with applications to biofluids. J Comput Phys 394:364–384

    Article  MathSciNet  MATH  Google Scholar 

  3. Carpentieri B, Duff IS, Giraud L(2001) Robust preconditioning of dense problems from electromagnetics. In: Numerical analysis and its applications, NAA 2000, Lecture Notes in computer science 1988:170-178. https://doi.org/10.1007/3-540-45262-1_21

  4. Evans DJ, Hatzopoulos M (1979) A parallel linear system solver. Int J Comput Math 7(3):227–238. https://doi.org/10.1080/00207167908803174

    Article  MathSciNet  MATH  Google Scholar 

  5. Bomhof CW, van der Vorst HA (2000) A parallel linear system solver for circuit simulation problems. Numer Linear Algebra Appl 7:649–665. https://doi.org/10.1002/1099-1506(200010/12)7:7/8<649::AID-NLA217>3.0.CO;2-W

    Article  MathSciNet  MATH  Google Scholar 

  6. Du P, Luszczek P, Dongarra J (2011) High performance dense linear system solver with soft error resilience. In: 2011 IEEE International Conference on Cluster Computing, pp 272-280. https://doi.org/10.1109/CLUSTER.2011.38

  7. Marques M, Quintana-Orti G, Quintana-Orti ES, van de Geijn RA(2009) Solving large dense matrix problems on multi-core processors. In: 2009 IEEE International Symposium on Parallel and Distributed Processing, pp 1-8. https://doi.org/10.1109/IPDPS.2009.5161162

  8. Saad Y (2003) Iterative methods for sparse linear systems. SIAM Press Philadelphia. https://doi.org/10.1137/1.9780898718003

    Article  MATH  Google Scholar 

  9. Gurieva YL, Ilin VP, Perevozkin DV (2017) Deflated Krylov iterations in domain decomposition methods. Domain Decompos Methods Sci Eng XXIII Lecture Notes Comput Sci Eng 116:345–352. https://doi.org/10.1007/978-3-319-52389-7_35

    Article  MathSciNet  MATH  Google Scholar 

  10. Ji H, Li Y (2012) Reusing random walks in Monte Carlo methods for linear systems. Procedia Comput Sci 9:383–392. https://doi.org/10.1016/j.procs.2012.04.041

    Article  Google Scholar 

  11. Bhavsar VC, Isaac JR (1987) Design and analysis of parallel Monte Carlo algorithms. SIAM J Sci Stat Comput 8(1):73–95. https://doi.org/10.1137/0908014

    Article  MathSciNet  MATH  Google Scholar 

  12. Forsythe GE, Leibler RA (1950) Matrix inversion by a Monte Carlo method. Math Comput 4:127–129

    Article  MathSciNet  Google Scholar 

  13. Halton JH (1994) Sequential Monte Carlo techniques for the solution of linear systems. J Sci Comput 9:213–257. https://doi.org/10.1007/BF01578388

    Article  MathSciNet  MATH  Google Scholar 

  14. Todorov V, Ikonomov N, Dimov I, Georgieva R (2018) A new Monte Carlo algorithm for linear algebraic systems based on the “Walk on Equations” algorithm. In: Proceedings of the Federated Conference on Computer Science and Information Systems, pp 257-260. https://doi.org/10.15439/2018F121

  15. Wasow W (1952) A note on the inversion of matrices by random walks. Math Tables Aids Other Comput 6:78–81

    Article  MathSciNet  MATH  Google Scholar 

  16. Tan C J K (2002) Antithetic Monte Carlo linear solver. In: Proceedings of the International Conference on Computational Science, Springer-Verlag, London, pp 383-392

  17. Ji H, Mascagni M, Li Y (2013) Convergence analysis of Markov chain Monte Carlo linear solvers using Ulam-von Neumann algorithm. SIAM J Numer Anal 51(4):2107–2122. https://doi.org/10.1137/130904867

    Article  MathSciNet  MATH  Google Scholar 

  18. Benzi M, Evans TM, Hamilton SP, Lupo Pasini M, Slattery SR (2017) Analysis of monte carlo accelerated iterative methods for sparse linear systems. Numer Linear Algebra Appl 24(3):e2088. https://doi.org/10.1002/nla.2088

    Article  MathSciNet  MATH  Google Scholar 

  19. Dimov I, Alexandrov V, Karaivanova A (2001) Parallel resolvent Monte Carlo algorithms for linear algebra problems. Math Comput Simul 55:25–35. https://doi.org/10.1016/S0378-4754(00)00243-3

    Article  MathSciNet  MATH  Google Scholar 

  20. Magalhães F, Monteiro J, Acebrón JA, Herrero JR (2022) A distributed Monte Carlo based linear algebra solver applied to the analysis of large complex networks. Future Gener Comput Syst 127:320–330. https://doi.org/10.1016/j.future.2021.09.014

    Article  Google Scholar 

  21. Jakovits P, Kromonov I, Srirama S N (2011) Monte Carlo linear system solver using mapreduce. In: 2011 Fourth IEEE International Conference on Utility and Cloud Computing, Melbourne, VIC, Australia, pp 293-299. https://doi.org/10.1109/UCC.2011.47

  22. Sabelfeld KK (2016) Vector Monte Carlo stochastic matrix-based algorithms for large linear systems. Monte Carlo Methods Appl 22(3):259–264. https://doi.org/10.1515/mcma-2016-0112

    Article  MathSciNet  MATH  Google Scholar 

  23. Walker AJ (1974) New fast method for generating discrete random numbers with arbitrary frequency distributions. Electr Lett 10(8):127–128. https://doi.org/10.1049/el:19740097

    Article  Google Scholar 

  24. Smith JC, Jacobson SH (2005) An analysis of the Alias method for discrete random-variate generation. INFORMS J Comput 17(3):321–327. https://doi.org/10.1287/ijoc.1030.0063

    Article  MATH  Google Scholar 

  25. Gantmacher FR (1959) Matrix Theory. Chelsea publishing, New York

    MATH  Google Scholar 

  26. O’Leary DP, Stewart GW, Vandergraft JS (1979) Estimating the largest Eigenvalue of a positive definite matrix. Math Comput 33(148):1289–1292. https://doi.org/10.1287/ijoc.1030.0063

    Article  MathSciNet  MATH  Google Scholar 

  27. Flury BD (1990) Acceptance–rejection sampling made easy. SIAM Rev 32(3):474–476. https://doi.org/10.1137/1032082

    Article  MathSciNet  MATH  Google Scholar 

  28. The Siberian Supercomputer Center of the Siberian Branch of the Russian Academy of Sciences. http://www.sscc.icmmg.nsc.ru. Accessed 27 July (2022)

Download references

Acknowledgements

The work is supported by the Russian Science Foundation, Grant 19-11-00019 in the part of randomized algorithms implementation, and Russian Fund of Basic Research, under Grant 20-51-18009, in the part of stochastic simulation theory development

Author information

Author notes

  1. All authors have contributed equally to this work.

Authors and Affiliations

  1. Institute of Computational Mathematics and Mathematical Geophysics, Russian Academy of Science, Lavrentiev Prosp. 6, Novosibirsk, Russia, 630090

    Karl K. Sabelfeld, Sergey Kireev & Anastasiya Kireeva

  2. Novosibirsk State University, Pirogova Str. 1, Novosibirsk, Russia, 630090

    Karl K. Sabelfeld & Sergey Kireev

Authors
  1. Karl K. Sabelfeld
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sergey Kireev
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Anastasiya Kireeva
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

KKS proposed and developed the randomized vector algorithm for solving large systems of linear equations; designed and directed the project. SK proposed the computational scheme for the implementations of the algorithm; contributed to the developing of the algorithm implementation, analysis of the results and writing of the manuscript. AK implemented a parallel version of the algorithm; wrote the manuscript; contributed to the analysis of the results. All authors discussed the results and commented on the manuscript.

Corresponding author

Correspondence to Anastasiya Kireeva.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethics approval

This article does not contain any studies with human participants or animals performed by any of the authors. The results/data/figures in this manuscript have not been published elsewhere, nor are they under consideration by another publisher.

Consent to participate

Consent to participate was obtained from all individual participants included in the study.

Consent for publication

Consent to participate was obtained from all individual participants included in the study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sabelfeld, K.K., Kireev, S. & Kireeva, A. Parallel implementations of randomized vector algorithm for solving large systems of linear equations. J Supercomput 79, 10555–10569 (2023). https://doi.org/10.1007/s11227-023-05079-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-023-05079-5

Keywords

Search

Navigation