A class of pseudoinverse-free greedy block nonlinear Kaczmarz methods for nonlinear systems of equations

Ying Lv; Li-Li Xing; Wen-Di Bao; Wei-Guo Li; Zhi-Wei Guo; Ying Lv; Li-Li Xing; Wen-Di Bao; Wei-Guo Li; Zhi-Wei Guo

doi:10.3934/nhm.2024014

Networks and Heterogeneous Media

2024, Volume 19, Issue 1: 305-323. doi: 10.3934/nhm.2024014

Previous Article Next Article

Research article Special Issues

A class of pseudoinverse-free greedy block nonlinear Kaczmarz methods for nonlinear systems of equations

College of Science, China University of Petroleum, Qingdao PR China

Received: 11 December 2023 Revised: 14 March 2024 Accepted: 21 March 2024 Published: 27 March 2024

In this research, we constructed a class of nonlinear greedy average block Kaczmarz methods to solve nonlinear problems without computing the Moore-Penrose pseudoinverse of the Jacobian matrix. These kinds of methods adopt the average technique of the Gaussian Kaczmarz method and combine the greedy strategy, which greatly reduces the amount of computation. The local convergence analysis and numerical experiments of the proposed methods are given. The numerical results show the effectiveness of the proposed methods.
- nonlinear equations,
- average technique,
- nonlinear Kaczmarz algorithm,
- block nonlinear Kaczmarz algorithm
Citation: Ying Lv, Li-Li Xing, Wen-Di Bao, Wei-Guo Li, Zhi-Wei Guo. A class of pseudoinverse-free greedy block nonlinear Kaczmarz methods for nonlinear systems of equations[J]. Networks and Heterogeneous Media, 2024, 19(1): 305-323. doi: 10.3934/nhm.2024014

Related Papers:

Abstract

In this research, we constructed a class of nonlinear greedy average block Kaczmarz methods to solve nonlinear problems without computing the Moore-Penrose pseudoinverse of the Jacobian matrix. These kinds of methods adopt the average technique of the Gaussian Kaczmarz method and combine the greedy strategy, which greatly reduces the amount of computation. The local convergence analysis and numerical experiments of the proposed methods are given. The numerical results show the effectiveness of the proposed methods.

References

[1]	H. B. An, Z. Z. Bai, Broyden method for nonlinear equation in several variables, Mathematica Numerica Sinica (Chinese Journal), 26 (2004), 385–400.
[2]	H. B. An, Z. Z. Bai, Directional secant method for nonlinear equations, J. Comput. Appl. Math., 175 (2005), 291–304. https://doi.org/10.1016/j.cam.2004.05.013 doi: 10.1016/j.cam.2004.05.013
[3]	Z. Z. Bai, L. Wang, On convergence rates of Kaczmarz-type methods with different selection rules of working rows, Appl. Numer. Math., 186 (2023), 289–319. https://doi.org/10.1016/j.apnum.2023.01.013 doi: 10.1016/j.apnum.2023.01.013
[4]	Z. Z. Bai, W. T. Wu, On greedy randomized Kaczmarz method for solving large sparse linear systems, SIAM J. Sci. Comput., 40 (2018), A592–A606. https://doi.org/10.1137/17M1137747 doi: 10.1137/17M1137747
[5]	J. Q. Chen, Z. D. Huang, On the error estimate of the randomized double block Kaczmarz method, Appl. Math. Comput., 370 (2020), 124907. https://doi.org/10.1016/j.amc.2019.124907 doi: 10.1016/j.amc.2019.124907
[6]	Q. P. Chen, W. R. Hao, A homotopy training algorithm for fully connected neural networks, P. Roy. Soc. A-Math. Phy., 475 (2019), 20190662. https://doi.org/10.1098/rspa.2019.0662 doi: 10.1098/rspa.2019.0662
[7]	K. Du, W. T. Si, X. H. Sun, Randomized extended average block Kaczmarz for solving least squares, SIAM J. Sci. Comput., 42 (2020), A3541–A3559. https://doi.org/10.1137/20M1312629 doi: 10.1137/20M1312629
[8]	K. Du, X. H. Sun, Pseudoinverse-free randomized block iterative algorithms for consistent and inconsistent linear systems, arXiv: 2011.10353 [preprint], (2020), [cited 2024 March 27]. Available from: https://doi.org/10.48550/arXiv.2011.10353
[9]	K. Du, X. H. Sun, A doubly stochastic block Gauss–Seidel algorithm for solving linear equations, Appl. Math. Comput., 408 (2021), 126373. https://doi.org/10.1016/j.amc.2021.126373 doi: 10.1016/j.amc.2021.126373
[10]	Y. C. Eldar, D. Needell, Acceleration of randomized Kaczmarz method via the Johnson-Lindenstrauss lemma, Numer. Algorithms, 58 (2011), 163–177. https://doi.org/10.1007/s11075-011-9451-z doi: 10.1007/s11075-011-9451-z
[11]	T. Elfving, Block-iterative methods for consistent and inconsistent linear equations, Numer. Math., 35 (1980), 1–12. https://doi.org/10.1007/BF01396365 doi: 10.1007/BF01396365
[12]	M. A. Gomes-Ruggiero, J. M. Martínez, A. C. Moretti, Comparing algorithms for solving sparse nonlinear systems of equations, SIAM J. Sci. Stat. Comput., 13 (1992), 459–483. https://doi.org/10.1137/0913025 doi: 10.1137/0913025
[13]	R. M. Gower, P. Richtárik, Randomized iterative methods for linear systems, SIAM J. Matrix Anal. A., 36 (2015), 1660–1690. https://doi.org/10.1137/15M1025487 doi: 10.1137/15M1025487
[14]	S. Karczmarz, Angenaherte auflosung von systemen linearer glei-chungen, Bull. Int. Acad. Pol. Sic. Let., Cl. Sci. Math. Nat., 35 (1937), 355–357.
[15]	K. Kawaguchi, Deep learning without poor local minima, In: D. D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, R. Garnett, Advances in neural information processing systems, New York: Curran Associates Inc., 29 (2016), 586–594.
[16]	J. Liu, S. J. Wright, An accelerated randomized Kaczmarz algorithm, Math. Comput., 85 (2016), 153–178. http://dx.doi.org/10.1090/mcom/2971 doi: 10.1090/mcom/2971
[17]	L. Lukšan, Hybrid methods for large sparse nonlinear least squares, J. Optimiz. Theory App., 89 (1996), 575–595.
[18]	A. Ma, D. Needell, A. Ramdas, Convergence properties of the randomized extended Gauss–Seidel and Kaczmarz methods, SIAM J. Matrix Anal. A., 36 (2015), 1590–1604. https://doi.org/10.1137/15M1014425 doi: 10.1137/15M1014425
[19]	L. Mirsky, Symmetric gauge functions and unitarily invariant norms, Q. J. Math., 11 (1960), 50–59. https://doi.org/10.1093/qmath/11.1.50 doi: 10.1093/qmath/11.1.50
[20]	J. J. Moré, B. S. Garbow, K. E. Hillstrom, Testing unconstrained optimization software, ACM T. Math. Software, 7 (1981), 17–41.
[21]	M. S. Morshed, M. S. Islam, M. Noor-E-Alam, Accelerated sampling Kaczmarz Motzkin algorithm for the linear feasibility problem, J. Global Optim., 77 (2020), 361–382. https://doi.org/10.1007/s10898-019-00850-6 doi: 10.1007/s10898-019-00850-6
[22]	I. Necoara, Faster randomized block Kaczmarz algorithms, SIAM J. Matrix Anal. A., 40 (2019), 1425–1452. https://doi.org/10.1137/19M1251643 doi: 10.1137/19M1251643
[23]	D. Needell, Randomized Kaczmarz solver for noisy linear systems, BIT, 50 (2010), 395–403. https://doi.org/10.1007/s10543-010-0265-5 doi: 10.1007/s10543-010-0265-5
[24]	D. Needell, J. A. Tropp, Paved with good intentions: Analysis of a randomized block Kaczmarz method, Linear Algebra Appl., 441 (2014), 199–221. https://doi.org/10.1016/j.laa.2012.12.022 doi: 10.1016/j.laa.2012.12.022
[25]	D. Needell, R. Zhao, A. Zouzias, Randomized block Kaczmarz method with projection for solving least squares, Linear Algebra Appl, 484 (2015), 322–343. https://doi.org/10.1016/j.laa.2015.06.027 doi: 10.1016/j.laa.2015.06.027
[26]	J. M. Ortega, W. C. Rheinboldt, Iterative Solution of Nonlinear Equations in Several Variables, Philadelphia: Society for Industrial and Applied Mathematics, 2000. https://doi.org/10.1137/1.9780898719468
[27]	T. Strohmer, R. Vershynin, A randomized Kaczmarz algorithm with exponential convergence, J. Fourier Anal. Appl., 15 (2009) 262–278. https://doi.org/10.1007/s00041-008-9030-4 doi: 10.1007/s00041-008-9030-4
[28]	Q. F. Wang, W. G. Li, W. D. Bao, X. Q. Gao, Nonlinear Kaczmarz algorithms and their convergence, J. Comput. Appl. Math., 399 (2022), 113720. https://doi.org/10.1016/j.cam.2021.113720 doi: 10.1016/j.cam.2021.113720
[29]	X. Z. Wang, M. L. Che, C. X. Mo, Y. M. Wei, Solving the system of nonsingular tensor equations via randomized Kaczmarz-like method, J. Comput. Appl. Math., 421 (2023), 114856. https://doi.org/10.1016/j.cam.2022.114856 doi: 10.1016/j.cam.2022.114856
[30]	A. Q. Xiao, J. F. Yin, N. Zheng, On fast greedy block Kaczmarz methods for solving large consistent linear systems, Comput. Appl. Math., 42 (2023), 119. https://doi.org/10.1007/s40314-023-02232-x doi: 10.1007/s40314-023-02232-x
[31]	R. Yuan, A. Lazaric, R. M. Gower, Sketched Newton-Raphson, SIAM J. Optimiz., 32 (2022), 1555–1583. https://doi.org/10.1137/21M139788X doi: 10.1137/21M139788X
[32]	J. H. Zhang, Y. Q. Wang, J. Zhao, On maximum residual nonlinear Kaczmarz-type algorithms for large nonlinear systems of equations, J. Comput. Appl. Math., 425 (2023), 115065. https://doi.org/10.1016/j.cam.2023.115065 doi: 10.1016/j.cam.2023.115065
[33]	Y. J. Zhang, H. Y. Li, Greedy capped nonlinear Kaczmarz methods, arXiv: 2210.00653 [preprint], (2022), [cited 2024 March 27]. Available from: https://doi.org/10.48550/arXiv.2210.00653
[34]	Y. J. Zhang, H. Y. Li, L. Tang, Greedy randomized sampling nonlinear Kaczmarz methods, arXiv: 2209.06082 [preprint], (2022), [cited 2024 March 27]. Available from: https://doi.org/10.48550/arXiv.2209.06082
[35]	A. Zouzias, N. M. Freris, Randomized extended Kaczmarz for solving least squares, SIAM J. Matrix Anal. A., 34 (2013), 773–793. https://doi.org/10.1137/120889897 doi: 10.1137/120889897

Reader Comments

Your name:*

Email:*
© 2024 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)