SIAM REVIEW © 2023 Society for Industrial and Applied Mathematics

Vol. 65, No. 4, pp. 1137–1151

The Reflection Method for the

Downloaded 03/28/24 to 115.244.99.163 . Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/terms-privacy

Numerical Solution of Linear

Systems\ast
Margherita Guida\dagger
Carlo Sbordone\dagger

Abstract. We present Cimmino's reflection algorithm for the numerical solution of linear systems,
which starts with an arbitrary point in \BbbR n that gets reflected with respect to the system's
hyperplanes. The centroid of the ensuing collection of points becomes the starting point
of the next iteration. We provide error estimates for the convergence at each step. A
probabilistic argument is also devised to improve this elegant geometrical algorithm. This
subject is an opportunity to show students how linear algebra can interact fruitfully not
only with algebra, geometry, and numerical analysis, but also with probability theory and
methods.

Key words. reflection, linear systems, numerical solution

MSC code. 65F10

DOI. 10.1137/22M1470463

1. Introduction. One of the basic objects of study in scientific computation is

systems of linear equations. Many problems in numerical partial differential equations
(PDEs) and integral equations, rather than eigenvalue problems and optimization,
ultimately reduce to solving such systems with the aid of sparse n \times n matrices and
``very large"" n. Discretization produces such systems.
Enter Gianfranco Cimmino (1908--1989), a former student of Mauro Picone (1885--
1977), who in 1927 founded in Naples the famous Institute of Numerical Computing),
and a younger colleague in Naples of the great mathematician Renato Caccioppoli
(1904--1959). In 1938 Cimmino published [6] an elegant iterative method for the
solution of the system

(1.1) A\bfitx = \bfitb

of n linear equations on \BbbR n , where A is a real n \times n sparse matrix that is initially
supposed nonsingular and \bfitx , \bfitb \in \BbbR n are column vectors.
Iterative methods are commonly used to solve problem (1.1) in practical applica-
tions, especially when in the presence of huge matrices A. These techniques produce
a sequence of successive approximations (\bfitx (k) ) which, under suitable conditions, con-
verge to a solution \bfitx as k \rightarrow \infty .

\ast Received by the editors January 12, 2022; accepted for publication (in revised form) January 18,

2023; published electronically November 7, 2023.

https://doi.org/10.1137/22M1470463
\dagger Department of Mathematics, Universit\à degli Studi di Napoli Federico II, Napoli, Italy, 80126

(maguida@unina.it, sbordone@unina.it).
1137

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

 1138 MARGHERITA GUIDA AND CARLO SBORDONE

In Cimmino's method the kth approximation \bfitx (k) is obtained by geometric re-
flections.
Downloaded 03/28/24 to 115.244.99.163 . Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/terms-privacy

The n equations of (1.1) can be identified with n distinct affine hyperplanes

meeting at a unique point Z, whose coordinates satisfy system (1.1) and are found
by multiple reflections, as we set out to explain.
The method starts by fixing an arbitrary point P (0) \not = Z as initial approximation
and then constructs the symmetric points Q(1) , . . . , Q(n) of P (0) with respect to the n
hyperplanes. These points belong to a spherical surface, the center of which is precisely
the common point Z to the n hyperplanes, namely, the linear system's solution. The
radius

(1.2) d1 = dist(P (0) , Z)

is the Euclidean distance between P (0) and Z.

Fig. 1 The reflection method for n = 2.

Let P (1) denote the centroid of the points Q(1) , . . . , Q(n) . This is also known as
the center of mass of the configuration of points since we may imagine assigning to
each point unit mass. In this way,

dist(P (1) , Z) < dist(P (0) , Z).

Now we replace P (0) with the centroid P (1) and iterate the reflecting procedure,
obtaining another point P (2) such that

dist(P (2) , Z) < dist(P (1) , Z).

Continuing in the same manner eventually produces a sequence of points (P (k) ) con-
verging to Z as k \rightarrow \infty , in which each P (k) will approximate Z:
\{ P (0) , P (1) , P (2) , . . .\} , P (k) \rightarrow Z.
We shall establish precise estimates for dist(P (k) , Z) against a given initial error
(see section 3). See Figure 1 for an illustration of the reflection method for n = 2.

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

 NUMERICAL SOLUTION OF LINEAR SYSTEMS 1139

The sequence (P (k) ) converges to a point Z whose coordinates \bfitxi = (\xi 1 , . . . , \xi n ) also
solve (1.1) when the system is singular, provided a solution does exist and rank(A) \geq
Downloaded 03/28/24 to 115.244.99.163 . Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/terms-privacy

2, which is what we assume here.

We must stress that the method is always convergent, but the number of iterations
might be considerable. In order to overcome the algorithm's efficiency issues, Cimmino
therefore upgraded his method by introducing probabilistic arguments (see section 4).
The idea is to add to the nonsingular system further constraints given by linear
combinations of the original n equations. The resulting system is manifestly still
nonsingular. Geometrically it corresponds to considering more hyperplanes passing
through Z.
For example, if we take all hyperplanes through Z and symmetrize P (0) , we obtain
the sphere centered at Z with radius d1 = dist(P (0) , Z). In this case the centroid
is precisely Z. Instead adding only some randomly chosen linear combinations to
the original n hyperplanes introduces a probabilistic element in the approximation
procedure. If we fix 0 < \delta < 1, we can then define the probability that the sequence
(P (k) ) is contained in the ball centered at Z with radius \delta \cdot d1 .
The principle underpinning this method is that one should augment the initial
linear system (1.1) so that a suitable increase in the number of equations results in
faster convergence. The choice of the new equations is mostly arbitrary. Consequently,
the algorithm, together with the problem's data, will contain random numbers and
the approximate solutions' accuracy will depend on ``chance.""
It is remarkable that in recent years randomized algorithms have become widely
used in numerical linear algebra.
In fact, via probabilistic approximation, it is possible to rigorously assess the
accuracy as follows. Given a certain quantity to be computed according to a suitable
measure, one has to evaluate the probability that the final error respects an a priori
bound. If this mathematical probability is sufficiently high, the approximate solution
is deemed acceptable.
Cimmino's idea (adding special extra equations to the system) should not be
considered as a tool of a practical nature. It should instead be regarded as a bridge
between two scenarios, namely, finite sets and continuous distributions of masses on
a hypersphere, and it eventually produces nice integral formulas for the solutions to
linear systems that go back to Jacobi [12].
All of this raises enlightening aspects that can improve the perception of mathe-
matics. For example, the classical method (due to the Swiss mathematician Gabriel
Cramer, 1704--1752) for solving a standard problem in linear algebra---one that stu-
dents know too---is revisited in this paper in the light of a simple, yet ostensibly
hidden, geometrical observation made by Cimmino:

Given two lines on a plane intersecting at Z and a point P \not = Z, the mirror
points of P with respect to the lines lie on the circle of center Z and radius
dist(P, Z).

No doubt every instructor will enjoy explaining this elegant result to students.
Cramer's rule also has a geometrically flavored proof based on the areas of paral-
lelograms and Cavalieri's principle.
In the probabilistic version of the numerical algorithm, the arguments are much
more involved and rely on knowing a few important integrals (see the formulas in
section 5, which are crucial to the rigorous and complete proof of the results).
For a long time Cimmino's method, in spite of its advantages, did not catch on. In

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

 1140 MARGHERITA GUIDA AND CARLO SBORDONE

all likelihood this was due to the fact that Gaussian elimination was the dominating
technique in linear systems of moderate size, while large linear systems used for PDE
Downloaded 03/28/24 to 115.244.99.163 . Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/terms-privacy

discretization were solved in a faster way by iterative methods adapted to the matrix's
specific structure [2]. Later, Krylov methods were typically preferred. Since the early
1980s, though, an increasing number of authors have returned to Cimmino's method
[1], [5], [9], [10], [16], [18],[19], [20].
An early extension of Cimmino's methods to the non-Euclidean setting deals with
regularization property applied to ill-posed problems for the equation

(1.3) A\bfitu = \bfitf .

Here A is a compact linear operator on an infinite-dimensional Banach space X.

The regularization property means that, after a certain number k of iterations,
one gets a sufficiently good approximation \bfitu (k) whose subsequent iterations do not
improve quality [2]. See also [3], where Landweber's work [14] is described as an
extension of Cimmino methods.
To conclude, let us mention that Cimmino's method has been shown to work well
in parallel computing, in particular for applications in the area of image reconstruction
via X-ray tomography (see, e.g., [4], [11], [15]).
2. Cimmino’s Method for \bfitn = 2. Consider, in the plane \BbbR 2 , the straight line r
of equation

(2.1) \langle \bfita , \bfitx \rangle = b,

\sum 2
where \bfitx = (x1 , x2 ), \bfita T = (a1 , a2 ), ai , b \in \BbbR , and \langle \bfita , \bfitx \rangle = i=1 ai xi . Its unit normal
is
\bfita
(2.2) \bfitnu =
\| \bfita \|
\sqrt{}
with \| \bfita \| = a21 + a22 = \langle \bfita , \bfita \rangle 1/2 .
(0) (0)
Let us fix P (0) = P (0) (\bfitx (0) ) = P (0) (x1 , x2 ) such that P (0) \in
/ r.
Lemma 2.1. Let R be the orthogonal projection of P (0) onto r and Q the sym-
metric point of P (0) with respect to R. See Figure 2. Then

b - \langle \bfita , \bfitx (0) \rangle

(2.3) R = \bfitx (0) + \bfita
\| \bfita \| 2

and

b - \langle \bfita , \bfitx (0) \rangle

(2.4) Q = \bfitx (0) + 2 \bfita .
\| \bfita \| 2

Proof. Since \bfitnu is parallel to P (0) Q, there exists \lambda > 0 such that

(2.5) Q \equiv \bfitx (0) + 2\lambda \bfitnu

and

(2.6) R \equiv \bfitx (0) + \lambda \bfitnu .

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

 NUMERICAL SOLUTION OF LINEAR SYSTEMS 1141
Downloaded 03/28/24 to 115.244.99.163 . Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/terms-privacy

Fig. 2 Lemma 2.1.

Stipulating that R belongs to the line r of (2.1), that is,

(2.7) a1 x1 + a2 x2 = b,

we find
(0) (0)
\lambda (a1 \nu 1 + a2 \nu 2 ) = b - (a1 x1 + a2 x2 )
and hence
(0) (0)
b - (a1 x1 + a2 x2 )
(2.8) \lambda = .
\| \bfita \|
At this point, (2.3) and (2.4) follow immediately.
Let us introduce the 2 \times 2 system
\Biggl\{
a11 x1 + a12 x2 = b1 ,
(2.9)
a21 x1 + a22 x2 = b2 ,

or, in vectorial notation, where \bfita Ti denotes the ith row of matrix A,

(2.10) \langle \bfita i , \bfitx \rangle = bi , i = 1, 2,

\bigl( \bfita T1 \bigr)
and assume A = \bfita T
is a matrix with det A \not = 0, so that the system has a unique
2
solution

(2.11) Z = Z(\xi 1 , \xi 2 ).

Lemma 2.2. Retaining the previous notation, if P (0) \not = Z and for each i = 1, 2,
the point Q(i) is symmetric to P (0) with respect to the straight line ri of (2.10), then

(2.12) dist(Q(i) , Z) = dist(P (0) , Z).

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

 1142 MARGHERITA GUIDA AND CARLO SBORDONE

Moreover, the centroid P (1) of Q(1) and Q(2) (each bearing mass 1) satisfies

dist(P (1) , Z) < dist(P (0) , Z) = d1 .

Downloaded 03/28/24 to 115.244.99.163 . Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/terms-privacy

(2.13)

Proof. By looking at Figure 3 it is clear that the triangle P (0) ZQ(2) is isosceles,
because r2 extends both the bisectrix of the angle at Z and the median line of the
edge P (0) Q(2) . By a similar reasoning Q(1) ZP (0) is isosceles as well. Hence (2.12)
holds for i = 1, 2.

Fig. 3 Q(i) is the mirror of P (0) with respect to ri for i = 1, 2.

As the centroid of a number of points lying on a circle falls inside the circle, we have
(2.13).
Given the initial approximation P (0) = P (0) (\bfitx (0) ), the second step P (1) = P (1) (\bfitx (1) )
is the centroid of two unit masses placed at the symmetric points Q(i) , i = 1, 2, of
P (0) with respect to the straight line \langle \bfita i , \bfitx \rangle = bi :
2
\sum bh - \langle \bfita h , \bfitx (0) \rangle
(2.14) \bfitx (1) = \bfitx (0) + \bfita h .
\| \bfita h \| 2
h=1

This P (1) (\bfitx (1) ) is taken as the starting point of the next iteration, and so on. At step
\nu + 1,
2
\sum bh - \langle \bfita h , \bfitx (\nu ) \rangle
(2.15) \bfitx (\nu +1) = \bfitx (\nu ) + \bfita h
\| \bfita h \| 2
h=1

(\nu = 0, 1, . . .).
In matrix form (2.15) reads

(2.16) \bfitx (\nu +1) = \bfitx (\nu ) + AT D(\bfitb - A\bfitx (\nu ) ),

\bigl( \| \bfita 1 \| - 2 0 \bigr)
where D is the diagonal matrix D = 0 \| \bfita 2 \| - 2
.
Cimmino's iterative algorithm was originally formulated using nonunitary masses
in the computation of the centroid, but we prefer here to consider the simplest case
above.

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

 NUMERICAL SOLUTION OF LINEAR SYSTEMS 1143

Let us point out that Kaczmarz [13] discovered in 1937 a similar (sequential) iter-
ative method. According to this, the step-\nu approximation \bfitx (\nu ) is projected orthog-
Downloaded 03/28/24 to 115.244.99.163 . Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/terms-privacy

onally (instead of reflected), sequentially, and not simultaneously (as for Cimmino).
Cimmino's algorithm has been found to suit parallel computing better, whereas Kacz-
marz's method tends to converge somewhat faster (see [11]).
3. Error Bounds. Let us indicate by Z = Z(\bfitxi ) the point whose coordinates
(\xi 1 , \xi 2 ) = \bfitxi solve system (2.9). Fix a point
P (0) = P (0) (\bfitx (0) ),
different from Z.
Using \langle \bfita i , \bfitxi \rangle = bi , i = 1, 2, we define the numbers
\eta i = \langle \bfita i , \bfitx (0) \rangle - bi = \langle \bfita i , \bfitx (0) - \bfitxi \rangle
and the vector
2
\sum \bfita i
(3.1) \bfitx (1) = \bfitx (0) - \langle \bfita i , \bfitx (0) - \bfitxi \rangle .
i=1
\| \bfita i \| 2

Now observe that

2
\bigm\| 2
\bigm\| 2
\eta i2
\bigm\| \sum
(1) 2 (0) 2
\sum 1 \bigm\|
(3.2) \| \bfitx - \bfitxi \| = \| \bfitx - \bfitxi \| - 2 2
+ \bigm\|
\bigm\|
2
\cdot \eta i \cdot \bfita i \bigm\|
\bigm\| .
i=1
\| \bfita i \| i=1
\| \bfita i \|

If we manage to prove that

2 \bigm\| 2
\eta i2
\bigm\|
\sum \bigm\| \sum \eta i \bigm\|
(3.3) 2 2
- \bigm\|
\bigm\|
2 \bigm\| \geq 0,
\bfita i \bigm\|
i=1
\| \bfita i \| i=1
\| \bfita i \|

the estimate
(3.4) \| \bfitx (1) - \bfitxi \| \leq \| \bfitx (0) - \bfitxi \|
follows. To prove (3.3) it suffices to observe that
\bigm\| 2 2 2
\bigm\| \sum \bigm\| 2 \biggl( \sum \biggr) \biggl( \sum \biggr)
2 2
\bigm\|
\bigm\|
\bigm\| p \eta \bfita
i i i \bigm\|
\bigm\| \leq p \eta
i i \cdot pi \| \bfita i \| ,
i=1 i=1 i=1
1
where pi = \| \bfita i \| 2 .
In conclusion, (3.4) tells us that \bfitx (1) is not any further than \bfitx (0) from the exact
solution \bfitxi to (3.1).
Now, iterating, we obtain a sequence P (\nu ) = P\nu (\bfitx (\nu ) ) and corresponding estimates
(3.5) \| \bfitx (\nu +1) - \bfitxi \| \leq \| \bfitx (\nu ) - \bfitxi \|
(\nu = 0, 1, . . .). As a matter of fact, there is equality in (3.5) if and only if \bfitx (\nu ) solves
the given system, and hence \bfitx (\nu ) = \bfitxi and also \bfitx (\nu +1) = \bfitx (\nu ) for any \nu \in \BbbN .
4. Probabilistic Arguments. In 1965, Cimmino devised an interesting way to
attempt to speed up the convergence of the approximating sequence. He exploited
the fact that in the system
\Biggl\{
a11 x1 + a12 x2 = b1 ,
(4.1)
a21 x1 + a22 x2 = b2 ,

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

 1144 MARGHERITA GUIDA AND CARLO SBORDONE

where det A = det(aij ) \not = 0, arbitrary linear combinations of the two equations rep-
resent straight lines containing the point Z(\bfitxi ) for which \bfitxi = (\xi 1 , \xi 2 ) solves (4.1).
Downloaded 03/28/24 to 115.244.99.163 . Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/terms-privacy

These new linear equations in (x1 , x2 ) make system (4.1) bigger without modifying
the solution. The reflection method applied to the augmented system geometrically
consists in replacing the approximation \bfitx (\nu - 1) of \bfitxi with another point \bfitx (\nu ) , which
is the centroid of as many unit masses as the number of equations in the augmented
system. These unit masses, together with \bfitx (\nu - 1) , belong to a circle of center \bfitxi .
If the number of these masses, randomly chosen on the circle with uniform dis-
tribution, grows, the probability that their centroid becomes closer to \bfitxi increases as
well. Uniform distribution means that the probability density is constant; that is, the
probability that each mass belongs to an arbitrary fixed portion of the circumference
is proportional to the length of the portion.
We always assume that system (4.1) has a unique solution \bfitxi = (\xi 1 , \xi 2 ). Let us
fix an integer N \geq 3 and consider an N \times 2 matrix \Gamma = (\gamma ik ) for i = 1, . . . , N and
k = 1, 2, and the enlarged system of N linear equations
2
\biggl( \sum \biggr) 2
\biggl( \sum \biggr) 2
\sum
(4.2) \gamma ih ah1 x1 + \gamma ih ah2 x2 = \gamma ih bh , i = 1, . . . , N.
h=1 h=1 h=1

We shall use a natural generalization of the previous sections' method. For sim-
plicity, we shall suppose P (0) = P (0) (0, 0).
In analogy to the case N = 2, one can calculate (see [7]) the coordinates of the
centroid of the N unit masses Q(1) , Q(2) , . . . , Q(N ) :
\Bigl( \sum \Bigr) \Bigl( \sum \Bigr)
N 2 2
2 h=1 \gamma i h ahk h=1 \gamma i h b h
(1)
\sum
(4.3) xk = \Bigr) 2 , k = 1, 2.
N i=1 \sum 2 \Bigl( \sum 2
m=1 h=1 ih hm \gamma a

Notice that (4.3) reduces to (2.14) by taking N = 2, \bfitx (0) = (0, 0), and \gamma ih = 1 if
i = h, \gamma ih = 0 if i \not = h.
For N large, with high probability (4.3) will give a good approximation of the
solution (\xi 1 , \xi 2 ) to (4.1). Namely, as N \rightarrow \infty , a single iteration of the method gives,
with probability that tends to 1, an approximate solution with relative error smaller
than any fixed 0 < \theta < 1. Here, by ``single iteration"" we mean that it suffices
to choose the centroid of Q(1) , Q(2) , . . . , Q(N ) (as N \rightarrow \infty ), which has an explicit
expression. No further iterations are required. Ultimately, Cimmino proved that
the centroid of a system of N masses randomly distributed on a circle tends, with
probability arbitrarily close to 1, to the center (as N \rightarrow \infty ).
Formula (4.3) represents a ``probabilistic"" solution to (4.1), meaning that if the
(1)
choice of the numbers \gamma ik is more/less fitting, then xk will give a more/less precise
approximation of \bfitxi . Let us pick 0 < \theta < 1 and call PN (\theta ) the probability that the
centroid P (1) of N points (randomly chosen on the circle \Sigma = \Sigma (Z, d) of center Z \in \BbbR 2
and radius d) has distance from Z satisfying
(4.4) dist(P (1) , Z) < \theta d.
Then we find limN \rightarrow \infty PN (\theta ) = 1.
This will work if the N masses are uniformly distributed on the circle \Sigma .
The N points can be considered as a point in
\Sigma N = \Sigma \times \cdot \cdot \cdot \times \Sigma ,
\underbrace{} \underbrace{}
N

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

 NUMERICAL SOLUTION OF LINEAR SYSTEMS 1145
Downloaded 03/28/24 to 115.244.99.163 . Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/terms-privacy

Fig. 4 Q(i) is symmetric to the origin O with respect to ri , i = 1, 2, 3.

and condition (4.4) on their centroid P (1) defines a subset TN of \Sigma N .

Namely, if Z = Z(\xi 1 , \xi 2 ) is the point for which (\xi 1 , \xi 2 ) solves system (4.1) and
P (0) = P (0) (0, 0) is the first approximation of Z with d = dist(P (0) , Z), we will write
\Sigma = \Sigma (Z, d).
For any 0 < \theta < 1 and N an integer, N \geq 3, TN is the set of N -tuples
(Q(1) , . . . , Q(N ) ) \in \Sigma N
such that the centroid P (1) of Q(1) , . . . , Q(N ) satisfies (4.4).
We consider on \Sigma N the measure
mN (\Sigma N ) = [\ell (\Sigma )]N ,
where \ell (\Sigma ) denotes the length of \Sigma and the induced measure on the subset TN of \Sigma N .
The ratio m[\ell N(\Sigma )]
(TN )
N will satisfy precisely the equality
mN (TN )
(4.5) PN (\theta ) =
[\ell (\Sigma )]N
(see [7]). This is the probability that (4.3) are approximations of the solution (\xi 1 , \xi 2 )
for which the relative error is smaller than \theta :
\Biggl\{ 2 \Biggr\} 12 \Biggl\{ 2 \Biggr\} 21
\sum (1) \sum
(xk - \xi k )2 \leq \theta \xi k2 .
k=1 k=1

Since we are assuming that the origin O = O(0, 0) of the Cartesian plane is the first
approximation P (0) of Z (Figure 4), the symmetric points Q(i) to O with respect to
the straight lines ri containing Z belong to the circle centered at Z of radius equal to
d = dist(O, Z).
In what follows we will consider some examples in the case N = 3, d = 1, where the
centroid P (1) of three points on the circle \Sigma = \Sigma (Z, 1) lies inside the disk of center Z
and radius 12 (Figure 6) or on the circle \Sigma (Z, 21 ) (Figures 5 and 7). The points Q(1)
and Q(2) are the same in all three figures.
From these figures we observe that when the points Q(3) move along the thick arc
of Figure 7, the centroids belong to the closed disk of half radius.

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

 1146 MARGHERITA GUIDA AND CARLO SBORDONE
Downloaded 03/28/24 to 115.244.99.163 . Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/terms-privacy

Fig. 5 The centroid P (1) of the triangle inscribed in the large disk lies on the circle of half radius.

Fig. 6 The centroid P (1) of the triangle inscribed in the large disk lies inside the disk of half radius.

5. Integrals in Place of Determinants. In this section we illustrate an ``uncom-

mon"" way to express the absolute value of the determinant det A of a generic n \times n
real matrix A = (aij ), i, j = 1, . . . , n, by means of multiple integrals [8], [17]. As an
added bonus, we also describe a Cramer-type formula, involving ratios of integrals
instead of determinants, for the solutions \bfitxi = (\xi 1 , . . . , \xi n ) of the linear system
n
\sum
(5.1) aik xk = bi , i = 1, . . . , n,
k=1

when det A \not = 0.

Although the essence of the results can only be truly appreciated when n is large,
for simplicity we will sketch the proofs only for the case n = 2.
Here is the first formula, where we denote by
\bfita Ti = (ai1 , ai2 , . . . , ain )
the ith row of A and by | \omega n | the (n - 1)-dimensional measure of the sphere

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

 NUMERICAL SOLUTION OF LINEAR SYSTEMS 1147
Downloaded 03/28/24 to 115.244.99.163 . Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/terms-privacy

Fig. 7 The centroid P (1) lies on the circle of half radius.

\biggl\{ n
\sum \biggr\}
n 2
\omega n = \bfitx = (x1 , . . . , xn ) \in \BbbR :\| \bfitx \| = x2i =1 .
i=1

We have
| \omega |
(5.2) | det A | = \int \sum n n - n dt \cdot \cdot \cdot dt
.
2 \BbbR n - 1
\| \bfita 1 + i=2 ti \bfita i \| 2 n

The Cramer-type formula for the solution \bfitxi = (\xi 1 , . . . , \xi n ) to (5.1) is
(5.3)
\int \bigm\| n \bigm\| - (n+2) \Biggl( n
\Biggr) \Biggl( n
\Biggr)
2n \bigm\| \sum \bigm\| \sum \sum
\bfitxi = | det A| \bigm\| \bfita 1 + ti \bfita i \bigm\| \bfita 1 + ti \bfita i b1 + tk bk dt2 . . . dtk .
| \omega n | \BbbR n - 1
\bigm\|
i=2
\bigm\|
i=2 k=2

The idea (cf. Figure 4) described in section 3 for n = 2 lies behind formulas (5.2)
and (5.3).
The points Q(i) symmetric to the origin O with respect to ri belong to the circle
centered at Z with radius | \bfitxi | , and their centroid P is inside the circle.
The probability that, for a given 0 < \theta < 1,

(5.4) dist(P, Z) < \theta | \bfitxi |

grows with the number of lines ri . We could even take a continuous family of lines,
generating a continuous distribution of masses on the circle. Formulas (5.2) and (5.3),
which we now set out to prove, arise from the probability of finding a continuous
distribution of suitable masses with centroid P for which the distance (5.4) reduces
to zero.
Next, recall that for n = 2 we have \bfita T1 = (a11 , a12 ), \bfita T2 = (a21 , a22 ), and A =
\bigl( \bfita T1 \bigr)
\bfita T
. Consider the linear system (5.1), where bi \in \BbbR , i = 1, 2. Our aim is to replace
2
the classical Cramer formula for the solution \bfitxi = (\xi 1 , \xi 2 ) by (5.3), which for n = 2
becomes
\int (\bfita 1 +t\bfita 2 )(b1 +tb2 )
\BbbR \| \bfita +t\bfita \| 4 dt
(5.5) \bfitxi = 2 \int 1 dt2
\BbbR \| \bfita 1 +t\bfita 2 \| 2

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

 1148 MARGHERITA GUIDA AND CARLO SBORDONE

thanks to (5.2).
Let us first show that (5.2) holds for n = 2.
Downloaded 03/28/24 to 115.244.99.163 . Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/terms-privacy

Theorem 5.1. The following formula holds:

\int
dt \pi
(5.6) 2
= .
\BbbR \| \bfita 1 + t\bfita 2 \| | det A |
Proof. Let us introduce the change of variables
(5.7) t = \gamma 1 + \gamma 2 \tau
with
1
(5.8) \gamma 2 = \sqrt{}
a221 + a222
and \gamma 1 given by imposing
(5.9) \langle \bfita 1 + \gamma 1 \bfita 2 , \bfita 2 \rangle = 0,
that is,
(a11 a21 + a12 a22 )
(5.10) \gamma 1 = - .
(a221 + a222 )
Our aim is to prove that there exists c \in \BbbR such that
(5.11) \| \bfita 1 + t\bfita 2 \| 2 = c2 + \tau 2 .
Changing variables and rearranging leads to
\| \bfita 1 + t\bfita 2 \| 2 = \langle \bfita 1 + t\bfita 2 , \bfita 1 + t\bfita 2 \rangle
\biggl\langle \Bigl( \Bigr) \Bigl( \Bigr) \biggr\rangle
= \bfita 1 + \gamma 1 + \gamma 2 \tau \bfita 2 , \bfita 1 + \gamma 1 + \gamma 2 \tau \bfita 2
\Bigl( \Bigr) \Bigl( \Bigr) 2
= \langle \bfita 1 , \bfita 1 \rangle + 2 \gamma 1 + \gamma 2 \tau \langle \bfita 1 , \bfita 2 \rangle + \gamma 1 + \gamma 2 \tau \langle \bfita 2 , \bfita 2 \rangle
\Bigl( \Bigr)
= \| \bfita 1 \| 2 +\gamma 1 \langle \bfita 1 , \bfita 2 \rangle + \tau 2 \gamma 22 \| \bfita 2 \| 2 +\gamma 1 \langle \bfita 1 + \gamma 1 \bfita 2 , \bfita 2 \rangle
\Bigl( \Bigr)
+2\tau \gamma 2 \langle \bfita 1 + \gamma 1 \bfita 2 , \bfita 2 \rangle .
1
Since \gamma 2 = \| \bfita 2 \| and \langle \bfita 1 + \gamma 1 \bfita 2 , \bfita 2 \rangle = 0, it follows that

\| \bfita 1 + t\bfita 2 \| 2 =\| \bfita 1 \| 2 +\gamma 1 \langle \bfita 1 , \bfita 2 \rangle + \tau 2 ,

so by setting c2 =\| \bfita 1 \| 2 +\gamma 1 \langle \bfita 1 , \bfita 2 \rangle we obtain (5.11).

Notice that we also have

(det A)2
(5.12) c2 = ,
\| \bfita 2 \| 2
so, using (5.11), (5.12), and (5.6), we easily end up with
\int \int
dt 1 ds \pi
(5.13) 2
= 2)
= .
\BbbR \| \bfita 1 + t\bfita 2 \| | det A | \BbbR (1 + s | det A |

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

 NUMERICAL SOLUTION OF LINEAR SYSTEMS 1149

Let us now compute the numerator in (5.5).

Theorem 5.2. The following equality holds:
Downloaded 03/28/24 to 115.244.99.163 . Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/terms-privacy

\int
(\bfita 1 + t\bfita 2 )(b1 + tb2 ) \pi \Bigl[ 1 b2 \bfita 2 \Bigr]
(5.14) 4
dt = \cdot 2 \cdot (\bfita 1 +\gamma 1 \bfita 2 )(b1 +\gamma 1 b2 )+
\BbbR \| \bfita 1 + t\bfita 2 \| 2 | det A | c \| \bfita 2 \| 2

with \gamma 1 given by (5.9) and (5.10) and c as in (5.12).

Proof. Using (5.7), (5.8), and (5.11) we have
(5.15) \Bigl[ \Bigr] \Bigl[ \Bigr]
\int \bfita 1 + (\gamma 1 + \tau )\bfita 2 b1 + b2 (\gamma 1 + \tau
| c | \| \bfita 2 \| )
\int
(\bfita 1 + t\bfita 2 )(b1 + tb2 ) \| \bfita 2 \|
4
dt = d\tau .
\BbbR \| \bfita 1 + t\bfita 2 \| | det A | \BbbR (c2 + \tau 2 )2
\tau
\int
It is easy to check that \BbbR (c2 +\tau 2 )2 d\tau = 0, so that (5.15) reads

| c |
\int \int
(\bfita 1 + t\bfita 2 )(b1 + tb2 ) 1
4
dt = \cdot (\bfita 1 + \gamma 1 \bfita 2 )(b 1 + \gamma 1 b2 ) 2 2 2
d\tau
\BbbR \| \bfita 1 + t\bfita 2 \| | det A | \BbbR (c + \tau )
| c | \tau 2
\int
b2 \bfita 2
(5.16) + \cdot d\tau .
| det A | \| \bfita 2 \| 2 \BbbR (c2 + \tau 2 )2
Since \int
1 \pi
d\tau =
\BbbR (c2 2
+ \tau )2 2 | c | 3
and
\tau 2
\int
\pi
2 2 2
d\tau = ,
\BbbR (c + \tau ) 2 | c |
formula (5.14) follows.
Theorem 5.3. The solution \bfitxi to the system

(5.17) \langle \bfita i , \bfitx \rangle = bi , i = 1, 2,

is given by
1 b2 \bfita 2
(5.18) \bfitxi = 2
(\bfita 1 + \gamma 1 \bfita 2 )(b1 + \gamma 1 b2 ) + .
c \| \bfita 2 \| 2

Proof. By virtue of (5.13) and (5.14), formula (5.5) turns into (5.18). It easy to
see that the dot product of (5.18) and \bfita 1 equals b1 , while the dot product with \bfita 2 is
b2 .
6. Conclusions. Cimmino's method has been described in a number of mono-
graphs (see [2] for a recent bibliography). In some of these books the Cimmino and
Kaczmarz methods are often presented together with comparisons and extensions,
and they have also been rediscovered by researchers in applied sciences.
A renewed interest in their algorithms in the period 1970--1980 was motivated by
the needs of two technological advances:
- first, the diffusion of tomography (CAT scans) in radiology which needed
regularizing effects in image reconstruction from projections (see [2, section
6] and references therein);
- second, the mushrooming of parallel computing (particularly regarding the
Cimmino algorithm).

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

 1150 MARGHERITA GUIDA AND CARLO SBORDONE

Over the years, Cimmino's method was applied in the following areas [2], [11]:
\bullet Convex mathematical programming [1], [18].
Downloaded 03/28/24 to 115.244.99.163 . Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/terms-privacy

\bullet Fast adaptation of radiation therapy planning [4].

\bullet Solution of ``inverse problems"" in medical physics [11].
\bullet Filtering in signal processing [19].
In many cases these problems are nonlinear and are expressed in terms of inequalities
[5], [18].
Notice that many articles quoting Cimmino's paper [6] in the 1938 Italian ver-
sion appeared in journals such as Physics in Medicine and Biology, Medical Physics,
Inverse Problems, International Journal of Radiation Oncology, Biology and Physics,
IEEE Transactions on Medical Imaging, IEEE Transactions on Image Processing,
IEEE Transactions on Signal Processing, Annals of Operations Research.
The story of Cimmino's method is interesting for several reasons. It shows that
good mathematical ideas may be appreciated and used even many years later. Im-
portant technological advances may transform the role of an algorithm from abstract
speculation to an applicable tool. It goes to show that good work in a specific disci-
pline, even though it might not attract much interest within its community at first,
might nevertheless be of use in unsuspected fields.

Acknowledgments. The authors are grateful to Prof. Michele Benzi of the Scuola
Normale Superiore in Pisa, who shared the English version of Cimmino's paper [6]
(written in 1932 and published in Italian in 1938) that he edited in 2004. See also [2].
Thanks are due to the referees for very useful observations.

REFERENCES

[1] R. Aharoni and Y. Censor, Block-iterative projection methods for parallel computation of
solutions to convex feasibility problems, Linear Algebra Appl., 120 (1989), pp. 165--175.
(Cited on pp. 1140, 1150)
[2] M. Benzi, Gianfranco Cimmino's contributions to numerical mathematics, Rend. Accad. Sci.
Fis. Mat. Napoli (4), 89 (2022), pp. 73--98. Updated version of the original paper: M.
Benzi, Gianfranco Cimmino's contributions to numerical mathematics, Atti del Seminario
di Analisi Matematica, Dip. di Mat. Univ. Bologna, Volume Speciale: Ciclo di Conferenze
in Memoria di Gianfranco Cimmino, 2004, Tecnoprint, Bologna, 2005, pp. 87--109. (Cited
on pp. 1140, 1149, 1150)
[3] P. Brianzi, F. Di Benedetto, and C. Estatico, Preconditioned iterative regularization in
Banach spaces, Comput. Optim. Appl., 54 (2013), pp. 263--282. (Cited on p. 1140)
[4] Y. Censor, M. D. Altschuler, and W. D. Powlis, On the use of Cimmino's simultaneous
projection method for computing a solution of the inverse problem in radiation therapy
treatment planning, Inverse Problems, 4 (1988), pp. 607--623. (Cited on pp. 1140, 1150)
[5] Y. Censor and T. Elfving, New methods for linear inequalities, Linear Algebra Appl., 42
(1982), pp. 199--211. (Cited on pp. 1140, 1150)
[6] G. Cimmino, Approximate computation of the solutions of systems of linear equations, Rend.
Accad. Sci. Fis. Mat. Napoli (4), 89 (2022), pp. 65--72; English translation by M. Benzi (Sc.
Norm. Sup. Pisa, 2015) of Calcolo approssimato per le soluzioni dei sistemi di equazioni
lineari, La Ricerca Scientifica, II, 9 (1938), pp. 326--333. (Cited on pp. 1137, 1150)
[7] G. Cimmino, Un metodo Monte Carlo per la risoluzione numerica dei sistemi di equazioni
lineari, Atti Accad. Sci. Ist. Bologna Cl. Sci. Fis. Rend. (12), 2 (1964--65) (1967), pp. 39--
44. (Cited on pp. 1144, 1145)
[8] G. Cimmino, An unusual way of solving linear systems, Atti Accad. Naz. Lincei Rend. Cl. Sci.
Fis. Mat. Nat. (8), 80 (1986), pp. 6--7. (Cited on p. 1146)
[9] T. Drummond, I. S. Duff, R. Guivarch, D. Ruiz, and M. Zenadi, Partitioning strategies for
the Block Cimmino algorithm, J. Engrg. Math., 93 (2014), pp. 21--39. (Cited on p. 1140)
[10] T. Elfving, A projection method for semidefinite linear systems and its applications, Linear
Algebra Appl., 391 (2004), pp. 57--73. (Cited on p. 1140)
[11] P. C. Hansen, Algebraic Iterative Methods for Computed Tomography, http://bme.elektro.dtu.

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.

 NUMERICAL SOLUTION OF LINEAR SYSTEMS 1151

dk/31545/notes/AIRslides.pdf. (Cited on pp. 1140, 1143, 1150)

[12] C. G. J. Jacobi, Dato systemate n equationum linearium inter n incognitas, valores incogni-
tarum per integralia definita (n - 1)-tuplicia exhibentur, Crelle J., 14 (1834), pp. 51--55;
Downloaded 03/28/24 to 115.244.99.163 . Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/terms-privacy

reprinted in Gesammelte Werke, Vol. 6, 2012, pp. 79--85. (Cited on p. 1139)

[13] S. Kaczmarz, Przyblizone rozwiazywanie uklad\' ow r\'
ownan liniowych. Angen\" aherte Aufl\"
osung
von Systemen linearer Gleichungen, Bull. Int. Acad. Polonaise Sci. Lett. Classe Sci. Math.
Natur. Ser. A, 35 (1937), pp. 355--357. (Cited on p. 1143)
[14] L. Landweber, An iteration formula for Fredholm integral equations of the first kind, Amer.
J. Math, 73 (1951), pp. 615--624. (Cited on p. 1140)
[15] S. Petra, C. Popa, and C. Schnorr, \" Extended and Constrained Cimmino-type Algorithms
with Applications in Tomographic Image Reconstruction, IWR preprint 8798, University
of Heidelberg, 2008; available at http://www.ub.uni-heidelberg.de/archiv/8798. (Cited on
p. 1140)
[16] C. Popa, On Cimmino's reflection algorithm, Proc. Rom. Acad Ser. A Math. Phys. Tech. Sci.
Inf. Sci., 9 (2008), pp. 13--19. (Cited on p. 1140)
[17] C. Sbordone and G. Trombetti, G. Cimmino, Opere Scelte, Giannini, Napoli, 2002. (Cited
on p. 1146)
[18] V. Simoncini, Cimmino's method and the next generation of iterative solvers, Universit\à di
Bologna, XXXIX Annual Conference of Italian Operational Research Society (AIRO), Is-
chia (NA), 2008, http://www.dm.unibo.it/\sim simoncin/cimmino.pdf. (Cited on pp. 1140,
1150)
[19] R. A. Soni, K. A. Gallivan, and W. K. Jenkins, Projection methods for improved perfor-
mance in FIR adaptive filters, in Proceedings of the 40th Midwest Symposium on Circuits
and Systems, Vol. 2, IEEE, 1997, pp. 746--749. (Cited on pp. 1140, 1150)
[20] F. S. Torun, M. Manguoglu, and C. Aykanat, A novel partitioning method for accelerating
the block Cimmino algorithm, SIAM J. Sci. Comput., 40 (2018), pp. C827--C850, https:
//doi.org/10.1137/18M1166407. (Cited on p. 1140)

Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.