Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Scribd Logo
Upload

Welcome to Scribd!

  • 15 views

    MAT 461/561: 5.1 Stationary Iterative Methods

    Uploaded by

    Debisa
    This document discusses stationary iterative methods for solving linear systems Ax = b. [1] Stationary iterative methods repeatedly apply a fixed function g(x) = M^-1(Nx + b) to generate approximations x(k+1) from x(k). [2] The Jacobi and Gauss-Seidel methods are examples of stationary iterative methods that use diagonal splitting of A. [3] Successive overrelaxation (SOR) is an acceleration of Gauss-Seidel that introduces a relaxation parameter ω to take steps of different lengths toward the solution.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Flag for inappropriate content

    MAT 461/561: 5.1 Stationary Iterative Methods

    Uploaded by

    Debisa
    15 views4 pages
    This document discusses stationary iterative methods for solving linear systems Ax = b. [1] Stationary iterative methods repeatedly apply a fixed function g(x) = M^-1(Nx + b) to generate approximations x(k+1) from x(k). [2] The Jacobi and Gauss-Seidel methods are examples of stationary iterative methods that use diagonal splitting of A. [3] Successive overrelaxation (SOR) is an acceleration of Gauss-Seidel that introduces a relaxation parameter ω to take steps of different lengths toward the solution.

    Original Description:

    Original Title

    notes5

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Report this Document
    This document discusses stationary iterative methods for solving linear systems Ax = b. [1] Stationary iterative methods repeatedly apply a fixed function g(x) = M^-1(Nx + b) to generate approximations x(k+1) from x(k). [2] The Jacobi and Gauss-Seidel methods are examples of stationary iterative methods that use diagonal splitting of A. [3] Successive overrelaxation (SOR) is an acceleration of Gauss-Seidel that introduces a relaxation parameter ω to take steps of different lengths toward the solution.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Flag for inappropriate content
    Download as pdf or txt
    15 views4 pages

    MAT 461/561: 5.1 Stationary Iterative Methods

    Uploaded by

    Debisa
    This document discusses stationary iterative methods for solving linear systems Ax = b. [1] Stationary iterative methods repeatedly apply a fixed function g(x) = M^-1(Nx + b) to generate approximations x(k+1) from x(k). [2] The Jacobi and Gauss-Seidel methods are examples of stationary iterative methods that use diagonal splitting of A. [3] Successive overrelaxation (SOR) is an acceleration of Gauss-Seidel that introduces a relaxation parameter ω to take steps of different lengths toward the solution.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Flag for inappropriate content
    Download as pdf or txt
    of 4

    MAT 461/561: 5.

    1 Stationary Iterative Methods

    James V. Lambers

    March 2, 2020

    Announcements

    Iterative Methods for Linear Systems


    Why elimination (direct) methods for Ax = b are bad:

    • What if we already have an approximation of x? Example: implicit time-stepping methods


    for systems of ODEs. Gaussian elimination has NO way of exploiting it

    • What if we don’t really need THAT much accuracy? Elimination has no way of “quitting
    early” in exchange for less accuracy

    • What if A is sparse (mostly zero entries)? Elimination (and pivoting) can lead to “fill-in”

    • What if we don’t even have A? Example: instead we have a function A(x) that returns Ax

    Stationary Iterative Methods


    A stationary iterative method for Ax = b is a method of the form

    x(k+1) = g(x(k) )

    where x(0) is an initial guess. This is based on fixed-point iteration for solving x = g(x).
    Let A = M − N be a splitting of A. Then from Ax = (M − N )x = b, we have:

    Mx = Nx + b

    which leads to the iteration


    M x(k+1) = N x(k) + b.
    This is a fixed-point iteration with

    g(x) = M −1 (N x + b).

    To be practical, systems involving M must be easy to solve (e.g. M diagonal or triangular)

    1
    Convergence Analysis
    How do we know whether this iteration will converge? From equations above:

    x(k+1) − x = M −1 N (x(k) − x)

    Over k iterations:
    x(k) − x = (M −1 N )k (x(0) − x)
    Then T = M −1 N is the iteration matrix for this method. Take the norm of both sides:

    kx(k) − xk ≤ kT kk kx(0) − xk

    To ensure error → 0 as k → ∞, we can require kT k < 1 in some norm. This is sufficient but not
    necessary.
    A condition that is necessary and sufficient is ρ(T ), the spectral radius of T , is less than 1.
    The spectral radius is:
    ρ(T ) = max |λ|,
    λ∈λ(T )

    the magnitude of the largest eigenvalue of T . Recall λ is an eigenvalue of T if T x = λx for some


    x 6= 0. Note that ρ(T ) ≤ kT k for any natural norm k · k. This is because if T x = λx, where x 6= 0
    and |λ| = ρ(T ), then
    ρ(T )kxk ≤ kT kkxk

    The Jacobi Method


    To describe specfic iterative methods, we write

    A=L+D+U

    where D is the diagonal part, L is the strictly lower triangular part, and U is the strictly upper
    triangular part.
    The Jacobi Method comes from M = D, N = −(L + U ). Then we have

    x(k+1) = D−1 [b − (L + U )x(k) ].

    Each component:  
    n
    (k+1) 1  X (k)
    xi = bi − aij xj  , i = 1, 2, . . . , n
    aii
    j=1,j6=i

    Implementation: given x(0) (can use x(0) = 0), have an outer loop for computing each new x(k+1)
    from x(k) , k = 0, 1, 2, . . . until some stopping criterion is met (example: kx(k+1) − x(k) k < ) but
    should also set a maximum number of iterations.

    2
    The Gauss-Seidel Method
    If we examine Jacobi more closely:
     
    (k+1) 1  X (k)
    X (k)
    xi = bi − aij xj − aij xj  , i = 1, 2, . . . , n
    aii
    j<i j>i

    (k)
    This formula uses old information xj , j < i. If we update:
     
    (k+1) 1  X (k+1)
    X (k)
    xi = bi − aij xj − aij xj  , i = 1, 2, . . . , n
    aii
    j<i j>i

    This is the Gauss-Seidel method:


    h i
    x(k+1) = D−1 b − Lx(k+1) − U x(k)

    or h i
    x(k+1) = (D + L)−1 b − U x(k) .

    That is, M = D + L and N = −U . In some cases, it can be proven that Gauss-Seidel converges
    more rapidly than Jacobi. Disadvantage: Jacobi can be parallelized, Gauss-Seidel can’t.

    Successive Overrelaxation
    We rewrite Gauss-Seidel as follows:
     
    (k+1) (k) 1  X (k+1)
    X (k) (k)
    xi = xi + bi − aij xj − aij xj − aii xi  , i = 1, 2, . . . , n
    aii
    j<i j>i

    (k+1)
    To take a step of different length in the direction of xGS − x(k) , we introduce a relaxation
    parameter ω:
     
    (k+1) (k) ω bi −
    X (k+1)
    X (k) (k)
    xi = xi + aij xj − aij xj − aii xi  , i = 1, 2, . . . , n
    aii
    j<i j>i

    This leads to:


     
    (k+1) (k) (k+1) (k)
    X X
    aii xi = (1 − ω)aii xi + ω bi − aij xj − aij xj  , i = 1, 2, . . . , n
    j<i j>i

    or, in matrix form:


    h i
    Dx(k+1) = (1 − ω)Dx(k) + ω b − Lx(k+1) − U x(k)

    or:
    (D + ωL)x(k+1) = [(1 − ω)D − U ] x(k) + ωb.
    This is called Successive Overrelaxation (SOR). Convergence requires 0 < ω < 2. Why?

    3
    The iteration matrix is
    Tω = (D + ωL)−1 [(1 − ω)D − U ] .
    But
    n
    Y n
    Y
    det(Tω ) = a−1
    ii (1 − ω)aii = (1 − ω)n
    i=1 i=1

    and we have
    n
    Y
    det(Tω ) = λi
    i=1

    and therefore
    ρ(Tω ) ≥ |1 − ω|.
    SOR was introduced by David Young in his PhD dissertation, 1950.

    You might also like