Alternating direction method with Gaussian back substitution for separable convex programming

B He, M Tao, X Yuan - SIAM Journal on Optimization, 2012 - SIAM
SIAM Journal on Optimization, 2012SIAM
We consider the linearly constrained separable convex minimization problem whose
objective function is separable into m individual convex functions with nonoverlapping
variables. A Douglas–Rachford alternating direction method of multipliers (ADM) has been
well studied in the literature for the special case of m=2. But the convergence of extending
ADM to the general case of m≥3 is still open. In this paper, we show that the straightforward
extension of ADM is valid for the general case of m≥3 if it is combined with a Gaussian back …
We consider the linearly constrained separable convex minimization problem whose objective function is separable into m individual convex functions with nonoverlapping variables. A Douglas–Rachford alternating direction method of multipliers (ADM) has been well studied in the literature for the special case of . But the convergence of extending ADM to the general case of is still open. In this paper, we show that the straightforward extension of ADM is valid for the general case of if it is combined with a Gaussian back substitution procedure. The resulting ADM with Gaussian back substitution is a novel approach towards the extension of ADM from to , and its algorithmic framework is new in the literature. For the ADM with Gaussian back substitution, we prove its convergence via the analytic framework of contractive-type methods, and we show its numerical efficiency by some application problems.
Society for Industrial and Applied Mathematics