Asymptotic behaviors of semidefinite programming with a covariance perturbation

Abstract

In this paper, we study asymptotic behaviors of semidefinite programming with a covariance perturbation. We obtain some moderate deviations, Cramér-type moderate deviations and a law of the iterated logarithm of estimates of the respective optimal value and optimal solutions when the covariance matrix is estimated by its sample covariance. As an example, we also apply the main results to the Minimum Trace factor Analysis.

Appendix: Differentiability properties of the optimal value and an optimal solution

For convenience, in this “Appendix”, we recall some results on differentiability properties of the optimal value $\vartheta (\Sigma )$ and an optimal solution ${\bar{x}}(\Sigma )$ of the following problem (A.1) considered as functions of matrix $\Sigma \in {\mathbb {S}}^p$ (see [26]):

$$\begin{aligned} \min _{x\in {\mathbb {R}}^n} c^Tx\quad \text{ subject } \text{ to } \Sigma +{\mathcal {A}}(x)\succcurlyeq 0, \end{aligned}$$

(A.1)

which can be viewed as an SDP problem parameterized by matrix $\Sigma \in {\mathbb {S}}^p$.

The (Lagrangian) dual of problem (A.1) can be written as

$$\begin{aligned} \max _{\Lambda \in {\mathbb {S}}^p_+ }\Lambda \bullet \Sigma \quad \text{ subject } \text{ to } \Lambda \bullet A_i=c_i,~i=1,\ldots ,n \end{aligned}$$

(A.2)

The problems (A.1) and (A.2) are refered as the primal (P) and dual (D) problems, respectively. We also use notation $\sigma :={\mathrm{vec}\,}(\Sigma )$, ${\bar{x}}(\sigma ):={\bar{x}}(\Sigma )$ and $\vartheta (\sigma ):=\vartheta (\Sigma )$.

Slater condition    It is said that Slater condition holds for the primal problem (P) if there exists $x^* \in {\mathbb {R}}^n$ such that $\Sigma +{\mathcal {A}}(x^*)\in {\mathbb {S}}^p_{++}$. If Slater condition holds, then optimal values of problems (P) and (D) are equal to each other.

Let ${\mathcal {W}}_r$ denote the space of matrices $A \in {\mathbb {S}}^p$ of ${\mathrm{rank}\,}(A) = r \le p$. Then by Proposition 1.1, Chapter 5 in [10], ${\mathcal {W}}_r$ is a smooth manifold of dimension

$$\begin{aligned} {\mathrm{dim}\,}({\mathcal {W}}_r) = p(p + 1)/2 - (p - r)(p - r + 1)/2 = pr - r(r - 1)/2, \end{aligned}$$

and the tangent space of the manifold ${\mathcal {W}}_r$ at $A \in {\mathcal {W}}_r$ is

$$\begin{aligned} T_{{\mathcal {W}}_r} (A)=\left\{ \Delta A+A\Delta ^T;~\Delta \text{ is } p\times p \text{ matrix } \right\} . \end{aligned}$$

Nondegenerate point    It is said that $x^*\in {\mathbb {R}}^n$ is a nondegenerate point of mapping $x \rightarrow \Sigma +{\mathcal {A}}(x)$ if for $\Upsilon := \Sigma + {\mathcal {A}}(x^*)$ and $r := {\mathrm{rank}\,}(\Upsilon )$ it follows that

$$\begin{aligned} {\mathcal {A}}({\mathbb {R}}^n) + T_{{\mathcal {W}}_r} (\Upsilon ) ={\mathbb {S}}^p, \end{aligned}$$

otherwise point $x^*$ is said to be degenerate.

1.1 A.1: Differentiability of the optimal value $\vartheta (\Sigma )$

Let ${\mathrm{Sol}\,}(P)$ denote the set of optimal solutions of the reference (true) problem (1.1), and let ${\mathrm{Sol}\,}(D)$ be the set of optimal solutions of its dual problem (A.1) for $\Sigma =\Sigma _0$. By the classical convex analysis and Theorem 4.1.9 in [25], the following result holds.

Proposition 3.1

(Proposition 3 in [26]) Suppose that Slater condition holds for the reference problem (1.1) and its optimal value $\vartheta (\Sigma _0)$ is finite. Then the set ${\mathrm{Sol}\,}(D)$ is nonempty, convex and compact and the optimal value function $\vartheta (\cdot )$ is continuous convex function and Fréchet directionally differentiable at $\Sigma _0$ with

$$\begin{aligned} \vartheta ^\prime (\Sigma _0, H) = \sup _{\Lambda \in {\mathrm{Sol}\,}(D)} \Lambda \bullet H. \end{aligned}$$

(A.3)

That is,

$$\begin{aligned} \vartheta (\Sigma )-\vartheta (\Sigma _0)=\sup _{\Lambda \in {\mathrm{Sol}\,}(D)} \Lambda \bullet (\Sigma -\Sigma _0)+o(\Vert \sigma -\sigma _0\Vert ) \end{aligned}$$

(A.4)

1.2 A.2: The second order differentiability of the optimal value $\vartheta (\Sigma )$

Suppose that ${\mathrm{Sol}\,}(P) = \{x^*\}$ and that $x^*$ is a nondegenerate point of $\Sigma _0 +{\mathcal {A}}(\cdot )$, and so ${\mathrm{Sol}\,}(D) = \{\Lambda \}$ is a singleton.

Complementarity condition    Assume that Slater condition holds for the reference problem (1.1). Then by the first order optimality conditions we have that for $x^* \in {\mathrm{Sol}\,}(P)$ and $\Lambda \in {\mathrm{Sol}\,}(D)$ the following complementarity condition follows

$$\begin{aligned} (\Sigma _0 +{\mathcal {A}}(x^*)) \bullet \Lambda = 0. \end{aligned}$$

Note that since $(\Sigma _0 +{\mathcal {A}}(x^*)) \succcurlyeq 0$ and $\Lambda \succcurlyeq 0$, this complementarity condition is equivalent to $(\Sigma _0 +{\mathcal {A}}(x^*)) \Lambda = 0$ and hence ${\mathrm{rank}\,}(\Lambda ) \le p- r$, where

$$\begin{aligned} r := {\mathrm{rank}\,}(\Sigma _0 +{\mathcal {A}}(x^*)). \end{aligned}$$

It is said that the strict complementarity condition holds at $\Lambda \in {\mathrm{Sol}\,}(D)$ if ${\mathrm{rank}\,}(\Lambda ) = p -r$.

Suppose also that the strict complementarity condition holds. Let $\Upsilon = NDN^T$ be the spectral decomposition of matrix $\Upsilon =\Sigma _0 +{\mathcal {A}}(x^*)$, and $\Lambda = EE^T$ for some $p\times (p-r)$ matrix E of rank $p-r$ such that $N^TE = 0$. It is known (see [26]) that the following optimization problem (A.5) depending on $\Delta \in {\mathbb {S}}^p$ has a unique optimal solution $J^T\delta$ and the optimal value is a quadratic function $\delta ^TQ\delta$ where $\delta :={\mathrm{vec}\,}(\Delta )$, J is a $p^2\times n$ matrix and Q is a $p^2\times p^2$ matrix.

$$\begin{aligned} \left\{ \begin{aligned} \min _{h\in {\mathbb {R}}^n}&\;\; {\mathrm{tr}\,}\left( \Lambda ({\mathcal {A}}(h) + \Delta )\Upsilon ^\dagger ({\mathcal {A}}(h) +\Delta )\right) \\ s.t.&\;\; E^T{\mathcal {A}}(h)E + E^T\Delta E = 0. \end{aligned} \right. \end{aligned}$$

(A.5)

The following result is Theorem 1 in [26] which can be obtained from Section 5.3.6 in [2].

Proposition 3.2

(Theorem 1 in [26]) Suppose that ${\mathrm{Sol}\,}(P) = \{x^*\}$ is a singleton, and that $x^*$ is a nondegenerate point of $\Sigma _0 +{\mathcal {A}}(\cdot )$ and the strict complementarity condition holds. Then ${\bar{x}}(\cdot )$ is differentiable at $\sigma _0 = {\mathrm{vec}\,}(\Sigma _0)$ and

$$\begin{aligned} {\bar{x}}(\sigma ) = {\bar{x}}(\sigma _0) + J^T(\sigma - \sigma _0) + o(\Vert \sigma -\sigma _0\Vert ), \end{aligned}$$

(A.6)

where $J^T\delta$ is the optimal solution of problem (A.5). Moreover

$$\begin{aligned} \vartheta (\sigma ) =\vartheta (\sigma _0) + \Lambda \bullet (\Sigma -\Sigma _0) + (\sigma -\sigma _0)^TQ(\sigma -\sigma _0) + o(\Vert \sigma -\sigma _0\Vert ^2),\nonumber \\ \end{aligned}$$

(A.7)

where $\Lambda$ is the optimal solution of the dual problem and $\delta ^TQ\delta$ is the optimal value of problem (A.5).