A coupling method of learning structured support correlation filters for visual tracking

Abstract

The correlation filtering method is one of the mainstream methods in the visual target tracking task. One of the reasons is that the introduction of cyclic samples facilitates the calculation of optimizing filters. But usual correlation filtering frameworks which are attributable to a ridge regression model based on least squares error with various regularizations put emphasis on modeling a linear system for samples themselves, so maybe likely result in over-fitting. With the appearance of either target or background varying, the credibility of the response obtained after filtering would be decrease. Some researchers thus have tried to incorporate other models such as SVMs to obtain better robustness. In this study, we propose a natural coupling method called StrucSCF of integrating structured SVM by background awareness into the correlation filtering framework, which put more emphasis on the discrepancy between the target and background samples to enhance the discrimination and robustness of tracking. Meanwhile, for the sake of online updating the filters based on structured SVM with real-time performance, we take advantage of the fast Fourier transform on the circulant samples to speed up solving the structured SVM-based filters. In addition, we extend the StrucSCF method with Laplacian temporal regularization to demonstrate that it has as good quality of extension as the conventional correlation filtering framework. The proposed StrucSCF has achieved competitive performance compared with the baseline and other advanced methods in mainstream benchmarks.

Appendices

Appendix A

1.1 Convergence of {$\varvec{z}^\kappa$}

Here we here introduce an operator called proximal operator [60]. For a closed convex proper function $f:{\mathbb {R}}^n\rightarrow {\mathbb {R}}\bigcup \{-\infty ,\infty \}$, the proximal operator ${\textbf {prox}}_{\lambda f}:{\mathbb {R}}^n\rightarrow {\mathbb {R}}^n$ is

$$\begin{aligned} {\textbf {prox}}_{\lambda f}\left( \varvec{v}\right) =\arg \min _{\varvec{x}}f\left( \varvec{x}\right) +\left( 1/2\lambda \right) \left\| \varvec{x}-\varvec{v}\right\| ^2_2, \end{aligned}$$

(A.1)

where $\lambda \ge 0$, and $\left( 1/2\lambda \right)$ is short for $\left( 1/(2\lambda )\right)$.

If f is an indicator function $\textrm{I}_{{\mathbb {R}}_+^n}\left( \varvec{x}\right) =\left\{ \begin{aligned}0&,x_i \ge 0,\forall i\\\infty&,\textrm{or} \end{aligned}\right.$, the proximal operator will be ${\textbf {prox}}_{\lambda f}\left( \varvec{v}\right) =\Pi _{{\mathbb {R}}_+^n}\left( \varvec{v}\right) =\left[ \varvec{v}\right] ^+$, where $\Pi _C$ denotes project mapping to set C.

According to proximal operator, here we define an operator ${{\textbf {prox}}}_{\lambda f;g}: {\mathbb {R}}^n\rightarrow {\mathbb {R}}^n$ as follows

$$\begin{aligned} {\textbf {prox}}_{\lambda f;g}\left( \varvec{v}\right) \overset{\triangle }{=}\arg \min _{\varvec{x}}f\left( \varvec{x}\right) +\left( 1/2\lambda \right) \left\| g\left( \varvec{x}\right) -\varvec{v}\right\| _2^2, \end{aligned}$$

(A.2)

where $f:{\mathbb {R}}^n\rightarrow {\mathbb {R}} \bigcup \left\{ \infty \right\}$ is differentiable convex closed function, and $g:{\mathbb {R}}^n\rightarrow {\mathbb {R}}^n$ is affine function $g\left( \varvec{x}\right) =\varvec{Ax}+\varvec{b},\varvec{A}\in {\mathbb {R}}^{n\times n}, \varvec{b}\in {\mathbb {R}}^n$. Note that if let $\varvec{v}^*=\varvec{Ax}^*+\varvec{b}$, where $\varvec{x}^*=\arg \min _{\varvec{x}}f\left( \varvec{x}\right)$, then $\varvec{v}^*=g\left( {\textbf {prox}}_{\lambda f;g}\left( \varvec{v}^*\right) \right)$. Defining a notation $T=g\circ {\textbf {prox}}_{\lambda f;g}$, we know there exists fixed point of T due to $\varvec{v}^*=T\varvec{v}^*$, and $\varvec{v}^*$ is one of fixed point of T. We denote the fixed points set of T by $\textrm{Fix}\left( T\right)$, thus $\varvec{v}^*\in \textrm{Fix}\left( T\right)$. Now, we are going to prove that

Proposition 1

T is an averaged operator, strictly, a 1/2-averaged operator (also called firmly nonexpansive).

Remark

If T is referred to as $\lambda$-average operator, that means there exists nonexpansiveness operator $T'$ for $\lambda \in \left( 0,1\right)$ such that $T=\left( 1-\lambda \right) I+\lambda T'$, where I is identity mapping. According to $T=\left( 1/2\right) I+\left( 1/2\right) T' \iff T'=2T-I$, we get

$$\begin{aligned} \begin{aligned}&\left\| \varvec{v}_1-\varvec{v}_2\right\| _2^2\ge \left\| \left( 2T-I\right) \varvec{v}_1-\left( 2T-I\right) \varvec{v}_2\right\| _2^2\\ \iff&\left\| \varvec{v}_1-\varvec{v}_2\right\| _2^2\ge \left\| \varvec{v}_1-\varvec{v}_2-\left( T\varvec{v}_1-T\varvec{v}_2\right) \right\| _2^2\\&\qquad \qquad \qquad +\left\| T\varvec{v}_1-T\varvec{v}_2\right\| _2^2\\ \iff&\left<\varvec{v}_1-\varvec{v}_2,T\varvec{v}_1-T\varvec{v}_2\right>\ge \left\| T\varvec{v}_1-T\varvec{v}_2\right\| _2^2,\\ \forall \varvec{v}_1,&\varvec{v}_2\in \textrm{dom}T. \end{aligned} \end{aligned}$$

(A.3)

The so-called firm nonexpansiveness means T meets condition (A.3), e.g., $\Pi _{{\mathbb {R}}_+^n}$ is firmly nonexpansive as a result of $\left<\varvec{x}_1-\varvec{x}_2,\Pi _{{\mathbb {R}}_+^n}\varvec{x}_1-\Pi _{{\mathbb {R}}_+^n}\varvec{x}_2\right>\ge \left\| \Pi _{{\mathbb {R}}_+^n}\varvec{x}_1-\Pi _{{\mathbb {R}}_+^n}\varvec{x}_2\right\| _2^2$. Obviously, the average operator is a subset of nonexpansiveness operators, and the composition of average operator is still an average operator.

Proof

Given arbitrary $\varvec{v}_1,\varvec{v}_2\in {\mathbb {R}}^n$, let

$$\begin{aligned} \varvec{x}_1= & {} {\textbf {prox}}_{\lambda f;g}\left( \varvec{v}_1\right) \nonumber \\= & {} \arg \min _{\varvec{x}}f\left( \varvec{x}\right) +\left( 1/2\lambda \right) \left\| \varvec{Ax}+\varvec{b}-\varvec{v}_1\right\| _2^2, \end{aligned}$$

(A.4)

$$\begin{aligned} \varvec{x}_2= & {} {\textbf {prox}}_{\lambda f;g}\left( \varvec{v}_2\right) \nonumber \\= & {} \arg \min _{\varvec{x}}f\left( \varvec{x}\right) +\left( 1/2\lambda \right) \left\| \varvec{Ax}+\varvec{b}-\varvec{v}_2\right\| _2^2. \end{aligned}$$

(A.5)

According to optimal condition, we have

$$\begin{aligned} \varvec{0}= & {} \nabla f\left( \varvec{x}_1\right) +\left( 1/2\lambda \right) \varvec{A}^\textsf{T}\left( \varvec{Ax}_1+\varvec{b}-\varvec{v}_1\right) \nonumber \\{} & {} \iff \nabla f\left( \varvec{x}_1\right) =\left( 1/2\lambda \right) \varvec{A}^\textsf{T}\left( \varvec{v}_1-\varvec{Ax}_1-\varvec{b}\right) , \end{aligned}$$

(A.6)

$$\begin{aligned} \varvec{0}= & {} \nabla f\left( \varvec{x}_2\right) +\left( 1/2\lambda \right) \varvec{A}^\textsf{T}\left( \varvec{Ax}_2+\varvec{b}-\varvec{v}_2\right) \nonumber \\{} & {} \iff \nabla f\left( \varvec{x}_2\right) =\left( 1/2\lambda \right) \varvec{A}^\textsf{T}\left( \varvec{v}_2-\varvec{Ax}_2-\varvec{b}\right) . \end{aligned}$$

(A.7)

Due to f differentiable convex function, $\nabla f$ is monotone operator, which means that

$$\begin{aligned} \left<\varvec{x}_1-\varvec{x}_2,\nabla f\left( \varvec{x}_1\right) -\nabla f\left( \varvec{x}_2\right) \right>\ge \varvec{0}, \end{aligned}$$

(A.8)

then the above formula is substituted with (A.6) and (A.7), and using $T\varvec{v}_1=\varvec{Ax}_1 + \varvec{b},T\varvec{v}_2=\varvec{Ax}_2 + \varvec{b}$, we get

$$\begin{aligned}&\left<\varvec{x}_1-\varvec{x}_2,\varvec{A}^\textsf{T}\left( \varvec{v}_1-\varvec{Ax}_1\right) -\varvec{A}^\textsf{T}\left( \varvec{v}_2-\varvec{Ax}_2\right) \right>\ge \varvec{0}\nonumber \\&\iff \left<\varvec{Ax}_1-\varvec{Ax}_2,\varvec{v}_1-\varvec{Ax}_1-\left( \varvec{v}_2-\varvec{Ax}_2\right) \right>\ge \varvec{0}\nonumber \\&\iff \left<T\varvec{v}_1-T\varvec{v}_2,\varvec{v}_1-T\varvec{v}_1-\left( \varvec{v}_2-T\varvec{v}_2\right) \right>\ge \varvec{0}\nonumber \\&\iff \left<\varvec{v}_1-\varvec{v}_2,T\varvec{v}_1-T\varvec{v}_2\right>\ge \left\| T\varvec{v}_1-T\varvec{v}_2\right\| _2^2. \end{aligned}$$

(A.9)

So T is firmly nonexpansive; in other words, it is a 1/2-averaged operator. $\square$

Since T has fixed point $\varvec{v}^*$, then for arbitrary $\varvec{v}^1 \in {\mathbb {R}}^n$,

$$\begin{aligned}&\left\| \varvec{v}^1-\varvec{v}^*\right\| ^2_2\ge \left\| T\varvec{v}^1-\varvec{v}^*\right\| ^2_2+\left\| \varvec{v}^1-T\varvec{v}^1\right\| ^2_2\nonumber \\ \ge&\left\| T\varvec{v}^2-\varvec{v}^*\right\| ^2_2+\left\| \varvec{v}^2-T\varvec{v}^2\right\| ^2_2+\left\| \varvec{v}^1-T\varvec{v}^1\right\| ^2_2\nonumber \\ \ge&\ldots \ge \left\| T\varvec{v}^k-\varvec{v}^*\right\| ^2_2+\sum _{i=1}^{k}\left\| \varvec{v}^i-T\varvec{v}^i\right\| ^2_2, \end{aligned}$$

(A.10)

so $\sum _{i=1}^{\infty }\left\| \varvec{v}^i-T\varvec{v}^i\right\| ^2_2$ is bounded, hence $T\varvec{v}^k\rightarrow \varvec{v}^k$ as $k\rightarrow \infty$, which means that the sequence $\left\{ \varvec{v}^k\right\} _{k=1}^\infty$ generated by iteration form $\varvec{v}^{k+1}=T\varvec{v}^k$ is bounded and converges to a fixed point of T denoted by $\varvec{v}'=\lim \limits _{k\rightarrow \infty } \varvec{v}^k\in \textrm{Fix}\left( T\right)$. Consider a bounded subset D on ${\mathbb {R}}^n$ that contains $\textrm{Fix}\left( T\right)$ and whose intersection with ${\mathbb {R}}^n_+$ is not empty, for arbitrary $\varvec{x}^1\in D\bigcap {\mathbb {R}}_+^n$, the sequence $\left\{ \varvec{x}^k\right\} _{k=1}^\infty$ on ${\mathbb {R}}_+^n$ generated from iteration form $\varvec{x}^{k+1}=\Pi _{{\mathbb {R}}_+^n}\left( T\varvec{x}^{k}\right)$ is bounded. Let $S=\Pi _{{\mathbb {R}}_+^n}\circ T$; hence, S is also an averaged operator. Let ${\overline{{\textbf {conv}}}}D$ denote the closure of convex hull of $D\bigcup \left\{ \varvec{x}^k\right\} _{k=1}^\infty$, then $\left\{ \varvec{x}^k\right\} _{k=1}^\infty \subset \overline{{\textbf {conv}}}D \bigcap {\mathbb {R}}_+^n$, S is a mapping from $\overline{{\textbf {conv}}}D \bigcap {\mathbb {R}}_+^n$ to itself. For $\overline{{\textbf {conv}}}D \bigcap {\mathbb {R}}_+^n$ is compact because of $\overline{{\textbf {conv}}}D \bigcap {\mathbb {R}}_+^n$ bounded and closed, there exists a convergent subsequence $\left\{ \varvec{x}^{k_i}\right\} _{i=1}^\infty$ such that $\varvec{u}=\lim \limits _{i\rightarrow \infty }\varvec{x}^{k_i} \in \overline{{\textbf {conv}}}D \bigcap {\mathbb {R}}_+^n$. Then, we bring in a set of operator $S_i\varvec{x}^{k_i}=\left( 1-\lambda _i\right) \varvec{x}^{k_1}+\lambda _iS\varvec{x}^{k_i}$ where $\lambda _i \in \left( 0,1\right)$ is set as

$$\begin{aligned} \lambda _i=\frac{\varvec{x}^{k_i}-\varvec{x}^{k_1}}{S\varvec{x}^{k_i}-\varvec{x}^{k_1}}=\frac{\varvec{x}^{k_i}-\varvec{x}^{k_1}}{\varvec{x}^{k_i+1}-\varvec{x}^{k_1}}\ge \frac{\varvec{x}^{k_i}-\varvec{x}^{k_1}}{S\varvec{x}^{k_{i+1}}-\varvec{x}^{k_1}}; \end{aligned}$$

(A.11)

hence, $\lambda _i\rightarrow 1$ as $i\rightarrow \infty$ and $\varvec{x}^{k_i}=S_i\varvec{x}^{k_i}$. So

$$\begin{aligned} \begin{aligned} \left\| S\varvec{x}^{k_i}-\varvec{x}^{k_i}\right\| _2&=\left\| S\varvec{x}^{k_i}-\varvec{x}^{k_i}\right\| _2=\left\| S\varvec{x}^{k_i}-S_i\varvec{x}^{k_i}\right\| _2\\&=\left\| S\varvec{x}^{k_i}-\left( 1-\lambda _i\right) \varvec{x}^{k_1}-\lambda _iS\varvec{x}^{k_i}\right\| _2\\&=\left\| \left( 1-\lambda _i\right) \left( S\varvec{x}^{k_i}-\varvec{x}^{k_1}\right) \right\| _2\\&=\left( 1-\lambda _i\right) \left\| S\varvec{x}^{k_i}-\varvec{x}^{k_i}\right\| _2\rightarrow 0\left( i\rightarrow \infty \right) , \end{aligned} \end{aligned}$$

(A.12)

thereby

$$\begin{aligned} \begin{aligned}&\left\| \varvec{u}-S\varvec{u}\right\| _2=\left\| \varvec{u}-\varvec{x}^{k_i}-S\varvec{u}+S\varvec{x}^{k_i}-S\varvec{x}^{k_i}+\varvec{x}^{k_i}\right\| _2\\ \le&\left\| \varvec{u}-\varvec{x}^{k_i}\right\| _2+\left\| S\varvec{u}-S\varvec{x}^{k_i}\right\| _2+\left\| S\varvec{x}^{k_i}-\varvec{x}^{k_1}\right\| _2\\ =&\;2\left\| \varvec{u}-\varvec{x}^{k_i}\right\| _2+\left\| S\varvec{x}^{k-i}-\varvec{x}^{k_i}\right\| _2\rightarrow 0 \left( i\rightarrow \infty \right) , \end{aligned} \end{aligned}$$

(A.13)

that is, $\varvec{u}=S\varvec{u}$ is a fixed point of S. And

$$\begin{aligned} \begin{aligned}&\left\| \varvec{x}^{k+1}-\varvec{u}\right\| _2=\left\| S\varvec{x}^{k}-\varvec{u}\right\| _2\le \left\| \varvec{x}^{k}-\varvec{u}\right\| _2\le \left\| \varvec{x}^{k_i}-\varvec{u}\right\| _2,\\&\;\forall \; k>k_i, \end{aligned} \end{aligned}$$

(A.14)

so $\lim \limits _{k\rightarrow \infty }\left\| \varvec{x}^{k}-\varvec{u}\right\| _2= \lim \limits _{i\rightarrow \infty }\left\| \varvec{x}^{k_i}-\varvec{u}\right\| _2=0$, $\left\{ \varvec{x}^k\right\}$ generated from iteration form $\varvec{x}^{k+1}=S\varvec{x}^{k}$ converges to $\varvec{u}$.

Proposition 2

{$\varvec{z}^\kappa$} is convergent.

Proof

Let

$$\begin{aligned} \begin{aligned} {\mathcal {L}}_0\left( \varvec{f},\varvec{\gamma };\lambda ,\varvec{z}^\kappa \right) =&\frac{1}{2}\left\| \varvec{1}\varvec{x}^\textsf{T}\varvec{f}-\varvec{Xf}-\varvec{y}-\varvec{z}^\kappa \right\| ^2+\frac{\lambda }{2}\left\| \varvec{f}\right\| ^2\\&+\gamma ^\textsf{T}\overline{\varvec{P}}\varvec{f}. \end{aligned} \end{aligned}$$

(A.15)

In Algorithm 1 with regard to the kth iteration $\varvec{f}^k,\varvec{g}^k$ in the ADMM steps, we have $\varvec{f}^k-\varvec{g}^k \rightarrow \varvec{0}$, and

$$\begin{aligned}&\frac{1}{2}\left\| \varvec{1}\varvec{x}^\textsf{T}\varvec{g}^k\right\| ^2\!-\!\left<\varvec{1}\varvec{x}^\textsf{T}\varvec{g}^k,\varvec{Xg}^k\!+\!\varvec{y}+\varvec{z}\right>\!+\!\frac{1}{2n}\left\| \varvec{1}^\textsf{T}\varvec{Xg}^k\right\| ^2\nonumber \\&-\frac{1}{2n}\left\| \varvec{1}^\textsf{T}\varvec{Xf}^k\right\| ^2+\frac{1}{2}\left\| \varvec{Xf}^k+\varvec{y}+\varvec{z}\right\| ^2+\frac{\lambda }{2}\left\| \varvec{f}^k\right\| ^2\nonumber \\&\rightarrow \min _{\varvec{f}}{\mathcal {L}}_0\left( \varvec{f},\varvec{\gamma }|_{z^\kappa };\lambda ,\varvec{z}^\kappa \right) , \end{aligned}$$

(A.16)

where $\varvec{\gamma }^k|_{\varvec{z}^\kappa } =\arg \max _{\varvec{\gamma }} \inf _{\varvec{f}} {\mathcal {L}}_0\left( \varvec{f},\varvec{\gamma };\lambda ,\varvec{z}^\kappa \right)$, so $\varvec{f}^k, \varvec{g}^k\rightarrow \arg \min _{\varvec{f}}{\mathcal {L}}_0\left( \varvec{f},\varvec{\gamma }^k|_{\varvec{z}^\kappa };\lambda ,\varvec{z}^\kappa \right)$.

Let $f\left( \varvec{f}\right) =\left\| \varvec{f}\right\| ^2+\left( 2/\lambda \right) \left( \varvec{\gamma }^k|_{\varvec{z}^\kappa }\right) ^\textsf{T}\overline{\varvec{P}}\varvec{f},g\left( \varvec{f}\right) =\left( \varvec{1}\varvec{x}^\textsf{T}-\varvec{X}\right) \varvec{f}-\varvec{y}$, and follow the iteration steps below

$$\begin{aligned} \begin{aligned} \left\{ \begin{aligned} \varvec{f}^k|_{\varvec{z}^\kappa }&={\textbf {prox}}_{\lambda f;g}\left( \varvec{z}^\kappa \right) =\arg \min _{\varvec{f}}{\mathcal {L}}_0\left( \varvec{f},\varvec{\gamma }^k|_{\varvec{z}^\kappa };\lambda ,\varvec{z}^\kappa \right) \\ \varvec{z}^{\kappa +.5}&=g\big (\varvec{f}^k|_{\varvec{z}^\kappa }\big )=\varvec{1}\varvec{x}^\textsf{T}\varvec{f}^k|_{\varvec{z}^\kappa }-\varvec{X}\varvec{f}^k|_{\varvec{z}^\kappa }-\varvec{y}\\ \varvec{z}^{\kappa +1}&=\Pi _{{\mathbb {R}}_+^n}\left( \varvec{z}^{\kappa +.5}\right) \end{aligned}s \right. \end{aligned} \end{aligned}$$

(A.17)

Since $\varvec{z}^{\kappa +1}=\Pi _{{\mathbb {R}}_+^n}\left( \varvec{z}^{\kappa +.5}\right) =\Pi _{{\mathbb {R}}_+^n}\left( g\left( {\textbf {prox}}_{\lambda f;g}\left( \varvec{z}^\kappa \right) \right) \right) =\Pi _{{\mathbb {R}}_+^n}\left( T\varvec{z}^k\right) =S\varvec{z}^\kappa$, {$\varvec{z}^\kappa$} is convergent. $\square$

Appendix B

1.1 Derivation of $\widehat{\varvec{f}_l}$

Fixed $\varvec{g}_l,l=1,2,\ldots ,L$, let the partial derivative of (20) with regard to $\varvec{f}_l,l=1,2,\ldots ,L$ be zeros as

$$\begin{aligned} \begin{aligned} \frac{\partial {\mathcal {L}}_p}{\partial \varvec{f}_l}=&\;\varvec{X}_l^\textsf{T}\left( \varvec{y}+\varvec{z}+\varvec{X}_l\varvec{f}_l+\sum _{i\ne l}^{L}\varvec{X}_i\varvec{f}_i\right) \\&-\frac{1}{n}\varvec{X}_l^\textsf{T}\varvec{1}\left( \varvec{1}^\textsf{T}\varvec{X}_l\varvec{f}_l+\varvec{1}^\textsf{T}\sum _{i\ne l}^L\varvec{X}_i\varvec{f}_i\right) \\&+\lambda \varvec{f}_l+\rho \left( \varvec{f}_l-\varvec{g}_l+\varvec{\nu }_l\right) =\varvec{0},l=1,\ldots ,L. \end{aligned} \end{aligned}$$

(B.1)

Then, take DFT on both sides of (B.1) to get

$$\begin{aligned} \begin{aligned} \hspace{-0.07in}\varvec{0}=&\;\widehat{\varvec{x}_l}\odot \left( \widehat{\varvec{y}}+\widehat{\varvec{z}}+\widehat{\varvec{x}_l}^*\odot \widehat{\varvec{f}_l}+\sum _{i\ne l}^{L}\widehat{\varvec{x}_i}^*\odot \widehat{\varvec{f}_i}\right) \\&-\widehat{\varvec{x}_l}\!\odot \!\left[ 1,0,\ldots ,0\right] ^\textsf{T}\odot \left( \widehat{\varvec{x}_l}^*\!\odot \!\widehat{\varvec{f}_l}\!+\!\sum _{i\ne l}^L\widehat{\varvec{x}_i}^*\!\odot \!\widehat{\varvec{f}_i}\right) \\&+\lambda \widehat{\varvec{f}_l}+\rho \left( \widehat{\varvec{f}_l}-\widehat{\varvec{g}_l}+\widehat{\varvec{\nu }_l}\right) ,l=1,2,\ldots ,L. \end{aligned} \end{aligned}$$

(B.2)

To solve above nL equations, let $\widehat{\varvec{\psi }_l}=-\widehat{\varvec{x}_l}\odot \left( \widehat{\varvec{y}}+\widehat{\varvec{z}}\right) +\rho \left( \widehat{\varvec{g}_l}-\widehat{\varvec{\nu }_l}\right)$ and

$$\begin{aligned} \varvec{s}=\sum _{i=1}^{L}\widehat{\varvec{x}_i}^*\odot \widehat{\varvec{f}_i}, \end{aligned}$$

(B.3)

we have

$$\begin{aligned} \widehat{\varvec{f}_l}=\frac{1}{\rho +\lambda }\left( \widehat{\varvec{\psi }_l}\!-\!\widehat{\varvec{x}_l}\odot \left( 0,1,\ldots ,1\right) ^\textsf{T}\odot \varvec{s}\right) ,l\!=\!1,\ldots ,L; \end{aligned}$$

(B.4)

then, plug (B.4) into (B.3) to get

$$\begin{aligned} \varvec{s}\!=\!\frac{1}{\rho +\lambda }\sum _{i=1}^{L}\widehat{\varvec{x}_i}^*\odot \left( \widehat{\varvec{\psi }_i}\!-\!\widehat{\varvec{x}_i}\odot \left( 0,1,\ldots ,1\right) ^\textsf{T}\odot \varvec{s}\right) , \end{aligned}$$

(B.5)

that is,

$$\begin{aligned} \varvec{s}=\frac{\sum _{i=1}^{L}\widehat{\varvec{x}_i}^*\odot \widehat{\varvec{\psi }_i}}{\left( \rho +\lambda \right) \varvec{1}+\sum _{i=1}^{L}\widehat{\varvec{x}_i}\odot \left( 0,1,\ldots ,1\right) ^\textsf{T}\odot \widehat{\varvec{x}_i}^*}. \end{aligned}$$

(B.6)

Plugging (B.6) into (B.4) as well as setting $\widehat{\varvec{\upsilon }_l}=\widehat{\varvec{x}_l}^*\odot \left( 0,1,\ldots ,1\right) \odot \widehat{\varvec{\psi }_l},\widehat{\varvec{\omega }_l}=\widehat{\varvec{x}_l}^*\odot \left( 0,1,\ldots ,1\right) \odot \widehat{\varvec{x}_l}$, then we get

$$\begin{aligned} \widehat{\varvec{f}_l}=\frac{1}{\rho +\lambda }\left( \widehat{\varvec{\psi }_l}-\frac{\widehat{\varvec{x}_l}\odot \sum _i^L\widehat{\varvec{\upsilon }_i}}{\left( \rho +\lambda \right) \varvec{1}+\sum _i^L\widehat{\varvec{\omega }_i}}\right) ,l=1,\ldots ,L. \end{aligned}$$

(B.7)

Appendix C

1.1 Derivation of $\varvec{f}_l^\star$

Let $\nabla ^2\varvec{f}_l$ denote a matrix of the same shape as $\varvec{f}_l$, whose elements, if taking 4-neighborhood difference as an example, are $(\nabla ^2\varvec{f}_l)[u,v]{:=}\Delta (\varvec{f}_l[u,v]) =4\varvec{f}_l[u,v]-\varvec{f}_l[(u+1)\bmod H,v]-\varvec{f}_l[(u-1)\bmod H,v]-\varvec{f}_l[u,(v+1)\bmod W]-\varvec{f}_l[u,(v-1)\bmod W], u\in \{0,1,\ldots ,H\!-\!1\},v\in \{0,1,\ldots ,W\!-\!1\}$. According to linear and spatial-shifting property of DFT, we have $\widehat{\nabla ^2\varvec{f}_l}[u,v]=(4-e^{i\frac{2\pi }{H}u}-e^{-i\frac{2\pi }{H}u}-e^{j\frac{2\pi }{W}v}-e^{-j\frac{2\pi }{W}v})\widehat{\varvec{f}_l}[u,v]=(4-2{{\mathfrak {R}}}{{\mathfrak {e}}}(e^{i\frac{2\pi }{H}u}+e^{j\frac{2\pi }{W}v}))\widehat{\varvec{f}_l}[u,v]$. For the sake of convenience in formula, we flatten $\widehat{\varvec{f}_l}$ out as a vector and $\widehat{\nabla ^2\varvec{f}_l}$ as well. Then, set a vector $\varvec{\delta }$ whose elements $\varvec{\delta }[u\times W+v]=4-2{{\mathfrak {R}}}{\mathfrak {e}}(e^{i\frac{2\pi }{H}u}+e^{j\frac{2\pi }{W}v})$ for all $u\in \{0,1,\ldots ,H-1\}, v\in \{0,1,\ldots ,W-1\}$. According to Parseval Identity, take DFT on (26) to get the equivalent optimization objective

$$\begin{aligned} \min _{\widehat{\varvec{f}_l}}&\;\frac{1}{2n}\left\| \widehat{\varvec{y}}+\widehat{\varvec{z}}+\sum _l^L\widehat{\varvec{x}_l}^*\odot \widehat{\varvec{f}_l}\right\| ^2+\frac{\rho }{2n}\sum _l^L\left\| \widehat{\varvec{f}_l}-\widehat{\varvec{g}_l}+\widehat{\varvec{\nu }_l}\right\| ^2\nonumber \\&+\frac{\lambda }{2n}\sum _l^L\left\| \widehat{\varvec{f}_l}\right\| ^2+\frac{\theta }{2n}\sum _l^L\left\| \varvec{\delta }\odot \left( \widehat{\varvec{f}_l}-\widehat{\varvec{f}_l^\textrm{pre}}\right) \right\| ^2\nonumber \\&-\frac{1}{2n}\sum _l^L\left\| (1,0,\ldots ,0)\odot \widehat{\varvec{x}_l}^*\odot \widehat{\varvec{f}_l}\right\| ^2. \end{aligned}$$

(C.1)

Let the partial derivative of (C.1) with regard to $\widehat{\varvec{f}_l^\star }, l=1,2,\ldots ,L$ be zeros as

$$\begin{aligned} \begin{aligned} \hspace{-0.07in}\varvec{0}=&\;\widehat{\varvec{x}_l}\odot \left( \widehat{\varvec{y}}+\widehat{\varvec{z}}+\widehat{\varvec{x}_l}^*\odot \widehat{\varvec{f}_l}+\sum _{i\ne l}^{L}\widehat{\varvec{x}_i}^*\odot \widehat{\varvec{f}_i}\right) \\&-\widehat{\varvec{x}_l}\!\odot \!\left[ 1,0,\ldots ,0\right] ^\textsf{T}\odot \left( \widehat{\varvec{x}_l}^*\!\odot \!\widehat{\varvec{f}_l}\!+\!\sum _{i\ne l}^L\widehat{\varvec{x}_i}^*\!\odot \!\widehat{\varvec{f}_i}\right) \\&+\lambda \widehat{\varvec{f}_l}+\rho \left( \widehat{\varvec{f}_l}-\widehat{\varvec{g}_l}+\widehat{\varvec{\nu }_l}\right) +\theta \varvec{d}\odot \left( \widehat{\varvec{f}_l}-\widehat{\varvec{f}_l^\textrm{pre}}\right) \!,\\ l=&\;1,2,\ldots ,L \end{aligned} \end{aligned}$$

(C.2)

where $\varvec{d}=\varvec{\delta }^*\odot \varvec{\delta }$. The rest steps are similar to that in Appendix B.

Let $\widehat{\varvec{\psi }^\Delta _l}=\widehat{\varvec{\psi }_l}+\theta \varvec{d}\odot \widehat{\varvec{f}_l^\textrm{pre}},\widehat{\varvec{\upsilon }^\Delta _l}=\widehat{\varvec{x}_l}^*\odot \left( 0,1,\ldots ,1\right) \odot \widehat{\varvec{\psi }^\Delta _l},l=1,2,\ldots ,L$, and finally, we have

$$\begin{aligned} \begin{aligned}&\widehat{\varvec{f}_l^\star }=\frac{\varvec{1}}{\left( \rho \!+\!\lambda \right) \!\varvec{1}\!+\!\theta \varvec{d}}\!\left( \widehat{\varvec{\psi }_l^\Delta }\!-\!\frac{\widehat{\varvec{x}_l}\odot \sum _i^L\widehat{\varvec{\upsilon }^\Delta _i}}{\left( \rho \!+\!\lambda \right) \!\varvec{1}\!+\!\theta \varvec{d}\!+\!\sum _i^L\widehat{\varvec{\omega }_i}}\right) ,\\&l=1,2,\ldots ,L, \end{aligned} \end{aligned}$$

(C.3)

and then, take IFFT to get $\varvec{f}_l^\star$.