Spatial regression with multiplicative errors, and its application with LiDAR measurements

Abstract

Multiplicative errors in addition to spatially referenced observations often arise in geodetic applications, particularly with light detection and ranging (LiDAR) measurements. However, regression involving multiplicative errors remains relatively unexplored in such applications. In this regard, we present a penalized modified least squares estimator to handle the complexities of a multiplicative error structure while identifying significant variables in spatially dependent observations. The proposed estimator can be also applied to classical additive error spatial regression. By establishing asymptotic properties of the proposed estimator under increasing domain asymptotics with stochastic sampling design, we provide a rigorous foundation for its effectiveness. A comprehensive simulation study confirms the superior performance of our proposed estimator in accurately estimating and selecting parameters, outperforming existing approaches. To demonstrate its real-world applicability, we employ our proposed method, along with other alternative techniques, to estimate a rotational landslide surface using LiDAR measurements. The results highlight the efficacy and potential of our approach in tackling complex spatial regression problems involving multiplicative errors.

Appendix A Proofs

We introduce a lemma with reference where the proof of the lemma is provided, so we omit the proof here.

Lemma 1

[Theorem 3.2 from Lahiri (2003)]

A random field $\{\xi (\varvec{s}): \varvec{s} \in {\mathcal {R}}^d\}$ is stationary with $\text{ E }\xi (\varvec{0}) =0$, $\text{ E }|\xi (\varvec{0})|^{2+\delta } < \infty$ for some $\delta >0$. In addition, if the conditions below are fulfilled,

• (a) There exists a function $H_1(\cdot )$ such that for all $\varvec{h} \in {\mathcal {R}}^d$, as $n \rightarrow \infty$,

  $$\begin{aligned} \left[ \int w_n^2(\eta _n \varvec{u}) \phi (\varvec{u}) d\varvec{u}\right] ^{-1}\int w_n(\eta _n \varvec{u}+\varvec{h})w_n(\eta _n \varvec{u}) \phi ^2(\varvec{u})d\varvec{u} \rightarrow H_1(\varvec{h}) \end{aligned}$$

• (b) $\gamma _{1n}^2 = O(n^a)\ \& \ o\left( [\log n]^{-2}\eta _n^{\frac{\tau -d}{4\tau }}\right)$ for some $a \in [0, 1/8)$ and $\tau >d$, where

  $$\begin{aligned} \gamma _{1n}^2 = \frac{\sup \{|w_n(\varvec{s})|^2:\varvec{s}\in \eta _n R_0\}}{\text{ E }[w_n^2(\eta _n \varvec{U}_1)]} \end{aligned}$$

• (c) $\phi (\varvec{u})$ is continuous and everywhere positive with support ${\bar{R}}_0$

• (d) For $\alpha _1(\cdot )$ from assumption,

  • 1. $\int _1^{\infty } t^{d-1} \alpha _1(t)^{\frac{\delta }{2+\delta }}<\infty$

  • 2. $\alpha _1(t) \sim t^{-\tau }L(t)$ as $t \rightarrow \infty$ for some slowly varying function $L(\cdot )$

• (e) For $d\ge 2$ and $\alpha _2(\cdot )$ from assumption, as $t \rightarrow \infty$

  $$\begin{aligned} \alpha _2(t) = o\left( [L(t)^{\frac{\tau +d}{4d\tau }}]^{-1}t^{\frac{\tau -d}{4d}} \right) \end{aligned}$$

then, with $n/\eta _n^d \rightarrow C_1 \in (0, \infty )$, as $n\rightarrow \infty$

$$\begin{aligned} \left( n \text{ E }[w_n^2(\eta _n \varvec{U}_1)]\right) ^{-1/2} \sum _{i=1}^{n} w_n(\varvec{s}_i)\xi (\varvec{s}_i) \overset{d}{\rightarrow }\ N\left( 0, \sigma (\varvec{0})+C_1\int \sigma (\varvec{h}) H_1(\varvec{h}) d\varvec{h}\right) \end{aligned}$$

Proof of Theorem 1

For a large enough n, we have

$$\begin{aligned} S_n(\varvec{\theta }_0+a_n\varvec{v}) -S_n(\varvec{\theta }_0)&= a_n\left( \nabla S_n(\varvec{\theta }_0)\right) ^{T}\varvec{v} +\frac{1}{2}n^{-1}\varvec{v}^{T}\nabla ^2 S_n(\varvec{\theta }_0+a_n\varvec{v}t) \varvec{v}\\&:={\mathbb {A}}+ {\mathbb {B}}, \end{aligned}$$

where $t \in (0,1)$. For the term ${\mathbb {A}}$,

$$\begin{aligned} {\mathbb {A}}&= a_n\left( \nabla S_n(\varvec{\theta }_0)\right) ^{T}\varvec{v}\\&=-2a_n\varvec{v}^{T}\varvec{\dot{G}}(\varvec{\theta }_0)^{T}\varvec{\Sigma }_n\varvec{\varepsilon },\\&=-2a_n\varvec{v}^{T}\varvec{\dot{G}}(\varvec{\theta }_0)^{T}\varvec{\Sigma }_n\varvec{\xi },\;\;\text{ where } \varvec{\xi }=\varvec{\varepsilon }-\mu \varvec{1} \end{aligned}$$

We follow a similar proof to You et al. (2024) and evaluate ${\text {Var}}({\mathbb {A}})$ since ${\mathbb {A}}-\text{ E }({\mathbb {A}}) = O_p\left( \sqrt{{\text {Var}}({\mathbb {A}})}\right)$.

$$\begin{aligned} {\text {Var}}({\mathbb {A}})&= 4 a_n^2 \varvec{v}^T \varvec{\dot{G}}(\varvec{\theta }_0)^{T} \varvec{\Sigma }_n \varvec{\Sigma }_\varepsilon \varvec{\Sigma }_n \varvec{\dot{G}}(\varvec{\theta }_0) \varvec{v} \\&\le 4a_n^2 \Vert \varvec{\dot{G}}(\varvec{\theta }_0) \varvec{v}\Vert ^2 \lambda _{\max }\left( \varvec{\Sigma }_n \varvec{\Sigma }_\varepsilon \varvec{\Sigma }_n\right) \\&= 4a_n^2 \Vert \varvec{\dot{G}}(\varvec{\theta }_0) \varvec{v}\Vert ^2 \lambda _{\max }\left( \varvec{\Sigma }_n \varvec{\Sigma }_\varepsilon \right) \\&\le 4a_n^2 \Vert \varvec{\dot{G}}(\varvec{\theta }_0) \varvec{v}\Vert ^2 \lambda _\varepsilon \\&= O\left( na_n^2 \lambda _\varepsilon \right) \Vert \varvec{v}\Vert ^2, \end{aligned}$$

where $\lambda _{max}(\varvec{M})$ refers to the maximum eigenvalue of a matrix $\varvec{M}$. The second equality holds since products of matrices AB and BA share the same non-zero eigenvalues and $\varvec{\Sigma }_n^2 = \varvec{\Sigma }_n$. The second inequality holds because $\varvec{\Sigma }_n$ has eigenvalues of 0 and 1’s. The last equality comes from the compactness of the domain ${\mathcal {D}}\times \Theta$ by Assumption 1-(1). Thus, ${\text {Var}}({\mathbb {A}}) = O\left( n a_n^2 \lambda _\varepsilon \right) \Vert \varvec{v} \Vert ^2$, which leads to

$$\begin{aligned} {\mathbb {A}} = O_p\left( n^{1/2} a_n \lambda _\varepsilon ^{1/2}\right) \Vert \varvec{v} \Vert \end{aligned}$$

(A1)

Now, we decompose ${\mathbb {B}}$ into 4 terms as follows.

$$\begin{aligned} {\mathbb {B}}&= \frac{1}{2}a_n^2 \varvec{v}^T \nabla ^2 \varvec{S}_n(\varvec{\theta }_n)\varvec{v} \\&= a_n^2 \varvec{v}^T \left( \varvec{\dot{G}}(\varvec{\theta }_n)^{T}\varvec{\Sigma }_n \varvec{\dot{G}}(\varvec{\theta }_n) - \varvec{\dot{G}}(\varvec{\theta }_0)^{T} \varvec{\Sigma }_n \varvec{\dot{G}}(\varvec{\theta }_0) \right) \varvec{v} +\varvec{v} ^T \varvec{\dot{G}}(\varvec{\theta }_0)^{T} \varvec{\Sigma }_n \varvec{\dot{G}}(\varvec{\theta }_0)\varvec{v} \\&~~~ + a_n^2 \varvec{v}^T \varvec{\ddot{G}}(\varvec{\theta }_n)^T \left( I\otimes \varvec{\Sigma }_n \varvec{d}(\varvec{\theta }_n, \varvec{\theta }_0) \right) \varvec{v} - a_n^2 \varvec{v}^T \varvec{\ddot{G}}(\varvec{\theta }_n)^T \left( I\otimes \varvec{\Sigma }_n \varvec{\xi }\right) \varvec{v} \\&= {\mathbb {B}}_1+{\mathbb {B}}_2+{\mathbb {B}}_3+{\mathbb {B}}_4 \end{aligned}$$

where $\varvec{\theta }_n = \varvec{\theta }_0 + a_n \varvec{v} t$ and $\varvec{d}(\varvec{\theta }, \varvec{\theta }') = (d_1(\varvec{\theta }, \varvec{\theta }'), \ldots , d_n(\varvec{\theta }, \varvec{\theta }'))^T$ with $d_i(\varvec{\theta }, \varvec{\theta }') = g(\varvec{x}(\varvec{s}_i); \varvec{\theta })-g(\varvec{x}(\varvec{s}_i); \varvec{\theta }')$. Since $n^{-1}{\varvec{\dot{G}}}({\varvec{\theta }})^{T}\varvec{\Sigma }_n {\varvec{\dot{G}}}({\varvec{\theta }})$ is a uniformly continuous function of $\varvec{\theta }$ and converges to $\varvec{\Sigma }_g = \varvec{\Delta }_{g^2}-\varvec{\Delta }_g\varvec{\Delta }_g^T$ at $\varvec{\theta }_0$ in probability by Assumptions 1-(4) and (5),

$$\begin{aligned} {\mathbb {B}}_1 = na_n^2\varvec{v}^T\left( n^{-1}\varvec{\dot{G}}(\varvec{\theta }_n)^{T}\varvec{\Sigma }_n \varvec{\dot{G}}(\varvec{\theta }_n) - n^{-1}\varvec{\dot{G}}(\varvec{\theta }_0)^{T} \varvec{\Sigma }_n \varvec{\dot{G}}(\varvec{\theta }_0)\right) \varvec{v} = o(na_n^2)\Vert \varvec{v}\Vert ^2. \end{aligned}$$

(A2)

By nature, ${\mathbb {B}}_2 = na_n^2 \varvec{v}^T \varvec{\Sigma }_g \varvec{v} (1+o(1))$. Note that ${\mathbb {B}}_2$ is positive with the order of $na_n^2 = \lambda _\varepsilon$ and $\lambda _\varepsilon$ does not vanish to zero. Next,

$$\begin{aligned} {\mathbb {B}}_3&= a_n^2 \varvec{v}^T \varvec{\ddot{G}}(\varvec{\theta }_n)^T \left( I\otimes \varvec{\Sigma }_n \varvec{d}(\varvec{\theta }_n, \varvec{\theta }_0) \right) \varvec{v} \\&= a_n^2 \varvec{v}^T \left[ \varvec{g}_{kl}(\varvec{\theta }_n)^T \varvec{\Sigma }_n \varvec{d}(\varvec{\theta }_n, \varvec{\theta }_0) \right] _{k, l = 1, \ldots , p} \varvec{v}, \end{aligned}$$

where $[a_{kl}]_{k,l=1, \ldots , p}$ is a matrix of which component at k-th row and l-th column equals to $a_{kl}$.

$$\begin{aligned} \left| \varvec{g}_{kl}(\varvec{\theta }_n)^T \varvec{\Sigma }_n \varvec{d}(\varvec{\theta }_n, \varvec{\theta }_0)\right|&\le \left| \varvec{g}_{kl}(\varvec{\theta }_n)^T \varvec{\Sigma }_n\varvec{g}_{kl}(\varvec{\theta }_n)\right| ^{1/2}\left| \varvec{d}(\varvec{\theta }_n, \varvec{\theta }_0)^T\varvec{\Sigma }_n \varvec{d}(\varvec{\theta }_n, \varvec{\theta }_0) \right| ^{1/2} \\&\le \Vert \varvec{g}_{kl}(\varvec{\theta }_n)\Vert \Vert \varvec{d}(\varvec{\theta }_n, \varvec{\theta }_0)\Vert \\&= O(n^{1/2}) O(\Vert \varvec{\theta }_n - \varvec{\theta }_0\Vert ) \\&= O(n^{1/2}a_n)\Vert \varvec{v}\Vert \end{aligned}$$

The first inequality is Cauchy-Schwarz inequality with semi-norm and the second inequality holds since the maximum eigenvalue of $\varvec{\Sigma }_n$ is 1. The compactness of the domain explains the first and second equalities. To sum up,

$$\begin{aligned} {\mathbb {B}}_3 = O(n^{1/2}a_n^3) \Vert \varvec{v}\Vert ^3. \end{aligned}$$

(A3)

Similarly, to deal with ${\mathbb {B}}_4$ we evaluate $\left[ \varvec{g}_{kl}(\varvec{\theta }_n)^T \varvec{\Sigma }_n \varvec{\xi }\right]$ for all $k,l =1,\ldots , p$.

$$\begin{aligned} {\text {Var}}\left( \varvec{g}_{kl}(\varvec{\theta }_n)^T \varvec{\Sigma }_n \varvec{\xi }\right)&= \varvec{g}_{kl}(\varvec{\theta }_n)^T \varvec{\Sigma }_n \varvec{\Sigma }_\varepsilon \varvec{\Sigma }_n \varvec{g}_{kl}(\varvec{\theta }_n) \\&\le \left\| \varvec{g}_{kl}(\varvec{\theta }_n)\right\| ^2 \lambda _\varepsilon \\&= O(n\lambda _\varepsilon ). \end{aligned}$$

Recall that the maximum eigenvalue of $\varvec{\Sigma }_n \varvec{\Sigma }_\varepsilon \varvec{\Sigma }_n$ is not greater than $\lambda _\varepsilon$. Since $\text{ E }(\varvec{g}_{kl}(\varvec{\theta }_n)^T \varvec{\Sigma }_n \varvec{\xi })=0$, $\varvec{g}_{kl}(\varvec{\theta }_n)^T \varvec{\Sigma }_n \varvec{\xi }=O_p(n^{1/2}\lambda _\varepsilon ^{1/2})$ and

$$\begin{aligned} {\mathbb {B}}_4 = O_p(n^{1/2}a_n^2\lambda _\varepsilon ^{1/2})\Vert \varvec{v}\Vert ^2. \end{aligned}$$

(A4)

Therefore, conditional on $\varvec{U}$,

$$\begin{aligned} {\mathbb {B}}&= {\mathbb {B}}_1 +{\mathbb {B}}_2+{\mathbb {B}}_3+{\mathbb {B}}_4 \nonumber \\&= o(na_n^2)\Vert \varvec{v}\Vert ^2 +na_n^2 \varvec{v}^T \varvec{\Sigma }_g \varvec{v} (1+o(1))+O(n^{1/2}a_n^3)\Vert \varvec{v}\Vert ^3 +O_p(n^{1/2}a_n^2 \lambda _\varepsilon ^{1/2})\Vert \varvec{v}\Vert ^2 \nonumber \\&= na_n^2 \varvec{v}^T\varvec{\Sigma }_g \varvec{v} (1+o(1)) + O_p(n^{1/2}a_n^2 \lambda _\varepsilon ^{1/2})\Vert \varvec{v}\Vert ^2 \nonumber \\&= na_n^2\varvec{v}^T\varvec{\Sigma }_g \varvec{v} (1+o_p(1)) \end{aligned}$$

(A5)

The last equality holds for $\lambda _\varepsilon =o(n)$ in Assumption 2-(1). Finally, by (A1) and (A5), we attain

$$\begin{aligned} \mathbb {A+B}&= O_p\left( n^{1/2}a_n \lambda _\varepsilon ^{1/2} \right) \Vert \varvec{v}\Vert + na_n^2\varvec{v}^T\varvec{\Sigma }_g \varvec{v} (1+o_p(1)) \nonumber \\&= O_p\left( \lambda _\varepsilon \right) \Vert \varvec{v}\Vert + \lambda _\varepsilon \varvec{v}^T\varvec{\Sigma }_g \varvec{v} (1+o_p(1)) \end{aligned}$$

(A6)

With large enough $\Vert \varvec{v}\Vert$, (A6) stays positive with probability tending to 1, which completes the proof.

Proof of Theorem  2

Let $b_n = a_n+c_n$,

$$\begin{aligned}{} & {} Q_n(\varvec{\theta }_0+b_n \varvec{v})-Q_n(\varvec{\theta }_0) \nonumber \\{} & {} ~~~ = \mathbb {A'+B'}+\displaystyle n \left( \sum _{i=1}^{p} p_{\lambda _{n}}(|\theta _{0i}+b_n v_i|)-p_{\lambda _{n}} (|\theta _{0i}|) \right) \nonumber \\{} & {} ~~~ =\mathbb {A'+B'}+\displaystyle n \left( \sum _{i=1}^{s} p_{\lambda _{n}}(|\theta _{0i}+b_n v_i|)-p_{\lambda _{n}} (|\theta _{0i}|) \right) +n\sum _{i=s+1}^{p} p_{\lambda _{n}}(|\theta _{0i}+b_n v_i|) \nonumber \\{} & {} ~~~ \ge \mathbb {A'+B'} +n\left( \sum _{i=1}^{s} q_{\lambda _{n}}(|\theta _{0i}|)sgn(\theta _{0i})b_n v_i +\frac{1}{2}q_{\lambda _n}'(|\theta _{0i}|)b_n^2 v_i^2(1+o(1))\right) \nonumber \\{} & {} ~~~ \ge \mathbb {A'+B'+C+D}, \end{aligned}$$

(A7)

where ${\mathbb {A}}'$ and ${\mathbb {B}}'$ are defined similarly to ${\mathbb {A}}$ and ${\mathbb {B}}$ in the proof of Theorem 1 with replacement of $a_n$ to $b_n$. ${\mathbb {C}}=n\sum _{i=1}^{s} q_{\lambda _{n}}(|\theta _{0i}|)sgn(\theta _{0i})b_n v_i$ and ${\mathbb {D}}=(n/2)\sum _{i=1}^{s}q_{\lambda _n}'(|\theta _{0i}|)b_n^2 v_i^2(1+o(1))$. The expression of ${\mathbb {D}}$ comes from Assumption 3-(2). Then, by (A1) and (A5),

$$\begin{aligned} {\mathbb {A}}'+{\mathbb {B}}' = O_p\left( n^{1/2}b_n\lambda _\varepsilon ^{1/2}\right) \Vert \varvec{v}\Vert +nb_n^2\varvec{v}^T \varvec{\Sigma }_g \varvec{v} (1+o_p(1)) \end{aligned}$$

For the desired result, it is enough to show that ${\mathbb {C}}=O(nb_n^2)\Vert \varvec{v}\Vert$ and ${\mathbb {D}}=o(nb_n^2)\Vert \varvec{v}\Vert ^2$. By the definition of $b_n$ and Assumption 3-(1),

$$\begin{aligned} |{\mathbb {C}}| \le n\sum _{i=1}^{s}|q_{\lambda _{n}}(|\theta _{0i}|)b_n v_i| \le n b_nc_n \sum _{i=1}^{s} |v_i| \le nb_n^2\sqrt{s}\Vert \varvec{v}\Vert \\ |{\mathbb {D}}| \le \displaystyle \max _{1\le i\le s}|q_{\lambda _n}'(|\theta _{0i}|)| n b_n^2 \Vert \varvec{v}\Vert ^2 = o(nb_n^2) \Vert \varvec{v}\Vert ^2 \end{aligned}$$

Putting the results above all together, we obtain ${\mathbb {B}}'$ dominates ${\mathbb {C}}$ and ${\mathbb {D}}$ with large enough $\Vert \varvec{v}\Vert$. By Assumption 3-(1), $b_n=O(n^{-1/2}\lambda _\varepsilon ^{1/2})$, so with the probability tending to 1, (A7) is positive.

Proof of Theorem  3

By the mean-value theorem of a vector-valued function (Feng et al., 2013),

$$\begin{aligned}&n^{-1/2}\nabla S_n(\varvec{\theta }_0) \\&\hspace{.5cm} = n^{-1/2}\left( \nabla S_n\left( \hat{\varvec{\theta }}^{(s)}\right) +\left( \int ^{1}_{0}\nabla ^2 S_n\left( \varvec{{{\hat{\theta }}}}^{(s)}+\left( \varvec{\theta }_0-\hat{\varvec{\theta }}^{(s)}\right) t\right) dt \right) ^{T}\left( \varvec{\theta }_0-\hat{\varvec{\theta }}^{(s)}\right) \right) \\&\hspace{.5cm} = n^{-1/2}\left( \int ^{1}_{0}\nabla ^2 S_n\left( \varvec{{{\hat{\theta }}}}^{(s)}+\left( \varvec{\theta }_0-\hat{\varvec{\theta }}^{(s)}\right) t\right) dt \right) ^{T}\left( \varvec{\theta }_0-\hat{\varvec{\theta }}^{(s)}\right) , \end{aligned}$$

For a arbitrary vector $\varvec{v} \in {\mathcal {R}}^d$, we consider $n^{-1/2} \varvec{v}^T \nabla S_n(\varvec{\theta }_0)$ as follows.

$$\begin{aligned} n^{-1/2} \varvec{v}^T \nabla S_n(\varvec{\theta }_0)=-2n^{-1/2}\varvec{v}^T\varvec{\dot{G}}(\varvec{\theta }_0)^T \varvec{\xi }+2n^{-1/2}\varvec{v}^T \varvec{\dot{G}}(\varvec{\theta }_0)^{T}\left( \frac{1}{n}\varvec{1} \varvec{1}^T\right) \varvec{\xi }. \end{aligned}$$

Since $\varvec{x}(\varvec{s}_i) = \varvec{x}(\eta _n\varvec{u}_i)$ is independent, by WLLN $n^{-1}\varvec{v}^{T}\varvec{\dot{G}}(\varvec{\theta }_0)^{T}\varvec{1}=\varvec{v}^T \varvec{\Delta }_g (1+o_p(1))$. Thus, conditional on $\varvec{U}$,

$$\begin{aligned} n^{-1/2} \varvec{v}^T \nabla S_n(\varvec{\theta }_0)&= -2n^{-1/2}\varvec{v}^{T}\varvec{\dot{G}}(\varvec{\theta }_0)^{T} \varvec{\xi }+ 2n^{-1/2}\varvec{v}^T \varvec{\Delta }_g \varvec{1}^T\varvec{\xi }+o_p(1)\\&= -2n^{-1/2}\sum _{i=1}^{n} \varvec{v}^{T}\left( \frac{\partial g(\varvec{x}(\varvec{s}_i);\varvec{\theta }_0)}{\partial \varvec{\theta }}-\varvec{\Delta }_g \right) \xi (\varvec{s}_i) +o_p(1) \end{aligned}$$

By Assumptions 1, 2 and the lemma 1 with $w_n(\varvec{s}_i) = \varvec{v}^{T}\left( \frac{\partial g(\varvec{x}(\varvec{s}_i);\varvec{\theta }_0)}{\partial \varvec{\theta }}-\varvec{\Delta }_g \right)$,

$$\begin{aligned} -2n^{-1/2} \sum _{i=1}^{n} w_n(\varvec{s}_i) \xi (\varvec{s}_i) \overset{d}{\rightarrow }\ N\left( 0, 4\varvec{v}^{T}\left( \sigma (\varvec{0}) \varvec{\Sigma }_g + \int \sigma (\varvec{h}) \varvec{r}(\varvec{h}) d\varvec{h} \right) \varvec{v} \right) \end{aligned}$$

(A8)

where $\varvec{\Sigma }_g = \varvec{\Delta }_{g^2}-\varvec{\Delta }_g\varvec{\Delta }_g^T,\ \varvec{r}(\varvec{h}) = \varvec{r}_2(\varvec{h})-\varvec{\Delta }_g(\varvec{r}_1(\varvec{0})+\varvec{r}_1(\varvec{h}))^T+\text{ E }[\phi (\varvec{U}_1)]\varvec{\Delta }_g\varvec{\Delta }_g^T$, and $\sigma (\varvec{h}) = \text{ E }\left[ \xi (\varvec{0})\xi (\varvec{h})\right]$. The variance expression follows from the lemma 1 since

$$\begin{aligned} \int w_n^2(\eta _n \varvec{u}) \phi (\varvec{u})d\varvec{u}&\overset{p}{\rightarrow }\ \varvec{v}^{T}\varvec{\Sigma }_g\varvec{v} \\ \int w_n(\eta _n \varvec{u}+\varvec{h})w_n(\eta _n \varvec{u}) \phi ^2(\varvec{u})d \varvec{u}&\overset{p}{\rightarrow }\ \varvec{v}^{T}\left( \varvec{r}_2(\varvec{h})-\varvec{\Delta }_g(\varvec{r}_1(\varvec{0})+\varvec{r}_1(\varvec{h}))^T+\text{ E }[\phi (\varvec{U}_1)]\varvec{\Delta }_g\varvec{\Delta }_g^T\right) \varvec{v} \end{aligned}$$

Therefore, by the conditional version of Cramer-Wold device,

$$\begin{aligned} n^{-1/2} \nabla S_n(\varvec{\theta }_0) \overset{d}{\longrightarrow } N\left( 0, 4 \varvec{\Sigma }_\theta \right) \end{aligned}$$

(A9)

where $\varvec{\Sigma }_\theta =\sigma (0)\varvec{\Sigma }_g + \int \sigma (\varvec{h}) \varvec{r}(\varvec{h}) d\varvec{h}$. The proof is completed by Slutsky’s theorem if we attain

$$\begin{aligned} n^{-1}\left( \int ^{1}_{0}\nabla ^2 S_n\left( \varvec{{{\hat{\theta }}}}^{(s)}+(\varvec{\theta }_0-\hat{\varvec{\theta }}^{(s)})t\right) dt \right) \overset{p}{\longrightarrow }\ 2\varvec{\Sigma }_g \end{aligned}$$

(A10)

where $\varvec{\Sigma }_g = \varvec{\Delta }_{g^2}-\varvec{\Delta }_{g}\varvec{\Delta }_{g}^T$. For (A10), it is enough to show that

$$\begin{aligned}&n^{-1}\nabla ^2 S_n(\varvec{\theta }_0) \overset{p}{\rightarrow } 2\varvec{\Sigma }_g, \end{aligned}$$

(A11)

$$\begin{aligned}&n^{-1/2} \left( \int ^{1}_{0}\nabla ^2 S_n\left( \varvec{{{\hat{\theta }}}}^{(s)}+(\varvec{\theta }_0-\hat{\varvec{\theta }}^{(s)})t\right) dt -\nabla ^2 S_n(\varvec{\theta }_0) \right) =o_p(1). \end{aligned}$$

(A12)

For (A11),

$$\begin{aligned} n^{-1} \nabla ^2 S_n(\varvec{\theta }_0) = 2n^{-1}\dot{\varvec{G}}(\varvec{\theta }_0)^T\varvec{\Sigma }_n \dot{\varvec{G}}(\varvec{\theta }_0)-2n^{-1}\ddot{\varvec{G}}(\varvec{\theta }_0)^T(I \otimes \varvec{\Sigma }_n \varvec{\xi }). \end{aligned}$$

The first term converges to $2\varvec{\Sigma }_g$ by Assumptions 1-(4) and (5) and the second term is $O_p(n^{-1/2}\lambda _\varepsilon ^{1/2})$ by similar procedure to (A4). Thus, by Assumption 2-(1), we achieve (A11). Next, since $\hat{\varvec{\theta }}^{(s)}$ is root-n-consistent under Assumption 2-(2),

$$\begin{aligned}&n^{-1}\left\| \int ^{1}_{0}\nabla ^2 S_n\left( \varvec{{{\hat{\theta }}}}^{(s)} + (\varvec{\theta }_0-\hat{\varvec{\theta }}^{(s)})t\right) dt-\nabla ^2 S_n(\varvec{\theta _0})\right\| \\&~~~~~~~~~~~~~~~~~~~~~~~~~~~ \le n^{-1}\max _{\tilde{\varvec{\theta }}\in B(\varvec{\theta }_0,n^{-1/2} C)}\left\| \nabla ^2 S_n(\tilde{\varvec{\theta }})- \nabla ^2 S_n(\varvec{\theta }_0)\right\| . \end{aligned}$$

The above term can be decomposed into 3 terms as follows.

$$\begin{aligned}&n^{-1}\max _{\tilde{\varvec{\theta }}\in B(\varvec{\theta }_0,n^{-1/2} C)}\left\| \nabla ^2 S_n(\tilde{\varvec{\theta }})- \nabla ^2 S_n(\varvec{\theta }_0)\right\| \\&\hspace{2cm} \le 2n^{-1}\max _{\tilde{\varvec{\theta }}\in B(\varvec{\theta }_0,n^{-1/2} C)}\left\| \dot{\varvec{G}}(\tilde{\varvec{\theta }})^T\varvec{\Sigma }_n \dot{\varvec{G}}(\tilde{\varvec{\theta }})-\dot{\varvec{G}}(\varvec{\theta }_0)^T\varvec{\Sigma }_n \dot{\varvec{G}}(\varvec{\theta }_0) \right\| \\&\hspace{2cm} + 2n^{-1}\max _{\tilde{\varvec{\theta }}\in B(\varvec{\theta }_0,n^{-1/2} C)}\left\| \ddot{\varvec{G}}(\tilde{\varvec{\theta }})^T(I\otimes \varvec{\Sigma }_n \varvec{d}(\tilde{\varvec{\theta }},\varvec{\theta }_0)) \right\| \\&\hspace{2cm} + 2n^{-1}\max _{\tilde{\varvec{\theta }}\in B(\varvec{\theta }_0,n^{-1/2} C)}\left\| \left( \ddot{\varvec{G}} (\tilde{\varvec{\theta }})-\ddot{\varvec{G}}(\varvec{\theta }_0)\right) ^T (I \otimes \varvec{\Sigma }_n \varvec{\xi }) \right\| . \end{aligned}$$

The first term is o(1) conditional on $\varvec{U}$ by similar procedure to (A2). The second part converges to 0 by

$$\begin{aligned} 2n^{-1}\left\| \ddot{\varvec{G}}(\tilde{\varvec{\theta }})^T\left( I \otimes \varvec{\Sigma }_n\varvec{d}(\tilde{\varvec{\theta }}, \varvec{\theta }_0)\right) \right\|&= 2n^{-1}\left\| \left[ \varvec{g}_{kl}(\tilde{\varvec{\theta }})^T\varvec{\Sigma }_n \varvec{d}(\tilde{\varvec{\theta }}, \varvec{\theta }_0)\right] _{k,l=\{1,\ldots ,p\}}\right\| \\&\le 2n^{-1}O\left( \max _{k,l}\left| \varvec{g}_{kl}(\tilde{\varvec{\theta }})^T\varvec{\Sigma }_n \varvec{d}(\tilde{\varvec{\theta }}, \varvec{\theta }_0)\right| \right) \\&\le 2n^{-1}O\left( \max _{k,l}\left\| \varvec{g}_{kl}(\tilde{\varvec{\theta }}) \right\| \left\| \varvec{d}(\tilde{\varvec{\theta }}, \varvec{\theta }_0)\right\| \right) \\&\le 2n^{-1} O\left( n^{1/2}\Vert \tilde{\varvec{\theta }}-\varvec{\theta }_0\Vert \right) \\&= O\left( n^{-1}\right) . \end{aligned}$$

The first and third inequalities hold since p is finite and g is bounded, respectively. For the last term, similarly to (A4) we evaluate the variance of (k, l)-th component.

$$\begin{aligned} {\text {Var}}\left( \left( \varvec{g}_{kl}(\tilde{\varvec{\theta }}) -\varvec{g}_{kl}(\varvec{\theta }_0)\right) ^T\varvec{\Sigma }_n \varvec{\xi }\right)&= \left( \varvec{g}_{kl}(\tilde{\varvec{\theta }}) -\varvec{g}_{kl}(\varvec{\theta }_0)\right) ^T\varvec{\Sigma }_n \varvec{\Sigma }_\varepsilon \varvec{\Sigma }_n \left( \varvec{g}_{kl}(\tilde{\varvec{\theta }}) -\varvec{g}_{kl}(\varvec{\theta }_0)\right) \\&\le \lambda _\varepsilon \left\| \varvec{g}_{kl}(\tilde{\varvec{\theta }}) -\varvec{g}_{kl}(\varvec{\theta }_0)\right\| ^2 \\&\le \lambda _\varepsilon \Vert \tilde{\varvec{\theta }}-\varvec{\theta }_0\Vert ^2 \\&= O\left( n^{-1}\lambda _\varepsilon \right) = O\left( n^{-1}\right) \end{aligned}$$

This indicates that the last term is $O_p(n^{-1/2})$. To sum up,

$$\begin{aligned} n^{-1}\max _{\tilde{\varvec{\theta }}\in B(\varvec{\theta }_0,n^{-1/2} C)}\left\| \nabla ^2 S_n(\tilde{\varvec{\theta }})- \nabla ^2 S_n(\varvec{\theta }_0)\right\| \le o(1)+O\left( n^{-1}\right) + O_p\left( n^{-1/2}\right) =o_p(1). \end{aligned}$$

Lastly, we attain (A12) and, therefore, (A10). Bringing (A9) and (A10) together with Slutsky’s theorem, we obtain

$$\begin{aligned} n^{1/2}\left( \hat{\varvec{\theta }}^{(s)}-\varvec{\theta }_0\right) \overset{d}{\longrightarrow }\ N\left( 0, \varvec{\Sigma }_g^{-1}\varvec{\Sigma }_\theta \varvec{\Sigma }_g^{-1} \right) . \end{aligned}$$

Proof of Theorem  4

Proof of (i) We show for $\epsilon >0$ & $\ i \in \{s+1, \ldots , p\},$ $P({\hat{\theta }}_i \ne 0) <\epsilon$. First, we break $({\hat{\theta }}_i\ne 0)$ into two sets:

$$\begin{aligned} \{{\hat{\theta }}_i\ne 0\}&= \left\{ {\hat{\theta }}_i\ne 0, |{\hat{\theta }}_i|\ge C n^{-1/2}\right\} +\left\{ {\hat{\theta }}_i\ne 0, |{\hat{\theta }}_i|< C n^{-1/2}\right\} \\&:= E_n+F_n. \end{aligned}$$

Then, it is enough to show for any $\epsilon > 0$, $P(E_n) <\epsilon /2$ and $P(F_n) < \epsilon /2$. For any $\epsilon >0$, we can show $P(E_n)<\epsilon /2$ for large enough n because $\Vert \varvec{{{\hat{\theta }}}}-\varvec{\theta }_0\Vert =O_p(n^{-1/2})$ under Assumption 2-(2). To verify $P(F_n)<\epsilon /2$ for large enough n, we first show $n^{1/2} q_{\lambda _{n}}(|{{\hat{\theta }}}_i|)=O_p(1)$ on the set $F_n$. By the mean-value theorem again,

$$\begin{aligned} n^{-1/2}\nabla S_n(\varvec{\theta }_0) = n^{-1/2}\left( \nabla S_n({\varvec{\theta }})+\left( \int ^{1}_{0}\nabla ^2 S_n\left( \varvec{\theta }+(\varvec{\theta }_0-{\varvec{\theta }})t\right) dt \right) ^{T}(\varvec{\theta }_0-{\varvec{\theta }})\right) . \end{aligned}$$

Recall that $n^{-1/2}\nabla S_n(\varvec{\theta }_0) = O_p(1)$ from (A9). In addition, with $\Vert \varvec{\theta }_0-\varvec{\theta }\Vert =O(n^{-1/2})$, the similar results of (A10) inform that

$$\begin{aligned} \left\| n^{-1/2}\left( \int ^{1}_{0}\nabla ^2 S_n\left( \varvec{\theta }+(\varvec{\theta }_0-{\varvec{\theta }})t\right) dt \right) ^{T}(\varvec{\theta }_0-{\varvec{\theta }})\right\| = O_p(1). \end{aligned}$$

(A13)

Then, we have

$$\begin{aligned} \left\| n^{-1/2}\nabla S_n(\varvec{\theta })\right\| =O_p(1)\;\hbox {for } \varvec{\theta }\text { satisfying }\Vert \varvec{\theta }-\varvec{\theta }_0\Vert \le Cn^{-1/2}. \end{aligned}$$

Since $\hat{\varvec{\theta }}$ is the local minimizer of $Q_n(\varvec{\theta })$ with $\Vert \hat{\varvec{\theta }}-\varvec{\theta }_0\Vert =O_p(n^{-1/2})$, we attain

$$\begin{aligned} n^{1/2} q_{\lambda _{n}}(|{\hat{\theta }}_i|)=O_p(1) \end{aligned}$$

from

$$\begin{aligned} n^{-1/2}\left. \frac{\partial Q_n(\varvec{\theta })}{\partial \theta _i}\right| _{\varvec{\theta }=\hat{\varvec{\theta }}}=n^{-1/2}\left. \frac{\partial S_n(\varvec{\theta })}{\partial \theta _i}\right| _{\varvec{\theta }=\hat{\varvec{\theta }}}+n^{1/2} q_{\lambda _{n}}(|\hat{\theta _i}|)sgn({{\hat{\theta }}}_i). \end{aligned}$$

Therefore, there exists $M^\prime$ such that $P\{n^{1/2} q_{\lambda _n}(|{\hat{\theta }}_i|)>M^{\prime }\}<\epsilon /2$ for large enough n, which implies $P\{{\hat{\theta }}_i\ne 0, |{\hat{\theta }}_i|<Cn^{-1/2}, n^{1/2} q_{\lambda _n}(|{\hat{\theta }}_i|)>M^{\prime }\}<\epsilon /2$. Finally, by Assumptions 3-(3) and (4), for large enough n,

$$\begin{aligned} \{{\hat{\theta }}_i\ne 0, |{\hat{\theta }}_i|<C n^{-1/2}, n^{1/2} q_{\lambda _n}(|{\hat{\theta }}_i|)>M^{\prime }\}=\{{\hat{\theta }}_i\ne 0, |{\hat{\theta }}_i|<C n^{-1/2}\}, \end{aligned}$$

which leads to $P(F_n)<\epsilon /2$.

Proof of (ii) By Taylor expansion,

$$\begin{aligned} n^{-1/2}\nabla Q_n(\hat{\varvec{\theta }})&= n^{-1/2}\nabla S_n(\hat{\varvec{\theta }})+n^{1/2}{\varvec{q}}_{\lambda _n}(\hat{\varvec{\theta }})\cdot sgn(\hat{\varvec{\theta }})\\&=n^{-1/2}\left( \nabla S_n(\varvec{\theta }_0)+\left( \int ^{1}_{0}\nabla ^2 S_n\left( \varvec{\theta }_0+(\varvec{{{\hat{\theta }}}}-\varvec{\theta }_0)t\right) \right) ^{T}(\varvec{{{\hat{\theta }}}}-\varvec{\theta }_0)\right) \\&+n^{1/2}\varvec{q}_{\lambda _n}(\hat{\varvec{\theta }})\cdot sgn(\hat{\varvec{\theta }}) \end{aligned}$$

where $\varvec{q}_{\lambda _n}(\hat{\varvec{\theta }}) \cdot sgn(\hat{\varvec{\theta }})=\left( q_{\lambda _n}(|{\hat{\theta }}_1|)sgn({{\hat{\theta }}}_1),..,q_{\lambda _n}(|{\hat{\theta }}_p|)sgn({{\hat{\theta }}}_p)\right) ^{T}.$ Since $\hat{\varvec{\theta }}$ is the local minimizer of $Q_n({\varvec{\theta }})$, $\nabla Q_n(\hat{\varvec{\theta }})=0$, which implies

$$\begin{aligned} -n^{-1/2}\nabla S_n({\varvec{\theta }_0})=n^{-1}\left( \int ^{1}_{0}\nabla ^2 S_n\left( \varvec{\theta }_0+(\varvec{{{\hat{\theta }}}}-\varvec{\theta }_0)t\right) dt \right) ^{T}\left( n^{1/2}(\hat{\varvec{\theta }}-\varvec{\theta }_0)\right) +n^{1/2}\varvec{q}_{\lambda _n}(\hat{\varvec{\theta }})\cdot sgn(\hat{\varvec{\theta }}). \end{aligned}$$

Since $n^{-1/2}\nabla S_n(\varvec{\theta }_0) \overset{d}{\rightarrow }N(4\varvec{\Sigma }_\theta )$ by (A9) and $n^{-1}\left( \int ^{1}_{0}\nabla ^2 S_n(\varvec{\theta }_0+(\varvec{{{\hat{\theta }}}}-\varvec{\theta }_0)t)dt \right) \overset{p}{\rightarrow }\ 2\varvec{\Sigma }_g$ by the similar results of (A10),

$$\begin{aligned} n^{1/2}\left( \hat{{\varvec{\theta }}}-\varvec{\theta }_{0}+(2\varvec{\Sigma }_g)^{-1}\varvec{q}_{\lambda _n}(\hat{\varvec{\theta }})sgn(\hat{\varvec{\theta }})\right) ~ {\mathop {\rightarrow }\limits ^{d}}N\left( 0,\varvec{\Sigma }_g^{-1}\varvec{\Sigma }_\theta \varvec{\Sigma }_g^{-1}\right) . \end{aligned}$$

Finally,

$$\begin{aligned} n^{1/2}\left( {\hat{{\varvec{\theta }}}}_{11}-\varvec{\theta }_{01}+(2\varvec{\Sigma }_g)_{11}^{-1}\varvec{\beta }_{n, s}\right) ~ {\mathop {\rightarrow }\limits ^{d}}N\left( 0,\left( \varvec{\Sigma }_g^{-1}\varvec{\Sigma }_{\theta }\varvec{\Sigma }_g^{-1}\right) _{11}\right) , \end{aligned}$$

where $(\varvec{\Sigma }_g)_{11}^{-1}$ is the $s\times s$ upper-left matrix of $\varvec{\Sigma }_g^{-1}$.