Interpretable sparse SIR for functional data

Abstract

We propose a semiparametric framework based on sliced inverse regression (SIR) to address the issue of variable selection in functional regression. SIR is an effective method for dimension reduction which computes a linear projection of the predictors in a low-dimensional space, without loss of information on the regression. In order to deal with the high dimensionality of the predictors, we consider penalized versions of SIR: ridge and sparse. We extend the approaches of variable selection developed for multidimensional SIR to select intervals that form a partition of the definition domain of the functional predictors. Selecting entire intervals rather than separated evaluation points improves the interpretability of the estimated coefficients in the functional framework. A fully automated iterative procedure is proposed to find the critical (interpretable) intervals. The approach is proved efficient on simulated and real data. The method is implemented in the R package SISIR available on CRAN at https://cran.r-project.org/package=SISIR.

Appendices

Equivalent expressions for $R^2(d)$

In this section, we show that $R^2(d) = \frac{1}{2}\mathbb {E} \left\| \varPi _d - \widehat{\varPi }_d \right\| ^2_F$. We have

$$\begin{aligned} \frac{1}{2} \left\| \varPi _d - \widehat{\varPi }_d \right\| ^2_F= & {} \frac{1}{2} \text{ Tr }\left[ \left( \varPi _d - \widehat{\varPi }_d \right) \left( \varPi _d - \widehat{\varPi }_d \right) ^\top \right] \\= & {} \frac{1}{2} \text{ Tr }\left[ \left( \varPi _d \varPi _d \right) \right] - \text{ Tr }\left[ \left( \varPi _d \widehat{\varPi }_d \right) \right] \\&+\frac{1}{2} \text{ Tr }\left[ \left( \widehat{\varPi }_d \widehat{\varPi }_d \right) \right] . \end{aligned}$$

The norm of a M-orthogonal projector onto a space of dimension d is equal to d, we thus have that

$$\begin{aligned} \frac{1}{2} \left\| \varPi _d - \widehat{\varPi }_d \right\| ^2_F = d - \text{ Tr }\left[ \left( \varPi _d \widehat{\varPi }_d \right) \right] , \end{aligned}$$

which concludes the proof.

Joint choice of the parameters $\mu _2$ and d

Notations:

• $\mathcal {L}_l$ are observations in fold number l and $\overline{\mathcal {L}_l}$ are the remaining observations;

• $\hat{A}(\mathcal {L}, \mu _2, d)$ and $\hat{C}(\mathcal {L}, \mu _2, d)$ are minimizers of the ridge regression problem restricted to observations $i \in \mathcal {L}$. Note that for $d_1 < d_2$, $\hat{A}(\tau , \mu _2, d_1)$ are the first $d_1$ columns of $\hat{A}(\mathcal {L}, \mu _2, d_2)$ (and similarly for $\hat{C}(\mathcal {L}, \mu _2, d)$);

• $\hat{p}_h^\mathcal {L}$, $\overline{X}_h^\mathcal {L}$, $\overline{X}^\mathcal {L}$ and $\widehat{\varSigma }^\mathcal {L}$ are, respectively, slices frequencies, conditional mean of X given the slices, mean of X given the slices and covariance of X for observations $i \in \mathcal {L}$;

• $\widehat{\varPi }_{d,\mu _2}^{\mathcal {L}}$ is the $(\widehat{\varSigma }^{\mathcal {L}}+\mu _2\mathbb {I}_p)$-orthogonal projector onto the space spanned by the first d columns of $\hat{A}(\mathcal {L},\mu _2,d_0)$ and $\widehat{\varPi }_{d,\mu _2}$ is $\widehat{\varPi }_{d,\mu _2}^{\mathcal {L}}$ for $\mathcal {L} = \{1,\,\ldots ,\,n\}$.