Mathematics > Statistics Theory
[Submitted on 4 Nov 2013 (v1), revised 27 Feb 2014 (this version, v2), latest version 4 Jun 2017 (v3)]
Title:Optimal Shrinkage of Eigenvalues in the Spiked Covariance Model
View PDFAbstract:Since the seminal work of Stein (1956) it has been understood that the empirical covariance matrix can be improved by shrinkage of the empirical eigenvalues. In this paper, we consider a proportional-growth asymptotic framework with $n$ observations and $p_n$ variables having limit $p_n/n \to \gamma \in (0,1]$. We assume the population covariance matrix $\Sigma$ follows the popular spiked covariance model, in which several eigenvalues are significantly larger than all the others, which all equal $1$. Factoring the empirical covariance matrix $S$ as $S = V \Lambda V'$ with $V$ orthogonal and $\Lambda$ diagonal, we consider shrinkers of the form $\hat{\Sigma} = \eta(S) = V \eta(\Lambda) V'$ where $\eta(\Lambda)_{ii} = \eta(\Lambda_{ii})$ is a scalar nonlinearity that operates individually on the diagonal entries of $\Lambda$. Many loss functions for covariance estimation have been considered in previous work. We organize and amplify the list, and study 26 loss functions, including Stein, Entropy, Divergence, Fréchet, Bhattacharya/Matusita, Frobenius Norm, Operator Norm, Nuclear Norm and Condition Number losses. For each of these loss functions, and each suitable fixed nonlinearity $\eta$, there is a strictly positive asymptotic loss which we evaluate precisely. For each of these 26 loss functions, there is a unique admissible shrinker dominating all other shrinkers; it takes the form $\hat{\Sigma}^* = V \eta^*(\Lambda) V'$ for a certain loss-dependent scalar nonlinearity $\eta^*$ which we characterize. For 17 of these loss functions, we derive a simple analytical expression for the optimal nonlinearity $\eta^*$; in all cases we tabulate the optimal nonlinearity and provide software to evaluate it numerically on a computer. We also tabulate the asymptotic slope, and, where relevant, the asymptotic shift of the optimal nonlinearity.
Submission history
From: Matan Gavish [view email][v1] Mon, 4 Nov 2013 20:55:31 UTC (64 KB)
[v2] Thu, 27 Feb 2014 06:37:49 UTC (71 KB)
[v3] Sun, 4 Jun 2017 20:46:43 UTC (939 KB)
Current browse context:
math.ST
References & Citations
Bibliographic and Citation Tools
Bibliographic Explorer (What is the Explorer?)
Litmaps (What is Litmaps?)
scite Smart Citations (What are Smart Citations?)
Code, Data and Media Associated with this Article
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub (What is DagsHub?)
Gotit.pub (What is GotitPub?)
Papers with Code (What is Papers with Code?)
ScienceCast (What is ScienceCast?)
Demos
Recommenders and Search Tools
Influence Flower (What are Influence Flowers?)
Connected Papers (What is Connected Papers?)
CORE Recommender (What is CORE?)
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.