article

Free access

Variable selection in high-dimensional varying-coefficient models with global optimality

Authors:

Annie QuAuthors Info & Claims

The Journal of Machine Learning Research, Volume 13

Pages 1973 - 1998

Published: 01 June 2012 Publication History

PDF eReader Publisher Site

Abstract

The varying-coefficient model is flexible and powerful for modeling the dynamic changes of regression coefficients. It is important to identify significant covariates associated with response variables, especially for high-dimensional settings where the number of covariates can be larger than the sample size. We consider model selection in the high-dimensional setting and adopt difference convex programming to approximate the L0 penalty, and we investigate the global optimality properties of the varying-coefficient estimator. The challenge of the variable selection problem here is that the dimension of the nonparametric form for the varying-coefficient modeling could be infinite, in addition to dealing with the high-dimensional linear covariates. We show that the proposed varying-coefficient estimator is consistent, enjoys the oracle property and achieves an optimal convergence rate for the non-zero nonparametric components for high-dimensional data. Our simulations and numerical examples indicate that the difference convex algorithm is efficient using the coordinate decent algorithm, and is able to select the true model at a higher frequency than the least absolute shrinkage and selection operator (LASSO), the adaptive LASSO and the smoothly clipped absolute deviation (SCAD) approaches.

References

[1]

L. An and P. Tao. Solving a class of linearly constrained indefinite quadratic problems by D.C. algorithms. Journal of Global Optimization, 11:253-285, 1997.

[2]

L. Breiman and A. Cutler. A deterministic algorithm for global optimization. Mathematical Programming, 58:179-199, 1993.

[3]

C. de Boor. A Practical Guide to Splines. Springer, New York, 2001.

[4]

J. Fan and T. Huang. Profile likelihood inferences on semiparametric varying-coefficient partially linear models. Bernoulli, 11:1031-1057, 2005.

[5]

J. Fan and R. Li. Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association, 96:1348-1360, 2001.

[6]

J. Fan and H. Peng. Nonconcave penalized likelihood with a diverging number of parameters. Annals of Statistics, 32:928-961, 2004.

[7]

J. Fan and J. Zhang. Two-step estimation of functional linear models with applications to longitudinal data. Journal of the Royal Statistical Society, Series B, 62:303-322, 2000.

[8]

T. Hastie and R. Tibshirani. Varying-coefficient models. Journal of the Royal Statistical Society, Series B, 55:757-796, 1993.

[9]

D. R. Hunter and R. Li. Variable selection using MM algorithms. Annals of Statistics, 33:1617- 1642, 2005.

[10]

D. R. Hoover, J. A. Rice, C. O. Wu, and L. Yang. Nonparametric smoothing estimates of time-varying coefficient models with longitudinal data. Biometrika, 85:809-822, 1998.

[11]

J. Huang, J. L. Horowitz, and F. Wei. Variable selection in nonparametric additive models. Annals of Statistics, 38:2282-2313, 2010.

[12]

J. Z. Huang, C. O. Wu, and L. Zhou. Varying-coefficient models and basis function approximations for the analysis of repeated measurements. Biometrika, 89:111-128, 2002.

[13]

J. Z. Huang, C. O. Wu, and L. Zhou. Polynomial spline estimation and inference for varying coefficient models with longitudinal data. Statistica Sinica, 14:763-788, 2004.

[14]

Y. Kim, H. Choi, and H. Oh. Smoothly clipped absolute deviation on high dimensions. Journal of the American Statistical Association, 103:1665-1673, 2008.

[15]

Y. Liu, X. Shen, and W. Wong. Computational development of psi-learning. Proc SIAM 2005 Int. Data Mining Conf., 1-12, 2005.

[16]

A. Qu, and R. Li. Quadratic inference functions for varying-coefficient models with longitudinal data. Biometrics, 62:379-391, 2006.

[17]

J. O. Ramsay, and B. W. Silverman. Functional Data Analysis. Springer-Verlag: New York, 1997.

[18]

X. Shen, W. Pan, Y. Zhu. Likelihood-based selection and sharp parameter estimation. Journal of the American Statistical Association, 107:223-232, 2012.

[19]

X. Shen, G. C. Tseng, X. Zhang, and W. H. Wong. On ¿-learning. Journal of the American Statistical Association, 98:724-734, 2003.

[20]

R. Tibshirani. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society, Series B, 58:267-288, 1996.

[21]

S. Van de Geer. High-dimensional generalized linear models and the Lasso. Annals of Statistics, 36:614-645, 2008.

[22]

H. Wang, and Y. Xia. Shrinkage estimation of the varying coefficient model. Journal of the American Statistical Association, 104:747-757, 2009.

[23]

L. Wang, H. Li, and J. Z. Huang. Variable selection in nonparametric varying-coefficient models for analysis of repeated measurements. Journal of the American Statistical Association, 103:1556- 1569, 2008.

[24]

F. Wei, J. Huang, and H. Li. Variable selection and estimation in high-dimensional varying coefficient models. Statistica Sinica, 21:1515-1540, 2011.

[25]

C. O. Wu, and C. Chiang. Kernel smoothing on varying coefficient models with longitudinal dependent variable. Statistica Sinica, 10:433-456, 2000.

[26]

L. Xue, A. Qu, and J. Zhou. Consistent model selection for marginal generalized additive model for correlated data. Journal of the American Statistical Association, 105:1518-1530, 2010.

[27]

M. Yuan, and Y. Lin. Model selection and estimation in regression with grouped variables. Journal of the Royal Statistical Society, Series B, 68:49-67, 2006.

[28]

C. H. Zhang. Nearly unbiased variable selection under minimax concave penalty. Annals of Statistics, 38:894-942, 2010.

[29]

P. Zhao, and B. Yu. On model selection consistency of Lasso. Journal of Machine Learning Research, 7:2541-2563, 2006.

[30]

S. Zhou, X. Shen, and D. A.Wolfe. Local asymptotics for regression splines and confidence regions. Annals of Statistics, 26:1760-1782, 1998.

[31]

H. Zou, and R. Li. One-step sparse estimates in nonconcave penalized likelihood models. Annals of Statistics, 36:1509-1533, 2008.

Cited By

(2019)Smooth-threshold estimating equations for varying coefficient partially nonlinear models based on orthogonality-projection methodJournal of Computational and Applied Mathematics10.1016/j.cam.2016.01.038302:C(24-37)Online publication date: 3-Jan-2019
https://dl.acm.org/doi/10.1016/j.cam.2016.01.038
Shim JHwang C(2018)Varying coefficient modeling via least squares support vector regressionNeurocomputing10.1016/j.neucom.2015.02.036161:C(254-259)Online publication date: 31-Dec-2018
https://dl.acm.org/doi/10.1016/j.neucom.2015.02.036
Fan ZLian H(2018)Quantile regression for additive coefficient models in high dimensionsJournal of Multivariate Analysis10.1016/j.jmva.2017.11.001164:C(54-64)Online publication date: 30-Dec-2018
https://dl.acm.org/doi/10.1016/j.jmva.2017.11.001
Show More Cited By

Index Terms

Variable selection in high-dimensional varying-coefficient models with global optimality

Recommendations

Variable selection in high-dimensional varying-coefficient models with global optimality

The varying-coefficient model is flexible and powerful for modeling the dynamic changes of regression coefficients. It is important to identify significant covariates associated with response variables, especially for high-dimensional settings where the ...
Non-convex penalized estimation in high-dimensional models with single-index structure

As promising alternatives to the LASSO, non-convex penalized methods, such as the SCAD and the minimax concave penalty method, produce asymptotically unbiased shrinkage estimates. By adopting non-convex penalties, in this paper we investigate uniformly ...
Variable selection for varying-coefficient models with the sparse regularization

Varying-coefficient models are useful tools for analyzing longitudinal data. They can effectively describe a relationship between predictors and responses which are repeatedly measured. We consider the problem of selecting variables in the varying-...

Comments

Information & Contributors

Information

Published In

cover image The Journal of Machine Learning Research

The Journal of Machine Learning Research Volume 13, Issue

3/1/2012

2065 pages

ISSN:1532-4435

EISSN:1533-7928

Issue’s Table of Contents

Publisher

JMLR.org

Publication History

Published: 01 June 2012

Published in JMLR Volume 13

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
306
Total Downloads

Downloads (Last 12 months)58
Downloads (Last 6 weeks)13

Reflects downloads up to 01 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

(2019)Smooth-threshold estimating equations for varying coefficient partially nonlinear models based on orthogonality-projection methodJournal of Computational and Applied Mathematics10.1016/j.cam.2016.01.038302:C(24-37)Online publication date: 3-Jan-2019
https://dl.acm.org/doi/10.1016/j.cam.2016.01.038
Shim JHwang C(2018)Varying coefficient modeling via least squares support vector regressionNeurocomputing10.1016/j.neucom.2015.02.036161:C(254-259)Online publication date: 31-Dec-2018
https://dl.acm.org/doi/10.1016/j.neucom.2015.02.036
Fan ZLian H(2018)Quantile regression for additive coefficient models in high dimensionsJournal of Multivariate Analysis10.1016/j.jmva.2017.11.001164:C(54-64)Online publication date: 30-Dec-2018
https://dl.acm.org/doi/10.1016/j.jmva.2017.11.001
Matsui HMisumi T(2018)Variable selection for varying-coefficient models with the sparse regularizationComputational Statistics10.1007/s00180-014-0520-330:1(43-55)Online publication date: 21-Dec-2018
https://dl.acm.org/doi/10.1007/s00180-014-0520-3
Gao QLee T(2017)High-dimensional variable selection in regression and classification with missing dataSignal Processing10.1016/j.sigpro.2016.07.014131:C(1-7)Online publication date: 1-Feb-2017
https://dl.acm.org/doi/10.1016/j.sigpro.2016.07.014
Guo CYang HLv J(2016)Generalized varying index coefficient modelsJournal of Computational and Applied Mathematics10.1016/j.cam.2015.11.025300:C(1-17)Online publication date: 1-Jul-2016
https://dl.acm.org/doi/10.1016/j.cam.2015.11.025

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Figures

Tables

Media

View Issue’s Table of Contents