Abstract
A statistician may face with a dataset that suffers from multicollinearity and outliers, simultaneously. The Huberized ridge (HR) estimator is a technique that can be used here. On the other hand, an expert may claim that some/all the variables should be removed from the analysis, due to inappropriateness, that imposes a prior information that all coefficients equal to zero (in the form of a restriction) to the analysis. In such situations, one may consider the HR estimation under the subspace restriction. In this paper, we introduce some improved estimators for verifying this claim. They are employed to improve the performance of the HR estimator in the multiple regression model. Advantages of the proposed estimators over the usual HR estimator are demonstrated through a Monte Carlo simulation as well as two real data examples.
Similar content being viewed by others
Notes
The data are available on the website (http://people.umass.edu/be640/yr2004/resources/data2002/VO2.txt).
This data set is available at http://www.stat.wisc.edu/~gvludwig/fall_2012/bodyfat.csv.
References
Akdeniz F (2002) More on the pre-test estimator in ridge regression. Commun Stat Theory Methods 31:987–994
Akdeniz F, Kaciranlar S (2001) More on the new biased estimator in linear regression. Sankhy B 63:321–325
Alheety MI, Kibria BMG (2014) A generalized stochastic restricted ridge regression estimator. Commun Stat Theory Methods 43:4415–4427
Alpu O, Samkar H (2010) Liu estimator based on an m estimator. Turk Klin J Biostat 2:49–53
Arslan O, Billor N (2000) Robust liu estimator for regression based on an M-estimator. J Appl Stat 27:39–47
Askin RG, Montgomery DC (1980) Augmented robust estimation. Technometrics 22:333–341
Askin RG, Montgomery DC (1984) An analysis of condtrained robust regression estimators. Navoal Logist Q 32:283–296
El-Salam M (2013) The efficiency of some robust ridge regression for handling multicollinearity and non-normals errors problems. Appl Math Sci 7:3831–3846
Gibbons DGA (1981) Simulation study of some ridge estimators. J Am Stat Assoc 76:131–139
Gruber MHJ (1986) Improving efficiency by shrinkage the James-Stein and ridge regression estimators. Springer, New York
Hoerl AE, Kennard RW (1970) Ridge regression: biased estimation for non-orthogonal problems. Technometrics 12:55–67
Huber PJ (1981) Robust statistics. Wiley, Hoboken
Jadhav NH, Kashid DN (2011) A jackknifed ridge M-estimator for regression model with multicollinearity and outliers. J Stat Theory Pract 5:659–673
Jadhav NH, Kashid DN (2014) Robust winsorized shrinkage estimators for linear regression models. J Modern Appl Stat Methods 13:131–150
James W, Stein C (1961) Estimation with quadratic loss. In: Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Contributions to the Theory of Statistics, pp. 361–379, University of California Press, Berkeley. http://projecteuclid.org/euclid.bsmsp/1200512173
Kan B, Alpu O, Yazici B (2013) Robust ridge and robust liu estimator for regression based on the lts estimator. J Appl Stat 40:644–655
Kibria BMG (1996) On preliminary test ridge regression estimators for linear restrictions in a regression model with non-normal disturbances. Commun Stat Theory Methods 25:2349–2369
Kibria BMG (2003) Performance of some new ridge regression estimators. Commun Stat Simul Comput 32:419–435
Kibria BMG (2004a) One some ridge regression estimators under possible stochastic constraints. Pak J Stat All Ser 20:1–24
Kibria BMG (2004b) Performance of the shrinkage preliminary test ridge regression estimators based on the conflicting of w, lr and lm tests. J Stat Comput Simul 74:793–810
Kibria BMG (2012) Some liu and ridge-type estimators and their properties under the ill-conditioned gaussian linear regression model. J Stat Comput Simul 82:1–17
Kibria BMG, Saleh AKME (2003) Effect of w, lr, and lm tests on the performance of preliminary test ridge regression estimators. J Jpn Stat Soc 33:119–136
Kibria BMG, Saleh AKME (2004a) Performance of positive rule estimator in the ill-conditioned gaussian regression model. Calcutta Stat Assoc Bull 55:209–239
Kibria BMG, Saleh AKME (2004b) Preliminary test ridge regression estimators with students t errors and conflicting test-statistics. Metrika 59:105–124
Lawrence KD, Marsh LC (1984) Robust ridge estimation methods for predicting US coal mining fatalities. Commun Stat Theory Methods 13:139–149
McDonald GC, Galarneau DI (1975) A monte carlo evaluation of some ridge-type estimators. J Am Stat Assoc 70:407–416
Montgomery DC, Askin RG (1981) Problems of nonnormality and multicollinearity for forecasting methods based on least squares. AIIE Trans 13:102–115
Montgomery DC, Peck EA (1992) Introduction to linear regression analysis, 2nd edn. Wiley, Hoboken
Muniz G, Kibria BMG (2010) On some ridge regression estimators: an empirical comparisons. Commun Stat Simul Comput 38:621–630
Muniz G, Kibria BMG, Mansson KM, Shukur G (2012) On developing ridge regression parameters: a graphical investigation. SORT 36:115–138
Norouzirad M, Arashi M (2017) Supplemental materials: The necessity of using shrinkage ridge M-estimator when multicollinearity and outliers are present in a dataset
Ozturk F, Akdeniz F (2000) Ill-conditioning and multicollinearity. Linear Algebra Appl 321:295–305
Pati KD, Adnan R, Rasheed BA (2014) Ridge least trimmed squares estimators in presence of multicollinearity and outliers. Nat Sci 12:1–8
Penrose K, Nelson A, Fisher A (1985) Generalized body composition prediction equation for men using simple measurement techniques. Med Sci Sports Exerc 17:189
Pfaffenberger RC, Dielman TE (1984) A modified ridge regression estimator using the least absolute value criterion in the multiple linear regression model. In: Proceedings of the American Institute for Decision Sciences, pp. 791–793
Pfaffenberger RC, Dielman TE (1985) A comparison of robust ridge estimators. In: Proceedings of the American Statistical Association Business and Economic Statistics Section, pp. 631–635
Saleh AKME (2006) Theory of preliminary test and Stein-type estimation with applications. Wiley, Hoboken
Saleh AKME, Kibria BMG (1993) Performance of some new preliminary test ridge regression estimators and their properties. Commun Stat Theory Methods 22:2747–2764
Saleh AKME, Shiraishi T (1989) On some r and m estimators of regression parameters under uncertain restriction. J Jpn Stat Soc 19:129–137
Samkar H, Alpu O (2010) Ridge regression based on some robust estimators. J Modern Appl Stat Methods 9:495–501
Sarker N (1992) A new estimator combining the ridge regression and the restricted least squares method of estimation. Commun Stat Theory Methods 21:1987–2000
Sengupta D, Jammalamadaka SR (2003) Linear models: an integrated approach. World Scientific Publishing Company, Singapore
Silvapull MJ (1991) Robust ridge regression based on an M estimator. Aust N Z J Stat 33:319–333
Susanti Y, Pratiwi H, Sulistijowati S, Liana T (2014) M estimation, S estimation, and MM estimation in robust regression. Int J Pure Appl Math 91:349–360
Tabakan G, Akdeniz F (2010) Difference-based ridge estimator of parameters in partial linear model. Stat Papers 51:357–368
Tabatabaey SMM, Kibria BGM, Saleh AKMdE (2004a) Estimation strategies for parameters of the linear regression models with spherically symmetric distributions. J Stat Res 38:13–31
Tabatabaey SMM, Saleh AKME, Kibria BGM (2004b) Simultaneous estimation of regression parameters with spherically symmetric errors under possible stochastic constraints. Int J Stat Sci 3:1–20
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Norouzirad, M., Arashi, M. Preliminary test and Stein-type shrinkage ridge estimators in robust regression. Stat Papers 60, 1849–1882 (2019). https://doi.org/10.1007/s00362-017-0899-3
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00362-017-0899-3