Augmented transfer regression learning with semi-non-parametric nuisance models
Article No.: 293, Pages 13924 - 13973
Abstract
We develop an augmented transfer regression learning (ATReL) approach that introduces an imputation model to augment the importance weighting equation to achieve double robustness for covariate shift correction. More significantly, we propose a novel semi-non-parametric (SNP) construction framework for the two nuisance models. Compared with existing doubly robust approaches relying on fully parametric or fully non-parametric (machine learning) nuisance models, our proposal is more exible and balanced to address model misspecification and the curse of dimensionality, achieving a better trade-off in terms of model complexity. The SNP construction presents a new technical challenge in controlling the first-order bias caused by the nuisance estimators. To overcome this, we propose a two-step calibrated estimating approach to construct the nuisance models that ensures the effective reduction of potential bias. Under this SNP framework, our ATReL estimator is n1/2-consistent when (i) at least one nuisance model is correctly specified and (ii) the non-parametric components are rate-doubly robust. Simulation studies demonstrate that our method is more robust and efficient than existing methods under various configurations. We also examine the utility of our method through a real transfer learning example of the phenotyping algorithm for rheumatoid arthritis across different time windows. Finally, we propose ways to enhance the intrinsic efficiency of our estimator and to incorporate modern machine-learning methods in the proposed SNP framework.
References
[1]
David Azriel, Lawrence D Brown, Michael Sklar, Richard Berk, Andreas Buja, and Linda Zhao. Semi-supervised linear regression. arXiv preprint arXiv:1612.02391, 2016.
[2]
Heejung Bang and James M Robins. Doubly robust estimation in missing data and causal inference models. Biometrics, 61(4):962-973, 2005.
[3]
Jay H Beder. A sieve estimator for the mean of a gaussian process. The Annals of Statistics, pages 59-78, 1987.
[4]
Alexandre Belloni, Victor Chernozhukov, Denis Chetverikov, and Kengo Kato. Some new asymptotic theory for least squares series: Pointwise and uniform results. Journal of Econometrics, 186(2):345-366, 2015.
[5]
Tianxi Cai, Mengyan Li, and Molei Liu. Semi-supervised triply robust inductive transfer learning. arXiv preprint arXiv:2209.04977, 2022.
[6]
Sebastian Calonico, Matias D Cattaneo, and Max H Farrell. On the effect of bias estimation on coverage accuracy in nonparametric inference. Journal of the American Statistical Association, 113(522):767-779, 2018.
[7]
Weihua Cao, Anastasios A Tsiatis, and Marie Davidian. Improving efficiency and robustness of the doubly robust estimator for a population mean with incomplete data. Biometrika, 96(3):723-734, 2009.
[8]
Raymond J Carroll, David Ruppert, and Alan H Welsh. Local estimating equations. Journal of the American Statistical Association, 93(441):214-227, 1998.
[9]
Robert J Carroll, Will K Thompson, Anne E Eyler, Arthur M Mandelin, Tianxi Cai, Raquel M Zink, Jennifer A Pacheco, Chad S Boomershine, Thomas A Lasko, Hua Xu, et al. Portability of an algorithm to identify rheumatoid arthritis in electronic health records. Journal of the American Medical Informatics Association, 19(e1):e162-e169, 2012.
[10]
Matias D Cattaneo. Efficient semiparametric estimation of multi-valued treatment effects under ignorability. Journal of Econometrics, 155(2):138-154, 2010.
[11]
Matias D Cattaneo, Max H Farrell, and Yingjie Feng. Large sample properties of partitioning-based series estimators. The Annals of Statistics, 48(3):1718-1741, 2020.
[12]
Abhishek Chakrabortty. Robust Semi-Parametric Inference in Semi-Supervised Settings. PhD thesis, 2016.
[13]
Abhishek Chakrabortty and Tianxi Cai. Efficient and adaptive linear regression in semi-supervised settings. The Annals of Statistics, 46(4):1541-1572, 2018.
[14]
Xiangli Chen, Mathew Monfort, Anqi Liu, and Brian D Ziebart. Robust covariate shift regression. In Artificial Intelligence and Statistics, pages 1270-1279, 2016.
[15]
Xiaohong Chen. Large sample sieve estimation of semi-nonparametric models. Handbook of econometrics, 6:5549-5632, 2007.
[16]
Xiaohong Chen, Han Hong, and Alessandro Tarozzi. Semiparametric efficiency in GMM models with auxiliary data. The Annals of Statistics, 36(2):808-843, 2008.
[17]
Victor Chernozhukov, Denis Chetverikov, Mert Demirer, Esther Duo, Christian Hansen, Whitney Newey, and James Robins. Double/debiased machine learning for treatment and structural parameters, 2018a.
[18]
Victor Chernozhukov, Whitney K Newey, and James Robins. Double/debiased machine learning using regularized riesz representers. Technical report, cemmap working paper, 2018b.
[19]
Oliver Dukes and Stijn Vansteelandt. Inference on treatment effect parameters in potentially misspecified high-dimensional models. Biometrika, 2020.
[20]
Oliver Dukes, Stijn Vansteelandt, and David Whitney. On doubly robust inference for double machine learning. arXiv preprint arXiv:2107.06124, 2021.
[21]
Shinto Eguchi and John Copas. A class of logistic-type discriminant functions. Biometrika, 89(1):1-22, 2002.
[22]
Jianqing Fan, Nancy E Heckman, and Matt P Wand. Local polynomial kernel regression for generalized linear models and quasi-likelihood functions. Journal of the American Statistical Association, 90(429):141-150, 1995.
[23]
Max H Farrell, Tengyuan Liang, and Sanjog Misra. Deep learning for individual heterogeneity: An automatic inference framework. arXiv preprint arXiv:2010.14694, 2020.
[24]
Max H Farrell, Tengyuan Liang, and Sanjog Misra. Deep neural networks for estimation and inference. Econometrica, 89(1):181-213, 2021.
[25]
Satyajit Ghosh and Zhiqiang Tan. Doubly robust semiparametric inference using regularized calibrated estimation with high-dimensional data. arXiv preprint arXiv:2009.12033, 2020.
[26]
Bryan S Graham, Cristine Campos de Xavier Pinto, and Daniel Egel. Efficient estimation of data combination models by the method of auxiliary-to-study tilting (ast). Journal of Business & Economic Statistics, 34(2):288-301, 2016.
[27]
Jessica Gronsbell, Molei Liu, Lu Tian, and Tianxi Cai. Efficient estimation and evaluation of prediction rules in semi-supervised settings under stratified sampling. arXiv preprint arXiv:2010.09443, 2020.
[28]
Jessica L Gronsbell and Tianxi Cai. Semi-supervised approaches to efficient evaluation of model prediction performance. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 80(3):579-594, 2018.
[29]
Jinyong Hahn. Functional restriction and efficiency in causal inference. The Review of Economics and Statistics, 86(1):73-76, 2004.
[30]
Peisong Han. Intrinsic efficiency and multiple robustness in longitudinal studies with dropout. Biometrika, 103(3):683-700, 2016.
[31]
Keisuke Hirano, Guido W Imbens, and Geert Ridder. Efficient estimation of average treatment effects using the estimated propensity score. Econometrica, 71(4):1161-1189, 2003.
[32]
Sicong Huang, Jie Huang, Tianrun Cai, Kumar P Dahal, Andrew Cagan, Zeling He, Jacklyn Stratton, Isaac Gorelik, Chuan Hong, Tianxi Cai, et al. Impact of icd10 and secular changes on electronic medical record rheumatoid arthritis algorithms. Rheumatology, 59 (12):3759-3766, 2020.
[33]
Guido W Imbens and Donald B Rubin. Causal inference in statistics, social, and biomedical sciences. Cambridge University Press, 2015.
[34]
Joseph DY Kang and Joseph L Schafer. Demystifying double robustness: A comparison of alternative strategies for estimating a population mean from incomplete data. Statistical science, 22(4):523-539, 2007.
[35]
Masanori Kawakita and Takafumi Kanamori. Semi-supervised learning with density-ratio estimation. Machine learning, 91(2):189-209, 2013.
[36]
Edward H Kennedy. Optimal doubly robust estimation of heterogeneous causal effects. arXiv preprint arXiv:2004.14497, 2020.
[37]
Seung-Soo Kim, Adam D Hudgins, Brenda Gonzalez, Sofiya Milman, Nir Barzilai, Jan Vijg, Zhidong Tu, and Yousin Suh. A compendium of age-related phewas and gwas traits for human genetic association studies, their networks and genetic correlations. Frontiers in Genetics, 12:842, 2021.
[38]
Isaac S Kohane. Using electronic health records to drive discovery in disease genomics. Nature Reviews Genetics, 12(6):417-428, 2011.
[39]
Isaac S Kohane, Susanne E Churchill, and Shawn N Murphy. A translational engine at the national scale: informatics for integrating biology and the bedside. Journal of the American Medical Informatics Association, 19(2):181-185, 2012.
[40]
Damian Kozbur. Inference in additively separable models with a high-dimensional set of conditioning variables. Journal of Business & Economic Statistics, 39(4):984-1000, 2021.
[41]
Katherine P Liao, Tianxi Cai, Vivian Gainer, Sergey Goryachev, Qing Zeng-treitler, Soumya Raychaudhuri, Peter Szolovits, Susanne Churchill, Shawn Murphy, Isaac Kohane, et al. Electronic medical records for discovery research in rheumatoid arthritis. Arthritis care & research, 62(8):1120-1127, 2010.
[42]
Katherine P Liao, Jiehuan Sun, Tianrun A Cai, Nicholas Link, Chuan Hong, Jie Huang, Jennifer E Huffman, Jessica Gronsbell, Yichi Zhang, and Yuk-Lam Ho. High-throughput multimodal automated phenotyping (map) with application to phewas. Journal of the American Medical Informatics Association, 26(11):1255-1262, 2019.
[43]
Xihong Lin and Raymond J Carroll. Semiparametric estimation in general repeated measures problems. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 68(1):69-88, 2006.
[44]
Anqi Liu and Brian D Ziebart. Robust covariate shift prediction with general losses and feature views. arXiv preprint arXiv:1712.10043, 2017.
[45]
Molei Liu, Yi Zhang, and Doudou Zhou. Double/debiased machine learning for logistic partially linear model. The Econometrics Journal, 24(3):559-588, 2021.
[46]
Whitney K Newey. Convergence rates and asymptotic normality for series estimators. Journal of econometrics, 79(1):147-168, 1997.
[47]
Whitney K Newey and James R Robins. Cross-fitting and fast remainder rates for semi-parametric estimation. arXiv preprint arXiv:1801.09138, 2018.
[48]
Yang Ning, Peng Sida, and Kosuke Imai. Robust estimation of causal effects via a high-dimensional covariate balancing propensity score. Biometrika, 107(3):533-554, 2020.
[49]
David Pollard. Empirical processes: theory and applications. In NSF-CBMS regional conference series in probability and statistics, pages i-86. JSTOR, 1990.
[50]
Jing Qin, Jun Shao, and Biao Zhang. Efficient and doubly robust imputation for covariate-dependent missing responses. Journal of the American Statistical Association, 103(482): 797-810, 2008.
[51]
Zhongjun Qu, Jungmo Yoon, and Pierre Perron. Inference on conditional quantile processes in partially linear models with applications to the impact of unemployment benefits. Review of Economics and Statistics, pages 1-45, 2022.
[52]
Laila Rasmy, Yonghui Wu, Ningtao Wang, Xin Geng, W Jim Zheng, Fei Wang, Hulin Wu, Hua Xu, and Degui Zhi. A study of generalizability of recurrent neural network-based predictive models for heart failure onset risk using a large and heterogeneous ehr data set. Journal of biomedical informatics, 84:11-16, 2018.
[53]
Sashank Jakkam Reddi, Barnabas Poczos, and Alex Smola. Doubly robust covariate shift correction. In Twenty-Ninth AAAI Conference on Artificial Intelligence, 2015.
[54]
Christoph Rothe and Sergio Firpo. Semiparametric two-step estimation using doubly robust moment conditions, 2015.
[55]
Andrea Rotnitzky, Quanhong Lei, Mariela Sued, and James M Robins. Improved doublerobust estimation in missing data and causal inference models. Biometrika, 99(2):439-456, 2012.
[56]
Vira Semenova and Victor Chernozhukov. Debiased machine learning of conditional average treatment effects and other causal functions. The Econometrics Journal, 24(2):264-289, 2021.
[57]
Xiaotong Shen. On methods of sieves and penalization. The Annals of Statistics, pages 2555-2591, 1997.
[58]
Heng Shu and Zhiqiang Tan. Improved estimation of average treatment effects on the treated: Local efficiency, double robustness, and beyond. arXiv preprint arXiv:1808.01408, 2018.
[59]
Ezequiel Smucler, Andrea Rotnitzky, and James M Robins. A unifying approach for doublyrobust ℓ1-regularized estimation of causal contrasts. arXiv preprint arXiv:1904.03737, 2019.
[60]
Zhiqiang Tan. Bounded, efficient and doubly robust estimation with inverse weighting. Biometrika, 97(3):661-682, 2010.
[61]
Zhiqiang Tan. Model-assisted inference for treatment effects using regularized calibrated estimation with high-dimensional data. Annals of Statistics, 48(2):811-837, 2020.
[62]
Mark J van der Laan and Susan Gruber. Collaborative double robust targeted maximum likelihood estimation. The international journal of biostatistics, 6(1), 2010.
[63]
Aad W Van der Vaart. Asymptotic statistics, volume 3. Cambridge University Press, 2000.
[64]
Tyler J VanderWeele. Surrogate measures and consistent surrogates. Biometrics, 69(3): 561-565, 2013.
[65]
Karel Vermeulen and Stijn Vansteelandt. Bias-reduced doubly robust estimation. Journal of the American Statistical Association, 110(511):1024-1036, 2015.
[66]
Junfeng Wen, Chun-Nam Yu, and Russell Greiner. Robust learning under uncertain test distributions: Relating covariate shift to model misspecification. In ICML, pages 631-639, 2014.
[67]
Chunhua Weng, Nigam H Shah, and George Hripcsak. Deep phenotyping: Embracing complexity and temporality--towards scalability, portability, and interoperability. Journal of Biomedical Informatics, 105:103433, 2020.
[68]
Sheng Yu, Abhishek Chakrabortty, Katherine P Liao, Tianrun Cai, Ashwin N Ananthakrishnan, Vivian S Gainer, Susanne E Churchill, Peter Szolovits, Shawn N Murphy, Isaac S Kohane, et al. Surrogate-assisted feature extraction for high-throughput phenotyping. Journal of the American Medical Informatics Association, 24(e1):e143-e149, 2017.
[69]
Qingyuan Zhao and Daniel Percival. Entropy balancing is doubly robust. Journal of Causal Inference, 5(1), 2017.
[70]
Doudou Zhou, Molei Liu, Mengyan Li, and Tianxi Cai. Doubly robust augmented model accuracy transfer inference with high dimensional features. arXiv preprint arXiv:2208.05134, 2022.
Index Terms
- Augmented transfer regression learning with semi-non-parametric nuisance models
Index terms have been assigned to the content through auto-classification.
Recommendations
Robust likelihood inference for multivariate correlated count data
A parametric robust approach for analyzing correlated count data is introduced. This method enables one to construct an asymptotically valid likelihood for the regression parameter when knowledge about the joint distribution for data is scarce or not ...
Asymptotically efficient estimation under semi-parametric random censorship models
We study the estimation of some linear functionals which are based on an unknown lifetime distribution. The observations are assumed to be generated under the semi-parametric random censorship model (SRCM), that is, a random censorship model where the ...
Likelihood ratio tests under model misspecification in high dimensions
AbstractWe investigate the likelihood ratio test for a large block-diagonal covariance matrix with an increasing number of blocks under the null hypothesis. While so far the likelihood ratio statistic has only been studied for normal ...
Comments
Information & Contributors
Information
Published In
January 2023
18881 pages
Copyright © 2023.
CC-BY 4.0
Publisher
JMLR.org
Publication History
Published: 06 March 2024
Accepted: 01 September 2023
Revised: 01 August 2023
Received: 01 June 2022
Published in JMLR Volume 24, Issue 1
Author Tags
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 31Total Downloads
- Downloads (Last 12 months)31
- Downloads (Last 6 weeks)2
Reflects downloads up to 10 Nov 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in