TY - JOUR
T1 - Stein-rule M-estimation in sparse partially linear models
AU - Raheem, Enayetur
AU - Ahmed, S. Ejaz
AU - Liu, Shuangzhe
N1 - Funding Information:
We would like to express our sincere gratitude to the Reviewers and Editors for their constructive comments and valuable feedback, which greatly contributed to the enhancement of this manuscript. Their meticulous review and thoughtful suggestions played a pivotal role in improving the quality and clarity of our work. Furthermore, S. Ejaz Ahmed would like to thank the colleagues at the University of Canberra for their hospitality and support. The support provided by the Natural Sciences and Engineering Research Council of Canada (NSERC) has been invaluable in conducting the research presented in this manuscript and is gratefully acknowledged.
Publisher Copyright:
© The Author(s) under exclusive licence to Japanese Federation of Statistical Science Associations 2023.
PY - 2024/6
Y1 - 2024/6
N2 - We propose and investigate the statistical properties of shrinkage M-estimators based on Stein-rule estimation for partially linear models under the assumption of sparsity. We are mainly interested in estimating regression coefficients parameter sub-vector with strong signals when the sparsity assumption may or may not hold. Thus, we consider two models, one including all the predictors, leading to a full (unrestricted, or over-fitted) model estimation; and the other with only a few influential predictors, resulting in a submodel (restricted, or under-fitted model) estimation problem. Generally speaking, submodel estimators perform better than full model estimators, when the assumption of sparsity is nearly correct. However, a small departure from this assumption makes submodel estimators biased and inefficient, questioning its applicability for practical reason. On the other hand, the full model estimators may not be desirable due to interpretability and higher estimation errors, specially when a large number of predictors are included in the model. For this reason, we propose shrinkage strategies which combine both full model and submodel estimators in an optimal way. The asymptotic properties of the suggested estimators have been studied both analytically and numerically. The asymptotic bias and risk of the estimators are derived in closed form. In addition, a simulation study is conducted to examine the performance of the estimators in practical settings when sparsity assumption may or may not hold. Our simulation results consolidate the theoretical properties of the estimators.
AB - We propose and investigate the statistical properties of shrinkage M-estimators based on Stein-rule estimation for partially linear models under the assumption of sparsity. We are mainly interested in estimating regression coefficients parameter sub-vector with strong signals when the sparsity assumption may or may not hold. Thus, we consider two models, one including all the predictors, leading to a full (unrestricted, or over-fitted) model estimation; and the other with only a few influential predictors, resulting in a submodel (restricted, or under-fitted model) estimation problem. Generally speaking, submodel estimators perform better than full model estimators, when the assumption of sparsity is nearly correct. However, a small departure from this assumption makes submodel estimators biased and inefficient, questioning its applicability for practical reason. On the other hand, the full model estimators may not be desirable due to interpretability and higher estimation errors, specially when a large number of predictors are included in the model. For this reason, we propose shrinkage strategies which combine both full model and submodel estimators in an optimal way. The asymptotic properties of the suggested estimators have been studied both analytically and numerically. The asymptotic bias and risk of the estimators are derived in closed form. In addition, a simulation study is conducted to examine the performance of the estimators in practical settings when sparsity assumption may or may not hold. Our simulation results consolidate the theoretical properties of the estimators.
KW - Asymptotic bias and risk
KW - Full model and submodel M-estimation
KW - Simulation
KW - Stein-rule M-estimation
UR - http://www.scopus.com/inward/record.url?scp=85180428785&partnerID=8YFLogxK
U2 - 10.1007/s42081-023-00231-0
DO - 10.1007/s42081-023-00231-0
M3 - Article
AN - SCOPUS:85180428785
SN - 2520-8764
VL - 7
SP - 507
EP - 535
JO - Japanese Journal of Statistics and Data Science
JF - Japanese Journal of Statistics and Data Science
IS - 1
ER -