Stein-rule M-estimation in sparse partially linear models

We propose and investigate the statistical properties of shrinkage M-estimators based on Stein-rule estimation for partially linear models under the assumption of sparsity. We are mainly interested in estimating regression coefficients parameter sub-vector with strong signals when the sparsity assumption may or may not hold. Thus, we consider two models, one including all the predictors, leading to a full (unrestricted, or over-fitted) model estimation; and the other with only a few influential predictors, resulting in a submodel (restricted, or under-fitted model) estimation problem. Generally speaking, submodel estimators perform better than full model estimators, when the assumption of sparsity is nearly correct. However, a small departure from this assumption makes submodel estimators biased and inefficient, questioning its applicability for practical reason. On the other hand, the full model estimators may not be desirable due to interpretability and higher estimation errors, specially when a large number of predictors are included in the model. For this reason, we propose shrinkage strategies which combine both full model and submodel estimators in an optimal way. The asymptotic properties of the suggested estimators have been studied both analytically and numerically. The asymptotic bias and risk of the estimators are derived in closed form. In addition, a simulation study is conducted to examine the performance of the estimators in practical settings when sparsity assumption may or may not hold. Our simulation results consolidate the theoretical properties of the estimators.