Abstract
We propose an efficient algorithm for calculating hold-out and cross-validation (CV) type of estimates for sparse regularized least-squares predictors. Holding out H data points with our method requires O(min(H 2 n,Hn 2)) time provided that a predictor with n basis vectors is already trained. In addition to holding out training examples, also some of the basis vectors used to train the sparse regularized least-squares predictor with the whole training set can be removed from the basis vector set used in the hold-out computation. In our experiments, we demonstrate the speed improvements provided by our algorithm in practice, and we empirically show the benefits of removing some of the basis vectors during the CV rounds.
| Original language | English |
|---|---|
| Pages (from-to) | 381-407 |
| Number of pages | 27 |
| Journal | Machine Learning |
| Volume | 87 |
| Issue number | 3 |
| DOIs | |
| Publication status | Published - Jun 2012 |
| Externally published | Yes |
Fingerprint
Cite this
}
Efficient cross-validation for kernelized least-squares regression with sparse basis expansions. / Pahikkala, Tapio; Suominen, Hanna; Boberg, Jorma.
In: Machine Learning, Vol. 87, No. 3, 06.2012, p. 381-407.Research output: Contribution to journal › Article
TY - JOUR
T1 - Efficient cross-validation for kernelized least-squares regression with sparse basis expansions
AU - Pahikkala, Tapio
AU - Suominen, Hanna
AU - Boberg, Jorma
PY - 2012/6
Y1 - 2012/6
N2 - We propose an efficient algorithm for calculating hold-out and cross-validation (CV) type of estimates for sparse regularized least-squares predictors. Holding out H data points with our method requires O(min(H 2 n,Hn 2)) time provided that a predictor with n basis vectors is already trained. In addition to holding out training examples, also some of the basis vectors used to train the sparse regularized least-squares predictor with the whole training set can be removed from the basis vector set used in the hold-out computation. In our experiments, we demonstrate the speed improvements provided by our algorithm in practice, and we empirically show the benefits of removing some of the basis vectors during the CV rounds.
AB - We propose an efficient algorithm for calculating hold-out and cross-validation (CV) type of estimates for sparse regularized least-squares predictors. Holding out H data points with our method requires O(min(H 2 n,Hn 2)) time provided that a predictor with n basis vectors is already trained. In addition to holding out training examples, also some of the basis vectors used to train the sparse regularized least-squares predictor with the whole training set can be removed from the basis vector set used in the hold-out computation. In our experiments, we demonstrate the speed improvements provided by our algorithm in practice, and we empirically show the benefits of removing some of the basis vectors during the CV rounds.
KW - Cross-validation
KW - Hold-out
KW - Kernel methods
KW - Least-squares support vector machine
KW - Regularized least-squares
KW - Sparse basis expansions
UR - http://www.scopus.com/inward/record.url?scp=84862027224&partnerID=8YFLogxK
U2 - 10.1007/s10994-012-5287-6
DO - 10.1007/s10994-012-5287-6
M3 - Article
VL - 87
SP - 381
EP - 407
JO - Machine Learning
JF - Machine Learning
SN - 0885-6125
IS - 3
ER -