From cross-validation to SURE: asymptotic risk of tuned regularized estimators

Working paper

We derive the asymptotic risk function of regularized empirical risk minimization (ERM) estimators tuned by n-fold cross-validation (CV). The out-of-sample prediction loss of such estimators converges in distribution to the squared-error loss (risk function) of shrinkage estimators in the normal means model, tuned by Stein’s unbiased risk estimate (SURE). This risk function provides a more fine-grained picture of predictive performance than uniform bounds on worst-case regret, which are common in learning theory: it quantifies how risk varies with the true parameter. As key intermediate steps, we show that (i) n-fold CV converges uniformly to SURE, and (ii) while SURE typically has multiple local minima, its global minimum is generically well separated. Well-separation ensures that uniform convergence of CV to SURE translates into convergence of the tuning parameter chosen by CV to that chosen by SURE.

Authors: Adusumilli, K; Kasy, M; Wilson, A

• Publication status: Published
• Publisher: University of Oxford
• Series: Department of Economics Discussion Paper Series
• Place of publication: Oxford, UK
• Publication date: 2026-03-19
• Paper number: 1111
• Language: English