This vignette demonstrates how works in practice and how it affects model performance in different contexts. Traditional regularization methods like elastic net penalize coefficients, but in many real-world datasets, variables can have unstable variance or sporadic large magnitudes, meaning that a few predictors can dominate the fitted model. By explicitly capping each variable’s maximum contribution \(|x_j\beta_j|\), provides a way to directly control how much influence each predictor can exert. This vignette compares standard elastic net, the standard model, and a walk-forward model designed for time-series data and greater stability.
The first example uses quick start example data from . Theoretically, we expect all models to perform similarly as all features are well-behaving and should rarely trigger large contribution penalties in the model.
ds_qs <- prepare_data("quickstart")
fits_qs <- fit_models(ds_qs$train_X, ds_qs$train_y, ds_qs$test_X, ds_qs$L, ds_qs$mu)
qs_enet_is <- mse_from_beta(ds_qs$train_X, ds_qs$train_y, fits_qs$enet$beta0, fits_qs$enet$beta)
qs_cap1_is <- mse_from_beta(ds_qs$train_X, ds_qs$train_y, fits_qs$cap1$beta0, fits_qs$cap1$beta)
qs_enet_oos <- mse_from_beta(ds_qs$test_X, ds_qs$test_y, fits_qs$enet$beta0, fits_qs$enet$beta)
qs_cap1_oos <- mse_from_beta(ds_qs$test_X, ds_qs$test_y, fits_qs$cap1$beta0, fits_qs$cap1$beta)
qs_cap2_oos <- mse_from_pred(fits_qs$cap2$predictions, ds_qs$test_y)
report_qs <- data.frame(
Model = c("Elan", "Optimized Cap", "Walk Cap"),
IS = c(qs_enet_is, qs_cap1_is, NA),
OOS = c(qs_enet_oos, qs_cap1_oos, qs_cap2_oos)
)
kable(report_qs, digits = 6)| Model | IS | OOS |
|---|---|---|
| Elan | 0.501179 | 0.591483 |
| Optimized Cap | 0.495869 | 0.580050 |
| Walk Cap | NA | 0.537670 |
The in sample (IS) and out of sample (OOS) mean squared errors (MSE) show that the elastic net model and the standard model have similar IS and OOS performances. The walk forward model achieved better OOS performance, which can be attributed to it being more adapted to each out of sample observation. This comes at higher time complexity and the absence of a single trained model that best capture the relationship between the feature and target variables.
To illustrate the strength of the model, I construct a synthetic dataset based on the latent data-generating process \(Z\rightarrow Y\). In this setup, the true predictors \(Z\) drive the outcome, but only a noisy proxy \(X\) is observed. Importantly, some variables in \(X\) exhibit unstable and high variance, occasionally taking on extreme values. Under these conditions, both the Optimized Cap and Walk Cap models outperform the standard elastic net, with the walk-forward implementation achieving the greatest improvement in generalization. These results demonstrate how ’s contribution-based constraints effectively limit the influence of high-variance features that might otherwise distort model predictions.
ds_px <- prepare_data("proxy")
fits_px <- fit_models(ds_px$train_X, ds_px$train_y, ds_px$test_X, ds_px$L, ds_px$mu)
px_enet_is <- mse_from_beta(ds_px$train_X, ds_px$train_y, fits_px$enet$beta0, fits_px$enet$beta)
px_cap1_is <- mse_from_beta(ds_px$train_X, ds_px$train_y, fits_px$cap1$beta0, fits_px$cap1$beta)
px_enet_oos <- mse_from_beta(ds_px$test_X, ds_px$test_y, fits_px$enet$beta0, fits_px$enet$beta)
px_cap1_oos <- mse_from_beta(ds_px$test_X, ds_px$test_y, fits_px$cap1$beta0, fits_px$cap1$beta)
px_cap2_oos <- mse_from_pred(fits_px$cap2$predictions, ds_px$test_y)
report_px <- data.frame(
Model = c("Elan", "Optimized Cap", "Walk Cap"),
IS = c(px_enet_is, px_cap1_is, NA),
OOS = c(px_enet_oos, px_cap1_oos, px_cap2_oos)
)
kable(report_px, digits = 6)| Model | IS | OOS |
|---|---|---|
| Elan | 0.860357 | 0.795325 |
| Optimized Cap | 0.803383 | 0.745417 |
| Walk Cap | NA | 0.681158 |