Penalized Regression with Correlation Based Penalty

A new regularization method for regression models is proposed. The
criterion to be minimized contains a penalty term which explicitly
links strength of penalization to the correlation between
predictors. As the elastic net, the method encourages a grouping
effect where strongly correlated predictors tend to be in or out of
the model together. A boosted version of the penalized estimator,
which is based on a new boosting method, allows to select
variables. Real world data and simulations show that the method
compares well to competing regularization techniques. In settings
where the number of predictors is smaller than the number of
observations it frequently performs better than competitors, in
high dimensional settings prediction measures favor the elastic net
while accuracy of estimation and stability of variable selection
favors the newly proposed method.