This library solves linear regression models. It find the parameter $\hat \beta$ of a linear model, such that given an input $x$ and it produces the prediction. $$\hat{y} = \hat\beta^\top x.$$

This library solves the regression problem (estimating the parameter $\hat\beta$) using adversarial training. The idea is to make the model robust by optimizing in the presence of disturbances. It considers training inputs have been contaminated with disturbances deliberately chosen to maximize the model error.

The code is companion to the paper:

• Efficient implementation of Adversarial Training for Linear Models. Antônio H. RIbeiro, Thomas B. Schön, Dave Zahariah, Francis Bach. International Conference on Artificial Intelligence and Statistics, AISTATS 2025

Move into the downloaded directory and install requirements with:

pip install -r requirements.txt

In sequence, install package with:

python setup.py install

You can reproduce the results and figures from the paper by running:

cd paper
bash generate_figs.sh

from linadvtrain.regression import lin_advregr
import numpy as np

# Generate dataset
rng = np.random.RandomState(5)
X = np.array([0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9])[:, None]
noise = 0.15 * rng.randn(len(x))
y = x + noise
# Adversarial estimation
estimated_params, info = lin_advregr(X, y)

The image ilustrate the adversarial training the output of the example above, with the adversarial radius being highlighted as error bars. See examples/one_dimensional.py for more details.

Bellow we illustrate one example using diabetes dataset. When p = 2 we are computing the $\ell_2$ adversarial training, and when p = np.inf we are computing the $\ell_\infty$ adversarial training. Since adv_radius is not set, the default value is used.

import numpy as np
from linadvtrain.regression import lin_advregr
from sklearn import datasets

X, y = datasets.load_diabetes(return_X_y=True)
# Standardize data
X -= X.mean(axis=0)
X /= X.std(axis=0)

# Compute l2adversarial training
estim_param, info = lin_advregr(X, y, p=2)

# Compute linf adversarial training
estim_param, info = lin_advregr(X, y, p=np.inf)

We also illustrate from bellow how the code can be used for reproducing results from the paper:

Regularization properties of adversarially-trained linear regression Antônio H. Ribeiro, Dave Zachariah, Francis Bach, Thomas B. Schön NeurIPS 2023.

$\ell_\infty$ adversarial training has similarities with Lasso. $\ell_2$ adversarial training has similarities with ridge regression. See examples/diabetes_path.py for more details.

Lasso	Adv. training $\ell_\infty$

Ridge	Adv. training $\ell_2$

For small delta adversarial training interpolates the training data. See examples/transition_into_interpolation.py for more details.

Lasso	Adv. training $\ell_\infty$

Ridge	Adv. training $\ell_2$