pyriemann.stats.PermutationModel

class pyriemann.stats.PermutationModel(n_perms=100, model=MDM(), cv=3, scoring=None, n_jobs=1, random_state=42)

Permutation test using any scikit-learn model for scoring.

Perform a permutation test using the cross-validation score of any scikit-learn compatible model. Score is obtained with cross_val_score from scikit-learn. The score should be a “higer is better” metric.

Parameters

n_permsint, default=100

The number of permutation. The minimum should be 20 for a resolution of 0.05 p-value.

modelsklearn compatible model, default=MDM()

The model for scoring.

cvint or cross-validation generator or an iterable, default=3

Determines the cross-validation splitting strategy. Possible inputs for cv are:

• None, to use the default 3-fold cross validation;

• integer, to specify the number of folds in a (Stratified)KFold;

• an object to be used as a cross-validation generator;

• an iterable yielding train, test splits.

For integer/None inputs, if the estimator is a classifier and y is either binary or multiclass, StratifiedKFold is used. In all other cases, KFold is used.

scoringstring or callable or None, default=None

A string (see model evaluation documentation) or a scorer callable object / function with signature scorer(estimator, X, y).

n_jobsinteger, default=1

The number of CPUs to use to do the computation. -1 means ‘all CPUs’.

random_stateint, default=42

random state for the permutation test.

See also

PermutationDistance

Attributes

p_value_float

the p-value of the test

scores_list

contain all scores for all permutations. The fist element is the non-permuted score.

__init__(n_perms=100, model=MDM(), cv=3, scoring=None, n_jobs=1, random_state=42)

Init.

plot(nbins=10, range=None, axes=None)

Plot results of the permutation test.

Parameters

nbinsinteger or array_like or ‘auto’, default=10

If an integer is given, bins + 1 bin edges are returned, consistently with np.histogram() for numpy version >= 1.3. Unequally spaced bins are supported if bins is a sequence.

rangetuple or None, default=None

The lower and upper range of the bins. Lower and upper outliers are ignored. If not provided, range is (x.min(), x.max()). Range has no effect if bins is a sequence. If bins is a sequence or range is specified, autoscaling is based on the specified bin range instead of the range of x.

axesaxes handle, default=None

Axes handle for matplotlib. if None a new figure will be created.

score(X, y, groups=None)

Score one permutation.

Parameters

Xarray-like

The data to fit. Can be, for example a list, or an array at least 2d.

yarray-like

The target variable to try to predict in the case of supervised learning.

test(X, y, groups=None, verbose=True)

Performs the permutation test

Parameters

Xarray-like

The data to fit. Can be, for example a list, or an array at least 2d.

yarray-like

The target variable to try to predict in the case of supervised learning.

verbosebool, default=True

If true, print progress.