permutation_importance(estimator, X, y, *, scoring=None, n_repeats=5, n_jobs=None, random_state=None)¶
Permutation importance for feature evaluation [BRE].
The estimator is required to be a fitted estimator.
Xcan be the data set used to train the estimator or a hold-out set. The permutation importance of a feature is calculated as follows. First, a baseline metric, defined by scoring, is evaluated on a (potentially different) dataset defined by the
X. Next, a feature column from the validation set is permuted and the metric is evaluated again. The permutation importance is defined to be the difference between the baseline metric and metric from permutating the feature column.
Read more in the User Guide.
- Xndarray or DataFrame, shape (n_samples, n_features)
Data on which permutation importance will be computed.
- yarray-like or None, shape (n_samples, ) or (n_samples, n_classes)
Targets for supervised or
- scoringstring, callable or None, default=None
Scorer to use. It can be a single string (see The scoring parameter: defining model evaluation rules) or a callable (see Defining your scoring strategy from metric functions). If None, the estimator’s default scorer is used.
- n_repeatsint, default=5
Number of times to permute a feature.
- n_jobsint or None, default=None
- random_stateint, RandomState instance, default=None
Pseudo-random number generator to control the permutations of each feature. Pass an int to get reproducible results across function calls. See :term:
Dictionary-like object, with the following attributes.
- importances_meanndarray, shape (n_features, )
Mean of feature importance over
- importances_stdndarray, shape (n_features, )
Standard deviation over
- importancesndarray, shape (n_features, n_repeats)
Raw permutation importance scores.
L. Breiman, “Random Forests”, Machine Learning, 45(1), 5-32, 2001. https://doi.org/10.1023/A:1010933404324
>>> from sklearn.linear_model import LogisticRegression >>> from sklearn.inspection import permutation_importance >>> X = [[1, 9, 9],[1, 9, 9],[1, 9, 9], ... [0, 9, 9],[0, 9, 9],[0, 9, 9]] >>> y = [1, 1, 1, 0, 0, 0] >>> clf = LogisticRegression().fit(X, y) >>> result = permutation_importance(clf, X, y, n_repeats=10, ... random_state=0) >>> result.importances_mean array([0.4666..., 0. , 0. ]) >>> result.importances_std array([0.2211..., 0. , 0. ])