sklearn.ensemble
.HistGradientBoostingClassifier¶

class
sklearn.ensemble.
HistGradientBoostingClassifier
(loss='auto', *, learning_rate=0.1, max_iter=100, max_leaf_nodes=31, max_depth=None, min_samples_leaf=20, l2_regularization=0.0, max_bins=255, categorical_features=None, monotonic_cst=None, warm_start=False, early_stopping='auto', scoring='loss', validation_fraction=0.1, n_iter_no_change=10, tol=1e07, verbose=0, random_state=None)[source]¶ Histogrambased Gradient Boosting Classification Tree.
This estimator is much faster than
GradientBoostingClassifier
for big datasets (n_samples >= 10 000).This estimator has native support for missing values (NaNs). During training, the tree grower learns at each split point whether samples with missing values should go to the left or right child, based on the potential gain. When predicting, samples with missing values are assigned to the left or right child consequently. If no missing values were encountered for a given feature during training, then samples with missing values are mapped to whichever child has the most samples.
This implementation is inspired by LightGBM.
Note
This estimator is still experimental for now: the predictions and the API might change without any deprecation cycle. To use it, you need to explicitly import
enable_hist_gradient_boosting
:>>> # explicitly require this experimental feature >>> from sklearn.experimental import enable_hist_gradient_boosting # noqa >>> # now you can import normally from ensemble >>> from sklearn.ensemble import HistGradientBoostingClassifier
Read more in the User Guide.
New in version 0.21.
 Parameters
 loss{‘auto’, ‘binary_crossentropy’, ‘categorical_crossentropy’}, default=’auto’
The loss function to use in the boosting process. ‘binary_crossentropy’ (also known as logistic loss) is used for binary classification and generalizes to ‘categorical_crossentropy’ for multiclass classification. ‘auto’ will automatically choose either loss depending on the nature of the problem.
 learning_ratefloat, default=0.1
The learning rate, also known as shrinkage. This is used as a multiplicative factor for the leaves values. Use
1
for no shrinkage. max_iterint, default=100
The maximum number of iterations of the boosting process, i.e. the maximum number of trees for binary classification. For multiclass classification,
n_classes
trees per iteration are built. max_leaf_nodesint or None, default=31
The maximum number of leaves for each tree. Must be strictly greater than 1. If None, there is no maximum limit.
 max_depthint or None, default=None
The maximum depth of each tree. The depth of a tree is the number of edges to go from the root to the deepest leaf. Depth isn’t constrained by default.
 min_samples_leafint, default=20
The minimum number of samples per leaf. For small datasets with less than a few hundred samples, it is recommended to lower this value since only very shallow trees would be built.
 l2_regularizationfloat, default=0
The L2 regularization parameter. Use 0 for no regularization.
 max_binsint, default=255
The maximum number of bins to use for nonmissing values. Before training, each feature of the input array
X
is binned into integervalued bins, which allows for a much faster training stage. Features with a small number of unique values may use less thanmax_bins
bins. In addition to themax_bins
bins, one more bin is always reserved for missing values. Must be no larger than 255. categorical_featuresarraylike of {bool, int} of shape (n_features) or shape (n_categorical_features,), default=None.
Indicates the categorical features.
None : no feature will be considered categorical.
boolean arraylike : boolean mask indicating categorical features.
integer arraylike : integer indices indicating categorical features.
For each categorical feature, there must be at most
max_bins
unique categories, and each categorical value must be in [0, max_bins 1].Read more in the User Guide.
New in version 0.24.
 monotonic_cstarraylike of int of shape (n_features), default=None
Indicates the monotonic constraint to enforce on each feature. 1, 1 and 0 respectively correspond to a negative constraint, positive constraint and no constraint. Read more in the User Guide.
New in version 0.23.
 warm_startbool, default=False
When set to
True
, reuse the solution of the previous call to fit and add more estimators to the ensemble. For results to be valid, the estimator should be retrained on the same data only. See the Glossary. early_stopping‘auto’ or bool, default=’auto’
If ‘auto’, early stopping is enabled if the sample size is larger than 10000. If True, early stopping is enabled, otherwise early stopping is disabled.
New in version 0.23.
 scoringstr or callable or None, default=’loss’
Scoring parameter to use for early stopping. It can be a single string (see The scoring parameter: defining model evaluation rules) or a callable (see Defining your scoring strategy from metric functions). If None, the estimator’s default scorer is used. If
scoring='loss'
, early stopping is checked w.r.t the loss value. Only used if early stopping is performed. validation_fractionint or float or None, default=0.1
Proportion (or absolute size) of training data to set aside as validation data for early stopping. If None, early stopping is done on the training data. Only used if early stopping is performed.
 n_iter_no_changeint, default=10
Used to determine when to “early stop”. The fitting process is stopped when none of the last
n_iter_no_change
scores are better than then_iter_no_change  1
thtolast one, up to some tolerance. Only used if early stopping is performed. tolfloat or None, default=1e7
The absolute tolerance to use when comparing scores. The higher the tolerance, the more likely we are to early stop: higher tolerance means that it will be harder for subsequent iterations to be considered an improvement upon the reference score.
 verboseint, default=0
The verbosity level. If not zero, print some information about the fitting process.
 random_stateint, RandomState instance or None, default=None
Pseudorandom number generator to control the subsampling in the binning process, and the train/validation data split if early stopping is enabled. Pass an int for reproducible output across multiple function calls. See Glossary.
 Attributes
 classes_array, shape = (n_classes,)
Class labels.
 do_early_stopping_bool
Indicates whether early stopping is used during training.
 n_iter_int
The number of iterations as selected by early stopping, depending on the
early_stopping
parameter. Otherwise it corresponds to max_iter. n_trees_per_iteration_int
The number of tree that are built at each iteration. This is equal to 1 for binary classification, and to
n_classes
for multiclass classification. train_score_ndarray, shape (n_iter_+1,)
The scores at each iteration on the training data. The first entry is the score of the ensemble before the first iteration. Scores are computed according to the
scoring
parameter. Ifscoring
is not ‘loss’, scores are computed on a subset of at most 10 000 samples. Empty if no early stopping. validation_score_ndarray, shape (n_iter_+1,)
The scores at each iteration on the heldout validation data. The first entry is the score of the ensemble before the first iteration. Scores are computed according to the
scoring
parameter. Empty if no early stopping or ifvalidation_fraction
is None. is_categorical_ndarray, shape (n_features, ) or None
Boolean mask for the categorical features.
None
if there are no categorical features.
Examples
>>> # To use this experimental feature, we need to explicitly ask for it: >>> from sklearn.experimental import enable_hist_gradient_boosting # noqa >>> from sklearn.ensemble import HistGradientBoostingClassifier >>> from sklearn.datasets import load_iris >>> X, y = load_iris(return_X_y=True) >>> clf = HistGradientBoostingClassifier().fit(X, y) >>> clf.score(X, y) 1.0
Methods
Compute the decision function of
X
.fit
(X, y[, sample_weight])Fit the gradient boosting model.
get_params
([deep])Get parameters for this estimator.
predict
(X)Predict classes for X.
Predict class probabilities for X.
score
(X, y[, sample_weight])Return the mean accuracy on the given test data and labels.
set_params
(**params)Set the parameters of this estimator.
Compute decision function of
X
for each iteration.Predict classes at each iteration.
Predict class probabilities at each iteration.

decision_function
(X)[source]¶ Compute the decision function of
X
. Parameters
 Xarraylike, shape (n_samples, n_features)
The input samples.
 Returns
 decisionndarray, shape (n_samples,) or (n_samples, n_trees_per_iteration)
The raw predicted values (i.e. the sum of the trees leaves) for each sample. n_trees_per_iteration is equal to the number of classes in multiclass classification.

fit
(X, y, sample_weight=None)[source]¶ Fit the gradient boosting model.
 Parameters
 Xarraylike of shape (n_samples, n_features)
The input samples.
 yarraylike of shape (n_samples,)
Target values.
 sample_weightarraylike of shape (n_samples,) default=None
Weights of training data.
New in version 0.23.
 Returns
 selfobject

get_params
(deep=True)[source]¶ Get parameters for this estimator.
 Parameters
 deepbool, default=True
If True, will return the parameters for this estimator and contained subobjects that are estimators.
 Returns
 paramsdict
Parameter names mapped to their values.

predict
(X)[source]¶ Predict classes for X.
 Parameters
 Xarraylike, shape (n_samples, n_features)
The input samples.
 Returns
 yndarray, shape (n_samples,)
The predicted classes.

predict_proba
(X)[source]¶ Predict class probabilities for X.
 Parameters
 Xarraylike, shape (n_samples, n_features)
The input samples.
 Returns
 pndarray, shape (n_samples, n_classes)
The class probabilities of the input samples.

score
(X, y, sample_weight=None)[source]¶ Return the mean accuracy on the given test data and labels.
In multilabel classification, this is the subset accuracy which is a harsh metric since you require for each sample that each label set be correctly predicted.
 Parameters
 Xarraylike of shape (n_samples, n_features)
Test samples.
 yarraylike of shape (n_samples,) or (n_samples, n_outputs)
True labels for
X
. sample_weightarraylike of shape (n_samples,), default=None
Sample weights.
 Returns
 scorefloat
Mean accuracy of
self.predict(X)
wrt.y
.

set_params
(**params)[source]¶ Set the parameters of this estimator.
The method works on simple estimators as well as on nested objects (such as
Pipeline
). The latter have parameters of the form<component>__<parameter>
so that it’s possible to update each component of a nested object. Parameters
 **paramsdict
Estimator parameters.
 Returns
 selfestimator instance
Estimator instance.

staged_decision_function
(X)[source]¶ Compute decision function of
X
for each iteration.This method allows monitoring (i.e. determine error on testing set) after each stage.
 Parameters
 Xarraylike of shape (n_samples, n_features)
The input samples.
 Yields
 decisiongenerator of ndarray of shape (n_samples,) or (n_samples, n_trees_per_iteration)
The decision function of the input samples, which corresponds to the raw values predicted from the trees of the ensemble . The classes corresponds to that in the attribute classes_.

staged_predict
(X)[source]¶ Predict classes at each iteration.
This method allows monitoring (i.e. determine error on testing set) after each stage.
New in version 0.24.
 Parameters
 Xarraylike of shape (n_samples, n_features)
The input samples.
 Yields
 ygenerator of ndarray of shape (n_samples,)
The predicted classes of the input samples, for each iteration.

staged_predict_proba
(X)[source]¶ Predict class probabilities at each iteration.
This method allows monitoring (i.e. determine error on testing set) after each stage.
 Parameters
 Xarraylike of shape (n_samples, n_features)
The input samples.
 Yields
 ygenerator of ndarray of shape (n_samples,)
The predicted class probabilities of the input samples, for each iteration.