sklearn.ensemble
.HistGradientBoostingRegressor¶

class
sklearn.ensemble.
HistGradientBoostingRegressor
(loss=’least_squares’, learning_rate=0.1, max_iter=100, max_leaf_nodes=31, max_depth=None, min_samples_leaf=20, l2_regularization=0.0, max_bins=256, scoring=None, validation_fraction=0.1, n_iter_no_change=None, tol=1e07, verbose=0, random_state=None)[source]¶ Histogrambased Gradient Boosting Regression Tree.
This estimator is much faster than
GradientBoostingRegressor
for big datasets (n_samples >= 10 000). The input dataX
is prebinned into integervalued bins, which considerably reduces the number of splitting points to consider, and allows the algorithm to leverage integerbased data structures. For small sample sizes,GradientBoostingRegressor
might be preferred since binning may lead to split points that are too approximate in this setting.This implementation is inspired by LightGBM.
Note
This estimator is still experimental for now: the predictions and the API might change without any deprecation cycle. To use it, you need to explicitly import
enable_hist_gradient_boosting
:>>> # explicitly require this experimental feature >>> from sklearn.experimental import enable_hist_gradient_boosting # noqa >>> # now you can import normally from ensemble >>> from sklearn.ensemble import HistGradientBoostingClassifier
Parameters:  loss : {‘least_squares’}, optional (default=’least_squares’)
The loss function to use in the boosting process. Note that the “least squares” loss actually implements an “half least squares loss” to simplify the computation of the gradient.
 learning_rate : float, optional (default=0.1)
The learning rate, also known as shrinkage. This is used as a multiplicative factor for the leaves values. Use
1
for no shrinkage. max_iter : int, optional (default=100)
The maximum number of iterations of the boosting process, i.e. the maximum number of trees.
 max_leaf_nodes : int or None, optional (default=31)
The maximum number of leaves for each tree. Must be strictly greater than 1. If None, there is no maximum limit.
 max_depth : int or None, optional (default=None)
The maximum depth of each tree. The depth of a tree is the number of nodes to go from the root to the deepest leaf. Must be strictly greater than 1. Depth isn’t constrained by default.
 min_samples_leaf : int, optional (default=20)
The minimum number of samples per leaf. For small datasets with less than a few hundred samples, it is recommended to lower this value since only very shallow trees would be built.
 l2_regularization : float, optional (default=0)
The L2 regularization parameter. Use
0
for no regularization (default). max_bins : int, optional (default=256)
The maximum number of bins to use. Before training, each feature of the input array
X
is binned into at mostmax_bins
bins, which allows for a much faster training stage. Features with a small number of unique values may use less thanmax_bins
bins. Must be no larger than 256. scoring : str or callable or None, optional (default=None)
Scoring parameter to use for early stopping. It can be a single string (see The scoring parameter: defining model evaluation rules) or a callable (see Defining your scoring strategy from metric functions). If None, the estimator’s default scorer is used. If
scoring='loss'
, early stopping is checked w.r.t the loss value. Only used ifn_iter_no_change
is not None. validation_fraction : int or float or None, optional (default=0.1)
Proportion (or absolute size) of training data to set aside as validation data for early stopping. If None, early stopping is done on the training data. Only used if
n_iter_no_change
is not None. n_iter_no_change : int or None, optional (default=None)
Used to determine when to “early stop”. The fitting process is stopped when none of the last
n_iter_no_change
scores are better than the ``n_iter_no_change  1``thtolast one, up to some tolerance. If None or 0, no earlystopping is done. tol : float or None, optional (default=1e7)
The absolute tolerance to use when comparing scores during early stopping. The higher the tolerance, the more likely we are to early stop: higher tolerance means that it will be harder for subsequent iterations to be considered an improvement upon the reference score.
 verbose: int, optional (default=0)
The verbosity level. If not zero, print some information about the fitting process.
 random_state : int, np.random.RandomStateInstance or None, optional (default=None)
Pseudorandom number generator to control the subsampling in the binning process, and the train/validation data split if early stopping is enabled. See random_state.
Attributes:  n_iter_ : int
The number of iterations as selected by early stopping (if n_iter_no_change is not None). Otherwise it corresponds to max_iter.
 n_trees_per_iteration_ : int
The number of tree that are built at each iteration. For regressors, this is always 1.
 train_score_ : ndarray, shape (max_iter + 1,)
The scores at each iteration on the training data. The first entry is the score of the ensemble before the first iteration. Scores are computed according to the
scoring
parameter. Ifscoring
is not ‘loss’, scores are computed on a subset of at most 10 000 samples. Empty if no early stopping. validation_score_ : ndarray, shape (max_iter + 1,)
The scores at each iteration on the heldout validation data. The first entry is the score of the ensemble before the first iteration. Scores are computed according to the
scoring
parameter. Empty if no early stopping or ifvalidation_fraction
is None.
Examples
>>> # To use this experimental feature, we need to explicitly ask for it: >>> from sklearn.experimental import enable_hist_gradient_boosting # noqa >>> from sklearn.ensemble import HistGradientBoostingRegressor >>> from sklearn.datasets import load_boston >>> X, y = load_boston(return_X_y=True) >>> est = HistGradientBoostingRegressor().fit(X, y) >>> est.score(X, y) 0.98...
Methods
fit
(self, X, y)Fit the gradient boosting model. get_params
(self[, deep])Get parameters for this estimator. predict
(self, X)Predict values for X. score
(self, X, y[, sample_weight])Returns the coefficient of determination R^2 of the prediction. set_params
(self, \*\*params)Set the parameters of this estimator. 
__init__
(self, loss=’least_squares’, learning_rate=0.1, max_iter=100, max_leaf_nodes=31, max_depth=None, min_samples_leaf=20, l2_regularization=0.0, max_bins=256, scoring=None, validation_fraction=0.1, n_iter_no_change=None, tol=1e07, verbose=0, random_state=None)[source]¶

fit
(self, X, y)[source]¶ Fit the gradient boosting model.
Parameters:  X : arraylike, shape=(n_samples, n_features)
The input samples.
 y : arraylike, shape=(n_samples,)
Target values.
Returns:  self : object

get_params
(self, deep=True)[source]¶ Get parameters for this estimator.
Parameters:  deep : boolean, optional
If True, will return the parameters for this estimator and contained subobjects that are estimators.
Returns:  params : mapping of string to any
Parameter names mapped to their values.

predict
(self, X)[source]¶ Predict values for X.
Parameters:  X : arraylike, shape (n_samples, n_features)
The input samples.
Returns:  y : ndarray, shape (n_samples,)
The predicted values.

score
(self, X, y, sample_weight=None)[source]¶ Returns the coefficient of determination R^2 of the prediction.
The coefficient R^2 is defined as (1  u/v), where u is the residual sum of squares ((y_true  y_pred) ** 2).sum() and v is the total sum of squares ((y_true  y_true.mean()) ** 2).sum(). The best possible score is 1.0 and it can be negative (because the model can be arbitrarily worse). A constant model that always predicts the expected value of y, disregarding the input features, would get a R^2 score of 0.0.
Parameters:  X : arraylike, shape = (n_samples, n_features)
Test samples. For some estimators this may be a precomputed kernel matrix instead, shape = (n_samples, n_samples_fitted], where n_samples_fitted is the number of samples used in the fitting for the estimator.
 y : arraylike, shape = (n_samples) or (n_samples, n_outputs)
True values for X.
 sample_weight : arraylike, shape = [n_samples], optional
Sample weights.
Returns:  score : float
R^2 of self.predict(X) wrt. y.
Notes
The R2 score used when calling
score
on a regressor will usemultioutput='uniform_average'
from version 0.23 to keep consistent withmetrics.r2_score
. This will influence thescore
method of all the multioutput regressors (except formultioutput.MultiOutputRegressor
). To specify the default value manually and avoid the warning, please either callmetrics.r2_score
directly or make a custom scorer withmetrics.make_scorer
(the builtin scorer'r2'
usesmultioutput='uniform_average'
).

set_params
(self, **params)[source]¶ Set the parameters of this estimator.
The method works on simple estimators as well as on nested objects (such as pipelines). The latter have parameters of the form
<component>__<parameter>
so that it’s possible to update each component of a nested object.Returns:  self