.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "auto_examples/ensemble/plot_ensemble_oob.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code or to run this example in your browser via JupyterLite or Binder .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_examples_ensemble_plot_ensemble_oob.py: ============================= OOB Errors for Random Forests ============================= The ``RandomForestClassifier`` is trained using *bootstrap aggregation*, where each new tree is fit from a bootstrap sample of the training observations :math:`z_i = (x_i, y_i)`. The *out-of-bag* (OOB) error is the average error for each :math:`z_i` calculated using predictions from the trees that do not contain :math:`z_i` in their respective bootstrap sample. This allows the ``RandomForestClassifier`` to be fit and validated whilst being trained [1]_. The example below demonstrates how the OOB error can be measured at the addition of each new tree during training. The resulting plot allows a practitioner to approximate a suitable value of ``n_estimators`` at which the error stabilizes. .. [1] T. Hastie, R. Tibshirani and J. Friedman, "Elements of Statistical Learning Ed. 2", p592-593, Springer, 2009. .. GENERATED FROM PYTHON SOURCE LINES 22-106 .. image-sg:: /auto_examples/ensemble/images/sphx_glr_plot_ensemble_oob_001.png :alt: plot ensemble oob :srcset: /auto_examples/ensemble/images/sphx_glr_plot_ensemble_oob_001.png :class: sphx-glr-single-img .. code-block:: Python # Author: Kian Ho # Gilles Louppe # Andreas Mueller # # License: BSD 3 Clause from collections import OrderedDict import matplotlib.pyplot as plt from sklearn.datasets import make_classification from sklearn.ensemble import RandomForestClassifier RANDOM_STATE = 123 # Generate a binary classification dataset. X, y = make_classification( n_samples=500, n_features=25, n_clusters_per_class=1, n_informative=15, random_state=RANDOM_STATE, ) # NOTE: Setting the `warm_start` construction parameter to `True` disables # support for parallelized ensembles but is necessary for tracking the OOB # error trajectory during training. ensemble_clfs = [ ( "RandomForestClassifier, max_features='sqrt'", RandomForestClassifier( warm_start=True, oob_score=True, max_features="sqrt", random_state=RANDOM_STATE, ), ), ( "RandomForestClassifier, max_features='log2'", RandomForestClassifier( warm_start=True, max_features="log2", oob_score=True, random_state=RANDOM_STATE, ), ), ( "RandomForestClassifier, max_features=None", RandomForestClassifier( warm_start=True, max_features=None, oob_score=True, random_state=RANDOM_STATE, ), ), ] # Map a classifier name to a list of (, ) pairs. error_rate = OrderedDict((label, []) for label, _ in ensemble_clfs) # Range of `n_estimators` values to explore. min_estimators = 15 max_estimators = 150 for label, clf in ensemble_clfs: for i in range(min_estimators, max_estimators + 1, 5): clf.set_params(n_estimators=i) clf.fit(X, y) # Record the OOB error for each `n_estimators=i` setting. oob_error = 1 - clf.oob_score_ error_rate[label].append((i, oob_error)) # Generate the "OOB error rate" vs. "n_estimators" plot. for label, clf_err in error_rate.items(): xs, ys = zip(*clf_err) plt.plot(xs, ys, label=label) plt.xlim(min_estimators, max_estimators) plt.xlabel("n_estimators") plt.ylabel("OOB error rate") plt.legend(loc="upper right") plt.show() .. rst-class:: sphx-glr-timing **Total running time of the script:** (0 minutes 4.453 seconds) .. _sphx_glr_download_auto_examples_ensemble_plot_ensemble_oob.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: binder-badge .. image:: images/binder_badge_logo.svg :target: https://mybinder.org/v2/gh/scikit-learn/scikit-learn/main?urlpath=lab/tree/notebooks/auto_examples/ensemble/plot_ensemble_oob.ipynb :alt: Launch binder :width: 150 px .. container:: lite-badge .. image:: images/jupyterlite_badge_logo.svg :target: ../../lite/lab/?path=auto_examples/ensemble/plot_ensemble_oob.ipynb :alt: Launch JupyterLite :width: 150 px .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: plot_ensemble_oob.ipynb ` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: plot_ensemble_oob.py ` .. include:: plot_ensemble_oob.recommendations .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_