.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "auto_examples/frozen/plot_frozen_examples.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_auto_examples_frozen_plot_frozen_examples.py>`
        to download the full example code. or to run this example in your browser via JupyterLite or Binder

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_auto_examples_frozen_plot_frozen_examples.py:


===================================
Examples of Using `FrozenEstimator`
===================================

This examples showcases some use cases of :class:`~sklearn.frozen.FrozenEstimator`.

:class:`~sklearn.frozen.FrozenEstimator` is a utility class that allows to freeze a
fitted estimator. This is useful, for instance, when we want to pass a fitted estimator
to a meta-estimator, such as :class:`~sklearn.model_selection.FixedThresholdClassifier`
without letting the meta-estimator refit the estimator.

.. GENERATED FROM PYTHON SOURCE LINES 13-17

.. code-block:: Python


    # Authors: The scikit-learn developers
    # SPDX-License-Identifier: BSD-3-Clause








.. GENERATED FROM PYTHON SOURCE LINES 18-28

Setting a decision threshold for a pre-fitted classifier
--------------------------------------------------------
Fitted classifiers in scikit-learn use an arbitrary decision threshold to decide
which class the given sample belongs to. The decision threshold is either `0.0` on the
value returned by :term:`decision_function`, or `0.5` on the probability returned by
:term:`predict_proba`.

However, one might want to set a custom decision threshold. We can do this by
using :class:`~sklearn.model_selection.FixedThresholdClassifier` and wrapping the
classifier with :class:`~sklearn.frozen.FrozenEstimator`.

.. GENERATED FROM PYTHON SOURCE LINES 28-46

.. code-block:: Python

    from sklearn.datasets import make_classification
    from sklearn.frozen import FrozenEstimator
    from sklearn.linear_model import LogisticRegression
    from sklearn.model_selection import FixedThresholdClassifier, train_test_split

    X, y = make_classification(n_samples=1000, random_state=0)
    X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=0)
    classifier = LogisticRegression().fit(X_train, y_train)

    print(
        "Probability estimates for three data points:\n"
        f"{classifier.predict_proba(X_test[-3:]).round(3)}"
    )
    print(
        "Predicted class for the same three data points:\n"
        f"{classifier.predict(X_test[-3:])}"
    )





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    Probability estimates for three data points:
    [[0.18 0.82]
     [0.29 0.71]
     [0.   1.  ]]
    Predicted class for the same three data points:
    [1 1 1]




.. GENERATED FROM PYTHON SOURCE LINES 47-51

Now imagine you'd want to set a different decision threshold on the probability
estimates. We can do this by wrapping the classifier with
:class:`~sklearn.frozen.FrozenEstimator` and passing it to
:class:`~sklearn.model_selection.FixedThresholdClassifier`.

.. GENERATED FROM PYTHON SOURCE LINES 51-56

.. code-block:: Python


    threshold_classifier = FixedThresholdClassifier(
        estimator=FrozenEstimator(classifier), threshold=0.9
    )








.. GENERATED FROM PYTHON SOURCE LINES 57-63

Note that in the above piece of code, calling `fit` on
:class:`~sklearn.model_selection.FixedThresholdClassifier` does not refit the
underlying classifier.

Now, let's see how the predictions changed with respect to the probability
threshold.

.. GENERATED FROM PYTHON SOURCE LINES 63-72

.. code-block:: Python

    print(
        "Probability estimates for three data points with FixedThresholdClassifier:\n"
        f"{threshold_classifier.predict_proba(X_test[-3:]).round(3)}"
    )
    print(
        "Predicted class for the same three data points with FixedThresholdClassifier:\n"
        f"{threshold_classifier.predict(X_test[-3:])}"
    )





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    Probability estimates for three data points with FixedThresholdClassifier:
    [[0.18 0.82]
     [0.29 0.71]
     [0.   1.  ]]
    Predicted class for the same three data points with FixedThresholdClassifier:
    [0 0 1]




.. GENERATED FROM PYTHON SOURCE LINES 73-79

We see that the probability estimates stay the same, but since a different decision
threshold is used, the predicted classes are different.

Please refer to
:ref:`sphx_glr_auto_examples_model_selection_plot_cost_sensitive_learning.py`
to learn about cost-sensitive learning and decision threshold tuning.

.. GENERATED FROM PYTHON SOURCE LINES 81-85

Calibration of a pre-fitted classifier
--------------------------------------
You can use :class:`~sklearn.frozen.FrozenEstimator` to calibrate a pre-fitted
classifier using :class:`~sklearn.calibration.CalibratedClassifierCV`.

.. GENERATED FROM PYTHON SOURCE LINES 85-99

.. code-block:: Python

    from sklearn.calibration import CalibratedClassifierCV
    from sklearn.metrics import brier_score_loss

    calibrated_classifier = CalibratedClassifierCV(
        estimator=FrozenEstimator(classifier)
    ).fit(X_train, y_train)

    prob_pos_clf = classifier.predict_proba(X_test)[:, 1]
    clf_score = brier_score_loss(y_test, prob_pos_clf)
    print(f"No calibration: {clf_score:.3f}")

    prob_pos_calibrated = calibrated_classifier.predict_proba(X_test)[:, 1]
    calibrated_score = brier_score_loss(y_test, prob_pos_calibrated)
    print(f"With calibration: {calibrated_score:.3f}")




.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    No calibration: 0.033
    With calibration: 0.032





.. rst-class:: sphx-glr-timing

   **Total running time of the script:** (0 minutes 0.019 seconds)


.. _sphx_glr_download_auto_examples_frozen_plot_frozen_examples.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example

    .. container:: binder-badge

      .. image:: images/binder_badge_logo.svg
        :target: https://mybinder.org/v2/gh/scikit-learn/scikit-learn/1.6.X?urlpath=lab/tree/notebooks/auto_examples/frozen/plot_frozen_examples.ipynb
        :alt: Launch binder
        :width: 150 px

    .. container:: lite-badge

      .. image:: images/jupyterlite_badge_logo.svg
        :target: ../../lite/lab/index.html?path=auto_examples/frozen/plot_frozen_examples.ipynb
        :alt: Launch JupyterLite
        :width: 150 px

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: plot_frozen_examples.ipynb <plot_frozen_examples.ipynb>`

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: plot_frozen_examples.py <plot_frozen_examples.py>`

    .. container:: sphx-glr-download sphx-glr-download-zip

      :download:`Download zipped: plot_frozen_examples.zip <plot_frozen_examples.zip>`


.. include:: plot_frozen_examples.recommendations


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_