.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "auto_examples/gaussian_process/plot_gpr_prior_posterior.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_auto_examples_gaussian_process_plot_gpr_prior_posterior.py>`
        to download the full example code. or to run this example in your browser via JupyterLite or Binder

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_auto_examples_gaussian_process_plot_gpr_prior_posterior.py:


==========================================================================
Illustration of prior and posterior Gaussian process for different kernels
==========================================================================

This example illustrates the prior and posterior of a
:class:`~sklearn.gaussian_process.GaussianProcessRegressor` with different
kernels. Mean, standard deviation, and 5 samples are shown for both prior
and posterior distributions.

Here, we only give some illustration. To know more about kernels' formulation,
refer to the :ref:`User Guide <gp_kernels>`.

.. GENERATED FROM PYTHON SOURCE LINES 15-19

.. code-block:: Python


    # Authors: The scikit-learn developers
    # SPDX-License-Identifier: BSD-3-Clause








.. GENERATED FROM PYTHON SOURCE LINES 20-32

Helper function
---------------

Before presenting each individual kernel available for Gaussian processes,
we will define an helper function allowing us plotting samples drawn from
the Gaussian process.

This function will take a
:class:`~sklearn.gaussian_process.GaussianProcessRegressor` model and will
drawn sample from the Gaussian process. If the model was not fit, the samples
are drawn from the prior distribution while after model fitting, the samples are
drawn from the posterior distribution.

.. GENERATED FROM PYTHON SOURCE LINES 32-81

.. code-block:: Python

    import matplotlib.pyplot as plt
    import numpy as np


    def plot_gpr_samples(gpr_model, n_samples, ax):
        """Plot samples drawn from the Gaussian process model.

        If the Gaussian process model is not trained then the drawn samples are
        drawn from the prior distribution. Otherwise, the samples are drawn from
        the posterior distribution. Be aware that a sample here corresponds to a
        function.

        Parameters
        ----------
        gpr_model : `GaussianProcessRegressor`
            A :class:`~sklearn.gaussian_process.GaussianProcessRegressor` model.
        n_samples : int
            The number of samples to draw from the Gaussian process distribution.
        ax : matplotlib axis
            The matplotlib axis where to plot the samples.
        """
        x = np.linspace(0, 5, 100)
        X = x.reshape(-1, 1)

        y_mean, y_std = gpr_model.predict(X, return_std=True)
        y_samples = gpr_model.sample_y(X, n_samples)

        for idx, single_prior in enumerate(y_samples.T):
            ax.plot(
                x,
                single_prior,
                linestyle="--",
                alpha=0.7,
                label=f"Sampled function #{idx + 1}",
            )
        ax.plot(x, y_mean, color="black", label="Mean")
        ax.fill_between(
            x,
            y_mean - y_std,
            y_mean + y_std,
            alpha=0.1,
            color="black",
            label=r"$\pm$ 1 std. dev.",
        )
        ax.set_xlabel("x")
        ax.set_ylabel("y")
        ax.set_ylim([-3, 3])









.. GENERATED FROM PYTHON SOURCE LINES 82-85

Dataset and Gaussian process generation
---------------------------------------
We will create a training dataset that we will use in the different sections.

.. GENERATED FROM PYTHON SOURCE LINES 85-90

.. code-block:: Python

    rng = np.random.RandomState(4)
    X_train = rng.uniform(0, 5, 10).reshape(-1, 1)
    y_train = np.sin((X_train[:, 0] - 2.5) ** 2)
    n_samples = 5








.. GENERATED FROM PYTHON SOURCE LINES 91-99

Kernel cookbook
---------------

In this section, we illustrate some samples drawn from the prior and posterior
distributions of the Gaussian process with different kernels.

Radial Basis Function kernel
............................

.. GENERATED FROM PYTHON SOURCE LINES 99-121

.. code-block:: Python

    from sklearn.gaussian_process import GaussianProcessRegressor
    from sklearn.gaussian_process.kernels import RBF

    kernel = 1.0 * RBF(length_scale=1.0, length_scale_bounds=(1e-1, 10.0))
    gpr = GaussianProcessRegressor(kernel=kernel, random_state=0)

    fig, axs = plt.subplots(nrows=2, sharex=True, sharey=True, figsize=(10, 8))

    # plot prior
    plot_gpr_samples(gpr, n_samples=n_samples, ax=axs[0])
    axs[0].set_title("Samples from prior distribution")

    # plot posterior
    gpr.fit(X_train, y_train)
    plot_gpr_samples(gpr, n_samples=n_samples, ax=axs[1])
    axs[1].scatter(X_train[:, 0], y_train, color="red", zorder=10, label="Observations")
    axs[1].legend(bbox_to_anchor=(1.05, 1.5), loc="upper left")
    axs[1].set_title("Samples from posterior distribution")

    fig.suptitle("Radial Basis Function kernel", fontsize=18)
    plt.tight_layout()




.. image-sg:: /auto_examples/gaussian_process/images/sphx_glr_plot_gpr_prior_posterior_001.png
   :alt: Radial Basis Function kernel, Samples from prior distribution, Samples from posterior distribution
   :srcset: /auto_examples/gaussian_process/images/sphx_glr_plot_gpr_prior_posterior_001.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 122-128

.. code-block:: Python

    print(f"Kernel parameters before fit:\n{kernel})")
    print(
        f"Kernel parameters after fit: \n{gpr.kernel_} \n"
        f"Log-likelihood: {gpr.log_marginal_likelihood(gpr.kernel_.theta):.3f}"
    )





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    Kernel parameters before fit:
    1**2 * RBF(length_scale=1))
    Kernel parameters after fit: 
    0.594**2 * RBF(length_scale=0.279) 
    Log-likelihood: -0.067




.. GENERATED FROM PYTHON SOURCE LINES 129-131

Rational Quadratic kernel
.........................

.. GENERATED FROM PYTHON SOURCE LINES 131-152

.. code-block:: Python

    from sklearn.gaussian_process.kernels import RationalQuadratic

    kernel = 1.0 * RationalQuadratic(length_scale=1.0, alpha=0.1, alpha_bounds=(1e-5, 1e15))
    gpr = GaussianProcessRegressor(kernel=kernel, random_state=0)

    fig, axs = plt.subplots(nrows=2, sharex=True, sharey=True, figsize=(10, 8))

    # plot prior
    plot_gpr_samples(gpr, n_samples=n_samples, ax=axs[0])
    axs[0].set_title("Samples from prior distribution")

    # plot posterior
    gpr.fit(X_train, y_train)
    plot_gpr_samples(gpr, n_samples=n_samples, ax=axs[1])
    axs[1].scatter(X_train[:, 0], y_train, color="red", zorder=10, label="Observations")
    axs[1].legend(bbox_to_anchor=(1.05, 1.5), loc="upper left")
    axs[1].set_title("Samples from posterior distribution")

    fig.suptitle("Rational Quadratic kernel", fontsize=18)
    plt.tight_layout()




.. image-sg:: /auto_examples/gaussian_process/images/sphx_glr_plot_gpr_prior_posterior_002.png
   :alt: Rational Quadratic kernel, Samples from prior distribution, Samples from posterior distribution
   :srcset: /auto_examples/gaussian_process/images/sphx_glr_plot_gpr_prior_posterior_002.png
   :class: sphx-glr-single-img


.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    /home/circleci/project/sklearn/gaussian_process/_gpr.py:523: RuntimeWarning:

    covariance is not symmetric positive-semidefinite.





.. GENERATED FROM PYTHON SOURCE LINES 153-159

.. code-block:: Python

    print(f"Kernel parameters before fit:\n{kernel})")
    print(
        f"Kernel parameters after fit: \n{gpr.kernel_} \n"
        f"Log-likelihood: {gpr.log_marginal_likelihood(gpr.kernel_.theta):.3f}"
    )





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    Kernel parameters before fit:
    1**2 * RationalQuadratic(alpha=0.1, length_scale=1))
    Kernel parameters after fit: 
    0.594**2 * RationalQuadratic(alpha=6.69e+08, length_scale=0.279) 
    Log-likelihood: -0.067




.. GENERATED FROM PYTHON SOURCE LINES 160-162

Exp-Sine-Squared kernel
.......................

.. GENERATED FROM PYTHON SOURCE LINES 162-188

.. code-block:: Python

    from sklearn.gaussian_process.kernels import ExpSineSquared

    kernel = 1.0 * ExpSineSquared(
        length_scale=1.0,
        periodicity=3.0,
        length_scale_bounds=(0.1, 10.0),
        periodicity_bounds=(1.0, 10.0),
    )
    gpr = GaussianProcessRegressor(kernel=kernel, random_state=0)

    fig, axs = plt.subplots(nrows=2, sharex=True, sharey=True, figsize=(10, 8))

    # plot prior
    plot_gpr_samples(gpr, n_samples=n_samples, ax=axs[0])
    axs[0].set_title("Samples from prior distribution")

    # plot posterior
    gpr.fit(X_train, y_train)
    plot_gpr_samples(gpr, n_samples=n_samples, ax=axs[1])
    axs[1].scatter(X_train[:, 0], y_train, color="red", zorder=10, label="Observations")
    axs[1].legend(bbox_to_anchor=(1.05, 1.5), loc="upper left")
    axs[1].set_title("Samples from posterior distribution")

    fig.suptitle("Exp-Sine-Squared kernel", fontsize=18)
    plt.tight_layout()




.. image-sg:: /auto_examples/gaussian_process/images/sphx_glr_plot_gpr_prior_posterior_003.png
   :alt: Exp-Sine-Squared kernel, Samples from prior distribution, Samples from posterior distribution
   :srcset: /auto_examples/gaussian_process/images/sphx_glr_plot_gpr_prior_posterior_003.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 189-195

.. code-block:: Python

    print(f"Kernel parameters before fit:\n{kernel})")
    print(
        f"Kernel parameters after fit: \n{gpr.kernel_} \n"
        f"Log-likelihood: {gpr.log_marginal_likelihood(gpr.kernel_.theta):.3f}"
    )





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    Kernel parameters before fit:
    1**2 * ExpSineSquared(length_scale=1, periodicity=3))
    Kernel parameters after fit: 
    0.799**2 * ExpSineSquared(length_scale=0.791, periodicity=2.87) 
    Log-likelihood: 3.394




.. GENERATED FROM PYTHON SOURCE LINES 196-198

Dot-product kernel
..................

.. GENERATED FROM PYTHON SOURCE LINES 198-221

.. code-block:: Python

    from sklearn.gaussian_process.kernels import ConstantKernel, DotProduct

    kernel = ConstantKernel(0.1, (0.01, 10.0)) * (
        DotProduct(sigma_0=1.0, sigma_0_bounds=(0.1, 10.0)) ** 2
    )
    gpr = GaussianProcessRegressor(kernel=kernel, random_state=0, normalize_y=True)

    fig, axs = plt.subplots(nrows=2, sharex=True, sharey=True, figsize=(10, 8))

    # plot prior
    plot_gpr_samples(gpr, n_samples=n_samples, ax=axs[0])
    axs[0].set_title("Samples from prior distribution")

    # plot posterior
    gpr.fit(X_train, y_train)
    plot_gpr_samples(gpr, n_samples=n_samples, ax=axs[1])
    axs[1].scatter(X_train[:, 0], y_train, color="red", zorder=10, label="Observations")
    axs[1].legend(bbox_to_anchor=(1.05, 1.5), loc="upper left")
    axs[1].set_title("Samples from posterior distribution")

    fig.suptitle("Dot-product kernel", fontsize=18)
    plt.tight_layout()




.. image-sg:: /auto_examples/gaussian_process/images/sphx_glr_plot_gpr_prior_posterior_004.png
   :alt: Dot-product kernel, Samples from prior distribution, Samples from posterior distribution
   :srcset: /auto_examples/gaussian_process/images/sphx_glr_plot_gpr_prior_posterior_004.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 222-228

.. code-block:: Python

    print(f"Kernel parameters before fit:\n{kernel})")
    print(
        f"Kernel parameters after fit: \n{gpr.kernel_} \n"
        f"Log-likelihood: {gpr.log_marginal_likelihood(gpr.kernel_.theta):.3f}"
    )





.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    Kernel parameters before fit:
    0.316**2 * DotProduct(sigma_0=1) ** 2)
    Kernel parameters after fit: 
    0.697**2 * DotProduct(sigma_0=0.454) ** 2 
    Log-likelihood: -18108182014.707




.. GENERATED FROM PYTHON SOURCE LINES 229-231

Matérn kernel
..............

.. GENERATED FROM PYTHON SOURCE LINES 231-252

.. code-block:: Python

    from sklearn.gaussian_process.kernels import Matern

    kernel = 1.0 * Matern(length_scale=1.0, length_scale_bounds=(1e-1, 10.0), nu=1.5)
    gpr = GaussianProcessRegressor(kernel=kernel, random_state=0)

    fig, axs = plt.subplots(nrows=2, sharex=True, sharey=True, figsize=(10, 8))

    # plot prior
    plot_gpr_samples(gpr, n_samples=n_samples, ax=axs[0])
    axs[0].set_title("Samples from prior distribution")

    # plot posterior
    gpr.fit(X_train, y_train)
    plot_gpr_samples(gpr, n_samples=n_samples, ax=axs[1])
    axs[1].scatter(X_train[:, 0], y_train, color="red", zorder=10, label="Observations")
    axs[1].legend(bbox_to_anchor=(1.05, 1.5), loc="upper left")
    axs[1].set_title("Samples from posterior distribution")

    fig.suptitle("Matérn kernel", fontsize=18)
    plt.tight_layout()




.. image-sg:: /auto_examples/gaussian_process/images/sphx_glr_plot_gpr_prior_posterior_005.png
   :alt: Matérn kernel, Samples from prior distribution, Samples from posterior distribution
   :srcset: /auto_examples/gaussian_process/images/sphx_glr_plot_gpr_prior_posterior_005.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 253-258

.. code-block:: Python

    print(f"Kernel parameters before fit:\n{kernel})")
    print(
        f"Kernel parameters after fit: \n{gpr.kernel_} \n"
        f"Log-likelihood: {gpr.log_marginal_likelihood(gpr.kernel_.theta):.3f}"
    )




.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    Kernel parameters before fit:
    1**2 * Matern(length_scale=1, nu=1.5))
    Kernel parameters after fit: 
    0.609**2 * Matern(length_scale=0.484, nu=1.5) 
    Log-likelihood: -1.185





.. rst-class:: sphx-glr-timing

   **Total running time of the script:** (0 minutes 1.507 seconds)


.. _sphx_glr_download_auto_examples_gaussian_process_plot_gpr_prior_posterior.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example

    .. container:: binder-badge

      .. image:: images/binder_badge_logo.svg
        :target: https://mybinder.org/v2/gh/scikit-learn/scikit-learn/1.6.X?urlpath=lab/tree/notebooks/auto_examples/gaussian_process/plot_gpr_prior_posterior.ipynb
        :alt: Launch binder
        :width: 150 px

    .. container:: lite-badge

      .. image:: images/jupyterlite_badge_logo.svg
        :target: ../../lite/lab/index.html?path=auto_examples/gaussian_process/plot_gpr_prior_posterior.ipynb
        :alt: Launch JupyterLite
        :width: 150 px

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: plot_gpr_prior_posterior.ipynb <plot_gpr_prior_posterior.ipynb>`

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: plot_gpr_prior_posterior.py <plot_gpr_prior_posterior.py>`

    .. container:: sphx-glr-download sphx-glr-download-zip

      :download:`Download zipped: plot_gpr_prior_posterior.zip <plot_gpr_prior_posterior.zip>`


.. include:: plot_gpr_prior_posterior.recommendations


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_