.. only:: html .. note:: :class: sphx-glr-download-link-note Click :ref:`here ` to download the full example code or to run this example in your browser via Binder .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_examples_gaussian_process_plot_gpc.py: ==================================================================== Probabilistic predictions with Gaussian process classification (GPC) ==================================================================== This example illustrates the predicted probability of GPC for an RBF kernel with different choices of the hyperparameters. The first figure shows the predicted probability of GPC with arbitrarily chosen hyperparameters and with the hyperparameters corresponding to the maximum log-marginal-likelihood (LML). While the hyperparameters chosen by optimizing LML have a considerable larger LML, they perform slightly worse according to the log-loss on test data. The figure shows that this is because they exhibit a steep change of the class probabilities at the class boundaries (which is good) but have predicted probabilities close to 0.5 far away from the class boundaries (which is bad) This undesirable effect is caused by the Laplace approximation used internally by GPC. The second figure shows the log-marginal-likelihood for different choices of the kernel's hyperparameters, highlighting the two choices of the hyperparameters used in the first figure by black dots. .. rst-class:: sphx-glr-horizontal * .. image:: /auto_examples/gaussian_process/images/sphx_glr_plot_gpc_001.png :alt: plot gpc :class: sphx-glr-multi-img * .. image:: /auto_examples/gaussian_process/images/sphx_glr_plot_gpc_002.png :alt: Log-marginal-likelihood :class: sphx-glr-multi-img .. rst-class:: sphx-glr-script-out Out: .. code-block:: none Log Marginal Likelihood (initial): -17.598 Log Marginal Likelihood (optimized): -3.875 Accuracy: 1.000 (initial) 1.000 (optimized) Log-loss: 0.214 (initial) 0.319 (optimized) | .. code-block:: default print(__doc__) # Authors: Jan Hendrik Metzen # # License: BSD 3 clause import numpy as np from matplotlib import pyplot as plt from sklearn.metrics import accuracy_score, log_loss from sklearn.gaussian_process import GaussianProcessClassifier from sklearn.gaussian_process.kernels import RBF # Generate data train_size = 50 rng = np.random.RandomState(0) X = rng.uniform(0, 5, 100)[:, np.newaxis] y = np.array(X[:, 0] > 2.5, dtype=int) # Specify Gaussian Processes with fixed and optimized hyperparameters gp_fix = GaussianProcessClassifier(kernel=1.0 * RBF(length_scale=1.0), optimizer=None) gp_fix.fit(X[:train_size], y[:train_size]) gp_opt = GaussianProcessClassifier(kernel=1.0 * RBF(length_scale=1.0)) gp_opt.fit(X[:train_size], y[:train_size]) print("Log Marginal Likelihood (initial): %.3f" % gp_fix.log_marginal_likelihood(gp_fix.kernel_.theta)) print("Log Marginal Likelihood (optimized): %.3f" % gp_opt.log_marginal_likelihood(gp_opt.kernel_.theta)) print("Accuracy: %.3f (initial) %.3f (optimized)" % (accuracy_score(y[:train_size], gp_fix.predict(X[:train_size])), accuracy_score(y[:train_size], gp_opt.predict(X[:train_size])))) print("Log-loss: %.3f (initial) %.3f (optimized)" % (log_loss(y[:train_size], gp_fix.predict_proba(X[:train_size])[:, 1]), log_loss(y[:train_size], gp_opt.predict_proba(X[:train_size])[:, 1]))) # Plot posteriors plt.figure() plt.scatter(X[:train_size, 0], y[:train_size], c='k', label="Train data", edgecolors=(0, 0, 0)) plt.scatter(X[train_size:, 0], y[train_size:], c='g', label="Test data", edgecolors=(0, 0, 0)) X_ = np.linspace(0, 5, 100) plt.plot(X_, gp_fix.predict_proba(X_[:, np.newaxis])[:, 1], 'r', label="Initial kernel: %s" % gp_fix.kernel_) plt.plot(X_, gp_opt.predict_proba(X_[:, np.newaxis])[:, 1], 'b', label="Optimized kernel: %s" % gp_opt.kernel_) plt.xlabel("Feature") plt.ylabel("Class 1 probability") plt.xlim(0, 5) plt.ylim(-0.25, 1.5) plt.legend(loc="best") # Plot LML landscape plt.figure() theta0 = np.logspace(0, 8, 30) theta1 = np.logspace(-1, 1, 29) Theta0, Theta1 = np.meshgrid(theta0, theta1) LML = [[gp_opt.log_marginal_likelihood(np.log([Theta0[i, j], Theta1[i, j]])) for i in range(Theta0.shape[0])] for j in range(Theta0.shape[1])] LML = np.array(LML).T plt.plot(np.exp(gp_fix.kernel_.theta)[0], np.exp(gp_fix.kernel_.theta)[1], 'ko', zorder=10) plt.plot(np.exp(gp_opt.kernel_.theta)[0], np.exp(gp_opt.kernel_.theta)[1], 'ko', zorder=10) plt.pcolor(Theta0, Theta1, LML) plt.xscale("log") plt.yscale("log") plt.colorbar() plt.xlabel("Magnitude") plt.ylabel("Length-scale") plt.title("Log-marginal-likelihood") plt.show() .. rst-class:: sphx-glr-timing **Total running time of the script:** ( 0 minutes 2.673 seconds) .. _sphx_glr_download_auto_examples_gaussian_process_plot_gpc.py: .. only :: html .. container:: sphx-glr-footer :class: sphx-glr-footer-example .. container:: binder-badge .. image:: https://mybinder.org/badge_logo.svg :target: https://mybinder.org/v2/gh/scikit-learn/scikit-learn/0.23.X?urlpath=lab/tree/notebooks/auto_examples/gaussian_process/plot_gpc.ipynb :width: 150 px .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: plot_gpc.py ` .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: plot_gpc.ipynb ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_