.. _sphx_glr_auto_examples_model_selection:

.. _model_selection_examples:

Model Selection
-----------------------

Examples related to the :mod:`sklearn.model_selection` module.



.. raw:: html

    <div class="sphx-glr-thumbnails">


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This example balances model complexity and cross-validated score by finding a decent accuracy w...">

.. only:: html

  .. image:: /auto_examples/model_selection/images/thumb/sphx_glr_plot_grid_search_refit_callable_thumb.png
    :alt:

  :ref:`sphx_glr_auto_examples_model_selection_plot_grid_search_refit_callable.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Balance model complexity and cross-validated score</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This example demonstrates the class_likelihood_ratios function, which computes the positive and...">

.. only:: html

  .. image:: /auto_examples/model_selection/images/thumb/sphx_glr_plot_likelihood_ratios_thumb.png
    :alt:

  :ref:`sphx_glr_auto_examples_model_selection_plot_likelihood_ratios.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Class Likelihood Ratios to measure classification performance</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Compare randomized search and grid search for optimizing hyperparameters of a linear SVM with S...">

.. only:: html

  .. image:: /auto_examples/model_selection/images/thumb/sphx_glr_plot_randomized_search_thumb.png
    :alt:

  :ref:`sphx_glr_auto_examples_model_selection_plot_randomized_search.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Comparing randomized search and grid search for hyperparameter estimation</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This example compares the parameter search performed by HalvingGridSearchCV and GridSearchCV.">

.. only:: html

  .. image:: /auto_examples/model_selection/images/thumb/sphx_glr_plot_successive_halving_heatmap_thumb.png
    :alt:

  :ref:`sphx_glr_auto_examples_model_selection_plot_successive_halving_heatmap.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Comparison between grid search and successive halving</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Example of confusion matrix usage to evaluate the quality of the output of a classifier on the ...">

.. only:: html

  .. image:: /auto_examples/model_selection/images/thumb/sphx_glr_plot_confusion_matrix_thumb.png
    :alt:

  :ref:`sphx_glr_auto_examples_model_selection_plot_confusion_matrix.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Confusion matrix</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This examples shows how a classifier is optimized by cross-validation, which is done using the ...">

.. only:: html

  .. image:: /auto_examples/model_selection/images/thumb/sphx_glr_plot_grid_search_digits_thumb.png
    :alt:

  :ref:`sphx_glr_auto_examples_model_selection_plot_grid_search_digits.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Custom refit strategy of a grid search with cross-validation</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Multiple metric parameter search can be done by setting the scoring parameter to a list of metr...">

.. only:: html

  .. image:: /auto_examples/model_selection/images/thumb/sphx_glr_plot_multi_metric_evaluation_thumb.png
    :alt:

  :ref:`sphx_glr_auto_examples_model_selection_plot_multi_metric_evaluation.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Demonstration of multi-metric evaluation on cross_val_score and GridSearchCV</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="In this example, we compare two binary classification multi-threshold metrics: the Receiver Ope...">

.. only:: html

  .. image:: /auto_examples/model_selection/images/thumb/sphx_glr_plot_det_thumb.png
    :alt:

  :ref:`sphx_glr_auto_examples_model_selection_plot_det.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Detection error tradeoff (DET) curve</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This example describes the use of the Receiver Operating Characteristic (ROC) metric to evaluat...">

.. only:: html

  .. image:: /auto_examples/model_selection/images/thumb/sphx_glr_plot_roc_thumb.png
    :alt:

  :ref:`sphx_glr_auto_examples_model_selection_plot_roc.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Multiclass Receiver Operating Characteristic (ROC)</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This example compares non-nested and nested cross-validation strategies on a classifier of the ...">

.. only:: html

  .. image:: /auto_examples/model_selection/images/thumb/sphx_glr_plot_nested_cross_validation_iris_thumb.png
    :alt:

  :ref:`sphx_glr_auto_examples_model_selection_plot_nested_cross_validation_iris.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Nested versus non-nested cross-validation</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This example shows how to use cross_val_predict together with PredictionErrorDisplay to visuali...">

.. only:: html

  .. image:: /auto_examples/model_selection/images/thumb/sphx_glr_plot_cv_predict_thumb.png
    :alt:

  :ref:`sphx_glr_auto_examples_model_selection_plot_cv_predict.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Plotting Cross-Validated Predictions</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="In this example, we show how to use the class LearningCurveDisplay to easily plot learning curv...">

.. only:: html

  .. image:: /auto_examples/model_selection/images/thumb/sphx_glr_plot_learning_curve_thumb.png
    :alt:

  :ref:`sphx_glr_auto_examples_model_selection_plot_learning_curve.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Plotting Learning Curves and Checking Models' Scalability</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="In this plot you can see the training scores and validation scores of an SVM for different valu...">

.. only:: html

  .. image:: /auto_examples/model_selection/images/thumb/sphx_glr_plot_validation_curve_thumb.png
    :alt:

  :ref:`sphx_glr_auto_examples_model_selection_plot_validation_curve.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Plotting Validation Curves</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Example of Precision-Recall metric to evaluate classifier output quality.">

.. only:: html

  .. image:: /auto_examples/model_selection/images/thumb/sphx_glr_plot_precision_recall_thumb.png
    :alt:

  :ref:`sphx_glr_auto_examples_model_selection_plot_precision_recall.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Precision-Recall</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This example presents how to estimate and visualize the variance of the Receiver Operating Char...">

.. only:: html

  .. image:: /auto_examples/model_selection/images/thumb/sphx_glr_plot_roc_crossval_thumb.png
    :alt:

  :ref:`sphx_glr_auto_examples_model_selection_plot_roc_crossval.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Receiver Operating Characteristic (ROC) with cross validation</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="The dataset used in this example is 20newsgroups_dataset which will be automatically downloaded...">

.. only:: html

  .. image:: /auto_examples/model_selection/images/thumb/sphx_glr_plot_grid_search_text_feature_extraction_thumb.png
    :alt:

  :ref:`sphx_glr_auto_examples_model_selection_plot_grid_search_text_feature_extraction.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Sample pipeline for text feature extraction and evaluation</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This example illustrates how to statistically compare the performance of models trained and eva...">

.. only:: html

  .. image:: /auto_examples/model_selection/images/thumb/sphx_glr_plot_grid_search_stats_thumb.png
    :alt:

  :ref:`sphx_glr_auto_examples_model_selection_plot_grid_search_stats.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Statistical comparison of models using grid search</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This example illustrates how a successive halving search (~sklearn.model_selection.HalvingGridS...">

.. only:: html

  .. image:: /auto_examples/model_selection/images/thumb/sphx_glr_plot_successive_halving_iterations_thumb.png
    :alt:

  :ref:`sphx_glr_auto_examples_model_selection_plot_successive_halving_iterations.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Successive Halving Iterations</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This example demonstrates the use of permutation_test_score to evaluate the significance of a c...">

.. only:: html

  .. image:: /auto_examples/model_selection/images/thumb/sphx_glr_plot_permutation_tests_for_classification_thumb.png
    :alt:

  :ref:`sphx_glr_auto_examples_model_selection_plot_permutation_tests_for_classification.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Test with permutations the significance of a classification score</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Illustration of how the performance of an estimator on unseen data (test data) is not the same ...">

.. only:: html

  .. image:: /auto_examples/model_selection/images/thumb/sphx_glr_plot_train_error_vs_test_error_thumb.png
    :alt:

  :ref:`sphx_glr_auto_examples_model_selection_plot_train_error_vs_test_error.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Train error vs Test error</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This example demonstrates the problems of underfitting and overfitting and how we can use linea...">

.. only:: html

  .. image:: /auto_examples/model_selection/images/thumb/sphx_glr_plot_underfitting_overfitting_thumb.png
    :alt:

  :ref:`sphx_glr_auto_examples_model_selection_plot_underfitting_overfitting.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Underfitting vs. Overfitting</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Choosing the right cross-validation object is a crucial part of fitting a model properly. There...">

.. only:: html

  .. image:: /auto_examples/model_selection/images/thumb/sphx_glr_plot_cv_indices_thumb.png
    :alt:

  :ref:`sphx_glr_auto_examples_model_selection_plot_cv_indices.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Visualizing cross-validation behavior in scikit-learn</div>
    </div>


.. raw:: html

    </div>


.. toctree::
   :hidden:

   /auto_examples/model_selection/plot_grid_search_refit_callable
   /auto_examples/model_selection/plot_likelihood_ratios
   /auto_examples/model_selection/plot_randomized_search
   /auto_examples/model_selection/plot_successive_halving_heatmap
   /auto_examples/model_selection/plot_confusion_matrix
   /auto_examples/model_selection/plot_grid_search_digits
   /auto_examples/model_selection/plot_multi_metric_evaluation
   /auto_examples/model_selection/plot_det
   /auto_examples/model_selection/plot_roc
   /auto_examples/model_selection/plot_nested_cross_validation_iris
   /auto_examples/model_selection/plot_cv_predict
   /auto_examples/model_selection/plot_learning_curve
   /auto_examples/model_selection/plot_validation_curve
   /auto_examples/model_selection/plot_precision_recall
   /auto_examples/model_selection/plot_roc_crossval
   /auto_examples/model_selection/plot_grid_search_text_feature_extraction
   /auto_examples/model_selection/plot_grid_search_stats
   /auto_examples/model_selection/plot_successive_halving_iterations
   /auto_examples/model_selection/plot_permutation_tests_for_classification
   /auto_examples/model_selection/plot_train_error_vs_test_error
   /auto_examples/model_selection/plot_underfitting_overfitting
   /auto_examples/model_selection/plot_cv_indices