.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "auto_examples/linear_model/plot_lasso_lars_ic.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_auto_examples_linear_model_plot_lasso_lars_ic.py>`
        to download the full example code or to run this example in your browser via JupyterLite or Binder

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_auto_examples_linear_model_plot_lasso_lars_ic.py:


==============================================
Lasso model selection via information criteria
==============================================

This example reproduces the example of Fig. 2 of [ZHT2007]_. A
:class:`~sklearn.linear_model.LassoLarsIC` estimator is fit on a
diabetes dataset and the AIC and the BIC criteria are used to select
the best model.

.. note::
    It is important to note that the optimization to find `alpha` with
    :class:`~sklearn.linear_model.LassoLarsIC` relies on the AIC or BIC
    criteria that are computed in-sample, thus on the training set directly.
    This approach differs from the cross-validation procedure. For a comparison
    of the two approaches, you can refer to the following example:
    :ref:`sphx_glr_auto_examples_linear_model_plot_lasso_model_selection.py`.

.. topic:: References

    .. [ZHT2007] :arxiv:`Zou, Hui, Trevor Hastie, and Robert Tibshirani.
       "On the degrees of freedom of the lasso."
       The Annals of Statistics 35.5 (2007): 2173-2192.
       <0712.0881>`

.. GENERATED FROM PYTHON SOURCE LINES 26-31

.. code-block:: Python


    # Author: Alexandre Gramfort
    #         Guillaume Lemaitre
    # License: BSD 3 clause








.. GENERATED FROM PYTHON SOURCE LINES 32-33

We will use the diabetes dataset.

.. GENERATED FROM PYTHON SOURCE LINES 33-39

.. code-block:: Python

    from sklearn.datasets import load_diabetes

    X, y = load_diabetes(return_X_y=True, as_frame=True)
    n_samples = X.shape[0]
    X.head()






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    <div>
    <style scoped>
        .dataframe tbody tr th:only-of-type {
            vertical-align: middle;
        }

        .dataframe tbody tr th {
            vertical-align: top;
        }

        .dataframe thead th {
            text-align: right;
        }
    </style>
    <table border="1" class="dataframe">
      <thead>
        <tr style="text-align: right;">
          <th></th>
          <th>age</th>
          <th>sex</th>
          <th>bmi</th>
          <th>bp</th>
          <th>s1</th>
          <th>s2</th>
          <th>s3</th>
          <th>s4</th>
          <th>s5</th>
          <th>s6</th>
        </tr>
      </thead>
      <tbody>
        <tr>
          <th>0</th>
          <td>0.038076</td>
          <td>0.050680</td>
          <td>0.061696</td>
          <td>0.021872</td>
          <td>-0.044223</td>
          <td>-0.034821</td>
          <td>-0.043401</td>
          <td>-0.002592</td>
          <td>0.019907</td>
          <td>-0.017646</td>
        </tr>
        <tr>
          <th>1</th>
          <td>-0.001882</td>
          <td>-0.044642</td>
          <td>-0.051474</td>
          <td>-0.026328</td>
          <td>-0.008449</td>
          <td>-0.019163</td>
          <td>0.074412</td>
          <td>-0.039493</td>
          <td>-0.068332</td>
          <td>-0.092204</td>
        </tr>
        <tr>
          <th>2</th>
          <td>0.085299</td>
          <td>0.050680</td>
          <td>0.044451</td>
          <td>-0.005670</td>
          <td>-0.045599</td>
          <td>-0.034194</td>
          <td>-0.032356</td>
          <td>-0.002592</td>
          <td>0.002861</td>
          <td>-0.025930</td>
        </tr>
        <tr>
          <th>3</th>
          <td>-0.089063</td>
          <td>-0.044642</td>
          <td>-0.011595</td>
          <td>-0.036656</td>
          <td>0.012191</td>
          <td>0.024991</td>
          <td>-0.036038</td>
          <td>0.034309</td>
          <td>0.022688</td>
          <td>-0.009362</td>
        </tr>
        <tr>
          <th>4</th>
          <td>0.005383</td>
          <td>-0.044642</td>
          <td>-0.036385</td>
          <td>0.021872</td>
          <td>0.003935</td>
          <td>0.015596</td>
          <td>0.008142</td>
          <td>-0.002592</td>
          <td>-0.031988</td>
          <td>-0.046641</td>
        </tr>
      </tbody>
    </table>
    </div>
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 40-48

Scikit-learn provides an estimator called
:class:`~sklearn.linear_model.LassoLarsIC` that uses either Akaike's
information criterion (AIC) or the Bayesian information criterion (BIC) to
select the best model. Before fitting
this model, we will scale the dataset.

In the following, we are going to fit two models to compare the values
reported by AIC and BIC.

.. GENERATED FROM PYTHON SOURCE LINES 48-55

.. code-block:: Python

    from sklearn.linear_model import LassoLarsIC
    from sklearn.pipeline import make_pipeline
    from sklearn.preprocessing import StandardScaler

    lasso_lars_ic = make_pipeline(StandardScaler(), LassoLarsIC(criterion="aic")).fit(X, y)









.. GENERATED FROM PYTHON SOURCE LINES 56-61

To be in line with the definition in [ZHT2007]_, we need to rescale the
AIC and the BIC. Indeed, Zou et al. are ignoring some constant terms
compared to the original definition of AIC derived from the maximum
log-likelihood of a linear model. You can refer to
:ref:`mathematical detail section for the User Guide <lasso_lars_ic>`.

.. GENERATED FROM PYTHON SOURCE LINES 61-66

.. code-block:: Python

    def zou_et_al_criterion_rescaling(criterion, n_samples, noise_variance):
        """Rescale the information criterion to follow the definition of Zou et al."""
        return criterion - n_samples * np.log(2 * np.pi * noise_variance) - n_samples









.. GENERATED FROM PYTHON SOURCE LINES 67-79

.. code-block:: Python

    import numpy as np

    aic_criterion = zou_et_al_criterion_rescaling(
        lasso_lars_ic[-1].criterion_,
        n_samples,
        lasso_lars_ic[-1].noise_variance_,
    )

    index_alpha_path_aic = np.flatnonzero(
        lasso_lars_ic[-1].alphas_ == lasso_lars_ic[-1].alpha_
    )[0]








.. GENERATED FROM PYTHON SOURCE LINES 80-92

.. code-block:: Python

    lasso_lars_ic.set_params(lassolarsic__criterion="bic").fit(X, y)

    bic_criterion = zou_et_al_criterion_rescaling(
        lasso_lars_ic[-1].criterion_,
        n_samples,
        lasso_lars_ic[-1].noise_variance_,
    )

    index_alpha_path_bic = np.flatnonzero(
        lasso_lars_ic[-1].alphas_ == lasso_lars_ic[-1].alpha_
    )[0]








.. GENERATED FROM PYTHON SOURCE LINES 93-96

Now that we collected the AIC and BIC, we can as well check that the minima
of both criteria happen at the same alpha. Then, we can simplify the
following plot.

.. GENERATED FROM PYTHON SOURCE LINES 96-98

.. code-block:: Python

    index_alpha_path_aic == index_alpha_path_bic





.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    True



.. GENERATED FROM PYTHON SOURCE LINES 99-101

Finally, we can plot the AIC and BIC criterion and the subsequent selected
regularization parameter.

.. GENERATED FROM PYTHON SOURCE LINES 101-117

.. code-block:: Python

    import matplotlib.pyplot as plt

    plt.plot(aic_criterion, color="tab:blue", marker="o", label="AIC criterion")
    plt.plot(bic_criterion, color="tab:orange", marker="o", label="BIC criterion")
    plt.vlines(
        index_alpha_path_bic,
        aic_criterion.min(),
        aic_criterion.max(),
        color="black",
        linestyle="--",
        label="Selected alpha",
    )
    plt.legend()
    plt.ylabel("Information criterion")
    plt.xlabel("Lasso model sequence")
    _ = plt.title("Lasso model selection via AIC and BIC")



.. image-sg:: /auto_examples/linear_model/images/sphx_glr_plot_lasso_lars_ic_001.png
   :alt: Lasso model selection via AIC and BIC
   :srcset: /auto_examples/linear_model/images/sphx_glr_plot_lasso_lars_ic_001.png
   :class: sphx-glr-single-img






.. rst-class:: sphx-glr-timing

   **Total running time of the script:** (0 minutes 0.103 seconds)


.. _sphx_glr_download_auto_examples_linear_model_plot_lasso_lars_ic.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example

    .. container:: binder-badge

      .. image:: images/binder_badge_logo.svg
        :target: https://mybinder.org/v2/gh/scikit-learn/scikit-learn/main?urlpath=lab/tree/notebooks/auto_examples/linear_model/plot_lasso_lars_ic.ipynb
        :alt: Launch binder
        :width: 150 px

    .. container:: lite-badge

      .. image:: images/jupyterlite_badge_logo.svg
        :target: ../../lite/lab/?path=auto_examples/linear_model/plot_lasso_lars_ic.ipynb
        :alt: Launch JupyterLite
        :width: 150 px

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: plot_lasso_lars_ic.ipynb <plot_lasso_lars_ic.ipynb>`

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: plot_lasso_lars_ic.py <plot_lasso_lars_ic.py>`


.. include:: plot_lasso_lars_ic.recommendations


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_