.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "auto_examples/release_highlights/plot_release_highlights_1_1_0.py"
.. LINE NUMBERS ARE GIVEN BELOW.
.. only:: html
.. note::
:class: sphx-glr-download-link-note
Click :ref:`here <sphx_glr_download_auto_examples_release_highlights_plot_release_highlights_1_1_0.py>`
to download the full example code or to run this example in your browser via Binder
.. rst-class:: sphx-glr-example-title
.. _sphx_glr_auto_examples_release_highlights_plot_release_highlights_1_1_0.py:
=======================================
Release Highlights for scikit-learn 1.1
=======================================
.. currentmodule:: sklearn
We are pleased to announce the release of scikit-learn 1.1! Many bug fixes
and improvements were added, as well as some new key features. We detail
below a few of the major features of this release. **For an exhaustive list of
all the changes**, please refer to the :ref:`release notes <changes_1_1>`.
To install the latest version (with pip)::
pip install --upgrade scikit-learn
or with conda::
conda install -c conda-forge scikit-learn
.. GENERATED FROM PYTHON SOURCE LINES 25-29
Quantile loss in :class:`ensemble.HistGradientBoostingRegressor`
----------------------------------------------------------------
:class:`ensemble.HistGradientBoostingRegressor` can model quantiles with
`loss="quantile"` and the new parameter `quantile`.
.. GENERATED FROM PYTHON SOURCE LINES 29-55
.. code-block:: default
from sklearn.ensemble import HistGradientBoostingRegressor
import numpy as np
import matplotlib.pyplot as plt
# Simple regression function for X * cos(X)
rng = np.random.RandomState(42)
X_1d = np.linspace(0, 10, num=2000)
X = X_1d.reshape(-1, 1)
y = X_1d * np.cos(X_1d) + rng.normal(scale=X_1d / 3)
quantiles = [0.95, 0.5, 0.05]
parameters = dict(loss="quantile", max_bins=32, max_iter=50)
hist_quantiles = {
f"quantile={quantile:.2f}": HistGradientBoostingRegressor(
**parameters, quantile=quantile
).fit(X, y)
for quantile in quantiles
}
fig, ax = plt.subplots()
ax.plot(X_1d, y, "o", alpha=0.5, markersize=1)
for quantile, hist in hist_quantiles.items():
ax.plot(X_1d, hist.predict(X), label=quantile)
_ = ax.legend(loc="lower left")
.. image-sg:: /auto_examples/release_highlights/images/sphx_glr_plot_release_highlights_1_1_0_001.png
:alt: plot release highlights 1 1 0
:srcset: /auto_examples/release_highlights/images/sphx_glr_plot_release_highlights_1_1_0_001.png
:class: sphx-glr-single-img
.. GENERATED FROM PYTHON SOURCE LINES 56-61
`get_feature_names_out` Available in all Transformers
-----------------------------------------------------
:term:`get_feature_names_out` is now available in all Transformers. This enables
:class:`pipeline.Pipeline` to construct the output feature names for more complex
pipelines:
.. GENERATED FROM PYTHON SOURCE LINES 61-89
.. code-block:: default
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import OneHotEncoder, StandardScaler
from sklearn.pipeline import make_pipeline
from sklearn.impute import SimpleImputer
from sklearn.feature_selection import SelectKBest
from sklearn.datasets import fetch_openml
from sklearn.linear_model import LogisticRegression
X, y = fetch_openml("titanic", version=1, as_frame=True, return_X_y=True)
numeric_features = ["age", "fare"]
numeric_transformer = make_pipeline(SimpleImputer(strategy="median"), StandardScaler())
categorical_features = ["embarked", "pclass"]
preprocessor = ColumnTransformer(
[
("num", numeric_transformer, numeric_features),
(
"cat",
OneHotEncoder(handle_unknown="ignore", sparse=False),
categorical_features,
),
],
verbose_feature_names_out=False,
)
log_reg = make_pipeline(preprocessor, SelectKBest(k=7), LogisticRegression())
log_reg.fit(X, y)
.. raw:: html
<div class="output_subarea output_html rendered_html output_result">
<style>#sk-container-id-1 {color: black;background-color: white;}#sk-container-id-1 pre{padding: 0;}#sk-container-id-1 div.sk-toggleable {background-color: white;}#sk-container-id-1 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-1 label.sk-toggleable__label-arrow:before {content: "▸";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-1 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-1 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-1 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-1 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-1 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-1 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: "▾";}#sk-container-id-1 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-1 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-1 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-1 div.sk-parallel-item::after {content: "";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-1 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-serial::before {content: "";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-1 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-1 div.sk-item {position: relative;z-index: 1;}#sk-container-id-1 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-1 div.sk-item::before, #sk-container-id-1 div.sk-parallel-item::before {content: "";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-1 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-1 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-1 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-1 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-1 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-1 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-1 div.sk-label-container {text-align: center;}#sk-container-id-1 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-1 div.sk-text-repr-fallback {display: none;}</style><div id="sk-container-id-1" class="sk-top-container"><div class="sk-text-repr-fallback"><pre>Pipeline(steps=[('columntransformer',
ColumnTransformer(transformers=[('num',
Pipeline(steps=[('simpleimputer',
SimpleImputer(strategy='median')),
('standardscaler',
StandardScaler())]),
['age', 'fare']),
('cat',
OneHotEncoder(handle_unknown='ignore',
sparse=False),
['embarked', 'pclass'])],
verbose_feature_names_out=False)),
('selectkbest', SelectKBest(k=7)),
('logisticregression', LogisticRegression())])</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class="sk-container" hidden><div class="sk-item sk-dashed-wrapped"><div class="sk-label-container"><div class="sk-label sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="sk-estimator-id-1" type="checkbox" ><label for="sk-estimator-id-1" class="sk-toggleable__label sk-toggleable__label-arrow">Pipeline</label><div class="sk-toggleable__content"><pre>Pipeline(steps=[('columntransformer',
ColumnTransformer(transformers=[('num',
Pipeline(steps=[('simpleimputer',
SimpleImputer(strategy='median')),
('standardscaler',
StandardScaler())]),
['age', 'fare']),
('cat',
OneHotEncoder(handle_unknown='ignore',
sparse=False),
['embarked', 'pclass'])],
verbose_feature_names_out=False)),
('selectkbest', SelectKBest(k=7)),
('logisticregression', LogisticRegression())])</pre></div></div></div><div class="sk-serial"><div class="sk-item sk-dashed-wrapped"><div class="sk-label-container"><div class="sk-label sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="sk-estimator-id-2" type="checkbox" ><label for="sk-estimator-id-2" class="sk-toggleable__label sk-toggleable__label-arrow">columntransformer: ColumnTransformer</label><div class="sk-toggleable__content"><pre>ColumnTransformer(transformers=[('num',
Pipeline(steps=[('simpleimputer',
SimpleImputer(strategy='median')),
('standardscaler',
StandardScaler())]),
['age', 'fare']),
('cat',
OneHotEncoder(handle_unknown='ignore',
sparse=False),
['embarked', 'pclass'])],
verbose_feature_names_out=False)</pre></div></div></div><div class="sk-parallel"><div class="sk-parallel-item"><div class="sk-item"><div class="sk-label-container"><div class="sk-label sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="sk-estimator-id-3" type="checkbox" ><label for="sk-estimator-id-3" class="sk-toggleable__label sk-toggleable__label-arrow">num</label><div class="sk-toggleable__content"><pre>['age', 'fare']</pre></div></div></div><div class="sk-serial"><div class="sk-item"><div class="sk-serial"><div class="sk-item"><div class="sk-estimator sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="sk-estimator-id-4" type="checkbox" ><label for="sk-estimator-id-4" class="sk-toggleable__label sk-toggleable__label-arrow">SimpleImputer</label><div class="sk-toggleable__content"><pre>SimpleImputer(strategy='median')</pre></div></div></div><div class="sk-item"><div class="sk-estimator sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="sk-estimator-id-5" type="checkbox" ><label for="sk-estimator-id-5" class="sk-toggleable__label sk-toggleable__label-arrow">StandardScaler</label><div class="sk-toggleable__content"><pre>StandardScaler()</pre></div></div></div></div></div></div></div></div><div class="sk-parallel-item"><div class="sk-item"><div class="sk-label-container"><div class="sk-label sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="sk-estimator-id-6" type="checkbox" ><label for="sk-estimator-id-6" class="sk-toggleable__label sk-toggleable__label-arrow">cat</label><div class="sk-toggleable__content"><pre>['embarked', 'pclass']</pre></div></div></div><div class="sk-serial"><div class="sk-item"><div class="sk-estimator sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="sk-estimator-id-7" type="checkbox" ><label for="sk-estimator-id-7" class="sk-toggleable__label sk-toggleable__label-arrow">OneHotEncoder</label><div class="sk-toggleable__content"><pre>OneHotEncoder(handle_unknown='ignore', sparse=False)</pre></div></div></div></div></div></div></div></div><div class="sk-item"><div class="sk-estimator sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="sk-estimator-id-8" type="checkbox" ><label for="sk-estimator-id-8" class="sk-toggleable__label sk-toggleable__label-arrow">SelectKBest</label><div class="sk-toggleable__content"><pre>SelectKBest(k=7)</pre></div></div></div><div class="sk-item"><div class="sk-estimator sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="sk-estimator-id-9" type="checkbox" ><label for="sk-estimator-id-9" class="sk-toggleable__label sk-toggleable__label-arrow">LogisticRegression</label><div class="sk-toggleable__content"><pre>LogisticRegression()</pre></div></div></div></div></div></div></div>
</div>
<br />
<br />
.. GENERATED FROM PYTHON SOURCE LINES 90-94
Here we slice the pipeline to include all the steps but the last one. The output
feature names of this pipeline slice are the features put into logistic
regression. These names correspond directly to the coefficients in the logistic
regression:
.. GENERATED FROM PYTHON SOURCE LINES 94-101
.. code-block:: default
import pandas as pd
log_reg_input_features = log_reg[:-1].get_feature_names_out()
pd.Series(log_reg[-1].coef_.ravel(), index=log_reg_input_features).plot.bar()
plt.tight_layout()
.. image-sg:: /auto_examples/release_highlights/images/sphx_glr_plot_release_highlights_1_1_0_002.png
:alt: plot release highlights 1 1 0
:srcset: /auto_examples/release_highlights/images/sphx_glr_plot_release_highlights_1_1_0_002.png
:class: sphx-glr-single-img
.. GENERATED FROM PYTHON SOURCE LINES 102-108
Grouping infrequent categories in :class:`OneHotEncoder`
--------------------------------------------------------
:class:`OneHotEncoder` supports aggregating infrequent categories into a single
output for each feature. The parameters to enable the gathering of infrequent
categories are `min_frequency` and `max_categories`. See the
:ref:`User Guide <one_hot_encoder_infrequent_categories>` for more details.
.. GENERATED FROM PYTHON SOURCE LINES 108-117
.. code-block:: default
from sklearn.preprocessing import OneHotEncoder
import numpy as np
X = np.array(
[["dog"] * 5 + ["cat"] * 20 + ["rabbit"] * 10 + ["snake"] * 3], dtype=object
).T
enc = OneHotEncoder(min_frequency=6, sparse=False).fit(X)
enc.infrequent_categories_
.. rst-class:: sphx-glr-script-out
.. code-block:: none
[array(['dog', 'snake'], dtype=object)]
.. GENERATED FROM PYTHON SOURCE LINES 118-120
Since dog and snake are infrequent categories, they are grouped together when
transformed:
.. GENERATED FROM PYTHON SOURCE LINES 120-123
.. code-block:: default
encoded = enc.transform(np.array([["dog"], ["snake"], ["cat"], ["rabbit"]]))
pd.DataFrame(encoded, columns=enc.get_feature_names_out())
.. raw:: html
<div class="output_subarea output_html rendered_html output_result">
<div>
<style scoped>
.dataframe tbody tr th:only-of-type {
vertical-align: middle;
}
.dataframe tbody tr th {
vertical-align: top;
}
.dataframe thead th {
text-align: right;
}
</style>
<table border="1" class="dataframe">
<thead>
<tr style="text-align: right;">
<th></th>
<th>x0_cat</th>
<th>x0_rabbit</th>
<th>x0_infrequent_sklearn</th>
</tr>
</thead>
<tbody>
<tr>
<th>0</th>
<td>0.0</td>
<td>0.0</td>
<td>1.0</td>
</tr>
<tr>
<th>1</th>
<td>0.0</td>
<td>0.0</td>
<td>1.0</td>
</tr>
<tr>
<th>2</th>
<td>1.0</td>
<td>0.0</td>
<td>0.0</td>
</tr>
<tr>
<th>3</th>
<td>0.0</td>
<td>1.0</td>
<td>0.0</td>
</tr>
</tbody>
</table>
</div>
</div>
<br />
<br />
.. GENERATED FROM PYTHON SOURCE LINES 124-164
Performance improvements
------------------------
Reductions on pairwise distances for dense float64 datasets has been refactored
to better take advantage of non-blocking thread parallelism. For example,
:meth:`neighbors.NearestNeighbors.kneighbors` and
:meth:`neighbors.NearestNeighbors.radius_neighbors` can respectively be up to ×20 and
×5 faster than previously. In summary, the following functions and estimators
now benefit from improved performance:
- :func:`metrics.pairwise_distances_argmin`
- :func:`metrics.pairwise_distances_argmin_min`
- :class:`cluster.AffinityPropagation`
- :class:`cluster.Birch`
- :class:`cluster.MeanShift`
- :class:`cluster.OPTICS`
- :class:`cluster.SpectralClustering`
- :func:`feature_selection.mutual_info_regression`
- :class:`neighbors.KNeighborsClassifier`
- :class:`neighbors.KNeighborsRegressor`
- :class:`neighbors.RadiusNeighborsClassifier`
- :class:`neighbors.RadiusNeighborsRegressor`
- :class:`neighbors.LocalOutlierFactor`
- :class:`neighbors.NearestNeighbors`
- :class:`manifold.Isomap`
- :class:`manifold.LocallyLinearEmbedding`
- :class:`manifold.TSNE`
- :func:`manifold.trustworthiness`
- :class:`semi_supervised.LabelPropagation`
- :class:`semi_supervised.LabelSpreading`
To know more about the technical details of this work, you can read
`this suite of blog posts <https://blog.scikit-learn.org/technical/performances/>`_.
Moreover, the computation of loss functions has been refactored using
Cython resulting in performance improvements for the following estimators:
- :class:`linear_model.LogisticRegression`
- :class:`linear_model.GammaRegressor`
- :class:`linear_model.PoissonRegressor`
- :class:`linear_model.TweedieRegressor`
.. GENERATED FROM PYTHON SOURCE LINES 166-175
MiniBatchNMF: an online version of NMF
--------------------------------------
The new class :class:`decomposition.MiniBatchNMF` implements a faster but less
accurate version of non-negative matrix factorization (:class:`decomposition.NMF`).
:class:`MiniBatchNMF` divides the data into mini-batches and optimizes the NMF model
in an online manner by cycling over the mini-batches, making it better suited for
large datasets. In particular, it implements `partial_fit`, which can be used for
online learning when the data is not readily available from the start, or when the
data does not fit into memory.
.. GENERATED FROM PYTHON SOURCE LINES 175-198
.. code-block:: default
import numpy as np
from sklearn.decomposition import MiniBatchNMF
rng = np.random.RandomState(0)
n_samples, n_features, n_components = 10, 10, 5
true_W = rng.uniform(size=(n_samples, n_components))
true_H = rng.uniform(size=(n_components, n_features))
X = true_W @ true_H
nmf = MiniBatchNMF(n_components=n_components, random_state=0)
for _ in range(10):
nmf.partial_fit(X)
W = nmf.transform(X)
H = nmf.components_
X_reconstructed = W @ H
print(
f"relative reconstruction error: ",
f"{np.sum((X - X_reconstructed) ** 2) / np.sum(X**2):.5f}",
)
.. rst-class:: sphx-glr-script-out
.. code-block:: none
relative reconstruction error: 0.00364
.. GENERATED FROM PYTHON SOURCE LINES 199-206
BisectingKMeans: divide and cluster
-----------------------------------
The new class :class:`cluster.BisectingKMeans` is a variant of :class:`KMeans`, using
divisive hierarchical clustering. Instead of creating all centroids at once, centroids
are picked progressively based on a previous clustering: a cluster is split into two
new clusters repeatedly until the target number of clusters is reached, giving a
hierarchical structure to the clustering.
.. GENERATED FROM PYTHON SOURCE LINES 206-225
.. code-block:: default
from sklearn.datasets import make_blobs
from sklearn.cluster import KMeans, BisectingKMeans
import matplotlib.pyplot as plt
X, _ = make_blobs(n_samples=1000, centers=2, random_state=0)
km = KMeans(n_clusters=5, random_state=0).fit(X)
bisect_km = BisectingKMeans(n_clusters=5, random_state=0).fit(X)
fig, ax = plt.subplots(1, 2, figsize=(10, 5))
ax[0].scatter(X[:, 0], X[:, 1], s=10, c=km.labels_)
ax[0].scatter(km.cluster_centers_[:, 0], km.cluster_centers_[:, 1], s=20, c="r")
ax[0].set_title("KMeans")
ax[1].scatter(X[:, 0], X[:, 1], s=10, c=bisect_km.labels_)
ax[1].scatter(
bisect_km.cluster_centers_[:, 0], bisect_km.cluster_centers_[:, 1], s=20, c="r"
)
_ = ax[1].set_title("BisectingKMeans")
.. image-sg:: /auto_examples/release_highlights/images/sphx_glr_plot_release_highlights_1_1_0_003.png
:alt: KMeans, BisectingKMeans
:srcset: /auto_examples/release_highlights/images/sphx_glr_plot_release_highlights_1_1_0_003.png
:class: sphx-glr-single-img
.. rst-class:: sphx-glr-timing
**Total running time of the script:** ( 0 minutes 9.704 seconds)
.. _sphx_glr_download_auto_examples_release_highlights_plot_release_highlights_1_1_0.py:
.. only:: html
.. container:: sphx-glr-footer sphx-glr-footer-example
.. container:: binder-badge
.. image:: images/binder_badge_logo.svg
:target: https://mybinder.org/v2/gh/scikit-learn/scikit-learn/1.1.X?urlpath=lab/tree/notebooks/auto_examples/release_highlights/plot_release_highlights_1_1_0.ipynb
:alt: Launch binder
:width: 150 px
.. container:: sphx-glr-download sphx-glr-download-python
:download:`Download Python source code: plot_release_highlights_1_1_0.py <plot_release_highlights_1_1_0.py>`
.. container:: sphx-glr-download sphx-glr-download-jupyter
:download:`Download Jupyter notebook: plot_release_highlights_1_1_0.ipynb <plot_release_highlights_1_1_0.ipynb>`
.. only:: html
.. rst-class:: sphx-glr-signature
`Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_