Selecting dimensionality reduction with Pipeline and GridSearchCV

This example constructs a pipeline that does dimensionality reduction followed by prediction with a support vector classifier. It demonstrates the use of GridSearchCV and Pipeline to optimize over different classes of estimators in a single CV run – unsupervised PCA and NMF dimensionality reductions are compared to univariate feature selection during the grid search.

Additionally, Pipeline can be instantiated with the memory argument to memoize the transformers within the pipeline, avoiding to fit again the same transformers over and over.

Note that the use of memory to enable caching becomes interesting when the fitting of a transformer is costly.

Illustration of Pipeline and GridSearchCV

This section illustrates the use of a Pipeline with GridSearchCV
# Authors: Robert McGibbon, Joel Nothman, Guillaume Lemaitre

from __future__ import print_function, division

import numpy as np
import matplotlib.pyplot as plt
from sklearn.datasets import load_digits
from sklearn.model_selection import GridSearchCV
from sklearn.pipeline import Pipeline
from sklearn.svm import LinearSVC
from sklearn.decomposition import PCA, NMF
from sklearn.feature_selection import SelectKBest, chi2

print(__doc__)

pipe = Pipeline([
    ('reduce_dim', PCA()),
    ('classify', LinearSVC())
])

N_FEATURES_OPTIONS = [2, 4, 8]
C_OPTIONS = [1, 10, 100, 1000]
param_grid = [
    {
        'reduce_dim': [PCA(iterated_power=7), NMF()],
        'reduce_dim__n_components': N_FEATURES_OPTIONS,
        'classify__C': C_OPTIONS
    },
    {
        'reduce_dim': [SelectKBest(chi2)],
        'reduce_dim__k': N_FEATURES_OPTIONS,
        'classify__C': C_OPTIONS
    },
]
reducer_labels = ['PCA', 'NMF', 'KBest(chi2)']

grid = GridSearchCV(pipe, cv=3, n_jobs=1, param_grid=param_grid)
digits = load_digits()
grid.fit(digits.data, digits.target)

mean_scores = np.array(grid.cv_results_['mean_test_score'])
# scores are in the order of param_grid iteration, which is alphabetical
mean_scores = mean_scores.reshape(len(C_OPTIONS), -1, len(N_FEATURES_OPTIONS))
# select score for best C
mean_scores = mean_scores.max(axis=0)
bar_offsets = (np.arange(len(N_FEATURES_OPTIONS)) *
               (len(reducer_labels) + 1) + .5)

plt.figure()
COLORS = 'bgrcmyk'
for i, (label, reducer_scores) in enumerate(zip(reducer_labels, mean_scores)):
    plt.bar(bar_offsets + i, reducer_scores, label=label, color=COLORS[i])

plt.title("Comparing feature reduction techniques")
plt.xlabel('Reduced number of features')
plt.xticks(bar_offsets + len(reducer_labels) / 2, N_FEATURES_OPTIONS)
plt.ylabel('Digit classification accuracy')
plt.ylim((0, 1))
plt.legend(loc='upper left')
../_images/sphx_glr_plot_compare_reduction_001.png

Caching transformers within a Pipeline

It is sometimes worthwhile storing the state of a specific transformer since it could be used again. Using a pipeline in GridSearchCV triggers such situations. Therefore, we use the argument memory to enable caching.

Warning

Note that this example is, however, only an illustration since for this specific case fitting PCA is not necessarily slower than loading the cache. Hence, use the memory constructor parameter when the fitting of a transformer is costly.

from tempfile import mkdtemp
from shutil import rmtree
from sklearn.externals.joblib import Memory

# Create a temporary folder to store the transformers of the pipeline
cachedir = mkdtemp()
memory = Memory(cachedir=cachedir, verbose=10)
cached_pipe = Pipeline([('reduce_dim', PCA()),
                        ('classify', LinearSVC())],
                       memory=memory)

# This time, a cached pipeline will be used within the grid search
grid = GridSearchCV(cached_pipe, cv=3, n_jobs=1, param_grid=param_grid)
digits = load_digits()
grid.fit(digits.data, digits.target)

# Delete the temporary cache before exiting
rmtree(cachedir)

Out:

________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(PCA(copy=True, iterated_power=7, n_components=2, random_state=None,
  svd_solver='auto', tol=0.0, whiten=False),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(PCA(copy=True, iterated_power=7, n_components=4, random_state=None,
  svd_solver='auto', tol=0.0, whiten=False),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(PCA(copy=True, iterated_power=7, n_components=8, random_state=None,
  svd_solver='auto', tol=0.0, whiten=False),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(NMF(alpha=0.0, beta_loss='frobenius', init=None, l1_ratio=0.0, max_iter=200,
  n_components=2, random_state=None, shuffle=False, solver='cd',
  tol=0.0001, verbose=0),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.1s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(NMF(alpha=0.0, beta_loss='frobenius', init=None, l1_ratio=0.0, max_iter=200,
  n_components=4, random_state=None, shuffle=False, solver='cd',
  tol=0.0001, verbose=0),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.1s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(NMF(alpha=0.0, beta_loss='frobenius', init=None, l1_ratio=0.0, max_iter=200,
  n_components=8, random_state=None, shuffle=False, solver='cd',
  tol=0.0001, verbose=0),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.1s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/1fa6ce67e7ead177fcbbca3fd7393951
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/ee719987452026f2eda33f23ae27db6d
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/e644fdede9912483f06b4b8e7fef522d
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/8fb3b7e6738d0769a7eab8ced6c15933
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/04b5cf068b88b9be247f622b6b813d32
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/c990dd18ef970d8977345b3d6d253163
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/1fa6ce67e7ead177fcbbca3fd7393951
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/ee719987452026f2eda33f23ae27db6d
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/e644fdede9912483f06b4b8e7fef522d
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/8fb3b7e6738d0769a7eab8ced6c15933
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/04b5cf068b88b9be247f622b6b813d32
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/c990dd18ef970d8977345b3d6d253163
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/1fa6ce67e7ead177fcbbca3fd7393951
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/ee719987452026f2eda33f23ae27db6d
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/e644fdede9912483f06b4b8e7fef522d
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/8fb3b7e6738d0769a7eab8ced6c15933
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/04b5cf068b88b9be247f622b6b813d32
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/c990dd18ef970d8977345b3d6d253163
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(SelectKBest(k=2, score_func=<function chi2 at 0x2b067d5af500>), None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(SelectKBest(k=4, score_func=<function chi2 at 0x2b067d5af500>), None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(SelectKBest(k=8, score_func=<function chi2 at 0x2b067d5af500>), None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/964c532e4d62a55facd03dcf4d9c0fba
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/30f65a5375870ad29e6cfd819b187c7f
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/a269d75014a15618c43cebd118e34b03
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/964c532e4d62a55facd03dcf4d9c0fba
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/30f65a5375870ad29e6cfd819b187c7f
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/a269d75014a15618c43cebd118e34b03
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/964c532e4d62a55facd03dcf4d9c0fba
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/30f65a5375870ad29e6cfd819b187c7f
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/a269d75014a15618c43cebd118e34b03
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(PCA(copy=True, iterated_power=7, n_components=2, random_state=None,
  svd_solver='auto', tol=0.0, whiten=False),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(PCA(copy=True, iterated_power=7, n_components=4, random_state=None,
  svd_solver='auto', tol=0.0, whiten=False),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(PCA(copy=True, iterated_power=7, n_components=8, random_state=None,
  svd_solver='auto', tol=0.0, whiten=False),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(NMF(alpha=0.0, beta_loss='frobenius', init=None, l1_ratio=0.0, max_iter=200,
  n_components=2, random_state=None, shuffle=False, solver='cd',
  tol=0.0001, verbose=0),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.1s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(NMF(alpha=0.0, beta_loss='frobenius', init=None, l1_ratio=0.0, max_iter=200,
  n_components=4, random_state=None, shuffle=False, solver='cd',
  tol=0.0001, verbose=0),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.1s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(NMF(alpha=0.0, beta_loss='frobenius', init=None, l1_ratio=0.0, max_iter=200,
  n_components=8, random_state=None, shuffle=False, solver='cd',
  tol=0.0001, verbose=0),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.1s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/80126468ee4bbfb7fd859b4d9804bb7a
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/5693d4a94c9062f482ff6007aef40095
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/88438488322c6b7f21dc0cbd937d2f93
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/75d2886569f67a3fa97bf8a1742cb14f
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/598173f8986a3f3c122996ef2cbbaf3b
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/4a19240fe67de357ffb3b6c55b18efbe
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/80126468ee4bbfb7fd859b4d9804bb7a
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/5693d4a94c9062f482ff6007aef40095
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/88438488322c6b7f21dc0cbd937d2f93
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/75d2886569f67a3fa97bf8a1742cb14f
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/598173f8986a3f3c122996ef2cbbaf3b
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/4a19240fe67de357ffb3b6c55b18efbe
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/80126468ee4bbfb7fd859b4d9804bb7a
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/5693d4a94c9062f482ff6007aef40095
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/88438488322c6b7f21dc0cbd937d2f93
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/75d2886569f67a3fa97bf8a1742cb14f
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/598173f8986a3f3c122996ef2cbbaf3b
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/4a19240fe67de357ffb3b6c55b18efbe
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(SelectKBest(k=2, score_func=<function chi2 at 0x2b067d5af500>), None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(SelectKBest(k=4, score_func=<function chi2 at 0x2b067d5af500>), None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(SelectKBest(k=8, score_func=<function chi2 at 0x2b067d5af500>), None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/9595441045c6e7cf06f92877e936cfaf
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/be2d7e414bd90556e908271c543e7d8d
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/79ea7c6b98ded79e4b8d6b2600aaaf7d
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/9595441045c6e7cf06f92877e936cfaf
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/be2d7e414bd90556e908271c543e7d8d
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/79ea7c6b98ded79e4b8d6b2600aaaf7d
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/9595441045c6e7cf06f92877e936cfaf
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/be2d7e414bd90556e908271c543e7d8d
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/79ea7c6b98ded79e4b8d6b2600aaaf7d
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(PCA(copy=True, iterated_power=7, n_components=2, random_state=None,
  svd_solver='auto', tol=0.0, whiten=False),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 4]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(PCA(copy=True, iterated_power=7, n_components=4, random_state=None,
  svd_solver='auto', tol=0.0, whiten=False),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 4]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(PCA(copy=True, iterated_power=7, n_components=8, random_state=None,
  svd_solver='auto', tol=0.0, whiten=False),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 4]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(NMF(alpha=0.0, beta_loss='frobenius', init=None, l1_ratio=0.0, max_iter=200,
  n_components=2, random_state=None, shuffle=False, solver='cd',
  tol=0.0001, verbose=0),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 4]))
________________________________________________fit_transform_one - 0.1s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(NMF(alpha=0.0, beta_loss='frobenius', init=None, l1_ratio=0.0, max_iter=200,
  n_components=4, random_state=None, shuffle=False, solver='cd',
  tol=0.0001, verbose=0),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 4]))
________________________________________________fit_transform_one - 0.1s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(NMF(alpha=0.0, beta_loss='frobenius', init=None, l1_ratio=0.0, max_iter=200,
  n_components=8, random_state=None, shuffle=False, solver='cd',
  tol=0.0001, verbose=0),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 4]))
________________________________________________fit_transform_one - 0.1s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/b39dcd5bce7d92e1a2c571b75c1acb31
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/53399e7fc8f030209e71abcf2a333970
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/200325a81fbdb3638fb27ee6a14af861
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/dc00c4ebf1a2c70541564d11283bdbc1
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/3790741f5784d2c0121112e2de592056
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/a394ba0bf29680d208b7547a36206473
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/b39dcd5bce7d92e1a2c571b75c1acb31
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/53399e7fc8f030209e71abcf2a333970
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/200325a81fbdb3638fb27ee6a14af861
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/dc00c4ebf1a2c70541564d11283bdbc1
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/3790741f5784d2c0121112e2de592056
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/a394ba0bf29680d208b7547a36206473
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/b39dcd5bce7d92e1a2c571b75c1acb31
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/53399e7fc8f030209e71abcf2a333970
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/200325a81fbdb3638fb27ee6a14af861
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/dc00c4ebf1a2c70541564d11283bdbc1
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/3790741f5784d2c0121112e2de592056
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/a394ba0bf29680d208b7547a36206473
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(SelectKBest(k=2, score_func=<function chi2 at 0x2b067d5af500>), None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 4]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(SelectKBest(k=4, score_func=<function chi2 at 0x2b067d5af500>), None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 4]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(SelectKBest(k=8, score_func=<function chi2 at 0x2b067d5af500>), None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 4]))
________________________________________________fit_transform_one - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/0078c2eb101e712f69a22f86f2bc653e
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/a377a2718d2a67d5f308fd44db73c2f9
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/d4dbdb17eb5e9b06e61c400f624deef0
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/0078c2eb101e712f69a22f86f2bc653e
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/a377a2718d2a67d5f308fd44db73c2f9
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/d4dbdb17eb5e9b06e61c400f624deef0
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/0078c2eb101e712f69a22f86f2bc653e
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/a377a2718d2a67d5f308fd44db73c2f9
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpsNkz0D/joblib/sklearn/pipeline/_fit_transform_one/d4dbdb17eb5e9b06e61c400f624deef0
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(PCA(copy=True, iterated_power=7, n_components=8, random_state=None,
  svd_solver='auto', tol=0.0, whiten=False),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min

The PCA fitting is only computed at the evaluation of the first configuration of the C parameter of the LinearSVC classifier. The other configurations of C will trigger the loading of the cached PCA estimator data, leading to save processing time. Therefore, the use of caching the pipeline using memory is highly beneficial when fitting a transformer is costly.

Total running time of the script: ( 1 minutes 26.348 seconds)

Generated by Sphinx-Gallery