Selecting dimensionality reduction with Pipeline and GridSearchCV

This example constructs a pipeline that does dimensionality reduction followed by prediction with a support vector classifier. It demonstrates the use of GridSearchCV and Pipeline to optimize over different classes of estimators in a single CV run – unsupervised PCA and NMF dimensionality reductions are compared to univariate feature selection during the grid search.

Additionally, Pipeline can be instantiated with the memory argument to memoize the transformers within the pipeline, avoiding to fit again the same transformers over and over.

Note that the use of memory to enable caching becomes interesting when the fitting of a transformer is costly.

Illustration of Pipeline and GridSearchCV

This section illustrates the use of a Pipeline with GridSearchCV
# Authors: Robert McGibbon, Joel Nothman, Guillaume Lemaitre

from __future__ import print_function, division

import numpy as np
import matplotlib.pyplot as plt
from sklearn.datasets import load_digits
from sklearn.model_selection import GridSearchCV
from sklearn.pipeline import Pipeline
from sklearn.svm import LinearSVC
from sklearn.decomposition import PCA, NMF
from sklearn.feature_selection import SelectKBest, chi2

print(__doc__)

pipe = Pipeline([
    ('reduce_dim', PCA()),
    ('classify', LinearSVC())
])

N_FEATURES_OPTIONS = [2, 4, 8]
C_OPTIONS = [1, 10, 100, 1000]
param_grid = [
    {
        'reduce_dim': [PCA(iterated_power=7), NMF()],
        'reduce_dim__n_components': N_FEATURES_OPTIONS,
        'classify__C': C_OPTIONS
    },
    {
        'reduce_dim': [SelectKBest(chi2)],
        'reduce_dim__k': N_FEATURES_OPTIONS,
        'classify__C': C_OPTIONS
    },
]
reducer_labels = ['PCA', 'NMF', 'KBest(chi2)']

grid = GridSearchCV(pipe, cv=3, n_jobs=1, param_grid=param_grid)
digits = load_digits()
grid.fit(digits.data, digits.target)

mean_scores = np.array(grid.cv_results_['mean_test_score'])
# scores are in the order of param_grid iteration, which is alphabetical
mean_scores = mean_scores.reshape(len(C_OPTIONS), -1, len(N_FEATURES_OPTIONS))
# select score for best C
mean_scores = mean_scores.max(axis=0)
bar_offsets = (np.arange(len(N_FEATURES_OPTIONS)) *
               (len(reducer_labels) + 1) + .5)

plt.figure()
COLORS = 'bgrcmyk'
for i, (label, reducer_scores) in enumerate(zip(reducer_labels, mean_scores)):
    plt.bar(bar_offsets + i, reducer_scores, label=label, color=COLORS[i])

plt.title("Comparing feature reduction techniques")
plt.xlabel('Reduced number of features')
plt.xticks(bar_offsets + len(reducer_labels) / 2, N_FEATURES_OPTIONS)
plt.ylabel('Digit classification accuracy')
plt.ylim((0, 1))
plt.legend(loc='upper left')
../_images/sphx_glr_plot_compare_reduction_001.png

Caching transformers within a Pipeline

It is sometimes worthwhile storing the state of a specific transformer since it could be used again. Using a pipeline in GridSearchCV triggers such situations. Therefore, we use the argument memory to enable caching.

Warning

Note that this example is, however, only an illustration since for this specific case fitting PCA is not necessarily slower than loading the cache. Hence, use the memory constructor parameter when the fitting of a transformer is costly.

from tempfile import mkdtemp
from shutil import rmtree
from sklearn.externals.joblib import Memory

# Create a temporary folder to store the transformers of the pipeline
cachedir = mkdtemp()
memory = Memory(cachedir=cachedir, verbose=10)
cached_pipe = Pipeline([('reduce_dim', PCA()),
                        ('classify', LinearSVC())],
                       memory=memory)

# This time, a cached pipeline will be used within the grid search
grid = GridSearchCV(cached_pipe, cv=3, n_jobs=1, param_grid=param_grid)
digits = load_digits()
grid.fit(digits.data, digits.target)

# Delete the temporary cache before exiting
rmtree(cachedir)

Out:

________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(PCA(copy=True, iterated_power=7, n_components=2, random_state=None,
  svd_solver='auto', tol=0.0, whiten=False),
None, array([[0., ..., 0.],
       ...,
       [0., ..., 0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(PCA(copy=True, iterated_power=7, n_components=2, random_state=None,
  svd_solver='auto', tol=0.0, whiten=False),
None, array([[0., ..., 0.],
       ...,
       [0., ..., 0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(PCA(copy=True, iterated_power=7, n_components=2, random_state=None,
  svd_solver='auto', tol=0.0, whiten=False),
None, array([[0., ..., 0.],
       ...,
       [0., ..., 0.]]), array([0, ..., 4]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(PCA(copy=True, iterated_power=7, n_components=4, random_state=None,
  svd_solver='auto', tol=0.0, whiten=False),
None, array([[0., ..., 0.],
       ...,
       [0., ..., 0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(PCA(copy=True, iterated_power=7, n_components=4, random_state=None,
  svd_solver='auto', tol=0.0, whiten=False),
None, array([[0., ..., 0.],
       ...,
       [0., ..., 0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(PCA(copy=True, iterated_power=7, n_components=4, random_state=None,
  svd_solver='auto', tol=0.0, whiten=False),
None, array([[0., ..., 0.],
       ...,
       [0., ..., 0.]]), array([0, ..., 4]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(PCA(copy=True, iterated_power=7, n_components=8, random_state=None,
  svd_solver='auto', tol=0.0, whiten=False),
None, array([[0., ..., 0.],
       ...,
       [0., ..., 0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(PCA(copy=True, iterated_power=7, n_components=8, random_state=None,
  svd_solver='auto', tol=0.0, whiten=False),
None, array([[0., ..., 0.],
       ...,
       [0., ..., 0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(PCA(copy=True, iterated_power=7, n_components=8, random_state=None,
  svd_solver='auto', tol=0.0, whiten=False),
None, array([[0., ..., 0.],
       ...,
       [0., ..., 0.]]), array([0, ..., 4]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(NMF(alpha=0.0, beta_loss='frobenius', init=None, l1_ratio=0.0, max_iter=200,
  n_components=2, random_state=None, shuffle=False, solver='cd',
  tol=0.0001, verbose=0),
None, array([[0., ..., 0.],
       ...,
       [0., ..., 0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.1s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(NMF(alpha=0.0, beta_loss='frobenius', init=None, l1_ratio=0.0, max_iter=200,
  n_components=2, random_state=None, shuffle=False, solver='cd',
  tol=0.0001, verbose=0),
None, array([[0., ..., 0.],
       ...,
       [0., ..., 0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.1s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(NMF(alpha=0.0, beta_loss='frobenius', init=None, l1_ratio=0.0, max_iter=200,
  n_components=2, random_state=None, shuffle=False, solver='cd',
  tol=0.0001, verbose=0),
None, array([[0., ..., 0.],
       ...,
       [0., ..., 0.]]), array([0, ..., 4]))
________________________________________________fit_transform_one - 0.1s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(NMF(alpha=0.0, beta_loss='frobenius', init=None, l1_ratio=0.0, max_iter=200,
  n_components=4, random_state=None, shuffle=False, solver='cd',
  tol=0.0001, verbose=0),
None, array([[0., ..., 0.],
       ...,
       [0., ..., 0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.1s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(NMF(alpha=0.0, beta_loss='frobenius', init=None, l1_ratio=0.0, max_iter=200,
  n_components=4, random_state=None, shuffle=False, solver='cd',
  tol=0.0001, verbose=0),
None, array([[0., ..., 0.],
       ...,
       [0., ..., 0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.1s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(NMF(alpha=0.0, beta_loss='frobenius', init=None, l1_ratio=0.0, max_iter=200,
  n_components=4, random_state=None, shuffle=False, solver='cd',
  tol=0.0001, verbose=0),
None, array([[0., ..., 0.],
       ...,
       [0., ..., 0.]]), array([0, ..., 4]))
________________________________________________fit_transform_one - 0.1s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(NMF(alpha=0.0, beta_loss='frobenius', init=None, l1_ratio=0.0, max_iter=200,
  n_components=8, random_state=None, shuffle=False, solver='cd',
  tol=0.0001, verbose=0),
None, array([[0., ..., 0.],
       ...,
       [0., ..., 0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.2s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(NMF(alpha=0.0, beta_loss='frobenius', init=None, l1_ratio=0.0, max_iter=200,
  n_components=8, random_state=None, shuffle=False, solver='cd',
  tol=0.0001, verbose=0),
None, array([[0., ..., 0.],
       ...,
       [0., ..., 0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.1s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(NMF(alpha=0.0, beta_loss='frobenius', init=None, l1_ratio=0.0, max_iter=200,
  n_components=8, random_state=None, shuffle=False, solver='cd',
  tol=0.0001, verbose=0),
None, array([[0., ..., 0.],
       ...,
       [0., ..., 0.]]), array([0, ..., 4]))
________________________________________________fit_transform_one - 0.1s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/6c9faf767180734537ee1460d687aa7e
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/fcd112fbe1836f85a32975165f6137db
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/571f5a1ed076993db04791fc833234d1
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/ae28f7e0b3597cf142e9d86a07e61d94
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/ff1dd0fc986140704ba90a99d238ff4c
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/5da0220bd504aa094c1184a66c83da04
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/586840f33912a6396ecdf756b2392e5c
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/d366660351b7d8ae30f884c98b91a404
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/7aaf51de31c01b79de9d29cb573cb570
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/4e5019335626f2a34b72f19ef3a94309
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/3dc6c0877e1e3599d87ca0f9083fc3cd
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/2763795884a037217f3b847e4d273878
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/77092ef32d818647bde9f3c80442a311
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/fad793302163ebf78158c2663bdee8a6
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/641667166004b7daf56bb2e1d7d07897
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/3bdacb7a1fdfec97d03052b520a53fda
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/72aa21c9e89f9c39cdde34a6eb18c4a5
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/33b991b23bfb8a4517dd51f338152ad9
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/6c9faf767180734537ee1460d687aa7e
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/fcd112fbe1836f85a32975165f6137db
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/571f5a1ed076993db04791fc833234d1
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/ae28f7e0b3597cf142e9d86a07e61d94
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/ff1dd0fc986140704ba90a99d238ff4c
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/5da0220bd504aa094c1184a66c83da04
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/586840f33912a6396ecdf756b2392e5c
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/d366660351b7d8ae30f884c98b91a404
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/7aaf51de31c01b79de9d29cb573cb570
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/4e5019335626f2a34b72f19ef3a94309
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/3dc6c0877e1e3599d87ca0f9083fc3cd
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/2763795884a037217f3b847e4d273878
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/77092ef32d818647bde9f3c80442a311
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/fad793302163ebf78158c2663bdee8a6
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/641667166004b7daf56bb2e1d7d07897
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/3bdacb7a1fdfec97d03052b520a53fda
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/72aa21c9e89f9c39cdde34a6eb18c4a5
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/33b991b23bfb8a4517dd51f338152ad9
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/6c9faf767180734537ee1460d687aa7e
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/fcd112fbe1836f85a32975165f6137db
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/571f5a1ed076993db04791fc833234d1
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/ae28f7e0b3597cf142e9d86a07e61d94
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/ff1dd0fc986140704ba90a99d238ff4c
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/5da0220bd504aa094c1184a66c83da04
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/586840f33912a6396ecdf756b2392e5c
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/d366660351b7d8ae30f884c98b91a404
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/7aaf51de31c01b79de9d29cb573cb570
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/4e5019335626f2a34b72f19ef3a94309
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/3dc6c0877e1e3599d87ca0f9083fc3cd
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/2763795884a037217f3b847e4d273878
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/77092ef32d818647bde9f3c80442a311
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/fad793302163ebf78158c2663bdee8a6
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/641667166004b7daf56bb2e1d7d07897
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/3bdacb7a1fdfec97d03052b520a53fda
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/72aa21c9e89f9c39cdde34a6eb18c4a5
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/33b991b23bfb8a4517dd51f338152ad9
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(SelectKBest(k=2, score_func=<function chi2 at 0x2b60c0903048>), None, array([[0., ..., 0.],
       ...,
       [0., ..., 0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(SelectKBest(k=2, score_func=<function chi2 at 0x2b60c0903048>), None, array([[0., ..., 0.],
       ...,
       [0., ..., 0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(SelectKBest(k=2, score_func=<function chi2 at 0x2b60c0903048>), None, array([[0., ..., 0.],
       ...,
       [0., ..., 0.]]), array([0, ..., 4]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(SelectKBest(k=4, score_func=<function chi2 at 0x2b60c0903048>), None, array([[0., ..., 0.],
       ...,
       [0., ..., 0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(SelectKBest(k=4, score_func=<function chi2 at 0x2b60c0903048>), None, array([[0., ..., 0.],
       ...,
       [0., ..., 0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(SelectKBest(k=4, score_func=<function chi2 at 0x2b60c0903048>), None, array([[0., ..., 0.],
       ...,
       [0., ..., 0.]]), array([0, ..., 4]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(SelectKBest(k=8, score_func=<function chi2 at 0x2b60c0903048>), None, array([[0., ..., 0.],
       ...,
       [0., ..., 0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(SelectKBest(k=8, score_func=<function chi2 at 0x2b60c0903048>), None, array([[0., ..., 0.],
       ...,
       [0., ..., 0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(SelectKBest(k=8, score_func=<function chi2 at 0x2b60c0903048>), None, array([[0., ..., 0.],
       ...,
       [0., ..., 0.]]), array([0, ..., 4]))
________________________________________________fit_transform_one - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/0c50184f7388eac6bdf390382f6148ab
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/b03168ef893036c56cec0c61e0f19053
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/3c428c47ad448999e93928024d512bb0
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/78cb432d6f087bf113dcf0b85abfe254
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/8de24a51226ce233d56b097ca8580a8b
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/0b9959b87f4003912026684a1f5d467d
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/fe2eddfc8beaf1174496078d2d423341
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/61bdce54b69043ef8c334ba66b85fd83
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/9fc00199cd62065fceaf38d2908d7811
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/0c50184f7388eac6bdf390382f6148ab
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/b03168ef893036c56cec0c61e0f19053
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/3c428c47ad448999e93928024d512bb0
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/78cb432d6f087bf113dcf0b85abfe254
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/8de24a51226ce233d56b097ca8580a8b
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/0b9959b87f4003912026684a1f5d467d
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/fe2eddfc8beaf1174496078d2d423341
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/61bdce54b69043ef8c334ba66b85fd83
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/9fc00199cd62065fceaf38d2908d7811
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/0c50184f7388eac6bdf390382f6148ab
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/b03168ef893036c56cec0c61e0f19053
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/3c428c47ad448999e93928024d512bb0
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/78cb432d6f087bf113dcf0b85abfe254
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/8de24a51226ce233d56b097ca8580a8b
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/0b9959b87f4003912026684a1f5d467d
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/fe2eddfc8beaf1174496078d2d423341
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/61bdce54b69043ef8c334ba66b85fd83
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmp7alkiyzr/joblib/sklearn/pipeline/_fit_transform_one/9fc00199cd62065fceaf38d2908d7811
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(NMF(alpha=0.0, beta_loss='frobenius', init=None, l1_ratio=0.0, max_iter=200,
  n_components=8, random_state=None, shuffle=False, solver='cd',
  tol=0.0001, verbose=0),
None, array([[0., ..., 0.],
       ...,
       [0., ..., 0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.2s, 0.0min

The PCA fitting is only computed at the evaluation of the first configuration of the C parameter of the LinearSVC classifier. The other configurations of C will trigger the loading of the cached PCA estimator data, leading to save processing time. Therefore, the use of caching the pipeline using memory is highly beneficial when fitting a transformer is costly.

plt.show()

Total running time of the script: ( 1 minutes 28.673 seconds)

Download Python source code: plot_compare_reduction.py
Download Jupyter notebook: plot_compare_reduction.ipynb

Generated by Sphinx-Gallery