Selecting dimensionality reduction with Pipeline and GridSearchCV

This example constructs a pipeline that does dimensionality reduction followed by prediction with a support vector classifier. It demonstrates the use of GridSearchCV and Pipeline to optimize over different classes of estimators in a single CV run – unsupervised PCA and NMF dimensionality reductions are compared to univariate feature selection during the grid search.

Additionally, Pipeline can be instantiated with the memory argument to memoize the transformers within the pipeline, avoiding to fit again the same transformers over and over.

Note that the use of memory to enable caching becomes interesting when the fitting of a transformer is costly.

Illustration of Pipeline and GridSearchCV

This section illustrates the use of a Pipeline with GridSearchCV
# Authors: Robert McGibbon, Joel Nothman, Guillaume Lemaitre

from __future__ import print_function, division

import numpy as np
import matplotlib.pyplot as plt
from sklearn.datasets import load_digits
from sklearn.model_selection import GridSearchCV
from sklearn.pipeline import Pipeline
from sklearn.svm import LinearSVC
from sklearn.decomposition import PCA, NMF
from sklearn.feature_selection import SelectKBest, chi2

print(__doc__)

pipe = Pipeline([
    ('reduce_dim', PCA()),
    ('classify', LinearSVC())
])

N_FEATURES_OPTIONS = [2, 4, 8]
C_OPTIONS = [1, 10, 100, 1000]
param_grid = [
    {
        'reduce_dim': [PCA(iterated_power=7), NMF()],
        'reduce_dim__n_components': N_FEATURES_OPTIONS,
        'classify__C': C_OPTIONS
    },
    {
        'reduce_dim': [SelectKBest(chi2)],
        'reduce_dim__k': N_FEATURES_OPTIONS,
        'classify__C': C_OPTIONS
    },
]
reducer_labels = ['PCA', 'NMF', 'KBest(chi2)']

grid = GridSearchCV(pipe, cv=3, n_jobs=1, param_grid=param_grid)
digits = load_digits()
grid.fit(digits.data, digits.target)

mean_scores = np.array(grid.cv_results_['mean_test_score'])
# scores are in the order of param_grid iteration, which is alphabetical
mean_scores = mean_scores.reshape(len(C_OPTIONS), -1, len(N_FEATURES_OPTIONS))
# select score for best C
mean_scores = mean_scores.max(axis=0)
bar_offsets = (np.arange(len(N_FEATURES_OPTIONS)) *
               (len(reducer_labels) + 1) + .5)

plt.figure()
COLORS = 'bgrcmyk'
for i, (label, reducer_scores) in enumerate(zip(reducer_labels, mean_scores)):
    plt.bar(bar_offsets + i, reducer_scores, label=label, color=COLORS[i])

plt.title("Comparing feature reduction techniques")
plt.xlabel('Reduced number of features')
plt.xticks(bar_offsets + len(reducer_labels) / 2, N_FEATURES_OPTIONS)
plt.ylabel('Digit classification accuracy')
plt.ylim((0, 1))
plt.legend(loc='upper left')
../_images/sphx_glr_plot_compare_reduction_001.png

Caching transformers within a Pipeline

It is sometimes worthwhile storing the state of a specific transformer since it could be used again. Using a pipeline in GridSearchCV triggers such situations. Therefore, we use the argument memory to enable caching.

Warning

Note that this example is, however, only an illustration since for this specific case fitting PCA is not necessarily slower than loading the cache. Hence, use the memory constructor parameter when the fitting of a transformer is costly.

from tempfile import mkdtemp
from shutil import rmtree
from sklearn.externals.joblib import Memory

# Create a temporary folder to store the transformers of the pipeline
cachedir = mkdtemp()
memory = Memory(cachedir=cachedir, verbose=10)
cached_pipe = Pipeline([('reduce_dim', PCA()),
                        ('classify', LinearSVC())],
                       memory=memory)

# This time, a cached pipeline will be used within the grid search
grid = GridSearchCV(cached_pipe, cv=3, n_jobs=1, param_grid=param_grid)
digits = load_digits()
grid.fit(digits.data, digits.target)

# Delete the temporary cache before exiting
rmtree(cachedir)

Out:

________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(PCA(copy=True, iterated_power=7, n_components=2, random_state=None,
  svd_solver='auto', tol=0.0, whiten=False),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(PCA(copy=True, iterated_power=7, n_components=4, random_state=None,
  svd_solver='auto', tol=0.0, whiten=False),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(PCA(copy=True, iterated_power=7, n_components=8, random_state=None,
  svd_solver='auto', tol=0.0, whiten=False),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(NMF(alpha=0.0, beta_loss='frobenius', init=None, l1_ratio=0.0, max_iter=200,
  n_components=2, random_state=None, shuffle=False, solver='cd',
  tol=0.0001, verbose=0),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.1s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(NMF(alpha=0.0, beta_loss='frobenius', init=None, l1_ratio=0.0, max_iter=200,
  n_components=4, random_state=None, shuffle=False, solver='cd',
  tol=0.0001, verbose=0),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.1s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(NMF(alpha=0.0, beta_loss='frobenius', init=None, l1_ratio=0.0, max_iter=200,
  n_components=8, random_state=None, shuffle=False, solver='cd',
  tol=0.0001, verbose=0),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.1s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/36f12c43349c7e6acab6ec381c224bce
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/ece915a6c6cba022fb711000ca1f1de8
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/c6cd6af44c46b6f83e312b4972d196c5
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/723b2346bf1613882d5a9d4df36834bc
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/9a006efe750da1fd731a9e6efb3af1c8
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/424a9093916ba8fbc9222849ee892720
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/36f12c43349c7e6acab6ec381c224bce
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/ece915a6c6cba022fb711000ca1f1de8
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/c6cd6af44c46b6f83e312b4972d196c5
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/723b2346bf1613882d5a9d4df36834bc
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/9a006efe750da1fd731a9e6efb3af1c8
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/424a9093916ba8fbc9222849ee892720
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/36f12c43349c7e6acab6ec381c224bce
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/ece915a6c6cba022fb711000ca1f1de8
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/c6cd6af44c46b6f83e312b4972d196c5
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/723b2346bf1613882d5a9d4df36834bc
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/9a006efe750da1fd731a9e6efb3af1c8
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/424a9093916ba8fbc9222849ee892720
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(SelectKBest(k=2, score_func=<function chi2 at 0x2b5e37fb38c8>), None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(SelectKBest(k=4, score_func=<function chi2 at 0x2b5e37fb38c8>), None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(SelectKBest(k=8, score_func=<function chi2 at 0x2b5e37fb38c8>), None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/866186639a40f74c0b70211394ea5aae
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/9e2fc008ca73eb521bf4bab30bc93b56
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/74003b170217804d6767d1946aaab4be
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/866186639a40f74c0b70211394ea5aae
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/9e2fc008ca73eb521bf4bab30bc93b56
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/74003b170217804d6767d1946aaab4be
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/866186639a40f74c0b70211394ea5aae
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/9e2fc008ca73eb521bf4bab30bc93b56
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/74003b170217804d6767d1946aaab4be
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(PCA(copy=True, iterated_power=7, n_components=2, random_state=None,
  svd_solver='auto', tol=0.0, whiten=False),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(PCA(copy=True, iterated_power=7, n_components=4, random_state=None,
  svd_solver='auto', tol=0.0, whiten=False),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(PCA(copy=True, iterated_power=7, n_components=8, random_state=None,
  svd_solver='auto', tol=0.0, whiten=False),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(NMF(alpha=0.0, beta_loss='frobenius', init=None, l1_ratio=0.0, max_iter=200,
  n_components=2, random_state=None, shuffle=False, solver='cd',
  tol=0.0001, verbose=0),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.1s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(NMF(alpha=0.0, beta_loss='frobenius', init=None, l1_ratio=0.0, max_iter=200,
  n_components=4, random_state=None, shuffle=False, solver='cd',
  tol=0.0001, verbose=0),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.1s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(NMF(alpha=0.0, beta_loss='frobenius', init=None, l1_ratio=0.0, max_iter=200,
  n_components=8, random_state=None, shuffle=False, solver='cd',
  tol=0.0001, verbose=0),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.1s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/d6fc2602640fc58b7407ce32d9eadab6
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/555970dead65b89480dbf8f56009fde1
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/2405f44dbd13199ec1eee178ea1e4233
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/80ab15c6464b9a1f30ed0ece0d9163ba
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/22a19ad7c42c33d0afc0449ae86c23e3
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/bdc4280674376de094ad6ab71895f62e
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/d6fc2602640fc58b7407ce32d9eadab6
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/555970dead65b89480dbf8f56009fde1
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/2405f44dbd13199ec1eee178ea1e4233
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/80ab15c6464b9a1f30ed0ece0d9163ba
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/22a19ad7c42c33d0afc0449ae86c23e3
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/bdc4280674376de094ad6ab71895f62e
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/d6fc2602640fc58b7407ce32d9eadab6
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/555970dead65b89480dbf8f56009fde1
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/2405f44dbd13199ec1eee178ea1e4233
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/80ab15c6464b9a1f30ed0ece0d9163ba
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/22a19ad7c42c33d0afc0449ae86c23e3
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/bdc4280674376de094ad6ab71895f62e
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(SelectKBest(k=2, score_func=<function chi2 at 0x2b5e37fb38c8>), None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(SelectKBest(k=4, score_func=<function chi2 at 0x2b5e37fb38c8>), None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(SelectKBest(k=8, score_func=<function chi2 at 0x2b5e37fb38c8>), None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/042d1095de98a19dc24504dc1e356d2a
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/2b609708440c996b13c4b48d581149aa
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/5ed11cc04b8f0528b8b1a647b30ed9ed
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/042d1095de98a19dc24504dc1e356d2a
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/2b609708440c996b13c4b48d581149aa
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/5ed11cc04b8f0528b8b1a647b30ed9ed
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/042d1095de98a19dc24504dc1e356d2a
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/2b609708440c996b13c4b48d581149aa
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/5ed11cc04b8f0528b8b1a647b30ed9ed
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(PCA(copy=True, iterated_power=7, n_components=2, random_state=None,
  svd_solver='auto', tol=0.0, whiten=False),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 4]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(PCA(copy=True, iterated_power=7, n_components=4, random_state=None,
  svd_solver='auto', tol=0.0, whiten=False),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 4]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(PCA(copy=True, iterated_power=7, n_components=8, random_state=None,
  svd_solver='auto', tol=0.0, whiten=False),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 4]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(NMF(alpha=0.0, beta_loss='frobenius', init=None, l1_ratio=0.0, max_iter=200,
  n_components=2, random_state=None, shuffle=False, solver='cd',
  tol=0.0001, verbose=0),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 4]))
________________________________________________fit_transform_one - 0.1s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(NMF(alpha=0.0, beta_loss='frobenius', init=None, l1_ratio=0.0, max_iter=200,
  n_components=4, random_state=None, shuffle=False, solver='cd',
  tol=0.0001, verbose=0),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 4]))
________________________________________________fit_transform_one - 0.1s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(NMF(alpha=0.0, beta_loss='frobenius', init=None, l1_ratio=0.0, max_iter=200,
  n_components=8, random_state=None, shuffle=False, solver='cd',
  tol=0.0001, verbose=0),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 4]))
________________________________________________fit_transform_one - 0.1s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/e734bdda3be2d347d55502dbeaa1ae67
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/392b25026a63273cb294c2a0ade6d4b0
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/ebcb31d1c68e778f3a8867ce9c49b3e9
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/803bb30e4325516d64c509ff3520383c
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/0e3dc0f70bbe0bca53ad24ccde309edc
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/b927cd627c234b6373eabbce03b95f1e
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/e734bdda3be2d347d55502dbeaa1ae67
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/392b25026a63273cb294c2a0ade6d4b0
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/ebcb31d1c68e778f3a8867ce9c49b3e9
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/803bb30e4325516d64c509ff3520383c
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/0e3dc0f70bbe0bca53ad24ccde309edc
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/b927cd627c234b6373eabbce03b95f1e
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/e734bdda3be2d347d55502dbeaa1ae67
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/392b25026a63273cb294c2a0ade6d4b0
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/ebcb31d1c68e778f3a8867ce9c49b3e9
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/803bb30e4325516d64c509ff3520383c
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/0e3dc0f70bbe0bca53ad24ccde309edc
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/b927cd627c234b6373eabbce03b95f1e
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(SelectKBest(k=2, score_func=<function chi2 at 0x2b5e37fb38c8>), None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 4]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(SelectKBest(k=4, score_func=<function chi2 at 0x2b5e37fb38c8>), None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 4]))
________________________________________________fit_transform_one - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(SelectKBest(k=8, score_func=<function chi2 at 0x2b5e37fb38c8>), None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 4]))
________________________________________________fit_transform_one - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/37220c8140570316db54b6379fb47487
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/80838a1026a06cd5a7013df01faf16fd
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/5f96e5724bf43670e637bf5c39504073
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/37220c8140570316db54b6379fb47487
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/80838a1026a06cd5a7013df01faf16fd
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/5f96e5724bf43670e637bf5c39504073
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/37220c8140570316db54b6379fb47487
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/80838a1026a06cd5a7013df01faf16fd
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
[Memory]    0.0s, 0.0min: Loading _fit_transform_one from /tmp/tmpaahm3uwv/joblib/sklearn/pipeline/_fit_transform_one/5f96e5724bf43670e637bf5c39504073
___________________________________fit_transform_one cache loaded - 0.0s, 0.0min
________________________________________________________________________________
[Memory] Calling sklearn.pipeline._fit_transform_one...
_fit_transform_one(PCA(copy=True, iterated_power=7, n_components=8, random_state=None,
  svd_solver='auto', tol=0.0, whiten=False),
None, array([[ 0., ...,  0.],
       ...,
       [ 0., ...,  0.]]), array([0, ..., 8]))
________________________________________________fit_transform_one - 0.0s, 0.0min

The PCA fitting is only computed at the evaluation of the first configuration of the C parameter of the LinearSVC classifier. The other configurations of C will trigger the loading of the cached PCA estimator data, leading to save processing time. Therefore, the use of caching the pipeline using memory is highly beneficial when fitting a transformer is costly.

Total running time of the script: ( 1 minutes 27.869 seconds)

Generated by Sphinx-Gallery