.. note:: :class: sphx-glr-download-link-note Click :ref:`here ` to download the full example code .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_examples_model_selection_plot_randomized_search.py: ========================================================================= Comparing randomized search and grid search for hyperparameter estimation ========================================================================= Compare randomized search and grid search for optimizing hyperparameters of a random forest. All parameters that influence the learning are searched simultaneously (except for the number of estimators, which poses a time / quality tradeoff). The randomized search and the grid search explore exactly the same space of parameters. The result in parameter settings is quite similar, while the run time for randomized search is drastically lower. The performance is slightly worse for the randomized search, though this is most likely a noise effect and would not carry over to a held-out test set. Note that in practice, one would not search over this many different parameters simultaneously using grid search, but pick only the ones deemed most important. .. rst-class:: sphx-glr-script-out Out: .. code-block:: none RandomizedSearchCV took 4.42 seconds for 20 candidates parameter settings. Model with rank: 1 Mean validation score: 0.939 (std: 0.024) Parameters: {'bootstrap': False, 'criterion': 'entropy', 'max_depth': None, 'max_features': 7, 'min_samples_split': 3} Model with rank: 2 Mean validation score: 0.933 (std: 0.022) Parameters: {'bootstrap': False, 'criterion': 'gini', 'max_depth': None, 'max_features': 6, 'min_samples_split': 6} Model with rank: 3 Mean validation score: 0.930 (std: 0.031) Parameters: {'bootstrap': True, 'criterion': 'gini', 'max_depth': None, 'max_features': 6, 'min_samples_split': 6} GridSearchCV took 13.24 seconds for 72 candidate parameter settings. Model with rank: 1 Mean validation score: 0.937 (std: 0.019) Parameters: {'bootstrap': False, 'criterion': 'entropy', 'max_depth': None, 'max_features': 10, 'min_samples_split': 2} Model with rank: 2 Mean validation score: 0.936 (std: 0.020) Parameters: {'bootstrap': False, 'criterion': 'gini', 'max_depth': None, 'max_features': 10, 'min_samples_split': 2} Model with rank: 3 Mean validation score: 0.931 (std: 0.029) Parameters: {'bootstrap': False, 'criterion': 'entropy', 'max_depth': None, 'max_features': 10, 'min_samples_split': 3} | .. code-block:: python print(__doc__) import numpy as np from time import time from scipy.stats import randint as sp_randint from sklearn.model_selection import GridSearchCV from sklearn.model_selection import RandomizedSearchCV from sklearn.datasets import load_digits from sklearn.ensemble import RandomForestClassifier # get some data digits = load_digits() X, y = digits.data, digits.target # build a classifier clf = RandomForestClassifier(n_estimators=20) # Utility function to report best scores def report(results, n_top=3): for i in range(1, n_top + 1): candidates = np.flatnonzero(results['rank_test_score'] == i) for candidate in candidates: print("Model with rank: {0}".format(i)) print("Mean validation score: {0:.3f} (std: {1:.3f})".format( results['mean_test_score'][candidate], results['std_test_score'][candidate])) print("Parameters: {0}".format(results['params'][candidate])) print("") # specify parameters and distributions to sample from param_dist = {"max_depth": [3, None], "max_features": sp_randint(1, 11), "min_samples_split": sp_randint(2, 11), "bootstrap": [True, False], "criterion": ["gini", "entropy"]} # run randomized search n_iter_search = 20 random_search = RandomizedSearchCV(clf, param_distributions=param_dist, n_iter=n_iter_search, cv=5, iid=False) start = time() random_search.fit(X, y) print("RandomizedSearchCV took %.2f seconds for %d candidates" " parameter settings." % ((time() - start), n_iter_search)) report(random_search.cv_results_) # use a full grid over all parameters param_grid = {"max_depth": [3, None], "max_features": [1, 3, 10], "min_samples_split": [2, 3, 10], "bootstrap": [True, False], "criterion": ["gini", "entropy"]} # run grid search grid_search = GridSearchCV(clf, param_grid=param_grid, cv=5, iid=False) start = time() grid_search.fit(X, y) print("GridSearchCV took %.2f seconds for %d candidate parameter settings." % (time() - start, len(grid_search.cv_results_['params']))) report(grid_search.cv_results_) **Total running time of the script:** ( 0 minutes 17.712 seconds) .. _sphx_glr_download_auto_examples_model_selection_plot_randomized_search.py: .. only :: html .. container:: sphx-glr-footer :class: sphx-glr-footer-example .. container:: sphx-glr-download :download:`Download Python source code: plot_randomized_search.py ` .. container:: sphx-glr-download :download:`Download Jupyter notebook: plot_randomized_search.ipynb ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_