.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "auto_examples/svm/plot_svm_anova.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code or to run this example in your browser via JupyterLite or Binder .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_examples_svm_plot_svm_anova.py: ================================================= SVM-Anova: SVM with univariate feature selection ================================================= This example shows how to perform univariate feature selection before running a SVC (support vector classifier) to improve the classification scores. We use the iris dataset (4 features) and add 36 non-informative features. We can find that our model achieves best performance when we select around 10% of features. .. GENERATED FROM PYTHON SOURCE LINES 14-16 Load some data to play with --------------------------- .. GENERATED FROM PYTHON SOURCE LINES 16-26 .. code-block:: Python import numpy as np from sklearn.datasets import load_iris X, y = load_iris(return_X_y=True) # Add non-informative features rng = np.random.RandomState(0) X = np.hstack((X, 2 * rng.random((X.shape[0], 36)))) .. GENERATED FROM PYTHON SOURCE LINES 27-29 Create the pipeline ------------------- .. GENERATED FROM PYTHON SOURCE LINES 29-45 .. code-block:: Python from sklearn.feature_selection import SelectPercentile, f_classif from sklearn.pipeline import Pipeline from sklearn.preprocessing import StandardScaler from sklearn.svm import SVC # Create a feature-selection transform, a scaler and an instance of SVM that we # combine together to have a full-blown estimator clf = Pipeline( [ ("anova", SelectPercentile(f_classif)), ("scaler", StandardScaler()), ("svc", SVC(gamma="auto")), ] ) .. GENERATED FROM PYTHON SOURCE LINES 46-48 Plot the cross-validation score as a function of percentile of features ----------------------------------------------------------------------- .. GENERATED FROM PYTHON SOURCE LINES 48-69 .. code-block:: Python import matplotlib.pyplot as plt from sklearn.model_selection import cross_val_score score_means = list() score_stds = list() percentiles = (1, 3, 6, 10, 15, 20, 30, 40, 60, 80, 100) for percentile in percentiles: clf.set_params(anova__percentile=percentile) this_scores = cross_val_score(clf, X, y) score_means.append(this_scores.mean()) score_stds.append(this_scores.std()) plt.errorbar(percentiles, score_means, np.array(score_stds)) plt.title("Performance of the SVM-Anova varying the percentile of features selected") plt.xticks(np.linspace(0, 100, 11, endpoint=True)) plt.xlabel("Percentile") plt.ylabel("Accuracy Score") plt.axis("tight") plt.show() .. image-sg:: /auto_examples/svm/images/sphx_glr_plot_svm_anova_001.png :alt: Performance of the SVM-Anova varying the percentile of features selected :srcset: /auto_examples/svm/images/sphx_glr_plot_svm_anova_001.png :class: sphx-glr-single-img .. rst-class:: sphx-glr-timing **Total running time of the script:** (0 minutes 0.300 seconds) .. _sphx_glr_download_auto_examples_svm_plot_svm_anova.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: binder-badge .. image:: images/binder_badge_logo.svg :target: https://mybinder.org/v2/gh/scikit-learn/scikit-learn/main?urlpath=lab/tree/notebooks/auto_examples/svm/plot_svm_anova.ipynb :alt: Launch binder :width: 150 px .. container:: lite-badge .. image:: images/jupyterlite_badge_logo.svg :target: ../../lite/lab/?path=auto_examples/svm/plot_svm_anova.ipynb :alt: Launch JupyterLite :width: 150 px .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: plot_svm_anova.ipynb ` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: plot_svm_anova.py ` .. include:: plot_svm_anova.recommendations .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_