.. _example_applications_plot_out_of_core_classification.py:
======================================================
Out-of-core classification of text documents
======================================================
This is an example showing how scikit-learn can be used for classification
using an out-of-core approach: learning from data that doesn't fit into main
memory. We make use of an online classifier, i.e., one that supports the
partial_fit method, that will be fed with batches of examples. To guarantee
that the features space remains the same over time we leverage a
HashingVectorizer that will project each example into the same feature space.
This is especially useful in the case of text classification where new
features (words) may appear in each batch.
The dataset used in this example is Reuters-21578 as provided by the UCI ML
repository. It will be automatically downloaded and uncompressed on first run.
The plot represents the learning curve of the classifier: the evolution
of classification accuracy over the course of the mini-batches. Accuracy is
measured on the first 1000 samples, held out as a validation set.
To limit the memory consumption, we queue examples up to a fixed amount before
feeding them to the learner.
.. rst-class:: horizontal
*
.. image:: images/plot_out_of_core_classification_001.png
:scale: 47
*
.. image:: images/plot_out_of_core_classification_002.png
:scale: 47
*
.. image:: images/plot_out_of_core_classification_003.png
:scale: 47
*
.. image:: images/plot_out_of_core_classification_004.png
:scale: 47
**Script output**::
Test set is 469 documents (49 positive)
Passive-Aggressive classifier : 870 train docs ( 58 positive) 469 test docs ( 49 positive) accuracy: 0.932 in 1.45s ( 599 docs/s)
Perceptron classifier : 870 train docs ( 58 positive) 469 test docs ( 49 positive) accuracy: 0.925 in 1.45s ( 598 docs/s)
SGD classifier : 870 train docs ( 58 positive) 469 test docs ( 49 positive) accuracy: 0.930 in 1.46s ( 597 docs/s)
NB Multinomial classifier : 870 train docs ( 58 positive) 469 test docs ( 49 positive) accuracy: 0.896 in 1.49s ( 584 docs/s)
Passive-Aggressive classifier : 3643 train docs ( 461 positive) 469 test docs ( 49 positive) accuracy: 0.945 in 4.10s ( 888 docs/s)
Perceptron classifier : 3643 train docs ( 461 positive) 469 test docs ( 49 positive) accuracy: 0.953 in 4.10s ( 887 docs/s)
SGD classifier : 3643 train docs ( 461 positive) 469 test docs ( 49 positive) accuracy: 0.949 in 4.11s ( 886 docs/s)
NB Multinomial classifier : 3643 train docs ( 461 positive) 469 test docs ( 49 positive) accuracy: 0.908 in 4.14s ( 880 docs/s)
Passive-Aggressive classifier : 6571 train docs ( 829 positive) 469 test docs ( 49 positive) accuracy: 0.979 in 6.80s ( 965 docs/s)
Perceptron classifier : 6571 train docs ( 829 positive) 469 test docs ( 49 positive) accuracy: 0.953 in 6.81s ( 965 docs/s)
SGD classifier : 6571 train docs ( 829 positive) 469 test docs ( 49 positive) accuracy: 0.951 in 6.81s ( 964 docs/s)
NB Multinomial classifier : 6571 train docs ( 829 positive) 469 test docs ( 49 positive) accuracy: 0.915 in 6.84s ( 960 docs/s)
Passive-Aggressive classifier : 9481 train docs ( 1176 positive) 469 test docs ( 49 positive) accuracy: 0.968 in 9.55s ( 992 docs/s)
Perceptron classifier : 9481 train docs ( 1176 positive) 469 test docs ( 49 positive) accuracy: 0.957 in 9.56s ( 992 docs/s)
SGD classifier : 9481 train docs ( 1176 positive) 469 test docs ( 49 positive) accuracy: 0.964 in 9.56s ( 991 docs/s)
NB Multinomial classifier : 9481 train docs ( 1176 positive) 469 test docs ( 49 positive) accuracy: 0.928 in 9.59s ( 988 docs/s)
Passive-Aggressive classifier : 11856 train docs ( 1454 positive) 469 test docs ( 49 positive) accuracy: 0.957 in 12.06s ( 983 docs/s)
Perceptron classifier : 11856 train docs ( 1454 positive) 469 test docs ( 49 positive) accuracy: 0.949 in 12.06s ( 982 docs/s)
SGD classifier : 11856 train docs ( 1454 positive) 469 test docs ( 49 positive) accuracy: 0.959 in 12.07s ( 982 docs/s)
NB Multinomial classifier : 11856 train docs ( 1454 positive) 469 test docs ( 49 positive) accuracy: 0.930 in 12.10s ( 979 docs/s)
Passive-Aggressive classifier : 14769 train docs ( 1827 positive) 469 test docs ( 49 positive) accuracy: 0.953 in 14.73s ( 1002 docs/s)
Perceptron classifier : 14769 train docs ( 1827 positive) 469 test docs ( 49 positive) accuracy: 0.945 in 14.73s ( 1002 docs/s)
SGD classifier : 14769 train docs ( 1827 positive) 469 test docs ( 49 positive) accuracy: 0.964 in 14.73s ( 1002 docs/s)
NB Multinomial classifier : 14769 train docs ( 1827 positive) 469 test docs ( 49 positive) accuracy: 0.945 in 14.76s ( 1000 docs/s)
Passive-Aggressive classifier : 17700 train docs ( 2188 positive) 469 test docs ( 49 positive) accuracy: 0.962 in 17.49s ( 1011 docs/s)
Perceptron classifier : 17700 train docs ( 2188 positive) 469 test docs ( 49 positive) accuracy: 0.966 in 17.50s ( 1011 docs/s)
SGD classifier : 17700 train docs ( 2188 positive) 469 test docs ( 49 positive) accuracy: 0.959 in 17.50s ( 1011 docs/s)
NB Multinomial classifier : 17700 train docs ( 2188 positive) 469 test docs ( 49 positive) accuracy: 0.947 in 17.53s ( 1009 docs/s)
**Python source code:** :download:`plot_out_of_core_classification.py <plot_out_of_core_classification.py>`
.. literalinclude:: plot_out_of_core_classification.py
:lines: 25-
**Total running time of the example:** 19.10 seconds
( 0 minutes 19.10 seconds)