.. include:: _contributors.rst
.. currentmodule:: sklearn
============
Version 0.16
============
.. _changes_0_16_1:
Version 0.16.1
===============
**April 14, 2015**
Changelog
---------
Bug fixes
.........
- Allow input data larger than ``block_size`` in
:class:`covariance.LedoitWolf` by `Andreas Müller`_.
- Fix a bug in :class:`isotonic.IsotonicRegression` deduplication that
caused unstable result in :class:`calibration.CalibratedClassifierCV` by
`Jan Hendrik Metzen`_.
- Fix sorting of labels in func:`preprocessing.label_binarize` by Michael Heilman.
- Fix several stability and convergence issues in
:class:`cross_decomposition.CCA` and
:class:`cross_decomposition.PLSCanonical` by `Andreas Müller`_
- Fix a bug in :class:`cluster.KMeans` when ``precompute_distances=False``
on fortran-ordered data.
- Fix a speed regression in :class:`ensemble.RandomForestClassifier`'s ``predict``
and ``predict_proba`` by `Andreas Müller`_.
- Fix a regression where ``utils.shuffle`` converted lists and dataframes to arrays, by `Olivier Grisel`_
.. _changes_0_16:
Version 0.16
============
**March 26, 2015**
Highlights
-----------
- Speed improvements (notably in :class:`cluster.DBSCAN`), reduced memory
requirements, bug-fixes and better default settings.
- Multinomial Logistic regression and a path algorithm in
:class:`linear_model.LogisticRegressionCV`.
- Out-of core learning of PCA via :class:`decomposition.IncrementalPCA`.
- Probability calibration of classifiers using
:class:`calibration.CalibratedClassifierCV`.
- :class:`cluster.Birch` clustering method for large-scale datasets.
- Scalable approximate nearest neighbors search with Locality-sensitive
hashing forests in `neighbors.LSHForest`.
- Improved error messages and better validation when using malformed input data.
- More robust integration with pandas dataframes.
Changelog
---------
New features
............
- The new `neighbors.LSHForest` implements locality-sensitive hashing
for approximate nearest neighbors search. By :user:`Maheshakya Wijewardena`.
- Added :class:`svm.LinearSVR`. This class uses the liblinear implementation
of Support Vector Regression which is much faster for large
sample sizes than :class:`svm.SVR` with linear kernel. By
`Fabian Pedregosa`_ and Qiang Luo.
- Incremental fit for :class:`GaussianNB `.
- Added ``sample_weight`` support to :class:`dummy.DummyClassifier` and
:class:`dummy.DummyRegressor`. By `Arnaud Joly`_.
- Added the :func:`metrics.label_ranking_average_precision_score` metrics.
By `Arnaud Joly`_.
- Add the :func:`metrics.coverage_error` metrics. By `Arnaud Joly`_.
- Added :class:`linear_model.LogisticRegressionCV`. By
`Manoj Kumar`_, `Fabian Pedregosa`_, `Gael Varoquaux`_
and `Alexandre Gramfort`_.
- Added ``warm_start`` constructor parameter to make it possible for any
trained forest model to grow additional trees incrementally. By
:user:`Laurent Direr`.
- Added ``sample_weight`` support to :class:`ensemble.GradientBoostingClassifier` and
:class:`ensemble.GradientBoostingRegressor`. By `Peter Prettenhofer`_.
- Added :class:`decomposition.IncrementalPCA`, an implementation of the PCA
algorithm that supports out-of-core learning with a ``partial_fit``
method. By `Kyle Kastner`_.
- Averaged SGD for :class:`SGDClassifier `
and :class:`SGDRegressor ` By
:user:`Danny Sullivan `.
- Added `cross_val_predict`
function which computes cross-validated estimates. By `Luis Pedro Coelho`_
- Added :class:`linear_model.TheilSenRegressor`, a robust
generalized-median-based estimator. By :user:`Florian Wilhelm `.
- Added :func:`metrics.median_absolute_error`, a robust metric.
By `Gael Varoquaux`_ and :user:`Florian Wilhelm `.
- Add :class:`cluster.Birch`, an online clustering algorithm. By
`Manoj Kumar`_, `Alexandre Gramfort`_ and `Joel Nothman`_.
- Added shrinkage support to :class:`discriminant_analysis.LinearDiscriminantAnalysis`
using two new solvers. By :user:`Clemens Brunner ` and `Martin Billinger`_.
- Added :class:`kernel_ridge.KernelRidge`, an implementation of
kernelized ridge regression.
By `Mathieu Blondel`_ and `Jan Hendrik Metzen`_.
- All solvers in :class:`linear_model.Ridge` now support `sample_weight`.
By `Mathieu Blondel`_.
- Added `cross_validation.PredefinedSplit` cross-validation
for fixed user-provided cross-validation folds.
By :user:`Thomas Unterthiner `.
- Added :class:`calibration.CalibratedClassifierCV`, an approach for
calibrating the predicted probabilities of a classifier.
By `Alexandre Gramfort`_, `Jan Hendrik Metzen`_, `Mathieu Blondel`_
and :user:`Balazs Kegl `.
Enhancements
............
- Add option ``return_distance`` in `hierarchical.ward_tree`
to return distances between nodes for both structured and unstructured
versions of the algorithm. By `Matteo Visconti di Oleggio Castello`_.
The same option was added in `hierarchical.linkage_tree`.
By `Manoj Kumar`_
- Add support for sample weights in scorer objects. Metrics with sample
weight support will automatically benefit from it. By `Noel Dawe`_ and
`Vlad Niculae`_.
- Added ``newton-cg`` and `lbfgs` solver support in
:class:`linear_model.LogisticRegression`. By `Manoj Kumar`_.
- Add ``selection="random"`` parameter to implement stochastic coordinate
descent for :class:`linear_model.Lasso`, :class:`linear_model.ElasticNet`
and related. By `Manoj Kumar`_.
- Add ``sample_weight`` parameter to
`metrics.jaccard_similarity_score` and :func:`metrics.log_loss`.
By :user:`Jatin Shah `.
- Support sparse multilabel indicator representation in
:class:`preprocessing.LabelBinarizer` and
:class:`multiclass.OneVsRestClassifier` (by :user:`Hamzeh Alsalhi ` with thanks
to Rohit Sivaprasad), as well as evaluation metrics (by
`Joel Nothman`_).
- Add ``sample_weight`` parameter to `metrics.jaccard_similarity_score`.
By `Jatin Shah`.
- Add support for multiclass in `metrics.hinge_loss`. Added ``labels=None``
as optional parameter. By `Saurabh Jha`.
- Add ``sample_weight`` parameter to `metrics.hinge_loss`.
By `Saurabh Jha`.
- Add ``multi_class="multinomial"`` option in
:class:`linear_model.LogisticRegression` to implement a Logistic
Regression solver that minimizes the cross-entropy or multinomial loss
instead of the default One-vs-Rest setting. Supports `lbfgs` and
`newton-cg` solvers. By `Lars Buitinck`_ and `Manoj Kumar`_. Solver option
`newton-cg` by Simon Wu.
- ``DictVectorizer`` can now perform ``fit_transform`` on an iterable in a
single pass, when giving the option ``sort=False``. By :user:`Dan
Blanchard `.
- :class:`model_selection.GridSearchCV` and
:class:`model_selection.RandomizedSearchCV` can now be configured to work
with estimators that may fail and raise errors on individual folds. This
option is controlled by the `error_score` parameter. This does not affect
errors raised on re-fit. By :user:`Michal Romaniuk `.
- Add ``digits`` parameter to `metrics.classification_report` to allow
report to show different precision of floating point numbers. By
:user:`Ian Gilmore `.
- Add a quantile prediction strategy to the :class:`dummy.DummyRegressor`.
By :user:`Aaron Staple `.
- Add ``handle_unknown`` option to :class:`preprocessing.OneHotEncoder` to
handle unknown categorical features more gracefully during transform.
By `Manoj Kumar`_.
- Added support for sparse input data to decision trees and their ensembles.
By `Fares Hedyati`_ and `Arnaud Joly`_.
- Optimized :class:`cluster.AffinityPropagation` by reducing the number of
memory allocations of large temporary data-structures. By `Antony Lee`_.
- Parellization of the computation of feature importances in random forest.
By `Olivier Grisel`_ and `Arnaud Joly`_.
- Add ``n_iter_`` attribute to estimators that accept a ``max_iter`` attribute
in their constructor. By `Manoj Kumar`_.
- Added decision function for :class:`multiclass.OneVsOneClassifier`
By `Raghav RV`_ and :user:`Kyle Beauchamp `.
- `neighbors.kneighbors_graph` and `radius_neighbors_graph`
support non-Euclidean metrics. By `Manoj Kumar`_
- Parameter ``connectivity`` in :class:`cluster.AgglomerativeClustering`
and family now accept callables that return a connectivity matrix.
By `Manoj Kumar`_.
- Sparse support for :func:`metrics.pairwise.paired_distances`. By `Joel Nothman`_.
- :class:`cluster.DBSCAN` now supports sparse input and sample weights and
has been optimized: the inner loop has been rewritten in Cython and
radius neighbors queries are now computed in batch. By `Joel Nothman`_
and `Lars Buitinck`_.
- Add ``class_weight`` parameter to automatically weight samples by class
frequency for :class:`ensemble.RandomForestClassifier`,
:class:`tree.DecisionTreeClassifier`, :class:`ensemble.ExtraTreesClassifier`
and :class:`tree.ExtraTreeClassifier`. By `Trevor Stephens`_.
- `grid_search.RandomizedSearchCV` now does sampling without
replacement if all parameters are given as lists. By `Andreas Müller`_.
- Parallelized calculation of :func:`metrics.pairwise_distances` is now supported
for scipy metrics and custom callables. By `Joel Nothman`_.
- Allow the fitting and scoring of all clustering algorithms in
:class:`pipeline.Pipeline`. By `Andreas Müller`_.
- More robust seeding and improved error messages in :class:`cluster.MeanShift`
by `Andreas Müller`_.
- Make the stopping criterion for `mixture.GMM`,
`mixture.DPGMM` and `mixture.VBGMM` less dependent on the
number of samples by thresholding the average log-likelihood change
instead of its sum over all samples. By `Hervé Bredin`_.
- The outcome of :func:`manifold.spectral_embedding` was made deterministic
by flipping the sign of eigenvectors. By :user:`Hasil Sharma `.
- Significant performance and memory usage improvements in
:class:`preprocessing.PolynomialFeatures`. By `Eric Martin`_.
- Numerical stability improvements for :class:`preprocessing.StandardScaler`
and :func:`preprocessing.scale`. By `Nicolas Goix`_
- :class:`svm.SVC` fitted on sparse input now implements ``decision_function``.
By `Rob Zinkov`_ and `Andreas Müller`_.
- `cross_validation.train_test_split` now preserves the input type,
instead of converting to numpy arrays.
Documentation improvements
..........................
- Added example of using :class:`pipeline.FeatureUnion` for heterogeneous input.
By :user:`Matt Terry `
- Documentation on scorers was improved, to highlight the handling of loss
functions. By :user:`Matt Pico `.
- A discrepancy between liblinear output and scikit-learn's wrappers
is now noted. By `Manoj Kumar`_.
- Improved documentation generation: examples referring to a class or
function are now shown in a gallery on the class/function's API reference
page. By `Joel Nothman`_.
- More explicit documentation of sample generators and of data
transformation. By `Joel Nothman`_.
- :class:`sklearn.neighbors.BallTree` and :class:`sklearn.neighbors.KDTree`
used to point to empty pages stating that they are aliases of BinaryTree.
This has been fixed to show the correct class docs. By `Manoj Kumar`_.
- Added silhouette plots for analysis of KMeans clustering using
:func:`metrics.silhouette_samples` and :func:`metrics.silhouette_score`.
See :ref:`sphx_glr_auto_examples_cluster_plot_kmeans_silhouette_analysis.py`
Bug fixes
.........
- Metaestimators now support ducktyping for the presence of ``decision_function``,
``predict_proba`` and other methods. This fixes behavior of
`grid_search.GridSearchCV`,
`grid_search.RandomizedSearchCV`, :class:`pipeline.Pipeline`,
:class:`feature_selection.RFE`, :class:`feature_selection.RFECV` when nested.
By `Joel Nothman`_
- The ``scoring`` attribute of grid-search and cross-validation methods is no longer
ignored when a `grid_search.GridSearchCV` is given as a base estimator or
the base estimator doesn't have predict.
- The function `hierarchical.ward_tree` now returns the children in
the same order for both the structured and unstructured versions. By
`Matteo Visconti di Oleggio Castello`_.
- :class:`feature_selection.RFECV` now correctly handles cases when
``step`` is not equal to 1. By :user:`Nikolay Mayorov `
- The :class:`decomposition.PCA` now undoes whitening in its
``inverse_transform``. Also, its ``components_`` now always have unit
length. By :user:`Michael Eickenberg `.
- Fix incomplete download of the dataset when
`datasets.download_20newsgroups` is called. By `Manoj Kumar`_.
- Various fixes to the Gaussian processes subpackage by Vincent Dubourg
and Jan Hendrik Metzen.
- Calling ``partial_fit`` with ``class_weight=='auto'`` throws an
appropriate error message and suggests a work around.
By :user:`Danny Sullivan `.
- :class:`RBFSampler ` with ``gamma=g``
formerly approximated :func:`rbf_kernel `
with ``gamma=g/2.``; the definition of ``gamma`` is now consistent,
which may substantially change your results if you use a fixed value.
(If you cross-validated over ``gamma``, it probably doesn't matter
too much.) By :user:`Dougal Sutherland `.
- Pipeline object delegate the ``classes_`` attribute to the underlying
estimator. It allows, for instance, to make bagging of a pipeline object.
By `Arnaud Joly`_
- :class:`neighbors.NearestCentroid` now uses the median as the centroid
when metric is set to ``manhattan``. It was using the mean before.
By `Manoj Kumar`_
- Fix numerical stability issues in :class:`linear_model.SGDClassifier`
and :class:`linear_model.SGDRegressor` by clipping large gradients and
ensuring that weight decay rescaling is always positive (for large
l2 regularization and large learning rate values).
By `Olivier Grisel`_
- When `compute_full_tree` is set to "auto", the full tree is
built when n_clusters is high and is early stopped when n_clusters is
low, while the behavior should be vice-versa in
:class:`cluster.AgglomerativeClustering` (and friends).
This has been fixed By `Manoj Kumar`_
- Fix lazy centering of data in :func:`linear_model.enet_path` and
:func:`linear_model.lasso_path`. It was centered around one. It has
been changed to be centered around the origin. By `Manoj Kumar`_
- Fix handling of precomputed affinity matrices in
:class:`cluster.AgglomerativeClustering` when using connectivity
constraints. By :user:`Cathy Deng `
- Correct ``partial_fit`` handling of ``class_prior`` for
:class:`sklearn.naive_bayes.MultinomialNB` and
:class:`sklearn.naive_bayes.BernoulliNB`. By `Trevor Stephens`_.
- Fixed a crash in :func:`metrics.precision_recall_fscore_support`
when using unsorted ``labels`` in the multi-label setting.
By `Andreas Müller`_.
- Avoid skipping the first nearest neighbor in the methods ``radius_neighbors``,
``kneighbors``, ``kneighbors_graph`` and ``radius_neighbors_graph`` in
:class:`sklearn.neighbors.NearestNeighbors` and family, when the query
data is not the same as fit data. By `Manoj Kumar`_.
- Fix log-density calculation in the `mixture.GMM` with
tied covariance. By `Will Dawson`_
- Fixed a scaling error in :class:`feature_selection.SelectFdr`
where a factor ``n_features`` was missing. By `Andrew Tulloch`_
- Fix zero division in :class:`neighbors.KNeighborsRegressor` and related
classes when using distance weighting and having identical data points.
By `Garret-R `_.
- Fixed round off errors with non positive-definite covariance matrices
in GMM. By :user:`Alexis Mignon `.
- Fixed a error in the computation of conditional probabilities in
:class:`naive_bayes.BernoulliNB`. By `Hanna Wallach`_.
- Make the method ``radius_neighbors`` of
:class:`neighbors.NearestNeighbors` return the samples lying on the
boundary for ``algorithm='brute'``. By `Yan Yi`_.
- Flip sign of ``dual_coef_`` of :class:`svm.SVC`
to make it consistent with the documentation and
``decision_function``. By Artem Sobolev.
- Fixed handling of ties in :class:`isotonic.IsotonicRegression`.
We now use the weighted average of targets (secondary method). By
`Andreas Müller`_ and `Michael Bommarito `_.
API changes summary
-------------------
- `GridSearchCV` and
`cross_val_score` and other
meta-estimators don't convert pandas DataFrames into arrays any more,
allowing DataFrame specific operations in custom estimators.
- `multiclass.fit_ovr`, `multiclass.predict_ovr`,
`predict_proba_ovr`,
`multiclass.fit_ovo`, `multiclass.predict_ovo`,
`multiclass.fit_ecoc` and `multiclass.predict_ecoc`
are deprecated. Use the underlying estimators instead.
- Nearest neighbors estimators used to take arbitrary keyword arguments
and pass these to their distance metric. This will no longer be supported
in scikit-learn 0.18; use the ``metric_params`` argument instead.
- `n_jobs` parameter of the fit method shifted to the constructor of the
LinearRegression class.
- The ``predict_proba`` method of :class:`multiclass.OneVsRestClassifier`
now returns two probabilities per sample in the multiclass case; this
is consistent with other estimators and with the method's documentation,
but previous versions accidentally returned only the positive
probability. Fixed by Will Lamond and `Lars Buitinck`_.
- Change default value of precompute in :class:`linear_model.ElasticNet` and
:class:`linear_model.Lasso` to False. Setting precompute to "auto" was found
to be slower when n_samples > n_features since the computation of the Gram
matrix is computationally expensive and outweighs the benefit of fitting the
Gram for just one alpha.
``precompute="auto"`` is now deprecated and will be removed in 0.18
By `Manoj Kumar`_.
- Expose ``positive`` option in :func:`linear_model.enet_path` and
:func:`linear_model.enet_path` which constrains coefficients to be
positive. By `Manoj Kumar`_.
- Users should now supply an explicit ``average`` parameter to
:func:`sklearn.metrics.f1_score`, :func:`sklearn.metrics.fbeta_score`,
:func:`sklearn.metrics.recall_score` and
:func:`sklearn.metrics.precision_score` when performing multiclass
or multilabel (i.e. not binary) classification. By `Joel Nothman`_.
- `scoring` parameter for cross validation now accepts `'f1_micro'`,
`'f1_macro'` or `'f1_weighted'`. `'f1'` is now for binary classification
only. Similar changes apply to `'precision'` and `'recall'`.
By `Joel Nothman`_.
- The ``fit_intercept``, ``normalize`` and ``return_models`` parameters in
:func:`linear_model.enet_path` and :func:`linear_model.lasso_path` have
been removed. They were deprecated since 0.14
- From now onwards, all estimators will uniformly raise ``NotFittedError``
when any of the ``predict`` like methods are called before the model is fit.
By `Raghav RV`_.
- Input data validation was refactored for more consistent input
validation. The ``check_arrays`` function was replaced by ``check_array``
and ``check_X_y``. By `Andreas Müller`_.
- Allow ``X=None`` in the methods ``radius_neighbors``, ``kneighbors``,
``kneighbors_graph`` and ``radius_neighbors_graph`` in
:class:`sklearn.neighbors.NearestNeighbors` and family. If set to None,
then for every sample this avoids setting the sample itself as the
first nearest neighbor. By `Manoj Kumar`_.
- Add parameter ``include_self`` in :func:`neighbors.kneighbors_graph`
and :func:`neighbors.radius_neighbors_graph` which has to be explicitly
set by the user. If set to True, then the sample itself is considered
as the first nearest neighbor.
- `thresh` parameter is deprecated in favor of new `tol` parameter in
`GMM`, `DPGMM` and `VBGMM`. See `Enhancements`
section for details. By `Hervé Bredin`_.
- Estimators will treat input with dtype object as numeric when possible.
By `Andreas Müller`_
- Estimators now raise `ValueError` consistently when fitted on empty
data (less than 1 sample or less than 1 feature for 2D input).
By `Olivier Grisel`_.
- The ``shuffle`` option of :class:`.linear_model.SGDClassifier`,
:class:`linear_model.SGDRegressor`, :class:`linear_model.Perceptron`,
:class:`linear_model.PassiveAggressiveClassifier` and
:class:`linear_model.PassiveAggressiveRegressor` now defaults to ``True``.
- :class:`cluster.DBSCAN` now uses a deterministic initialization. The
`random_state` parameter is deprecated. By :user:`Erich Schubert `.
Code Contributors
-----------------
A. Flaxman, Aaron Schumacher, Aaron Staple, abhishek thakur, Akshay, akshayah3,
Aldrian Obaja, Alexander Fabisch, Alexandre Gramfort, Alexis Mignon, Anders
Aagaard, Andreas Mueller, Andreas van Cranenburgh, Andrew Tulloch, Andrew
Walker, Antony Lee, Arnaud Joly, banilo, Barmaley.exe, Ben Davies, Benedikt
Koehler, bhsu, Boris Feld, Borja Ayerdi, Boyuan Deng, Brent Pedersen, Brian
Wignall, Brooke Osborn, Calvin Giles, Cathy Deng, Celeo, cgohlke, chebee7i,
Christian Stade-Schuldt, Christof Angermueller, Chyi-Kwei Yau, CJ Carey,
Clemens Brunner, Daiki Aminaka, Dan Blanchard, danfrankj, Danny Sullivan, David
Fletcher, Dmitrijs Milajevs, Dougal J. Sutherland, Erich Schubert, Fabian
Pedregosa, Florian Wilhelm, floydsoft, Félix-Antoine Fortin, Gael Varoquaux,
Garrett-R, Gilles Louppe, gpassino, gwulfs, Hampus Bengtsson, Hamzeh Alsalhi,
Hanna Wallach, Harry Mavroforakis, Hasil Sharma, Helder, Herve Bredin,
Hsiang-Fu Yu, Hugues SALAMIN, Ian Gilmore, Ilambharathi Kanniah, Imran Haque,
isms, Jake VanderPlas, Jan Dlabal, Jan Hendrik Metzen, Jatin Shah, Javier López
Peña, jdcaballero, Jean Kossaifi, Jeff Hammerbacher, Joel Nothman, Jonathan
Helmus, Joseph, Kaicheng Zhang, Kevin Markham, Kyle Beauchamp, Kyle Kastner,
Lagacherie Matthieu, Lars Buitinck, Laurent Direr, leepei, Loic Esteve, Luis
Pedro Coelho, Lukas Michelbacher, maheshakya, Manoj Kumar, Manuel, Mario
Michael Krell, Martin, Martin Billinger, Martin Ku, Mateusz Susik, Mathieu
Blondel, Matt Pico, Matt Terry, Matteo Visconti dOC, Matti Lyra, Max Linke,
Mehdi Cherti, Michael Bommarito, Michael Eickenberg, Michal Romaniuk, MLG,
mr.Shu, Nelle Varoquaux, Nicola Montecchio, Nicolas, Nikolay Mayorov, Noel
Dawe, Okal Billy, Olivier Grisel, Óscar Nájera, Paolo Puggioni, Peter
Prettenhofer, Pratap Vardhan, pvnguyen, queqichao, Rafael Carrascosa, Raghav R
V, Rahiel Kasim, Randall Mason, Rob Zinkov, Robert Bradshaw, Saket Choudhary,
Sam Nicholls, Samuel Charron, Saurabh Jha, sethdandridge, sinhrks, snuderl,
Stefan Otte, Stefan van der Walt, Steve Tjoa, swu, Sylvain Zimmer, tejesh95,
terrycojones, Thomas Delteil, Thomas Unterthiner, Tomas Kazmar, trevorstephens,
tttthomasssss, Tzu-Ming Kuo, ugurcaliskan, ugurthemaster, Vinayak Mehta,
Vincent Dubourg, Vjacheslav Murashkin, Vlad Niculae, wadawson, Wei Xue, Will
Lamond, Wu Jiang, x0l, Xinfan Meng, Yan Yi, Yu-Chin