Version 1.0.0

In Development

Legend for changelogs

• Major Feature : something big that you couldn’t do before.

• Feature : something that you couldn’t do before.

• Efficiency : an existing feature now may not require as much computation or memory.

• Enhancement : a miscellaneous minor improvement.

• Fix : something that previously didn’t work as documentated – or according to reasonable expectations – should now work.

• API Change : you will need to change your code to have the same effect in the future; or a feature will be removed in the future.

Minimal dependencies

Version 1.0.0 of scikit-learn requires python 3.7+, numpy 1.14.6+ and scipy 1.1.0+. Optional minimal dependency is matplotlib 2.2.2+.

Enforcing keyword-only arguments

In an effort to promote clear and non-ambiguous use of the library, most constructor and function parameters must now be passed as keyword arguments (i.e. using the param=value syntax) instead of positional. If a keyword-only parameter is used as positional, a TypeError is now raised. #15005 #20002 by Joel Nothman, Adrin Jalali, Thomas Fan, Nicolas Hug, and Tom Dupre la Tour. See SLEP009 for more details.

Put the changes in their relevant module.

Changed models

The following estimators and functions, when fit with the same data and parameters, may produce different models from the previous version. This often occurs due to changes in the modelling logic (bug fixes or enhancements), or in random sampling procedures.

• Fix manifold.TSNE now avoids numerical underflow issues during affinity matrix computation.

• Fix The splitting criterion of tree.DecisionTreeClassifier and tree.DecisionTreeRegressor can be impacted by a fix in the handling of rounding errors. Previously some extra spurious splits could occur.

Details are listed in the changelog below.

(While we are trying to better inform users by providing this information, we cannot assure that this list is complete.)

Changelog

• API Change The option for using the squared error via loss and criterion parameters was made more consistent. The preferred way is by setting the value to "squared_error". Old option names are still valid, produce the same models, but are deprecated and will be removed in version 1.2. #19310 by Christian Lorentzen.

  • For ensemble.ExtraTreesRegressor, criterion="mse" is deprecated, use "squared_error" instead which is now the default.

  • For ensemble.GradientBoostingRegressor, loss="ls" is deprecated, use "squared_error" instead which is now the default.

  • For ensemble.RandomForestRegressor, criterion="mse" is deprecated, use "squared_error" instead which is now the default.

  • For ensemble.HistGradientBoostingRegressor, loss="least_squares" is deprecated, use "squared_error" instead which is now the default.

  • For linear_model.RANSACRegressor, loss="squared_loss" is deprecated, use "squared_error" instead.

  • For linear_model.SGDRegressor, loss="squared_loss" is deprecated, use "squared_error" instead which is now the default.

  • For tree.DecisionTreeRegressor, criterion="mse" is deprecated, use "squared_error" instead which is now the default.

  • For tree.ExtraTreeRegressor, criterion="mse" is deprecated, use "squared_error" instead which is now the default.

• API Change The option for using the absolute error via loss and criterion parameters was made more consistent. The preferred way is by setting the value to "absolute_error". Old option names are still valid, produce the same models, but are deprecated and will be removed in version 1.2. #19733 by Christian Lorentzen.

  • For ensemble.ExtraTreesRegressor, criterion="mae" is deprecated, use "absolute_error" instead.

  • For ensemble.GradientBoostingRegressor, loss="lad" is deprecated, use "absolute_error" instead.

  • For ensemble.RandomForestRegressor, criterion="mae" is deprecated, use "absolute_error" instead.

  • For ensemble.HistGradientBoostingRegressor, loss="least_absolute_deviation" is deprecated, use "absolute_error" instead.

  • For linear_model.RANSACRegressor, loss="absolute_loss" is deprecated, use "absolute_error" instead which is now the default.

  • For tree.DecisionTreeRegressor, criterion="mae" is deprecated, use "absolute_error" instead.

  • For tree.ExtraTreeRegressor, criterion="mae" is deprecated, use "absolute_error" instead.

• API Change np.matrix usage is deprecated in 1.0 and will raise a TypeError in 1.2. #20165 by Thomas Fan.

sklearn.base

• Fix config_context is now threadsafe. #18736 by Thomas Fan.

sklearn.calibration

• Fix The predict and predict_proba methods of calibration.CalibratedClassifierCV can now properly be used on prefitted pipelines. #19641 by Alek Lefebvre.

• Fix Fixed an error when using a :ensemble.VotingClassifier as base_estimator in :calibration.CalibratedClassifierCV. #20087 by Clément Fauchereau.

sklearn.cluster

• Efficiency The “k-means++” initialization of cluster.KMeans and cluster.MiniBatchKMeans is now faster, especially in multicore settings. #19002 by Jon Crall and Jérémie du Boisberranger.

• Efficiency cluster.KMeans with algorithm='elkan' is now faster in multicore settings. #19052 by Yusuke Nagasaka.

• Efficiency cluster.MiniBatchKMeans is now faster in multicore settings. #17622 by Jérémie du Boisberranger.

• Enhancement The predict and fit_predict methods of cluster.AffinityPropagation now accept sparse data type for input data. #20117 by Venkatachalam Natchiappan

• Fix Fixed a bug in cluster.MiniBatchKMeans where the sample weights were partially ignored when the input is sparse. #17622 by Jérémie du Boisberranger.

• Fix Improved convergence detection based on center change in cluster.MiniBatchKMeans which was almost never achievable. #17622 by Jérémie du Boisberranger.

• Fix cluster.AgglomerativeClustering now supports readonly memory-mapped datasets. #19883 by Julien Jerphanion.

• API Change cluster.Birch attributes, fit_ and partial_fit_, are deprecated and will be removed in 1.2. #19297 by Thomas Fan.

• API Change the default value for the batch_size parameter of MiniBatchKMeans was changed from 100 to 1024 due to efficiency reasons. The n_iter_ attribute of MiniBatchKMeans now reports the number of started epochs and the n_steps_ attribute reports the number of mini batches processed. #17622 by Jérémie du Boisberranger.

sklearn.compose

• Enhancement compose.ColumnTransformer now records the output of each transformer in output_indices_. #18393 by Luca Bittarello.

• Enhancement compose.ColumnTransformer now allows DataFrame input to have its columns appear in a changed order in transform. Further, columns that are dropped will not be required in transform, and additional columns will be ignored if remainder='drop'. #19263 by Thomas Fan

• Fix compose.ColumnTransformer.get_feature_names supports non-string feature names returned by any of its transformers. #18459 by Albert Villanova del Moral and Alonso Silva Allende.

• Fix compose.TransformedTargetRegressor now takes nD targets with an adequate transformer. #18898 by Oras Phongpanagnam.

sklearn.datasets

• Enhancement datasets.fetch_openml now supports categories with missing values when returning a pandas dataframe. #19365 by Thomas Fan and Amanda Dsouza and EL-ATEIF Sara.

• Enhancement datasets.fetch_kddcup99 raises a better message when the cached file is invalid. #19669 Thomas Fan.

• Fix Shorten data file names in the openml tests to better support installing on Windows and its default 260 character limit on file names. #20209 by Thomas Fan.

sklearn.decomposition

• API Change In decomposition.DictionaryLearning, decomposition.MiniBatchDictionaryLearning, dict_learning and dict_learning_online, transform_alpha will be equal to alpha instead of 1.0 by default starting from version 1.2 #19159 by Benoît Malézieux.

• Fix Fixes incorrect multiple data-conversion warnings when clustering boolean data. #19046 by Surya Prakash.

• Fix Fixed dict_learning, used by DictionaryLearning, to ensure determinism of the output. Achieved by flipping signs of the SVD output which is used to initialize the code. #18433 by Bruno Charron.

• Enhancement added a new approximate solver (randomized SVD, available with eigen_solver='randomized') to decomposition.KernelPCA. This significantly accelerates computation when the number of samples is much larger than the desired number of components. #12069 by Sylvain Marié.

• Fix Fixed a bug in MiniBatchDictionaryLearning, MiniBatchSparsePCA and dict_learning_online where the update of the dictionary was incorrect. #19198 by Jérémie du Boisberranger.

• Fix Fixed a bug in DictionaryLearning, SparsePCA, MiniBatchDictionaryLearning, MiniBatchSparsePCA, dict_learning and dict_learning_online where the restart of unused atoms during the dictionary update was not working as expected. #19198 by Jérémie du Boisberranger.

sklearn.ensemble

• Fix Do not allow to compute out-of-bag (OOB) score in ensemble.RandomForestClassifier and ensemble.ExtraTreesClassifier with multiclass-multioutput target since scikit-learn does not provide any metric supporting this type of target. Additional private refactoring was performed. #19162 by Guillaume Lemaitre.

• Enhancement HistGradientBoostingClassifier and HistGradientBoostingRegressor are no longer experimental. They are now considered stable and are subject to the same deprecation cycles as all other estimators. #19799 by Nicolas Hug.

• Enhancement Improve the HTML rendering of the ensemble.StackingClassifier and ensemble.StackingRegressor. #19564 by Thomas Fan.

• Enhancement Documented and tested support of the Poisson criterion for ensemble.RandomForestRegressor. #19836 by Brian Sun.

• Fix Fixed the range of the argument max_samples to be (0.0, 1.0] in ensemble.RandomForestClassifier, ensemble.RandomForestRegressor, where max_samples=1.0 is interpreted as using all n_samples for bootstrapping. #20159 by @murata-yu.

sklearn.feature_extraction

• Fix Fixed a bug in class:feature_extraction.HashingVectorizer where some input strings would result in negative indices in the transformed data. #19035 by Liu Yu.

sklearn.feature_selection

• Feature feature_selection.r_regression computes Pearson’s R correlation coefficients between the features and the target. #17169 by Dmytro Lituiev and Julien Jerphanion.

• API Change Raises an error in feature_selection.VarianceThreshold when the variance threshold is negative. #20207 by Tomohiro Endo

sklearn.inspection

• Fix Allow multiple scorers input to permutation_importance. #19411 by Simona Maggio.

sklearn.linear_model

• Feature Added linear_model.QuantileRegressor which implements linear quantile regression with L1 penalty. #9978 by David Dale and Christian Lorentzen.

• Feature The new linear_model.SGDOneClassSVM provides an SGD implementation of the linear One-Class SVM. Combined with kernel approximation techniques, this implementation approximates the solution of a kernelized One Class SVM while benefitting from a linear complexity in the number of samples. #10027 by Albert Thomas.

• Efficiency The implementation of linear_model.LogisticRegression has been optimised for dense matrices when using solver='newton-cg' and multi_class!='multinomial'. #19571 by Julien Jerphanion.

• Enhancement Validate user-supplied gram matrix passed to linear models via the precompute argument. #19004 by Adam Midvidy.

• Fix ElasticNet.fit no longer modifies sample_weight in place. #19055 by Thomas Fan.

• Fix Lasso, ElasticNet no longer have a dual_gap_ not corresponding to their objective. #19172 by Mathurin Massias

• API Change : The parameter normalize of linear_model.LinearRegression is deprecated and will be removed in 1.2. Motivation for this deprecation: normalize parameter did not take any effect if fit_intercept was set to False and therefore was deemed confusing. The behavior of the deprecated LinearModel(normalize=True) can be reproduced with Pipeline with LinearModel (where LinearModel is LinearRegression, Ridge, RidgeClassifier, RidgeCV or RidgeClassifierCV) as follows: make_pipeline(StandardScaler(with_mean=False), LinearModel()). LinearRegression was deprecated in: #17743 by Maria Telenczuk and Alexandre Gramfort. The normalize parameter in Ridge, RidgeClassifier, RidgeCV or RidgeClassifierCV were deprecated and will be removed in 1.2. #17772 by Maria Telenczuk and Alexandre Gramfort. Same for BayesianRidge, ARDRegression in: #17746 by Maria Telenczuk. The normalize parameter of linear_model.OrthogonalMatchingPursuit linear_model.OrthogonalMatchingPursuitCV will default to False in 1.2 and will be removed in 1.4. #17750 by Maria Telenczuk and Alexandre Gramfort. The normalize parameter of linear_model.Lars linear_model.LarsCV linear_model.LassoLars linear_model.LassoLarsCV linear_model.LassoLarsIC will default to False in 1.2 and will be removed in 1.4. #17769 by Maria Telenczuk and Alexandre Gramfort.

• Fix sample_weight are now fully taken into account in linear models when normalize=True for both feature centering and feature scaling. #19426 by Alexandre Gramfort and Maria Telenczuk.

• Efficiency The implementation of fit for PolynomialFeatures transformer is now faster. This is especially noticeable on large sparse input. #19734 by Fred Robinson.

• Enhancement fit method preserves dtype for numpy.float32 in Lars, LassoLars, LassoLars, LarsCV and LassoLarsCV. #20155 by Takeshi Oura.

• Fix Sample weight invariance for class:Ridge was fixed in #19616 by Oliver Grisel and Christian Lorentzen.

sklearn.manifold

• Fix Change numerical precision to prevent underflow issues during affinity matrix computation for manifold.TSNE. #19472 by Dmitry Kobak.

• Enhancement Implement 'auto' heuristic for the learning_rate in manifold.TSNE. It will become default in 1.2. The default initialization will change to pca in 1.2. PCA initialization will be scaled to have standard deviation 1e-4 in 1.2. #19491 by Dmitry Kobak.

sklearn.metrics

• API Change metrics.ConfusionMatrixDisplay exposes two class methods from_estimator and from_predictions allowing to create a confusion matrix plot using an estimator or the predictions. metrics.plot_confusion_matrix is deprecated in favor of these two class methods and will be removed in 1.2. #18543 by Guillaume Lemaitre.

• Enhancement A fix to raise an error in metrics.hinge_loss when pred_decision is 1d whereas it is a multiclass classification or when pred_decision parameter is not consistent with the labels parameter. #19643 by Pierre Attard.

• Feature metrics.mean_pinball_loss exposes the pinball loss for quantile regression. #19415 by Xavier Dupré and Oliver Grisel.

• Efficiency Improved speed of metrics.confusion_matrix when labels are integral. #9843 by Jon Crall.

• Fix metrics.ConfusionMatrixDisplay.plot uses the correct max for colormap. #19784 by Thomas Fan.

• Fix Samples with zero sample_weight values do not affect the results from metrics.det_curve, metrics.precision_recall_curve and metrics.roc_curve. #18328 by Albert Villanova del Moral and Alonso Silva Allende.

sklearn.mixture

• Fix Ensure that the best parameters are set appropriately in the case of divergency for mixture.GaussianMixture and mixture.BayesianGaussianMixture. #20030 by Tingshan Liu and Benjamin Pedigo.

sklearn.model_selection

• Feature added model_selection.StratifiedGroupKFold, that combines model_selection.StratifiedKFold and model_selection.GroupKFold, providing an ability to split data preserving the distribution of classes in each split while keeping each group within a single split. #18649 by Leandro Hermida and Rodion Martynov.

sklearn.naive_bayes

• Fix The fit and partial_fit methods of the discrete naive Bayes classifiers (naive_bayes.BernoulliNB, naive_bayes.CategoricalNB, naive_bayes.ComplementNB, and naive_bayes.MultinomialNB) now correctly handle the degenerate case of a single class in the training set. #18925 by David Poznik.

• API Change The attribute sigma_ is now deprecated in naive_bayes.GaussianNB and will be removed in 1.2. Use var_ instead. #18842 by Hong Shao Yang.

sklearn.neighbors

• Enhancement The creation of neighbors.KDTree and neighbors.BallTree has been improved for their worst-cases time complexity from \(\mathcal{O}(n^2)\) to \(\mathcal{O}(n)\). #19473 by jiefangxuanyan and Julien Jerphanion.

• Fix neighbors.DistanceMetric subclasses now support readonly memory-mapped datasets. #19883 by Julien Jerphanion.

• Fix neighbors.NearestNeighbors, neighbors.KNeighborsClassifier, neighbors.RadiusNeighborsClassifier, neighbors.KNeighborsRegressor and neighbors.RadiusNeighborsRegressor does not validate weights in __init__ and validates weights in fit instead. #20072 by Juan Carlos Alfaro Jiménez.

sklearn.pipeline

• API Change The predict_proba and predict_log_proba methods of the Pipeline class now support passing prediction kwargs to the final estimator. #19790 by Christopher Flynn.

sklearn.preprocessing

• Feature The new preprocessing.SplineTransformer is a feature preprocessing tool for the generation of B-splines, parametrized by the polynomial degree of the splines, number of knots n_knots and knot positioning strategy knots. #18368 by Christian Lorentzen. preprocessing.SplineTransformer also supports periodic splines via the extrapolation argument. #19483 by Malte Londschien.

• Feature preprocessing.OrdinalEncoder supports passing through missing values by default. #19069 by Thomas Fan.

• Fix preprocessing.scale, preprocessing.StandardScaler and similar scalers detect near-constant features to avoid scaling them to very large values. This problem happens in particular when using a scaler on sparse data with a constant column with sample weights, in which case centering is typically disabled. #19527 by Oliver Grisel and Maria Telenczuk and #19788 by Jérémie du Boisberranger.

• Fix preprocessing.StandardScaler.inverse_transform now correctly handles integer dtypes. #19356 by @makoeppel.

• Feature preprocessing.OneHotEncoder now supports handle_unknown='ignore' and dropping categories. #19041 by Thomas Fan.

• Fix preprocessing.OrdinalEncoder.inverse_transform is not supporting sparse matrix and raise the appropriate error message. #19879 by Guillaume Lemaitre.

• Efficiency Changed algorithm argument for cluster.KMeans in preprocessing.KBinsDiscretizer from auto to full. #19934 by Gleb Levitskiy.

• API Change The n_input_features_ attribute of preprocessing.PolynomialFeatures is deprecated in favor of n_features_in_ and will be removed in 1.2. #20240 by Jérémie du Boisberranger.

sklearn.tree

• Enhancement Add fontname argument in tree.export_graphviz for non-English characters. #18959 by Zero and wstates.

• Fix Fixed a bug in tree.DecisionTreeClassifier, tree.DecisionTreeRegressor where a node could be split whereas it should not have been due to incorrect handling of rounding errors. #19336 by Jérémie du Boisberranger.

• API Change The n_features_ attribute of tree.DecisionTreeClassifier, tree.DecisionTreeRegressor, tree.ExtraTreeClassifier and tree.ExtraTreeRegressor is deprecated in favor of n_features_in_ and will be removed in 1.2. #20272 by Jérémie du Boisberranger.

sklearn.utils

• Enhancement Deprecated the default value of the random_state=0 in randomized_svd. Starting in 1.2, the default value of random_state will be set to None. #19459 by Cindy Bezuidenhout and Clifford Akai-Nettey.

• Fix Fixed a bug in utils.sparsefuncs.mean_variance_axis where the precision of the computed variance was very poor when the real variance is exactly zero. #19766 by Jérémie du Boisberranger.

sklearn.validation

• Fix Support for np.matrix is deprecated in check_array in 1.0 and will raise a TypeError in 1.2. #20165 by Thomas Fan.

Code and Documentation Contributors

Thanks to everyone who has contributed to the maintenance and improvement of the project since version 0.24, including:

TODO: update at the time of the release.