Contributing¶

How to make a good bug report¶

When you submit an issue to Github, please do your best to follow these guidelines! This will make it a lot easier to provide you with good feedback:

• The ideal bug report contains a short reproducible code snippet, this way anyone can try to reproduce the bug easily (see this for more details). If your snippet is longer than around 50 lines, please link to a gist or a github repo.

• If not feasible to include a reproducible snippet, please be specific about what estimators and/or functions are involved and the shape of the data.

• If an exception is raised, please provide the full traceback.

• Please include your operating system type and version number, as well as your Python, scikit-learn, numpy, and scipy versions. This information can be found by running the following code snippet:

  >>> import sklearn
  >>> sklearn.show_versions()  
  

  Note

  This utility function is only available in scikit-learn v0.20+. For previous versions, one has to explicitly run:

  import platform; print(platform.platform())
  import sys; print("Python", sys.version)
  import numpy; print("NumPy", numpy.__version__)
  import scipy; print("SciPy", scipy.__version__)
  import sklearn; print("Scikit-Learn", sklearn.__version__)
  

• Please ensure all code snippets and error messages are formatted in appropriate code blocks. See Creating and highlighting code blocks for more details.

If you want to help curate issues, read the following.

Video resources¶

These videos are step-by-step introductions on how to contribute to scikit-learn, and are a great companion to the following text guidelines. Please make sure to still check our guidelines below, since they describe our latest up-to-date workflow.

• Crash Course in Contributing to Scikit-Learn & Open Source Projects: Video, Transcript

• Example of Submitting a Pull Request to scikit-learn: Video, Transcript

• Sprint-specific instructions and practical tips: Video, Transcript

How to contribute¶

The preferred way to contribute to scikit-learn is to fork the main repository on GitHub, then submit a “pull request” (PR).

In the first few steps, we explain how to locally install scikit-learn, and how to set up your git repository:

1. Create an account on GitHub if you do not already have one.

2. Fork the project repository: click on the ‘Fork’ button near the top of the page. This creates a copy of the code under your account on the GitHub user account. For more details on how to fork a repository see this guide.

3. Clone your fork of the scikit-learn repo from your GitHub account to your local disk:

   git clone git@github.com:YourLogin/scikit-learn.git  # add --depth 1 if your connection is slow
   cd scikit-learn
   

4. Install the development dependencies:

   pip install cython pytest pytest-cov flake8 mypy black==21.6b0
   

5. Install scikit-learn in editable mode:

   pip install --no-build-isolation --editable .
   

   If you receive errors in building scikit-learn, see the Building from source section.

6. Add the upstream remote. This saves a reference to the main scikit-learn repository, which you can use to keep your repository synchronized with the latest changes:

   git remote add upstream https://github.com/scikit-learn/scikit-learn.git
   

You should now have a working installation of scikit-learn, and your git repository properly configured. The next steps now describe the process of modifying code and submitting a PR:

7. Synchronize your main branch with the upstream main branch:

   git checkout main
   git pull upstream main
   

8. Create a feature branch to hold your development changes:

   git checkout -b my_feature
   

   and start making changes. Always use a feature branch. It’s good practice to never work on the main branch!

9. (Optional) Install pre-commit to run code style checks before each commit:

   pip install pre-commit
   pre-commit install
   

   pre-commit checks can be disabled for a particular commit with git commit -n.

10. Develop the feature on your feature branch on your computer, using Git to do the version control. When you’re done editing, add changed files using git add and then git commit:

    git add modified_files
    git commit
    

    to record your changes in Git, then push the changes to your GitHub account with:

    git push -u origin my_feature
    

11. Follow these instructions to create a pull request from your fork. This will send an email to the committers. You may want to consider sending an email to the mailing list for more visibility.

pip install --no-build-isolation -e .

It is often helpful to keep your local feature branch synchronized with the latest changes of the main scikit-learn repository:

git fetch upstream
git merge upstream/main

Subsequently, you might need to solve the conflicts. You can refer to the Git documentation related to resolving merge conflict using the command line.

Pull request checklist¶

Before a PR can be merged, it needs to be approved by two core developers. Please prefix the title of your pull request with [MRG] if the contribution is complete and should be subjected to a detailed review. An incomplete contribution – where you expect to do more work before receiving a full review – should be prefixed [WIP] (to indicate a work in progress) and changed to [MRG] when it matures. WIPs may be useful to: indicate you are working on something to avoid duplicated work, request broad review of functionality or API, or seek collaborators. WIPs often benefit from the inclusion of a task list in the PR description.

In order to ease the reviewing process, we recommend that your contribution complies with the following rules before marking a PR as [MRG]. The bolded ones are especially important:

1. Give your pull request a helpful title that summarises what your contribution does. This title will often become the commit message once merged so it should summarise your contribution for posterity. In some cases “Fix <ISSUE TITLE>” is enough. “Fix #<ISSUE NUMBER>” is never a good title.

2. Make sure your code passes the tests. The whole test suite can be run with pytest, but it is usually not recommended since it takes a long time. It is often enough to only run the test related to your changes: for example, if you changed something in sklearn/linear_model/logistic.py, running the following commands will usually be enough:

   • pytest sklearn/linear_model/logistic.py to make sure the doctest examples are correct

   • pytest sklearn/linear_model/tests/test_logistic.py to run the tests specific to the file

   • pytest sklearn/linear_model to test the whole linear_model module

   • pytest doc/modules/linear_model.rst to make sure the user guide examples are correct.

   • pytest sklearn/tests/test_common.py -k LogisticRegression to run all our estimator checks (specifically for LogisticRegression, if that’s the estimator you changed).

   There may be other failing tests, but they will be caught by the CI so you don’t need to run the whole test suite locally. For guidelines on how to use pytest efficiently, see the Useful pytest aliases and flags.

3. Make sure your code is properly commented and documented, and make sure the documentation renders properly. To build the documentation, please refer to our Documentation guidelines. The CI will also build the docs: please refer to Generated documentation on CircleCI.

4. Tests are necessary for enhancements to be accepted. Bug-fixes or new features should be provided with non-regression tests. These tests verify the correct behavior of the fix or feature. In this manner, further modifications on the code base are granted to be consistent with the desired behavior. In the case of bug fixes, at the time of the PR, the non-regression tests should fail for the code base in the main branch and pass for the PR code.

5. Run black to auto-format your code.

   black .
   

   See black’s editor integration documentation to configure your editor to run black.

6. Make sure that your PR does not add PEP8 violations. To check the code that you changed, you can run the following command (see above to set up the upstream remote):

   git diff upstream/main -u -- "*.py" | flake8 --diff
   

   or make flake8-diff which should work on unix-like system.

7. Follow the Coding guidelines.

8. When applicable, use the validation tools and scripts in the sklearn.utils submodule. A list of utility routines available for developers can be found in the Utilities for Developers page.

9. Often pull requests resolve one or more other issues (or pull requests). If merging your pull request means that some other issues/PRs should be closed, you should use keywords to create link to them (e.g., Fixes #1234; multiple issues/PRs are allowed as long as each one is preceded by a keyword). Upon merging, those issues/PRs will automatically be closed by GitHub. If your pull request is simply related to some other issues/PRs, create a link to them without using the keywords (e.g., See also #1234).

10. PRs should often substantiate the change, through benchmarks of performance and efficiency (see Monitoring performance) or through examples of usage. Examples also illustrate the features and intricacies of the library to users. Have a look at other examples in the examples/ directory for reference. Examples should demonstrate why the new functionality is useful in practice and, if possible, compare it to other methods available in scikit-learn.

11. New features have some maintenance overhead. We expect PR authors to take part in the maintenance for the code they submit, at least initially. New features need to be illustrated with narrative documentation in the user guide, with small code snippets. If relevant, please also add references in the literature, with PDF links when possible.

12. The user guide should also include expected time and space complexity of the algorithm and scalability, e.g. “this algorithm can scale to a large number of samples > 100000, but does not scale in dimensionality: n_features is expected to be lower than 100”.

You can also check our Code Review Guidelines to get an idea of what reviewers will expect.

You can check for common programming errors with the following tools:

• Code with a good unittest coverage (at least 80%, better 100%), check with:

  pip install pytest pytest-cov
  pytest --cov sklearn path/to/tests_for_package
  

  see also Testing and improving test coverage

  Run static analysis with mypy:

  mypy sklearn
  

  must not produce new errors in your pull request. Using # type: ignore annotation can be a workaround for a few cases that are not supported by mypy, in particular,

  • when importing C or Cython modules

  • on properties with decorators

Bonus points for contributions that include a performance analysis with a benchmark script and profiling output (see Monitoring performance).

Also check out the How to optimize for speed guide for more details on profiling and Cython optimizations.

Issues for New Contributors¶

New contributors should look for the following tags when looking for issues. We strongly recommend that new contributors tackle “easy” issues first: this helps the contributor become familiar with the contribution workflow, and for the core devs to become acquainted with the contributor; besides which, we frequently underestimate how easy an issue is to solve!

Building the documentation¶

First, make sure you have properly installed the development version.

Building the documentation requires installing some additional packages:

pip install sphinx sphinx-gallery numpydoc matplotlib Pillow pandas \
            scikit-image packaging seaborn sphinx-prompt

To build the documentation, you need to be in the doc folder:

cd doc

In the vast majority of cases, you only need to generate the full web site, without the example gallery:

make

The documentation will be generated in the _build/html/stable directory. To also generate the example gallery you can use:

make html

This will run all the examples, which takes a while. If you only want to generate a few examples, you can use:

EXAMPLES_PATTERN=your_regex_goes_here make html

This is particularly useful if you are modifying a few examples.

Set the environment variable NO_MATHJAX=1 if you intend to view the documentation in an offline setting.

To build the PDF manual, run:

make latexpdf

Guidelines for writing documentation¶

It is important to keep a good compromise between mathematical and algorithmic details, and give intuition to the reader on what the algorithm does.

Basically, to elaborate on the above, it is best to always start with a small paragraph with a hand-waving explanation of what the method does to the data. Then, it is very helpful to point out why the feature is useful and when it should be used - the latter also including “big O” (\(O\left(g\left(n\right)\right)\)) complexities of the algorithm, as opposed to just rules of thumb, as the latter can be very machine-dependent. If those complexities are not available, then rules of thumb may be provided instead.

Secondly, a generated figure from an example (as mentioned in the previous paragraph) should then be included to further provide some intuition.

Next, one or two small code examples to show its use can be added.

Next, any math and equations, followed by references, can be added to further the documentation. Not starting the documentation with the maths makes it more friendly towards users that are just interested in what the feature will do, as opposed to how it works “under the hood”.

Finally, follow the formatting rules below to make it consistently good:

• Add “See Also” in docstrings for related classes/functions.

• “See Also” in docstrings should be one line per reference, with a colon and an explanation, for example:

  See Also
  --------
  SelectKBest : Select features based on the k highest scores.
  SelectFpr : Select features based on a false positive rate test.
  

• When documenting the parameters and attributes, here is a list of some well-formatted examples:

  n_clusters : int, default=3
      The number of clusters detected by the algorithm.
  
  some_param : {'hello', 'goodbye'}, bool or int, default=True
      The parameter description goes here, which can be either a string
      literal (either `hello` or `goodbye`), a bool, or an int. The default
      value is True.
  
  array_parameter : {array-like, sparse matrix} of shape (n_samples, n_features) or (n_samples,)
      This parameter accepts data in either of the mentioned forms, with one
      of the mentioned shapes. The default value is
      `np.ones(shape=(n_samples,))`.
  
  list_param : list of int
  
  typed_ndarray : ndarray of shape (n_samples,), dtype=np.int32
  
  sample_weight : array-like of shape (n_samples,), default=None
  
  multioutput_array : ndarray of shape (n_samples, n_classes) or list of such arrays
  

  In general have the following in mind:

  1. Use Python basic types. (bool instead of boolean)

  2. Use parenthesis for defining shapes: array-like of shape (n_samples,) or array-like of shape (n_samples, n_features)

  3. For strings with multiple options, use brackets: input: {'log', 'squared', 'multinomial'}

  4. 1D or 2D data can be a subset of {array-like, ndarray, sparse matrix, dataframe}. Note that array-like can also be a list, while ndarray is explicitly only a numpy.ndarray.

  5. Specify dataframe when “frame-like” features are being used, such as the column names.

  6. When specifying the data type of a list, use of as a delimiter: list of int. When the parameter supports arrays giving details about the shape and/or data type and a list of such arrays, you can use one of array-like of shape (n_samples,) or list of such arrays.

  7. When specifying the dtype of an ndarray, use e.g. dtype=np.int32 after defining the shape: ndarray of shape (n_samples,), dtype=np.int32. You can specify multiple dtype as a set: array-like of shape (n_samples,), dtype={np.float64, np.float32}. If one wants to mention arbitrary precision, use integral and floating rather than the Python dtype int and float. When both int and floating are supported, there is no need to specify the dtype.

  8. When the default is None, None only needs to be specified at the end with default=None. Be sure to include in the docstring, what it means for the parameter or attribute to be None.

• For unwritten formatting rules, try to follow existing good works:

  • For “References” in docstrings, see the Silhouette Coefficient (sklearn.metrics.silhouette_score).

• When editing reStructuredText (.rst) files, try to keep line length under 80 characters when possible (exceptions include links and tables).

• Do not modify sphinx labels as this would break existing cross references and external links pointing to specific sections in the scikit-learn documentation.

• Before submitting your pull request check if your modifications have introduced new sphinx warnings and try to fix them.

Generated documentation on CircleCI¶

When you change the documentation in a pull request, CircleCI automatically builds it. To view the documentation generated by CircleCI, simply go at the bottom of your PR page and look for the “ci/circleci: doc artifact” link.

Writing matplotlib related tests¶

Test fixtures ensure that a set of tests will be executing with the appropriate initialization and cleanup. The scikit-learn test suite implements a fixture which can be used with matplotlib.

pyplot

The pyplot fixture should be used when a test function is dealing with matplotlib. matplotlib is a soft dependency and is not required. This fixture is in charge of skipping the tests if matplotlib is not installed. In addition, figures created during the tests will be automatically closed once the test function has been executed.

To use this fixture in a test function, one needs to pass it as an argument:

def test_requiring_mpl_fixture(pyplot):
    # you can now safely use matplotlib

Workflow to improve test coverage¶

To test code coverage, you need to install the coverage package in addition to pytest.

1. Run ‘make test-coverage’. The output lists for each file the line

   numbers that are not tested.

2. Find a low hanging fruit, looking at which lines are not tested,

   write or adapt a test specifically for these lines.

3. Loop.

Deprecation¶

If any publicly accessible method, function, attribute or parameter is renamed, we still support the old one for two releases and issue a deprecation warning when it is called/passed/accessed. E.g., if the function zero_one is renamed to zero_one_loss, we add the decorator deprecated (from sklearn.utils) to zero_one and call zero_one_loss from that function:

from ..utils import deprecated

def zero_one_loss(y_true, y_pred, normalize=True):
    # actual implementation
    pass

@deprecated("Function 'zero_one' was renamed to 'zero_one_loss' "
            "in version 0.13 and will be removed in release 0.15. "
            "Default behavior is changed from 'normalize=False' to "
            "'normalize=True'")
def zero_one(y_true, y_pred, normalize=False):
    return zero_one_loss(y_true, y_pred, normalize)

If an attribute is to be deprecated, use the decorator deprecated on a property. Please note that the property decorator should be placed before the deprecated decorator for the docstrings to be rendered properly. E.g., renaming an attribute labels_ to classes_ can be done as:

@deprecated("Attribute labels_ was deprecated in version 0.13 and "
            "will be removed in 0.15. Use 'classes_' instead")
@property
def labels_(self):
    return self.classes_

If a parameter has to be deprecated, a FutureWarning warning must be raised too. In the following example, k is deprecated and renamed to n_clusters:

import warnings

def example_function(n_clusters=8, k='deprecated'):
    if k != 'deprecated':
        warnings.warn("'k' was renamed to n_clusters in version 0.13 and "
                      "will be removed in 0.15.",
                      FutureWarning)
        n_clusters = k

When the change is in a class, we validate and raise warning in fit:

import warnings

class ExampleEstimator(BaseEstimator):
    def __init__(self, n_clusters=8, k='deprecated'):
        self.n_clusters = n_clusters
        self.k = k

    def fit(self, X, y):
        if self.k != 'deprecated':
            warnings.warn("'k' was renamed to n_clusters in version 0.13 and "
                          "will be removed in 0.15.",
                          FutureWarning)
            self._n_clusters = self.k
        else:
            self._n_clusters = self.n_clusters

As in these examples, the warning message should always give both the version in which the deprecation happened and the version in which the old behavior will be removed. If the deprecation happened in version 0.x-dev, the message should say deprecation occurred in version 0.x and the removal will be in 0.(x+2), so that users will have enough time to adapt their code to the new behaviour. For example, if the deprecation happened in version 0.18-dev, the message should say it happened in version 0.18 and the old behavior will be removed in version 0.20.

In addition, a deprecation note should be added in the docstring, recalling the same information as the deprecation warning as explained above. Use the .. deprecated:: directive:

.. deprecated:: 0.13
   ``k`` was renamed to ``n_clusters`` in version 0.13 and will be removed
   in 0.15.

What’s more, a deprecation requires a test which ensures that the warning is raised in relevant cases but not in other cases. The warning should be caught in all other tests (using e.g., @pytest.mark.filterwarnings), and there should be no warning in the examples.

Change the default value of a parameter¶

If the default value of a parameter needs to be changed, please replace the default value with a specific value (e.g., warn) and raise FutureWarning when users are using the default value. In the following example, we change the default value of n_clusters from 5 to 10 (current version is 0.20):

import warnings

def example_function(n_clusters='warn'):
    if n_clusters == 'warn':
        warnings.warn("The default value of n_clusters will change from "
                      "5 to 10 in 0.22.", FutureWarning)
        n_clusters = 5

When the change is in a class, we validate and raise warning in fit:

import warnings

class ExampleEstimator:
    def __init__(self, n_clusters='warn'):
        self.n_clusters = n_clusters

    def fit(self, X, y):
        if self.n_clusters == 'warn':
          warnings.warn("The default value of n_clusters will change from "
                        "5 to 10 in 0.22.", FutureWarning)
          self._n_clusters = 5

Similar to deprecations, the warning message should always give both the version in which the change happened and the version in which the old behavior will be removed. The docstring needs to be updated accordingly. We need a test which ensures that the warning is raised in relevant cases but not in other cases. The warning should be caught in all other tests (using e.g., @pytest.mark.filterwarnings), and there should be no warning in the examples.