sklearn.metrics
.cohen_kappa_score¶

sklearn.metrics.
cohen_kappa_score
(y1, y2, labels=None, weights=None, sample_weight=None)[source]¶ Cohen’s kappa: a statistic that measures interannotator agreement.
This function computes Cohen’s kappa [1], a score that expresses the level of agreement between two annotators on a classification problem. It is defined as
\[\kappa = (p_o  p_e) / (1  p_e)\]where \(p_o\) is the empirical probability of agreement on the label assigned to any sample (the observed agreement ratio), and \(p_e\) is the expected agreement when both annotators assign labels randomly. \(p_e\) is estimated using a perannotator empirical prior over the class labels [2].
Read more in the User Guide.
 Parameters
 y1array, shape = [n_samples]
Labels assigned by the first annotator.
 y2array, shape = [n_samples]
Labels assigned by the second annotator. The kappa statistic is symmetric, so swapping
y1
andy2
doesn’t change the value. labelsarray, shape = [n_classes], optional
List of labels to index the matrix. This may be used to select a subset of labels. If None, all labels that appear at least once in
y1
ory2
are used. weightsstr, optional
Weighting type to calculate the score. None means no weighted; “linear” means linear weighted; “quadratic” means quadratic weighted.
 sample_weightarraylike of shape = [n_samples], optional
Sample weights.
 Returns
 kappafloat
The kappa statistic, which is a number between 1 and 1. The maximum value means complete agreement; zero or lower means chance agreement.
References