Reliability assessment for remote sensing data: beyond Cohen's kappa

Abstract

Cohen’s kappa coefficient has been widely used for assessing classification results derived from remote sensing data. It however presents several limitations, which are preventing both an efficient use as well as a generalisation of its use. This paper reviews these problems and proposes as an alternative to prefer the Krippendorff’s alpha- coefficient over Cohen’s kappa. Krippendorff’s alpha indeed presents less flaws while dealing with more data types - hence allowing the rating of quantitative data - and managing the case where more than two judgements are issued and finally dealing with cases where no judgement is issued. These concepts are be illustrated by some exemplary data-sets.