On the noise resilience of ranking measures

Berrar, Daniel (2016). On the noise resilience of ranking measures. In: Neural Information Processing ICONIP 2016 (Hirose, A.; Ozawa, S.; Doya, K.; Ikeda, K.; Lee, M. and Liu, D. eds.), Lecture Notes in Computer Science (LNTCS), Springer-Verlag, pp. 47–55.

DOI: https://doi.org/10.1007/978-3-319-46672-9_6


Performance measures play a pivotal role in the evaluation and selection of machine learning models for a wide range of applications. Using both synthetic and real-world data sets, we investigated the resilience to noise of various ranking measures. Our experiments revealed that the area under the ROC curve AUC and a related measure, the truncated average Kolmogorov-Smirnov statistic taKS, can reliably discriminate between models with truly different performance under various types and levels of noise. With increasing class skew, however, the H-measure and estimators of the area under the precision-recall curve become preferable measures. Because of its simple graphical interpretation and robustness, the lower trapezoid estimator of the area under the precision-recall curve is recommended for highly imbalanced data sets.

Viewing alternatives


Public Attention

Altmetrics from Altmetric

Number of Citations

Citations from Dimensions
No digital document available to download for this item

Item Actions