[PDF][PDF] A Skew-Sensitive Evaluation Framework for Imbalanced Data Classification
arXiv preprint arXiv:2010.05995, 2020•dmlr.ai
Class distribution skews in imbalanced datasets may lead to models with prediction bias
towards majority classes, making fair assessment of classifiers a challenging task. Metrics
such as Balanced Accuracy are commonly used to evaluate a classifier's prediction
performance under such scenarios. However, these metrics fall short when classes vary in
importance. In this paper, we propose a simple and general-purpose evaluation framework
for imbalanced data classification that is sensitive to arbitrary skews in class cardinalities …
towards majority classes, making fair assessment of classifiers a challenging task. Metrics
such as Balanced Accuracy are commonly used to evaluate a classifier's prediction
performance under such scenarios. However, these metrics fall short when classes vary in
importance. In this paper, we propose a simple and general-purpose evaluation framework
for imbalanced data classification that is sensitive to arbitrary skews in class cardinalities …
Abstract
Class distribution skews in imbalanced datasets may lead to models with prediction bias towards majority classes, making fair assessment of classifiers a challenging task. Metrics such as Balanced Accuracy are commonly used to evaluate a classifier’s prediction performance under such scenarios. However, these metrics fall short when classes vary in importance. In this paper, we propose a simple and general-purpose evaluation framework for imbalanced data classification that is sensitive to arbitrary skews in class cardinalities and importances. Experiments with several stateof-the-art classifiers tested on real-world datasets from three different domains show the effectiveness of our framework–not only in evaluating and ranking classifiers, but also training them.
dmlr.ai