article

Free access

Probability Estimates for Multi-class Classification by Pairwise Coupling

Authors:

Ruby C. WengAuthors Info & Claims

The Journal of Machine Learning Research, Volume 5

Pages 975 - 1005

Published: 01 December 2004 Publication History

Abstract

Pairwise coupling is a popular multi-class classification method that combines all comparisons for each pair of classes. This paper presents two approaches for obtaining class probabilities. Both methods can be reduced to linear systems and are easy to implement. We show conceptually and experimentally that the proposed approaches are more stable than the two existing popular methods: voting and the method by Hastie and Tibshirani (1998)

References

[1]

E. L. Allwein, R. E. Schapire, and Y. Singer. Reducing multiclass to binary: a unifying approach for margin classiers. Journal of Machine Learning Research, 1:113-141, 2001. ISSN 1533-7928.

Digital Library

[2]

C. L. Blake and C. J. Merz. UCI repository of machine learning databases. Technical report, University of California, Department of Information and Computer Science, Irvine, CA, 1998. Available at http://www.ics.uci.edu/~mlearn/MLRepository.html.

[3]

B. Boser, I. Guyon, and V. Vapnik. A training algorithm for optimal margin classifiers. In Proceedings of the Fifth Annual Workshop on Computational Learning Theory, 1992.

Digital Library

[4]

L. Breiman. Random forests. Machine Learning, 45(1):5-32, 2001. URL citeseer.nj. nec.com/breiman01random.html.

Digital Library

[5]

G. W. Brier. Verification of forecasts expressed in probabilities. Monthly Weather Review, 78:1-3, 1950.

[6]

C.-C. Chang and C.-J. Lin. LIBSVM: a library for support vector machines, 2001. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm.

[7]

C. Cortes and V. Vapnik. Support-vector network. Machine Learning, 20:273-297, 1995.

Digital Library

[8]

K. Duan and S. S. Keerthi. Which is the best multiclass SVM method? An empirical study. Technical Report CD-03-12, Control Division, Department of Mechanical Engineering, National University of Singapore, 2003.

[9]

J. Friedman. Another approach to polychotomous classification. Technical report, Department of Statistics, Stanford University, 1996. Available at http://www-stat.stanford.edu/reports/friedman/poly.ps.Z.

[10]

T. Hastie and R. Tibshirani. Classification by pairwise coupling. The Annals of Statistics, 26(1):451-471, 1998.

[11]

J. J. Hull. A database for handwritten text recognition research. IEEE Transactions on Pattern Analysis and Machine Intelligence, 16(5):550-554, May 1994.

Digital Library

[12]

D. R. Hunter. MM algorithms for generalized Bradley-Terry models. The Annals of Statistics , 32:386-408, 2004.

[13]

S. Knerr, L. Personnaz, and G. Dreyfus. Single-layer learning revisited: a stepwise procedure for building and training a neural network. In J. Fogelman, editor, Neurocomputing: Algorithms, Architectures and Applications. Springer-Verlag, 1990.

[14]

Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-2324, November 1998. MNIST database available at http://yann.lecun.com/exdb/mnist/.

[15]

A. Liaw and M. Wiener. Classification and regression by randomForest. R News, 2/3:18-22, December 2002. URL http://cran.r-project.org/doc/Rnews/Rnews_2002-3.pdf.

[16]

H.-T. Lin, C.-J. Lin, and R. C. Weng. A note on Platt's probabilistic outputs for support vector machines. Technical report, Department of Computer Science, National Taiwan University, 2003. URL http://www.csie.ntu.edu.tw/~cjlin/papers/plattprob.ps.

[17]

D. Michie, D. J. Spiegelhalter, and C. C. Taylor. Machine Learning, Neural and Statistical Classification. Prentice Hall, Englewood Cliffs, N.J., 1994. Data available at http: //www.ncc.up.pt/liacc/ML/statlog/datasets.html.

Digital Library

[18]

J. Platt. Probabilistic outputs for support vector machines and comparison to regularized likelihood methods. In A. J. Smola, P. L. Bartlett, B. Schölkopf, and D. Schuurmans, editors, Advances in Large Margin Classifiers, Cambridge, MA, 2000. MIT Press. URL citeseer.nj.nec.com/platt99probabilistic.html.

[19]

D. Price, S. Knerr, L. Personnaz, and G. Dreyfus. Pairwise nerual network classifiers with probabilistic outputs. In G. Tesauro, D. Touretzky, and T. Leen, editors, Neural Information Processing Systems, volume 7, pages 1109-1116. The MIT Press, 1995.

[20]

P. Refregier and F. Vallet. Probabilistic approach for multiclass classification with neural networks. In Proceedings of International Conference on Artificial Networks, pages 1003- 1007, 1991.

[21]

S. Ross. Stochastic Processes. John Wiley & Sons, Inc., second edition, 1996.

[22]

V. Sventnik, A. Liaw, C. Tong, J. C. Culberson, R. P. Sheridan, and B. P. Feuston. Random forest: a tool for classification and regression in compound classification and QSAR modeling. Journal of Chemical Information and Computer Science, 43(6):1947-1958, 2003.

[23]

T.-F. Wu, C.-J. Lin, and R. C. Weng. Probability estimates for multi-class classification by pairwise coupling. In S. Thrun, L. Saul, and B. Schölkopf, editors, Advances in Neural Information Processing Systems 16. MIT Press, Cambridge, MA, 2004. URL http://www.csie.ntu.edu.tw/~cjlin/papers/svmprob.pdf.

[24]

E. Zermelo. Die berechnung der turnier-ergebnisse als ein maximumproblem der wahrscheinlichkeitsrechnung. Mathematische Zeitschrift, 29:436-460, 1929.

[25]

T. Zhang. Statistical behavior and consistency of classification methods based on convex risk minimization. The Annals of Statistics, 32(1):56-134, 2004.

Cited By

Liu JTang YZhao HWang XLi FZhang J(2024)CPS Attack Detection under Limited Local Information in Cyber Security: An Ensemble Multi-Node Multi-Class Classification ApproachACM Transactions on Sensor Networks10.1145/358552020:2(1-27)Online publication date: 9-Jan-2024
https://dl.acm.org/doi/10.1145/3585520
Corbara SMoreo ASebastiani F(2023)Same or Different? Diff-Vectors for Authorship AnalysisACM Transactions on Knowledge Discovery from Data10.1145/360922618:1(1-36)Online publication date: 6-Sep-2023
https://dl.acm.org/doi/10.1145/3609226
Resck LPonciano JNonato LPoco J(2023)LegalVis: Exploring and Inferring Precedent Citations in Legal DocumentsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2022.315245029:6(3105-3120)Online publication date: 1-Jun-2023
https://dl.acm.org/doi/10.1109/TVCG.2022.3152450
Show More Cited By

Probability Estimates for Multi-class Classification by Pairwise Coupling
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Supervised learning
    2. Machine learning approaches

Recommendations

Cost-sensitive multi-class classification from probability estimates
ICML '08: Proceedings of the 25th international conference on Machine learning

For two-class classification, it is common to classify by setting a threshold on class probability estimates, where the threshold is determined by ROC curve analysis. An analog for multi-class classification is learning a new class partitioning of the ...
TrueSkill-Based pairwise coupling for multi-class classification
ICANN'12: Proceedings of the 22nd international conference on Artificial Neural Networks and Machine Learning - Volume Part II

A multi-class classification problem can be solved efficiently via decomposition of the problem into multiple binary classification problems. As a way of such decomposition, we propose a novel pairwise coupling method based on the TrueSkill ranking ...
Probability estimates for multi-class classification by pairwise coupling
NIPS'03: Proceedings of the 16th International Conference on Neural Information Processing Systems

Pairwise coupling is a popular multi-class classification method that combines together all pairwise comparisons for each pair of classes. This paper presents two approaches for obtaining class probabilities. Both methods can be reduced to linear ...

Comments

Information & Contributors

Information

Published In

cover image The Journal of Machine Learning Research

The Journal of Machine Learning Research Volume 5, Issue

12/1/2004

1571 pages

ISSN:1532-4435

EISSN:1533-7928

Issue’s Table of Contents

Publisher

JMLR.org

Publication History

Published: 01 December 2004

Published in JMLR Volume 5

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

309
Total Citations
View Citations
2,588
Total Downloads

Downloads (Last 12 months)64
Downloads (Last 6 weeks)14

Reflects downloads up to 13 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Liu JTang YZhao HWang XLi FZhang J(2024)CPS Attack Detection under Limited Local Information in Cyber Security: An Ensemble Multi-Node Multi-Class Classification ApproachACM Transactions on Sensor Networks10.1145/358552020:2(1-27)Online publication date: 9-Jan-2024
https://dl.acm.org/doi/10.1145/3585520
Corbara SMoreo ASebastiani F(2023)Same or Different? Diff-Vectors for Authorship AnalysisACM Transactions on Knowledge Discovery from Data10.1145/360922618:1(1-36)Online publication date: 6-Sep-2023
https://dl.acm.org/doi/10.1145/3609226
Resck LPonciano JNonato LPoco J(2023)LegalVis: Exploring and Inferring Precedent Citations in Legal DocumentsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2022.315245029:6(3105-3120)Online publication date: 1-Jun-2023
https://dl.acm.org/doi/10.1109/TVCG.2022.3152450
Ye XZhao J(2023)Graph Convolutional Network With Unknown Class NumberIEEE Transactions on Multimedia10.1109/TMM.2022.318340125(4800-4813)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1109/TMM.2022.3183401
Liu ZZheng JLin JWang LWu K(2023)$\mathtt {Radar}$: Adversarial Driving Style Representation Learning With Data AugmentationIEEE Transactions on Mobile Computing10.1109/TMC.2022.320826522:12(7070-7085)Online publication date: 1-Dec-2023
https://dl.acm.org/doi/10.1109/TMC.2022.3208265
Fu SChen PYe Z(2023)Simplex-Based Proximal Multicategory Support Vector MachineIEEE Transactions on Information Theory10.1109/TIT.2022.322226669:4(2427-2451)Online publication date: 1-Apr-2023
https://dl.acm.org/doi/10.1109/TIT.2022.3222266
Lyu GFeng SLiu WLiu SLang C(2022)Redundant Label Learning via Subspace Representation and Global DisambiguationACM Transactions on Intelligent Systems and Technology10.1145/355854714:1(1-19)Online publication date: 9-Nov-2022
https://dl.acm.org/doi/10.1145/3558547
Alshammari NAlharbi F(2022)Combining a Novel Scoring Approach with Arabic Stemming Techniques for Arabic Chatbots Conversation EngineACM Transactions on Asian and Low-Resource Language Information Processing10.1145/351121521:4(1-21)Online publication date: 20-Jan-2022
https://dl.acm.org/doi/10.1145/3511215
Qian XHe FHu XWang TIpsita ARamani K(2022)ScalAR: Authoring Semantically Adaptive Augmented Reality Experiences in Virtual RealityProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3517665(1-18)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3517665
Shu YYang PLiu NZhang SZhao GLiu Y(2022)Emotion Distribution Learning Based on Peripheral Physiological SignalsIEEE Transactions on Affective Computing10.1109/TAFFC.2022.316360914:3(2470-2483)Online publication date: 30-Mar-2022
https://dl.acm.org/doi/10.1109/TAFFC.2022.3163609
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Media

Figures

Other

Tables

View Issue’s Table of Contents