research-article

ABC-boost: adaptive base class boost for multi-class classification

Author:

Ping LiAuthors Info & Claims

ICML '09: Proceedings of the 26th Annual International Conference on Machine Learning

Pages 625 - 632

https://doi.org/10.1145/1553374.1553455

Published: 14 June 2009 Publication History

Abstract

We propose abc-boost (adaptive base class boost) for multi-class classification and present abc-mart, an implementation of abc-boost, based on the multinomial logit model. The key idea is that, at each boosting iteration, we adaptively and greedily choose a base class. Our experiments on public datasets demonstrate the improvement of abc-mart over the original mart algorithm.

References

[1]

Allwein, E. L., Schapire, R. E., & Singer, Y. (2000). Reducing multiclass to binary: A unifying approach for margin classifiers. J. of Machine Learning Research, 1, 113--141.

Digital Library

[2]

Bartlett, P., Freund, Y., Lee, W. S., & Schapire, R. E. (1998). Boosting the margin: a new explanation for the effectiveness of voting methods. The Annals of Statistics, 26, 1651--1686.

[3]

Cossock, D., & Zhang, T. (2006). Subset ranking using regression. Conf. on Learning Theory, 605--619.

Digital Library

[4]

Freund, Y. (1995). Boosting a weak learning algorithm by majority. Inf. Comput., 121, 256--285.

Digital Library

[5]

Freund, Y., & Schapire, R. E. (1997). A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. Syst. Sci., 55, 119--139.

Digital Library

[6]

Friedman, J. H. (2001). Greedy function approximation: A gradient boosting machine. The Annals of Statistics, 29, 1189--1232.

[7]

Friedman, J. H., Hastie, T. J., & Tibshirani, R. (2000). Additive logistic regression: a statistical view of boosting. The Annals of Statistics, 28, 337--407.

[8]

Lee, Y., Lin, Y., & Wahba, G. (2004). Multicategory support vector machines: Theory and application to the classification of microarray data and satellite radiance data. J. of Amer. Stat. Asso., 99, 67--81.

[9]

Li, P., Burges, C. J., & Wu, Q. (2008). Mcrank: Learning to rank using classification and gradient boosting. Neur. Inf. Proc. Sys. Conf. 897--904.

[10]

Mason, L., Baxter, J., Bartlett, P., & Frean, M. (2000). Boosting algorithms as gradient descent. Neur. Inf. Proc. Sys. Conf. 512--518.

[11]

Schapire, R. (1990). The strength of weak learnability. Machine Learning, 5, 197--227.

Digital Library

[12]

Schapire, R. E., & Singer, Y. (1999). Improved boosting algorithms using confidence-rated predictions. Machine Learning, 37, 297--336.

Digital Library

[13]

Schapire, R. E., & Singer, Y. (2000). Boostexter: A boosting-based system for text categorization. Machine Learning, 39, 135--168.

Digital Library

[14]

Tewari, A., & Bartlett, P. L. (2007). On the consistency of multiclass classification methods. J. of Machine Learning Research, 8, 1007--1025.

Digital Library

[15]

Zhang, T. (2004). Statistical analysis of some multi-category large margin classification methods. J. of Machine Learning Research, 5, 1225--1251.

Digital Library

[16]

Zheng, Z., Zha, H., Zhang, T., Chapelle, O., Chen, K., Sun, G. (2008). A General Boosting Method and its Application to Learning Ranking Functions for Web Search Neur. Inf. Proc. Sys. Conf. 1697--1704.

[17]

Zou, H., Zhu, J., & Hastie, T. (2008). New multi-category boosting algorithms based on multicategory fisher-consistent losses. The Annals of Applied Statistics, 2, 1290--1306.

Cited By

Wang XZhao Y(2024)Multi-Branch Line Fault Arc Detection Method Based on the Improved Northern Goshawk Optimization Adaptive Base Class LogitBoost AlgorithmEnergies10.3390/en1704095417:4(954)Online publication date: 19-Feb-2024
https://doi.org/10.3390/en17040954
Emami SMartínez-Muñoz G(2024)Condensed-gradient boostingInternational Journal of Machine Learning and Cybernetics10.1007/s13042-024-02279-0Online publication date: 23-Jul-2024
https://doi.org/10.1007/s13042-024-02279-0
Swapna MViswanadhula UAluvalu RVardharajan VKotecha K(2022)Bio-Signals in Medical Applications and Challenges Using Artificial IntelligenceJournal of Sensor and Actuator Networks10.3390/jsan1101001711:1(17)Online publication date: 25-Feb-2022
https://doi.org/10.3390/jsan11010017
Show More Cited By

Index Terms

ABC-boost: adaptive base class boost for multi-class classification
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Supervised learning
        Supervised learning by classification
    2. Machine learning approaches
      1. Classification and regression trees
  2. Modeling and simulation
    1. Model development and analysis
      1. Model verification and validation
      2. Modeling methodologies

Recommendations

Adaptive Ensemble Undersampling-Boost

We propose a framework for imbalanced classification tasks with ensemble learning.We propose a weight modification method to give each classifier an adaptive weight.We introduce OTSU algorithm to determine the optimal threshold adaptively.We make ...
AR-boost: reducing overfitting by a robust data-driven regularization strategy
ECMLPKDD'13: Proceedings of the 2013th European Conference on Machine Learning and Knowledge Discovery in Databases - Volume Part III

We introduce a novel, robust data-driven regularization strategy called Adaptive Regularized Boosting (AR-Boost), motivated by a desire to reduce overfitting. We replace AdaBoost's hard margin with a regularized soft margin that trades-off between a ...
Ent-Boost: Boosting using entropy measures for robust object detection

Recently, boosting has come to be used widely in object-detection applications because of its impressive performance in both speed and accuracy. However, learning weak classifiers which is one of the most significant tasks in using boosting is left to ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICML '09: Proceedings of the 26th Annual International Conference on Machine Learning

June 2009

1331 pages

ISBN:9781605585161

DOI:10.1145/1553374

General Chair:
Andrea Danyluk
Williams College
,
Program Chairs:
Léon Bottou
NEC Laboratories America
,
Michael Littman
Rutgers University

Copyright © 2009 Copyright 2009 by the author(s)/owner(s).

Sponsors

NSF
Microsoft Research: Microsoft Research
MITACS

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 June 2009

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Funding Sources

Conference

ICML '09

Sponsor:

Microsoft Research

ICML '09: The 26th Annual International Conference on Machine Learning held in conjunction with the 2007 International Conference on Inductive Logic Programming

June 14 - 18, 2009

Quebec, Montreal, Canada

Acceptance Rates

Overall Acceptance Rate 140 of 548 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

22
Total Citations
View Citations
229
Total Downloads

Downloads (Last 12 months)4
Downloads (Last 6 weeks)0

Reflects downloads up to 10 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Wang XZhao Y(2024)Multi-Branch Line Fault Arc Detection Method Based on the Improved Northern Goshawk Optimization Adaptive Base Class LogitBoost AlgorithmEnergies10.3390/en1704095417:4(954)Online publication date: 19-Feb-2024
https://doi.org/10.3390/en17040954
Emami SMartínez-Muñoz G(2024)Condensed-gradient boostingInternational Journal of Machine Learning and Cybernetics10.1007/s13042-024-02279-0Online publication date: 23-Jul-2024
https://doi.org/10.1007/s13042-024-02279-0
Swapna MViswanadhula UAluvalu RVardharajan VKotecha K(2022)Bio-Signals in Medical Applications and Challenges Using Artificial IntelligenceJournal of Sensor and Actuator Networks10.3390/jsan1101001711:1(17)Online publication date: 25-Feb-2022
https://doi.org/10.3390/jsan11010017
Wang XYang PChen SLiu LZhao LGuo JSun MLi PDemartini GZuccon GCulpepper JHuang ZTong H(2021)Efficient Learning to Learn a Robust CTR Model for Web-scale Online Sponsored Search AdvertisingProceedings of the 30th ACM International Conference on Information & Knowledge Management10.1145/3459637.3481912(4203-4213)Online publication date: 26-Oct-2021
https://dl.acm.org/doi/10.1145/3459637.3481912
Wang XLi XYu JSun MLi P(2020)Improved Touch-screen Inputting Using Sequence-level Prediction GenerationProceedings of The Web Conference 202010.1145/3366423.3380080(3077-3083)Online publication date: 20-Apr-2020
https://dl.acm.org/doi/10.1145/3366423.3380080
Fan MHuang JZhuo ALi YLi PWang HZhu WTao DCheng XCui PRundensteiner ECarmel DHe QXu Yu J(2019)MONOPOLY: Learning to Price Public Facilities for Revaluing Private Properties with Large-Scale Urban DataProceedings of the 28th ACM International Conference on Information and Knowledge Management10.1145/3357384.3357810(2655-2663)Online publication date: 3-Nov-2019
https://dl.acm.org/doi/10.1145/3357384.3357810
Li DZhang JLi P(2019)Large Scale Semantic Indexing with Deep Level-wise Extreme Multi-label LearningThe World Wide Web Conference10.1145/3308558.3313636(950-960)Online publication date: 13-May-2019
https://dl.acm.org/doi/10.1145/3308558.3313636
Wang Z(2018)Multi-class HingeBoostMethods of Information in Medicine10.3414/ME11-02-002051:02(162-167)Online publication date: 19-Jan-2018
https://doi.org/10.3414/ME11-02-0020
Li DZhang JLi P(2018)Representation Learning for Question Classification via Topic Sparse Autoencoder and Entity Embedding2018 IEEE International Conference on Big Data (Big Data)10.1109/BigData.2018.8622331(126-133)Online publication date: Dec-2018
https://doi.org/10.1109/BigData.2018.8622331
Hu JLi P(2016)Improved and Scalable Bradley-Terry Model for Collaborative Ranking2016 IEEE 16th International Conference on Data Mining (ICDM)10.1109/ICDM.2016.0118(949-954)Online publication date: Dec-2016
https://doi.org/10.1109/ICDM.2016.0118
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents