research-article

trNon-greedy active learning for text categorization using convex ansductive experimental design

Authors: Kai Yu, Shenghuo Zhu, Wei Xu, Yihong GongAuthors Info & Claims

SIGIR '08: Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval

Pages 635 - 642

https://doi.org/10.1145/1390334.1390442

Published: 20 July 2008 Publication History

Abstract

In this paper we propose a non-greedy active learning method for text categorization using least-squares support vector machines (LSSVM). Our work is based on transductive experimental design (TED), an active learning formulation that effectively explores the information of unlabeled data. Despite its appealing properties, the optimization problem is however NP-hard and thus--like most of other active learning methods--a greedy sequential strategy to select one data example after another was suggested to find a suboptimum. In this paper we formulate the problem into a continuous optimization problem and prove its convexity, meaning that a set of data examples can be selected with a guarantee of global optimum. We also develop an iterative algorithm to efficiently solve the optimization problem, which turns out to be very easy-to-implement. Our text categorization experiments on two text corpora empirically demonstrated that the new active learning algorithm outperforms the sequential greedy algorithm, and is promising for active text categorization applications.

References

[1]

A. C. Atkinson and A. N. Donev. Optimum experiment designs. Oxford Statistical Science Series. Oxford University Press, 1992.

[2]

O. Chapelle. Active learning for Parzen window classifier. In Proceedings of the Tenth International Workshop on Artificial Intelligence and Statistics, pages 49--56, 2005.

[3]

D. Cohn and Z. Ghahramani. Active learning with statistical models. Journal of Arti¯cial Intelligence Research, 4:129--145, 1996.

Digital Library

[4]

D. Donoho. For most large underdetermined systems of linear equations, the minimal l1-norm solution is also the sparsest solution. Communications on Pure and Applied Mathematics, 59(6), 2006.

[5]

Y. Freund, H. S. Seung, E. Shamir, and N. Tishby. Selective sampling using the query by committee algorithm. Machine Learning, 28(2-3):133--168, 1997.

Digital Library

[6]

C. Guestrin, A. Krause, and A. Singh. Near-optimal sensor placements in gaussian processes. In Proc. of the International Conference on Machine Learning (ICML), 2005.

Digital Library

[7]

X. He, W. Min, D. Cai, and K. Zhou. Laplacian optimal design for image retrieval. In ACM SIGIR Conference, 2007.

Digital Library

[8]

S. C. H. Hoi, R. Jin, J. Zhu, and M. R. Lyu. Batch mode active learning and its application to medical image classi¯cation. In International Conference on Machine Learning (ICML), 2006.

Digital Library

[9]

D. D. Lewis, Y. Yang, T. Rose, and F. Li. RCV1: A new benchmark collection for text categorization research. Journal of Machine Learning Research, 2005.

Digital Library

[10]

D. MacKay. Information-based objective functions for active data selection. Neural Computation, 4(4):590--604, 1992.

Digital Library

[11]

D. MacKay. Information-based objective functions for active data selection. Neural Computation, 4(4):590--604, 1992.

Digital Library

[12]

A. Schein and L. Ungar. Optimality for active learning of logistic regression classi¯ers. Technical Report Technical Report MS-CIS-04-07, The University of Pennsylvania, Department of Computer and Information Science, 2004.

[13]

G. Schohn and D. Cohn. Less is more: Active learning with support vector machines. In International Conference on Machine Learning, 2000.

Digital Library

[14]

J. Suykens and J. Vandewalle. Least squares support vector machine classifiers. Neural Processing Letters, 1999.

Digital Library

[15]

R. Tibshirani. Regression shrinkage and selection via the lasso. J. Royal. Statist. Soc B, 58(1), 1996.

[16]

S. Tong and D. Koller. Support vector machine active learning with applications to text classification. Journal of Machine Learning Research, 2, 2001.

Digital Library

[17]

K. Yu, J. Bi, and V. Tresp. Active learning via transductive experimental design. In International Conference on Machine Learning (ICML), 2006.

Digital Library

[18]

J. Zhang and Y. Yang. Robustness of regularized linear classifcation methods in text categorization. In The 26th Annual International SIGIR Conference (SIGIR'99), 2003.

Digital Library

[19]

T. Zhang and F. J. Oles. Text categorization based on regularized linear classi¯cation methods. Information Retrieval, (4):5--31, 2001.

Digital Library

[20]

W. V. Zhang, X. He, B. Rey, and R. Jones. Query rewritting using active learning for sponsored search. In ACM SIGIR Conference, 2007.

Digital Library

Cited By

Li HDel Castillo E(2022)Optimal Design of Experiments on Riemannian ManifoldsJournal of the American Statistical Association10.1080/01621459.2022.2146587119:546(875-886)Online publication date: 12-Dec-2022
https://doi.org/10.1080/01621459.2022.2146587
Foggo BYu N(2021)Analyzing Data Selection Techniques with Tools from the Theory of Information Losses2021 IEEE International Conference on Big Data (Big Data)10.1109/BigData52589.2021.9671861(7-16)Online publication date: 15-Dec-2021
https://doi.org/10.1109/BigData52589.2021.9671861
Wan CJin FQiao ZZhang WYuan Y(2021)Unsupervised active learning with loss predictionNeural Computing and Applications10.1007/s00521-021-06480-y35:5(3587-3595)Online publication date: 13-Sep-2021
https://doi.org/10.1007/s00521-021-06480-y
Show More Cited By

Index Terms

trNon-greedy active learning for text categorization using convex ansductive experimental design
1. Information systems
  1. Information retrieval

Recommendations

Large-scale text categorization by batch mode active learning
WWW '06: Proceedings of the 15th international conference on World Wide Web

Large-scale text categorization is an important research topic for Web data mining. One of the challenges in large-scale text categorization is how to reduce the human efforts in labeling text documents for building reliable classification models. In ...
A Novel Active Learning Method Using SVM for Text Classification

Support vector machines (SVMs) are a popular class of supervised learning algorithms, and are particularly applicable to large and high-dimensional classification problems. Like most machine learning methods for data classification and information ...
Batch Mode Active Learning with Applications to Text Categorization and Image Retrieval

Most machine learning tasks in data classification and information retrieval require manually labeled data examples in the training stage. The goal of active learning is to select the most informative examples for manual labeling in these learning ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGIR '08: Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval

July 2008

934 pages

ISBN:9781605581644

DOI:10.1145/1390334

General Chairs:
Tat-Seng Chua
National University of Singapore
,
Mun-Kew Leong
National Library Board, Singapore
,
Program Chairs:
Syung Hyon Myaeng
Information and Communications University, Korea
,
Douglas W. Oard
University of Maryland, College Park, USA
,
Fabrizio Sebastiani
Consiglio Nazionale delle Ricerche, Italy

Copyright © 2008 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 July 2008

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SIGIR '08

Sponsor:

SIGIR '08: The 31st Annual International ACM SIGIR Conference

July 20 - 24, 2008

Singapore, Singapore

Acceptance Rates

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

22
Total Citations
View Citations
651
Total Downloads

Downloads (Last 12 months)9
Downloads (Last 6 weeks)0

Reflects downloads up to 04 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Li HDel Castillo E(2022)Optimal Design of Experiments on Riemannian ManifoldsJournal of the American Statistical Association10.1080/01621459.2022.2146587119:546(875-886)Online publication date: 12-Dec-2022
https://doi.org/10.1080/01621459.2022.2146587
Foggo BYu N(2021)Analyzing Data Selection Techniques with Tools from the Theory of Information Losses2021 IEEE International Conference on Big Data (Big Data)10.1109/BigData52589.2021.9671861(7-16)Online publication date: 15-Dec-2021
https://doi.org/10.1109/BigData52589.2021.9671861
Wan CJin FQiao ZZhang WYuan Y(2021)Unsupervised active learning with loss predictionNeural Computing and Applications10.1007/s00521-021-06480-y35:5(3587-3595)Online publication date: 13-Sep-2021
https://doi.org/10.1007/s00521-021-06480-y
Xiong XFan MYu CHong Z(2020)A Novel Active Learning Algorithm for Robust Image ClassificationIEEE Access10.1109/ACCESS.2020.29680828(71106-71116)Online publication date: 2020
https://doi.org/10.1109/ACCESS.2020.2968082
Li HDel Castillo ERunger G(2020)On active learning methods for manifold dataTEST10.1007/s11749-019-00694-yOnline publication date: 2-Jan-2020
https://doi.org/10.1007/s11749-019-00694-y
Xu LZhang CSingh SMarkovitch S(2017)Bridging video content and commentsProceedings of the Thirty-First AAAI Conference on Artificial Intelligence10.5555/3298239.3298473(1611-1617)Online publication date: 4-Feb-2017
https://dl.acm.org/doi/10.5555/3298239.3298473
Shi LShen Y(2016)Diversifying convex transductive experimental design for active learningProceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence10.5555/3060832.3060900(1997-2003)Online publication date: 9-Jul-2016
https://dl.acm.org/doi/10.5555/3060832.3060900
Wang HDu LZhou PShi LQian YShen Y(2015)Experimental Design with Multiple KernelsProceedings of the 2015 IEEE International Conference on Data Mining (ICDM)10.1109/ICDM.2015.107(419-428)Online publication date: 14-Nov-2015
https://dl.acm.org/doi/10.1109/ICDM.2015.107
Ji MZhao WLiu Z(2015)A statistical design approach to unsupervised codeword selection in image retrievalNeurocomputing10.1016/j.neucom.2014.10.030157(323-334)Online publication date: Jun-2015
https://doi.org/10.1016/j.neucom.2014.10.030
He ZChen CBu JWang CZhang LCai DHe X(2015)Unsupervised document summarization from data reconstruction perspectiveNeurocomputing10.1016/j.neucom.2014.07.046157(356-366)Online publication date: Jun-2015
https://doi.org/10.1016/j.neucom.2014.07.046
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents