research-article

Substantial improvements in the set-covering projection classifier CHIRP (composite hypercubes on iterated random projections)

Authors:

Leland Wilkinson,

Tuan Nhon DangAuthors Info & Claims

ACM Transactions on Knowledge Discovery from Data (TKDD), Volume 6, Issue 4

Article No.: 19, Pages 1 - 18

https://doi.org/10.1145/2382577.2382583

Published: 18 December 2012 Publication History

Abstract

In Wilkinson et al. [2011] we introduced a new set-covering random projection classifier that achieved average error lower than that of other classifiers in the Weka platform. This classifier was based on an L^∞ norm distance function and exploited an iterative sequence of three stages (projecting, binning, and covering) to deal with the curse of dimensionality, computational complexity, and nonlinear separability. We now present substantial changes that improve robustness and reduce training and testing time by almost an order of magnitude without jeopardizing CHIRP's outstanding error performance.

References

[1]

Abdullah, M. R., Toh, K.-A., and Srinivasan, D. 2006. A framework for empirical classifiers comparison. In Proceedings of the Symposium on Industrial Electronics and Applications. IEEE.

[2]

Achlioptas, D. 2001. Database-Friendly random projections. In Proceedings of the 20th ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems (PODS'01). ACM, New York, 274--281.

Digital Library

[3]

Agrawal, R., Gehrke, J., Gunopulos, D., and Raghavan, P. 1998. Automatic subspace clustering of high dimensional data for data mining applications. In Proceedings of the ACM SIGMOD Conference on Management of Data. 94--105.

Digital Library

[4]

Aguilar, J., Riquelme, J., and Toro, M. 1998. Decision queue classifier for supervised learning using rotated hyperboxes. In Proceedings of the 6th Ibero-American Conference on AI: Progress in Artificial Intelligence. Lecture Notes in Computer Science, vol. 4045, Springer, 326--336.

Digital Library

[5]

Alpern, B. and Carter, L. 1991. The hyperbox. In Proceedings of the IEEE Information Visualization Conference. 133--134.

Digital Library

[6]

Anand, A., Wilkinson, L., and Tuan, D. N. 2009. An L-infinity norm visual classifier. In Proceedings of the International Conference on Data Mining (ICDM). 687--692.

Digital Library

[7]

Asuncion, A. and Newman, D. 2007. UCI machine learning repository. http://www.ics.uci.edu/~mlearn/MLRepository.html.

[8]

Bickel, P. and Levina, E. 2004. Some theory for Fisher's linear discriminant function, ‘naive Bayes’, and some alternatives when there are many more variables than observations. Bernoulli 10, 989--1010.

[9]

Breiman, L., Friedman, J., Olshen, R., and Stone, C. 1984. Classification and Regression Trees. Wadsworth, Belmont, CA.

[10]

Bu, S., Lakshmanan, L. V. S., and Ng, R. T. 2005. MDL summarization with holes. In Proceedings of the 31st International Conference on Very Large Data Bases (VLDB '05). VLDB Endowment, 433--444.

Digital Library

[11]

Dietterich, T. G. and Bakiri, G. 1995. Solving multiclass learning problems via error-correcting output codes. J. AI Res. 2, 263--286.

Digital Library

[12]

Flick, T. E., Jones, L. K., Priest, R. G., and Herman, C. 1990. Pattern classification using projection pursuit. Pattern Recogn. 23, 1367--1376.

Digital Library

[13]

Gao, B. 2002. Hyper-rectangle-based discriminative data generalization and applications in data. Ph.D. thesis, Simon Fraser University.

Digital Library

[14]

Gao, B. J. and Ester, M. 2006. Turning clusters into patterns: Rectangle-based discriminative data description. In Proceedings of the 6th International Conference on Data Mining (ICDM'06). IEEE Computer Society, 200--211.

Digital Library

[15]

Guo, Y., Hastie, T., and Tibshirani, R. 2005. Regularized discriminant analysis and its application in microarrays. Biostatist. 1, 1--18.

[16]

Hanczar, B., Hua, J., Sima, C., Weinstein, J., Bittner, M., and Dougherty, E. R. 2010. Small-sample precision of roc-related estimates. Bioinf. 26, 822--820.

Digital Library

[17]

Hand, D. 2010. Measuring classifier performance: A coherent alternative to the area under the roc curve. Mach. Learn. 77, 103--123.

Digital Library

[18]

Hastie, T., Tibshirani, R., and Friedman, J. H. 2001. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer.

[19]

Hegde, C. and Baraniuk, R. 2007. Random projections for manifold learning. In Proceedings of the Conference on Advances in Neural Information Processing Systems (NIPS'07). MIT Press, Cambridge, MA.

[20]

Hosking, J. 1990. L-Moments: Analysis and estimation of distributions using linear combinations of order statistics. J. Roy. Statist. Soc. B 52, 105--124.

[21]

Jimenez, L. O. and Landgrebe, D. A. 1995. Projection pursuit for high dimensional feature reduction: paralleland sequential approaches. In Proceedings of the Geoscience and Remote Sensing Symposium (IGARSS '95. Vol. 1. 148--150.

[22]

Johnson, W. B. and Lindenstrauss, J. 1984. Lipschitz mapping into Hilbert space. Contemp. Math. 26, 189--206.

[23]

King, R., Feng, C., and Sutherland, A. 1995. Statlog: Comparison of classification algorithms on large real-world problems. Appl. Artif. Intell. 9.

[24]

Lazarsfeld, P. F. and Henry, N. 1968. Latent Structure Analysis. Houghton Mifflin, Boston.

[25]

Lee, E.-K., Cook, D., Klinke, S., and Lumley, T. 2005. Projection pursuit for exploratory supervised classification. J. Comput. Graph. Statis. 14, 831--846.

[26]

Li, P. 2010. Robust LogitBoost and Adaptive Base Class (ABC) LogitBoost. In Proceedings of the IEEE UAI.

[27]

Li, P., Hastie, T. J., and Church, K. W. 2006. Very sparse random projections. In Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD'06). ACM, New York, 287--296.

Digital Library

[28]

Marchand, M. and Shawe-Taylor, J. 2002. The set covering machine. J. Mach. Learn. Res. 3, 723--746.

[29]

Mosteller, F. 1948. A k-sample slippage test for an extreme population. Ann. Math. Statist. 19, 58--65.

[30]

Pu, K. Q. and Mendelzon, A. O. 2005. Concise descriptions of subsets of structured sets. ACM Trans. Datab. Syst. 30, 1, 211--248.

Digital Library

[31]

Quinlan, J. R. 1993. C4.5: Programs for Machine Learning (Morgan Kaufmann Series in Machine Learning). Morgan Kaufmann.

Digital Library

[32]

Rivest, R. L. 1987. Learning decision lists. Mach. Learn. 2, 229--246.

Digital Library

[33]

Scott, D. W. 1979. On optimal and data-based histograms. Biometrika 66, 605--610.

[34]

Silverman, B. 1986. Density Estimation for Statistics and Data Analysis. Chapman & Hall, New York.

[35]

Simpson, P. K. 1992. Fuzzy min-max neural network, i: Classification. IEEE Trans. Neural Netw. 3, 776--786.

Digital Library

[36]

Sokolova, M., Japkowicz, N., Marchand, M., and Shawe-taylor, J. 2003. The decision list machine. In Advances in Neural Information Processing Systems 15, MIT Press, 921--928.

[37]

Statnikov, A., Aliferis, C. F., Tsamardinos, I., Hardin, D., and Levy, S. 2005. A comprehensive evaluation of multicategory classification methods for microarray gene expression cancer diagnosis. Bioinf. 21, 5, 631--643.

Digital Library

[38]

Sturges, H. A. 1926. The choice of a class interval. J. Amer. Statist. Assoc. 21, 65--66.

[39]

Tibshirani, R. 1995. Regression shrinkage and selection via the lasso. J. Roy. Statist. Soc. B 58, 267--288.

[40]

Tukey, J. 1959. A quick, compact, two-sample test to Duckworth's specifications. Technometrics, 31--48.

[41]

Üney, F. and Türkay, M. 2006. A mixed-integer programming approach to multi-class data classification problem. Euro, J. Oper. Res. 173, 910--920.

[42]

Wainer, H. 1976. Estimating coefficients in linear models: It don't make no nevermind. Psychol. Bull. 83, 2, 213--217.

[43]

Wand, M. P. 1997. Data-Based choice of histogram bin width. The Amer. Statist. 51, 1, 59--64.

[44]

Wilkinson, L., Anand, A., and Dang, T. 2011. CHIRP: A new classier based on composite hypercubes on iterated random projections. In Proceedings of ACM Conference on Knowledge Discovery and Data Mining.

Digital Library

[45]

Witten, I. H., Frank, E., Trigg, L., Hall, M., Holmes, G., and Cunningham, S. J. 1999. Weka: Practical machine learning tools and techniques with Java implementations. In Proceedings of the ICONIP/ANZIIS/ANNES'99 Workshop on Emerging Knowledge Engineering and Connectionist-Based Information Systems. 192--196.

Cited By

Pham B(2018)A Novel Classifier Based on Composite Hyper-cubes on Iterated Random Projections for Assessment of Landslide SusceptibilityJournal of the Geological Society of India10.1007/s12594-018-0862-591:3(355-362)Online publication date: 21-Mar-2018
https://doi.org/10.1007/s12594-018-0862-5

Index Terms

Substantial improvements in the set-covering projection classifier CHIRP (composite hypercubes on iterated random projections)
1. Information systems
  1. Information systems applications

Recommendations

CHIRP: a new classifier based on composite hypercubes on iterated random projections
KDD '11: Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining

We introduce a classifier based on the L-infinity norm. This classifier, called CHIRP, is an iterative sequence of three stages (projecting, binning, and covering) that are designed to deal with the curse of dimensionality, computational complexity, and ...
Random Projections of Smooth Manifolds

We propose a new approach for nonadaptive dimensionality reduction of manifold-modeled data, demonstrating that a small number of random linear projections can preserve key information about a manifold-modeled signal. We center our analysis on the ...
Random projections as regularizers: learning a linear discriminant from fewer observations than dimensions

We prove theoretical guarantees for an averaging-ensemble of randomly projected Fisher linear discriminant classifiers, focusing on the case when there are fewer training observations than data dimensions. The specific form and simplicity of this ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Knowledge Discovery from Data

ACM Transactions on Knowledge Discovery from Data Volume 6, Issue 4

Special Issue on the Best of SIGKDD 2011

December 2012

141 pages

ISSN:1556-4681

EISSN:1556-472X

DOI:10.1145/2382577

Issue’s Table of Contents

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 December 2012

Accepted: 01 October 2012

Revised: 01 October 2012

Received: 01 October 2011

Published in TKDD Volume 6, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
215
Total Downloads

Downloads (Last 12 months)1
Downloads (Last 6 weeks)0

Reflects downloads up to 14 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Pham B(2018)A Novel Classifier Based on Composite Hyper-cubes on Iterated Random Projections for Assessment of Landslide SusceptibilityJournal of the Geological Society of India10.1007/s12594-018-0862-591:3(355-362)Online publication date: 21-Mar-2018
https://doi.org/10.1007/s12594-018-0862-5

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents