research-article

Ensemble Reduction via Logic Minimization

Authors:

R. D. (Shawn) BlantonAuthors Info & Claims

ACM Transactions on Design Automation of Electronic Systems (TODAES), Volume 21, Issue 4

Article No.: 67, Pages 1 - 17

https://doi.org/10.1145/2897515

Published: 27 May 2016 Publication History

Abstract

An ensemble of machine learning classifiers usually improves generalization performance and is useful for many applications. However, the extra memory storage and computational cost incurred from the combined models often limits their potential applications. In this article, we propose a new ensemble reduction method called CANOPY that significantly reduces memory storage and computations. CANOPY uses a technique from logic minimization for digital circuits to select and combine particular classification models from an initial pool in the form of a Boolean function, through which the reduced ensemble performs classification. Experiments on 20 UCI datasets demonstrate that CANOPY either outperforms or is very competitive with the initial ensemble and one state-of-the-art ensemble reduction method in terms of generalization error, and is superior to all existing reduction methods surveyed for identifying the smallest numbers of models in the reduced ensembles.

References

[1]

A. V. Assche and H. Blockeel. 2007. Seeing the forest through the trees: Learning a comprehensible model from an ensemble. In Proceedings of the 18th European Conference on Machine Learning. 418--429.

Digital Library

[2]

R. Bitirgen, E. Ipek, and J. F. Martinez. 2008. Coordinated management of multiple interacting resources in chip multiprocessors: A machine learning approach. In Proceedings of the 41st Annual IEEE/ACM International Symposium on Microarchitecture. 318--329.

Digital Library

[3]

R. D. Blanton, X. Li, K. Mai, D. Marculescu, R. Marculescu, J. Paramesh, J. Schneider, and D. E. Thomas. 2014. SLIC: Statistical learning in chip. In Proceedings of the 2014 14th International Symposium on Integrated Circuits (ISIC). 119--123.

[4]

R. K. Brayton, G. D. Hachtel, C. T. McMullen, and A. L. Sangiovanni-Vincentelli. 1984. Logic Minimization Algorithms for VLSI Synthesis. Kluwer Academic Publishers.

Digital Library

[5]

L. Breiman. 1996. Bagging predictors. Machine Learning 24, 2 (1996), 123--140.

[6]

L. Breiman. 2001. Random forests. Machine Learning 45, 1 (2001), 5--32.

Digital Library

[7]

L. Breiman, J. H. Friedman, R. A. Olshen, and C. J. Stone. 1984. Classification and Regression Trees. Chapman & Hall, NewYork.

[8]

C. Buciluǎ, R. Caruana, and A. Niculescu-Mizil. 2006. Model compression. In Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’06). 535--541.

Digital Library

[9]

R. Caruana, A. Niculescu-Mizil, G. Crew, and A. Ksikes. 2004. Ensemble selection from libraries of models. In Proceedings of the 21st International Conference on Machine Learning. 18.

Digital Library

[10]

C. Chou and R. Marculescu. 2010. Run-time task allocation considering user behavior in embedded multiprocessor networks-on-chip. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 29, 1 (2010), 78--91.

Digital Library

[11]

P. Domingos. 1997. Knowledge acquisition from examples via multiple models. In Proceedings of the 14th International Conference on Machine Learning. 98--106.

Digital Library

[12]

ESPRESSO. http://embedded.eecs.berkeley.edu/pubs/downloads/espresso/.

[13]

A. Frank and A. Asuncion. 2010. UCI machine learning repository. School of Information and Computer Science, University of California. Retrieved from http://archive.ics.uci.edu/ml.

[14]

D. Hernández-Lobato, G. Martínez-Muñoz, and A. Suárez. 2009. Statistical instance-based pruning in ensembles of independent classifiers. IEEE Transactions on Pattern Analysis and Machine Intelligence 31, 2 (2009), 364--369.

Digital Library

[15]

E. Ipek, O. Mutlu, J. F. Martinez, and R. Caruana. 2008. Self-optimizing memory controllers: A reinforcement learning approach. In Proceedings of the International Symposium on Computer Architecture. 39--50.

Digital Library

[16]

Z. Lu, X. Wu, X. Zhu, and J. Bongard. 2010. Ensemble pruning via individual contribution ordering. In Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’10). 871--880.

Digital Library

[17]

D. Margineantu and T. Dietterich. 1997. Pruning adaptive boosting. In Proceedings of the 14th International Conference on Machine Learning. 211--218.

Digital Library

[18]

G. Martínez-Muñoz and A. Suárez. 2004. Aggregation ordering in bagging. In Proceedings of the International Conference on Artificial Intelligence and Applications (IASTED). Acta Press. 258--263.

[19]

G. Martínez-Muñoz and A. Suárez. 2006. Pruning in ordered bagging ensembles. In Proceedings of the 23rd International Conference on Machine Learning. ACM, 609--616.

Digital Library

[20]

G. D. Micheli. 1994. Synthesis and Optimization of Digital Circuits. McGraw-Hill, Hightstown, NJ, 556--558.

Digital Library

[21]

D. Opitz and J. Shavlik. 1996. Generating accurate and diverse members of a neural-network ensemble. Advanceds in Neural Information Processing Systems 8 (1996), 535--541.

[22]

I. Partalas, G. Tsoumakas, and I. Vlahavas. 2008. Focused ensemble selection: A diversity-based method for greedy ensemble selection. In Proceedings of the 18th European Conference on Artificial Intelligence. IOS Press, Patras, Greece, 117--121.

Digital Library

[23]

X. Ren, M. Martin, and R. D. Blanton. 2015a. Improving accuracy of on-chip diagnosis via incremental learning. In Proceedings of the 2015 IEEE 33rd VLSI Test Symposium (VTS). 1--6.

[24]

X. Ren, V. G. Tavares, and R. D. Blanton. 2015b. Detection of illegitimate access to JTAG via statistical learning in chip. In Proceedings of the 2015 Design, Automation & Test in Europe Conference & Exhibition (DATE). 109--114.

Digital Library

[25]

G. Rizzoni. 1993. Principles and Applications of Electrical Engineering. Irwin, Homewood, IL, 616--640.

[26]

R. L. Rudell. 1986. Multiple-valued logic minimization for PLA synthesis. Memorandum No. UCB/ERL M86-65, UC Berkeley.

[27]

R. Salvador, A. Otero, J. Mora, E. de la Torre, T. Riesgo, and L. Sekanina. 2013. Self-reconfigurable evolvable hardware system for adaptive image processing. IEEE Transactions on Computers 1481--1493.

Digital Library

[28]

MATLAB. 2010. Statistic Toolbox, Version R2010a. The MathWorks, Inc. Retrieved from http://www.mathworks.com/.

[29]

W. N. Streat and Y. A. Kim. 2001. Streaming ensemble algorithm (SEA) for large-scale classification. In Proceedings of the 7th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’01). 377--382.

Digital Library

[30]

Design Compiler. http://www.synopsys.com/Tools/Implementation/RTLSynthesis/Pages/default.aspx.

[31]

C. Tamon and J. Xiang. 2000. On the boosting pruning problem. In Proceedings of the 11th European Conference on Machine Learning. Springer-Verlag, 404--412.

Digital Library

[32]

D. M. J. Tax and R. P. W. Duin. 2002. Using two-class classifiers for multiclass classification. In Proceedings International Conference on Pattern Recognition. 124--127.

[33]

K. M. Ting and I. H. Witten. 1999. Issues in stacked generalization. Journal of Articial Intelligence Research 10 (1999), 271--289.

Digital Library

[34]

B. Van Essen, C. Macaraeg, M. Gokhale, and R. Prenger. 2012. Accelerating a random forest classifier: Multi-core, GP-GPU, or FPGA? In Proceedings of the IEEE 20th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM’12). 232--239.

Digital Library

[35]

D. H. Wolpert. 1992. Stacked generalization. Neural Networks 5, 2 (1992), 241--259.

Digital Library

[36]

M. Won, H. Albalawi, X. Li, and D. E. Thomas. 2014. Low-power hardware implementation of movement decoding for brain computer interface with reduced-resolution discrete cosine transform. In Proceedings of the 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). 1626--1629.

[37]

F. Ye, F. Firouzi, Y. Yang, K. Chakrabarty, and M. B. Tahoori. 2014. On-chip voltage-droop prediction using support-vector machines. In Proceedings of the 2014 IEEE 32nd VLSI Test Symposium (VTS). 1--6.

[38]

P. Zhang, J. Li, P. Wang, B. Gao, X. Zhu, and L. Guo. 2011. Enabling fast prediction for ensemble models on data streams. In Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’11). 177--185.

Digital Library

[39]

P. Zhang, X. Zhu, J. Tan, and L. Guo. 2010. Classifier and cluster ensembles for mining concept drifting data streams. In Proceedings of the IEEE International Conference on Data Mining (ICDM’10). 1175--1180.

Digital Library

[40]

Y. Zhang, S. Burer, and W. N. Street. 2006. Ensemble pruning via semi-definite programming. Journal of Machine Learning Research 7 (2006), 1315--1338.

Digital Library

[41]

Z. H. Zhou, J. Wu, and W. Tang. 2002. Ensembling neural networks: Many could be better than all. Artificial Intelligence 137, 1 (2002), 239--263.

Digital Library

Cited By

Wang HWu ZWang XBian LJin H(2023)HardGBM: A Framework for Accurate and Hardware-Efficient Gradient Boosting MachinesIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems10.1109/TCAD.2022.321850942:7(2122-2135)Online publication date: 1-Jul-2023
https://dl.acm.org/doi/10.1109/TCAD.2022.3218509
Wang HLi JHe K(2019)Hierarchical Ensemble Reduction and Learning for Resource-constrained ComputingACM Transactions on Design Automation of Electronic Systems10.1145/336522425:1(1-21)Online publication date: 4-Dec-2019
https://dl.acm.org/doi/10.1145/3365224
Wang HHe K(2019)Improving Test and Diagnosis Efficiency through Ensemble Reduction and LearningACM Transactions on Design Automation of Electronic Systems10.1145/332875424:5(1-26)Online publication date: 5-Jun-2019
https://dl.acm.org/doi/10.1145/3328754
Show More Cited By

Index Terms

Ensemble Reduction via Logic Minimization

Recommendations

Improving Test and Diagnosis Efficiency through Ensemble Reduction and Learning

Machine learning is a powerful lever for developing, improving, and optimizing test methodologies to cope with the demand from the advanced nodes. Ensemble methods are a particular learning paradigm that uses multiple models to boost performance. In ...
Hierarchical Ensemble Reduction and Learning for Resource-constrained Computing

Generic tree ensembles (such as Random Forest, RF) rely on a substantial amount of individual models to attain desirable performance. The cost of maintaining a large ensemble could become prohibitive in applications where computing resources are ...
Classifier ensemble reduction using a modified firefly algorithm

We propose a FA variant for classifier ensemble reduction.It incorporates both accelerated attractiveness and evading strategies.The attractiveness search operation is directed by local and global best solutions.The evading mechanism leads the search to ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Design Automation of Electronic Systems

ACM Transactions on Design Automation of Electronic Systems Volume 21, Issue 4

September 2016

423 pages

ISSN:1084-4309

EISSN:1557-7309

DOI:10.1145/2939671

Editor:
Naehyuck Chang
Korea Advanced Institute of Science and Technology, Korea

Issue’s Table of Contents

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Journal Family

ACM Journals for the Design of Smart and Connected Systems

Publication History

Published: 27 May 2016

Accepted: 01 February 2016

Revised: 01 January 2016

Received: 01 October 2015

Published in TODAES Volume 21, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
206
Total Downloads

Downloads (Last 12 months)12
Downloads (Last 6 weeks)1

Reflects downloads up to 28 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Wang HWu ZWang XBian LJin H(2023)HardGBM: A Framework for Accurate and Hardware-Efficient Gradient Boosting MachinesIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems10.1109/TCAD.2022.321850942:7(2122-2135)Online publication date: 1-Jul-2023
https://dl.acm.org/doi/10.1109/TCAD.2022.3218509
Wang HLi JHe K(2019)Hierarchical Ensemble Reduction and Learning for Resource-constrained ComputingACM Transactions on Design Automation of Electronic Systems10.1145/336522425:1(1-21)Online publication date: 4-Dec-2019
https://dl.acm.org/doi/10.1145/3365224
Wang HHe K(2019)Improving Test and Diagnosis Efficiency through Ensemble Reduction and LearningACM Transactions on Design Automation of Electronic Systems10.1145/332875424:5(1-26)Online publication date: 5-Jun-2019
https://dl.acm.org/doi/10.1145/3328754
Wang HLi JHe KCai WShrivastava APasricha S(2018)Hierarchical ensemble learning for resource-aware FPGA computingProceedings of the International Conference on Hardware/Software Codesign and System Synthesis10.5555/3283568.3283586(1-2)Online publication date: 30-Sep-2018
https://dl.acm.org/doi/10.5555/3283568.3283586
Wang HLi JHe KCai W(2018)Work-in-Progress: Hierarchical Ensemble Learning for Resource-Aware FPGA Computing2018 International Conference on Hardware/Software Codesign and System Synthesis (CODES+ISSS)10.1109/CODESISSS.2018.8525906(1-2)Online publication date: Oct-2018
https://doi.org/10.1109/CODESISSS.2018.8525906
Kuo CLu Y(2017)Scheduling algorithm for parallel real-time tasks on multiprocessor systemsACM SIGAPP Applied Computing Review10.1145/3040575.304057716:4(14-24)Online publication date: 13-Jan-2017
https://dl.acm.org/doi/10.1145/3040575.3040577

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents