Article

Recommending learning algorithms and their associated hyperparameters

Authors:

Michael R. Smith,

Logan Mitchell,

Christophe Giraud-Carrier,

Tony MartinezAuthors Info & Claims

MLAS'14: Proceedings of the 2014 International Conference on Meta-learning and Algorithm Selection - Volume 1201

Pages 39 - 40

Published: 19 September 2014 Publication History

Abstract

The success of machine learning on a given task depends on, among other things, which learning algorithm is selected and its associated hyperparameters. Selecting an appropriate learning algorithm and setting its hyperparameters for a given data set can be a challenging task, especially for users who are not experts in machine learning. Previous work has examined using meta-features to predict which learning algorithm and hyperparameters should be used. However, choosing a set of metafeatures that are predictive of algorithm performance is difficult. Here, we propose to apply collaborative filtering techniques to learning algorithm and hyperparameter selection, and find that doing so avoids determining which meta-features to use and outperforms traditional meta-learning approaches in many cases.

References

[1]

S. Ali and K.A. Smith, 'On Learning Algorithm Selection for Classification', Applied Soft Computing, 62, 119-138, (2006).

Digital Library

[2]

S. Ali and K.A. Smith-Miles, 'A Meta-learning Approach to Automatic Kernel Selection for Support Vector Machines', Neurocomputing, 70, 173-186, (2006).

[3]

J. Bergstra and Y. Bengio, 'Random search for hyper-parameter optimization', Journal of Machine Learning Research, 13, 281-305, (2012).

Digital Library

[4]

P. B. Brazdil, C. Soares, and J. Pinto Da Costa, 'Ranking learning algorithms: Using IBL and meta-learning on accuracy and time results', Machine Learning, 50(3), 251-277, (2003).

Digital Library

[5]

P. Brazdil and C. Giraud-Carrier and C. Soares and R. Vilalta, 'Metalearning: Applications to Data Mining', Springer, (2009).

Digital Library

[6]

M. S. Gashler, 'Waffles: A machine learning toolkit', Journal of Machine Learning Research, MLOSS 12, 2383-2387, (July 2011).

Digital Library

[7]

M. S. Gashler, M. R. Smith, R. Morris, and T. Martinez, 'Missing value imputation with unsupervised backpropagation', Computational Intelligence, Accepted, (2014).

[8]

M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann, and I. H. Witten, 'The weka data mining software: an update', SIGKDD Explorations Newsletter, 11(1), 10-18, (2009).

Digital Library

[9]

Y. Koren, R. Bell, and C. Volinsky, 'Matrix factorization techniques for recommender systems', Computer, 42(8), 30-37, (2009).

Digital Library

[10]

R. Leite, P. Brazdil, and J. Vanschoren, 'Selecting classification algorithms with active testing', in Machine Learning and Data Mining in Pattern Recognition, ed., Petra Perner, volume 7376 of Lecture Notes in Computer Science, 117-131, Springer Berlin / Heidelberg, (2012).

Digital Library

[11]

D. Li, J. Deogun, W. Spaulding, and B. Shuart, 'Towards missing data imputation: A study of fuzzy k-means clustering method', in Rough Sets and Current Trends in Computing, volume 3066 of Lecture Notes in Computer Science, 573-579, Springer Berlin / Heidelberg, (2004).

[12]

B. Pfahringer, H. Bensusan, and C. G. Giraud-Carrier, 'Meta-learning by landmarking various learning algorithms', in Proceedings of the 17th International Conference on Machine Learning, pp. 743-750, San Francisco, CA, USA, (2000). Morgan Kaufmann Publishers Inc.

Digital Library

[13]

M. Scholz, F. Kaplan, C. L. Guy, J. Kopka, and J. Selbig, 'Non-linear pca: a missing data approach', Bioinformatics, 21 (20), 3887-3895, (2005).

Digital Library

[14]

M. R. Smith, T. Martinez, and C. Giraud-Carrier, 'An instance level analysis of data complexity', Machine Learning, 95(2), 225-256, (2014).

Digital Library

[15]

J. Snoek, H. Larochelle, and R. Adams, 'Practical bayesian optimization of machine learning algorithms', in Advances in Neural Information Processing Systems 25, eds., F. Pereira, C.J.C. Burges, L. Bottou, and K.Q. Weinberger, 2951-2959, (2012).

Digital Library

[16]

C. Thornton, F. Hutter, H. H. Hoos, and K. Leyton-Brown, 'Auto-weka: combined selection and hyperparameter optimization of classification algorithms', in proceedings of the 19th International Conference on Knowledge Discoveryand Data Mining, pp. 847-855, (2013).

Digital Library

[17]

J. Vanschoren, H. Blockeel, B. Pfahringer, and G. Holmes, 'Experiment databases - a new way to share, organize and learn from experiments', Machine Learning, 87(2), 127-158, (2012).

Digital Library

Cited By

Anelli VDi Noia TDi Sciascio EPomo CRagone ABogers TSaid ABrusilovsky PTikk D(2019)On the discriminative power of hyper-parameters in cross-validation and how to choose themProceedings of the 13th ACM Conference on Recommender Systems10.1145/3298689.3347010(447-451)Online publication date: 10-Sep-2019
https://dl.acm.org/doi/10.1145/3298689.3347010
Yao YXiao ZWang BViswanath BZheng HZhao BUhlig SMaennel O(2017)Complexity vs. performanceProceedings of the 2017 Internet Measurement Conference10.1145/3131365.3131372(384-397)Online publication date: 1-Nov-2017
https://dl.acm.org/doi/10.1145/3131365.3131372

Recommending learning algorithms and their associated hyperparameters
1. Computing methodologies

Recommendations

Reinforcement learning algorithms: A brief survey
Highlights
- RL can be used to solve problems involving sequential decision-making.
- RL is based on trial-and-error learning through rewards and punishments.
- The ultimate goal of an RL agent is to maximize cumulative reward.
- RL agent tries ...
Abstract
Reinforcement Learning (RL) is a machine learning (ML) technique to learn sequential decision-making in complex problems. RL is inspired by trial-and-error based human/animal learning. It can learn an optimal policy autonomously with knowledge ...
Recommending Serendipitous Items using Transfer Learning
CIKM '18: Proceedings of the 27th ACM International Conference on Information and Knowledge Management

Most recommender algorithms are designed to suggest relevant items, but suggesting these items does not always result in user satisfaction. Therefore, the efforts in recommender systems recently shifted towards serendipity, but generating serendipitous ...
Meta-learning with adaptive hyperparameters
NIPS '20: Proceedings of the 34th International Conference on Neural Information Processing Systems

The ability to quickly learn and generalize from only few examples is an essential goal of few-shot learning. Meta-learning algorithms effectively tackle the problem by learning how to learn novel tasks. In particular, model-agnostic meta-learning (MAML) ...

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

MLAS'14: Proceedings of the 2014 International Conference on Meta-learning and Algorithm Selection - Volume 1201

September 2014

60 pages

ISBN:16130073

Editors:
Joaquin Vanschoren
Eindhoven University of Technology, The Netherlands
,
Pavel Brazdil
University of Porto, Portugal
,
Carlos Soares
University of Porto, Portugal
,
Lars Kotthoff
University College Cork, Ireland

Publisher

CEUR-WS.org

Aachen, Germany

Publication History

Published: 19 September 2014

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 17 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Anelli VDi Noia TDi Sciascio EPomo CRagone ABogers TSaid ABrusilovsky PTikk D(2019)On the discriminative power of hyper-parameters in cross-validation and how to choose themProceedings of the 13th ACM Conference on Recommender Systems10.1145/3298689.3347010(447-451)Online publication date: 10-Sep-2019
https://dl.acm.org/doi/10.1145/3298689.3347010
Yao YXiao ZWang BViswanath BZheng HZhao BUhlig SMaennel O(2017)Complexity vs. performanceProceedings of the 2017 Internet Measurement Conference10.1145/3131365.3131372(384-397)Online publication date: 1-Nov-2017
https://dl.acm.org/doi/10.1145/3131365.3131372

View Options

View options

Media

Figures

Other

Tables

View Table of Contents