Authors:
João Freitas
1
;
Nuno Lavado
1
and
Jorge Bernardino
2
Affiliations:
1
Coimbra Polytechnic – ISEC, Rua Pedro Nunes, 3030-199 Coimbra and Portugal
;
2
Coimbra Polytechnic – ISEC, Rua Pedro Nunes, 3030-199 Coimbra, Portugal, CISUC - Centre of Informatics and Systems of University of Coimbra, DEI, Polo 2, 3030-290 Coimbra and Portugal
Keyword(s):
Data Science, Machine Learning, AutoML, Auto-WEKA, OpenML, Benchmarking.
Related
Ontology
Subjects/Areas/Topics:
Business Analytics
;
Data Engineering
;
Predictive Modeling
Abstract:
Machine Learning model building is an important and complex task in Data Science but also a good target for automation as recently exploited by AutoML. In general, free and open-source packages offer a joint space of learning algorithms and their respective hyperparameter settings and an optimization method for model search and tuning. In this paper, Auto-WEKA’s performance has been tested by running it for short periods of time (5, 15 and 30 minutes) using a commodity machine and suitable datasets with a limited number of observations and features. Benchmarking was performed against the best human-generated solution available in OpenML for each selected dataset. We concluded that increasing the overall time budget available over the previous values didn’t improve significantly classifiers’ performance.