Abstract
Feature construction is essential for solving many complex learning problems. Unfortunately, the construction of features usually implies searching a very large space of possibilities and is often computationally demanding. In this work, we propose a case based approach to feature construction. Learning tasks are stored together with a corresponding set of constructed features in a case base and can be retrieved to speed up feature construction for new tasks. The essential part of our method is a new representation model for learning tasks and a corresponding distance measure. Learning tasks are compared using relevance weights on a common set of base features only. Therefore, the case base can be built and queried very efficiently. In this respect, our approach is unique and enables us to apply case based feature construction not only on a large scale, but also in distributed learning scenarios in which communication costs play an important role. We derive a distance measure for heterogeneous learning tasks by stating a set of necessary conditions. Although the conditions are quite basic, they constraint the set of applicable methods to a surprisingly small number.
Chapter PDF
Similar content being viewed by others
References
Blum, A.L., Langley, P.: Selection of relevant features and examples in machine learning. Artificial Intelligence, 245–271 (1997)
Dash, M., Liu, H.: Feature selection for classification. International Journal of Intelligent Data Analysis 1, 131–156 (1997)
Koller, D., Sahami, M.: Toward optimal feature selection. In: Proc. of the ICML, pp. 129–134 (1996)
Mierswa, I., Morik, K.: Automatic feature extraction for classifying audio data. Machine Learning Journal 58, 127–149 (2005)
Wolpert, D., Macready, W.: No free lunch theorems for optimisation. IEEE Trans. on Evolutionary Computation 1, 67–82 (1997)
Vilalta, R., Drissi, Y.: A perspective view and survey of meta-learning. Artificial Intelligence Review 18, 77–95 (2002)
Baxter, J.: Learning internal representations. In: Proc. of the eighth annual conference on Computational learning theory 1995, pp. 311–320. ACM Press, New York (1995)
Baxter, J.: A model of inductive bias learning. Journal of Artificial Intelligence Research 12, 149–198 (2000)
Ben-David, S., Schuller, R.: Exploiting task relatedness for multiple task learning. In: Proc. of the Sixteenth Annual Conference on Learning Theory 2003 (2003)
Thrun, S., O’Sullivan, J.: Discovering structure in multiple learning tasks: The TC algorithm. In: Saitta, L. (ed.) Proc. of the ICML, San Mateo, CA. Morgen Kaufmann, San Francisco (1996)
Quinlan, R.: Induction of decision trees. Machine Learning 1, 81–106 (1986)
Kira, K., Rendell, I.A.: The feature selection problem: Traditional methods and a new algoirthm. In: 10th National Conference on Artificial Intelligence, pp. 129–134. MIT Press, Cambridge (1992)
Vapnik, V.N.: The Nature of Statistical Learning Theory. Springer, New York (1995)
Schölkopf, B., Smola, A.J.: Learning with Kernels – Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press, Cambridge (2002)
Mierswa, I., Wurst, M.: Efficient feature construction by meta learning – guiding the search in meta hypothesis space. In: Proc. of the ICML Workshop on Meta Learning (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Mierswa, I., Wurst, M. (2005). Efficient Case Based Feature Construction. In: Gama, J., Camacho, R., Brazdil, P.B., Jorge, A.M., Torgo, L. (eds) Machine Learning: ECML 2005. ECML 2005. Lecture Notes in Computer Science(), vol 3720. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11564096_64
Download citation
DOI: https://doi.org/10.1007/11564096_64
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-29243-2
Online ISBN: 978-3-540-31692-3
eBook Packages: Computer ScienceComputer Science (R0)