article

A survey of kernels for structured data

Author:

Thomas GärtnerAuthors Info & Claims

ACM SIGKDD Explorations Newsletter, Volume 5, Issue 1

Pages 49 - 58

https://doi.org/10.1145/959242.959248

Published: 01 July 2003 Publication History

Abstract

Kernel methods in general and support vector machines in particular have been successful in various learning tasks on data represented in a single table. Much 'real-world' data, however, is structured - it has no natural representation in a single table. Usually, to apply kernel methods to 'real-world' data, extensive pre-processing is performed to embed the data into areal vector space and thus in a single table. This survey describes several approaches of defining positive definite kernels on structured instances directly.

References

[1]

N. Aronszajn. Theory of reproducing kernels. Transactions of the American Mathematical Society, 68, 1950.]]

[2]

K. Bennett and C. Campbell. Support vector machines: Hype or hallelujah? SIGKDD Explorations, 2(2), 2000. http://www.acm.org/sigs/sigkdd/explorations/issue2-2/bennett.pdf.]]

Digital Library

[3]

B. E. Boser, I. M. Guyon, and V. N. Vapnik. A training algorithm for optimal margin classifiers. In D. Haussler, editor, Proceedings of the 5th Annual ACM Workshop on Computational Learning Theory, pages 144--152. ACM Press, July 1992.]]

Digital Library

[4]

M. Collins and N. Duffy. Convolution kernels for natural language. In T. G. Dietterich, S. Becker, and Z. Ghahramani, editors, Advances in Neural Information Processing Systems, volume 14. MIT Press, 2002.]]

[5]

N. Cristianini and J. Shawe-Taylor. An Introduction to Support Vector Machines (and Other Kernel-Based Learning Methods). Cambridge University Press, 2000.]]

Digital Library

[6]

T. G. Dietterich, R. H. Lathrop, and T. Lozano-Pérez. Solving the multiple instance problem with axis-parallel rectangles. Artificial Intelligence, 89(1--2):31--71, 1997.]]

Digital Library

[7]

R. Durbin, S. Eddy, A. Krogh, and G. Mitchison. Biological Sequence Analysis: Probabilistic Models of Proteins and Nucleic Acids. Cambridge University Press, 1998.]]

[8]

S. Džeroski and N. Lavrač, editors. Relational Data Mining. Springer-Verlag, 2001.]]

Digital Library

[9]

S. Džeroski, S. Schulze-Kremer, K. Heidtke, K. Siems, D. Wettschereck, and H. Blockeel. Diterpene structure elucidation from 13C NMR spectra with inductive logic programming. Applied Artificial Intelligence, 12(5):363--383, July-Aug. 1998. Special Issue on First-Order Knowledge Discovery in Databases.]]

[10]

T. Gärtner. Exponential and geometric kernels for graphs. In NIPS Workshop on Unreal Data: Principles of Modeling Nonvectorial Data, 2002.]]

[11]

T. Gärtner, K. Driessens, and J. Ramon. Graph kernels and gaussian processes for relational reinforcement learning. In Proceedings of the 13th International Conference on Inductive Logic Programming, 2003.]]

[12]

T. Gärtner, P. A. Flach, A. Kowalczyk, and A. J. Smola. Multi-instance kernels. In C. Sammut and A. Hoffmann, editors, Proceedings of the 19th International Conference on Machine Learning, pages 179--186. Morgan Kaufmann, June 2002.]]

Digital Library

[13]

T. Gärtner, P. A. Flach, and S. Wrobel. On graph kernels: Hardness results and efficient alternatives. In Proceedings of the 16th Annual Conference on Computa tional Learning Theory and the 7th Kernel Workshop, 2003.]]

[14]

T. Gärtner, J. W. Lloyd, and P. A. Flach. Kernels for structured data. In Proceedings of the 12th International Conference on Inductive Logic Programming. Springer-Verlag, 2002.]]

Digital Library

[15]

D. Haussler. Convolution kernels on discrete structures. Technical report, Department of Computer Science, University of California at Santa Cruz, 1999.]]

[16]

T. Jaakkola, M. Diekhans, and D. Haussler. A discriminative framework for detecting remote protein homologies. Journal of Computational Biology, 7(1, 2), 2000.]]

[17]

T. Jaakkola and D. Haussler. Exploiting generative models in discriminative classifiers. In Advances in Neural Information Processing Systems, volume 10, 1999.]]

Digital Library

[18]

T. Jaakkola and D. Haussler. Probabilistic kernel regression models. In Proceedings of the 1999 Conference on AI and Statistics, 1999.]]

[19]

T. Joachims. Learning to Classify Text using Support Vector Machines. Kluwer Academic Publishers, 2002.]]

Digital Library

[20]

R. Karchin, K. Karplus, and D. Haussler. Classifying g-protein coupled receptors with support vector machines. Bioinformatics, 18(1):147--159, 2002.]]

[21]

H. Kashima and A. Inokuchi. Kernels for graph classification. In ICDM Workshop on Active Mining, 2002.]]

[22]

H. Kashima and T. Koyanagi. Kernels for semistructured data. In C. Sammut and A. Hoffmann, editors, Proceedings of the 19th International Conference on Machine Learning. Morgan Kaufmann, 2002.]]

Digital Library

[23]

H. Kashima, K. Tsuda, and A. Inokuchi. Marginalized kernels between labeled graphs. In Proceedings of the 20th International Conference on Machine Learning, 2003.]]

Digital Library

[24]

R. I. Kondor and J. Lafferty. Diffusion kernels on graphs and other discrete input spaces. In C. Sammut and A. Hoffmann, editors, Proceedings of the 19th International Conference on Machine Learning, pages 315--322. Morgan Kaufmann, 2002.]]

Digital Library

[25]

S. Kramer, N. Lavrač, and P. A. Flach. Propositionalization approaches to relational data mining. In Džeroski and Lavrač {8}, chapter 11.]]

Digital Library

[26]

M.-A. Krogel and S. Wrobel. Transformation-based learning using multirelational aggregation. In C. Rouveirol and M. Sebag, editors, Proceedings of the 11th International Conference on Inductive Logic Programming. Springer-Verlag, 2001.]]

Digital Library

[27]

C. Leslie, E. Eskin, andW. Noble. The spectrum kernel: A string kernel for svm protein classification. In Proceedings of the Pacific Symposium on Biocomputing, pages 564--575, 2002.]]

[28]

C. Leslie, E. Eskin, J. Weston, and W. Noble. Mismatch string kernels for svm protein classification. In S. Becker, S. Thrun, and K. Obermayer, editors, Advances in Neural Information Processing Systems, volume 15. MIT Press, 2003.]]

[29]

J. W. Lloyd. Logic for Learning. Springer-Verlag, 2002.]]

[30]

H. Lodhi, C. Saunders, J. Shawe-Taylor, N. Cristianini, and C. Watkins. Text classification using string kernels. Journal of Machine Learning Research, 2, 2002.]]

Digital Library

[31]

H. Lodhi, J. Shawe-Taylor, N. Christianini, and C. Watkins. Text classification using string kernels. In T. Leen, T. Dietterich, and V. Tresp, editors, Advances in Neural Information Processing Systems, volume 13. MIT Press, 2001.]]

[32]

K.-R. Müller, S. Mika, G. Rätsch, K. Tsuda, and B. Schölkopf. An introduction to kernel-based learning algorithms. IEEE Transactions on Neural Networks, 2(2), 2001.]]

Digital Library

[33]

G. Paass, E. Leopold, M. Larson, J. Kindermann, and S. Eickeler. Svm classification using sequences of phonemes and syllables. In T. Elomaa, H. Mannila, and H. Toivonen, editors, Proceedings of the 6th European Conference on Principles of Data Mining and Knowledge Discovery, pages 373--384. Springer-Verlag, 2002.]]

Digital Library

[34]

P. Pavlidis, T. Furey, M. Liberto, D. Haussler, and W. Grundy. Promoter region-based classification of genes. In Proceedings of the Pacific Symposium on Biocomputing, pages 151--163, 2001.]]

[35]

L. R. Rabiner. A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE, 77(2):257--285, Feb. 1989.]]

[36]

C. Saunders, J. Shawe-Taylor, and A. Vinokourov. String kernels, fisher kernels and finite state automata. In S. Becker, S. Thrun, and K. Obermayer, editors, Advances in Neural Information Processing Systems, volume 15. MIT Press, 2003.]]

[37]

B. Schölkopf and A. J. Smola. Learning with Kernels. MIT Press, 2002.]]

[38]

N. Smith and M. Gales. Speech recognition using SVMs. In T. Dietterich, S. Becker, and Z. Ghahramani, editors, Advances in Neural Information Processing Systems, volume 14. MIT Press, 2002.]]

[39]

K. Tsuda, M. Kawanabe, G. Rätsch, S. Sonnenburg, and K.-R. Müller. A new discriminative kernel from probabilistic models. In T. Dietterich, S. Becker, and Z. Ghahramani, editors, Advances in Neural Information Processing Systems, volume 14. MIT Press, 2002.]]

[40]

K. Tsuda, T. Kin, and K. Asai. Marginalized kernels for biological sequences. Bioinformatics, 2002.]]

[41]

V. Vapnik. The Nature of Statistical Learning Theory. Springer-Verlag, 1995.]]

Digital Library

[42]

J.-P. Vert. A tree kernel to analyze phylogenetic profiles. Bioinformatics, 2002.]]

[43]

J.-P. Vert and M. Kanehisa. Graph driven features extraction from microarray data using diffusion kernels and kernel cca. In S. Becker, S. Thrun, and K. Ober mayer, editors, Advances in Neural Information Processing Systems, volume 15. MIT Press, 2003.]]

[44]

S. Vishwanathan and A. Smola. Fast kernels for string and tree matching. In S. Becker, S. Thrun, and K. Obermayer, editors, Advances in Neural Information Processing Systems, volume 15. MIT Press, 2003.]]

[45]

C. Watkins. Dynamic alignment kernels. Technical report, Department of Computer Science, Royal Holloway, University of London, 1999.]]

[46]

C. Watkins. Kernels from matching operations. Technical report, Department of Computer Science, Royal Holloway, University of London, 1999.]]

[47]

A, Zien, G. Ratsch, S. Mika, B. Schölkopf, T. Lengauer, and K.-R. Muller. Engineering support vector machine kernels that recognize translation initiation sites. Bioinforrnatics, 16(9):799--807, 2000.]]

Cited By

Cui WFeng ZChen JXu XTian YZhao HWang C(2024)Long-Tailed Effect Study in Remote Sensing Semantic Segmentation Based on Graph Kernel PrinciplesRemote Sensing10.3390/rs1608139816:8(1398)Online publication date: 15-Apr-2024
https://doi.org/10.3390/rs16081398
Chen HGupta ASun YShroff N(2024)Model-Free Change Point Detection for Mixing ProcessesIEEE Open Journal of Control Systems10.1109/OJCSYS.2024.33985303(202-213)Online publication date: 2024
https://doi.org/10.1109/OJCSYS.2024.3398530
Münch MBohnsack KSchleif FVillmann T(2024)Data-distribution-informed Nyström approximation for structured data using vector quantization-based landmark determinationNeurocomputing10.1016/j.neucom.2024.128100596(128100)Online publication date: Sep-2024
https://doi.org/10.1016/j.neucom.2024.128100
Show More Cited By

Index Terms

A survey of kernels for structured data
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Supervised learning
        Supervised learning by classification

Index terms have been assigned to the content through auto-classification.

Recommendations

Kernels and Distances for Structured Data

This paper brings together two strands of machine learning of increasing importance: kernel methods and highly structured data. We propose a general method for constructing a kernel following the syntactic structure of the data, as defined by its type ...
Design of ETL Tool for Structured Data Based on Data Warehouse
CSAE '20: Proceedings of the 4th International Conference on Computer Science and Application Engineering

This paper takes the current business system of a mobile communication-equipment-chain sales-service-company as an example, and analyzes the problem that the data from multiple data sources cannot directly be loaded into the data warehouse by the ...
Evolutionary strategies for hyperparameters of support vector machines based on multi-scale radial basis function kernels

Kernel functions are used in support vector machines (SVM) to compute inner product in a higher dimensional feature space. SVM classification performance depends on the chosen kernel. The radial basis function (RBF) kernel is a distance-based kernel ...

Comments

Information & Contributors

Information

Published In

cover image ACM SIGKDD Explorations Newsletter

ACM SIGKDD Explorations Newsletter Volume 5, Issue 1

July 2003

101 pages

ISSN:1931-0145

EISSN:1931-0153

DOI:10.1145/959242

Issue’s Table of Contents

Copyright © 2003 Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 July 2003

Published in SIGKDD Volume 5, Issue 1

Check for updates

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

374
Total Citations
View Citations
4,179
Total Downloads

Downloads (Last 12 months)50
Downloads (Last 6 weeks)8

Reflects downloads up to 01 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Cui WFeng ZChen JXu XTian YZhao HWang C(2024)Long-Tailed Effect Study in Remote Sensing Semantic Segmentation Based on Graph Kernel PrinciplesRemote Sensing10.3390/rs1608139816:8(1398)Online publication date: 15-Apr-2024
https://doi.org/10.3390/rs16081398
Chen HGupta ASun YShroff N(2024)Model-Free Change Point Detection for Mixing ProcessesIEEE Open Journal of Control Systems10.1109/OJCSYS.2024.33985303(202-213)Online publication date: 2024
https://doi.org/10.1109/OJCSYS.2024.3398530
Münch MBohnsack KSchleif FVillmann T(2024)Data-distribution-informed Nyström approximation for structured data using vector quantization-based landmark determinationNeurocomputing10.1016/j.neucom.2024.128100596(128100)Online publication date: Sep-2024
https://doi.org/10.1016/j.neucom.2024.128100
Bonnier POberhauser HSzabó ZOh ANaumann TGloberson ASaenko KHardt MLevine S(2023)Kernelized cumulantsProceedings of the 37th International Conference on Neural Information Processing Systems10.5555/3666122.3666610(11049-11074)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.5555/3666122.3666610
Münch MRöder MSchleif FFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)Unlocking the Potential of Non-PSD Kernel Matrices: A Polar Decomposition-based Transformation for Improved Prediction ModelsProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3615102(1867-1876)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3615102
Nienkötter AJiang X(2023)Kernel-Based Generalized Median Computation for Consensus LearningIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2022.320256545:5(5872-5888)Online publication date: 1-May-2023
https://dl.acm.org/doi/10.1109/TPAMI.2022.3202565
Rebafka T(2023)Model-based clustering of multiple networks with a hierarchical algorithmStatistics and Computing10.1007/s11222-023-10329-w34:1Online publication date: 7-Nov-2023
https://dl.acm.org/doi/10.1007/s11222-023-10329-w
Pasa LNavarin NErb WSperduti A(2023)A unified framework for backpropagation-free soft and hard gated graph neural networksKnowledge and Information Systems10.1007/s10115-023-02024-z66:4(2393-2416)Online publication date: 26-Dec-2023
https://doi.org/10.1007/s10115-023-02024-z
Navarin NPasa LGallicchio CSperduti A(2023)An Untrained Neural Model for Fast and Accurate Graph ClassificationArtificial Neural Networks and Machine Learning – ICANN 202310.1007/978-3-031-44216-2_23(278-290)Online publication date: 22-Sep-2023
https://doi.org/10.1007/978-3-031-44216-2_23
Adamson DDeligkas AGusev VPotapov I(2023)The k-Centre Problem for Classes of Cyclic WordsSOFSEM 2023: Theory and Practice of Computer Science10.1007/978-3-031-23101-8_26(385-400)Online publication date: 15-Jan-2023
https://dl.acm.org/doi/10.1007/978-3-031-23101-8_26
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents