Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
SpringerLink
Log in

Graph Kernels from the Jensen-Shannon Divergence

  • Published:
Journal of Mathematical Imaging and Vision Aims and scope Submit manuscript

Abstract

Graph-based representations have been proved powerful in computer vision. The challenge that arises with large amounts of graph data is that of computationally burdensome edit distance computation. Graph kernels can be used to formulate efficient algorithms to deal with high dimensional data, and have been proved an elegant way to overcome this computational bottleneck. In this paper, we investigate whether the Jensen-Shannon divergence can be used as a means of establishing a graph kernel. The Jensen-Shannon kernel is nonextensive information theoretic kernel, and is defined using the entropy and mutual information computed from probability distributions over the structures being compared. To establish a Jensen-Shannon graph kernel, we explore two different approaches. The first of these is based on the von Neumann entropy associated with a graph. The second approach uses the Shannon entropy associated with the probability state vector for a steady state random walk on a graph. We compare the two resulting graph kernels for the problem of graph clustering. We use kernel principle components analysis (kPCA) to embed graphs into a feature space. Experimental results reveal that the method gives good classification results on graphs extracted both from an object recognition database and from an application in bioinformation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Anand, K., Bianconi, G., Severini, S.: Shannon and von Neumann entropy of random networks with heterogeneous expected degree. Phys. Rev. E 83, 036109 (2011)

    Article  MathSciNet  Google Scholar 

  2. Borgwardt, K.M., Kriegel, H.P.: Shortest-path kernels on graphs. In: Proceedings of IEEE International Conference on Data Mining, pp. 74–81 (2005)

    Chapter  Google Scholar 

  3. Braunstein, S., Ghosh, S., Severini, S.: The Laplacian of a graph as a density matrix: a basic combinatorial approach to separability of mixed states. Ann. Comb. 10, 291–317 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  4. Chung, F.R.K.: Spectral Graph Theory. American Mathematical Society, Providence (1997)

    MATH  Google Scholar 

  5. Desobry, F., Davy, M., Fitzgerald, W.: Density kernels on unordered sets for kernel-based signal processing. In: Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 417–420 (2003)

    Google Scholar 

  6. Emms, D., Wilson, R.C., Hancock, E.R.: Graph matching using the interference of continuous-time quantum walks. Pattern Recognit. 42, 985–1002 (2009)

    Article  MATH  Google Scholar 

  7. Emms, D., Wilson, R.C., Hancock, E.R.: Graph matching using the interference of discrete-time quantum walks. Image Vis. Comput. 27, 934–949 (2009)

    Article  Google Scholar 

  8. Feldman, D., Crutchfield, J.: Measures of statistical complexity: why? Phys. Lett. A 238, 244,252 (1998)

    Article  MathSciNet  Google Scholar 

  9. Ferrer, M., Valveny, E., Serratosa, F., Riesen, K., Bunke, H.: Generalized median graph computation by means of graph embedding in vector spaces. Pattern Recognit. 43, 1642–1655 (2010)

    Article  MATH  Google Scholar 

  10. Gärtner, T.: A survey of kernels for structured data. ACM Special Interest Group on Knowledge Discovery and Data Mining 5(1), 49–58 (2003)

    Google Scholar 

  11. Gell–Mann, M., Tsallis, C.: Nonextensive Entropy: Interdisciplinary Application. Oxford University Press, London (2004)

    Google Scholar 

  12. Godsil, C., Royle, G.: Algebraic Graph Theory. Springer, New York (2001)

    Book  MATH  Google Scholar 

  13. Han, L., Hancock, E.R., Wilson, R.C.: Characterizing graphs using approximate von Neumann entropy. In: Proceedings of the Iberian Conference on Pattern Recognition and Image Analysis, pp. 484–491 (2011)

    Chapter  Google Scholar 

  14. Havrda, M., Charát, F.: Quantification method of classification processes: concept of structural α-entropy. Kybernetika 3, 30–35 (1967)

    MathSciNet  MATH  Google Scholar 

  15. Jebara, T., Kondor, R.I., Howard, A.: Probability product kernels. J. Mach. Learn. Res. 5, 819–844 (2004)

    MathSciNet  MATH  Google Scholar 

  16. Kashima, H., Tsuda, K., Inokuchi, A.: Marginalized kernels between labeled graphs. In: Proceedings of International Conference on Machine Learning, pp. 321–328 (2003)

    Google Scholar 

  17. Ketkar, N.S., Holder, L.B., Cook, D.J.: Faster computation of the direct product kernel for graph classification. In: Proceedings of the IEEE Symposium on Computational Intelligence and Data Mining, pp. 267–274 (2009)

    Google Scholar 

  18. Kondor, R.I., Lafferty, J.D.: Diffusion kernels on graphs and other discrete input spaces. In: Proceedings of International Conference on Machine Learning, pp. 315–322 (2002)

    Google Scholar 

  19. Lafferty, J.D., Lebanon, G.: Diffusion kernels on statistical manifolds. J. Mach. Learn. Res. 6, 129–163 (2005)

    MathSciNet  MATH  Google Scholar 

  20. Lamberti, P., Majtey, A., Borras, A., Casas, M., Plastino, A.: Metric character of the quantum Jensen-Shannon divergence. Phys. Rev. A 77, 052311 (2008)

    Article  Google Scholar 

  21. Majtey, A., Borrás, A., Casas, M., Lamberti, P., Plastino, A.: Jensen-Shannon divergence as a measure of the degree of entanglement. Int. J. Quantum Inf. 6, 715–720 (2008)

    Article  Google Scholar 

  22. Majtey, A., Lamberti, P., Prato, D.: Jensen-Shannon divergence as a measure of distinguishability between mixed quantum states. Phys. Rev. A 72, 052310 (2005)

    Article  Google Scholar 

  23. Martins, A.F.T., Smith, N.A., Xing, E.P., Aguiar, P.M.Q., Figueiredo, M.A.T.: Nonextensive information theoretic kernels on measures. J. Mach. Learn. Res., 935–975 (2009)

  24. Passerini, F., Severini, S.: Quantifying complexity in networks: the von Neumann entropy. Int. J. Agent Technol. Syst. 1, 58–67 (2009)

    Article  Google Scholar 

  25. Qiu, H., Hancock, E.R.: Clustering and embedding using commute times. IEEE Trans. Pattern Anal. Mach. Intell. 29, 1873–1890 (2007)

    Article  Google Scholar 

  26. Ren, P., Aleksić, T., Emms, D., Wilson, R.C., Hancock, E.R.: Quantum walks, Ihara zeta functions and cospectrality in regular graphs. Quantum Inf. Process. 10, 405–417 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  27. Ren, P., Wilson, R.C., Hancock, E.R.: Graph characteristics from the Ihara zeta function. In: Proceedings of Joint IAPR International Workshops on Structural and Syntactic Pattern Recognition (SSPR) and Statistical Techniques in Pattern Recognition (SPR), pp. 257–266 (2008)

    Chapter  Google Scholar 

  28. Robles-Kelly, A., Hancock, E.R.: A Riemannian approach to graph embedding. Pattern Recognit. 40, 1042–1056 (2007)

    Article  MATH  Google Scholar 

  29. Schölkopf, B., Smola, A.: Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press, Cambridge (2002)

    Google Scholar 

  30. Schölkopf, B., Smola, A., Müller, K.: Kernel principal component analysis. Artif. Neural Netw., 583–588 (1997)

  31. Suau, P., Escolano, F.: Bayesian optimization of the scale saliency filter. Image Vis. Comput. 26, 1207–1218 (2008)

    Article  Google Scholar 

  32. Taylor, J.S., Cristianini, N.: Kernel Methods for Pattern Analysis. Cambridge University Press, Cambridge (2004)

    Book  Google Scholar 

  33. Torsello, A., Hancock, E.R.: Graph embedding using tree edit-union. Pattern Recognit. 40, 1393–1405 (2007)

    Article  MATH  Google Scholar 

  34. Torsello, A., Robles-Kelly, A., Hancock, E.R.: Discovering shape classes using tree edit-distance and pairwise clustering. Int. J. Comput. Vis. 72, 259–285 (2007)

    Article  Google Scholar 

  35. Wilson, R.C., Hancock, E.R., Luo, B.: Pattern vectors from algebraic graph theory. IEEE Trans. Pattern Anal. Mach. Intell. 27, 1112–1124 (2005)

    Article  Google Scholar 

  36. Xiao, B., Hancock, E.R., Wilson, R.C.: Graph characteristics from the heat kernel trace. Pattern Recognit. 42, 2589–2606 (2009)

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Deramore Lane, University of York, Heslington, York, YO10 5GH, UK

    Lu Bai & Edwin R. Hancock

Authors
  1. Lu Bai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Edwin R. Hancock
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lu Bai.

Additional information

Edwin R. Hancock is supported by a Royal Society Wolfson Research Merit Award.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Bai, L., Hancock, E.R. Graph Kernels from the Jensen-Shannon Divergence. J Math Imaging Vis 47, 60–69 (2013). https://doi.org/10.1007/s10851-012-0383-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10851-012-0383-6

Keywords

Search

Navigation