research-article

Nonlinear Diffusion for Community Detection and Semi-Supervised Learning

Authors:

Rania Ibrahim and

David GleichAuthors Info & Claims

WWW '19: The World Wide Web Conference

May 2019

Pages 739 - 750

https://doi.org/10.1145/3308558.3313483

Published: 13 May 2019 Publication History

Abstract

Diffusions, such as the heat kernel diffusion and the PageRank vector, and their relatives are widely used graph mining primitives that have been successful in a variety of contexts including community detection and semi-supervised learning. The majority of existing methods and methodology involves linear diffusions, which then yield simple algorithms involving repeated matrix-vector operations. Recent work, however, has shown that sophisticated and complicated techniques based on network embeddings and neural networks can give empirical results superior to those based on linear diffusions. In this paper, we illustrate a class of nonlinear graph diffusions that are competitive with state of the art embedding techniques and outperform classic diffusions. Our new methods enjoy much of the simplicity underlying classic diffusion methods as well. Formally, they are based on nonlinear dynamical systems that can be realized with an implementation akin to applying a nonlinear function after each matrix-vector product in a classic diffusion. This framework also enables us to easily integrate results from multiple data representations in a principled fashion. Furthermore, we have some theoretical relationships that suggest choices of the nonlinear term. We demonstrate the benefits of these techniques on a variety of synthetic and real-world data.

References

[1]

S. Amghibech. 2003. Eigenvalues of the Discrete p-Laplacian for Graphs. Ars Comb. 67(2003).

[2]

Reid Andersen, Fan Chung, and Kevin Lang. 2006. Local Graph Partitioning using PageRank Vectors. In Proceedings of the 47th Annual IEEE Symposium on Foundations of Computer Science. IEEE, IEEE, Washington, D.C., USA, 475-486. http://www.math.ucsd.edu/~fan/wp/localpartition.pdf

Digital Library

[3]

Mikhail Belkin, Partha Niyogi, and Vikas Sindhwani. 2006. Manifold regularization: A geometric framework for learning from labeled and unlabeled examples. Journal of machine learning research 7, Nov (2006), 2399-2434.

Digital Library

[4]

Philippe Be´nilan and Michael G Crandall. 1981. The continuous dependence on f of solutions of u t- ?f (u)= 0. Indiana University Mathematics Journal 30, 2 (1981), 161-177.

[5]

Austin Benson, David F. Gleich, and Jure Leskovec. 2016. Higher-order organization of complex networks. Submitted. Science 353, 6295 (2016), 163-166.

[6]

James G Berryman and Charles J Holland. 1978. Nonlinear diffusion problem arising in plasma physics. Physical Review Letters 40, 26 (1978), 1720.

[7]

M. Bonforte, J. Dolbeault, G. Grillo, and J. L. Vazquez. 2010. Sharp rates of decay of solutions to the nonlinear fast diffusion equation via functional inequalities. Proceedings of the National Academy of Sciences 107, 38 (sep 2010), 16459-16464.

[8]

Nick Bridle and Xiaojin Zhu. 2013. p-voltages: Laplacian regularization for semi-supervised learning on high-dimensional data. In Eleventh Workshop on Mining and Learning with Graphs (MLG2013). ACM, New York, NY, USA.

[9]

Thomas Bühler and Matthias Hein. 2009. Spectral clustering based on the graph p-Laplacian. In Proceedings of the 26th Annual International Conference on Machine Learning. ACM, ACM, New York, NY, USA, 81-88.

Digital Library

[10]

O Chapelle and B Sch. 2006. olkopf, and A. Zien, Semi-Supervised Learning, vol. 2.

[11]

Fan Chung. 2005. Laplacians and the Cheeger inequality for directed graphs. Annals of Combinatorics 9, 1 (2005), 1-19.

[12]

Fan Chung. 2007. Four proofs for the Cheeger inequality and graph partition algorithms. In Proceedings of ICCM, Vol. 2. ScienTech, Mason, OH, USA, 378.

[13]

Fan Chung. 2007. The heat kernel as the pagerank of a graph. Proceedings of the National Academy of Sciences 104, 50(2007), 19735-19740.

[14]

Fan Chung and Olivia Simpson. 2013. Solving Linear Systems with Boundary Conditions Using Heat Kernel Pagerank. In Algorithms and Models for the Web Graph. Springer, 203-219.

[15]

Fan R. L. Chung. 1992. Spectral Graph Theory. American Mathematical Society, Providence, RI, USA.

Digital Library

[16]

George Cybenko. 1989. Approximation by superpositions of a sigmoidal function. Mathematics of control, signals and systems 2, 4 (1989), 303-314.

[17]

Rumi Ghosh, Shang-hua Teng, Kristina Lerman, and Xiaoran Yan. 2014. The interplay between dynamics and networks: centrality, communities, and cheeger inequality. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, ACM, New York, NY, USA, 1406-1415.

Digital Library

[18]

David F. Gleich. 2015. PageRank beyond the Web. SIAM Rev. 57, 3 (August 2015), 321-363.

Digital Library

[19]

David F Gleich and Michael W Mahoney. 2015. Using local spectral methods to robustify graph-based learning algorithms. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, ACM, New York, NY, USA, 359-368.

Digital Library

[20]

David F. Gleich and Michael W. Mahoney. 2016. Mining large graphs. In Handbook of Big Data, Peter Bühlmann, Petros Drineas, Michael Kane, and Mark van de Laan (Eds.). CRC Press, Boca Raton, Florida, USA, 191-220.

[21]

David F. Gleich and Ryan A. Rossi. 2014. A Dynamical System for PageRank with Time-Dependent Teleportation. Internet Mathematics 10, 1-2 (June 2014), 188-217.

[22]

Aditya Grover and Jure Leskovec. 2016. Node2Vec: Scalable Feature Learning for Networks. In Proceedings of the 22Nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining(KDD '16). ACM, New York, NY, USA, 855-864.

Digital Library

[23]

FM Henderson and RA Wooding. 1964. Overland flow and groundwater flow from a steady rainfall of finite duration. Journal of Geophysical Research 69, 8 (1964), 1531-1540.

[24]

Thorsten Joachims. 2003. Transductive Learning via Spectral Graph Partitioning. In ICML. ACM, New York, NY, USA, 290-297. http://www.aaai.org/Papers/ICML/2003/ICML03-040.pdf

Digital Library

[25]

J. R. KING. 1988. Extremely High Concentration Dopant Diffusion in Silicon. IMA Journal of Applied Mathematics 40, 3 (1988), 163-181.

[26]

Thomas N Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 abs/1609.02907 (2016).

[27]

Kyle Kloster and David F Gleich. 2014. Heat kernel based community detection. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, ACM, New York, NY, USA, 1386-1395.

Digital Library

[28]

Andrea Lancichinetti, Santo Fortunato, and Filippo Radicchi. 2008. Benchmark graphs for testing community detection algorithms. Physical review E 78, 4 (2008), 046110.

[29]

LS Leibenzon. 1930. The motion of a gas in a porous medium. Complete Works, Vol. 2, Acad. Sciences URSS, Moscow,(Russian) 63 (1930), 8-9.

[30]

Jure Leskovec and Andrej Krevl. 2014. {SNAP Datasets}:{Stanford} Large Network Dataset Collection. http://snap.stanford.edu/data.

[31]

Qimai Li, Zhichao Han, and Xiao-Ming Wu. 2018. Deeper Insights Into Graph Convolutional Networks for Semi-Supervised Learning. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018. AAAI Press, Palo Alto, CA, USA, 3538-3545.

[32]

Yixuan Li, Kun He, David Bindel, and John E. Hopcroft. 2015. Uncovering the Small Community Structure in Large Networks: A Local Spectral Approach. In Proceedings of the 24th International Conference on World Wide Web(WWW '15). International World Wide Web Conferences Steering Committee, Geneva, Switzerland, 658-668.

Digital Library

[33]

Qing Lu and Lise Getoor. 2003. Link-based classification. In Proceedings of the 20th International Conference on Machine Learning (ICML-03). ACM, New York, NY, USA, 496-503.

Digital Library

[34]

Ulrike Luxburg. 2007. A tutorial on spectral clustering. Statistics and Computing 17, 4 (December 2007), 395-416.

Digital Library

[35]

Michael W. Mahoney, Lorenzo Orecchia, and Nisheeth K. Vishnoi. 2012. A Local Spectral Method for Graphs: With Applications to Improving Graph Partitions and Exploring Data Graphs Locally. Journal of Machine Learning Research 13 (August 2012), 2339-2365. http://www.jmlr.org/papers/volume13/mahoney12a/mahoney12a.pdf

Digital Library

[36]

Boaz Nadler, Nathan Srebro, and Xueyuan Zhou. 2009. Semi-supervised learning with the graph laplacian: The limit of infinite unlabelled data. Advances in neural information processing systems 22 (2009), 1330-1338.

Digital Library

[37]

Lawrence Page, Sergey Brin, Rajeev Motwani, and Terry Winograd. 1999. The PageRank Citation Ranking: Bringing Order to the Web. Technical Report 1999-66. Stanford University. http://dbpubs.stanford.edu:8090/pub/1999-66

[38]

Jia-Yu Pan, Hyung-Jeong Yang, Christos Faloutsos, and Pinar Duygulu. 2004. Automatic multimedia cross-modal correlation discovery. In KDD '04: Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, New York, NY, USA, 653-658.

Digital Library

[39]

Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. 2014. Deepwalk: Online learning of social representations. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, ACM, New York, NY, USA, 701-710.

Digital Library

[40]

A. P. Riascos and Jose´ L. Mateos. 2014. Fractional dynamics on networks: Emergence of anomalous diffusion and Le´vy flights. Physical Review E 90, 3 (sep 2014).

[41]

Sam T Roweis and Lawrence K Saul. 2000. Nonlinear dimensionality reduction by locally linear embedding. science 290, 5500 (2000), 2323-2326.

[42]

F. Scarselli, M. Gori, Ah Chung Tsoi, M. Hagenbuchner, and G. Monfardini. 2009. The Graph Neural Network Model. IEEE Transactions on Neural Networks 20, 1 (jan 2009), 61-80.

Digital Library

[43]

Satu Elisa Schaeffer. 2007. Graph clustering. Computer Science Review 1, 1 (2007), 27-64.

Digital Library

[44]

Lawrence F Shampine, Skip Thompson, JA Kierzenka, and GD Byrne. 2005. Non-negative solutions of ODEs. Appl. Math. Comput. 170, 1 (2005), 556-569.

[45]

Dane Taylor, Sean A. Myers, Aaron Clauset, Mason A. Porter, and Peter J. Mucha. 2017. Eigenvector-Based Centrality Measures for Temporal Networks. Multiscale Modeling & Simulation 15, 1 (jan 2017), 537-574.

Digital Library

[46]

Joshua B Tenenbaum, Vin De Silva, and John C Langford. 2000. A global geometric framework for nonlinear dimensionality reduction. science 290, 5500 (2000), 2319-2323.

[47]

JL Vázquez. 2005. Smoothing and decay estimates for nonlinear parabolic equations of porous medium type. preprint (2005).

[48]

Juan Luis Vázquez. 2007. Perspectives in nonlinear diffusion: between analysis, physics and geometry. In Proceedings of the International Congress of Mathematicians Madrid, August 22-30, 2006. American Mathematical Society, Providence, RI, USA, 609-634.

[49]

Juan Luis Vázquez. 2017. The Mathematical Theories of Diffusion: Nonlinear and Fractional Diffusion. In Nonlocal and Nonlinear Diffusions and Interactions: New Methods and Directions. Springer International Publishing, New York, NY, USA, 205-278.

[50]

Luke N. Veldt, David F. Gleich, and Michael W. Mahoney. 2016. A Simple and Strongly-Local Flow-Based Method for Cut Improvement. In International Conference on Machine Learning. ACM, New York, NY, USA, 1938-1947. http://jmlr.org/proceedings/papers/v48/veldt16.html

Digital Library

[51]

James Voss, Mikhail Belkin, and Luis Rademacher. 2016. The Hidden Convexity of Spectral Clustering. In AAAI. AAAI Press, Palo Alto, CA, USA, 2108-2114. https://www.aaai.org/ocs/index.php/AAAI/AAAI16/paper/view/12307/11849

Digital Library

[52]

Di Wang, Kimon Fountoulakis, Monika Henzinger, Michael W. Mahoney, and Satish Rao. 2017. Capacity Releasing Diffusion for Speed and Locality. In Proceedings of the 34th International Conference on Machine Learning(Proceedings of Machine Learning Research), Doina Precup and Yee Whye Teh (Eds.), Vol. 70. PMLR, International Convention Centre, Sydney, Australia, 3598-3607. http://proceedings.mlr.press/v70/wang17b.html

Digital Library

[53]

Jason Weston, Fre´de´ric Ratle, Hossein Mobahi, and Ronan Collobert. 2012. Deep learning via semi-supervised embedding. In Neural Networks: Tricks of the Trade. Springer, New York, NY, USA, 639-655.

[54]

Han Xiao, Kashif Rasul, and Roland Vollgraf. 2017. Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms. CoRR abs/1708.07747(2017).

[55]

Xiaoran Yan, Shang-hua Teng, Kristina Lerman, and Rumi Ghosh. 2016. Capturing the interplay of dynamics and networks through parameterizations of Laplacian operators. PeerJ Computer Science 2(2016), e57.

[56]

Zhilin Yang, William Cohen, and Ruslan Salakhudinov. 2016. Revisiting Semi-Supervised Learning with Graph Embeddings. In International Conference on Machine Learning. ACM, New York, NY, USA, 40-48.

Digital Library

[57]

Denny Zhou, Olivier Bousquet, Thomas N Lal, Jason Weston, and Bernhard Schölkopf. 2004. Learning with local and global consistency. In Advances in neural information processing systems. MIT Press, Cambridge, MA, USA, 321-328.

Digital Library

[58]

Xiaojin Zhu, Zoubin Ghahramani, and John D Lafferty. 2003. Semi-supervised learning using gaussian fields and harmonic functions. In Proceedings of the 20th International conference on Machine learning (ICML-03). ACM, New York, NY, USA, 912-919.

Digital Library

Cited By

Zhong HZhang YYan CXuan ZYu TZhang JYing SGao Y(2024)Penalized Flow Hypergraph Local ClusteringIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2023.3319019(1-16)Online publication date: 2024
https://doi.org/10.1109/TKDE.2023.3319019
Chen YYe WLi D(2023)Spectral Clustering Community Detection Algorithm Based on Point-Wise Mutual Information Graph KernelEntropy10.3390/e2512161725:12(1617)Online publication date: 3-Dec-2023
https://doi.org/10.3390/e25121617
Ni LGe JZhang YLuo WSheng V(2023)Semi-Supervised Local Community DetectionIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2023.3290095(1-17)Online publication date: 2023
https://doi.org/10.1109/TKDE.2023.3290095
Show More Cited By

Recommendations

A Generic Diffusion Kernel for Semi-supervised Learning
ISNN '08: Proceedings of the 5th international symposium on Neural Networks: Advances in Neural Networks

In this paper, we present a generic diffusion kernel for graph-based semi-supervised learning, whose kernel matrix is generated with a Taylor expansion on the generating function of diffusion similarity matrix. The generic diffusion kernel subsumes ...
Read More
Anisotropic diffusion for image denoising based on diffusion tensors

In this paper, the anisotropic diffusion for image denoising is considered. A new method to construct diffusion tensors is proposed. The tensors obtained by our approach depend on four directional derivatives of the intensity of an image, and hence they ...
Read More
Crossing-Preserving Coherence-Enhancing Diffusion on Invertible Orientation Scores

Many image processing problems require the enhancement of crossing elongated structures. These problems cannot easily be solved by commonly used coherence-enhancing diffusion methods. Therefore, we propose a method for coherence-enhancing diffusion on ...
Read More

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

WWW '19: The World Wide Web Conference

May 2019

3620 pages

ISBN:9781450366748

DOI:10.1145/3308558

Editors:
Ling Liu
Georgia Tech, USA
,
Ryen White
Microsoft Research, USA

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

IW3C2: International World Wide Web Conference Committee

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 May 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

WWW '19

WWW '19: The Web Conference

May 13 - 17, 2019

CA, San Francisco, USA

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

12
Total Citations
View Citations
665
Total Downloads

Downloads (Last 12 months)40
Downloads (Last 6 weeks)2

Other Metrics

View Author Metrics

Citations

Cited By

Zhong HZhang YYan CXuan ZYu TZhang JYing SGao Y(2024)Penalized Flow Hypergraph Local ClusteringIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2023.3319019(1-16)Online publication date: 2024
https://doi.org/10.1109/TKDE.2023.3319019
Chen YYe WLi D(2023)Spectral Clustering Community Detection Algorithm Based on Point-Wise Mutual Information Graph KernelEntropy10.3390/e2512161725:12(1617)Online publication date: 3-Dec-2023
https://doi.org/10.3390/e25121617
Ni LGe JZhang YLuo WSheng V(2023)Semi-Supervised Local Community DetectionIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2023.3290095(1-17)Online publication date: 2023
https://doi.org/10.1109/TKDE.2023.3290095
He CFei XCheng QLi HHu ZTang Y(2022)A Survey of Community Detection in Complex Networks Using Nonnegative Matrix FactorizationIEEE Transactions on Computational Social Systems10.1109/TCSS.2021.31144199:2(440-457)Online publication date: Apr-2022
https://doi.org/10.1109/TCSS.2021.3114419
He CZheng YFei XLi HHu ZTang Y(2022)Boosting Nonnegative Matrix Factorization Based Community Detection With Graph Attention Auto-EncoderIEEE Transactions on Big Data10.1109/TBDATA.2021.31032138:4(968-981)Online publication date: 1-Aug-2022
https://doi.org/10.1109/TBDATA.2021.3103213
Kang YLee WLee YHan KKim S(2021)Adversarial Learning of Balanced Triangles for Accurate Community Detection on Signed Networks2021 IEEE International Conference on Data Mining (ICDM)10.1109/ICDM51629.2021.00137(1150-1155)Online publication date: Dec-2021
https://doi.org/10.1109/ICDM51629.2021.00137
Kumar PJain RChaudhary SKumar S(2021)Solving Community Detection in Social Networks: A comprehensive study2021 5th International Conference on Computing Methodologies and Communication (ICCMC)10.1109/ICCMC51019.2021.9418412(239-345)Online publication date: 8-Apr-2021
https://doi.org/10.1109/ICCMC51019.2021.9418412
Fountoulakis KWang DYang SDaumé HSingh A(2020)p-Norm flow diffusion for local graph clusteringProceedings of the 37th International Conference on Machine Learning10.5555/3524938.3525240(3222-3232)Online publication date: 13-Jul-2020
https://dl.acm.org/doi/10.5555/3524938.3525240
Liu MGleich DLarochelle HRanzato MHadsell RBalcan MLin H(2020)Strongly local p-norm-cut algorithms for semi-supervised learning and local graph clusteringProceedings of the 34th International Conference on Neural Information Processing Systems10.5555/3495724.3496146(5023-5035)Online publication date: 6-Dec-2020
https://dl.acm.org/doi/10.5555/3495724.3496146
Jia JBenson AGupta RLiu YShah MRajan STang JPrakash B(2020)Residual Correlation in Graph Neural Network RegressionProceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining10.1145/3394486.3403101(588-598)Online publication date: 23-Aug-2020
https://dl.acm.org/doi/10.1145/3394486.3403101
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents