research-article

Distributed subgraph matching on timely dataflow

Authors:

Zhengping Qian,

Jingren ZhouAuthors Info & Claims

Proceedings of the VLDB Endowment, Volume 12, Issue 10

Pages 1099 - 1112

https://doi.org/10.14778/3339490.3339494

Published: 01 June 2019 Publication History

Abstract

Recently there emerge many distributed algorithms that aim at solving subgraph matching at scale. Existing algorithm-level comparisons failed to provide a systematic view of distributed subgraph matching mainly due to the intertwining of strategy and optimization. In this paper, we identify four strategies and three general-purpose optimizations from representative state-of-the-art algorithms. We implement the four strategies with the optimizations based on the common Timely dataflow system for systematic strategy-level comparison. Our implementation covers all representative algorithms. We conduct extensive experiments for both unlabelled matching and labelled matching to analyze the performance of distributed subgraph matching under various settings, which is finally summarized as a practical guide.

References

[1]

The challenge9 datasets. http://www.dis.uniroma1.it/challenge9.

[2]

The clubweb12 dataset. https://lemurproject.org/clueweb12.

[3]

Compressed sparse row. https://en.wikipedia.org/wiki/Sparse_matrix.

[4]

Distributed subgraph matching on timely dataflow -the full paper. https://goo.gl/zkTkL4.

[5]

Giraph. http://giraph.apache.org/.

[6]

The implementation of bigjoin. https://github.com/frankmcsherry/dataflow-join/.

[7]

Ldbc benchmarks. http://ldbcouncil.org/benchmarks.

[8]

The ldbc social network benchmark. https://ldbc.github.io/ldbc_snb_docs/ldbc-snb-specification.pdf.

[9]

The open-sourced timely dataflow system. https://github.com/frankmcsherry/timely-dataflow.

[10]

The snap datasets. http://snap.stanford.edu/data/index.html.

[11]

The webgraph datasets. http://law.di.unimi.it/datasets.php.

[12]

C. R. Aberger, S. Tu, K. Olukotun, and C. Ré. Emptyheaded: A relational engine for graph processing. SIGMOD '16, pages 431--446.

Digital Library

[13]

F. N. Afrati, D. Fotakis, and J. D. Ullman. Enumerating subgraph instances using map-reduce. In Proc. of ICDE'13, 2013.

Digital Library

[14]

K. Ammar, F. McSherry, S. Salihoglu, and M. Joglekar. Distributed evaluation of subgraph queries using worst-case optimal low-memory dataflows. PVLDB, 11(6):691--704, 2018.

Digital Library

[15]

F. Bi, L. Chang, X. Lin, L. Qin, and W. Zhang. Efficient subgraph matching by postponing cartesian products. SIGMOD '16, pages 1199--1214, 2016.

Digital Library

[16]

P. Carbone, A. Katsifodimos, . Kth, S. Sweden, S. Ewen, V. Markl, S. Haridi, and K. Tzoumas. Apache flink: Stream and batch processing in a single engine. 38, 01 2015.

[17]

D. Chakrabarti, Y. Zhan, and C. Faloutsos. R-mat: A recursive model for graph mining. In SDM, 2004.

[18]

S. Choudhury, L. B. Holder, G. Chin, K. Agarwal, and J. Feo. A selectivity based approach to continuous pattern detection in streaming graphs. In EDBT, 2015.

[19]

S. Chu, M. Balazinska, and D. Suciu. From theory to practice: Efficient join query evaluation in a parallel database system. SIGMOD '15, pages 63--78.

Digital Library

[20]

F. R. K. Chung, L. Lu, and V. H. Vu. The spectra of random graphs with given expected degrees. Internet Mathematics, 1(3), 2003.

[21]

D. J. DeWitt and J. Gray. Parallel database systems: The future of database processing or a passing fad? SIGMOD Rec., 19(4):104--112.

Digital Library

[22]

P. Erdos and A. Renyi. On the evolution of random graphs. In Publ. Math. Inst. Hungary. Acad. Sci., 1960.

[23]

W. Fan, J. Li, J. Luo, Z. Tan, X. Wang, and Y. Wu. Incremental graph pattern matching. SIGMOD '11, pages 925--936, 2011.

Digital Library

[24]

J. A. Grochow and M. Kellis. Network motif discovery using subgraph enumeration and symmetry-breaking. In Proc. of RECOMB'07, 2007.

Digital Library

[25]

W.-S. Han, J. Lee, and J.-H. Lee. Turboiso: Towards ultrafast and robust subgraph isomorphism search in large graph databases. In Proc. of SIGMOD'13, 2013.

Digital Library

[26]

J. He, S. Zhang, and B. He. In-cache query co-processing on coupled cpu-gpu architectures. PVLDB, 8(4):329--340, 2014.

Digital Library

[27]

M. Herlihy and N. Shavit. The Art of Multiprocessor Programming. 2008.

Digital Library

[28]

H. Inoue, M. Ohara, and K. Taura. Faster set intersection with simd instructions by reducing branch mispredictions. PVLDB, 8(3):293--304, 2014.

Digital Library

[29]

Y. E. Ioannidis and Y. C. Kang. Left-deep vs. bushy trees: An analysis of strategy spaces and its implications for query optimization. In SIGMOD'91, pages 168--177, 1991.

Digital Library

[30]

C. Kankanamge, S. Sahu, A. Mhedbhi, J. Chen, and S. Salihoglu. Graphflow: An active graph database. SIGMOD '17, pages 1695--1698.

Digital Library

[31]

H. Kim, J. Lee, S. S. Bhowmick, W.-S. Han, J. Lee, S. Ko, and M. H. Jarrah. Dualsim: Parallel subgraph enumeration in a massive graph on a single machine. SIGMOD '16, pages 1231--1245, 2016.

Digital Library

[32]

L. Lai, L. Qin, X. Lin, and L. Chang. Scalable subgraph enumeration in mapreduce. PVLDB, 8(10):974--985, 2015.

Digital Library

[33]

L. Lai, L. Qin, X. Lin, and L. Chang. Scalable subgraph enumeration in mapreduce: A cost-oriented approach. The VLDB Journal, 26(3):421--446, June 2017.

Digital Library

[34]

L. Lai, L. Qin, X. Lin, Y. Zhang, L. Chang, and S. Yang. Scalable distributed subgraph enumeration. PVLDB, 10(3):217--228, 2016.

Digital Library

[35]

B. Liu, L. Yuan, X. Lin, L. Qin, W. Zhang, and J. Zhou. Efficient (α, β)-core computation: An index-based approach. WWW, pages 1130--1141, 2019.

Digital Library

[36]

Y. Low, D. Bickson, J. Gonzalez, C. Guestrin, A. Kyrola, and J. M. Hellerstein. Distributed graphlab: A framework for machine learning and data mining in the cloud. PVLDB, 5(8):716--727, 2012.

Digital Library

[37]

F. McSherry, M. Isard, and D. G. Murray. Scalability! but at what cost? HOTOS'15, 2015.

Digital Library

[38]

D. G. Murray, F. McSherry, R. Isaacs, M. Isard, P. Barham, and M. Abadi. Naiad: A timely dataflow system. SOSP '13, pages 439--455, 2013.

Digital Library

[39]

H. Q. Ngo, E. Porat, C. Ré, and A. Rudra. Worst-case optimal join algorithms. J. ACM, 65(3), 2018.

Digital Library

[40]

D. Olteanu and M. Schleich. Factorized databases. SIGMOD Rec., 45(2):5--16, Sept. 2016.

Digital Library

[41]

M. Qiao, H. Zhang, and H. Cheng. Subgraph matching: On compression and computation. PVLDB, 11(2):176--188, 2017.

Digital Library

[42]

X. Qiu, W. Cen, Z. Qian, Y. Peng, Y. Zhang, X. Lin, and J. Zhou. Real-time constrained cycle detection in large dynamic graphs. PVLDB, 11(12):1876--1888.

Digital Library

[43]

X. Ren and J. Wang. Exploiting vertex relationships in speeding up subgraph isomorphism over large graphs. PVLDB, 8(5):617--628, 2015.

Digital Library

[44]

R. Shamir and D. Tsur. Faster subtree isomorphism. In Proceedings of the Fifth Israeli Symposium on Theory of Computing and Systems, pages 126--131, 1997.

Digital Library

[45]

H. Shang, Y. Zhang, X. Lin, and J. X. Yu. Taming verification hardness: An efficient algorithm for testing subgraph isomorphism. PVLDB, 1(1):364--375, 2008.

Digital Library

[46]

B. Shao, H. Wang, and Y. Li. Trinity: A distributed graph engine on a memory cloud. SIGMOD '13, pages 505--516, 2013.

Digital Library

[47]

Y. Shao, B. Cui, L. Chen, L. Ma, J. Yao, and N. Xu. Parallel subgraph listing in a large-scale graph. In SIGMOD'14, pages 625--636. ACM, 2014.

Digital Library

[48]

Z. Sun, H. Wang, H. Wang, B. Shao, and J. Li. Efficient subgraph matching on billion node graphs. PVLDB, 5(9):788--799, 2012.

Digital Library

[49]

T. L. Veldhuizen. Triejoin: A simple, worst-case optimal join algorithm. In ICDT, 2014.

[50]

K. Wang, X. Lin, L. Qin, W. Zhang, and Y. Zhang. Vertex priority based butterfly counting for large-scale bipartite networks. arXiv preprint arXiv:1812.00283, 2018.

[51]

H. Wei, J. X. Yu, C. Lu, and X. Lin. Speedup graph processing by graph ordering. SIGMOD '16, pages 1813--1828.

Digital Library

[52]

M. Zaharia, M. Chowdhury, M. J. Franklin, S. Shenker, and I. Stoica. Spark: Cluster computing with working sets. In HotCloud'10, pages 10--10.

Digital Library

Cited By

Li QYu J(2024)Fast Local Subgraph CountingProceedings of the VLDB Endowment10.14778/3659437.365945117:8(1967-1980)Online publication date: 1-Apr-2024
https://dl.acm.org/doi/10.14778/3659437.3659451
Zhu JMao YChen LGe CWei ZGao Y(2024)FusionQuery: On-demand Fusion Queries over Multi-source Heterogeneous DataProceedings of the VLDB Endowment10.14778/3648160.364817417:6(1337-1349)Online publication date: 3-May-2024
https://doi.org/10.14778/3648160.3648174
Lou YLai LLyu BYang YZhou XYu WZhang YZhou J(2024)Towards a Converged Relational-Graph Optimization FrameworkProceedings of the ACM on Management of Data10.1145/36988282:6(1-27)Online publication date: 20-Dec-2024
https://dl.acm.org/doi/10.1145/3698828
Show More Cited By

Recommendations

Efficient Subgraph Matching: Harmonizing Dynamic Programming, Adaptive Matching Order, and Failing Set Together
SIGMOD '19: Proceedings of the 2019 International Conference on Management of Data

Subgraph matching (or subgraph isomorphism) is one of the fundamental problems in graph analysis. Extensive research has been done to develop practical solutions for subgraph matching. The state-of-the-art algorithms such as \textsfCFL-Match and \...
On Perfect Matching Coverings and Even Subgraph Coverings

A perfect matching covering of a graph G is a set of perfect matchings of G such that every edge of G is contained in at least one member of it. Berge conjectured that every bridgeless cubic graph admits a perfect matching covering of order at most 5 we ...
A subgraph matching algorithm based on subgraph index for knowledge graph
Abstract
The problem of subgraph matching is one fundamental issue in graph search, which is NP-Complete problem. Recently, subgraph matching has become a popular research topic in the field of knowledge graph analysis, which has a wide range of ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the VLDB Endowment

Proceedings of the VLDB Endowment Volume 12, Issue 10

June 2019

177 pages

ISSN:2150-8097

Editors:
Lei Chen
HKUST
,
Fatma Özcan
IBM Research - Almaden

Issue’s Table of Contents

Publisher

VLDB Endowment

Publication History

Published: 01 June 2019

Published in PVLDB Volume 12, Issue 10

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

36
Total Citations
View Citations
438
Total Downloads

Downloads (Last 12 months)60
Downloads (Last 6 weeks)7

Reflects downloads up to 23 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Li QYu J(2024)Fast Local Subgraph CountingProceedings of the VLDB Endowment10.14778/3659437.365945117:8(1967-1980)Online publication date: 1-Apr-2024
https://dl.acm.org/doi/10.14778/3659437.3659451
Zhu JMao YChen LGe CWei ZGao Y(2024)FusionQuery: On-demand Fusion Queries over Multi-source Heterogeneous DataProceedings of the VLDB Endowment10.14778/3648160.364817417:6(1337-1349)Online publication date: 3-May-2024
https://doi.org/10.14778/3648160.3648174
Lou YLai LLyu BYang YZhou XYu WZhang YZhou J(2024)Towards a Converged Relational-Graph Optimization FrameworkProceedings of the ACM on Management of Data10.1145/36988282:6(1-27)Online publication date: 20-Dec-2024
https://dl.acm.org/doi/10.1145/3698828
Meng LShao YYuan LLai LCheng PLi XYu WZhang WLin XZhou J(2024)A Survey of Distributed Graph Algorithms on Massive GraphsACM Computing Surveys10.1145/369496657:2(1-39)Online publication date: 10-Oct-2024
https://dl.acm.org/doi/10.1145/3694966
Zhao YWang WChen SZhang BWang JWang Z(2024)Speeding Up Subgraph Matching Queries with Schema Guided IndexProceedings of the 2024 3rd International Conference on Networks, Communications and Information Technology10.1145/3672121.3672129(34-38)Online publication date: 7-Jun-2024
https://dl.acm.org/doi/10.1145/3672121.3672129
Sharma AMehta DWu B(2024)Understanding High-Performance Subgraph Pattern Matching: A Systems PerspectiveProceedings of the 7th Joint Workshop on Graph Data Management Experiences & Systems (GRADES) and Network Data Analytics (NDA)10.1145/3661304.3661897(1-12)Online publication date: 14-Jun-2024
https://dl.acm.org/doi/10.1145/3661304.3661897
Zhang ZLu YZheng WLin X(2024)A Comprehensive Survey and Experimental Study of Subgraph Matching: Trends, Unbiasedness, and InteractionProceedings of the ACM on Management of Data10.1145/36393152:1(1-29)Online publication date: 26-Mar-2024
https://dl.acm.org/doi/10.1145/3639315
Zheng LLi YXin JLiu HZheng RLiao XJin H(2024)Minimal Context-Switching Data Race Detection with Dataflow TrackingJournal of Computer Science and Technology10.1007/s11390-023-1569-739:1(211-226)Online publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1007/s11390-023-1569-7
Zhang TCai XChen LYang ZGao YCao BFan J(2024)Towards efficient simulation-based constrained temporal graph pattern matchingWorld Wide Web10.1007/s11280-024-01259-227:3Online publication date: 3-Apr-2024
https://dl.acm.org/doi/10.1007/s11280-024-01259-2
Wang ZLai LLiu YShui BTian CZhong S(2024)Parallelization of butterfly counting on hierarchical memoryThe VLDB Journal — The International Journal on Very Large Data Bases10.1007/s00778-024-00856-x33:5(1453-1484)Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1007/s00778-024-00856-x
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents