Abstract
We present Phoenix, a scalable hypergraph analytics framework for data analytics and knowledge discovery that was implemented on the leadership class computing platforms at Oak Ridge National Laboratory (ORNL). Our software framework comprises a distributed implementation of a streaming server architecture which acts as a gateway for various hypergraph generators/external sources to connect. Phoenix has the capability to utilize diverse hypergraph generators, including HyGen, a very large-scale hypergraph generator developed by ORNL. Phoenix incorporates specific algorithms for efficient data representation by exploiting hidden structures of the hypergraphs. Our experimental results demonstrate Phoenix’s scalable and stable performance on massively parallel computing platforms. Phoenix’s superior performance is due to the merging of high-performance computing with data analytic.
This manuscript has been authored in part by UT-Battelle, LLC, under contract no. DE-AC05-00OR22725 with the US Department of Energy (DOE). The US government retains and the publisher, by accepting the article for publication, acknowledges that the US government retains a nonexclusive, paid-up, irrevocable, worldwide license to publish or reproduce the published form of this manuscript, or allow others to do so, for US government purposes. DOE will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan (http://energy.gov/downloads/doe-public-access-plan).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
S. Agarwal, K. Branson, S. Belongie, Higher order learning with graphs, in Proceedings of the 23rd international conference on Machine learning (2006), pp. 17–24
A. Antelmi et al., SimpleHypergraphs.jl–novel software framework for modelling and analysis of hypergraphs, in International Workshop on Algorithms and Models for the Web-Graph (Springer, Cham, 2019), pp. 115–129
BalancedGo (2020). https://github.com/cem-okulmus/BalancedGo. Online. Accessed 03 Apr 2020
Chapel Hypergraph Library (2020). https://github.com/pnnl/chgl. Online. Accessed 03 Apr 2020
D. Ediger et al., STINGER: High performance data structure for streaming graphs, in 2012 IEEE Conference on High Performance Extreme Computing (2012), pp. 1–5
E. Estrada, J. Rodriguez-Velazquez, Complex networks as hypergraphs (2005) . Arxiv preprint physics 0505137
G. Feng, X. Meng, K. Ammar, DISTINGER: A distributed graph data structure for massive dynamic graph processing, in 2015 IEEE International Conference on Big Data (Big Data) (2015), pp. 1814–1822
Graph Signal Processing Toolbox (GSPBox) (2020). https://github.com/epfl-lts2/gspbox. Online. Accessed 03 Apr 2020
B. Heintz, A. Chandra, Beyond graphs: toward scalable hypergraph analysis systems. SIGMETRICS Perform. Eval. Rev. 41(4), 94–97 (2014). ISSN: 0163-5999. https://doi.org/10.1145/2627534.2627563
V. Hernandez, J.E. Roman, V. Vidal, SLEPc: A scalable and flexible toolkit for the solution of eigenvalue problems. ACM Trans. Math. Softw. (TOMS) 31(3), 351–362 (2005)
P. Hintjens, ZeroMQ: messaging for many applications (O’Reilly Media, Newton, 2013)
HyperGCN: A New Method of Training Graph Convolutional Networks on Hypergraphs (2020). https://github.com/malllabiisc/HyperGCN. Online. Accessed 03 Apr 2020
Hypergraph Algorithms Package (2020). https://murali-group.github.io/halp/. Online. Accessed 03 Apr 2020
HyperGraphLib (2020). https://github.com/alex87/HyperGraphLib. Online. Accessed 03 Apr 2020
W. Jiang et al., HyperX: A scalable hypergraph framework. IEEE Trans. Knowl. Data Eng. 31(5), 909–922 (2019)
KaHyPar - Karlsruhe Hypergraph Partitioning (2020). https://github.com/kahypar/kahypar. Online. Accessed 03 Apr 2020
J. Leskovec et al., Kronecker graphs: An approach to modeling networks. J. Mach. Learn. Res. 11, 985–1042 (2010)
A. Lugowski et al., Scalable complex graph analysis with the knowledge discovery toolbox, in 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (2012), pp. 5345–5348
multihypergraph (2020). https://github.com/vaibhavkarve/multihypergraph. Online. Accessed 03 Apr 2020
networkR - An R package for analysing social and economic networks (2020). https://github.com/O1sims/networkR. Online. Accessed 03 Apr 2020
pnnl/HyperNetX (2020). https://pnnl.github.io/HyperNetX/build/index.html. Online. Accessed 03 Apr 2020
Pygraph (2020). https://github.com/jciskey/pygraph. Online. Accessed 03 Apr 2020
RHEA: A conduit for large-scale scientific discovery by pre- and post- processing and analysis of simulation data. https://www.olcf.ornl.gov/olcf-resources/compute-systems/rhea/. Accessed 14 Apr 2020
S. Schlag et al., K-way hypergraph partitioning via n-level recursive bisection, in 2016 Proceedings of the Eighteenth Workshop on Algorithm Engineering and Experiments (ALENEX) (SIAM, Philadelphia, 2016), pp. 53–67
P. Sen et al., Collective classification in network data. AI Mag.29(3), 93–93 (2008)
N. Sundaram et al., GraphMat: high performance graph analytics made productive. Proc. VLDB Endow. 8(11), 1214–1225 (2015). ISSN: 2150-8097. https://doi.org/10.14778/2809974.2809983
J. Winick, S. Jamin, Inet-3.0: Internet topology generator. Technical Report CSE-TR-456-02, University of Michigan, 2002
M.M. Wolf, A.M. Klinvex, D.M. Dunlavy, Advantages to modeling relational data using hypergraphs versus graphs, in 2016 IEEE High Performance Extreme Computing Conference (HPEC) (2016), pp. 1–7
D.E. Womble et al., Early experiences on summit: data analytics and AI applications. IBM J. Res. Devel. 63(6), 2:1–2:9 (2019)
N. Yadati et al., HyperGCN: a new method for training graph convolutional networks on hypergraphs, in Advances in Neural Information Processing Systems (2019), pp. 1509–1520
A. Yoo, A.H. Baker, R. Pearce, A scalable eigensolver for large scale-free graphs using 2D graph partitioning, in Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis (2011), pp. 1–11
J. You et al., Graphrnn: Generating realistic graphs with deep autoregressive models (2018). Preprint. arXiv:1802.08773
D. Zhou, J. Huang, B. Schölkopf, Learning with hypergraphs: Clustering, classification, and embedding, in Advances in Neural Information Processing Systems (2007), pp. 1601–1608
Acknowledgements
Support for this work was provided by the US Department of Defense. We used resources of the Computational Research and Development Programs and the Oak Ridge Leadership Computing Facility at the Oak Ridge National Laboratory, which is supported by the Office of Science of the US Department of Energy under contract no. DE-AC05-00OR22725.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Kurte, K., Imam, N., Hasan, S.M.S., Kannan, R. (2021). Phoenix: A Scalable Streaming Hypergraph Analysis Framework. In: Stahlbock, R., Weiss, G.M., Abou-Nasr, M., Yang, CY., Arabnia, H.R., Deligiannidis, L. (eds) Advances in Data Science and Information Engineering. Transactions on Computational Science and Computational Intelligence. Springer, Cham. https://doi.org/10.1007/978-3-030-71704-9_1
Download citation
DOI: https://doi.org/10.1007/978-3-030-71704-9_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-71703-2
Online ISBN: 978-3-030-71704-9
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)