research-article

<sc>NVGraph</sc>: Enforcing Crash Consistency of Evolving Network Analytics in NVMM Systems

Authors:

Xuechen ZhangAuthors Info & Claims

IEEE Transactions on Parallel and Distributed Systems, Volume 31, Issue 6

Pages 1255 - 1269

https://doi.org/10.1109/TPDS.2020.2965452

Published: 01 June 2020 Publication History

Abstract

Many complex networks can be modeled as evolving graphs. Existing in-memory graph data structures are designed for <sc>DRAM</sc>. Thus, they cannot effectively exploit the current and ongoing adoption of emerging non-volatile main memory (<sc>NVMM</sc>) for two reasons. (1) Ephemeral graph data structures are not crash-consistent for <sc>NVMM</sc>. (2) <sc>NVMM</sc> writes and reads and may incur higher latency than <sc>DRAM</sc>. In this article, we propose a novel persistent evolving graph data structure, named <sc>NVGraph</sc>, for both computing and in-memory storage of evolving graphs in <sc>NVMM</sc>. We devise <sc>NVGraph</sc> as a multi-version data structure, wherein a minimum of one version of its data is stored in <sc>NVMM</sc> to provide the desired durability at runtime for failure recovery, and another version is stored in both <sc>DRAM</sc> and <sc>NVMM</sc> to reduce the <sc>NVMM</sc>-induced memory latency. <sc>NVGraph</sc> is also a partitioned data structure. We dynamically transform the layout of <sc>NVGraph</sc> (e.g., changing the size of its partition in <sc>DRAM</sc>) exploiting network properties and data access patterns of workloads. For the evaluation of <sc>NVGraph</sc>, we implement four representative real-world graph applications: pagerank, BFS, influence maximization, and rumor source detection. The experimental results show that the performance of <sc>NVGraph</sc> is comparable to other in-memory data structure (e.g., CSR and LLAMA) while using 70 percent less DRAM. It scales well up to 10 billion edges and 201 snapshots and supports crash consistency. It offers up to the 21X speedup of execution time compared to the scale-up graph computation approaches (e.g., GraphChi and X-stream).

References

[1]

S. Lim, J. Hao, Z. Lu, X. Zhang, and Z. Zhang, “Approximating the k-minimum distance rumor source detection in online social networks,” in Proc. 27th Int. Conf. Comput. Commun. Netw., 2018, pp. 1–9.

[2]

X. Zhang, U. Khanal, X. Zhao, and S. Ficklin, “Making sense of performance in in-memory computing frameworks for scientific data analysis: A case study of the spark system,” J. Parallel Distrib. Comput., vol. 120, pp. 369–382, 2018.

Digital Library

[3]

G. Liu, et al., “Evolving graph based power system EMS real time analysis framework,” in Proc. IEEE Int. Symp. Circuits Syst., May 2018, pp. 1–5.

[4]

R. Pearce, M. Gokhale, and N. M. Amato, “Multithreaded asynchronous graph traversal for in-memory and semi-external memory,” in Proc. ACM/IEEE Int. Conf. High Perform. Comput. Netw. Storage Anal., 2010, pp. 1–11.

[5]

P. Macko, V. J. Marathe, D. W. Margo, and M. I. Seltzer, “LLAMA: Efficient graph analytics using large multiversioned arrays,” in Proc. IEEE 31st Int. Conf. Data Eng., 2015, pp. 363–374.

[6]

X. Ju, D. Williams, H. Jamjoom, and K. G. Shin, “Version traveler: Fast and memory-efficient version switching in graph processing systems,” in Proc. USENIX Annu. Tech. Conf., 2016, pp. 523–536.

[7]

J. E. Gonzalez, Y. Low, H. Gu, D. Bickson, and C. Guestrin, “PowerGraph: Distributed graph-parallel computation on natural graphs,” in Proc. 10th USENIX Symp. Operating Syst. Des. Implementation, 2012, pp. 17–30.

[8]

Y. Low, D. Bickson, J. Gonzalez, C. Guestrin, A. Kyrola, and J. M. Hellerstein, “Distributed GraphLab: A framework for machine learning and data mining in the cloud,” Proc. VLDB Endowment, vol. 5, no. 8, pp. 716–727, Apr. 2012.

[9]

J. Ousterhout, et al., “The RAMCloud storage system,” ACM Trans. Comput. Syst., vol. 33, no. 3, pp. 7:1–7:55, Aug. 2015.

Digital Library

[10]

D. Zheng, D. Mhembere, R. Burns, J. Vogelstein, C. E. Priebe, and A. S. Szalay, “FlashGraph: Processing billion-node graphs on an array of commodity SSDs,” in Proc. 13th USENIX Conf. File Storage Technol., 2015, pp. 45–58.

[11]

A. Kyrola, G. Blelloch, and C. Guestrin, “GraphChi: Large-scale graph computation on just a PC,” in Proc. 10th USENIX Symp. Operating Syst. Des. Implementation, 2012, pp. 31–46.

[12]

J. J. Yang and R. S. Williams, “Memristive devices in computing system: Promises and challenges,” J. Emerg. Technol. Comput. Syst., vol. 9, no. 2, May 2013, Art. no.

[13]

D. X. Memory, 2019. [Online]. Available: https://www.intel.com/content/www/us/en/architecture-and-technology/intel-optane-technology.html

[14]

Intel optane DIMMs, 2018. [Online]. Available: https://blocksandfiles.com/2018/12/13/intel-confirms-optane-dimm-and-ssd-speed/

[15]

S. Venkataraman, N. Tolia, P. Ranganathan, and R. H. Campbell, “Consistent and durable data structures for non-volatile byte-addressable memory,” in Proc. 9th USENIX Conf. File Stroage Technol., 2011, Art. no.

[16]

J. Yang, Q. Wei, C. Chen, C. Wang, K. L. Yong, and B. He, “NV-Tree: Reducing consistency cost for NVM-based single level systems,” in Proc. 13th USENIX Conf. File Storage Technol., 2015, pp. 167–181.

[17]

S. Chen, P. B. Gibbons, and S. Nath, “Rethinking database algorithms for phase change memory,” in Proc. 5th Biennial Conf. Innovative Data Syst. Res., 2011, pp. 21–31.

[18]

S. Chen and Q. Jin, “Persistent B⁺-trees in non-volatile main memory,” Proc. VLDB Endowment, vol. 8, no. 7, pp. 786–797, Feb. 2015.

[19]

B. Nguyen, H. Tan, and X. Zhang, “Large-scale adaptive mesh simulations through non-volatile byte-addressable memory,” in Proc. Int. Conf. High Perform. Comput. Netw. Storage Anal., 2017, pp. 1–12.

[20]

B. Nguyen, H. Tan, K. Davis, and X. Zhang, “Persistent octrees for parallel mesh refinement through non-volatile byte-addressable memory,” IEEE Trans. Parallel Distrib. Syst., vol. 30, pp. 677–691, 2019, Art. no. 1.

[21]

A. Roy, I. Mihailovic, and W. Zwaenepoel, “X-stream: Edge-centric graph processing using streaming partitions,” in Proc. 24th ACM Symp. Operating Syst. Princ., 2013, pp. 472–488.

[22]

H. Elnawawy, M. Alshboul, J. Tuck, and Y. Solihin, “Efficient checkpointing of loop-based codes for non-volatile main memory,” in Proc. 26th Int. Conf. Parallel Archit. Compilation Techn., Sep. 2017, pp. 318–329.

[23]

B. C. Lee, E. Ipek, O. Mutlu, and D. Burger, “Architecting phase change memory as a scalable DRAM alternative,” in Proc. 36th Annu. Int. Symp. Comput. Archit., 2009, pp. 2–13.

[24]

Persistent memory programming, 2019. [Online]. Available: https://pmem.io/

[25]

J. Wang and M. Balazinska, “Elastic memory management for cloud data analytics,” in Proc. USENIX Annu. Tech. Conf., 2017, pp. 745–758.

[26]

J. R. Quinlan, “Learning with continuous classes,” in Proc. 5th Aust. Joint Conf. Artif. Intell., 1992, pp. 343–348.

[27]

U. Khurana and A. Deshpande, “Efficient snapshot retrieval over historical graph data,” in Proc. IEEE 29th Int. Conf. Data Eng., 2013, pp. 997–1008.

[28]

W. Han, et al., “Chronos: A graph engine for temporal graph analysis,” in Proc. 9th Eur. Conf. Comput. Syst., 2014, pp. 1–14.

[29]

Y. Miao, et al., “ImmortalGraph: A system for storage and analysis of temporal graphs,” Trans. Storage, vol. 11, no. 3, pp. 14:1–14:34, Jul. 2015.

[30]

K. Vora, R. Gupta, and G. Xu, “Synergistic analysis of evolving graphs,” ACM Trans. Archit. Code Optim., vol. 13, no. 4, pp. 32:1–32:27, Oct. 2016.

[31]

P. Kumar and H. H. Huang, “GraphOne: A data store for real-time analytics on evolving graphs,” in Proc. 17th USENIX Conf. File Storage Technol., Feb. 2019, pp. 249–263.

[32]

S. Ko and W.-S. Han, “TurboGraph++: A scalable and fast graph analytics system,” in Proc. Int. Conf. Manage. Data, 2018, pp. 395–410.

[33]

K. Nilakant, V. Dalibard, A. Roy, and E. Yoneki, “PrefEdge: SSD prefetcher for large-scale graph traversal,” in Proc. Int. Conf. Syst. Storage, 2014, pp. 1–12.

[34]

S. Maass, C. Min, S. Kashyap, W. Kang, M. Kumar, and T. Kim, “Mosaic: Processing a trillion-edge graph on a single machine,” in Proc. 12th Eur. Conf. Comput. Syst., 2017, pp. 527–543.

[35]

J. Malicevic, S. Dulloor, N. Sundaram, N. Satish, J. Jackson, and W. Zwaenepoel, “Exploiting NVM in large-scale graph analytics,” in Proc. 3rd Workshop Interact. NVM/FLASH Operating Syst. Workloads, 2015, pp. 1–9.

[36]

X. Zhu, W. Han, and W. Chen, “GridGraph: Large-scale graph processing on a single machine using 2-level hierarchical partitioning,” in Proc. USENIX Annu. Tech. Conf., 2015, pp. 375–386.

[37]

H. Liu and H. H. Huang, “Graphene: Fine-grained IO management for graph computing,” in Proc. 15th USENIX Conf. File Storage Technol., 2017, pp. 285–300.

[38]

S.-W. Jun, A. Wright, S. Zhang, S. Xu, and Arvind, “GraFBoost: Using accelerated flash storage for external graph analytics,” in Proc. 45th Annu. Int. Symp. Comput. Archit., 2018, pp. 411–424.

[39]

N. Elyasi, C. Choi, and A. Sivasubramaniam, “Large-scale graph processing on emerging storage devices,” in Proc. 17th USENIX Conf. File Storage Technol., 2019, pp. 309–316.

[40]

R. Hagmann, “Reimplementing the cedar file system using logging and group commit,” in Proc. 11th ACM Symp. Operating Syst. Principles, 1987, pp. 155–162.

[41]

J. R. Driscoll, N. Sarnak, D. D. Sleator, and R. E. Tarjan, “Making data structures persistent,” J. Comput. Syst. Sci., vol. 38, no. 1, pp. 86–124, Feb. 1989.

Digital Library

[42]

J. Bent, et al., “PLFS: A checkpoint filesystem for parallel applications,” in Proc. Conf. High Perform. Comput. Netw. Storage Anal., 2009, pp. 1–12.

[43]

P. J. Varman and R. M. Verma, “An efficient multiversion access structure,” IEEE Trans. Knowl. Data Eng., vol. 9, no. 3, pp. 391–409, May 1997.

Digital Library

[44]

C. Mohan, “Repeating history beyond ARIES,” in Proc. 25th Int. Conf. Very Large Data Bases, 1999, pp. 1–17.

[45]

J. Condit, et al., “Better I/O through byte-addressable, persistent memory,” in Proc. ACM SIGOPS 22nd Symp. Operating Syst. Princ., 2009, pp. 133–146.

[46]

S. Chen, F. Zhang, L. Liu, and L. Peng, “Efficient GPU NVRAM persistence with helper warps,” in Proc. 56th Annu. Des. Autom. Conf., 2019, pp. 155:1–155:6.

[47]

Q. Liu, J. Izraelevitz, S. K. Lee, M. L. Scott, S. H. Noh, and C. Jung, “iDO: Compiler-directed failure atomicity for nonvolatile memory,” in Proc. 51st Annu. IEEE/ACM Int. Symp. Microarchit., 2018, pp. 258–270.

[48]

S. Popinet, “Gerris: A tree-based adaptive solver for the incompressible Euler equations in complex geometries,” J. Comput. Phys., vol. 190, no. 2, pp. 572–600, 2003.

Digital Library

[49]

W. X. Wang, et al., “Gyro-Kinetic simulation of global turbulent transport properties in tokamak experiments,” Phys. Plasmas, vol. 13, no. 9, 2006, Art. no.

[50]

X. Zhang, K. Davis, and S. Jiang, “IOrchestrator: Improving the performance of multi-node I/O systems via inter-server coordination,” in Proc. ACM/IEEE Int. Conf. High Perform. Comput. Netw. Storage Anal., 2010, pp. 1–11.

[51]

X. Zhang, K. Davis, and S. Jiang, “QoS support for end users of I/O-intensive applications using shared storage systems,” in Proc. Int. Conf. High Perform. Comput. Netw. Storage Anal., 2011, pp. 1–12.

[52]

S. Kannan, A. Gavrilovska, K. Schwan, and D. Milojicic, “Optimizing checkpoints using NVM as virtual memory,” in Proc. IEEE 27th Int. Symp. Parallel Distrib. Process., 2013, pp. 29–40.

[53]

A. M. Caulfield, et al., “Understanding the impact of emerging non-volatile memories on high-performance, IO-intensive computing,” in Proc. ACM/IEEE Int. Conf. High Perform. Comput. Netw. Storage Anal., 2010, pp. 1–11.

[54]

L. Liu, S. Yang, L. Peng, and X. Li, “Hierarchical hybrid memory management in OS for tiered memory systems,” IEEE Trans. Parallel Distrib. Syst., vol. 30, no. 10, pp. 2223–2236, Oct. 2019.

[55]

L. Liu, Y. Li, C. Ding, H. Yang, and C. Wu, “Rethinking memory management in modern operating system: Horizontal, vertical or random?” IEEE Trans. Comput., vol. 65, no. 6, pp. 1921–1935, Jun. 2016.

Digital Library

[56]

C.-C. J. Li, E. M. Stewart, and W. K. Fuchs, “Compiler-assisted full checkpointing,” Softw.: Practice Experience, vol. 24, no. 10, pp. 871–886, 1994.

Digital Library

[57]

A. Ziv and J. Bruck, “An on-line algorithm for checkpoint placement,” IEEE Trans. Comput., vol. 46, no. 9, pp. 976–985, Sep. 1997.

Digital Library

[58]

M. K. Qureshi, V. Srinivasan, and J. A. Rivers, “Scalable high performance main memory system using phase-change memory technology,” in Proc. 36th Annu. Int. Symp. Comput. Archit., 2009, pp. 24–33.

[59]

P. Zhou, B. Zhao, J. Yang, and Y. Zhang, “A durable and energy efficient main memory using phase change memory technology,” in Proc. 36th Annu. Int. Symp. Comput. Archit., 2009, pp. 14–23.

[60]

M. Newman, Networks : An Introduction. London, U.K.: Oxford Univ. Press, 2010.

[61]

476 million twitter tweets, 2018. [Online]. Available: https://snap.stanford.edu/data/twitter7.html

[62]

B. Bahmani, A. Chowdhury, and A. Goel, “Fast incremental and personalized PageRank,” Proc. VLDB Endowment, vol. 4, no. 3, pp. 173–184, Dec. 2010.

[63]

M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann, and I. H. Witten, “The WEKA data mining software: An update,” SIGKDD Explorations Newslett., vol. 11, no. 1, pp. 10–18, Nov. 2009.

Digital Library

[64]

D. Chakrabarti, Y. Zhan, and C. Faloutsos, R-MAT: A Recursive Model for Graph Mining, pp. 442–446. [Online]. Available: https://epubs.siam.org/doi/abs/10.1137/1.9781611972740.43

[65]

H. Volos, A. J. Tack, and M. M. Swift, “Mnemosyne: Lightweight persistent memory,” SIGPLAN Notices, vol. 47, no. 4, pp. 91–104, Mar. 2011.

[66]

J. Ou, J. Shu, and Y. Lu, “A high performance file system for non-volatile main memory,” in Proc. 11th Eur. Conf. Comput. Syst., 2016, pp. 1–16.

[67]

J. Huang, K. Schwan, and M. K. Qureshi, “NVRAM-aware logging in transaction systems,” in Proc. VLDB Endowment, Dec. 2014, vol. 8, no. 4, pp. 389–400.

[68]

H. Volos, S. Nalli, S. Panneerselvam, V. Varadarajan, P. Saxena, and M. M. Swift, “Aerie: Flexible file-system interfaces to storage-class memory,” in Proc. 9th Eur. Conf. Comput. Syst., 2014, pp. 1–14.

[69]

L. Backstrom, D. Huttenlocher, J. Kleinberg, and X. Lan, “Group formation in large social networks: Membership, growth, and evolution,” in Proc. 12th ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, 2006, pp. 44–54.

[70]

H. Kwak, C. Lee, H. Park, and S. Moon, “What is Twitter, a social network or a news media?” in Proc. 19th Int. Conf. World Wide Web, 2010, pp. 591–600.

[71]

L. Page, S. Brin, R. Motwani, and T. Winograd, “The PageRank citation ranking: Bringing order to the web,” Stanford InfoLab, Stanford, CA, USA, Tech. Rep. 1999–66, Nov. 1999, previous number = SIDL-WP-1999-0120.

[72]

F. Bonchi, “Influence propagation in social networks: A data mining perspective,” in Proc. IEEE/WIC/ACM Int. Conf. Web Intell. Intell. Agent Technol., Aug 2011, pp. 2–2.

[73]

S. Hong, H. Chafi, E. Sedlar, and K. Olukotun, “Green-marl: A DSL for easy and efficient graph analysis,” in Proc. 17th Int. Conf. Archit. Support Program. Lang. Operating Syst., 2012, pp. 349–362.

[74]

D. Zheng, R. Burns, and A. S. Szalay, “A parallel page cache: IOPS and caching for multicore systems,” in Proc. 4th USENIX Workshop Hot Topics Storage File Syst., 2012, Art. no.

[75]

Graph 500, 2019. [Online]. Available: https://graph500.org/

Cited By

Lavinsky BZhang X(2022)PM-Rtree: A Highly-Efficient Crash-Consistent R-tree for Persistent MemoryProceedings of the 34th International Conference on Scientific and Statistical Database Management10.1145/3538712.3538713(1-11)Online publication date: 6-Jul-2022
https://dl.acm.org/doi/10.1145/3538712.3538713
Baumstark AGötze PJibril MSattler K(2021)Instant Graph Query Recovery on Persistent MemoryProceedings of the 17th International Workshop on Data Management on New Hardware10.1145/3465998.3466011(1-4)Online publication date: 20-Jun-2021
https://dl.acm.org/doi/10.1145/3465998.3466011

Index Terms

<sc>NVGraph</sc>: Enforcing Crash Consistency of Evolving Network Analytics in NVMM Systems
1. Information systems
  1. Data management systems
  2. Information storage systems

Index terms have been assigned to the content through auto-classification.

Recommendations

True-Damage-Aware Enumerative Coding for Improving <sc>nand</sc> Flash Memory Endurance
This brief presents a technique that can fully exploit the data dependency of flash memory cell damage to improve the program/erase (P/E) cycling endurance of nand flash memory. The key is to opportunistically leverage data lossless compressibility and ...
i-NVMM: a secure non-volatile main memory system with incremental encryption
ISCA '11: Proceedings of the 38th annual international symposium on Computer architecture

Emerging technologies for building non-volatile main memory (NVMM) systems suffer from a security vulnerability where information lingers on long after the system is powered down, enabling an attacker with physical access to the system to extract ...
Compiler aided checkpointing using crash-consistent data structures in NVMM systems
ICS '20: Proceedings of the 34th ACM International Conference on Supercomputing

Scientific applications use checkpointing for failure recovery. The existing checkpointing approaches were proposed for storing persistent states of applications as checkpoints in disk-based file systems via the block interface. As non-volatile main ...

Comments

Information & Contributors

Information

Published In

cover image IEEE Transactions on Parallel and Distributed Systems

IEEE Transactions on Parallel and Distributed Systems Volume 31, Issue 6

June 2020

164 pages

ISSN:1045-9219

Issue’s Table of Contents

1045-9219 © 2020 Personal use is permitted, but republication/redistribution requires IEEE permission. See https://www.ieee.org/publications/rights/index.html for more information.

Publisher

IEEE Press

Publication History

Published: 01 June 2020

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 06 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Lavinsky BZhang X(2022)PM-Rtree: A Highly-Efficient Crash-Consistent R-tree for Persistent MemoryProceedings of the 34th International Conference on Scientific and Statistical Database Management10.1145/3538712.3538713(1-11)Online publication date: 6-Jul-2022
https://dl.acm.org/doi/10.1145/3538712.3538713
Baumstark AGötze PJibril MSattler K(2021)Instant Graph Query Recovery on Persistent MemoryProceedings of the 17th International Workshop on Data Management on New Hardware10.1145/3465998.3466011(1-4)Online publication date: 20-Jun-2021
https://dl.acm.org/doi/10.1145/3465998.3466011

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents