Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article

Scalable and High-Performance Large-Scale Dynamic Graph Storage and Processing System

Published: 11 February 2025 Publication History

Abstract

Existing in-memory graph storage systems that rely on DRAM have scalability issues because of the limited capacity and volatile nature of DRAM. The emerging persistent memory (PMEM) offers us a chance to solve these issues through its larger capacity and non-volatile characteristics. However, simply adapting existing DRAM-based graph storage systems to PMEM would result in inefficient PMEM stores and accesses, including high read and write amplification to PMEM, imbalanced work division for PMEM accesses, and costly remote PMEM access across NUMA nodes. These issues severely limit the performance of large graph processing.
In this article, we aim at achieving scalable and high-performance graph processing in PMEM. We first propose an XPLine-friendly graph storage model that uses vertex-centric graph buffering, hierarchical vertex buffer managing, and in-place vertex block merging to optimize PMEM graph storage. Furthermore, we develop a scalable graph processing model that leverages multi-threaded work dividing and NUMA-friendly graph accessing to optimize PMEM graph accesses. Based on these techniques, we implement XPGraph, a PMEM-based graph storage system for large-scale evolving graphs, and several variants for different system settings. Our experiments demonstrate that XPGraph surpasses the state-of-the-art in-memory graph storage system on a PMEM-based system by 3.07× to 4.99× in update performance and up to 5.87× in query performance, and performs much better in highly parallel multi-threaded scenarios.

References

[1]
Zhiyuan Ai, Mingxing Zhang, Yongwei Wu, Xuehai Qian, Kang Chen, and Weimin Zheng. 2017. Squeezing out all the value of loaded data: An out-of-core graph processing system with reduced disk I/O. In Proceedings of the 2017 USENIX Annual Technical Conference. 125–137.
[2]
Scott Beamer, Krste Asanović, and David Patterson. 2015. The GAP benchmark suite. arXiv:1508.03619. Retrieved from https://arxiv.org/abs/1508.03619
[3]
Scott Beamer, Krste Asanovic, and David Patterson. 2015. Locality exists in graph processing: Workload characterization on an ivy bridge server. In Proceedings of the 2015 IEEE International Symposium on Workload Characterization. 56–65.
[4]
Lawrence Benson, Hendrik Makait, and Tilmann Rabl. 2021. Viper: An efficient hybrid pmem-dram key-value store. PVLDB 14, 9 (2021), 1544–1556.
[5]
Kumud Bhandari, Dhruva R. Chakrabarti, and Hans-J. Boehm. 2016. Makalu: Fast recoverable allocation of non-volatile memory. ACM SIGPLAN Notices 51, 10 (2016), 677–694.
[6]
Federico Busato, Oded Green, Nicola Bombieri, and David A. Bader. 2018. Hornet: An efficient data structure for dynamic sparse graphs and matrices on GPUs. In Proceedings of the IEEE HPEC. 1–7.
[7]
Wentao Cai, Haosen Wen, H. Alan Beadle, Chris Kjellqvist, Mohammad Hedayati, and Michael L Scott. 2020. Understanding and optimizing persistent memory allocation. In Proceedings of the ACM SIGPLAN ISMM. 60–73.
[8]
Paris Carbone, Asterios Katsifodimos, Stephan Ewen, Volker Markl, Seif Haridi, and Kostas Tzoumas. 2015. Apache flink: Stream and batch processing in a single engine. IEEE TCDE 38, 4 (2015), 28–38.
[9]
Hongzhi Chen, Miao Liu, Yunjian Zhao, Xiao Yan, Da Yan, and James Cheng. 2018. G-Miner: An efficient task-oriented graph mining system. In Proceedings of the ACM EuroSys. 1–12.
[10]
Qi Chen, Hao Hu, Cai Deng, Dingbang Liu, Shiyi Li, Bo Tang, Ting Yao, and Wen Xia. 2023. EEPH: An efficient extendible perfect hashing for hybrid PMem-DRAM. In Proceedings of the IEEE ICDE. IEEE, 1366–1378.
[11]
Rong Chen, Jiaxin Shi, Yanzhe Chen, and Haibo Chen. 2015. Powerlyra: Differentiated graph computation and partitioning on skewed graphs. In Proceedings of the ACM EuroSys. 1–39.
[12]
Shimin Chen and Qin Jin. 2015. Persistent B+-Trees in non-nolatile main memory. PVLDB 8, 7 (2015), 786–797.
[13]
Youmin Chen, Youyou Lu, Pei Chen, and Jiwu Shu. 2018. Efficient and consistent NVMM cache for SSD-based file system. IEEE TC 68, 8 (2018), 1147–1158.
[14]
Youmin Chen, Youyou Lu, Kedong Fang, Qing Wang, and Jiwu Shu. 2020. UTree: A persistent b+-tree with low tail latency. PVLDB 13, 12 (2020), 2634–2648.
[15]
Youmin Chen, Youyou Lu, Fan Yang, Qing Wang, Yang Wang, and Jiwu Shu. 2020. Flatstore: An efficient log-structured key-value storage engine for persistent memory. In Proceedings of the ACM ASPLOS. 1077–1091.
[16]
Youmin Chen, Jiwu Shu, Jiaxin Ou, and Youyou Lu. 2018. HiNFS: A persistent memory file system with both buffering and direct-access. ACM ToS 14, 1 (2018), 1–30.
[17]
Zhiwen Chen, Wenkui Che, Daokun Hu, Xin He, Jianhua Sun, and Hao Chen. 2023. On the performance intricacies of persistent memory aware storage engines. IEEE TKDE 35, 10 (2023), 10365–10382.
[18]
Raymond Cheng, Ji Hong, Aapo Kyrola, Youshan Miao, Xuetian Weng, Ming Wu, Fan Yang, Lidong Zhou, Feng Zhao, and Enhong Chen. 2012. Kineograph: Taking the pulse of a fast-changing and connected world. In Proceedings of the ACM EuroSys. 85–98.
[19]
Disa Mhembere Da Zheng, Randal Burns, Joshua Vogelstein, Carey E Priebe, and Alexander S. Szalay. 2015. FlashGraph: Processing billion-node graphs on an array of commodity SSDs. In Proceedings of the USENIX FAST. 45–58.
[20]
Zheng Dang, Shuibing He, Peiyi Hong, Zhenxin Li, Xuechen Zhang, Xian-He Sun, and Gang Chen. 2022. NVAlloc: Rethinking heap metadata management in persistent memory allocators. In Proceedings of the ACM ASPLOS. 115–127.
[21]
Zheng Dang, Shuibing He, Xuechen Zhang, Peiyi Hong, Zhenxin Li, Xinyu Chen, Haozhe Song, Xian-He Sun, and Gang Chen. 2024. PMAlloc: A holistic approach to improving persistent memory allocation. ACM TC 42, 3–4 (2024), 1–52.
[22]
Peter Desnoyers, Ian Adams, Tyler Estro, Anshul Gandhi, Geoff Kuenning, Mike Mesnier, Carl Waldspurger, Avani Wildani, and Erez Zadok. 2023. Persistent memory research in the post-optane era. In Proceedings of the DIMES. 23–30.
[23]
Aleksandar Dragojević, Dushyanth Narayanan, Edmund B. Nightingale, Matthew Renzelmann, Alex Shamis, Anirudh Badam, and Miguel Castro. 2015. No compromises: Distributed transactions with consistency, availability, and performance. In Proceedings of the ACM SOSP. 54–70.
[24]
Subramanya R. Dulloor, Sanjay Kumar, Anil Keshavamurthy, Philip Lantz, Dheeraj Reddy, Rajesh Sankaran, and Jeff Jackson. 2014. System software for persistent memory. In Proceedings of the ACM EuroSys. 1–15.
[25]
David Ediger, Rob McColl, Jason Riedy, and David A. Bader. 2012. Stinger: High performance data structure for streaming graphs. In Proceedings of the IEEE HPEC. 1–5.
[26]
Nima Elyasi, Changho Choi, and Anand Sivasubramaniam. 2019. Large-scale graph processing on emerging storage devices. In Proceedings of the USENIX FAST. 309–316.
[27]
Friendster Graph. 2025. Friendster. Retrieved from http://konect.cc/networks/friendster/
[28]
Xuran Ge, Mingche Lai, Yang Liu, Lizhou Wu, Zhutao Zhuang, Yang Ou, Zhiguang Chen, and Nong Xiao. 2022. SpacKV: A pmem-aware key-value separation store based on LSM-tree. In Proceedings of the ACM NPC. Springer, 327–339.
[29]
Joseph E. Gonzalez, Yucheng Low, Haijie Gu, Danny Bickson, and Carlos Guestrin. 2012. PowerGraph: Distributed graph-parallel computation on natural graphs. In Proceedings of the USENIX OSDI. 17–30.
[30]
Joseph E. Gonzalez, Reynold S. Xin, Ankur Dave, Daniel Crankshaw, Michael J. Franklin, and Ion Stoica. 2014. GraphX: Graph processing in a distributed dataflow framework. In Proceedings of the USENIX OSDI. 599–613.
[31]
Graph500 Generator. 2025. Graph500. Retrieved from https://graph500.org/.
[32]
Oded Green and David A. Bader. 2016. cuSTINGER: Supporting dynamic graph algorithms for GPUs. In Proceedings of the IEEE HPEC. 1–6.
[33]
Sungpack Hong, Hassan Chafi, Edic Sedlar, and Kunle Olukotun. 2012. Green-Marl: A DSL for easy and efficient graph analysis. ACM SIGPLAN Notices 47, 4 (2012), 349–362.
[34]
Daokun Hu, Zhiwen Chen, Wenkui Che, Jianhua Sun, and Hao Chen. 2022. Halo: A hybrid PMem-DRAM persistent hash index with fast recovery. In Proceedings of the ACM SIGMOD. 1049–1063.
[35]
Yihe Huang, Matej Pavlovic, Virendra Marathe, Margo Seltzer, Tim Harris, and Steve Byan. 2018. Closing the performance gap between volatile and persistent key-value stores using cross-referencing logs. In Proceedings of the USENIX ATC. 967–979.
[36]
Deukyeon Hwang, Wook-Hee Kim, Youjip Won, and Beomseok Nam. 2018. Endurable transient inconsistency in byte-addressable persistent b+-tree. In Proceedings of the USENIX FAST. 187–200.
[37]
Abdullah Al Raqibul Islam and Dong Dai. 2023. DGAP: Efficient dynamic graph analysis on persistent memory. In Proceedings of the ACM SC. 1–13.
[38]
Anand Padmanabha Iyer, Li Erran Li, Tathagata Das, and Ion Stoica. 2016. Time-evolving graph processing at scale. In Proceedings of the ACM GRADES. 1–6.
[39]
Myung-Hwan Jang, Jeong-Min Park, Ikhyeon Jo, Duck-Ho Bae, and Sang-Wook Kim. 2023. RealGraph+: A high-performance single-machine-based graph engine that utilizes IO bandwidth effectively. In Proceedings of the ACM WWW. 276–279.
[40]
Sang-Woo Jun, Andy Wright, Sizhuo Zhang, Shuotao Xu, and Arvind. 2018. GraFBoost: Using accelerated flash storage for external graph analytics. In Proceedings of the IEEE ISCA. 411–424.
[41]
Rajat Kateja, Anirudh Badam, Sriram Govindan, Bikash Sharma, and Greg Ganger. 2017. Viyojit: Decoupling battery and DRAM capacities for battery-backed DRAM. In Proceedings of the IEEE ISCA. 613–626.
[42]
Arijit Khan, Gustavo Segovia, and Donald Kossmann. 2018. On smart query routing: for distributed graph querying with decoupled storage. In Proceedings of the USENIX ATC. 401–412.
[43]
Juno Kim and Steven Swanson. 2022. Blaze: Fast graph processing on fast SSDs. In Proceedings of the IEEE SC. IEEE, 1–15.
[44]
Pradeep Kumar and H. Howie Huang. 2019. GraphOne: A data store for real-time analytics on evolving graphs. In Proceedings of the USENIX FAST. 249–263.
[45]
Pradeep Kumar and Sarah Revillar. 2023. G-Bench: Fair benchmarking to support innovations in streaming graph systems. In Proceedings of the IEEE IPDPSW. IEEE, 179–188.
[46]
Aapo Kyrola, Guy E. Blelloch, and Carlos Guestrin. 2012. Graphchi: Large-scale graph computation on just a PC. In Proceedings of the USENIX OSDI. 31–46.
[47]
Changji Li, Hongzhi Chen, Shuai Zhang, Yingqian Hu, Chao Chen, Zhenjie Zhang, Meng Li, Xiangchen Li, Dongqing Han, Xiaohui Chen, et al. 2022. ByteGraph: A high-performance distributed graph database in bytedance. PVLDB 15, 12 (2022), 3306–3318.
[48]
Zhenxin Li, Shuibing He, Zheng Dang, Peiyi Hong, Xuechen Zhang, Rui Wang, and Fei Wu. 2024. CCL-BTree: A crash-consistent locality-aware b+-tree for reducing xpbuffer-induced write amplification in persistent memory. In Proceedings of the 19th European Conference on Computer Systems. 441–455.
[49]
Hang Liu and H. Howie Huang. 2017. Graphene: Fine-grained io management for graph computing. In Proceedings of the USENIX FAST. 285–300.
[50]
Jihang Liu, Shimin Chen, and Lujun Wang. 2020. LB+ Trees: Optimizing persistent index performance on 3dxpoint memory. PVLDB 13, 7 (2020), 1078–1090.
[51]
Yucheng Low, Danny Bickson, Joseph Gonzalez, Carlos Guestrin, Aapo Kyrola, and Joseph M. Hellerstein. 2012. Distributed graphlab: A framework for machine learning and data mining in the cloud. PVLDB 5, 8 (2012), 716–727.
[52]
Shaonan Ma, Kang Chen, Shimin Chen, Mengxing Liu, Jianglang Zhu, Hongbo Kang, and Yongwei Wu. 2021. ROART: Range-query optimized persistent ART. In Proceedings of the USENIX FAST. 1–16.
[53]
Steffen Maass, Changwoo Min, Sanidhya Kashyap, Woonhak Kang, Mohan Kumar, and Taesoo Kim. 2017. Mosaic: Processing a trillion-edge graph on a single machine. In Proceedings of the ACM EuroSys. 527–543.
[54]
Peter Macko, Virendra J. Marathe, Daniel W. Margo, and Margo I. Seltzer. 2015. Llama: Efficient graph analytics using large multiversioned arrays. In Proceedings of the IEEE ICDE. 363–374.
[55]
Grzegorz Malewicz, Matthew H. Austern, Aart J. C. Bik, James C. Dehnert, Ilan Horn, Naty Leiser, and Grzegorz Czajkowski. 2010. Pregel: A system for large-scale graph processing. In Proceedings of the ACM SIGMOD. 135–146.
[56]
Derek G. Murray, Frank McSherry, Rebecca Isaacs, Michael Isard, Paul Barham, and Martín Abadi. 2013. Naiad: A timely dataflow system. In Proceedings of the ACM SOSP. 439–455.
[57]
Neo4j. 2025. Neo4j 4.4.6. Retrieved from https://github.com/neo4j/neo4j/tree/4.4.6
[58]
Neo4j Graph Database. 2025. Neo4j Inc. Retrieved from https://neo4j.com/
[59]
Donald Nguyen, Andrew Lenharth, and Keshav Pingali. 2013. A lightweight infrastructure for graph analytics. In Proceedings of the ACM SOSP. 456–471.
[61]
Ismail Oukid, Daniel Booss, Adrien Lespinasse, Wolfgang Lehner, Thomas Willhalm, and Grégoire Gomes. 2017. Memory management techniques for large-scale persistent-main-memory systems. PVLDB 10, 11 (2017), 1166–1177.
[62]
PCM Tool. 2025. Intel. Processor Counter Monitor (PCM). Retrieved from https://github.com/opcm/pcm
[63]
Vijayan Prabhakaran, Ming Wu, Xuetian Weng, Frank McSherry, Lidong Zhou, and Maya Haradasan. 2012. Managing large graphs on multi-cores with graph awareness. In Proceedings of the USENIX ATC. 41–52.
[64]
pthread setaffinity np. 2025. pthread_setaffinity_np(3) – Linux manual page. Retrieved from https://man7.org/linux/man-pages/man3/pthread_setaffinity_np.3.html
[65]
Hao Qi, Yiyang Wu, Ligang He, Yu Zhang, Kang Luo, Minzhi Cai, Hai Jin, Zhan Zhang, and Jin Zhao. 2024. LSGraph: A locality-centric high-performance streaming graph engine. In Proceedings of the ACM EuroSys. 33–49.
[66]
Amitabha Roy, Ivo Mihailovic, and Willy Zwaenepoel. 2013. X-stream: Edge-centric graph processing using streaming partitions. In Proceedings of the ACM SOSP. 472–488.
[67]
Samsung Memory-Semantic SSD. [n. d.]. Samsung Memory-Semantic SSD. Retrieved from https://semiconductor.samsung.com/us/news-events/tech-blog/webinar-memory-semantic-ssd/
[68]
David Schwalb, Tim Berning, Martin Faust, Markus Dreseler, and Hasso Plattner. 2015. NVM malloc: Memory allocation for NVRAM. In Proceedings of ADMS. 61–72.
[69]
Dipanjan Sengupta and Shuaiwen Leon Song. 2017. Evograph: On-the-fly efficient mining of evolving graphs on GPU. In Proceedings of the ACM ISC. 97–119.
[70]
Dipanjan Sengupta, Narayanan Sundaram, Xia Zhu, Theodore L. Willke, Jeffrey Young, Matthew Wolf, and Karsten Schwan. 2016. GraphIn: An online high performance incremental graph processing framework. In Proceedings of the Euro-Par. 319–333.
[71]
Xiaogang Shi, Bin Cui, Yingxia Shao, and Yunhai Tong. 2016. Tornado: A system for real-time iterative analysis over evolving data. In Proceedings of the ACM SIGMOD. 417–430.
[72]
Julian Shun and Guy E. Blelloch. 2013. Ligra: A lightweight graph processing framework for shared memory. In Proceedings of the ACM SIGPLAN. 135–146.
[73]
Ying Tai, Jian Yang, Xiaoming Liu, and Chunyan Xu. 2017. Memnet: A persistent memory network for image restoration. In Proceedings of the IEEE ICCV. 4539–4547.
[74]
Carlos H. C. Teixeira, Alexandre J. Fonseca, Marco Serafini, Georgos Siganos, Mohammed J. Zaki, and Ashraf Aboulnaga. 2015. Arabesque: A system for distributed graph mining. In Proceedings of the ACM SOSP. 425–440.
[75]
Twitter Graph. 2025. Twitter. Retrieved from http://an.kaist.ac.kr/traces/WWW2010.html
[76]
UKdomain Graph. 2025. UK domain (2007) network dataset – KONECT. Retrieved from http://konect.cc/networks/dimacs10-uk-2007-05
[77]
Haris Volos, Andres Jaan Tack, and Michael M. Swift. 2011. Mnemosyne: Lightweight persistent memory. ACM SIGARCH CAN 39, 1 (2011), 91–104.
[78]
Keval Vora. 2019. LUMOS: Dependency-driven disk-based graph processing. In Proceedings of the USENIX ATC. 429–442.
[79]
Keval Vora, Guoqing (Harry) Xu, and Rajiv Gupta. 2016. Load the edges you need: A generic I/O optimization for disk-based graph processing. In Proceedings of the USENIX ATC. 507–522.
[80]
Jing Wang, Youyou Lu, Qing Wang, Yuhao Zhang, and Jiwu Shu. 2024. Perseid: A secondary indexing mechanism for LSM-based storage systems. ACM TS 20, 2 (2024), 1–28.
[81]
Qing Wang, Youyou Lu, Junru Li, and Jiwu Shu. 2021. Nap: A black-box approach to NUMA-aware persistent memory indexes. In Proceedings of the USENIX OSDI. 93–111.
[82]
Rui Wang, Yongkun Li, Hong Xie, Yinlong Xu, and John C. S. Lui. 2020. GraphWalker: An I/O-efficient and resource-friendly graph analytic system for fast and scalable random walks. In Proceedings of the USENIX ATC. 559–571.
[83]
Rui Wang, Weixu Zong, Shuibing He, Xinyu Chen, Zhenxin Li, and Zheng Dang. 2024. Efficient large graph processing with chunk-based graph representation model. In Proceedings of the USENIX ATC. 1239–1255.
[84]
Martin Winter, Daniel Mlakar, Rhaleb Zayer, Hans-Peter Seidel, and Markus Steinberger. 2018. FaimGraph: High performance management of fully-dynamic graphs under tight memory constraints on the GPU. In Proceedings of the IEEE SC. 754–766.
[85]
Martin Winter, Rhaleb Zayer, and Markus Steinberger. 2017. Autonomous, independent management of dynamic graphs on GPUs. In Proceedings of the IEEE HPEC. 1–7.
[86]
Hobin Woo, Daegyu Han, Seungjoon Ha, Sam H. Noh, and Beomseok Nam. 2023. On stacking a persistent memory file system on legacy file systems. In Proceedings of the USENIX FAST. 281–296.
[87]
Fei Xia, Dejun Jiang, Jin Xiong, and Ninghui Sun. 2017. HiKV: A hybrid index key-value store for DRAM-NVM memory systems. In Proceedings of the USENIX ATC. 349–362.
[88]
Xiangyu Xiang, Yu Hua, and Hao Xu. 2023. PMA: A persistent memory allocator with high efficiency and crash consistency guarantee. In Proceedings of the IEEE ICCD. IEEE, 182–189.
[89]
Jian Xu and Steven Swanson. 2016. NOVA: A log-structured file system for hybrid volatile/non-volatile main memories. In Proceedings of the USENIX FAST. 323–338.
[90]
Jian Xu, Lu Zhang, Amirsaman Memaripour, Akshatha Gangadharaiah, Amit Borase, Tamires Brito Da Silva, Steven Swanson, and Andy Rudoff. 2017. NOVA-fortis: A fault-tolerant non-volatile main memory file system. In Proceedings of the ACM SOSP. 478–496.
[91]
Xianghao Xu, Hong Jiang, Fang Wang, Yongli Cheng, and Peng Fang. 2022. Graphsd: A state and dependency aware out-of-core graph processing system. In Proceedings of the ACM ICPP. 1–11.
[92]
Yahoo Graph. 2025. Yahoo Webscope. Yahoo! AltaVista Web Page Hyperlink Connectivity Graph. Retrieved from http://webscope.sandbox.yahoo.com
[93]
Jian Yang, Juno Kim, Morteza Hoseinzadeh, Joseph Izraelevitz, and Steve Swanson. 2020. An empirical guide to the behavior and use of scalable persistent memory. In Proceedings of the USENIX FAST. 169–182.
[94]
Tsun-Yu Yang, Yizou Chen, Yuhong Liang, and Ming-Chang Yang. 2024. Seraph: Towards scalable and efficient fully-external graph computation via on-demand processing. In Proceedings of the USENIX FAST. 373–387.
[95]
Yiwen Zhang, Jian Zhou, Xinhao Min, Song Ge, Jiguang Wan, Ting Yao, and Daohui Wang. 2022. PetaKV: Building efficient key-value store for file system metadata on persistent memory. IEEE TPDS 34, 3 (2022), 843–855.
[96]
Xiaowei Zhu, Wenguang Chen, Weimin Zheng, and Xiaosong Ma. 2016. Gemini: A computation-centric distributed graph processing system. In Proceedings of the USENIX OSDI. 301–316.
[97]
Xiaowei Zhu, Guanyu Feng, Serafini Marco, Xiaosong Ma, Jiping Yu, Lei Xie, Aboulnaga Ashraf, and Wenguang Chen. 2020. LiveGraph: A transactional graph storage system with purely sequential adjacency list scans. PVLDB 13, 7 (2020), 1020–1034.
[98]
Xiaowei Zhu, Wentao Han, and Wenguang Chen. 2015. GridGraph: Large-scale graph processing on a single machine using 2-level hierarchical partitioning. In Proceedings of the USENIX ATC. 375–386.

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Storage
ACM Transactions on Storage  Volume 21, Issue 2
May 2025
244 pages
EISSN:1553-3093
DOI:10.1145/3697221
Issue’s Table of Contents

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 February 2025
Online AM: 25 January 2025
Accepted: 12 January 2025
Revised: 23 December 2024
Received: 29 September 2024
Published in TOS Volume 21, Issue 2

Check for updates

Author Tags

  1. Dynamic graphs processing
  2. persistent memory
  3. graph storage

Qualifiers

  • Research-article

Funding Sources

  • Science and Technology Program of Zhejiang Province
  • Hangzhou Joint Fund of the Zhejiang Provincial Natural Science Foundation of China
  • Zhejiang University Education Foundation Qizhen Scholar Foundation
  • National Science Foundation of China
  • Zhejiang Province
  • Pioneer and Leading Goose R&D Program of Zhejiang Province

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 184
    Total Downloads
  • Downloads (Last 12 months)184
  • Downloads (Last 6 weeks)184
Reflects downloads up to 07 Mar 2025

Other Metrics

Citations

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Full Text

View this article in Full Text.

Full Text

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media