research-article

Public Access

Exploring Performance Characteristics of the Optane 3D Xpoint Storage Technology

Authors:

David J. LiljaAuthors Info & Claims

ACM Transactions on Modeling and Performance Evaluation of Computing Systems (TOMPECS), Volume 5, Issue 1

Article No.: 4, Pages 1 - 28

https://doi.org/10.1145/3372783

Published: 04 February 2020 Publication History

All formats PDF

Abstract

Intel’s Optane solid-state nonvolatile storage device is constructed using their new 3D Xpoint technology. Although it is claimed that this technology can deliver substantial performance improvements compared to NAND-based storage systems, its performance characteristics have not been well studied. In this study, intensive experiments and measurements have been carried out to extract the intrinsic performance characteristics of the Optane SSD, including the basic I/O performance behavior, advanced interleaving technology, performance consistency under a highly intensive I/O workload, influence of unaligned request size, elimination of write-driven garbage collection, read disturb issues, and tail latency problem. The performance is compared to that of a conventional NAND SSD to indicate the performance difference of the Optane SSD in each scenario. In addition, by using TPC-H, a read-intensive benchmark, a database system’s performance has been studied on our target storage devices to quantify the potential benefits of the Optane SSD to a real application. Finally, the performance impact of hybrid Optane and NAND SSD storage systems on a database application has been investigated.

References

[1]

Transaction Processing Performance Council. TPC-H Benchmark. http://www.tpc.org/tpch/.

[2]

Jens Axboe. 2016. Flexible I/O Tester. https://github.com/axboe/fio.

[3]

Aleph One Company. 2012. Yet Another Flash File System (YAFFS). https://yaffs.net/.

[4]

Intel Corporation. 2014. Intel SSD DC P3700 Series. https://ark.intel.com/content/www/us/en/ark/products/series/79628/intel-ssd-dc-p3700-series.html.

[5]

Oracle Corporation. 2017. MySQL 5.7. https://dev.mysql.com/doc/refman/5.7/en/.

[6]

Intel Corporation. 2018. Product Brief: Intel Optane SSD DC P4800X Series. Retrieved from https://www.intel.com/content/www/us/en/solid-state-drives/optane-ssd-dc-p4800x-brief.html.

[7]

Intel Corporation. 2018. World’s Most Responsive Data Center SSD. Retrieved from https://www.intel.com/content/dam/www/public/us/en/documents/product-briefs/optane-ssd-dc-p4800x-brief.pdf.

[8]

Nitin Agrawal, Vijayan Prabhakaran, Ted Wobber, John D. Davis, Mark S. Manasse, and Rina Panigrahy. 2008. Design tradeoffs for SSD performance. In USENIX Annual Technical Conference, Vol. 57.

[9]

Alan D. Brunelle. 2006. Block I/O layer tracing: blktrace. HP, Gelato-Cupertino, CA.

[10]

Mustafa Canim, George A. Mihaila, Bishwaranjan Bhattacharjee, Kenneth A. Ross, and Christian A. Lang. 2010. SSD bufferpool extensions for database systems. Proceedings of the VLDB Endowment 3, 1–2 (2010), 1435--1446.

Digital Library

[11]

Feng Chen, David A. Koufaty, and Xiaodong Zhang. 2009. Understanding intrinsic characteristics and system implications of flash memory based solid state drives. In ACM SIGMETRICS Performance Evaluation Review, Vol. 37. ACM, 181--192.

Digital Library

[12]

Feng Chen, Rubao Lee, and Xiaodong Zhang. 2011. Essential roles of exploiting internal parallelism of flash memory based solid state drives in high-speed data processing. In IEEE 17th International Symposium on High Performance Computer Architecture (HPCA’11). IEEE, 266--277.

[13]

Cagdas Dirik and Bruce Jacob. 2009. The performance of PC solid-state disks (SSDs) as a function of bandwidth, concurrency, device architecture, and system organization. In ACM SIGARCH Computer Architecture News, Vol. 37. ACM, 279--289.

Digital Library

[14]

Assaf Eisenman, Darryl Gardner, Islam AbdelRahman, Jens Axboe, Siying Dong, Kim Hazelwood, Chris Petersen, Asaf Cidon, and Sachin Katti. 2018. Reducing DRAM footprint with NVM in Facebook. In Proceedings of the 13th EuroSys Conference. ACM, 42.

Digital Library

[15]

Holloway H. Frost, Charles J. Camp, Timothy J. Fisher, James A. Fuxa, and Lance W. Shelton. 2010. Efficient reduction of read disturb errors in NAND flash memory. US Patent 7,818,525.

[16]

Eran Gal and Sivan Toledo. 2005. A transactional flash file system for microcontrollers. In USENIX Annual Technical Conference, General Track. 89--104.

[17]

Aayush Gupta, Youngjae Kim, and Bhuvan Urgaonkar. 2009. DFTL: A Flash Translation Layer Employing Demand-Based Selective Caching of Page-level Address Mappings, Vol. 44. ACM.

Digital Library

[18]

Frank T. Hady, Annie Foong, Bryan Veal, and Dan Williams. 2017. Platform storage performance with 3D XPoint technology. Proceedings of the IEEE 105, 9 (2017), 1822--1833.

[19]

Mingzhe Hao, Gokul Soundararajan, Deepak R. Kenchammana-Hosekote, Andrew A. Chien, and Haryadi S. Gunawi. 2016. The tail at store: A revelation from millions of hours of disk and SSD deployments. In 14th USENIX Conference on File and Storage Technologies (FAST'16). 263--276.

[20]

Yang Hu, Hong Jiang, Dan Feng, Lei Tian, Hao Luo, and Chao Ren. 2013. Exploring and exploiting the multilevel parallelism inside SSDs for improved performance and endurance. IEEE Transactions on Computers 62, 6 (2013), 1141--1155.

Digital Library

[21]

Joseph Izraelevitz, Jian Yang, Lu Zhang, Juno Kim, Xiao Liu, Amirsaman Memaripour, Yun Joon Soh, Zixuan Wang, Yi Xu, Subramanya R. Dulloor, Jishen Zhao, and Steven Swanson. 2019. Basic performance measurements of the Intel Optane DC persistent memory module. arXiv preprint arXiv:1903.05714 (2019).

[22]

Seongwook Jin, Jaehong Kim, Jaegeuk Kim, Jaehyuk Huh, and Seungryoul Maeng. 2011. Sector log: Fine-grained storage management for solid state drives. In Proceedings of the 2011 ACM Symposium on Applied Computing. ACM, 360--367.

Digital Library

[23]

William K. Josephson, Lars A. Bongo, Kai Li, and David Flynn. 2010. DFS: A file system for virtualized flash storage. ACM Transactions on Storage (TOS) 6, 3 (2010), 14.

[24]

Dawoon Jung, Jeong-Uk Kang, Heeseung Jo, Jin-Soo Kim, and Joonwon Lee. 2010. Superblock FTL: A superblock-based flash translation layer with a hybrid address translation scheme. ACM Transactions on Embedded Computing Systems (TECS) 9, 4 (2010), 40.

Digital Library

[25]

Jeong-Uk Kang, Heeseung Jo, Jin-Soo Kim, and Joonwon Lee. 2006. A superblock-based flash translation layer for NAND flash memory. In Proceedings of the 6th ACM ' IEEE International Conference on Embedded Software. ACM, 161--170.

Digital Library

[26]

Jesung Kim, Jong Min Kim, Sam H. Noh, Sang Lyul Min, and Yookun Cho. 2002. A space-efficient flash translation layer for CompactFlash systems. IEEE Transactions on Consumer Electronics 48, 2 (2002), 366--375.

Digital Library

[27]

Jesung Kim, Jong Min Kim, Sam H. Noh, Sang Lyul Min, and Yookun Cho. 2002. A space-efficient flash translation layer for CompactFlash systems. IEEE Transactions on Consumer Electronics 48, 2 (2002), 366--375.

Digital Library

[28]

Jaeho Kim, Yongseok Oh, Eunsam Kim, Jongmoo Choi, Donghee Lee, and Sam H. Noh. 2009. Disk schedulers for solid state drivers. In Proceedings of the 7th ACM International Conference on Embedded Software. ACM, 295--304.

[29]

Jaehong Kim, Sangwon Seo, Dawoon Jung, Jin-Soo Kim, and Jaehyuk Huh. 2012. Parameter-aware I/O management for solid state disks (SSDs). IEEE Transactions on Computers 61, 5 (2012), 636--649.

Digital Library

[30]

Michael Kund, Gerhard Beitel, C.-U. Pinnow, Thomas Rohr, Jorg Schumann, Ralf Symanczyk, K. Ufert, and Gerhard Muller. 2005. Conductive bridging RAM (CBRAM): An emerging non-volatile memory technology scalable to sub 20nm. In IEEE International Electron Devices Meeting, 2005. IEDM Technical Digest. IEEE, 754--757.

[31]

Simone Lavizzari, Daniele Ielmini, Deepak Sharma, and Andrea Leonardo Lacaita. 2008. Transient effects of delay, switching and recovery in phase change memory (PCM) devices. In IEEE International Electron Devices Meeting, 2008 (IEDM’08). IEEE, 1--4.

[32]

Sang-Won Lee, Bongki Moon, and Chanik Park. 2009. Advances in flash memory SSD technology for enterprise database applications. In Proceedings of the 2009 ACM SIGMOD International Conference on Management of Data. ACM, 863--870.

Digital Library

[33]

Sang-Won Lee, Bongki Moon, Chanik Park, Jae-Myung Kim, and Sang-Woo Kim. 2008. A case for flash memory SSD in enterprise database applications. In Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data. ACM, 1075--1086.

Digital Library

[34]

Sang-Won Lee, Dong-Joo Park, Tae-Sun Chung, Dong-Ho Lee, Sangwon Park, and Ha-Joo Song. 2007. A log buffer-based flash translation layer using fully-associative sector translation. ACM Transactions on Embedded Computing Systems (TECS) 6, 3 (2007), 18.

Digital Library

[35]

Sang-Won Lee, Dong-Joo Park, Tae-Sun Chung, Dong-Ho Lee, Sangwon Park, and Ha-Joo Song. 2007. A log buffer-based flash translation layer using fully-associative sector translation. ACM Transactions on Embedded Computing Systems (TECS) 6, 3 (2007), 18.

Digital Library

[36]

Youyou Lu, Jiwu Shu, and Weimin Zheng. 2013. Extending the lifetime of flash-based storage through reducing write amplification from file systems. In 11th USENIX Conference on File and Storage Technologies (FAST'13), Vol. 13.

[37]

Tian Luo, Rubao Lee, Michael Mesnier, Feng Chen, and Xiaodong Zhang. 2012. hStorage-DB: Heterogeneity-aware data management to exploit the full capability of hybrid storage systems. Proceedings of the Conference on Very Large Data Bases (VLDB) Endowment 5, 10 (2012), 1076--1087.

Digital Library

[38]

Paul Saab Mohan Srinivasan. 2010. Flashcache. Retrieved from https://github.com/facebookarchive/flashcache.

[39]

Mark Moshayedi and Patrick Wilkison. 2008. Enterprise SSDs. Queue 6, 4 (2008), 32--39.

Digital Library

[40]

Prashant J. Nair, Chiachen Chou, Bipin Rajendran, and Moinuddin K. Qureshi. 2015. Reducing read latency of phase change memory via early read and turbo read. In 2015 IEEE 21st International Symposium on High Performance Computer Architecture (HPCA’15). IEEE, 309--319.

[41]

Hiroyuki Nakamoto, Daisuke Yamazaki, Takuji Yamamoto, Hajime Kurata, Satoshi Yamada, Kenji Mukaida, Tsuzumi Ninomiya, Takashi Ohkawa, Shoichi Masui, and Kunihiko Gotoh. 2007. A passive UHF RF identification CMOS tag IC using ferroelectric RAM in 0.35-um technology. IEEE Journal of Solid-State Circuits 42, 1 (2007), 101--110.

[42]

Dushyanth Narayanan, Austin Donnelly, and Antony Rowstron. 2008. Write off-loading: Practical power management for enterprise storage. ACM Transactions on Storage (TOS) 4, 3 (2008), 10.

Digital Library

[43]

Seon-yeong Park, Euiseong Seo, Ji-Yong Shin, Seungryoul Maeng, and Joonwon Lee. 2010. Exploiting internal parallelism of flash-based SSDs. IEEE Computer Architecture Letters 9, 1 (2010), 9--12.

Digital Library

[44]

Gokul Soundararajan, Vijayan Prabhakaran, Mahesh Balakrishnan, and Ted Wobber. 2010. Extending SSD lifetimes with disk-based write caches. In USENIX Conference on File and Storage Technologies (FAST'10), Vol. 10. 101--114.

[45]

Hermann Strass. 2016. An Introduction to NVMe. https://labs.seagate.com/wp-content/uploads/sites/7/2018/09/an-introduction-to-nvme-tp690-1-1605us.pdf.

[46]

Beth Trushkowsky, Peter Bodík, Armando Fox, Michael J. Franklin, Michael I. Jordan, and David A. Patterson. 2011. The SCADS director: Scaling a distributed storage system under stringent performance requirements. In USENIX Conference on File and Storage Technologies (FAST'11), Vol. 11. 163--176.

[47]

H.-S. Philip Wong, Simone Raoux, SangBum Kim, Jiale Liang, John P. Reifenberg, Bipin Rajendran, Mehdi Asheghi, and Kenneth E. Goodson. 2010. Phase change memory. Proceedings of the IEEE 98, 12 (2010), 2201--2227.

[48]

David Woodhouse. 2001. JFFS: The journalling flash file system. In Ottawa Linux Symposium, Vol. 2001.

[49]

Qiumin Xu, Huzefa Siyamwala, Mrinmoy Ghosh, Tameesh Suri, Manu Awasthi, Zvika Guz, Anahita Shayesteh, and Vijay Balakrishnan. 2015. Performance analysis of NVMe SSDs and their implication on real world databases. In Proceedings of the 8th ACM International Systems and Storage Conference. ACM, 6.

Digital Library

[50]

Jinfeng Yang and David J. Lilja. 2018. Reducing relational database performance bottlenecks using 3D XPoint storage technology. In 2018 17th IEEE International Conference on Trust, Security and Privacy in Computing and Communications/12th IEEE International Conference on Big Data Science and Engineering (TrustCom/BigDataSE’18). IEEE, 1804--1808.

[51]

Jiachen Zhang, Peng Li, Bo Liu, Trent G. Marbach, Xiaoguang Liu, and Gang Wang. 2018. Performance analysis of 3D XPoint SSDs in virtualized and non-virtualized environments. In 2018 IEEE 24th International Conference on Parallel and Distributed Systems (ICPADS’18). IEEE, 1--10.

Cited By

Boopathy ESathish Kumar DGeorge A(2024)Advancements in Next-Generation Memory TechnologiesInterdisciplinary Approaches to AI, Internet of Everything, and Machine Learning10.4018/979-8-3373-1032-9.ch002(25-48)Online publication date: 20-Dec-2024
https://doi.org/10.4018/979-8-3373-1032-9.ch002
Li ZZhang GWang Y(2024)Flash-oriented Coded Storage: Research Status and Future DirectionsACM Transactions on Storage10.1145/370899521:1(1-37)Online publication date: 19-Dec-2024
https://dl.acm.org/doi/10.1145/3708995
Jaranilla CShin HYoo SCho SChoi J(2024)Tiered Storage in Modern Key-Value Stores: Performance, Storage-Efficiency, and Cost-Efficiency Considerations2024 IEEE International Conference on Big Data and Smart Computing (BigComp)10.1109/BigComp60711.2024.00032(151-158)Online publication date: 18-Feb-2024
https://doi.org/10.1109/BigComp60711.2024.00032
Show More Cited By

Index Terms

Exploring Performance Characteristics of the Optane 3D Xpoint Storage Technology
1. General and reference
  1. Cross-computing tools and techniques
    1. Evaluation
    2. Performance
2. Information systems
  1. Information storage systems
    1. Information storage technologies
      1. Storage class memory
    2. Storage management
      1. Hierarchical storage management

Recommendations

The Nonvolatile Memory Transformation of Client Storage

The introduction of solid-state drives in mainstream computing systems is reshaping the client storage landscape. PCI Express is supplanting Serial ATA to deliver higher performance, while novel caching solutions are extending the benefits of ...
Impact of flash memory on video-on-demand storage: analysis of tradeoffs
MMSys '11: Proceedings of the second annual ACM conference on Multimedia systems

There is no doubt that video-on-demand (VoD) services are very popular these days. However, disk storage is a serious bottleneck limiting the scalability of a VoD server. Disk throughput degrades dramatically due to seek time overhead when the server is ...
Read-Performance Optimization for Deduplication-Based Storage Systems in the Cloud

Data deduplication has been demonstrated to be an effective technique in reducing the total data transferred over the network and the storage space in cloud backup, archiving, and primary storage systems, such as VM (virtual machine) platforms. However, ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Modeling and Performance Evaluation of Computing Systems

ACM Transactions on Modeling and Performance Evaluation of Computing Systems Volume 5, Issue 1

Special Issue on Qest 2018 and Regular Papers

March 2020

137 pages

ISSN:2376-3639

EISSN:2376-3647

DOI:10.1145/3381922

Editors:
Sem Borst
Nokia Bell Labs/Eindhoven University of Technology, the Netherlands
,
Carey Williamson
University of Calgary, Canada

Issue’s Table of Contents

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 February 2020

Accepted: 01 November 2019

Revised: 01 September 2019

Received: 01 April 2019

Published in TOMPECS Volume 5, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

National Science Foundation
Center for Research in Intelligent Storage (CRIS)

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

19
Total Citations
View Citations
2,599
Total Downloads

Downloads (Last 12 months)540
Downloads (Last 6 weeks)73

Reflects downloads up to 22 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Boopathy ESathish Kumar DGeorge A(2024)Advancements in Next-Generation Memory TechnologiesInterdisciplinary Approaches to AI, Internet of Everything, and Machine Learning10.4018/979-8-3373-1032-9.ch002(25-48)Online publication date: 20-Dec-2024
https://doi.org/10.4018/979-8-3373-1032-9.ch002
Li ZZhang GWang Y(2024)Flash-oriented Coded Storage: Research Status and Future DirectionsACM Transactions on Storage10.1145/370899521:1(1-37)Online publication date: 19-Dec-2024
https://dl.acm.org/doi/10.1145/3708995
Jaranilla CShin HYoo SCho SChoi J(2024)Tiered Storage in Modern Key-Value Stores: Performance, Storage-Efficiency, and Cost-Efficiency Considerations2024 IEEE International Conference on Big Data and Smart Computing (BigComp)10.1109/BigComp60711.2024.00032(151-158)Online publication date: 18-Feb-2024
https://doi.org/10.1109/BigComp60711.2024.00032
Lu RXu EZhang YZhu FZhu ZWang MZhu ZXue GShu JLi MWu JNaor DGoel A(2023)PERSEUSProceedings of the 21st USENIX Conference on File and Storage Technologies10.5555/3585938.3585942(49-63)Online publication date: 21-Feb-2023
https://dl.acm.org/doi/10.5555/3585938.3585942
Park JEom Y(2023)Filesystem Fragmentation on Modern Storage SystemsACM Transactions on Computer Systems10.1145/361138641:1-4(1-27)Online publication date: 18-Dec-2023
https://dl.acm.org/doi/10.1145/3611386
Cai ZTang CLi MTrahay FLi JSha ZWu JYang FLiao J(2023)Re-aligning Across-page Requests for Flash-based Solid-state DrivesProceedings of the 52nd International Conference on Parallel Processing10.1145/3605573.3605652(736-745)Online publication date: 7-Aug-2023
https://dl.acm.org/doi/10.1145/3605573.3605652
Carniel AAguiar C(2023)Spatial Index Structures for Modern Storage Devices: A SurveyIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2023.324220735:9(9578-9597)Online publication date: 1-Sep-2023
https://doi.org/10.1109/TKDE.2023.3242207
Lee HChoi KLee HSim J(2023)SDM: Sharing-enabled Disaggregated Memory System with Cache Coherent Compute Express LinkProceedings of the 32nd International Conference on Parallel Architectures and Compilation Techniques10.1109/PACT58117.2023.00016(86-98)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1109/PACT58117.2023.00016
Oliveira GGhose SGómez-Luna JBoroumand ASavery ARao SQazi SGrignou GThakur RShiu EMutlu O(2023)Extending Memory Capacity in Modern Consumer Systems With Emerging Non-Volatile Memory: Experimental Analysis and Characterization Using the Intel Optane SSDIEEE Access10.1109/ACCESS.2023.331788411(105843-105871)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3317884
Hill JBlanco JShey JRakvic RWalker O(2022)Toward Classification of Phase Change Memory and 3D NAND Flash SSDs Using Power-based Side-channel Analysis in the Time-domainProceedings of the 2022 International Symposium on Memory Systems10.1145/3565053.3565065(1-7)Online publication date: 3-Oct-2022
https://dl.acm.org/doi/10.1145/3565053.3565065
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Figures

Tables

Media

View Issue’s Table of Contents