research-article

Open access

Project Silica: Towards Sustainable Cloud Archival Storage in Glass

Authors:

Patrick Anderson,

Erika Blancada Aranas,

Raphael Behrendt,

Marco Caballero,

Pashmina Cameron,

Thales De Carvalho,

Andromachi Chatzieleftheriou,

Rebekah Storan Clarke,

Daniel Cletheroe,

Bridgette Cooper,

Austin Donnelly,

Rokas Drevinskas,

Alexander Gaunt,

Christos Gkantsidis,

Ariel Gomez Diaz,

Teodora Ilieva,

Shashidhar Joshi,

Sergey Legtchenko,

Fanglin Linda Liu,

Bruno Magalhaes,

Jayashree Mohan,

Sebastian Nowozin,

Antony Rowstron,

Masaaki Sakakura,

Nina Schreiner,

Ioan Stefanovici,

Jonathan Westcott,

Charles Whittaker,

Pablo Wilke Berenguer,

Thomas Winkler, and

Stefan WinzeckAuthors Info & Claims

SOSP '23: Proceedings of the 29th Symposium on Operating Systems Principles

October 2023

Pages 166 - 181

https://doi.org/10.1145/3600006.3613208

Published: 23 October 2023 Publication History

Abstract

Sustainable and cost-effective long-term storage remains an unsolved problem. The most widely used storage technologies today are magnetic (hard disk drives and tape). They use media that degrades over time and has a limited lifetime, which leads to inefficient, wasteful, and costly solutions for long-lived data. This paper presents Silica: the first cloud storage system for archival data underpinned by quartz glass, an extremely resilient media that allows data to be left in situ indefinitely. The hardware and software of Silica have been co-designed and co-optimized from the media up to the service level with sustainability as a primary objective. The design follows a cloud-first, data-driven methodology underpinned by principles derived from analyzing the archival workload of a large public cloud service. Silica can support a wide range of archival storage workloads and ushers in a new era of sustainable, cost-effective storage.

References

[1]

2023. Life Cycle Assessment of Different Storage Media. Report prepared for Microsoft by WSP Consulting.

[2]

U.S. Environmental Protection Agency. 1990. Magnetic Tape Manufacturing. In AP 42, Fifth Edition, Volume I. 4.2.2.13.

[3]

U.S. Environmental Protection Agency. 2006. Magnetic Tape Manufacturing Operations: National Emission Standards for Hazardous Air Pollutants (NESHAP). https://www.epa.gov/stationary-sources-air-pollution/magnetic-tape-manufacturing-operations-national-emission-standards

[4]

National Digital Stewardship Alliance. [n. d.]. Checking Your Digital Content. http://hdl.loc.gov/loc.gdc/lcpub.2013655117.1

[5]

George Amvrosiadis, Alina Oprea, and Bianca Schroeder. 2012. Practical scrubbing: Getting to the bad sector at the right time. In IEEE/IFIP International Conference on Dependable Systems and Networks (DSN 2012). 1--12.

[6]

AWS. 2023. Amazon S3 Glacier storage classes. https://docs.aws.amazon.com/AmazonS3/latest/userguide/storage-class-intro.html

[7]

AWS. 2023. What is Amazon S3 Glacier? https://docs.aws.amazon.com/glacier/index.html

[8]

Shobana Balakrishnan, Richard Black, Austin Donnelly, Paul England, Adam Glass, Dave Harper, Sergey Legtchenko, Aaron Ogus, Eric Peterson, and Antony Rowstron. 2014. Pelican: A Building Block for Exascale Cold Data Storage. In 11th USENIX Symposium on Operating Systems Design and Implementation (OSDI 14). USENIX Association, Broomfield, CO, 351--365. https://www.usenix.org/conference/osdi14/technical-sessions/presentation/balakrishnan

[9]

Zahy Bnaya and Ariel Felner. 2014. Conflict-Oriented Windowed Hierarchical Cooperative A*. In 2014 IEEE International Conference on Robotics and Automation (ICRA). IEEE Robotics and Automation Society, 3743--3748.

[10]

Brad Calder, Ju Wang, Aaron Ogus, Niranjan Nilakantan, Arild Skjolsvold, Sam McKelvie, Yikang Xu, Shashwat Srivastav, Jiesheng Wu, Huseyin Simitci, Jaidev Haridas, Chakravarthy Uddaraju, Hemal Khatri, Andrew Edwards, Vaman Bedekar, Shane Mainali, Rafay Abbasi, Arpit Agarwal, Mian Fahim ul Haq, Muhammad Ikram ul Haq, Deepali Bhardwaj, Sowmya Dayanand, Anitha Adusumilli, Marvin McNett, Sriram Sankaran, Kavitha Manivannan, and Leonidas Rigas. 2011. Windows Azure Storage: A Highly Available Cloud Storage Service with Strong Consistency. In Proceedings of the Twenty-Third ACM Symposium on Operating Systems Principles (SOSP '11). Association for Computing Machinery, New York, NY, USA, 143--157.

Digital Library

[11]

Germ Cancio, Vladim Bahyl, Daniele Francesco Kruse, Julien Leduc, Eric Cano, and Steven Murray. 2015. Experiences and challenges running CERN's high capacity tape archive. Journal of Physics: Conference Series. 4 (2015).

[12]

Robert Allen Carlton. 2011. Polarized Light Microscopy. Springer New York, New York, NY, 7--64.

[13]

Luis Ceze, Jeff Nivala, and Karin Strauss. 2019. Molecular Digital Data Storage using DNA. Nature Reviews Genetics (May 2019). https://www.microsoft.com/en-us/research/publication/molecular-digital-data-storage-using-dna/

[14]

Andromachi Chatzieleftheriou, Ioan Stefanovici, Dushyanth Narayanan, Benn Thomsen, and Antony Rowstron. 2020. Could cloud storage be disrupted in the next decade?. In 12th USENIX Workshop on Hot Topics in Storage and File Systems (HotStorage 20). USENIX Association. https://www.usenix.org/conference/hotstorage20/presentation/chatzieleftheriou

[15]

Lin Chen, Yaonan Wang, Yang Mo, Zhiqiang Miao, Hesheng Wang, Mingtao Feng, and Sifei Wang. 2023. Multiagent Path Finding Using Deep Reinforcement Learning Coupled With Hot Supervision Contrastive Loss. IEEE Transactions on Industrial Electronics 70, 7 (2023), 7032--7040.

[16]

D. Colarelli and D. Grunwald. 2002. Massive Arrays of Idle Disks For Storage Archives. In SC '02: Proceedings of the 2002 ACM/IEEE Conference on Supercomputing. 47--47.

[17]

Hans J Coufal, Demetri Psaltis, and Glenn T Sincerbox (Eds.). 2000. Holographic Data Storage. Springer Series in Optical Sciences, Vol. 76. Springer.

[18]

Michael W. Davidson and Gary E. Lofgren. 1991. Photomicrography in the Geological Sciences. Journal of Geological Education 39, 5 (1991), 403--418.

[19]

Boris de Wilde, Adriaan W. ter Mors, and Cees Witteveen. 2013. Push and Rotate: Cooperative Multi-Agent Path Planning. In Proceedings of the 2013 International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS '13). International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC, 87--94.

[20]

George Dickinson, Golam Mortuza, William Clay, Luca Piantanida, Christopher Green, Chad Watson, Eric Hayden, Tim Andersen, Wan Kuang, Elton Graugnard, and William Hughes. 2021. An alternative approach to nucleic acid memory. Nature Communications 12 (04 2021).

[21]

Simeon Furrer, Mark A. Lantz, Peter Reininger, Angeliki Pantazi, Hugo E. Rothuizen, Roy D. Cideciyan, Giovanni Cherubini, Walter Haeberle, Evangelos Eleftheriou, Junichi Tachibana, Noboru Sekiguchi, Takashi Aizawa, Tetsuo Endo, Tomoe Ozaki, Teruo Sai, Ryoichi Hiratsuka, Satoshi Mitamura, and Atsushi Yamaguchi. 2018. 201 Gb/in² Recording Areal Density on Sputtered Magnetic Tape. IEEE Transactions on Magnetics 54, 2 (Feb. 2018), 1--8. http://ieeexplore.ieee.org/document/7984852/

[22]

Christos Gkantsidis and Pablo Rodriguez Rodriguez. 2005. Network coding for large scale content distribution. In 24th Annual Joint Conference of the IEEE Computer and Communications Societies (INFOCOM, Vol. 4). IEEE, 2235--2245.

[23]

E. N. Glezer, M. Milosavljevic, L. Huang, R. J. Finlay, T.-H. Her, J. P. Callan, and E. Mazur. 1996. Three-dimensional optical storage inside transparent materials. Opt. Lett. 21, 24 (Dec 1996), 2023--2025.

[24]

Chao He, Honghui He, Jintao Chang, Binguo Chen, Hui Ma, and Martin Booth. 2021. Polarisation optics for biomedical and clinical applications: a review. Light: Science & Applications 10 (09 2021), 194.

[25]

IBM. 2023. TS4500 Tape Libray Documentation. https://www.ibm.com/docs/en/STQRQ9_1.9.1/pdf/ts4500-tape-library-1.9.1-documentation.pdf

[26]

IDC. 2018. The Digitization of the World From Edge to Core. https://www.seagate.com/files/www-content/our-story/trends/files/idc-seagate-dataage-whitepaper.pdf

[27]

RTI International. 2005. Hazardous Air Pollutant Emissions From Magnetic Tape Manufacturing Operations - Background Information for Technology and Residual Risk Review. Technical Report. https://nepis.epa.gov/Exe/ZyPURL.cgi?Dockey=9100923X.TXT

[28]

Adib Keikhosravi, Michael Shribak, Matthew Conklin, Yuming Liu, Bin Li, Agnes Loeffler, Richard Levenson, and Kevin Eliceiri. 2021. RealTime Polarization Microscopy of Fibrillar Collagen in Histopathology. Scientific Reports (05 2021).

[29]

Edward Lam, Pierre Le Bodic, Daniel Harabor, and Peter J. Stuckey. 2022. Branch-and-cut-and-price for multi-agent path finding. Computers & Operations Research 144 (2022), 105809.

Digital Library

[30]

David J. C. MacKay. 2003. Information Theory, Inference, and Learning Algorithms. Copyright Cambridge University Press.

Digital Library

[31]

Microsoft. 2023. Azure Archive Storage. https://learn.microsoft.com/en-us/azure/storage/blobs/access-tiers-overview

[32]

Microsoft. 2023. Azure Archive Storage. https://azure.microsoft.com/en-us/products/storage/

[33]

Rich Miller. 2015. Inside Facebook's Blu-Ray Cold Storage Data Center. https://www.datacenterfrontier.com/cloud/article/11431537/inside-facebook8217s-blu-ray-cold-storage-data-centerl

[34]

Keisuke Okumura and Sébastien Tixeuil. 2022. Fault-Tolerant Offline Multi-Agent Path Planning.

[35]

The Council on Library and Information Resources. [n. d.]. Magnetic Tape Storage and Handling: A Guide for Libraries and Archives. https://www.clir.org/wp-content/uploads/sites/6/pub54.pdf

[36]

Google Cloud Plattform. 2023. Storage classes. https://cloud.google.com/storage/docs/storage-classes#archive

[37]

Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-Net: Convolutional Networks for Biomedical Image Segmentation. In Medical Image Computing and Computer-Assisted Intervention - MICCAI 2015, Nassir Navab, Joachim Hornegger, William M. Wells, and Alejandro F. Frangi (Eds.). Springer International Publishing, Cham, 234--241.

[38]

Bianca Schroeder, Sotirios Damouras, and Phillipa Gill. 2010. Understanding Latent Sector Errors and How to Protect against Them. ACM Trans. Storage 6, 3 (sep 2010).

Digital Library

[39]

T.J.E. Schwarz, Qin Xin, E.L. Miller, D.D.E. Long, A. Hospodor, and S. Ng. 2004. Disk scrubbing in large archival storage systems. In The IEEE Computer Society's 12th Annual International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunications Systems, 2004. (MASCOTS 2004). Proceedings. 409--418.

[40]

Yasuhiko Shimotsuma, Masaaki Sakakura, Peter Kazansky, Martynas Beresna, Jianrong Qiu, Jiarong Qiu, Kiyotaka Miura, and Kazuyuki Hirao. 2010. Ultrafast Manipulation of Self-Assembled Form Birefringence in Glass. Advanced materials (Deerfield Beach, Fla.) 22 (09 2010), 4039--43.

[41]

Manabu Shiozawa, Takao Watanabe, Eriko Tatsu, Mariko Umeda, Toshiyuki Mine, Yasuhiko Shimotsuma, Masaaki Sakakura, Miki Nakabayashi, Kiyotaka Miura, and Koichi Watanabe. 2013. Simultaneous Multi-Bit Recording in Fused Silica for Permanent Storage. Japanese Journal of Applied Physics 52, 9S2 (sep 2013), 09LA01.

[42]

M. Shribak and R Oldenbourg. 2003. Techniques for fast and sensitive measurements of two-dimensional birefringence distributions. Appl. Opt. 42, 16 (June 2003), 3009--3017.

[43]

Karen Simonyan and Andrew Zisserman. 2014. Very Deep Convolutional Networks for Large-Scale Image Recognition. CoRR abs/1409.1556 (2014). http://arxiv.org/abs/1409.1556

[44]

Roni Stern, Nathan R. Sturtevant, Ariel Felner, Sven Koenig, Hang Ma, Thayne T. Walker, Jiaoyang Li, Dor Atzmon, Liron Cohen, T. K. Satish Kumar, Roman Barták, and Eli Boyarski. 2019. MultiAgent Pathfinding: Definitions, Variants, and Benchmarks. In Twelfth Annual Symposium on Combinatorial Search. Association for the Advancement of Artificial Intelligence, 151--158. https://www.aaai.org/ocs/index.php/SOCS/SOCS19/paper/view/18341

[45]

Wikipedia. 2023. Birefringence. https://en.wikipedia.org/wiki/Birefringence

[46]

Wikipedia. 2023. Crypto-shredding. https://en.wikipedia.org/wiki/Crypto-shredding

[47]

Velvet Wu. 2023. Tape Storage Might Be Computing's Climate Savior. https://spectrum.ieee.org/tape-storage-sustainable-option

[48]

Wenrui Yan, Jie Yao, Qiang Cao, Changsheng Xie, and Hong Jiang. 2017. ROS: A Rack-Based Optical Storage System with Inline Accessibility for Long-Term Data Preservation. In Proceedings of the Twelfth European Conference on Computer Systems (EuroSys '17). Association for Computing Machinery, New York, NY, USA, 161--174.

Digital Library

[49]

Jingjin Yu and Steven M. LaValle. 2013. Structure and Intractability of Optimal Multi-Robot Path Planning on Graphs. In Proceedings of the Twenty-Seventh AAAI Conference on Artificial Intelligence (AAAI'13). AAAI Press, 1443--1449.

[50]

Jingyu Zhang, Mindaugas Gecevičius, Martynas Beresna, and Peter G. Kazansky. 2014. Seemingly Unlimited Lifetime Data Storage in Nanostructured Glass. Phys. Rev. Lett. 112 (Jan 2014), 033901.

[51]

J. Zhang, A. Čerkauskaitė, R. Drevinskas, A. Patel, M. Beresna, and P. G. Kazansky. 2016. Eternal 5D data storage by ultrafast laser writing in glass. In Laser-based Micro- and Nanoprocessing X, Udo Klotzbach, Kunihiko Washio, and Craig B. Arnold (Eds.), Vol. 9736. International Society for Optics and Photonics, SPIE, 97360U.

[52]

Kai Zhao, Wenzhe Zhao, Hongbin Sun, Xiaodong Zhang, Nanning Zheng, and Tong Zhang. 2013. LDPC-in-SSD: Making Advanced Error Correction Codes Work Effectively in Solid State Drives. In 11th USENIX Conference on File and Storage Technologies (FAST 13). USENIX Association, San Jose, CA, 243--256. https://www.usenix.org/conference/fast13/technical-sessions/presentation/zhao

Cited By

Stefanovici IAnderson PAranas EBlack RCabellero MCanakci BChatzieleftheriou AClegg JCletheroe DCooper BDeegan TDonnelly ADrevinskas RGomez Diaz AIlieva TJoyce RLegtchenko SMagalhaes BRowstron ASakakura MSchreiner NSweeney DWhittaker CWilliams HWinkler TWinzeck S(2024)Project Silica: sustainable cloud archival storage in glassFrontiers in Ultrafast Optics: Biomedical, Scientific, and Industrial Applications XXIV10.1117/12.3010515(39)Online publication date: 12-Mar-2024
https://doi.org/10.1117/12.3010515
Xu ZChen XLan XTang RJiang SShen C(2024)Empowering Data Owners: An Efficient and Verifiable Scheme for Secure Data DeletionComputers & Security10.1016/j.cose.2024.103978144(103978)Online publication date: Sep-2024
https://doi.org/10.1016/j.cose.2024.103978

Recommendations

Lonestar: An Energy-Aware Disk Based Long-Term Archival Storage System
ICPADS '11: Proceedings of the 2011 IEEE 17th International Conference on Parallel and Distributed Systems

We present the architecture for an disk based archival storage system and propose a new RAID scheme that is designed for "write once, read sometimes" workloads. By intertwining parity groups into a multi-dimensional RAID and improving the single disk ...
Read More
Robust benchmarking for archival storage tiers
PDSW '11: Proceedings of the sixth workshop on Parallel Data Storage

Until recently archival storage tiers have consisted of tape-based devices with a large storage capacity, but limited I/O performance for data retrieval. However, the growing capacity and shrinking cost of disk-based devices means that disk-based ...
Read More
Measurement for Improving the Design of Commodity Archival Storage Tiers
UCC '11: Proceedings of the 2011 Fourth IEEE International Conference on Utility and Cloud Computing

Archival data storage plays a critical role in data preservation as almost all current data will eventually be archived. In addition, the demands placed on archival storage tiers are growing because of large regularly-scheduled backups. Archival storage ...
Read More

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SOSP '23: Proceedings of the 29th Symposium on Operating Systems Principles

October 2023

802 pages

ISBN:9798400702297

DOI:10.1145/3600006

Conference Chairs:
Jason Flinn
Meta
,
Margo Seltzer
University of British Columbia
,
General Chairs:
Peter Druschel
Max Planck Institute for Software Systems (MPI-SWS)
,
Antoine Kaufmann
Max Planck Institute for Software Systems (MPI-SWS)
,
Jonathan Mace
Max Planck Institute for Software Systems (MPI-SWS) and Microsoft Research

Copyright © 2023 Owner/Author(s).

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs International 4.0 License.

Sponsors

SIGOPS: ACM Special Interest Group on Operating Systems

In-Cooperation

USENIX

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 October 2023

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SOSP '23

Sponsor:

SIGOPS

SOSP '23: 29th Symposium on Operating Systems Principles

October 23 - 26, 2023

Koblenz, Germany

Acceptance Rates

SOSP '23 Paper Acceptance Rate 43 of 232 submissions, 19%;

Overall Acceptance Rate 131 of 716 submissions, 18%

Upcoming Conference

SOSP '24

Sponsor:
sigops

ACM SIGOPS 30th Symposium on Operating Systems Principles

November 5 - 8, 2024

Austin , TX , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
3,758
Total Downloads

Downloads (Last 12 months)3,758
Downloads (Last 6 weeks)146

Other Metrics

View Author Metrics

Citations

Cited By

Stefanovici IAnderson PAranas EBlack RCabellero MCanakci BChatzieleftheriou AClegg JCletheroe DCooper BDeegan TDonnelly ADrevinskas RGomez Diaz AIlieva TJoyce RLegtchenko SMagalhaes BRowstron ASakakura MSchreiner NSweeney DWhittaker CWilliams HWinkler TWinzeck S(2024)Project Silica: sustainable cloud archival storage in glassFrontiers in Ultrafast Optics: Biomedical, Scientific, and Industrial Applications XXIV10.1117/12.3010515(39)Online publication date: 12-Mar-2024
https://doi.org/10.1117/12.3010515
Xu ZChen XLan XTang RJiang SShen C(2024)Empowering Data Owners: An Efficient and Verifiable Scheme for Secure Data DeletionComputers & Security10.1016/j.cose.2024.103978144(103978)Online publication date: Sep-2024
https://doi.org/10.1016/j.cose.2024.103978

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents