research-article

Open access

Replication, history, and grafting in the Ori file system

Authors:

Ali José Mashtizadeh,

Yifeng Frank Huang,

David MazièresAuthors Info & Claims

SOSP '13: Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles

Pages 151 - 166

https://doi.org/10.1145/2517349.2522721

Published: 03 November 2013 Publication History

Abstract

Ori is a file system that manages user data in a modern setting where users have multiple devices and wish to access files everywhere, synchronize data, recover from disk failure, access old versions, and share data. The key to satisfying these needs is keeping and replicating file system history across devices, which is now practical as storage space has outpaced both wide-area network (WAN) bandwidth and the size of managed data. Replication provides access to files from multiple devices. History provides synchronization and offline access. Replication and history together subsume backup by providing snapshots and avoiding any single point of failure. In fact, Ori is fully peer-to-peer, offering opportunistic synchronization between user devices in close proximity and ensuring that the file system is usable so long as a single replica remains. Cross-file system data sharing with history is provided by a new mechanism called grafting. An evaluation shows that as a local file system, Ori has low overhead compared to a File system in User Space (FUSE) loopback driver; as a network file system, Ori over a WAN outperforms NFS over a LAN.

Supplementary Material

MP4 File (d1-10-ali-mashtizadeh.mp4)

Download
1103.35 MB

References

[1]

Dropbox Inc. Dropbox - simplify your life. http://www.dropbox.com/, September 2012.

[2]

R. Rodrigues and C. Blake. When multi-hop peer-to-peer routing matters. In 3rd International Workshop on Peer-to-Peer Systems, La Jolla, CA, 2004.

Digital Library

[3]

Inc. Software Freedom Conservancy. Git. http://www.git-scm.com/, September 2012.

[4]

Yupu Zhang, Chris Dragga, Andrea Arpaci-Dusseau, and Remzi Arpaci-Dusseau. *-box: Towards reliability and consistency in dropbox-like file synchronization services. In Proceedings of the The Fifth Workshop on Hot Topics in Storage and File Systems, HotStorage '13, Berkeley, CA, USA, 2013. USENIX Association.

Digital Library

[5]

Sean Quinlan and Sean Dorward. Venti: A new approach to archival storage. In Proceedings of the Conference on File and Storage Technologies, FAST '02, pages 89--101, Berkeley, CA, USA, 2002. USENIX Association.

Digital Library

[6]

Mercurial scm. http://mercurial.selenic.com/, September 2012.

[7]

Athicha Muthitacharoen, Benjie Chen, and David Mazières. A low-bandwidth network file system. In Proceedings of the eighteenth ACM symposium on Operating systems principles, SOSP '01, pages 174--187, New York, NY, USA, 2001. ACM.

Digital Library

[8]

Filebench. http://sourceforge.net/apps/mediawiki/filebench/index.php, 2013.

[9]

Russell Coker. Bonnie++ now at 1.03e. http://www.coker.com.au/bonnie++/, September 2012.

[10]

Joey Hess. git-annex. http://git-annex.branchable.com/, September 2012.

[11]

B. Nowicki. NFS: Network File System Protocol specification. RFC 1094 (Informational), March 1989.

[12]

Microsoft Corporation. Common internet file system (cifs) protocol specification. http://msdn.microsoft.com/en-us/library/ee442092.aspx, September 2012.

[13]

Jose Barreto. Smb2, a complete redesign of the main remote file protocol for windows. http://blogs.technet.com/b/josebda/archive/2008/12/05/smb2-a-complete-redesign-of-the-main-remote-file-protocol-for-windows.aspx, December 2008.

[14]

Gursharan S. Sidhu, Richard F. Andrews, and Alan B. Oppenheimer. Inside AppleTalk. Addison-Wesley, Boston, 1990.

[15]

Mahadev Satyanarayanan. Scalable, secure, and highly available distributed file access. Computer, 23(5):9--18, 20--21, May 1990.

Digital Library

[16]

Mahadev Satyanarayanan, James J. Kistler, Puneet Kumar, Maria E. Okasaki, Ellen H. Siegel, and David C. Steere. Coda: A highly available file system for a distributed workstation environment. IEEE Trans. Comput., 39(4):447--459, April 1990.

Digital Library

[17]

P. J. Braam, M. Callahan, and P. Schwan. The inter-mezzo file system. The Perl Conference 3, August 1999.

[18]

Peter Reiher, John Heidemann, David Ratner, Greg Skinner, and Gerald Popek. Resolving file conflicts in the ficus file system. In Proceedings of the USENIX Summer 1994 Technical Conference on USENIX Summer 1994 Technical Conference - Volume 1, USTC'94, pages 12--12, Berkeley, CA, USA, 1994. USENIX Association.

Digital Library

[19]

Björn Grönvall, Ian Marsh, and Stephen Pink. A multicast-based distributed file system for the internet. In Proceedings of the 7th workshop on ACM SIGOPS European workshop: Systems support for worldwide applications, EW 7, pages 95--102, New York, NY, USA, 1996. ACM.

Digital Library

[20]

GlusterFS. http://www.gluster.org/.

[21]

Jacob Strauss, Justin Mazzola Paluska, Chris Lesniewski-Laas, Bryan Ford, Robert Morris, and Frans Kaashoek. Eyo: device-transparent personal storage. In Proceedings of the 2011 USENIX conference on USENIX annual technical conference, USENIXATC'11, pages 35--35, Berkeley, CA, USA, 2011. USENIX Association.

Digital Library

[22]

Venugopalan Ramasubramanian, Thomas L. Rodeheffer, Douglas B. Terry, Meg Walraed-Sullivan, Ted Wobber, Catherine C. Marshall, and Amin Vahdat. Cimbiosys: a platform for content-based partial replication. In Proceedings of the 6th USENIX symposium on Networked systems design and implementation, NSDI'09, pages 261--276, Berkeley, CA, USA, 2009. USENIX Association.

Digital Library

[23]

Daniel Peek and Jason Flinn. Ensemblue: integrating distributed storage and consumer electronics. In Proceedings of the 7th symposium on Operating systems design and implementation, OSDI '06, pages 219--232, Berkeley, CA, USA, 2006. USENIX Association.

Digital Library

[24]

Nalini Belaramani, Mike Dahlin, Lei Gao, Amol Nayate, Arun Venkataramani, Praveen Yalagandula, and Jiandan Zheng. Practi replication. In Proceedings of the 3rd conference on Networked Systems Design & Implementation - Volume 3, NSDI'06, pages 5--5, Berkeley, CA, USA, 2006. USENIX Association.

Digital Library

[25]

Dave Hitz, James Lau, and Michael Malcolm. File system design for an nfs file server appliance. In Proceedings of the USENIX Winter 1994 Technical Conference on USENIX Winter 1994 Technical Conference, WTEC'94, pages 19--19, Berkeley, CA, USA, 1994. USENIX Association.

Digital Library

[26]

J. Bonwick and B. Moore. Zfs: The last word in file systems. http://hub.opensolaris.org/bin/download/Community+Group+zfs/docs/zfslast.pdf, September 2012.

[27]

Douglas J. Santry, Michael J. Feeley, Norman C. Hutchinson, and Alistair C. Veitch. Elephant: The file system that never forgets. In Proceedings of the The Seventh Workshop on Hot Topics in Operating Systems, HOTOS '99, pages 2--, Washington, DC, USA, 1999. IEEE Computer Society.

Digital Library

[28]

S. Quinlan, J. McKie, and R. Cox. Fossil, an archival file server. http://www.cs.bell-labs.com/sys/doc/fossil.pdf, 2003.

[29]

Brian Cornell, Peter A. Dinda, and Fabián E. Bustamante. Wayback: a user-level versioning file system for linux. In Proceedings of the annual conference on USENIX Annual Technical Conference, ATEC '04, pages 27--27, Berkeley, CA, USA, 2004. USENIX Association.

Digital Library

[30]

Kiran-Kumar Muniswamy-Reddy, Charles P. Wright, Andrew Himmer, and Erez Zadok. A versatile and user-oriented versioning file system. In Proceedings of the 3rd USENIX Conference on File and Storage Technologies, FAST '04, pages 115--128, Berkeley, CA, USA, 2004. USENIX Association.

Digital Library

[31]

Wikipedia. Time machine (mac os). http://en.wikipedia.org/wiki/Time_Machine_(Mac_OS), March 2013.

[32]

Steve Sinofsky. Protecting user files with file history. http://blogs.msdn.com/b/b8/archive/2012/07/10/protecting-user-files-with-file-history.aspx, March 2013.

[33]

Siddhartha Annapureddy, Michael J. Freedman, and David Mazières. Shark: scaling file servers via cooperative caching. In Proceedings of the 2nd conference on Symposium on Networked Systems Design & Implementation - Volume 2, NSDI'05, pages 129--142, Berkeley, CA, USA, 2005. USENIX Association.

Digital Library

[34]

Kevin Fu, M. Frans Kaashoek, and David Mazières. Fast and secure distributed read-only file system. ACM Transactions on Computer Systems, 20(1):1--24, February 2002.

Digital Library

[35]

Jinyuan Li, Maxwell Krohn, David Mazières, and Dennis Shasha. Secure untrusted data repository (sundr). In Proceedings of the 6th conference on Symposium on Opearting Systems Design & Implementation - Volume 6, OSDI'04, pages 9--9, Berkeley, CA, USA, 2004. USENIX Association.

Digital Library

[36]

Ralph C. Merkle. A digital signature based on a conventional encryption function. In Carl Pomerance, editor, Advances in Cryptology---CRYPTO '87, volume 293 of Lecture Notes in Computer Science, pages 369--378, Berlin, 1987. Springer-Verlag.

Digital Library

Cited By

Nikolaidis SRefanidis I(2022)Consolidating incentivization in distributed neural network training via decentralized autonomous organizationNeural Computing and Applications10.1007/s00521-022-07374-3Online publication date: 22-May-2022
https://doi.org/10.1007/s00521-022-07374-3
Mostafaei HKumar DLospoto GChiesa MBattista G(2021)DeSI: A Decentralized Software-Defined Network Architecture for Internet Exchange PointsIEEE Transactions on Network Science and Engineering10.1109/TNSE.2021.30825758:3(2198-2212)Online publication date: 1-Jul-2021
https://doi.org/10.1109/TNSE.2021.3082575
Mendes ROliveira TCogo VNeves NBessani A(2021)Charon: A Secure Cloud-of-Clouds System for Storing and Sharing Big DataIEEE Transactions on Cloud Computing10.1109/TCC.2019.29168569:4(1349-1361)Online publication date: 1-Oct-2021
https://doi.org/10.1109/TCC.2019.2916856
Show More Cited By

Recommendations

Replication in the harp file system

This paper describes the design and implementation of the Harp file system. Harp is a replicated Unix file system accessible via the VFS interface. It provides highly available and reliable storage for files and guarantees that file operations are ...
File Migration and File Replication: A Symbiotic Relationship

Much of the past research on file migration and file replication has examined these two resource management strategies in isolation or in an environment where they do not work together. We establish through simulation that these two strategies can be ...
Replication in the harp file system
SOSP '91: Proceedings of the thirteenth ACM symposium on Operating systems principles

This paper describes the design and implementation of the Harp file system. Harp is a replicated Unix file system accessible via the VFS interface. It provides highly available and reliable storage for files and guarantees that file operations are ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SOSP '13: Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles

November 2013

498 pages

ISBN:9781450323888

DOI:10.1145/2517349

General Chair:
Michael Kaminsky
Intel Labs
,
Program Chair:
Mike Dahlin
Google and UT Austin

Copyright © 2013 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

SIGOPS: ACM Special Interest Group on Operating Systems

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 November 2013

Check for updates

Qualifiers

Research-article

Funding Sources

Conference

SOSP '13

Sponsor:

SIGOPS

SOSP '13: ACM SIGOPS 24th Symposium on Operating Systems Principles

November 3 - 6, 2013

Pennsylvania, Farminton

Acceptance Rates

Overall Acceptance Rate 131 of 716 submissions, 18%

Upcoming Conference

SOSP '24

Sponsor:
sigops

ACM SIGOPS 30th Symposium on Operating Systems Principles

November 4 - 6, 2024

Austin , TX , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

25
Total Citations
View Citations
11,527
Total Downloads

Downloads (Last 12 months)175
Downloads (Last 6 weeks)27

Reflects downloads up to 15 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Nikolaidis SRefanidis I(2022)Consolidating incentivization in distributed neural network training via decentralized autonomous organizationNeural Computing and Applications10.1007/s00521-022-07374-3Online publication date: 22-May-2022
https://doi.org/10.1007/s00521-022-07374-3
Mostafaei HKumar DLospoto GChiesa MBattista G(2021)DeSI: A Decentralized Software-Defined Network Architecture for Internet Exchange PointsIEEE Transactions on Network Science and Engineering10.1109/TNSE.2021.30825758:3(2198-2212)Online publication date: 1-Jul-2021
https://doi.org/10.1109/TNSE.2021.3082575
Mendes ROliveira TCogo VNeves NBessani A(2021)Charon: A Secure Cloud-of-Clouds System for Storing and Sharing Big DataIEEE Transactions on Cloud Computing10.1109/TCC.2019.29168569:4(1349-1361)Online publication date: 1-Oct-2021
https://doi.org/10.1109/TCC.2019.2916856
Nikolaidis SRefanidis I(2021)Using distributed ledger technology to democratize neural network trainingApplied Intelligence10.1007/s10489-021-02340-3Online publication date: 31-Mar-2021
https://doi.org/10.1007/s10489-021-02340-3
Konstantopoulos MChondros NRoussopoulos M(2020)A social link based private storage cloud2020 IEEE 26th International Conference on Parallel and Distributed Systems (ICPADS)10.1109/ICPADS51040.2020.00065(447-455)Online publication date: Dec-2020
https://doi.org/10.1109/ICPADS51040.2020.00065
Nikolaidis SRefanidis I(2020)Privacy preserving distributed training of neural networksNeural Computing and Applications10.1007/s00521-020-04880-0Online publication date: 11-May-2020
https://doi.org/10.1007/s00521-020-04880-0
Bozorgi AJadidi MAnderson J(2020)Challenges in Designing a Distributed Cryptographic File SystemSecurity Protocols XXVII10.1007/978-3-030-57043-9_17(177-192)Online publication date: 21-Aug-2020
https://doi.org/10.1007/978-3-030-57043-9_17
Bijlani ARamachandran UDan TDahlia M(2019)Extension framework for file systems in user spaceProceedings of the 2019 USENIX Conference on Usenix Annual Technical Conference10.5555/3358807.3358819(121-134)Online publication date: 10-Jul-2019
https://dl.acm.org/doi/10.5555/3358807.3358819
Noma TYamamoto K(2019)Chain of Equipments: A Scheme for Life-Time Personal Photo Storage2019 IEEE International Conference on Consumer Electronics (ICCE)10.1109/ICCE.2019.8662025(1-5)Online publication date: Jan-2019
https://doi.org/10.1109/ICCE.2019.8662025
Nikolaidis SRefanidis I(2019)Learnae: Distributed and Resilient Deep Neural Network Training for Heterogeneous Peer to Peer TopologiesPädiatrie10.1007/978-3-030-20257-6_24(286-298)Online publication date: 15-May-2019
https://doi.org/10.1007/978-3-030-20257-6_24
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents