research-article

StackSync: bringing elasticity to dropbox-like file synchronization

Authors:

Pedro Garcia Lopez,

Marc Sanchez-Artigas,

Cristian Cotes,

John LentonAuthors Info & Claims

Middleware '14: Proceedings of the 15th International Middleware Conference

Pages 49 - 60

https://doi.org/10.1145/2663165.2663332

Published: 08 December 2014 Publication History

Abstract

The design of elastic file synchronization services like Dropbox is an open and complex issue yet not unveiled by the major commercial providers, as it includes challenges like fine-grained programmable elasticity and efficient change notification to millions of devices. In this paper, we propose a novel architecture for file synchronization which aims to solve the above two major challenges. At the heart of our proposal lies ObjectMQ, a lightweight framework for providing programmatic elasticity to distributed objects using messaging. The efficient use of indirect communication: i) enables programmatic elasticity based on queue message processing, ii) simplifies change notifications offering simple unicast and multicast primitives; and iii) provides transparent load balancing based on queues.

Our reference implementation is StackSync, an open source elastic file synchronization Cloud service developed in the context of the FP7 project CloudSpaces. StackSync supports both predictive and reactive provisioning policies on top of ObjectMQ that adapt to real traces from the Ubuntu One service. The feasibility of our approach has been extensively validated with an open benchmark, including commercial synchronization services like Dropbox or OneDrive.

References

[1]

I. Drago, M. Mellia, M. Munafo, A. Sperotto, R. Sadre, and A. Pras, "Inside dropbox: Understanding personal cloud storage services," in Proc. of ACM Internet Measurement Conference (IMC), 2012, pp. 481--494.

Digital Library

[2]

"How we have scaled dropbox," https://www.youtube.com/watch?v=PE4gwstWhmc.

[3]

K. Jayaram, "Elastic remote methods," in ACM/IFIP/USENIX Middleware 2013, 2013, pp. 143--162.

[4]

I. Drago, E. Bocchi, M. Mellia, H. Slatman, and A. Pras, "Benchmarking personal cloud storage," in Proc. of ACM Internet Measurement Conference (IMC), 2013.

Digital Library

[5]

R. Dievendorff, P. J. Helland, G. Chopra, and M. Al-Ghosein, "Queued method invocations on distributed component applications," Jul. 23 2002, US Patent 6,425,017.

[6]

A. Lima, W. Cirne, F. Brasileiro, and D. Fireman, "A case for event-driven distributed objects," in On the Move to Meaningful Internet Systems 2006: CoopIS, DOA, GADA, and ODBASE, 2006, pp. 1705--1721.

Digital Library

[7]

"Mongrel: message-queue-based-load-balancing," http://zef.me/4502/message-queue-based-load-balancingMongrel2.

[8]

P. Eugster, "Type-based publish/subscribe: Concepts and experiences," ACM Transactions on Programming Languages and Systems (TOPLAS), vol. 29, no. 1, p. 6, 2007.

Digital Library

[9]

P. T. Eugster, R. Guerraoui, and J. Sventek, "Distributed asynchronous collections: Abstractions for publish/subscribe interaction," in European Conference on Object-Oriented Programming (ECOOP), 2000, pp. 252--276.

Digital Library

[10]

C. Pairot, P. García, and A. F. G. Skarmeta, "Dermi: a decentralized peer-to-peer event-based object middleware," in Proc. of IEEE International Conference on Distributed Computing Systems (ICDCS), 2004, pp. 236--243.

Digital Library

[11]

D. A. Menasce, "Mom vs. rpc: Communication models for distributed applications," IEEE Internet Computing, vol. 9, no. 2, pp. 90--93, 2005.

Digital Library

[12]

OASIS, "Amqp: Advanced message queueing protocol," http://www.amqp.org/.

[13]

S. C. Kendall, J. Waldo, A. Wollrath, and G. Wyant, "A note on distributed computing," Mountain View, CA, USA, Tech. Rep., 1994.

Digital Library

[14]

"Kryo: Fast, efficient java serialization and cloning," http://code.google.com/p/kryo/.

[15]

R. Gracia-Tinedo, M. Sánchez-Artigas, A. Moreno-Martínez, C. Cotes-González, and P. García-López, "Actively Measuring Personal Cloud Storage," in Proc. of IEEE CLOUD'13, 2013, pp. 301--308.

Digital Library

[16]

H. F. G. Y. Songbin Liu, Xiaomeng Huang, "Understanding data characteristics and access patterns in a cloud storage system," in Proc. of IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing (CCGRID), 2013, pp. 327--334.

[17]

P. Heckel, "Syncany open source file synchronization," http://www.syncany.org/.

[18]

"Objectmq mom-rpc middleware," https://github.com/cloudspaces/objectmq.

[19]

D. Harnik, B. Pinkas, and A. Shulman-Peleg, "Side channels in cloud services: Deduplication in cloud storage," IEEE Security & Privacy, vol. 8, no. 6, pp. 40--47, 2010.

Digital Library

[20]

A. Muthitacharoen, B. Chen, and D. Mazières, "A low-bandwidth network file system," ACM SIGOPS Operating Systems Review (OSR), vol. 35, no. 5, pp. 174--187, 2001.

Digital Library

[21]

K. Eshghi and H. K. Tang, "A Framework for Analyzing and Improving Content-Based Chunking Algorithms," http://www.hpl.hp.com/techreports/2005/HPL-2005-30R1.pdf, 2005.

[22]

B. Urgaonkar, P. Shenoy, A. Chandra, P. Goyal, and T. Wood, "Agile dynamic provisioning of multi-tier internet applications," ACM Transactions on Autonomous and Adaptive Systems (TAAS), vol. 3, no. 1, pp. 1:1--1:39, 2008.

Digital Library

[23]

V. Tarasov, A. Mudrankit, W. Buik, P. Shilane, G. Kuenning, and E. Zadok, "Generating realistic datasets for deduplication analysis," in Proc. of USENIX ATC, 2012, pp. 24--24.

Digital Library

Cited By

Lim M(2023)An Active File Mode Transition Mechanism Based on Directory Activation Ratio in File Synchronization ServiceApplied Sciences10.3390/app1310597013:10(5970)Online publication date: 12-May-2023
https://doi.org/10.3390/app13105970
Henziger ECarlsson NJohnson KSpillner JKlusáček DAnjum A(2019)The Overhead of Confidentiality and Client-side Encryption in Cloud Storage SystemsProceedings of the 12th IEEE/ACM International Conference on Utility and Cloud Computing10.1145/3344341.3368808(209-217)Online publication date: 2-Dec-2019
https://dl.acm.org/doi/10.1145/3344341.3368808
Henziger ECarlsson N(2019)Delta Encoding Overhead Analysis of Cloud Storage Systems Using Client-Side Encryption2019 IEEE International Conference on Cloud Computing Technology and Science (CloudCom)10.1109/CloudCom.2019.00036(183-190)Online publication date: Dec-2019
https://doi.org/10.1109/CloudCom.2019.00036
Show More Cited By

Index Terms

StackSync: bringing elasticity to dropbox-like file synchronization
1. Information systems
  1. Information retrieval
    1. Search engine architectures and scalability
      1. Distributed retrieval
      2. Peer-to-peer retrieval
  2. Information storage systems
    1. Storage architectures
      1. Distributed storage

Recommendations

Physics and microeconomics-based metrics for evaluating cloud computing elasticity

Currently, many customers and broadband providers are using cloud resources, such as processing and storage, for their applications and services. With the increase of computational resources usage, elasticity has become quite attractive and a key ...
Automatic elasticity in OpenStack
SDMCMM '12: Proceedings of the Workshop on Secure and Dependable Middleware for Cloud Monitoring and Management

Cloud computing infrastructures are the most recent approach to the development and conception of computational systems. Cloud infrastructures are complex environments with various subsystems, each one with their own challenges. Cloud systems should be ...
A Framework and Middleware for Application-Level Cloud Bursting on Top of Infrastructure-as-a-Service Clouds
UCC '13: Proceedings of the 2013 IEEE/ACM 6th International Conference on Utility and Cloud Computing

A core idea of cloud computing is elasticity, i.e., enabling applications to adapt to varying load by dynamically acquiring and releasing cloud resources. One concrete realization is cloud bursting, which is the migration of applications or parts of ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

Middleware '14: Proceedings of the 15th International Middleware Conference

December 2014

334 pages

ISBN:9781450327855

DOI:10.1145/2663165

General Chair:
Laurent Réveillère
LaBRI, University of Bordeaux, France
,
Program Chairs:
Lucy Cherkasova
HP Labs, USA
,
François Taïani
Université de Rennes 1 / IRISA, France

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Orange
Conseil Régional d'Aquitaine
LaBRI: LaBRI
Raytheon BBN Technologies: Raytheon BBN Technologies
ACM: Association for Computing Machinery
Red Hat JBoss Middleware: Red Hat JBoss Middleware
Bordeaux: City of Bordeaux
USENIX Assoc: USENIX Assoc
GDR ASR: GDR Architecture, Systèmes et Réseaux
IBM: IBM
HP: HP
IFIP

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 December 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Spanish research projects DELFIN (TIN-2010-20140-C03-03)
Seventh Framework Programme
Cloud Services and Community Clouds (TIN2013-47245-C2-2-R) funded by the Ministry of Science and Innovation

Conference

Middleware '14

Sponsor:

LaBRI
Raytheon BBN Technologies
ACM
Red Hat JBoss Middleware
Bordeaux
USENIX Assoc
GDR ASR
IBM
HP

Middleware '14: 15th International Middleware Conference

December 8 - 12, 2014

Bordeaux, France

Acceptance Rates

Middleware '14 Paper Acceptance Rate 27 of 144 submissions, 19%;

Overall Acceptance Rate 203 of 948 submissions, 21%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

18
Total Citations
View Citations
405
Total Downloads

Downloads (Last 12 months)8
Downloads (Last 6 weeks)0

Reflects downloads up to 26 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Lim M(2023)An Active File Mode Transition Mechanism Based on Directory Activation Ratio in File Synchronization ServiceApplied Sciences10.3390/app1310597013:10(5970)Online publication date: 12-May-2023
https://doi.org/10.3390/app13105970
Henziger ECarlsson NJohnson KSpillner JKlusáček DAnjum A(2019)The Overhead of Confidentiality and Client-side Encryption in Cloud Storage SystemsProceedings of the 12th IEEE/ACM International Conference on Utility and Cloud Computing10.1145/3344341.3368808(209-217)Online publication date: 2-Dec-2019
https://dl.acm.org/doi/10.1145/3344341.3368808
Henziger ECarlsson N(2019)Delta Encoding Overhead Analysis of Cloud Storage Systems Using Client-Side Encryption2019 IEEE International Conference on Cloud Computing Technology and Science (CloudCom)10.1109/CloudCom.2019.00036(183-190)Online publication date: Dec-2019
https://doi.org/10.1109/CloudCom.2019.00036
Gracia-Tinedo RZou CSanchez-Artigas MGarcia-Lopez P(2018)BenchBox: A User-Driven Benchmarking Framework for Fat-Client Storage SystemsIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2018.281965729:10(2191-2205)Online publication date: 1-Oct-2018
https://doi.org/10.1109/TPDS.2018.2819657
Akter MGani ARahman MHassan MAlmogren AAhmad S(2018)Performance Analysis of Personal Cloud Storage Services for Mobile Multimedia Health Record ManagementIEEE Access10.1109/ACCESS.2018.28698486(52625-52638)Online publication date: 2018
https://doi.org/10.1109/ACCESS.2018.2869848
Gracia-Tinedo RCotes CZamora-Gómez EOrtiz GMoreno-Martínez ASánchez-Artigas MGarcía-López PSánchez RGómez AIllana A(2018)Giving wings to your data: A first experience of Personal Cloud interoperabilityFuture Generation Computer Systems10.1016/j.future.2017.01.02778(1055-1070)Online publication date: Jan-2018
https://doi.org/10.1016/j.future.2017.01.027
Hao PBai YZhang XZhang YZhang JChiang MMaggs B(2017)EdgecourierProceedings of the Second ACM/IEEE Symposium on Edge Computing10.1145/3132211.3134447(1-14)Online publication date: 12-Oct-2017
https://dl.acm.org/doi/10.1145/3132211.3134447
Viotti PDobre DVukolić M(2017)HybrisACM Transactions on Storage10.1145/311989613:3(1-32)Online publication date: 28-Sep-2017
https://dl.acm.org/doi/10.1145/3119896
Saiz-Laudo RSanchez-Artigas MGarcia-Lopez P(2017)RSD: Rate-Based Sync Deferment for Personal Cloud Storage ServicesIEEE Communications Letters10.1109/LCOMM.2017.273184821:11(2384-2387)Online publication date: Nov-2017
https://doi.org/10.1109/LCOMM.2017.2731848
Sutra PRiviere ECotes CArtigas MLopez PBernard EBurns WZamarreno G(2017)CRESON: Callable and Replicated Shared Objects over NoSQL2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS)10.1109/ICDCS.2017.239(115-128)Online publication date: Jun-2017
https://doi.org/10.1109/ICDCS.2017.239
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten