research-article

Public Access

Efficient, Consistent Distributed Computation with Predictive Treaties

Authors:

Johannes Gehrke, and

Andrew C. MyersAuthors Info & Claims

EuroSys '19: Proceedings of the Fourteenth EuroSys Conference 2019

March 2019

Article No.: 36, Pages 1 - 16

https://doi.org/10.1145/3302424.3303987

Published: 25 March 2019 Publication History

Abstract

To achieve good performance, modern applications often partition their state across multiple geographically distributed nodes. While this approach reduces latency in the common case, it can be challenging for programmers to use correctly, especially in applications that require strong consistency. We introduce predictive treaties, a mechanism that can significantly reduce distributed coordination without losing strong consistency. The central insight behind our approach is that many computations can be expressed in terms of predicates over distributed state that can be partitioned and enforced locally. Predictive treaties improve on previous work by allowing the locally enforced predicates to depend on time. Intuitively, by predicting the evolution of system state, coordination can be significantly reduced compared to static approaches. We implemented predictive treaties in a distributed system that exposes them in an intuitive programming model. We evaluate performance on several benchmarks, including TPC-C, showing that predictive treaties can significantly increase performance by orders of magnitude and can even outperform customized algorithms.

References

[1]

Acar, U. A., Ahmed, A., And Blume, M. Imperative self-adjusting computation. In 35th ACM Symp. on Principles of Programming Languages (POPL) (2008), pp. 309--322.

Digital Library

[2]

Adya, A., Gruber, R., Liskov, B., And Maheshwari, U. Efficient optimistic concurrency control using loosely synchronized clocks. In ACM SIGMOD International Conference on Management of Data (SIGMOD) (San Jose, CA, May 1995), pp. 23--34.

Digital Library

[3]

Adya, A., And Liskov, B. Lazy consistency using loosely synchronized clocks. In 16th ACM Symp. on Principles of Distributed Computing (Aug. 1997), PODC '97, pp. 73--82.

Digital Library

[4]

Aho, A. V., Lam, M. S., Sethi, R., And Ullman, J. D. Compilers: Principles, Techniques, and Tools (2nd Edition). Addison-Wesley Longman Publishing Co., Inc., 2006.

Digital Library

[5]

Arlitt, M., And Jin, T. A Workload Characterization Study Of The 1998 World Cup Web Site. Ieee Network 14, 3 (2000), 30--37.

Digital Library

[6]

Babcock, B., And Olston, C. Distributed top-k monitoring. In Proceedings of the 2003 ACM SIGMOD International Conference on Management of Data (New York, NY, USA, 2003), SIGMOD '03, ACM, pp. 28--39.

Digital Library

[7]

Bailis, P., Fekete, A., Franklin, M. J., Ghodsi, A., Hellerstein, J. M., And StoicA, I. Coordination avoidance in database systems. PVLDB 8 (2014), 185--196.

Digital Library

[8]

Balegas, V., Duarte, S., Ferreira, C., Rodrigues, R., PreguiÇA, N., Najafzadeh, M., And Shapiro, M. Putting consistency back into eventual consistency. In Proceedings of the Tenth European Conference on Computer Systems (New York, NY, USA, 2015), EuroSys '15, ACM, pp. 6:1--6:16.

Digital Library

[9]

Balegas, V., Serra, D., Duarte, S., Ferreira, C., Shapiro, M., Rodrigues, R., And PreguiÇA, N. Extending eventually consistent cloud databases for enforcing numeric invariants. In IEEE Symp. on Reliable Distributed Systems (SRDS) (Sept. 2015).

Digital Library

[10]

BarbarÁ-MillÁ, D., And Garcia-Molina, H. The demarcation protocol: A technique for maintaining constraints in distributed database systems. The VLDB Journal 3, 3 (July 1994), 325--353.

Digital Library

[11]

Bernstein, P. A., Hadzilacos, V., And Goodman, N. Concurrency Control and Recovery in Database Systems. Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA, 1987.

Digital Library

[12]

Bhatotia, P., Wieder, A., Rodrigues, R., Acar, U. A., And Pasquini, R. Incoop: MapReduce for incremental computations. In ACM Symp. Cloud Computing (Oct. 2011).

Digital Library

[13]

Brown, R. G. Exponential smoothing for predicting demand. Operations Research 5, 1 (1957), 145--145.

Digital Library

[14]

Chong, C.-Y., And Kumar, S. P. Sensor networks: Evolution, opportunities, and challenges. Proceedings of the IEEE 91, 8 (2003), 1247--1256.

[15]

Corbett, J. C., Dean, J., Epstein, M., Fikes, A., Frost, C., Furman, J. J., Ghemawat, S., Gubarev, A., Heiser, C., Hochschild, P., et al. Spanner: Google's globally distributed database. ACM Transactions on Computer Systems (TOCS) 31, 3 (2013), 8.

Digital Library

[16]

Cormode, G. The continuous distributed monitoring model. ACM SIGMOD Record 42, 1 (May 2013), 5--14.

Digital Library

[17]

Durrett, R. Probability: Theory and Examples, 4th ed. Cambridge University Press, 2010.

Digital Library

[18]

Ebbinghaus, H., Flum, J., And Thomas, W. Mathematical Logic. Undergraduate Texts in Mathematics. Springer New York, 1996.

[19]

Eswaran, K. P., Gray, J. N., Lorie, R. A., And Traiger, I. L. The notions of consistency and predicate locks in a database system. Comm. of the ACM 19, 11 (Nov. 1976), 624--633. Also published as IBM RJ1487, December 1974.

Digital Library

[20]

Garofalakis, M., Keren, D., And Samoladas, V. Sketch-based geometric monitoring of distributed stream queries. Proceedings of the VLDB Endowment 6, 10 (Aug. 2013), 937--948.

Digital Library

[21]

Geng, Y., Liu, S., Yin, Z., Naik, A., Prabhakar, B., Rosenblum, M., And Vahdat, A. Exploiting a natural network effect for scalable, fine-grained clock synchronization. In 15th USENIX Symp. on Networked Systems Design and Implementation (NSDI) (Apr. 2018), pp. 81--94.

Digital Library

[22]

Giatrakos, N., Deligiannakis, A., Garofalakis, M., Sharfman, I., And Schuster, A. Prediction-based geometric monitoring over distributed data streams. In Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data (New York, NY, USA, 2012), SIGMOD '12, ACM, pp. 265--276.

Digital Library

[23]

Gray, J., Lorie, R., Putzolu, G., And Traiger, I. Granularity of locks and degrees of consistency in a shared database. In Modeling in Data Base Management Systems. Amsterdam: Elsevier North-Holland, 1976. Also available in Chapter 3 of Readings in Database Systems, Second Edition, M. Stonebraker Editor, Morgan Kaufmann, 1994.

Digital Library

[24]

Gupta, A., Mumick, I. S., And Subrahmanian, V. S. Maintaining views incrementally. In ACM SIGMOD International Conference on Management of Data (SIGMOD) (1993), pp. 157--166.

Digital Library

[25]

Herlihy, M., And Wing, J. Linearizability: A correctness condition for concurrent objects. Technical Report CMU-CS-88-120, Carnegie Mellon University, Pittsburgh, Pa., 1988.

[26]

Holt, C. C. Forecasting trends and seasonals by exponentially weighted averages. carnegie institute of technology. Tech. rep., Pittsburgh ONR memorandum, 1957.

[27]

Keren, D., Sharfman, I., Schuster, A., And Livne, A. Shape sensitive geometric monitoring. IEEE Transactions on Knowledge and Data Engineering 24, 8 (Aug. 2012), 1520--1535.

Digital Library

[28]

Kraska, T., Pang, G., Franklin, M. J., Madden, S., And Fekete, A. Mdcc: Multi-data center consistency. In ACM SIGOPS/EuroSys European Conference on Computer Systems (Apr. 2013).

Digital Library

[29]

Lee, K. S., Wang, H., Shrivastav, V., And Weatherspoon, H. Globally synchronized time via datacenter networks. In SIGCOMM (2016), pp. 454--467.

Digital Library

[30]

Liskov, B. Practical uses of synchronized clocks in distributed systems. In 10th ACM Symp. on Principles of Distributed Computing (Aug. 1991), PODC '91, pp. 1--9.

Digital Library

[31]

Liskov, B., Shrira, L., And Wroclawski, J. Efficient at-most-once messages based on synchronized clocks. ACM Trans. on Computer Systems 9, 2 (May 1991), 125--142.

Digital Library

[32]

Liu, J., George, M. D., Vikram, K., Qi, X., Waye, L., And Myers, A. C. Fabric: A platform for secure distributed computation and storage. In 22nd ACM Symp. on Operating System Principles (SOSP) (Oct. 2009), pp. 321--334.

Digital Library

[33]

Liu, J., Magrino, T., Arden, O., George, M. D., And Myers, A. C. Warranties for faster strong consistency. In 11th USENIX Symp. on Networked Systems Design and Implementation (NSDI) (Apr. 2014), pp. 513--517.

Digital Library

[34]

Lloyd, W., Freedman, M. J., Kaminsky, M., And Andersen, D. G. Don't settle for eventual: scalable causal consistency for wide-area storage with COPS. In 23rd ACM Symp. on Operating System Principles (SOSP) (2011).

Digital Library

[35]

Marzullo, K. Loosely-Coupled Distributed Services: A Distributed Time Service. PhD thesis, Stanford University, Stanford, Ca., 1983.

[36]

Menth, M., And Hauser, F. On moving averages, histograms and time-dependentrates for online measurement. In Proceedings of the 8th ACM/SPEC on International Conference on Performance Engineering (New York, NY, USA, 2017), ICPE '17, ACM, pp. 103--114.

Digital Library

[37]

Mills, D. L. Network time protocol (version 3) specification, implementation and analysis. Network Working Report RFC 1305, Mar. 1992.

Digital Library

[38]

O'Neil, P. The escrow transactional method. ACM Trans. on Database Systems 11, 4 (Dec. 1986), 405--430.

Digital Library

[39]

Pantel, L., And Wolf, L. C. On the suitability of dead reckoning schemes for games. In Proceedings of the 1st Workshop on Network and System Support for Games (New York, NY, USA, 2002), NetGames '02, ACM, pp. 79--84.

Digital Library

[40]

Papadimitriou, C. H. The serializability of concurrent database updates. Journal of the ACM 26, 4 (Oct. 1979), 631--653.

Digital Library

[41]

Piantoni, R., And Stancescu, C. Implementing the Swiss exchange trading system. In Fault-Tolerant Computing, 1997. FTCS-27. Digest of Papers., Twenty-Seventh Annual International Symposium on (1997), IEEE, pp. 309--313.

Digital Library

[42]

Ports, D. R. K., Clements, A. T., Zhang, I., Madden, S., And Liskov, B. Transactional consistency and automatic management in an application data cache. In 9th USENIX Symp. on Operating Systems Design and Implementation (OSDI) (2010).

Digital Library

[43]

PreguiçA, N., Martins, J. L., Cunha, M., And Domingos, H. Reservations for conflict avoidance in a mobile database system. In Proceedings of the 1st International Conference on Mobile Systems, Applications and Services (New York, NY, USA, 2003), MobiSys '03, ACM, pp. 43--56.

Digital Library

[44]

Ramakrishnan, R., And Gehrke, J. Database Management Systems, 3 ed. McGraw-Hill, Inc., New York, NY, USA, 2003.

Digital Library

[45]

Roy, S., Kot, L., Bender, G., Ding, B., Hojjat, H., Koch, C., Foster, N., And Gehrke, J. The homeostasis protocol: Avoiding transaction coordination through program analysis. In ACM SIGMOD International Conference on Management of Data (SIGMOD) (2015).

Digital Library

[46]

Shamis, A., Renzelmann, M., Novakovic, S., Chatzopouios, G., Dragojevic, A., Narayanan, D., And Castro, M. Fast general distributed transactions with opacity. In ACM SIGMOD International Conference on Management of Data (SIGMOD) (June 2019).

Digital Library

[47]

Shapiro, M., PreguiçA, N., Baquero, C., And Zawirski, M. Conflict-free replicated data types. In Proceedings of the 13th International Conference on Stabilization, Safety, and Security of Distributed Systems (Berlin, Heidelberg, 2011), SSS'11, Springer-Verlag, pp. 386--400.

Digital Library

[48]

Shapiro, M., PreguiçA, N. M., Baquero, C., And Zawirski, M. Convergent and commutative replicated data types. Bulletin of the EATCS 104 (2011), 67--88.

[49]

Sharfman, I., Schuster, A., And Keren, D. A Geometric Approach To Monitoring Threshold Functions Over Distributed Data Streams. Acm Transactions on Database Systems 32, 4 (Nov. 2007).

Digital Library

[50]

Singhal, S. K., And Cheriton, D. R. Using a position history-based protocol for distributed object visualization. Technical Report CS-TR-94-1505, Stanford University, Department of Computer Science, Stanford, CA, USA, Feb. 1994.

Digital Library

[51]

Taylor, S. J., Saville, J., And Sudra, R. Developing interest management techniques in distributed interactive simulation using java. In 1999 Winter Simulation Conference Proceedings (Dec. 1999), vol. 1, IEEE, pp. 518--523.

Digital Library

[52]

Terry, D. B., Prabhakaran, V., Kotla, R., Balakrishnan, M., Aguilera, M. K., And Abu-Libdeh, H. Consistency-based service level agreements for cloud storage. In 24th ACM Symp. on Operating System Principles (SOSP) (2013).

Digital Library

[53]

Vogels, W. Eventually consistent. Commun. ACM 52, 1 (Jan. 2009), 40--44.

Digital Library

[54]

Yu, H., And Vahdat, A. Design and evaluation of a continuous consistency model for replicated services. In 4th USENIX Symp. on Operating Systems Design and Implementation (OSDI) (2000).

Digital Library

[55]

Zaharia, M., Chowdhury, M., Das, T., Dave, A., Ma, J., McCauley, M., Franklin, M. J., Shenker, S., And Stoica, I. Resilient distributed datasets: A fault-tolerant abstraction for in-memory cluster computing. In 9th USENIX Symp. on Networked Systems Design and Implementation (NSDI) (2012).

Digital Library

Cited By

Chen YLuo LGuo DRottenstreich OWu J(2023)SDTP: Accelerating Wide-Area Data Analytics With Simultaneous Data Transfer and ProcessingIEEE Transactions on Cloud Computing10.1109/TCC.2021.311999111:1(911-926)Online publication date: 1-Jan-2023
https://doi.org/10.1109/TCC.2021.3119991
Donkervliet JCuijpers JIosup A(2021)Dyconits: Scaling Minecraft-like Services through Dynamically Managed Inconsistency2021 IEEE 41st International Conference on Distributed Computing Systems (ICDCS)10.1109/ICDCS51616.2021.00021(126-137)Online publication date: Jul-2021
https://doi.org/10.1109/ICDCS51616.2021.00021
Donkervliet JTrivedi AIosup APhanishayee AStutsman R(2020)Towards supporting millions of users in modifiable virtual environments by redesigning minecraft-like games as serverless systemsProceedings of the 12th USENIX Conference on Hot Topics in Cloud Computing10.5555/3485849.3485851(2-2)Online publication date: 13-Jul-2020
https://dl.acm.org/doi/10.5555/3485849.3485851
Show More Cited By

Recommendations

Finding Consistent Global Checkpoints in a Distributed Computation

Consistent global checkpoints have many uses in distributed computations. A central question in applications that use consistent global checkpoints is to determine whether a consistent global checkpoint that includes a given set of local checkpoints can ...
Read More
Implementing and programming weakly consistent memories
Read More
Flexible Integration of Eventually Consistent Distributed Storage with Strongly Consistent Databases
NCCA '12: Proceedings of the 2012 Second Symposium on Network Cloud Computing and Applications

In order to design distributed business applications or services, the common practice consists in setting up a multi-tier architecture on top of a relational database. Due to the recent evolution of the needs in terms of scalability and availability in ...
Read More

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

EuroSys '19: Proceedings of the Fourteenth EuroSys Conference 2019

March 2019

714 pages

ISBN:9781450362818

DOI:10.1145/3302424

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGOPS: ACM Special Interest Group on Operating Systems

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 March 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Conference

EuroSys '19

Sponsor:

SIGOPS

EuroSys '19: Fourteenth EuroSys Conference 2019

March 25 - 28, 2019

Dresden, Germany

Acceptance Rates

Overall Acceptance Rate 241 of 1,308 submissions, 18%

Upcoming Conference

EuroSys '25

Sponsor:
sigops

Twentieth European Conference on Computer Systems

March 30 - April 3, 2025

Rotterdam , Netherlands

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
569
Total Downloads

Downloads (Last 12 months)47
Downloads (Last 6 weeks)5

Other Metrics

View Author Metrics

Citations

Cited By

Chen YLuo LGuo DRottenstreich OWu J(2023)SDTP: Accelerating Wide-Area Data Analytics With Simultaneous Data Transfer and ProcessingIEEE Transactions on Cloud Computing10.1109/TCC.2021.311999111:1(911-926)Online publication date: 1-Jan-2023
https://doi.org/10.1109/TCC.2021.3119991
Donkervliet JCuijpers JIosup A(2021)Dyconits: Scaling Minecraft-like Services through Dynamically Managed Inconsistency2021 IEEE 41st International Conference on Distributed Computing Systems (ICDCS)10.1109/ICDCS51616.2021.00021(126-137)Online publication date: Jul-2021
https://doi.org/10.1109/ICDCS51616.2021.00021
Donkervliet JTrivedi AIosup APhanishayee AStutsman R(2020)Towards supporting millions of users in modifiable virtual environments by redesigning minecraft-like games as serverless systemsProceedings of the 12th USENIX Conference on Hot Topics in Cloud Computing10.5555/3485849.3485851(2-2)Online publication date: 13-Jul-2020
https://dl.acm.org/doi/10.5555/3485849.3485851
Köhler MEskandani NWeisenburger PMargara ASalvaneschi G(2020)Rethinking safe consistency in distributed object-oriented programmingProceedings of the ACM on Programming Languages10.1145/34282564:OOPSLA(1-30)Online publication date: 13-Nov-2020
https://dl.acm.org/doi/10.1145/3428256

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents