SPIRIT: a runtime system for distributed irregular tree applications
Article No.: 51, Pages 1 - 2
Abstract
Repeated, depth-first traversal of trees is a common algorithmic pattern in an important set of applications from diverse domains such as cosmological simulations, data mining, and computer graphics. As these applications operate over massive data sets, it is often necessary to distribute the trees to process all of the data.
In this work, we introduce SPIRIT, a runtime system to ease the writing of distributed tree applications. SPIRIT automates the challenging tasks of tree distribution, optimizing communication and parallelizing independent computations. The common algorithmic pattern in tree traversals is exploited to effectively schedule parallel computations and improve locality. As a result, pipeline parallelism in distributed traversals is identified, which is complemented by load-balancing, and locality-enhancing, message aggregation optimizations. Evaluation of SPIRIT on tree traversal in Point Correlation (PC) shows a scalable system, achieving speedups upto 38x on a 16-node, 64 process system compared to a 1-node, baseline configuration. We also find that SPIRIT results in substantially less communication and achieves significant performance improvements over implementations in other distributed graph systems.
References
[1]
J. Chhugani, C. Kim, H. Shukla, J. Park, P. Dubey, J. Shalf, and H. D. Simon. Billion-particle simd-friendly two-point correlation on large-scale hpc cluster systems. In Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, page 1, 2012.
[2]
D. Gregor and A. Lumsdaine. The parallel bgl: A generic library for distributed graph computations. Parallel Object-Oriented Scientific Computing (POOSC), 2:1--18, 2005.
[3]
Y. Jo and M. Kulkarni. Automatically enhancing locality for tree traversals with traversal splicing. In Proceedings of the ACM international conference on Object oriented programming systems languages and applications, OOPSLA '12, pages 355--374, New York, NY, USA, 2012.
[4]
M. Kulkarni. The Galois System: optimistic parallelization of irregular programs. PhD thesis, Ithaca, NY, USA, 2008. Adviser-Pingali, Keshav.
[5]
Y. Low, D. Bickson, J. Gonzalez, C. Guestrin, A. Kyrola, and J. M. Hellerstein. Distributed graphlab: A framework for machine learning and data mining in the cloud. Proc. VLDB Endow., 5(8):716--727, 2012.
[6]
G. Malewicz, M. H. Austern, A. J. Bik, J. C. Dehnert, I. Horn, N. Leiser, and G. Czajkowski. Pregel: A system for large-scale graph processing. In Proceedings of the 2010 ACM SIGMOD International Conference on Management of Data, pages 135--146, New York, NY, USA, 2010.
[7]
J. H. Reif. Depth-first search is inherently sequential. Information Processing Letters, 20(5):229--234, 1985.
Index Terms
- SPIRIT: a runtime system for distributed irregular tree applications
Recommendations
SPIRIT: a runtime system for distributed irregular tree applications
PPoPP '16: Proceedings of the 21st ACM SIGPLAN Symposium on Principles and Practice of Parallel ProgrammingRepeated, depth-first traversal of trees is a common algorithmic pattern in an important set of applications from diverse domains such as cosmological simulations, data mining, and computer graphics. As these applications operate over massive data sets, ...
Automatically enhancing locality for tree traversals with traversal splicing
OOPSLA '12Generally applicable techniques for improving temporal locality in irregular programs, which operate over pointer-based data structures such as trees and graphs, are scarce. Focusing on a subset of irregular programs, namely, tree traversal algorithms ...
Automatically enhancing locality for tree traversals with traversal splicing
OOPSLA '12: Proceedings of the ACM international conference on Object oriented programming systems languages and applicationsGenerally applicable techniques for improving temporal locality in irregular programs, which operate over pointer-based data structures such as trees and graphs, are scarce. Focusing on a subset of irregular programs, namely, tree traversal algorithms ...
Comments
Information & Contributors
Information
Published In
August 2016
405 pages
February 2016420 pages
Copyright © 2016 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 27 February 2016
Published in SIGPLAN Volume 51, Issue 8
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 170Total Downloads
- Downloads (Last 12 months)5
- Downloads (Last 6 weeks)0
Reflects downloads up to 15 Oct 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in