Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3626183.3660269acmconferencesArticle/Chapter ViewAbstractPublication PagesspaaConference Proceedingsconference-collections
extended-abstract

Brief Announcement: Red-Blue Pebbling with Multiple Processors: Time, Communication and Memory Trade-offs

Published: 17 June 2024 Publication History

Abstract

The well-studied red-blue pebble game models the execution of an arbitrary computational DAG by a single processor over a two-level memory hierarchy. We present a natural generalization to a multiprocessor setting where each processor has its own limited fast memory, and all processors share unlimited slow memory. To our knowledge, this is the first thorough study that combines pebbling and DAG scheduling problems, capturing the computation of general workloads on multiple processors with memory constraints and communication costs. Our pebbling model enables us to analyze trade-offs between workload balancing, communication and memory limitations, and it captures real-world factors such as superlinear speedups due to parallelization. Our results include upper and lower bounds on the pebbling cost, an analysis of a greedy pebbling strategy, and an extension of NP-hardness results for specific DAG classes from simpler models. For our main technical contribution, we show two inapproximability results that already hold for the long-standing problem of standard red-blue pebbling: (i) the optimal I/O cost cannot be approximated to any finite factor, and (ii) the optimal total cost (I/O+computation) can only be approximated to a limited constant factor, i.e., it does not allow for a polynomial-time approximation scheme. These results also carry over naturally to our multiprocessor pebbling model.

References

[1]
Gianfranco Bilardi, Michele Scquizzato, and Francesco Silvestri. 2018. A lower bound technique for communication in BSP. ACM Transactions on Parallel Computing (TOPC), Vol. 4, 3 (2018), 1--27.
[2]
Guy E Blelloch, Rezaul A Chowdhury, Phillip B Gibbons, Vijaya Ramachandran, Shimin Chen, and Michael Kozuch. 2008. Provably good multicore cache performance for divide-and-conquer algorithms. In Proceedings of the 19th annual ACM-SIAM Symposium on Discrete Algorithms (SODA). 501--510.
[3]
Jeremiah Blocki, Ling Ren, and Samson Zhou. 2018. Bandwidth-hard functions: Reductions and lower bounds. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security. 1820--1836.
[4]
Timothy Carpenter, Fabrice Rastello, P Sadayappan, and Anastasios Sidiropoulos. 2016. Brief announcement: Approximating the I/O complexity of one-shot red-blue pebbling. In Proceedings of the 28th ACM Symposium on Parallelism in Algorithms and Architectures (SPAA). 161--163.
[5]
Rezaul Alam Chowdhury and Vijaya Ramachandran. 2008. Cache-efficient dynamic programming algorithms for multicores. In Proceedings of the 20th ACM Symposium on Parallelism in Algorithms and Architectures (SPAA). 207--216.
[6]
Erik D Demaine and Quanquan C Liu. 2018. Red-blue pebble game: Complexity of computing the trade-off between cache size and memory transfers. In Proceedings of the 30th ACM Symposium on Parallelism in Algorithms and Architectures (SPAA). 195--204.
[7]
Venmugil Elango, Fabrice Rastello, Louis-Noël Pouchet, Jagannathan Ramanujam, and Ponnuswamy Sadayappan. 2014. On characterizing the data movement complexity of computational DAGs for parallel execution. In Proceedings of the 26th ACM Symposium on Parallelism in Algorithms and Architectures (SPAA). 296--306.
[8]
Venmugil Elango, Fabrice Rastello, Louis-Noël Pouchet, Jagannathan Ramanujam, and Ponnuswamy Sadayappan. 2015. On characterizing the data access complexity of programs. In Proceedings of the 42nd ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL). 567--580.
[9]
Niels Gleinig and Torsten Hoefler. 2022. The red-blue pebble game on trees and dags with large input. In International Colloquium on Structural Information and Communication Complexity (SIROCCO). Springer, 135--153.
[10]
Jia-Wei Hong and Hsiang-Tsung Kung. 1981. I/O complexity: The red-blue pebble game. In Proceedings of the 13th ACM Symposium on Theory of Computing (STOC). 326--333.
[11]
Saachi Jain and Matei Zaharia. 2020. Spectral Lower Bounds on the I/O Complexity of Computation Graphs. In Proceedings of the 32nd ACM Symposium on Parallelism in Algorithms and Architectures (SPAA). 329--338.
[12]
Grzegorz Kwasniewski, Tal Ben-Nun, Lukas Gianinazzi, Alexandru Calotoiu, Timo Schneider, Alexandros Nikolaos Ziogas, Maciej Besta, and Torsten Hoefler. 2021. Pebbles, graphs, and a pinch of combinatorics: Towards tight I/O lower bounds for statically analyzable programs. In Proceedings of the 33rd ACM Symposium on Parallelism in Algorithms and Architectures (SPAA). 328--339.
[13]
Grzegorz Kwasniewski, Marko Kabić, Maciej Besta, Joost VandeVondele, Raffaele Solcà, and Torsten Hoefler. 2019. Red-blue pebbling revisited: near optimal parallel matrix-matrix multiplication. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis. 1--22.
[14]
WF McColl and A Tiskin. 1999. Memory-efficient matrix computations in the BSP model. Algorithmica, Vol. 24 (1999), 3--4.
[15]
Pál András Papp and Roger Wattenhofer. 2020. On the hardness of red-blue pebble games. In Proceedings of the 32nd ACM Symposium on Parallelism in Algorithms and Architectures (SPAA). 419--429.
[16]
Pál András Papp Papp, Georg Anegg, and A. N. Yzelman. 2023. DAG Scheduling in the BSP Model. arXiv preprint arXiv:2303.05989.
[17]
Desh Ranjan, John Savage, and Mohammad Zubair. 2012. Upper and lower I/O bounds for pebbling r-pyramids. Journal of Discrete Algorithms, Vol. 14 (2012), 2--12.
[18]
John E Savage. 1995. Extending the Hong-Kung model to memory hierarchies. In International Computing and Combinatorics Conference. Springer, 270--281.
[19]
John E Savage and Mohammad Zubair. 2008. A unified model for multicore architectures. In Proceedings of the 1st international forum on next-generation multicore/manycore technologies. 1--12.
[20]
Jacob Scott, Olga Holtz, and Oded Schwartz. 2015. Matrix multiplication I/O-complexity by path routing. In Proceedings of the 27th ACM symposium on Parallelism in Algorithms and Architectures (SPAA). 35--45.
[21]
Ravi Sethi. 1973. Complete register allocation problems. In Proceedings of the 5th ACM Symposium on Theory of Computing. 182--195.
[22]
Edgar Solomonik, Erin Carson, Nicholas Knight, and James Demmel. 2017. Trade-offs between synchronization, communication, and computation in parallel linear algebra computations. ACM Transactions on Parallel Computing, Vol. 3, 1 (2017), 1--47.
[23]
Alexandre Tiskin. 1998. The bulk-synchronous parallel random access machine. Theoretical Computer Science, Vol. 196, 1--2 (1998), 109--130.
[24]
Leslie G Valiant. 1990. A bridging model for parallel computation. Commun. ACM, Vol. 33, 8 (1990), 103--111.
[25]
Leslie G Valiant. 2011. A bridging model for multi-core computing. J. Comput. System Sci., Vol. 77, 1 (2011), 154--166.

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
SPAA '24: Proceedings of the 36th ACM Symposium on Parallelism in Algorithms and Architectures
June 2024
510 pages
ISBN:9798400704161
DOI:10.1145/3626183
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 June 2024

Check for updates

Author Tags

  1. approximation
  2. communication costs
  3. limited memory
  4. parallel computing
  5. red-blue pebble game
  6. scheduling

Qualifiers

  • Extended-abstract

Conference

SPAA '24
Sponsor:

Acceptance Rates

Overall Acceptance Rate 447 of 1,461 submissions, 31%

Upcoming Conference

SPAA '25
37th ACM Symposium on Parallelism in Algorithms and Architectures
July 28 - August 1, 2025
Portland , OR , USA

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 47
    Total Downloads
  • Downloads (Last 12 months)47
  • Downloads (Last 6 weeks)9
Reflects downloads up to 15 Jan 2025

Other Metrics

Citations

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media