research-article

Public Access

On Characterizing the Data Access Complexity of Programs

Authors:

Venmugil Elango,

Fabrice Rastello,

Louis-Noël Pouchet,

P. SadayappanAuthors Info & Claims

POPL '15: Proceedings of the 42nd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages

Pages 567 - 580

https://doi.org/10.1145/2676726.2677010

Published: 14 January 2015 Publication History

Abstract

Technology trends will cause data movement to account for the majority of energy expenditure and execution time on emerging computers. Therefore, computational complexity will no longer be a sufficient metric for comparing algorithms, and a fundamental characterization of data access complexity will be increasingly important. The problem of developing lower bounds for data access complexity has been modeled using the formalism of Hong and Kung's red/blue pebble game for computational directed acyclic graphs (CDAGs). However, previously developed approaches to lower bounds analysis for the red/blue pebble game are very limited in effectiveness when applied to CDAGs of real programs, with computations comprised of multiple sub-computations with differing DAG structure. We address this problem by developing an approach for effectively composing lower bounds based on graph decomposition. We also develop a static analysis algorithm to derive the asymptotic data-access lower bounds of programs, as a function of the problem size and cache size.

Supplementary Material

MPG File (p567-sidebyside.mpg)

Download
527.41 MB

References

[1]

G. Ballard, J. Demmel, O. Holtz, and O. Schwartz. Minimizing communication in numerical linear algebra. SIAM J. Matrix Analysis Applications, 32(3):866--901, 2011.

[2]

G. Ballard, J. Demmel, O. Holtz, B. Lipshitz, and O. Schwartz. Brief announcement: Strong scaling of matrix multiplication algorithms and memory-independent communication lower bounds. In Proc. SPAA'12, pages 77--79, 2012.

Digital Library

[3]

G. Ballard, J. Demmel, O. Holtz, and O. Schwartz. Graph expansion and communication costs of fast matrix multiplication. J. ACM, 59(6): 32, 2012.

Digital Library

[4]

A. Barvinok. Computing the Ehrhart polynomial of a convex l attice polytope. Discrete and Computational Geometry, 12:35--48, 1994.

Digital Library

[5]

J. Bennett, A. Carbery, M. Christ, and T. Tao. Finite bounds for Holder-Brascamp-Lieb multilinear inequalities. Mathematical Research Letters, 55(4):647--666, 2010.

[6]

K. Bergman, S. Borkar, et al. Exascale computing study: Technology challenges in achieving exascale systems. DARPA IPTO, Tech. Rep, 2008.

[7]

G. Bilardi and E. Peserico. A characterization of temporal locality and its portability across memory hierarchies. Automata, Languages and Programming, pages 128--139, 2001.

Digital Library

[8]

G. Bilardi and F. P. Preparata. Processor - Time Tradeoffs under Bounded-Speed Message Propagation: Part II, Lower Bounds. Theory Comput. Syst., 32(5):531--559, 1999.

[9]

G. Bilardi, A. Pietracaprina, and P. D'Alberto. On the space and access complexity of computation DAGs. In Graph-Theoretic Concepts in Computer Science, volume 1928 of LNCS, pages 81--92. 2000.

Digital Library

[10]

G. Bilardi, M. Scquizzato, and F. Silvestri. A lower bound technique for communication on bsp with application to the fft. In Euro-Par, pages 676--687, 2012.

Digital Library

[11]

M. Christ, J. Demmel, N. Knight, T. Scanlon, and K. Yelick. Communication Lower Bounds and Optimal Algorithms for Programs That Reference Arrays Part 1. EECS Technical Report EECS'2013--6 1, UC Berkeley, May 2013.

[12]

S. A. Cook. An observation on time-storage trade off. J. Comput. Syst. Sci., 9(3):308--316, 1974.

Digital Library

[13]

J. Demmel, L. Grigori, M. Hoemmen, and J. Langou. Communication-optimal parallel and sequential QR and LU factorizations. SIAM J. Scientific Computing, 34(1), 2012.

Digital Library

[14]

V. Elango, F. Rastello, L.-N. Pouchet, J. Ramanujam, and P. Sadayappan. Data access complexity: The red/blue pebble game revisited. Technical report, OSU/INRIA/LSU/UCLA, Sept. 2013. OSU-CI SRC-7/13-TR16.

[15]

V. Elango, F. Rastello, L. Pouchet, J. Ramanujam, and P. Sadayappan. On characterizing the data movement complexity of computational dags for parallel execution. In 26th ACM Symposium on Parallelism in Algorithms and Architectures, SPAA '14, pages 296--306, 2014.

Digital Library

[16]

P. Feautrier. Parametric integer programming. RAIRO Recherche Operationnelle, 22(3):243--268, 1988.

[17]

S. H. Fuller and L. I. Millett. The Future of Computing Performance: Game Over or Next Level? The National Academies Press, 2011.

Digital Library

[18]

S. Girbal, N. Vasilache, C. Bastoul, A. Cohen, D. Parello, M. Sigler, and O. Temam. Semi-automatic composition of loop transformations for deep parallelism and memory hierarchies. Intl. J. of Parallel Programming, 34(3), 2006.

Digital Library

[19]

G. Gupta and S. Rajopadhye. The Z-polyhedral model. In ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (PPoPP'07), pages 237--248. ACM, 2007.

Digital Library

[20]

J.-W. Hong and H. T. Kung. I/O complexity: The red-blue pebble game. In Proc. of the 13th annual ACM sympo. on Theory of computing (STOC'81), pages 326--333. ACM, 1981.

Digital Library

[21]

D. Irony, S. Toledo, and A. Tiskin. Communication lower bounds for distributed-memory matrix multiplication. J. Parallel Distrib. Comput., 64(9):1017--1026, 2004.

Digital Library

[22]

A. Ketterlin and P. Clauss. Profiling data-dependence to assist parallelization: Framework, scope, and optimization. In MICRO, pages 437--448, 2012.

Digital Library

[23]

L. Loomis and H. Whitney. An inequality related to the isoperimetric inequality. Bull. Am. Math. Soc., 55:961--962, 1949.

[24]

D. Ranjan and M. Zubair. Vertex isoperimetric parameter of a computation graph. Int. J. Found. Comput. Sci., 23(4):941--964, 2012.

[25]

D. Ranjan, J. Savage, and M. Zubair. Strong I/O lower bounds for binomial and FFT computation graphs. In Computing and Combinatorics, volume 6842 of LNCS, pages 134--145. Springer, 2011.

Digital Library

[26]

D. Ranjan, J. E. Savage, and M. Zubair. Upper and lower I/O bounds for pebbling r-pyramids. J. Discrete Algorithms, 14:2--12, 2012.

Digital Library

[27]

J. Savage. Extending the Hong-Kung model to memory hierarchies. In Computing and Combinatorics, volume 959 of LNCS, pages 270--281. 1995.

Digital Library

[28]

J. E. Savage. Models of Computation. Addison-Wesley, 1998.

[29]

J. E. Savage and M. Zubair. A unified model for multicore arc hi-tectures. In Proceedings of the 1st International Forum on Next-generation Multicore/Manycore Technologies, page 9. ACM, 2008.

Digital Library

[30]

J. E. Savage and M. Zubair. Cache-optimal algorithms for option pricing. ACM Trans. Math. Softw., 37(1), 2010.

Digital Library

[31]

M. Scquizzato and F. Silvestri. Communication lower bounds for distributed-memory computations. CoRR, abs/1307.1805, 2013.

[32]

J. Shalf, S. Dosanjh, and J. Morrison. Exascale computing technology challenges. High Performance Computing for Computational Science-VECPAR 2010, pages 1--25, 2011.

Digital Library

[33]

E. Solomonik, A. Buluc, and J. Demmel. Minimizing communication in all-pairs shortest paths. In IPDPS, 2013.

Digital Library

[34]

S. I. Valdimarsson. The Brascamp-Lieb polyhedron. Can. J. Math., 62(4):870--888, 2010.

[35]

L. G. Valiant. A bridging model for multi-core computing. J. Comput. Syst. Sci., 77:154--166, Jan. 2011.

Digital Library

[36]

S. Verdoolaege. isl: An integer set library for the poly hedral model. In Mathematical Software-ICMS 2010, pages 299--302. Springer, 2010.

Digital Library

Cited By

Liu FZhu YSun SDing CSmith WHosseini K(2024)Parallel Loop Locality Analysis for Symbolic Thread CountsProceedings of the 2024 International Conference on Parallel Architectures and Compilation Techniques10.1145/3656019.3676948(219-232)Online publication date: 14-Oct-2024
https://dl.acm.org/doi/10.1145/3656019.3676948
Pouchet LTucker EZhang NChen HPal DRodríguez GZhang ZZhang ZPutnam A(2024)Formal Verification of Source-to-Source Transformations for HLSProceedings of the 2024 ACM/SIGDA International Symposium on Field Programmable Gate Arrays10.1145/3626202.3637563(97-107)Online publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1145/3626202.3637563
Böhnlein TPapp PYzelman AAgrawal KPetrank E(2024)Brief Announcement: Red-Blue Pebbling with Multiple Processors: Time, Communication and Memory Trade-offsProceedings of the 36th ACM Symposium on Parallelism in Algorithms and Architectures10.1145/3626183.3660269(285-287)Online publication date: 17-Jun-2024
https://dl.acm.org/doi/10.1145/3626183.3660269
Show More Cited By

Index Terms

On Characterizing the Data Access Complexity of Programs
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Empirical software validation
  2. Software organization and properties
    1. Software functional properties
      1. Formal methods
2. Theory of computation
  1. Design and analysis of algorithms

Recommendations

Pebbles, Graphs, and a Pinch of Combinatorics: Towards Tight I/O Lower Bounds for Statically Analyzable Programs
SPAA '21: Proceedings of the 33rd ACM Symposium on Parallelism in Algorithms and Architectures

Determining I/O lower bounds is a crucial step in obtaining communication-efficient parallel algorithms, both across the memory hierarchy and between processors. Current approaches either study specific algorithms individually, disallow programmatic ...
Automated derivation of parametric data movement lower bounds for affine programs
PLDI 2020: Proceedings of the 41st ACM SIGPLAN Conference on Programming Language Design and Implementation

Researchers and practitioners have for long worked on improving the computational complexity of algorithms, focusing on reducing the number of operations needed to perform a computation. However the hardware trend nowadays clearly shows a higher ...
Red-Blue Pebble Game: Complexity of Computing the Trade-Off between Cache Size and Memory Transfers
SPAA '18: Proceedings of the 30th on Symposium on Parallelism in Algorithms and Architectures

The red-blue pebble game was formulated in the 1980s~\citeHK81 to model the I/O complexity of algorithms on a two-level memory hierarchy. Given a directed acyclic graph representing computations (vertices) and their dependencies (edges), the red-blue ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

POPL '15: Proceedings of the 42nd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages

January 2015

716 pages

ISBN:9781450333009

DOI:10.1145/2676726

General Chair:
Sriram Rajamani
Microsoft Research, India
,
Program Chair:
David Walker
Princeton University, USA

ACM SIGPLAN Notices Volume 50, Issue 1
POPL '15
January 2015
682 pages
ISSN:0362-1340
EISSN:1558-1160
DOI:10.1145/2775051
Editor:
Andy Gill
University of Kansas, Lawrence, KS
Issue’s Table of Contents

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGPLAN: ACM Special Interest Group on Programming Languages

In-Cooperation

SIGACT: ACM Special Interest Group on Algorithms and Computation Theory

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 January 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

POPL '15

Sponsor:

SIGPLAN

POPL '15: The 42nd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages

January 15 - 17, 2015

Mumbai, India

Acceptance Rates

POPL '15 Paper Acceptance Rate 52 of 227 submissions, 23%;

Overall Acceptance Rate 824 of 4,130 submissions, 20%

Upcoming Conference

POPL '25

Sponsor:
sigplan

The 52nd Annual ACM SIGPLAN Symposium on Principles of Programming Languages

January 19 - 25, 2025

Denver , CO , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

14
Total Citations
View Citations
918
Total Downloads

Downloads (Last 12 months)156
Downloads (Last 6 weeks)14

Reflects downloads up to 15 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Liu FZhu YSun SDing CSmith WHosseini K(2024)Parallel Loop Locality Analysis for Symbolic Thread CountsProceedings of the 2024 International Conference on Parallel Architectures and Compilation Techniques10.1145/3656019.3676948(219-232)Online publication date: 14-Oct-2024
https://dl.acm.org/doi/10.1145/3656019.3676948
Pouchet LTucker EZhang NChen HPal DRodríguez GZhang ZZhang ZPutnam A(2024)Formal Verification of Source-to-Source Transformations for HLSProceedings of the 2024 ACM/SIGDA International Symposium on Field Programmable Gate Arrays10.1145/3626202.3637563(97-107)Online publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1145/3626202.3637563
Böhnlein TPapp PYzelman AAgrawal KPetrank E(2024)Brief Announcement: Red-Blue Pebbling with Multiple Processors: Time, Communication and Memory Trade-offsProceedings of the 36th ACM Symposium on Parallelism in Algorithms and Architectures10.1145/3626183.3660269(285-287)Online publication date: 17-Jun-2024
https://dl.acm.org/doi/10.1145/3626183.3660269
Eyraud-Dubois LIooss GLangou JRastello FAgrawal KPetrank E(2024)Tightening I/O Lower Bounds through the Hourglass Dependency PatternProceedings of the 36th ACM Symposium on Parallelism in Algorithms and Architectures10.1145/3626183.3659986(183-193)Online publication date: 17-Jun-2024
https://dl.acm.org/doi/10.1145/3626183.3659986
Ding CChen DLiu FReber BSmith W(2022)CARL: Compiler Assigned Reference LeasingACM Transactions on Architecture and Code Optimization10.1145/349873019:1(1-28)Online publication date: 17-Mar-2022
https://dl.acm.org/doi/10.1145/3498730
Olivry AIooss GTollenaere NRountev ASadayappan PRastello FFreund SYahav E(2021)IOOpt: automatic derivation of I/O complexity bounds for affine programsProceedings of the 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation10.1145/3453483.3454103(1187-1202)Online publication date: 19-Jun-2021
https://dl.acm.org/doi/10.1145/3453483.3454103
Kwasniewski GBen-Nun TGianinazzi LCalotoiu ASchneider TZiogas ABesta MHoefler TAgrawal KAzar Y(2021)Pebbles, Graphs, and a Pinch of CombinatoricsProceedings of the 33rd ACM Symposium on Parallelism in Algorithms and Architectures10.1145/3409964.3461796(328-339)Online publication date: 6-Jul-2021
https://dl.acm.org/doi/10.1145/3409964.3461796
Sun ZPedretti GMannocci PAmbrosi EBricalli AIelmini D(2020)Time Complexity of In-Memory Solution of Linear SystemsIEEE Transactions on Electron Devices10.1109/TED.2020.299243567:7(2945-2951)Online publication date: Jul-2020
https://doi.org/10.1109/TED.2020.2992435
Yuan LDing CSmith WDenning PZhang Y(2019)A Relational Theory of LocalityACM Transactions on Architecture and Code Optimization10.1145/334110916:3(1-26)Online publication date: 20-Aug-2019
https://dl.acm.org/doi/10.1145/3341109
Demaine ELiu QScheideler CFineman J(2018)Red-Blue Pebble GameProceedings of the 30th on Symposium on Parallelism in Algorithms and Architectures10.1145/3210377.3210387(195-204)Online publication date: 11-Jul-2018
https://dl.acm.org/doi/10.1145/3210377.3210387
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents