article

Free access

Latency lags bandwith

Author:

David A. PattersonAuthors Info & Claims

Communications of the ACM, Volume 47, Issue 10

Pages 71 - 75

https://doi.org/10.1145/1022594.1022596

Published: 01 October 2004 Publication History

All formats PDF

Abstract

As I review performance trends, I am struck by a consistent theme across many technologies: bandwidth improves much more quickly than latency. Here, I list a half-dozen performance milestones to document this observation, many reasons why it happens, a few ways to cope with it, a rule of thumb to quantify it, plus an example of how to design systems differently based on this observation.

References

[1]

Gries, M. A survey of synchronous RAM architectures. Computer Engineering and Networks Laboratory (TIK). Zurich, Germany, (Apr. 1999).

Google Scholar

[2]

Grochowski, E. and Halem, R. Technological impact of magnetic hard disk drives on storage systems. IBM Systems J. 42, 2 (July 2003), 338-346.

Digital Library

Google Scholar

[3]

Hennessy, J. and Patterson, D. Computer Architecture: A Quantitative Approach. Morgan Kauffman, San Francisco, CA, 1990, 1996, 2003. (Most of the historical data in <zref=T1>Table 1<zrefx> comes from the three editions of this book.)

Digital Library

Google Scholar

[4]

IC Knowledge. History of the Integrated Circuit; www.icknowledge.com/history/history.html (2003)

Google Scholar

[5]

Patterson, D. and Hennessy, J. Computer Organization and Design: The Hardware/Software Interface. Morgan Kauffman San Francisco, CA, 1994, 1998, 2004. (Some historical data in <zref=T1>Table 1<zrefx> comes from the three editions of this book.)

Digital Library

Google Scholar

[6]

Ross, P. 5 commandments of engineering. IEEE Spectrum (Dec. 2003).

Google Scholar

Cited By

View all

Li YWei FZhang CZhang HSalakhutdinov RKolter ZHeller KWeller AOliver NScarlett JBerkenkamp F(2024)EAGLEProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3693232(28935-28948)Online publication date: 21-Jul-2024
https://dl.acm.org/doi/10.5555/3692070.3693232
Kim SHooper CGholami ADong ZLi XShen SMahoney MKeutzer KSalakhutdinov RKolter ZHeller KWeller AOliver NScarlett JBerkenkamp F(2024)SqueezeLLMProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3693028(23901-23923)Online publication date: 21-Jul-2024
https://dl.acm.org/doi/10.5555/3692070.3693028
Butler BYu SMazaheri AJannesari A(2024)PipeInfer: Accelerating LLM Inference using Asynchronous Pipelined SpeculationProceedings of the International Conference for High Performance Computing, Networking, Storage, and Analysis10.1109/SC41406.2024.00046(1-19)Online publication date: 17-Nov-2024
https://dl.acm.org/doi/10.1109/SC41406.2024.00046
Show More Cited By

Index Terms

Latency lags bandwith

Recommendations

Latency Lags Bandwidth
ICCD '05: Proceedings of the 2005 International Conference on Computer Design

As I review performance trends, I am struck by a consistent theme across many technologies over many years: bandwidth improves much more quickly than latency for four different technologies: disks, networks, memories and processors. A rule of thumb to ...
Reducing web latency: the virtue of gentle aggression

To serve users quickly, Web service providers build infrastructure closer to clients and use multi-stage transport connections. Although these changes reduce client-perceived round-trip times, TCP's current mechanisms fundamentally limit latency ...
Reducing web latency: the virtue of gentle aggression
SIGCOMM '13: Proceedings of the ACM SIGCOMM 2013 conference on SIGCOMM

To serve users quickly, Web service providers build infrastructure closer to clients and use multi-stage transport connections. Although these changes reduce client-perceived round-trip times, TCP's current mechanisms fundamentally limit latency ...

Comments

Information & Contributors

Information

Published In

Communications of the ACM Volume 47, Issue 10

Voting systems

October 2004

95 pages

ISSN:0001-0782

EISSN:1557-7317

DOI:10.1145/1022594

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 October 2004

Published in CACM Volume 47, Issue 10

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

164
Total Citations
View Citations
10,087
Total Downloads

Downloads (Last 12 months)2,039
Downloads (Last 6 weeks)276

Reflects downloads up to 16 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Li YWei FZhang CZhang HSalakhutdinov RKolter ZHeller KWeller AOliver NScarlett JBerkenkamp F(2024)EAGLEProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3693232(28935-28948)Online publication date: 21-Jul-2024
https://dl.acm.org/doi/10.5555/3692070.3693232
Kim SHooper CGholami ADong ZLi XShen SMahoney MKeutzer KSalakhutdinov RKolter ZHeller KWeller AOliver NScarlett JBerkenkamp F(2024)SqueezeLLMProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3693028(23901-23923)Online publication date: 21-Jul-2024
https://dl.acm.org/doi/10.5555/3692070.3693028
Butler BYu SMazaheri AJannesari A(2024)PipeInfer: Accelerating LLM Inference using Asynchronous Pipelined SpeculationProceedings of the International Conference for High Performance Computing, Networking, Storage, and Analysis10.1109/SC41406.2024.00046(1-19)Online publication date: 17-Nov-2024
https://dl.acm.org/doi/10.1109/SC41406.2024.00046
Gholami AYao ZKim SHooper CMahoney MKeutzer K(2024)AI and Memory WallIEEE Micro10.1109/MM.2024.337376344:3(33-39)Online publication date: 1-May-2024
https://dl.acm.org/doi/10.1109/MM.2024.3373763
Ding NMaris PNam HGroves TAwan MLindsey LDaley CSelvitopi OOliker LWright NWilliams S(2024)Evaluating the potential of disaggregated memory systems for HPC applicationsConcurrency and Computation: Practice and Experience10.1002/cpe.814736:19Online publication date: 31-May-2024
https://doi.org/10.1002/cpe.8147
Girase HAgarwal NChoi CMangalam K(2023)Latency Matters: Real-Time Action Forecasting Transformer2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52729.2023.01799(18759-18769)Online publication date: Jun-2023
https://doi.org/10.1109/CVPR52729.2023.01799
Fariborz MYoo S(2022)High Throughput Memory with Silicon Photonics in Chiplet-based Architectures for Irregular Workloads2022 27th OptoElectronics and Communications Conference (OECC) and 2022 International Conference on Photonics in Switching and Computing (PSC)10.23919/OECC/PSC53152.2022.9849864(1-3)Online publication date: 3-Jul-2022
https://doi.org/10.23919/OECC/PSC53152.2022.9849864
Scheuner JDeng RSteghöfer JLeitner P(2022)CrossFit: Fine-grained Benchmarking of Serverless Application Performance across Cloud Providers2022 IEEE/ACM 15th International Conference on Utility and Cloud Computing (UCC)10.1109/UCC56403.2022.00016(51-60)Online publication date: Dec-2022
https://doi.org/10.1109/UCC56403.2022.00016
Ding NWilliams SNam HGroves TAwan MLindsey LDaley CSelvitopi OOliker LWright N(2022)Methodology for Evaluating the Potential of Disaggregated Memory Systems2022 IEEE/ACM International Workshop on Resource Disaggregation in High-Performance Computing (REDIS)10.1109/RESDIS56595.2022.00006(1-11)Online publication date: Nov-2022
https://doi.org/10.1109/RESDIS56595.2022.00006
Beck MMoore T(2022)Is Universal Broadband Service Impossible?2022 IEEE 19th International Conference on Mobile Ad Hoc and Smart Systems (MASS)10.1109/MASS56207.2022.00064(403-409)Online publication date: Oct-2022
https://doi.org/10.1109/MASS56207.2022.00064
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Magazine Site

View this article on the magazine site (external)

Magazine Site

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Abstract

References

Cited By

Index Terms

Recommendations

Latency Lags Bandwidth

Reducing web latency: the virtue of gentle aggression

Reducing web latency: the virtue of gentle aggression

Comments

Information

Published In

Publisher

Publication History

Permissions

Check for updates

Qualifiers

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

PDF

eReader

Magazine Site

Login options

Full Access

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations