research-article

Comparing high-performance multi-core web-server architectures

Authors:

Ashif S. Harji,

Tim BrechtAuthors Info & Claims

SYSTOR '12: Proceedings of the 5th Annual International Systems and Storage Conference

Article No.: 1, Pages 1 - 12

https://doi.org/10.1145/2367589.2367591

Published: 04 June 2012 Publication History

Abstract

In this paper, we study how web-server architecture and implementation affect performance when trying to obtain high throughput on a 4-core system servicing static content. We focus on static content as a growing numbers of servers are dedicated to workloads comprised of songs, photos, software, and videos chunked for HTTP downloads. Two representative static-content workloads are used: one serviced entirely from the file-system cache and the other requires significant disk I/O. We focus on 4-core systems as: 1) it is a widely used configurations in data-centers and cloud services, 2) recent studies show large SMP systems may operate more efficiently when subdivided into smaller subsystems, 3) understanding performance with a smaller number of cores is essential before scaling to a larger number of cores, 4) and 4-cores may be sufficient for many web servers.

Two high-performance web-servers, with event-driven (μserver) and pipelined (WatPipe) architectures, are developed and tested for a multi-core environment. By carefully implementing and tuning the two web-servers, both achieve performance comparable to running independent copies of the server on each processor (N-copy). The new web-servers achieve high throughput (4,000--6,000 Mbps) with 40,000 to 70,000 connects/second; performance in all cases is better than nginx, lighttpd, and Apache. We conclude that implementation and tuning of web servers is perhaps more important than server architecture. We also find it is better to use blocking rather than non-blocking calls to sendfile, when the requested files do not all fit in the file-system cache.

References

[1]

V. Anand and B. Hartner. TCPIP network stack performance in Linux kernel 2.4 and 2.5. In Proc. of the 4th Ottawa Linux Symp., June 2002.

[2]

Apache Software Foundation. The Apache web server. http://httpd.apache.org.

[3]

G. Banga and P. Druschel. Measuring the capacity of a web server. In Proc. of the USENIX Symp. on Internet Technologies and Systems. USENIX Association, 1997.

Digital Library

[4]

P. Barford and M. Crovella. Generating representative web workloads for network and server performance evaluation. In Proc. of ACM SIGMETRICS 1998, Madison, Wis., 1998.

Digital Library

[5]

D. Beaver, S. Kumar, H. Li, J. Sobel, and P. Vajgel. Finding a needle in Haystack: Facebook's photo storage. In Operating System Design and Implementation, 2010.

Digital Library

[6]

A. C. Begen, T. Akgul, and M. Baugher. Watching video over the web: Part 1: Streaming protocols. IEEE Internet Computing, 15(2): 54--63, 2011. ISSN 1089-7801.

Digital Library

[7]

S. Boyd-Wickizer, A. T. Clements, Y. Mao, A. Pesterev, M. F. Kaashoek, R. Morris, and N. Zeldovich. An analysis of linux scalability to many cores. In Proc. of the 9th USENIX Symp. on Operating Sys. Design and Impl., pages 1--16, Oct. 2010.

Digital Library

[8]

T. Brecht, G. J. Janakiraman, B. Lynn, V. Saletore, and Y. Turner. Evaluating network processing efficiency with processor partitioning and asynchronous I/O. In EuroSys'06, pages 265--278, April 2006.

Digital Library

[9]

B. Burns, K. Grimaldi, A. Kostadinov, E. D. Berger, and M. D. Corner. Flux: A language for programming high-performance servers. In Proc. of the USENIX Annual Tech. Conf., pages 129--142, 2006.

Digital Library

[10]

G. S. Choi, J.-H. Kim, D. Ersoz, and C. R. Das. A multi-threaded pipelined web server architecture for SMP/SoC machines. In Proc. of the 14th international conf. on World Wide Web, pages 730--739, 2005.

Digital Library

[11]

M. Dobrescu, N. Egi, K. Argyraki, B. Chun, K. Fall, G. Iannaccone, A. Knies, M. Manesh, and S. Ratnasamy. Route-bricks: exploiting parallelism to scale software routers. In Proc. of the 22nd ACM Symp. on Operating Systems Principles, pages 15--28, 2010.

Digital Library

[12]

Facebook. Open compute project. http://opencompute.org/specs.

[13]

A. Foong, J. Fung, and D. Newell. An in-depth analysis of the impact of processor affinity on network performance. In Proc 12th IEEE International Conference on Networks, volume 1, pages 244--250, Nov. 2004.

[14]

A. S. Harji. Performance Comparison of Uniprocessor and Multiprocessor Web Server Architectures. PhD thesis, University of Waterloo, 2010. http://uwspace.uwaterloo.ca/-bitstream/10012/5040/1/Harji_thesis.pdf

[15]

A. S. Harji, P. A. Buhr, and T. Brecht. Our troubles with linux and why you should care. In 2nd ACM SIGOPS Asia-Pacific Workshop on Systems (APSys 2011), July 2011.

Digital Library

[16]

P. Joubert, R. King, R. Neves, M. Russinovich, and J. Tracey. High-performance memory-based Web servers: Kernel and user-space performance. In Proc. of the USENIX Annual Tech. Conf., pages 175--188, 2001.

Digital Library

[17]

Jupiter Research. Retail website performance: Consumer reaction to a poor online shopping experience. http://www.akamai.com/4seconds, 2006.

[18]

M. Krohn. Building secure high-performance web services with OKWS. In Proc. of the USENIX Annual Tech. Conf. USENIX, 2004.

Digital Library

[19]

Microsoft. Microsoft reveals its specialty servers, racks. http://www.datacenterknowledge.com/archives/2011/04/25/-microsoft-reveals-its-specialty-servers-racks.

[20]

D. Mosberger and T. Jin. httperf tool for measuring web server performance. ACM SIGMETRICS, 26(3): 31--37, 1998.

Digital Library

[21]

Netcraft. Oct. 2011 Web Server Survey, 2011. http://news.-netcraft.com/archives/2011/10/06/october-2011-web-server-survey.html.

[22]

J. Nielsen. Designing Web Usability. New Riders, 2000.

[23]

V. S. Pai, P. Druschel, and W. Zwaenepoel. Flash: An efficient and portable Web server. In Proc. of the USENIX Annual Tech. Conf., 1999.

Digital Library

[24]

D. Pariag, T. Brecht, A. Harji, P. Buhr, and A. Shukla. Comparing the performance of web server architectures. In Proc. of the 2nd ACM EuroSys Conf. on Computer Systems, pages 231--243, March 2007. ISBN 978-1-59593-636-3.

Digital Library

[25]

B. Schroeder, A. Wierman, and M. Harchol-Balter. Open versus closed: A cautionary tale. In Network System Design and Implementation, 2006. http://www.usenix.org/events/-nsdi06/tech/schroeder.html.

Digital Library

[26]

X. Song. Personal communication, 2011.

[27]

X. Song, H. Chen, R. Chen, Y Wang, and B. Zang. A case for scaling applications to many-core with OS clustering. In EuroSys Conf. on Computer Systems, 2011.

Digital Library

[28]

G. Upadhyaya, V. S. Pai, and S. P. Midkiff Expressing and exploiting concurrency in networked applications with Aspen. In Proc. of the 12th ACM SIGPLAN Symp. on Principles and Practice of Parallel Programming, pages 13--23, 2007.

Digital Library

[29]

B. Veal and A. Foong. Performance scalability of a multi-core web server. In Proc. of the 3rd ACM/IEEE Symp. on Architecture for Networking and Communications Systems, pages 57--66. ACM, 2007.

Digital Library

[30]

VMware. Consolidating web applications using VMware infrastructure. http://www.vmware.com/files/pdf/consolidating-_webapps_vi3_wp.pdf

[31]

R. von Behren, J. Condit, F. Zhou, G. C. Necula, and E. Brewer. Capriccio: Scalable threads for Internet services. In Proc. of the 19th ACM Symp. on Operating Systems Principles, pages 268--281, 2003.

Digital Library

[32]

I. Voras and M. Žagar. Characteristics of multithreading models for high-performance IO driven network applications. In AFRICON, 2009. AFRICON '09., pages 1--6, Sept. 2009.

[33]

M. Welsh, D. Culler, and E. Brewer. SEDA: An architecture for well-conditioned, scalable Internet services. In Proc. of the 18th ACM Symp. on Operating Systems Principles, pages 230--243. ACM Press, 2001.

Digital Library

[34]

N. Zeldovich, A. Yip, F. Dabek, R. T. Morris, D. Mazières, and F. Kaashoek. Multiprocessor support for event-driven programs. In Proc. of the USENIX Annual Tech. Conf., pages 239--252, June 2003.

Cited By

Soundararajan NKarne RWijesinha AOrdouie NChang H(2020)Design Issues in Running a Web Server on Bare PC Multi-Core Architecture2020 IEEE 44th Annual Computers, Software, and Applications Conference (COMPSAC)10.1109/COMPSAC48688.2020.0-195(555-564)Online publication date: Jul-2020
https://doi.org/10.1109/COMPSAC48688.2020.0-195
Cai ZLi Jzhang J(2020)Research on Performance Optimization of Web Application System based on JAVA EEJournal of Physics: Conference Series10.1088/1742-6596/1437/1/0120391437(012039)Online publication date: 7-Jan-2020
https://doi.org/10.1088/1742-6596/1437/1/012039
You GWang X(2020)A server-side accelerator framework for multi-core CPUs and Intel Xeon Phi co-processor systemsCluster Computing10.1007/s10586-019-03030-zOnline publication date: 1-Jan-2020
https://doi.org/10.1007/s10586-019-03030-z
Show More Cited By

Recommendations

Performance scalability of a multi-core web server
ANCS '07: Proceedings of the 3rd ACM/IEEE Symposium on Architecture for networking and communications systems

Today's large multi-core Internet servers support thousands of concurrent connections or ows. The computation ability of future server platforms will depend on increasing numbers of cores. The key to ensure that performance scales with cores is to ...
Evaluating multi-core and many-core architectures through accelerating the three-dimensional Lax-Wendroff correction stencil

Wave propagation forward modeling is a widely used computational method in oil and gas exploration. The iterative stencil loops in such problems have broad applications in scientific computing. However, executing such loops can be highly time-consuming, ...
Boundary element quadrature schemes for multi- and many-core architectures

In the paper we study the performance of the regularized boundary element quadrature routines implemented in the BEM4I library developed by the authors. Apart from the results obtained on the classical multi-core architecture represented by the Intel ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

SYSTOR '12: Proceedings of the 5th Annual International Systems and Storage Conference

June 2012

183 pages

ISBN:9781450314480

DOI:10.1145/2367589

General Chair:
Michael Vinov
IBM Haifa
,
Program Chairs:
Dan Tsafrir
Technion
,
Erez Zadok
Stony Brook University

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

The Technion - Israel Institute of Techn.: The Technion - Israel Institute of Technology

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 June 2012

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Funding Sources

Natural Sciences and Engineering Research Council of Canada

Conference

SYSTOR '12

Sponsor:

The Technion - Israel Institute of Techn.

SYSTOR '12: The 5th Annual International Systems and Storage Conference

June 4 - 6, 2012

Haifa, Israel

Acceptance Rates

Overall Acceptance Rate 108 of 323 submissions, 33%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
509
Total Downloads

Downloads (Last 12 months)15
Downloads (Last 6 weeks)1

Reflects downloads up to 09 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Soundararajan NKarne RWijesinha AOrdouie NChang H(2020)Design Issues in Running a Web Server on Bare PC Multi-Core Architecture2020 IEEE 44th Annual Computers, Software, and Applications Conference (COMPSAC)10.1109/COMPSAC48688.2020.0-195(555-564)Online publication date: Jul-2020
https://doi.org/10.1109/COMPSAC48688.2020.0-195
Cai ZLi Jzhang J(2020)Research on Performance Optimization of Web Application System based on JAVA EEJournal of Physics: Conference Series10.1088/1742-6596/1437/1/0120391437(012039)Online publication date: 7-Jan-2020
https://doi.org/10.1088/1742-6596/1437/1/012039
You GWang X(2020)A server-side accelerator framework for multi-core CPUs and Intel Xeon Phi co-processor systemsCluster Computing10.1007/s10586-019-03030-zOnline publication date: 1-Jan-2020
https://doi.org/10.1007/s10586-019-03030-z
Wang QZhang SKanemasa YPu C(2019)Mitigating Tail Response Time of n-Tier ApplicationsACM Transactions on Internet Technology10.1145/334046219:3(1-25)Online publication date: 25-Jul-2019
https://dl.acm.org/doi/10.1145/3340462
Wang QZhang SKanemasa YPu CPalanisamy BHarada LKawaba M(2019)Optimizing N-Tier Application Scalability in the CloudACM Transactions on Modeling and Performance Evaluation of Computing Systems10.1145/33261204:2(1-27)Online publication date: 13-Jun-2019
https://dl.acm.org/doi/10.1145/3326120
Zhang SWang QKanemasa YShan HHu L(2019)The Impact of Event Processing Flow on Asynchronous Server EfficiencyIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2019.2938500(1-1)Online publication date: 2019
https://doi.org/10.1109/TPDS.2019.2938500
Holzschuher FPeinl R(2016)Querying a graph database - language selection and performance considerationsJournal of Computer and System Sciences10.1016/j.jcss.2015.06.00682:1(45-68)Online publication date: 1-Feb-2016
https://dl.acm.org/doi/10.1016/j.jcss.2015.06.006
You GWang XZhao Y(2016)An Adaptive Dynamic Request Scheduling Model for Multi-socket, Multi-core Web ServersArabian Journal for Science and Engineering10.1007/s13369-016-2350-942:2(751-764)Online publication date: 27-Oct-2016
https://doi.org/10.1007/s13369-016-2350-9
Fan QWang Q(2015)Performance Comparison of Web Servers with Different ArchitecturesProceedings of the 2015 Third IEEE Workshop on Hot Topics in Web Systems and Technologies (HotWeb)10.1109/HotWeb.2015.11(37-42)Online publication date: 12-Nov-2015
https://dl.acm.org/doi/10.1109/HotWeb.2015.11
Chen XHo COsman RHarrison PKnottenbelt WLange KMurphy JBinder WMerseguer J(2014)Understanding, modelling, and improving the performance of web applications in multicore virtualised environmentsProceedings of the 5th ACM/SPEC international conference on Performance engineering10.1145/2568088.2568102(197-207)Online publication date: 22-Mar-2014
https://dl.acm.org/doi/10.1145/2568088.2568102
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents