Networking support for large scale multiprocessor servers
Pages 116 - 125
Abstract
Over the next several years the performance demands on globally available information servers are expected to increase dramatically. These servers must be capable of sending and receiving data over hundreds or even thousands of simultaneous connections. In this paper, we show that connection-level parallel protocols (where different connections are processed in parallel) running on a shared-memory multiprocessor can deliver high network bandwidth across a large number of connections.We experimentally evaluate connection-level parallel implementations of both TCP/IP and UDP/IP protocol stacks. We focus on three questions in our performance evaluation: how throughput scales with the number of processors, how throughput changes as the number of connections increases, and how fairly the aggregate bandwidth is distributed across connections. We show how several factors impact performance: the number of processors used, the number of threads in the system, the number of connections assigned to each thread, and the type of protocols in the stack (i.e., TCP versus UDP).Our results show that with careful implementation connection-level parallel protocol stacks scale well with the number of processors, and deliver high throughput which is, for the most part, sustained as the number of connections increases. Maximizing the number of threads in the system yields the best overall throughput. However, the best fairness behavior is achieved by matching the number of threads to the number of processors and scheduling connections assigned to threads in a round-robin manner.
References
[1]
M. L. Bailey, B. Gopal, M. A. Pagels, L. L Peterson, and E Sarkar PathFinder: a pattern-based packet classifier in First USENIX Symposium on Operating Systems Design and Implementation (OSDI), pages 115-123, Monterey, CA, Nov. 1994]]
[2]
J. M. Barton and N. Bitar. A scalable multi-discipline, multipleprocessor scheduling framework for IRIX. In IPPS '95 Workshop on Job Scheduling Strategies for Parallel Processing, pages 24-40, Santa Barbara, CA, Apr. 1995]]
[3]
M. Bjorkman and P. Gunningberg. Locking effects in multiprocessor implementations of protocols. In SIGCOMM Symposium on Communications Architectures and Protocols, pages 74-83, San Francisco, CA, Sept. 1993. ACM.]]
[4]
K. C Claffy, H.-W. Braun, and G. C. Polyzos. Internet traffic flow profiling. Technical Report UCSD Report CS93-328, SDSC Report GA-A21526, University of Califoma at San Diego, March 1994.]]
[5]
P. Druschel, L. Peterson, and B. Davie. Experiences with a high-speed network adaptor: A software perspective. In SIGCOMM Symposium on Communications Architectures and Protocols, pages 2-13, London, England, Aug. 1994. ACM.]]
[6]
P. Druschel and L. L. Peterson. Fbufs: A high-bandwidth cross-domain transfer facility. In Proceedings of the Fourteenth A CM Symposium on Operating Systems Principles, pages 189-202, Asheville, NC, Dec. 1993.]]
[7]
M. Gatles and E. Williams. Performance optimizations, implementation, and verification of the SGI Challenge multiprocessor. Technical report, Silicon Graphics Inc., Mt. View, CA, May 1994.]]
[8]
A. Garg. Parallel STREAMS: a multi-processor implementation. In Proceedings o.f the Winter 1990 USENIX Conference, Washington, D.C., Jan. 1990.]]
[9]
D. Giarrizzo, M. Kaiserswerth, T. Wicki, and R. C. Williamson. Highspeed parallel protocol implementation. First IFIP WG6.1/WG6.4 lnternattonal Workshop on ProtocolsJor High-Speed Networks, pages 165-180, May 1989.]]
[10]
M. W. Goldberg, G. W. Neufeld, and M. R. lto. A parallel approach to OSI connection-oriented protocols. Third IFIP WG6.1/WG6.4 international Workshop on Protocols for High-Speed Networks, pages 219-232, May 1992.]]
[11]
N. C. Hutchinson and L. L. Peterson. The x-Kernel: An architecture for implementing network protocols. IEEE Transactions on Software Engineering, 17(1):64-76, Jan. 1991.]]
[12]
V. Jacobson, R. Braden, and D. Borman. TCP extensions for high performance. In Network ln tormanon Center RFC 1323, pages 1-37, Menlo Park, CA, May 1992. SRI International.]]
[13]
N. Jain, M. Schwartz, and T. R. Bashkow. Transport protocol processing at Gbps rates. In SIGCOMM Symposium on Communications Architectures and Protocols, pages 188-199, Philadelphia, PA, Sept. 1990. ACM.]]
[14]
J. Kay and J. Pasquale. Measurement, analysis, and improvement of UDP/IP throughput for the DECStation 5000. In Proceedings oJ the Winter 1993 USENIX Conterence, pages 249-258, San Diego, CA, 1993.]]
[15]
O. G. Koufopavlou and M. Zitterbart Parallel TCP for high performance communication subsystems, in Proceedings of the Global Telecommunications Con.terence (GLOBECOM), pages 1395-1399, 1992.]]
[16]
T. F. La Porta and M. Schwartz. A high-speed protocol parallel implementation: Design and analysls. Fourth IFIP TC6.1/WG6.4 International ConJerence on High Performance Networking, pages 135-150, Dec. 1992.]]
[17]
B. Lindgren, B. Krupczak, M. Ammar, and K. Schwan. An architecture and toolkit for parallel and configurable protocols. In Proceedings of the International Conference on Network Protocols, pages 234--242, San Francisco, CA, Mar. 1993.]]
[18]
C. Maeda and B. N. Bershad. Protocol service decomposition for high-performance networking. In Proceedings ot the Fourteenth A CM Sympostum on Operating Systems Prmczptes, pages 244-255, Dec 1993.]]
[19]
S. McCanne and V. Jacobson. The BSD packet filter. A new architecture for user-level packet capture. In Proceedings o~ the Winter 1993 USENIX Conference, pages 259-269, Jan. 1993.]]
[20]
J. Mogul, R. Rashid, and M. Accetta. The packet filter: An efficient mechanism for user-level network code In Proceedings llth Symposium on Operating System Principles, pages 39-51, Austin, TX, November 1987.]]
[21]
A B. Montz, D. Mosberger, S. W. O'Malley, L. L. Peterson, T A Proebsting, and J. H. Hartman Scout: A communications-oriented operating system. Technical Report TR 94-20, University of Arizona, Tuscon, AZ, June 1994.]]
[22]
E. Nahum, D. J. Yates, S. O'Malley, H. Orman, and R Schroeppel Parallelized network security protocols. In Proceedings o.t the lnternet Society Symposium on Network and Distributed System Security, San Diego, CA, Feb. 1996.]]
[23]
E. M. Nahum, D. J. Yates, J. E Kurose, and D. Towsley. Performance issues in parallelized network protocols. In First USENIX Symposium on Operating Systems Designand Implementation (OSDI), pages 125- 137, Monterey, CA, Nov. 1994.]]
[24]
A. N. Netravali, W. D. Roome, and K. Sabnani. Design and implementation of a high-speed transport protocol. IEEE Transacttons on Commumcations, 38(11):2010-2024, Nov. 1990.]]
[25]
D. Presotto. Multiprocessor streams for Plan 9. In Proceedings o.t the United Kingdom UNIX Users Group, Jan. 1993.]]
[26]
J. D. Salehi, J F. Kurose, and D. Towsley. The performance impact of scheduling for cache affinity in parallel network processing. In International Symposium on High Performance Distributed Computmg (HPDC-4), Pentagon City, VA, Aug. 1995.]]
[27]
J.D. Salehi, J. E Kurose, andD. Towsley. The effectiveness of affinitybased scheduling in multiprocessor networking. In Proceedings ot the Conference on Computer Communications (IEEE lnfocom), page 2C.3, San Francisco, CA, Mar. 1996.]]
[28]
S. Saxena, J. K. Peacock, F. Yang, V. Verma, and M. Krishnan. Pitfalls in multithreading SVR4 STREAMS and other weightless processes. In Proceedings oJ the Winter 1993 USENIX Conference, pages 85-96, San Diego, CA, Jan. 1993.]]
[29]
D. C. Schmidt and T. Suda. Measuring the performance of parallel message-based process architectures. In Proceedings ~t the Conference on Computer Commumcations (IEEE Infocom), pages 624-633, Boston, MA, Apr. 1995]]
[30]
C. A. Thekkath, T. D. Nguyen, E. Moy, and E. D. Lazowska. Implementing network protocols at user level. In SIGCOMM Symposium on Communications Architectures and Protocols, pages 64-73, San Francisco, CA, Sept. 1993. ACM.]]
[31]
D. J. Yates, E. M. Nahum, J. F. Kurose, and D. Towsley. Networking support for large scale multiprocessor servers. Technical Report CMPSCI 95-83 (in preparation), URL = ftp://gaia.cs.umass.edu/pub/Yate95:Networking.ps.Z, University of Massachusetts, Amherst, MA, Dec. 1995.]]
[32]
M. Yuhara, B. N. Bershad, C. Maeda, and J. E. Moss. Efficient packet demultiplexing for multiple endpoints and large messages. In Proceedings of the Winter 1994 USENIX Conference, Jan. 1994.]]
Index Terms
- Networking support for large scale multiprocessor servers
Recommendations
Networking support for large scale multiprocessor servers
SIGMETRICS '96: Proceedings of the 1996 ACM SIGMETRICS international conference on Measurement and modeling of computer systemsOver the next several years the performance demands on globally available information servers are expected to increase dramatically. These servers must be capable of sending and receiving data over hundreds or even thousands of simultaneous connections. ...
Mobility support in hybrid wireless/IP networking
The ramifications of Transmission Control Protocol (TCP)/Internet Protocol (IP) networking with mobile hosts and wireless links in a hybrid wireless/IP environment are examined. Two issues that affect the performance of TCP/IP are mobility support in ...
Contention-based congestion management in large-scale networks
MICRO-49: The 49th Annual IEEE/ACM International Symposium on MicroarchitectureGlobal adaptive routing exploits non-minimal paths to improve performance on adversarial traffic patterns and load-balance network channels in large-scale networks. However, most prior work on global adaptive routing have assumed admissible traffic ...
Comments
Information & Contributors
Information
Published In
May 1996
273 pages
May 1996279 pages
Copyright © 1996 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 15 May 1996
Published in SIGMETRICS Volume 24, Issue 1
Check for updates
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations12Total Citations
- 342Total Downloads
- Downloads (Last 12 months)43
- Downloads (Last 6 weeks)11
Reflects downloads up to 09 Sep 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in