Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3127479.3132252acmconferencesArticle/Chapter ViewAbstractPublication PagesmodConference Proceedingsconference-collections
research-article
Public Access

UNO: uniflying host and smart NIC offload for flexible packet processing

Published: 24 September 2017 Publication History
  • Get Citation Alerts
  • Abstract

    Increasingly, smart Network Interface Cards (sNICs) are being used in data centers to offload networking functions (NFs) from host processors thereby making these processors available for tenant applications. Modern sNICs have fully programmable, energy-efficient multi-core processors on which many packet processing functions, including a full-blown programmable switch, can run. However, having multiple switch instances deployed across the host hypervisor and the attached sNICs makes controlling them difficult and data plane operations more complex.
    This paper proposes a generalized SDN-controlled NF offload architecture called UNO. It can transparently offload dynamically selected host processors' packet processing functions to sNICs by using multiple switches in the host while keeping the data centerwide network control and management planes unmodified. UNO exposes a single virtual control plane to the SDN controller and hides dynamic NF offload behind a unified virtual management plane. This enables UNO to make optimal use of host's and sNIC's combined packet processing capabilities with local optimization based on locally observed traffic patterns and resource consumption, and without central controller involvement. Experimental results based on a real UNO prototype in realistic scenarios show promising results: it can save processing worth up to 8 CPU cores, reduce power usage by up to 2x, and reduce the control plane overhead by more than 50%.

    References

    [1]
    Accolade ANIC. https://accoladetechnology.com/whitepapers/ANIC-Features-Overview.pdf.
    [2]
    Cavium LiquidIO. http://www.cavium.com/pdfFiles/LiquidIO_Server_Adapters_PB_Rev1.0.pdf.
    [3]
    Data Center Market Trends. http://www.te.com/content/dam/te-com/documents/broadband-network-solutions/global/data-center/brochures/presentation-data-center-market-trends.pdf.
    [4]
    Emerging Smart NIC Technology. http://www.csit.qub.ac.uk/News/Events/Belfast-2016-6th-Cyber-Security-Summit/PDFs/Filetoupload,631658,en.pdf.
    [5]
    Floodlight. http://www.projectfloodlight.org/floodlight/.
    [6]
    How to Port Open vSwitch to New Software or Hardware. http://openvswitch.org/support/dist-docs-2.5/PORTING.md.html.
    [7]
    Intel Advanced Encryption Standard (Intel AES) Instructions Set - Rev 3.01. https://software.intel.com/en-us/articles/intel-advanced-encryption-standard-aes-instructions-set.
    [8]
    Intel Clear Containers: A Breakthrough Combination of Speed and Workload Isolation. https://clearlinux.org/sites/default/files/vmscontainers_wp_v5.pdf.
    [9]
    Intel Gigabit Server Adapters. http://ark.intel.com/products/family/46829.
    [10]
    Intel QuickAssist Adapter Family for Servers. http://www.intel.com/content/www/us/en/ethernet-products/gigabit-server-adapters/quickassist-adapter-for-servers.html.
    [11]
    Max-flow min-cut theorem. https://en.wikipedia.org/wiki/Max-flow_min-cut_theorem.
    [12]
    Mellanox BlueField. http://www.mellanox.com/related-docs/npu-multicore-processors/PB_Bluefield_SoC.pdf.
    [13]
    Mellanox ConnectX-4. http://www.mellanox.com/related-docs/prod_adapter_cards/PB_ConnectX-4_VPI_Card.pdf.
    [14]
    Mellanox ConnectX-5. http://www.mellanox.com/related-docs/user_manuals/ConnectX-5_VPI_Card.pdf.
    [15]
    Minimum k-cut. https://en.wikipedia.org/wiki/Minimum_k-cut.
    [16]
    nDPI. http://www.ntop.org/products/deep-packet-inspection/ndpi/.
    [17]
    Netronome Agilio vRouter. https://netronome.com/media/redactor_files/SB_Netronome_Juniper_vRouter.pdf.
    [18]
    OpenDaylight. https://www.opendaylight.org.
    [19]
    OpenStack. https://www.openstack.org.
    [20]
    OpenStack Compute API. https://developer.openstack.org/api-ref/compute/.
    [21]
    OPNFV. https://www.opnfv.org.
    [22]
    PRADS - Passive Real-time Asset Detection System. https://gamelinux.github.io/prads/.
    [23]
    Processor Counter Monitor. https://github.com/opcm/pcm.
    [24]
    Programming Intel QuickAssist Technology Hardware Accelerators for Optimal Performance. https://01.org/sites/default/files/page/332125_002_0.pdf.
    [25]
    Putting Smart NICs in White Boxes. https://www.sdxcentral.com/articles/analysis/nics-white-boxes/2016/11/.
    [26]
    SD-WAN. https://en.wikipedia.org/wiki/SD-WAN.
    [27]
    Setting up the Python API of CPLEX. http://www.ibm.com/support/knowledgecenter/SSSA5P_12.5.1/ilog.odms.cplex.help/CPLEX/GettingStarted/topics/set_up/Python_setup.html.
    [28]
    TILEncore-Gx36. http://www.mellanox.com/related-docs/prod_multi_core/PB_TILEncore-Gx36.pdf.
    [29]
    Tilera Rescues CPU Cycles with Network Coprocessors. https://www.enterprisetech.com/2013/10/16/tilera-free-expensive-cpu-cycles-network-coprocessors/.
    [30]
    VMware. Data Center Micro-Segmentation. http://blogs.vmware.com/networkvirtualization/files/2014/06/VMware-SDDC-Micro-Segmentation-White-Paper.pdf.
    [31]
    Watts Up Meter. https://www.wattsupmeters.com.
    [32]
    TILE Processor Architecture Overview for the TILE-Gx Series. Technical report, Mellanox, 2012. Doc. No. UG130.
    [33]
    OpenFlow Switch Specification 1.5.0. Open Network Foundation, 2014.
    [34]
    A. Al-Shabibi et al. OpenVirteX: Make Your Virtual SDNs Programmable. In Proc. ACM HotSDN, 2014.
    [35]
    S. P. Antoine Kaufmann and N. K. Sharma. High Performance Packet Processing with FlexNIC. In Proc. ASPLOS, 2016.
    [36]
    H. Ballani et al. Enabling End-host Network Functions. In Proc. ACM SIGCOMM, 2015.
    [37]
    A. Belay, G. Prekas, A. Klimovic, S. Grossman, C. Kozyrakis, and E. Bugnion. IX: A Protected Dataplane Operating System for High Throughput and Low Latency. In Proc. USENIX OSDI, 2014.
    [38]
    M. Blott and K. Vissers. Dataflow Architectures for 10Gbps Line-rate Key-value-Stores. In Proc. IEEE Hot Chips 25 Symposium, 2013.
    [39]
    P. Bosshart et al. P4: Programming Protocol-Independent Packet Processors. ACM SIGCOMM Computer Communication Review, 44(3), 2014.
    [40]
    Z. Bozakov and P. Papadimitriou. AutoSlice: Automated and Scalable Slicing for Software-Defined Networks. In Proc. ACM CoNEXT, 2012.
    [41]
    M. Casado, T. Koponen, S. Shenker, and A. Tootoonchian. Fabric: A Retrospective on Evolving SDN. In Proc. ACM HotSDN, 2012.
    [42]
    H. Chang, S. Mukherjee, L. Wang, T. Lakshman, Y. Le, A. Akella, and M. Swift. UNO: Unifying Host and Smart NIC Offload for Flexible Packet Processing. Technical Report ITD-16-56788B, Nokia, 2016.
    [43]
    Cisco. Data Center Microsegmentation: Enhance Security for Data Center Traffic. http://www.cisco.com/c/en/us/solutions/collateral/data-center-virtualization/application-centric-infrastructure/white-paper-c11-732943.html.
    [44]
    E. Cuervo et al. MAUI: Making Smartphones Last Longer with Code Offload. In Proc. ACM MobiSys, 2010.
    [45]
    H. T. Dang et al. Network Hardware-Accelerated Consensus. In USI Technical Report Series in Informatics, 2016.
    [46]
    R. R. David F. Bacon and S. Shukla. FPGA Programming for the Masses. ACM QUEUE, 11(2), 2013.
    [47]
    W. Dietz, J. Cranmer, N. Dautenhahn, and V. Adve. Slipstream: Automatic Interprocess Communication Optimization. In Proc. USENIX ATC, 2015.
    [48]
    S. K. Fayazbakhsh, L. Chiang, V. Sekar, M. Yu, and J. C. Mogul. Enforcing Network-Wide Policies in the Presence of Dynamic Middlebox Actions using FlowTags. In Proc. USENIX NSDI, 2014.
    [49]
    D. Firestone. SmartNIC: Accelerating Azure's Network with FPGAs on OCS Servers. Open Compute Project, 2016.
    [50]
    X. Ge, Y. Liu, D. H. Du, L. Zhang, H. Guan, J. Chen, Y. Zhao, and X. Hu. OpenANFV: Accelerating Network Function Virtualization with a Consolidated Framework in OpenStack. In Proc. ACM SIGCOMM, 2014.
    [51]
    A. Gember, P. Prabhu, Z. Ghadiyali, and A. Akella. Toward Software-defined Middlebox Networking. In Proc. ACM HotNets-XI, 2012.
    [52]
    A. Gember-Jacobson et al. OpenNF: Enabling Innovation in Network Function Control. ACM SIGCOMM Computer Communication Review, 44(4), 2015.
    [53]
    B. Grot et al. Optimizing Data-Center TCO with Scale-Out Processors. IEEE Micro, 32(5), 2012.
    [54]
    B. Han, V. Gopalakrishnan, L. Ji, and S. Lee. Network Functions Virtualization: Challenges and Opportunities for Innovations. IEEE Communication Magazine, 53(2), 2015.
    [55]
    S. Han, K. Jang, A. Panda, S. Palkar, D. Han, and S. Ratnasamy. SoftNIC: A Software NIC to Augment Hardware. Technical Report UCB/EECS-2015-155, University of California, Berkeley, 2015.
    [56]
    A. Holt et al. Cloud Computing Takes Off. https://www.morganstanley.com/views/perspectives/cloud_computing.pdf. Morgan Stanley.
    [57]
    M. Honda, F. Huici, G. Lettieri, and L. Rizzo. mSwitch: A Highly-Scalable, Modular Software Switch. In Proc. ACM SOSR, 2015.
    [58]
    J. Hwang, K. K. Ramakrishnan, and T. Wood. NetVM: High Performance and Flexible Networking using Virtualization on Commodity Platforms. In Proc. USENIX NSDI, 2014.
    [59]
    Z. Istvan, D. Sidler, G. Alonso, and M. Vukolic. Consensus in a Box: Inexpensive Coordination in Hardware. In Proc. USENIX NSDI, 2016.
    [60]
    E. J. Jackson, M. Walls, A. Panda, J. Pettit, B. Pfaff, J. Rajahalme, T. Koponen, and S. Shenker. SoftFlow: A Middlebox Architecture for Open vSwitch. In Proc. USENIX ATC, 2016.
    [61]
    M. Kablan, A. Alsudais, E. Keller, and F. Le. Stateless Network Functions: Breaking the Tight Coupling of State and Processing. In Proc. USENIX NSDI, 2017.
    [62]
    N. Kang, Z. Liu, J. Rexford, and D. Walker. Optimizing the One Big Switch Abstraction in Software-Defined Networks. In Proc. ACM CoNEXT, 2013.
    [63]
    Y. Kanizo, D. Hay, and I. Keslassy. Palette: Distributing Tables in Software-Defined Networks. In Proc. ACM CoNEXT, 2013.
    [64]
    N. P. Katta, J. Rexford, and D. Walker. Incremental Consistent Updates. In Proc. ACM SIGCOMM Workshop on Hot Topics in Software Defined Networking, 2013.
    [65]
    S. Kent. IP Encapsulating Security Payload (ESP). RFC 4303, 2005.
    [66]
    A. Khrabrov and E. de Lara. Accelerating Complex Data Transfer for Cluster Computing. In Proc. USENIX HotCloud, 2016.
    [67]
    Kindervag, J. Build Security Into Your Network's DNA: The Zero Trust Network Architecture.
    [68]
    S. Larsen and B. Lee. Platform IO DMA Transaction Acceleration. In Proc. ACM Workshop on Characterizing Applications for Heterogeneous Exascale Systems, 2011.
    [69]
    J. Li, E. Michael, N. K. Sharma, A. Szekeres, and D. R. K. Ports. Just say NO to Paxos Overhead: Replacing Consensus with Network Ordering. In Proc. USENIX OSDI, 2016.
    [70]
    K. Lim et al. Thin Servers with Smart Pipes: Designing SoC Accelerators for Memcached. In Proc. ISCA, 2013.
    [71]
    Y. Luo, E. Murray, and T. L. Ficarra. Accelerated Virtual Switching with Programmable NICs for Scalable Data Center Networking. In Proc. ACM VISA, 2010.
    [72]
    H. Mekky, F. Hao, S. Mukherjee, Z.-L. Zhang, and T. Lakshman. Application-aware Data Plane Processing in SDN. In Proc. ACM HotSDN, 2014.
    [73]
    M. Moshref, M. Yu, A. Sharma, and R. Govindan. vCRIB: Virtualized Rule Management in the Cloud. In Proc. USENIX HotCloud, 2012.
    [74]
    M. Moshref, M. Yu, A. Sharma, and R. Govindan. Scalable Rule Management for Data Centers. In Proc. USENIX NSDI, 2013.
    [75]
    J. Nam, M. Jamshed, B. Choi, D. Han, and K. Park. Scaling the Performance of Network Intrusion Detection with Many-core Processors. In Proc. ACM/IEEE ANCS, 2015.
    [76]
    S. Palkar, C. Lan, S. Han, K. Jang, A. Panda, S. Ratnasamy, L. Rizzo, and S. Shenker. E2: A Framework for NFV Applications. In Proc. ACM SOSP, 2015.
    [77]
    Palo Alto Networks. Getting Started With a Zero Trust Approach to Network Security. https://www.paloaltonetworks.com/resources/whitepapers/zero-trust-network-security.html.
    [78]
    T. Park, Y. Kim, and S. Shin. UNISAFE: A Union of Security Actions for Software Switches. In Proc. SDN-NFV Security, 2016.
    [79]
    S. Peter, J. Li, I. Zhang, D. R. K. Ports, D. Woos, A. Krishnamurthy, T. Anderson, and T. Roscoe. Arrakis: The Operating System is the Control Plane. In Proc. USENIX OSDI, 2014.
    [80]
    J. Pettit. Open vSwitch and the Intelligent Edge. In Proc. OpenStack Summit Atlanta, 2014.
    [81]
    B. Pfaff et al. The Design and Implementation of Open vSwitch. In Proc. USENIX NSDI, 2015.
    [82]
    Z. A. Qazi, C.-C. Tu, L. Chiang, R. Miao, V. Sekar, and M. Yu. SIMPLE-fying Middlebox Policy Enforcement Using SDN. In Proc. ACM SIGCOMM, 2013.
    [83]
    S. Radhakrishnan, Y. Geng, V. Jeyakumar, A. Kabbani, G. Porter, and A. Vahdat. SENIC: Scalable NIC for End-Host Rate Limiting. In Proc. USENIX NSDI, 2014.
    [84]
    B. Raghavan et al. Software-Defined Internet Architecture: Decoupling Architecture from Infrastructure. In Proc. ACM HotNets-XI, 2012.
    [85]
    K. K. Ram, A. L. Cox, M. Chadha, and S. Rixner. Hyper-switch: A scalable software virtual switching architecture. In Proc. USENIX ATC, 2013.
    [86]
    K. K. Ram et al. sNICh: Efficient Last Hop Networking in the Data Center. In Proc. ACM/IEEE ANCS, 2010.
    [87]
    L. Rizzo, P. Valente, G. Lettieri, and V. Maffione. PSPAT: software packet scheduling at hardware speed. Preprint, 2016.
    [88]
    G. Sabin and M. Rashti. Security Offload Using the SmartNIC, A Programmable 10 Gbps Ethernet NIC. In Proc. Aerospace and Electronics Conference, 2015.
    [89]
    V. Sekar, N. Egi, S. Ratnasamy, M. K. Reiter, and G. Shi. Design and Implementation of a Consolidated Middlebox Architecture. In Proc. USENIX NSDI, 2012.
    [90]
    A. Shaikh, J. Rexford, and K. G. Shin. Load-Sensitive Routing of Long-Lived IP Flows. In Proc. ACM SIGCOMM, 1999.
    [91]
    J. Sherry, S. Hasan, C. Scott, A. Krishnamurthy, S. Ratnasamy, and V. Sekar. Making Middleboxes Someone else's Problem: Network Processing As a Cloud Service. In Proc. ACM SIGCOMM, 2012.
    [92]
    R. Sherwood et al. FlowVisor: A Network Virtualization Layer. In OpenFlow Switch Consortium, 2009.
    [93]
    P. Shinde, A. Kaufmann, T. Roscoe, and S. Kaestle. We need to talk about NICs. In Proc. USENIX HotOS, 2013.
    [94]
    D. Sturgeon. HW Acceleration of Memcached. In Proc. Flash Memory Summit, 2014.
    [95]
    A. Tootoonchian and Y. Ganjali. HyperFlow: A Distributed Control Plane for OpenFlow. In Proc. Internet Network Management Conference on Research on Enterprise Networking, 2010.
    [96]
    A. Wang, Y. Guo, F. Hao, T. V. Lakshman, and S. Chen. UMON: Flexible and Fine Grained Traffic Monitoring in Open vSwitch. In Proc. ACM CoNEXT, 2015.
    [97]
    Z. Wang, K. Liu, Y. Shen, J. Y. B. Lee, M. Chen, and L. Zhang. Intra-host Rate Control with Centralized Approach. In Proc. IEEE International Conference on Cluster Computing, 2016.
    [98]
    Y. Weinsberg, D. Dolev, P. Wyckoff, and T. Anker. Accelerating Distributed Computing Applications Using a Network Offloading Framework. In Proc. IEEE Parallel and Distributed Processing Symposium, 2007.
    [99]
    M. Yu, J. Rexford, M. J. Freedman, and J. Wang. Scalable Flow-Based Networking with DIFANE. In Proc. ACM SIGCOMM, 2010.

    Cited By

    View all
    • (2024)DPC: DPU-accelerated High-Performance File System ClientProceedings of the 53rd International Conference on Parallel Processing10.1145/3673038.3673123(63-72)Online publication date: 12-Aug-2024
    • (2024)μMon: Empowering Microsecond-level Network Monitoring with WaveletsProceedings of the ACM SIGCOMM 2024 Conference10.1145/3651890.3672236(274-290)Online publication date: 4-Aug-2024
    • (2024)Hoda: a High-performance Open vSwitch Dataplane with Multiple Specialized Data PathsProceedings of the Nineteenth European Conference on Computer Systems10.1145/3627703.3629564(82-98)Online publication date: 22-Apr-2024
    • Show More Cited By

    Index Terms

    1. UNO: uniflying host and smart NIC offload for flexible packet processing

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      SoCC '17: Proceedings of the 2017 Symposium on Cloud Computing
      September 2017
      672 pages
      ISBN:9781450350280
      DOI:10.1145/3127479
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 24 September 2017

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. networking and SDNs
      2. virtualization and containers

      Qualifiers

      • Research-article

      Funding Sources

      Conference

      SoCC '17
      Sponsor:
      SoCC '17: ACM Symposium on Cloud Computing
      September 24 - 27, 2017
      California, Santa Clara

      Acceptance Rates

      Overall Acceptance Rate 169 of 722 submissions, 23%

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)419
      • Downloads (Last 6 weeks)49
      Reflects downloads up to 12 Aug 2024

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)DPC: DPU-accelerated High-Performance File System ClientProceedings of the 53rd International Conference on Parallel Processing10.1145/3673038.3673123(63-72)Online publication date: 12-Aug-2024
      • (2024)μMon: Empowering Microsecond-level Network Monitoring with WaveletsProceedings of the ACM SIGCOMM 2024 Conference10.1145/3651890.3672236(274-290)Online publication date: 4-Aug-2024
      • (2024)Hoda: a High-performance Open vSwitch Dataplane with Multiple Specialized Data PathsProceedings of the Nineteenth European Conference on Computer Systems10.1145/3627703.3629564(82-98)Online publication date: 22-Apr-2024
      • (2024)LiteFlow: Toward High-Performance Adaptive Neural Networks for Kernel DatapathIEEE/ACM Transactions on Networking10.1109/TNET.2023.329315232:1(627-642)Online publication date: Feb-2024
      • (2024)Un-IOV: Achieving Bare-Metal Level I/O Virtualization Performance for Cloud Usage With Migratability, Scalability and TransparencyIEEE Transactions on Computers10.1109/TC.2024.337558973:7(1655-1668)Online publication date: Jul-2024
      • (2024)A Comprehensive Survey on SmartNICs: Architectures, Development Models, Applications, and Research DirectionsIEEE Access10.1109/ACCESS.2024.343720312(107297-107336)Online publication date: 2024
      • (2024)FlowLever: Leverage Flow Director for Packet Dispatch Acceleration in NFVIEEE Access10.1109/ACCESS.2024.337370512(36122-36134)Online publication date: 2024
      • (2023)Resource scheduling techniques in cloud from a view of coordination: a holistic survey从协同视角论云资源调度技术:综述Frontiers of Information Technology & Electronic Engineering10.1631/FITEE.210029824:1(1-40)Online publication date: 23-Jan-2023
      • (2023)Toward Fast Query Serving in Key-Value Store Migration with Approximate TelemetryACM SIGMETRICS Performance Evaluation Review10.1145/3626570.362660451:2(91-93)Online publication date: 2-Oct-2023
      • (2023)YamaProceedings of the 2023 ACM Symposium on Cloud Computing10.1145/3620678.3624792(572-587)Online publication date: 30-Oct-2023
      • Show More Cited By

      View Options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Get Access

      Login options

      Media

      Figures

      Other

      Tables

      Share

      Share

      Share this Publication link

      Share on social media