ClusterLink: A Multi-Cluster Application Interconnect Kfir Toledo, Pravein G. Kannan, Malka M., Lev-Ran E., Barabash K., Bortnikov V. {kfir.toledo,pravein.govindan.kannan,michal.malka}@ibm.com,{etail,kathy,vita}@il.ibm.com IBM Research ABSTRACT Enterprises often deploy their business applications in multiple clouds as well as in multiple traditional environments. This work focuses on the connectivity aspects of this new way of operating and consuming digital services. We define the related requirements, analyze the challenges, and present ClusterLink, our solution for interconnecting today’s and future multi-cloud applications. CCS CONCEPTS Figure 1: ClusterLink design model · Networks → Cloud computing. Connection oriented. To satisfy enterprise-grade requirements for controlled, secure, and flexible application-level connectivity with predictable performance, the platform must operate on business-meaningful connections, e.g., service-to-service or clientto-service communication flows, or at the L4 level of the networking stack. The above principles have been used to design ClusterLink, a solution for cross-cluster service connectivity (Fig. 1). KEYWORDS SDN, Cloud Networking, Multi-cloud Networking 1 INTRODUCTION Increasingly more businesses host applications in multiple clouds, e.g., to ensure global presence and availability or to avoid vendor lock-in. Together with the obvious benefits, this increases the complexity involved in operating enterprise workloads, especially as it comes to establishing connectivity across different environments. We identify two main challenges: (1) Lack of uniform abstraction, as different solutions offer different ways of specifying applications and their connectivity needs (e.g., VPC, Skupper [3], Istio). (2) Lack of interoperability and of architectural openness as most solutions operate as private overlays (e.g., Aviatrix [1]) and do not support ecosystem-driven extensibility and innovation. Due to these and other challenges, no solution available today can satisfy the connectivity needs of truly global enterprises across multiple clouds, private data centers, branch offices, and remote facilities. We envision an extensible multi-cloud connectivity platform to enable seamless application-centric connectivity across the existing private network providers. This poster presents the platform’s first prototype, ClusterLink, created to connect K8s services across multiple clusters. 2 3 IMPLEMENTATION & EVALUATION ClusterLink prototype is implemented in Golang following the generic principles outlined above as a set of unprivileged gateways serving connections to and from K8s services according to policies defined through the management APIs. ClusterLink gateways establish mTLS connections between them and continuously exchange control-plane information, forming a secure distributed control plane. In addition, ClusterLink gateways represent the remotely deployed services to applications running in a local cluster, acting as L4 proxies. On connection establishment, the control plane components in the source and the target ClusterLink gateways validate and establish the connection based on specified policies, then promote the control connection into a data plane session. The distributed control plane and the fine grained connection establishment control are the main advantages of ClusterLink over its competitors, e.g. Skupper. ClusterLink prototype is demoed with Istio BookInfo [2] application, with micro-services deployed in different clusters, located in different cloud and on-prem environments, and powered by different CNI plugins. The prototype supports the two most prominent connection establishment policies, access control and load balancing. Performance evaluation on clusters deployed in the same zone of the Google Cloud with 7𝐺𝑏𝑝𝑠 provisioned bandwidth shows that ClusterLink can outperform Skupper by 1.87×, with a 5.05𝐺𝑏𝑝𝑠 for a single iPerf3 connection, compared to Skupper’s 2.7𝐺𝑏𝑝𝑠. We now work on open-sourcing the ClusterLink and on enhancing its capabilities, e.g., to support non-cluster environments richer set of policies, as well as to improve automation and scalability. DESIGN PRINCIPLES ClusterLink design is guided by the following principles: Programmable. The platform must support creating use-case specific end-to-end connectivity solutions, e.g. by following the SDN paradigm and exposing programmable interfaces between the control, the data, and the management components. Open and Extensible. The platform must operate across administrative, technology, and vendor boundaries without interfering with the existing networks installed in the interconnected locations. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s). SYSTOR ’23, June 5–7, 2023, Haifa, Israel © 2023 Copyright held by the owner/author(s). ACM ISBN 978-1-4503-9962-3/23/06. https://doi.org/10.1145/3579370.3594747 REFERENCES [1] Aviatrix. 2023. Aviatrix documentation. https://docs.aviatrix.com [2] Istio Bookinfo. 2023. Istio Bookinfo. https://istio.io/latest/docs/examples/bookinfo [3] Skupper. 2023. Multicloud communication for Kubernetes. https://skupper.io 138