Minimizing latency in geo-distributed clouds

J Supercomput (2015) 71:4423–4445 DOI 10.1007/s11227-015-1538-1 Minimizing latency in geo-distributed clouds Marzieh Malekimajd1 · Ali Movaghar1 · Seyedmahyar Hosseinimotlagh2 Published online: 26 October 2015 © Springer Science+Business Media New York 2015 Abstract Geo-distributed data-centers in cloud computing are becoming increasingly popular due to their lower end-user perceived latency and increased reliability in distributed applications. The important challenge of resource allocation in cloud management is more pronounced in geo-distributed data-centers compared to traditional data-centers. A geo-distributed cloud manager faces applications whose virtual machines (VMs) are far apart and need to interact with end users, access distributed data and communicate with each other. In such applications, the service level agreement is not met if the communication latency is not bounded. In this paper, we focus on the problem of finding data-centers for hosting VMs while the requested VMs are located in different geo-distributed data-centers and are sensitive to the communication latency. We propose an algorithm to minimize communication latency by taking into account the cloud network topology to be either a tree or that of the Internet. Moreover, our algorithm can utilize user’s locations to find better candidate solutions. In the case of tree topology, we prove that our algorithm finds a solution whose latency is minimum. In addition, we show that our algorithm performs well in the Internet topology, with simulation results indicating that it can reduce the communication latency up to 92 % compared to existing algorithms. B Marzieh Malekimajd m_malekimajd@yahoo.com; malekimajd@ce.sharif.edu Ali Movaghar movaghar@sharif.edu Seyedmahyar Hosseinimotlagh seyedmah@ualberta.ca 1 Department of Computer Engineering, Sharif University of Technology, Tehran, Iran 2 Department of Electrical and Computer Engineering, University of Alberta, Edmonton, Canada 123 4424 M. Malekimajd et al. Keywords Geo-distributed cloud · Resource allocation · Communication latency · Internet treeness 1 Introduction Cloud computing is a model for providing access to computing resources from everywhere, in any time, while the access should be easy, on demand and thorough a network [5,24]. In cloud environment, an infrastructure provider uses cooperative data-centers to serve customers [35]. Nowadays, large companies such as Google, Yahoo and Microsoft are moving towards using and building geographically distributed datacenters, which are interconnected via the Internet [10]. Geo-distributed data-centers are usually smaller in size and larger in number compared to traditional data-centers. Geo-distribution of data-centers leads to more power efficient clouds [31]. Infrastructure providers achieve more reliability and less cost by leveraging the geographic distribution of data-centers to meet the requirements of location diversity [16]. Indeed, infrastructure providers can benefit from data-centers located in different geographical regions to serve a user request from closer datacenters to lower communication latency in comparison with traditional data-centers. Therefore, they can offer better services. It is worthwhile to mention that minimizing latency leads to less bandwidth consumption [25]. One of the most important challenging problems for infrastructure providers is resource management. This problem has a significant impact on the revenue of the provider and user satisfaction. A cloud manager fulfills its responsibility by monitoring the cloud infrastructure, maintaining an updated view of the available resources and finding an effective resource allocation for each request. An effective resource management approach will be achieved by exploiting a proper model of requests and resources. The model has to be based on the need of the users and the cloud infrastructure. The challenge of resource allocation is more pronounced in geo-distributed clouds because of the large number of data-centers and the noticeable communication latency. In a geo-distributed cloud, in some situations the allocator needs to consider the geodistribution of allocated resources. Therefore, it is possible that more than a single data-center can serve a request submitted by a user. This can arise due to constraints from users. For instance, when users reveal some locations to be near to their allocated VMs, they bound the maximum number of allocated VMs on a single data-center or restrict the locations of data-centers that host their VMs. The need for multiple data-centers can also arise due to security issues or lack of free capacity on a single data-center. In the infrastructure-as-a-service layer, cloud managers provide VMs to the users. Therefore, each user requests a number of VMs with certain specifications. Moreover, the cloud manager can get information from the users about the interaction points with VMs, or the locations where the user’s data are stored. When the geo-distributed cloud manager receives a request from a user, the resource allocator considers both the cloud profile and the submitted request to find candidate solutions. The allocator has to find a set of data-centers and decide on how many VMs should be hosted in each 123 Minimizing latency in geo-distributed clouds 4425 data-center. Therefore, the allocator needs an algorithm to find a set of data-centers with guaranteed communication latency. In this paper, we aim to optimize data-intensive resource allocation in geodistributed clouds. We focus on the problem of finding a set of data-centers to allocate the requested VMs. We propose three algorithms for finding a set of hosts, geodistributed data-centers, in different situations. First algorithm works for data-centers interconnected by a tree network. Second algorithm works for data-centers interconnected by the Internet. The solution found by this algorithm is better where the Internet network is more like a tree. Third algorithm is also for data-centers interconnected by a tree. This algorithm uses the revealed locations of the user and also provides the guaranteed communication latency. The initial novelty of our algorithm is in modeling the graph of the communication latency as a tree, because the Internet topology with respect to latency and bandwidth is tree-like, as argued by Ramasubramanian et al. in [28]. In this work, the authors introduce a metric to quantify the degree to which the Internet topology can be approximated by a tree with respect to the latency and bandwidth. One difference of our network model with that of some prior studies is that we do not consider triangle inequality (e.g., [4] assumes the cloud network to be a complete graph with triangle inequality). In fact, Eriksson et al. [13] showed that the triangle inequality is violated in the Internet network. Another novelty is in providing a solution in which the set of data-centers with low communication latency is found with low time complexity. Also, it is proved that the proposed algorithm finds a latency optimal solution for tree networks. Moreover, the experimental results show that the proposed algorithm for the Internet network reduces the latency of communication in geo-distributed environments by up to 92 %. It is worthwhile to mention that the proposed method in this paper can be used as a preprocessing phase before VM allocation process in other frameworks like [15,21] is executed. The rest of this paper is organized as follows: In Sect. 2, we give a brief review of the related work. In Sect. 3, we present our model. Then, we describe our problem in Sect. 4. We provide our three proposed algorithms in Sect. 5. The experiment results are discussed in Sect. 6. Finally, Sect. 7 concludes the paper. 2 Related work Resource allocation is a challenging problem in cloud computing. It is responsible for improving resource utilization and providing guarantees for the quality of services. The main challenges of the resource allocation process in distributed clouds are categorized with respect to modeling, offering, monitoring and selecting resources [12], which are defined briefly in the next section. In this paper, we consider the challenge of selecting resources and our focus is on selecting geo-distributed data-centers to host requested VMs regarding communication latency. We do not cover the server selection sub-process, because the latency between co-located servers is negligible compared to the intra-data-center latency in a geo-distributed cloud (i.e., the latency between geo-distributed data-centers). On the other hand, the server selection process can be adopted from literature with respect to different policies like [6,11,18]. 123 4426 M. Malekimajd et al. Resource management and resource allocation have received a considerable amount of attention in the literature. Different problem solving techniques are used in resource management e.g., bin packing, heuristics and optimization techniques [17]. A comprehensive survey of research on resource management was provided in [19]. Some examples of resource management in geo-distributed clouds are the studies for resource allocation that consider data access latency [20], cost efficiency [8], communication cost [33], user’s location [23] and VM migration [3]. One of the important aspects of resource management in geo-distributed datacenters is the communication latency caused by the network. In this regard, geodistributed clouds and their networks have been modeled in a variety of ways and levels. For instance, there are studies on considering networks as virtual resources. Papagianni et al. [27] modeled both request of a user and substrate network as a weighted undirected graph and formulated the networked cloud mapping problem as a mixed integer programming problem. There is still a need to provide models and solutions in resource allocation by considering the importance of communication latency. The model of the cloud network has significant impact on the efficiency of resource allocation, especially in geo-distributed data-centers. There are efforts to identify network topology of clouds [33] and examine the similarity of the Internet latencies to tree metrics [1]. Ramasubramanian et al. [28] showed that the structure of the Internet is like a tree, especially regarding to latency and bandwidth. They called this structure property as the treeness property, which we adopt in this paper. Also, they proposed a method to embed network latency into a tree. Besides, Eriksson et al. [14] build the logical tree structure of the Internet by tomographic methodologies in a practical way, which can be used to produce the Internet latency tree. The limitation of this tomography method is that the resulting tree depends on the probing node. Consequently, the methods in [14,28] can be considered to address the challenges of network modeling in a geo-distributed cloud. These methods show the similarity of the Internet network topology to a tree network in respect of latency. They can be used to approximate the network of a geo-distributed cloud by a tree. The studies in [4,22], similar to our study, considered the problem of minimizing latency for VMs placement in distributed cloud. These studies assumed that the communication network is a complete graph with triangle inequality. The authors in [22] considered access to data nodes in addition to VM communications. It is shown in [4,22] that there is no solution better than 2-approximation when the triangle inequality is satisfied for the graph topology of the network. Also, these studies proposed 2-approximation algorithms while they did not provide explicit time complexity of their proposed algorithm. The initial difference between our study and the studies in [4,22] is how the latency is modeled in a cloud network. The approximate treeness of the Internet network inspires modeling the cloud network by a tree graph. This model leads to a change in the hardness of the problem. Overall, we do not assume a complete graph with the triangle inequality for the network topology. We replace this assumption with treeness of the network topology. In the case of a tree network, the problem can be solved in polynomial time. This is another difference of our algorithm with prior studies. We propose an algorithm that finds an optimal latency for the VM allocation in tree 123 Minimizing latency in geo-distributed clouds 4427 networks. Moreover, our algorithm is extended for tree-like networks e.g., latencies caused by the Internet. Hence, we can have a better VM allocation policy with a lower communication latency for the case of the Internet. 3 Background We consider a geo-distributed cloud in which data-centers are located in different geographical locations and interconnected through the Internet network [34]. There is a manager in the geo-distributed cloud who is in the charge of the whole cloud and manages various activities within the cloud. This cloud manager can be decentralized to avoid the potential of having a single point of failure [26]. One of the most important responsibilities of the cloud manager is resource allocation. Takako Endo et al. [12] defined four challenges for the resource allocation process in geo-distributed clouds. These four challenges are modeling, offering, monitoring and selecting resources. Modeling of cloud resources describes how the cloud deals with infrastructural resources. After the cloud resources are modeled, several interfaces are needed to handle resource treatments and requests. The challenge of resource offering includes the designing of such interfaces. However, resource modeling can be independent of how resources are offered. In resource monitoring, the cloud manager focuses on the discovery of the proximity status of resources, in a way that the discovery causes minimal impact on the network performance. The last challenge, the process of resource selection, uses optimization algorithms to find configurations that fulfill the requirements of the user while optimizing the utilization of the cloud infrastructure. In the underlying cloud system, the manager has an updated profile of cloud resources status. The monitoring part of resource management provides this updated profile for the cloud manager [29]. This profile has information including the free capacity and bandwidth of each data-center and the intra-latency between data-centers. This information in the profile shows how many VMs can be hosted by a data-center in various conditions. The cloud manager uses this profile for resource allocation decisions. In the process of resource management, the cloud manager receives request for a number of VMs by user. Such request could have some constraints in addition to the number of requested VMs. For example, a user can express a set of multiple geographical locations besides the number of requested VMs. These revealed locations are where the end users interact with allocated VMs or where the dedicated data of allocated VMs are stored. The cloud manger has to estimate the communication latencies between these revealed locations and each data-center. Hence, submitting a request to the cloud interface, the cloud manager examines the request and the cloud profile to find the appropriate resource allocation candidates. The considered problem in this paper can be applied to interactive online applications, communication intensive applications and MapReduce applications that need to be distributed. In these applications, the latency between every two terminals (let a terminal be either a VM, a data-center, or a revealed location) is important and has significant impact on job completion time. In this paper, the latency is considered as 123 4428 M. Malekimajd et al. the time needed for sending or receiving data between terminals. The latency between two VMs is assumed to be equal to the latency between their hosting data-centers. Also, the latencies are assumed to be symmetric i.e., the latencies of receiving and sending data are equal. Otherwise, the greater one is adopted as the communication latency, because in this paper we focus on minimizing the maximum latency. When the cloud manager receives a request, it searches for a set of data-centers to host VMs. Therefore, the cloud manager selects a set of data-centers as a candidate set. Then it decides how many VMs from each data-centers should be allocated to the user. The focus of this paper is on designing an algorithm as a part of resource allocation procedure to find a candidate set of data-centers for hosting the requested VMs. It is also assumed that the free capacity of each data-center, the number of requested VMs and the communication latencies between data-centers are available from the monitoring process [2,7]. Other challenges of the resource management can be dealt with as in [9,30]. The other challenges include allocating and deallocating VMs from a specified data-centers, interacting with users, and holding an updated cloud profile. Also, a cloud management system, like [15,21], can use our algorithms as a part of its resource allocation process. 4 Problem description We model the problem of finding an appropriate set of data-centers as a weighted graph problem. In our model, there is no need to construct the graph with all the details i.e., all nodes, edges and weights. Moreover, there is no need to know all the cloud properties to construct the considered partial graph. In the following, it is stated how the problem of selecting data-centers is considered as a graph problem. Each node of the graph represents one of following three entities: (1) A data-center that is represented by a weighted node in the graph model. The weight shows data-center’s free capacity. (2) A revealed location of a user represented by a zero weight node in the graph model. (3) A switch that is not available in the graph model (actually these nodes are the absent nodes of the graph in our model). The number of these nodes is not required. The weight of the graph edges represents the communication latencies. For example, for two nodes of entities (1) and (2), the weight of a path between them equals their communication latency. The considered path between two nodes of entities (1) and (2) is not necessarily the shortest or longest path between them. The considered path for routing in the network may not be the shortest path e.g., because of the security or fault issues. Hence, a subset of graph’s nodes with the weight of a path between every two nodes of this subset is constituted the graph model of the cloud profile. In the graph model, the problem of finding a set of data-centers with a capacity of at least r is equivalent to selecting a set of nodes with a weight of at least r (let r be the number of VMs requested by a user). In addition, the problem of minimizing the latency of the selected set of data-centers is equivalent to minimizing the latency of the 123 Minimizing latency in geo-distributed clouds 4429 Table 1 Notation definitions Notation Definition r The number of VMs that a user requests ltnc(m 1 , m 2 ) ltnc of two data-centers m 1 and m 2 is their communication latency. Equally, ltnc of two nodes is the weight of a path between them. ltnc of two nodes of the tree graph equals to their distance. In addition, ltnc(m 1 , m 2 ) equals to ltnc(m 2 , m 1 ) ltnc(M) For a set M of data-centers (or nodes), ltnc is defined as the maximum value of the set {ltnc(m1, m2)|m 1 ∈ Mand m 2 ∈ M} weight For a set of data-centers, weight is defined as the summation of free capacity of these data-centers in terms of VMs. Equally, in a graph, weight of a subset of nodes is the summation of their weights u–v The shortest path on the graph that connects the node u to the node v is showed by u–v D The set of nodes that corresponding to the geo-distributed data-centers in the cloud L The set of nodes that corresponding to the revealed geographical locations of the user Dpaired A list of pairs {(i, j)|i  = j, i ∈ D and j ∈ D} that is sorted decreasingly with respect to the latency of pairs selected set of nodes. In the rest of this section, three different versions of the problem are given in detail. The notation adopted in this paper is summarized in Table 1. In the first version of the problem, the network graph is a tree. Each data-center is modeled as a leaf of the tree while the switches are the non-leaf nodes. In this version, there is no revealed location from the users. There is a unique path between any two leaves because the given graph is a tree. The total weight of edges along this path represents the latency between two data-centers, which equals the weight of the unique path between their corresponding leaves. Other information about the tree e.g., the weight of an individual edge, is neither available, nor required in our algorithm. For leveraging this proposed algorithm in cloud infrastructure, the topology of the network in respect of latency needs to be a tree or be approximated by a tree. In the second version, there is no assumption on the graph topology. The distance and the latency between two nodes, which represent data-centers, would be different. Other assumptions about the problem are the same as the first version. The performance of the proposed solution depends on the level that the network topology is approximated by a tree. In this case, the topology of the Internet is known as the network topology of geo-distributed data-centers, which can be well approximated by a tree [1]. The third version is obtained from the first one by assuming that the user reveals multiple locations for storing data or communicating with requested VMs. In this version, the revealed locations are modeled as the leaves with a weight of zero. The distance between two nodes that represent the locations of the user is not used. However, for other pairs of leaves, the distance of two leaves is set to the their communication latency. The considered problem in the graph is to find a subset of nodes such that the summation of their weights will be at least the number of the requested VMs. Moreover, the aim of minimizing latency is modeled by minimizing the diameter of selected nodes in association with the revealed locations. 123 4430 M. Malekimajd et al. 5 Proposed algorithm In this section, we propose three algorithms for the three graph problems explained in the previous section. Also, we discuss the time complexity and optimality of these algorithms. These algorithms only need to know the weight of the data-center representative nodes, the latency between the nodes that represent the data-centers and the locations of the user (if they are revealed). The goal of each algorithm is to find a subset of nodes with the weight of at least r while the ltnc of this subset is as low as possible. In the first version of the problem, the graph topology is a tree. For this version, we propose the TreeOpt algorithm, which returns a candidate set of nodes with the minimum ltnc. In the second version, the graph topology is not necessarily a tree. For this version, we propose the NetOpt algorithm to find a candidate set of nodes. In this algorithm, if the triangle inequality is satisfied for the latency between nodes, the ltnc of the output set is at most twice the minimum ltnc. In the third version of the problem, the graph topology is a tree and a subset of leaves represents revealed locations of the user. In this version, we propose the TreeOptExt algorithm, which returns a candidate set of nodes with optimal latency. In other words, the candidate set in association with user representative nodes has minimum ltnc compared to other possible sets. 5.1 TreeOpt algorithm The TreeOpt algorithm takes the number r (which denotes the number of requested VMs), the subset D of tree leaves and the ltnc of each pair in the set {(i, j)|i ∈ D, and j ∈ D} as input. This algorithm returns a subset of D with the weight of at least r while there is no other subset of D with the weight of at least r and smaller latency. Since the TreeOpt algorithm uses the FindSet function as a subroutine, first the details of the FindSet function are explained, and then the steps of the TreeOpt algorithm are discussed. It is worthwhile to mention that ltnc of a subset of leaves is equal to its diameter and can be used interchangeably, because the latency and distance are equal in a tree. The FindSet function takes two leaves u and v of a tree as input and finds a maximal subset of D including u and v with diameter ltnc(u, v). Initially, the FindSet function assumes a set P that consists of only two elements u and v. Then, it checks each leaf d ∈ D to see whether this leaf can be added to the set P or not. A leaf can be added to the set P if it does not increase the ltnc(P). Hence, if ltnc(d, u) is not greater than ltnc(u, v) and ltnc(d, v) is not greater than ltnc(u, v), the FindSet function adds the node d to the set P. Finally, when the FindSet function examines all the members of D, it returns the set P as output. The pseudo-code of the FindSet(u, v) function is shown in Algorithm 1. Having the operation of the FindSet(u, v) function, the TreeOpt algorithm can be described. The TreeOpt algorithm examines all possible candidates to find the optimal subset of leaves. Hence, for each pair of leaves, the TreeOpt algorithm produces a candidate set. Then, the algorithm compares it with the set S, which is the best candidate set found so far. Initially, the set S is initialized to the set D. If the weight of 123 Minimizing latency in geo-distributed clouds 4431 Algorithm 1: FindSet function Require: (u, v), D and {ltnc(u, v)| u ∈ D and v ∈ D} P ← {u, v} for each d ∈ D do if ltnc(d, u) ≤ ltnc(u, v) and ltnc(d, v) ≤ ltnc(u, v) then P ← P ∪ {d} end if end for return P D is less than r , no possible candidate can be found and the TreeOpt algorithm returns Null. However, since the problem is originated from the cloud environment, the free capacity of the whole cloud is usually greater than the resources requested by a user. Therefore, the weight of D is assumed to be always greater than r . More specifically, the TreeOpt algorithm takes every pair of leaves u and v of the set {(u, v)|u ∈ D and v ∈ D}. If the latency between u and v is not less than the latency of the set S, the TreeOpt algorithm skips the pair (u, v) and proceeds to another pair of leaves. Otherwise, it calls the FindSet(u, v) function that produces a candidate set including both u and v with the latency ltnc(u, v). If the weight of the candidate set reported by FindSet(u, v) is more than or equal to r , the TreeOpt algorithm replaces the set S with the output set of the FindSet(u, v) function. The pseudo-code of the TreeOpt algorithm is shown in Algorithm 2. Algorithm 2: TreeOpt algorithm Require: r, D and {ltnc(u, v)| u ∈ D and v ∈ D} S←D for each (i, j) ∈ D × D(i  = j) do if ltnc(i, j) < ltnc(S) then tempS ← Find Set (i, j) if weight (tempS) > r then S ← tempS end if end if end for return S In the following, first we state a lemma on the output of the FindSet(u, v) function. Second, we propose a theorem on the optimality of the TreeOpt(r ) algorithm. At last, we discuss the time complexity of the TreeOpt algorithm. Lemma 1 While the nodes of the set D in the FindSet(u, v) function are leaves of a tree, FindSet(u, v) returns a maximal set including u and v such that the latency of this output set is equal to ltnc(u, v). Proof See the “Appendix”. Theorem 1 The TreeOpt algorithm takes the leaves of a tree, the latency between these leaves and a number r as input. The TreeOpt algorithm returns a subset of the 123 4432 M. Malekimajd et al. tree leaves with the weight of at least r while its latency is less than or equal to any subset of weight at least r . Proof Let S be the output of the TreeOpt algorithm and Smin be the optimal set such that Smin has the minimum latency and its weight is at least r . To prove the Theorem 1, it is sufficient to show that ltnc(S) is no more than ltnc(Smin ). Let a and b be two members of Smin that ltnc(a, b) equals to ltnc(Smin ). The TreeOpt algorithm calls the FindSet function on each pair of leaves from the input tree. Therefore, it calls the FindSet function on the nodes a and b too. By Lemma 1, the FindSet(a, b) function returns a maximal set including a and b with the latency ltnc(a, b). Since this set is maximal, it includes all members of the set Smin . Therefore, the output of FindSet(a, b) has the weight of at least r . The output of FindSet(a, b), as a candidate set, is checked by the TreeOpt algorithm while at each step of the algorithm, the best candidate is held or improved. Consequently, the latency of the final solution S (the best candidate set that is found by the TreeOpt algorithm) is better than or equal to the output of FindSet(a, b) and ltnc(S) is not greater than ltnc(a, b) = ltnc(Smin ). Time complexity of TreeOpt Let n be the number of the given tree leaves. The TreeOpt algorithm calls the FindSet function on each pair of leaves from the given tree that there are O(n 2 ) pairs of such leaves. The FindSet function takes O(n) time, because it checks each leaf of the given tree. Therefore, in the worst case that all pairs of leaves are examined, the time complexity of the TreeOpt algorithm is O(n 3 ). Discussion on parallelism The iterations of the main loop (the For loop) in the TreeOpt algorithm are independent and can be executed simultaneously (in parallel). As a result, this algorithm can be run in a distributed fashion that leads to reduction in the time complexity. Let us consider there are m cores to run the TreeOpt algorithm. The set D × D is divided to m subsets while each core executes the algorithm on one subset. When all cores executed the algorithm, a minimum selection algorithm finds the best candidate set S from the candidate sets that are calculated by the cores. Overall, the time complexity is O(n 2 /m × n + m) that equals to O(n 3 /m) because m < n. 5.2 NetOpt algorithm The NetOpt algorithm takes the number r , a subset D of the graph nodes and the latency between any two nodes of D as input. This algorithm returns a subset of D with the weight of at least r as output. Also, the latency of the output of the NetOpt algorithm is less than twice of the latency of the optimal solution while the triangle inequality holds for the latencies of the set {(i, j)|i ∈ D, and j ∈ D}. The NetOpt algorithm uses the FindSet function and the PureSet function as its subroutines. The FindSet function is already defined in the TreeOpt algorithm. TreeOpt calls this function on two leaves and a subset of graph leaves, rather than two nodes 123 Minimizing latency in geo-distributed clouds 4433 and a subset of graph nodes. Therefore, the input of the FindSet function consists of nodes rather than leaves. In the following, the PureSet function is defined, and then the details of the NetOpt algorithm are illustrated. Finally, the optimality and the time complexity of this algorithm are discussed. The PureSet function takes a subset S of the graph nodes D as input. This function iteratively removes a node from the set S in order to decrease the latency of the set S. The removing process continues until the weight of the set S is at least r . The PureSet function needs a decreasing sorted list of ltnc between members of the set S to start the removing process. Therefore, the PureSet function uses the list Dpaired to compute the needed decreasing sorted list. The NetOpt algorithm provides the sorted list Dpaired = {(i, j)|i = j, i ∈ D and j ∈ D} that its members are sorted decreasingly in respect of their latency. The PureSet function scans the NetOpt list once and removes the redundant pairs from the list Dpaired to make the new list NewDpaired , which the pairs of its input nodes are sorted decreasingly. After computing the decreasing sorted list NewDpaired , the PureSet function starts its processing from the beginning of this list. At each step, it removes the first pair from the sorted list NewDpaired and checks the two nodes of this pair. If both nodes are in the set S, it removes one of them from the set S. Otherwise, one of them was already removed from the set S and there is no need to remove one of them. The PureSet function continues checking the pairs and removing nodes from the set S as long as the weight of the set S is not less than r . When removing a node from the set S makes its weight less than r , the PureSet function stops the processing of the list NewDpaired . Finally, the PureSet function ends by returning the set S to the NetOpt algorithm. The pseudo-code of the PureSet function is shown in Algorithm 3. Algorithm 3: PureSet function Require: r , S and D pair ed compute an orders list N ewD pair ed = {(i, j)|i  = j, i ∈ S and j ∈ S} (from the set D pair ed ) while weight (S) > r do (i, j) ← next o f D pair ed if i ∈ S and j ∈ S then S ← S − { j} end if end while return S The details of the NetOpt algorithm are similar to the TreeOpt algorithm. The difference is that the NetOpt algorithm calls the PureSet function on the output of the FindSet function. Therefore, it finds the best candidate set from the outputs of the PureSet function rather than the FindSet function. The pseudo-code of the NetOpt algorithm is shown in Algorithm 4. The optimality of the NetOpt algorithm, by considering the triangle inequality, is outlined in the following. The Lemma 1 is used to show the optimality of the TreeOpt algorithm. When the network graph is not a tree, the output latency of FindSet(i, j) is not necessarily equal to ltnc(i, j) and the Lemma 1 is not held anymore. 123 4434 M. Malekimajd et al. Algorithm 4: NetOpt algorithm Require: r , D and {ltnc(u, v)| u ∈ D and v ∈ D} S←D for each (i, j) ∈ D × D(i  = j) do if ltnc(i, j) < ltnc(S) then tempS ← Find Set (i, j) tempS ← Pur eSet (tempS) if weight (tempS) ≥ r and ltnc(tempS) < ltnc(S) then S ← tempS end if end if end for return S Lemma 2 While the triangle inequality is satisfied for the latencies of all pairs in the set D, the latency of the output set of the FindSet(u, v) function is at most twice ltnc(u, v). Proof See the “Appendix”. Theorem 2 Let the NetOpt algorithm take a subset D of the graph nodes, the latencies between these nodes and the number r as input. If the triangle inequality is satisfied for the latencies between the nodes of D, the NetOpt algorithm finds a subset of nodes D with the weight of at least r (that its latency is at most twice of the latency of the optimal solution with the weight of at least r ). Proof Let u and v be two nodes of an optimal solution Sopt such that the ltnc(Sopt ) is equal to ltnc(u, v). Also, let Sout be the output of FindSet(u, v). The set Sout is a superset of the set Sopt because each member of the set Sopt stands the comparisons in FindSet(u, v). Consequently, the set Sout has the weight of at least r . In addition, by Lemma 2, the latency of the set Sout is at most twice ltnc(u, v) = ltnc(Sopt ). The PureSet function removes some nodes from the set Sout such that its weight stays greater than or equal to r while its latency does not increase. Therefore, the output of PureSet(u, v) has the latency of at most 2 × ltnc(Sopt ) and the weight of at least r . Since the NetOpt algorithm examines the output of PureSet(u, v) as a candidate output, this algorithm returns a subset with the latency of at most 2 × ltnc(Sopt ). In other words, the subset’s latency is at most twice of the latency of optimal solution. Time complexity of NetOpt The time complexity of NetOpt is divided into two main parts. The first part is the time complexity of computing a sorted list of latencies and the second part is the time complexity of calling subroutines FindSet and PureSet. Each of these subroutines is called at most n 2 times (let the cardinality of the set D be n). The NetOpt algorithm sorts the list Dpaired = {(i, j)|i = j, i ∈ D and j ∈ D} in a time complexity of O(n 2 log(n)). Similar to the time analysis of the TreeOpt algorithm, the time complexity of FindSet is O(n). In addition, PureSet can compute the sorted list NewDpaired in O(n 2 ) time by scanning the sorted list NewDpaired in linear time. In the worst case, the PureSet 123 Minimizing latency in geo-distributed clouds 4435 algorithm has to scan the sorted list completely and spends O(1) time for each member of the list NewDpaired . Hence, the time complexity of PureSet is O(n 2 ). By adding up these time complexities, O(n 2 log(n) + O(n 2 ) × O(n 2 )) time is obtained and the time complexity of NetOpt is O(n 4 ). Based on the simulation result, the run time of the NetOpt algorithm is much less than the time complexity in the worst case. Moreover, if the TreeOpt algorithm is used and the PureSet function is called on the output of TreeOpt algorithm, the time complexity remains O(n 3 ). Therefore, there is a trade-off between the time complexity and the chance of finding better candidates. 5.3 TreeOptExt algorithm In this section, we extend our proposed TreeOpt algorithm to the TreeOptExt algorithm. The TreeOptExt algorithm takes one more subset of leaves as input, in addition to the mentioned inputs of the TreeOpt algorithm. This subset is L = {l1 , l2 , . . . , lm }. The TreeOptExt algorithm considers this set while it examines the latency of a candidate set. Another difference to the treeOpt algorithm is that the TreeOptExt algorithm uses the FinsSetExt function rather than the FindSet function. The FindSetExt(u, v) function first checks all the members of L to see whether they stand the latency comparisons. Then, it checks for each node l ∈ L whether both ltnv(l, u) and ltnv(l, v) are less than ltnc(u, v) or not. If each member of the set L stands the comparisons, the rest of the FindSetExt(u, v) function works similar to FindSet(u, v). Otherwise, FindSetExt(u, v) returns NULL to TreeOptExt. The pseudo-code of the FindSetExt function is shown in Algorithm 5. Algorithm 5: FindSetExt algorithm Require: (u, v) , D , L and {ltnc(u, v)| u ∈ D and v ∈ D} T empSet ← {u, v} if there is an l ∈ L such that ltnc(l, u) > ltnc(u, v) or ltnc(l, v) > ltnc(u, v) then return N U L L end if for each d ∈ D do if ltnc(d, u) < ltnc(u, v) and ltnc(d, v) < ltnc(u, v) then T empSet ← T empSet ∪ {d} end if end for return T empSet By considering the TreeOpt algorithm, the TreeOptExt algorithm calls the FindSetExt function on the larger set {(i, j)|i ∈ D, j ∈ L} ∪ {(i, j)|i, j ∈ D}. However, the TreeOptExt algorithm is similar to the TreeOpt algorithm regarding other details. The pseudo-code of the TreeOptExt algorithm is shown in Algorithm 6. The TreeOptExt algorithm can be extended, similar to the extension of TreeOpt to NetOpt, to cover any graph topology as a latency graph. Therefore, the extended algorithm works for a network that its nodes are connected by the Internet. The details 123 4436 M. Malekimajd et al. Algorithm 6: TreeOptExt algorithm Require: r, L , D and {ltnc(u, v)| u ∈ D and v ∈ D ∪ L} S←D for each (i, j) ∈ {(i, j)|i ∈ D , j ∈ D ∪ L} do if ltnc(i, j) < ltnc(S) then tempS ← Find Set E xt (i, j) if weight (tempS) > r then S ← tempS end if end if end for return S of the extension of TreeOptExt algorithm are omitted because of similarity to extension of the TreeOpt algorithm. The optimality and time complexity of the TreeOptExt algorithm are similar to the TreeOpt algorithm. In the following, the Theorem 3 is provided to show the optimality of the TreeOptExt algorithm and its time complexity. Theorem 3 The TreeOptExt algorithm takes two subsets of tree leaves, the latency between these leaves and the number r as input. The TreeOptExt algorithm returns an optimal candidate set with the weight of at least r . The latency of this candidate in association with the set L is minimum, compared to other possible candidates. Proof Let the set S and the set Sopt be the output of TreeOpt and the optimal set, respectively. Also, let u 1 and u 2 be members of the set Sopt ∪ L, which ltnc(u 1 , u 2 ) = ltnc(Sopt ∪ L). The latencies between members of the set L are neither available nor used by the cloud manager. Therefore, the leaves u 1 and u 2 have three different membership possibilities as follows: u 1 ∈ D and u 2 ∈ D u 1 ∈ L and u 2 ∈ D u 1 ∈ D and u 2 ∈ L In any of above cases, FindOptExt(u 1 , u 2 ) is called. The comparisons in FindOptExt(u 1 , u 2 ) are stood for each member of Sopt because of the definition of Sopt ∪ L, the value of ltnc(Sopt ∪ L) and the comparisons that are made on l ∈ L. Therefore, the output of FindOptExt(u 1 , u 2 ) is a superset of Sopt , which has the weight of at least r and the latency less than ltnc(Sopt ). As a result, TreeOptExt examines a superset of Sopt and the output of TreeOptExt cannot be worse than Sopt with respect to latency. Time complexity of TreeOptExt Let m be the cardinality of the set L. In comparison with the FindSet function, a loop with order O(m) is added to the FindSetExt function. Besides, a loop with order O(mn) is added to the TreeOptExt algorithm compared to the TreeOpt algorithm. 123 Minimizing latency in geo-distributed clouds 4437 Therefore, the time complexity of the TreeOptExt algorithm is O((n + m)3 ). Because of O(m) ≤ O(n), it can be stated that the time complexity of the TreeOptExt is O(n 3 ) too. 6 Evaluation In this section, the performance of the NetOpt algorithm is evaluated. This algorithm is used to find a set of data-centers for hosting the requested VMs. The NetOpt algorithm solves the problem when the network is modeled by a graph. This algorithm helps to minimize the latency of VM allocation since the network latency of the hosting datacenters has significant impact on the performance of resources allocated to a user. For the purpose of evaluation, we compare our algorithm to the proposed algorithm MinDiameterGraph in [4] and an algorithm for random selection. Hence, we implemented the three algorithms by high level programming languages. The random selection algorithm iteratively opts a random data-center from the set of all data-centers and adds it to a desired set. This algorithm continues selection process until the free capacity of the desired set of data-centers becomes at least the number of the requested VMs. At last, the random selection algorithm returns the desired set with weight at least the number of the requested VMs. This set is a candidate set of data-centers for hosting the requested VMs. To have a comprehensive evaluation, we run experiments on both real latencies adopted from the Internet and randomly generated values. In the following, we outline three scenarios that we used to compare these algorithms. It is worthwhile to remind that Alicherry et al. [4] assumed that the triangle inequality is implied on communication latencies. Also, they assumed that for any three nodes that violate the triangle inequality, the violation is fixed by forcing the longer edge to take a path formed by the shorter edges. Hence, we take into account both establishing and not establishing the triangle inequality for latency. In the first scenario, which is taken from Ref. [4], users request 1000 VMs and the number of data-centers takes 5 different values among 10, 25, 50, 75 and 100. In the case of 100 data-center, the number of VMs per data-center is chosen uniformly random between 50 and 100. The upper and lower bounds for the number of VMs per data-center are relatively scaled in other number of data-centers. The locations of data-centers are randomly selected from a 1000 × 1000 grid and the latency between two data-centers is the Euclidean distance between them. Therefore, the latencies are randomly generated and the triangle inequality is applied to the latencies. For the first scenario, we run 100 experiments for each number of data-centers. In each case, the result of each algorithm is a set of data-centers that its latency is matter in our simulation. The average and standard deviation of the output latency of running each algorithm 100 times is illustrated in Fig. 1. In this figure, our algorithm, the algorithm in [4] and the random one are showed by labels Proposed, Approx and Random, respectively. In all cases, our algorithm has better latency, although it is negligible. By considering the result of the experiment in Fig. 1, we can conclude that our algorithm works similar or better than the algorithm in [4] while the latencies are random values and the triangle inequality is applied to them. Also, the result of this 123 4438 M. Malekimajd et al. Fig. 1 Latency of the placement for a request of 1000 VMs scenario can verify the implementation of algorithm in [4], because this scenario is taken from Ref. [4]. In the second scenario, there are 200 data-centers such that each data-center’s capacity is equal to one and the cardinality of requests varies from 2 to 92 VMs (by steps of 10). The response time between 200 distributed PlaneLab nodes across the Internet [36] is adopted as communication latencies between 200 data-centers. Zhu et al. [36] used the response times to verify the accuracy of predicting response time with corresponding Euclidean distances. However, the triangle inequality does not hold for these latencies. We consider two different inputs. The (unchanged) data from [36] are the latencies in one of the inputs, which we call it the Pure input. Moreover, we call another one the Advance input. In the Advance input, the latencies are modified in a way that triangle inequality is applied to them (the network graph of data-centers is considered). The latency between every two data-centers is changed regarding to the lowest latency path between them. The output of each algorithm is a set of data-centers that its latency matters. Therefore, the result of each scenario is the latency of the output set. Figure 2 illustrates the results of this scenario while the y-axis (latency of output set) is presented on logarithmic scale. A-Proposed and P-Proposed show the results of our algorithm for the Advance input and the Pure input, respectively. Additionally, A-Approx and P-Approx show the results of the algorithm in [4] for the Advance input and the Pure input, respectively. We do not report the output of the random algorithm in Fig. 2, because it is far worse than the others. In this scenario, the latencies are close to Euclidean distances, although based on the results, we can conclude that our algorithm works better or similar to other algorithms. However, we use the next scenario to show the advantage of our algorithm in a large-scale network over the Internet. Third scenario is obtained by changing the second one such that cloud infrastructure includes 2500 geo-distributed data-centers. The latencies between these data-centers are adopted from Ref. [32], which the Internet latencies are calculated by considering 123 Minimizing latency in geo-distributed clouds 4439 Fig. 2 The communication latency of VM allocation in 200 geo-distributed data-centers (note: latency is presented in logarithmic scale) Fig. 3 The communication latency of VM allocation in 2500 geo-distributed data-centers (note: latency is presented in logarithmic scale) a large-scale distributed system. These real-world data are the most suitable one for verifying our algorithm because of its scale, distribution and realness. The result of this scenario is illustrated in Fig. 3 while the y-axis (latency of output set) is presented on logarithmic scale. The scenarios show that our algorithm finds a set of data-centers with a latency far less than the latency of the algorithm in [4] or the random selection. A large-scale geodistributed system is assumed in the third scenario with latencies adopted from real communications in the Internet. Our major concern is when the latencies are real data from the Internet. From third scenario, we can conclude that our algorithm finds a far better solution with respect to latency when the number of data-centers is increased. In this scenario, there is an experiment that our algorithm can reduce the communication latency up to 92 % compared to the proposed algorithm in [4]. 123 4440 M. Malekimajd et al. (a) (b) Fig. 4 The average run time of an experiment in third scenario. a The triangle inequality is satisfied. b The triangle inequality is not necessarily satisfied Let us consider Fig. 3. The results of A-Proposed and A-Approx are close and small, although these results are not provided to show a candidate latency. Actually, the results of A-Proposed and A-Approx in comparison to P-Proposed and P-Approx show that the modified data are not useful to select a set of data-centers. Another important result is that the algorithm [4] is not suitable in large-scale geo-distributed systems such that it can have a result worse than Random. The performance of the selected set of data-centers and the time complexity of our algorithms are illustrated in this section and Sect. 5, respectively. The run time of the third scenario is shown in Fig. 4. We omit the run time of other scenarios, since they are short. As Fig. 4a shows, the run time of our algorithm is low for latencies that dependent to Euclidean distance. Moreover, as Fig. 4b shows, the run time of our algorithm is proportional to the number of geo-distributed data-centers that connected through the Internet. It is worthwhile to mention that the number of requested VMs can be huge in proportion to the number of hosting data-centers; however, the capacity 123 Minimizing latency in geo-distributed clouds 4441 of each data-center is assumed to be one. Hence, our algorithm works in reasonable and low time even in the case of requesting a large number of VMs. 7 Conclusion and future work A solid resource allocation is an essential part of the cloud management, especially in geo-distributed clouds. Lots of small data-centers with diverse geographical locations construct a geo-distributed cloud. Our goal is to find a set of geo-distributed datacenters to host the requested communicating VMs while the free capacity and the communication latency of selected data-centers are a matter of importance. Our proposed algorithm for finding a set of data-centers in geo-distributed cloud is inspired by the treeness of the Internet latency. The proper performance of our proposed algorithm is due to the approximate treeness of the latency of the Internet topology. Moreover, our proposed algorithm finds a latency optimal set of data-centers in geo-distributed data-centers with tree networks. As a future work, one can extend our algorithm to handle more complicated user requests. For example, a user can state which VMs do not need to communicate with each other. Appendix Proof of Lemma 1: Proof Let Findset(u, v) return the set S ⊂ D. First, we prove that the latency of each two members of the set S is less than or equal to ltnc(u, v). Second, we prove that the set S is maximal. Bound on the latency To prove a bound on the latency of the set S, it is enough to verify the latency between two optional leaves of the set S. It is clear from the definition of the set D that all members of the set S are leaves of a given tree. Since the latency and distance between two nodes are equal in a tree, it suffices to verify the distance, rather than their latency, between two arbitrary nodes a and b of the set S. Let us consider the nodes a, b, u, v ∈ S and the paths between them. The tree paths a–u, a–v and u–v have at least a common node. Because if these three paths do not meet at a common node, they build a cycle in the graph and contradict the assumption that the given graph is a tree. In the same way, three paths b–u, b–v and u–v have at least a common node. Let t1 be a common node on paths a–u, a–v and u–v and t2 be a common node on paths b–u, b–v and u–v. Since there is a unique path between any two distinct nodes on the tree, the path a–u can split in two disjoint paths a–t1 and t1 –u such that the following equation on their distances is obtained: ltnc(a, u) = ltnc(a, t1 ) + ltnc(t1 , u) (1) Also, we have the following equations for other mentioned paths: ltnc(v, u) = ltnc(v, t1 ) + ltnc(t1 , u) (2) 123 4442 M. Malekimajd et al. ltnc(b, u) = ltnc(b, t2 ) + ltnc(t2 , u) (3) ltnc(v, u) = ltnc(v, t2 ) + ltnc(t2 , u) (4) In addition, we have the following equations from the comparisons made in the FindSet function: ltnc(a, u) ≤ ltnc(u, v) ltnc(a, v) ≤ ltnc(u, v) (5) (6) ltnc(b, u) ≤ ltnc(u, v) ltnc(b, v) ≤ ltnc(u, v) (7) (8) By knowing ltnc(i, j) equals to ltnc( j, i) and by substituting Eqs. (1) and (2) into Eq. (5), the following equation is obtained: ltnc(a, u) ≤ ltnc(u, v) ⇒ ltnc(a, t1 ) + ltnc(t1 , u) ≤ ltnc(v, t1 ) + ltnc(t1 , u) ⇒ ltnc(a, t1 ) ≤ ltnc(v, t1 ) (9) Again, by substituting Eqs. (3) and (4) in Eq. (7), the following equation is obtained: ltnc(b, t2 ) ≤ ltnc(u, t2 ) (10) We assume in the path from u to v, first we visit t1 and second we visit t2 . If it is vice versa, it is enough to rewrite the proof by changing the notations. The following equation is established by considering the unique path from u to v in the tree: ltnc(v, u) = ltnc(v, t1 ) + ltnc(t1 , t2 ) + ltnc(t2 , u) (11) Adding up Eqs. (9)–(11) leads to ltnc(v, u) ≥ ltnc(a, t1 ) + ltnc(t1 , t2 ) + ltnc(t2 , b) (12) Following equation is derived for latencies between the nodes a , b , t1 and t2 : ltnc(a, t1 ) + ltnc(t1 , t2 ) + ltnc(t2 , b) ≥ ltnc(a, b) (13) Finally, ltnc(v, u) ≥ ltnc(a, b) is concluded from Eqs. (12) and (13). Because we assume that a and b are two arbitrary nodes of the set S, we can conclude that the distance between any two nodes of the set S is less than or equal to ltnc(u, v). Consequently, ltnc(S) equals ltnc(u, v). 123 Minimizing latency in geo-distributed clouds 4443 Maximality Let the node d be a member of the set D, which is not in the set S (d ∈ /S and d ∈ D). According to the definition of the FindSet function, at least one of the inequalities ltnc(d, u) > ltnc(u, v) or ltnc(d, v) > ltnc(u, v) is satisfied. Therefore, if the node d is added to the set S, the latency of the set S is increased and is not equal to ltnc(u, v) anymore. Since each optional node d ∈ / S cannot be added to the set S, it is a maximal set. Proof of Lemma 2: Proof Let the set S be the output of the FindSet(u, v) function. Suppose on the contrary that n 1 and n 2 are two members of the set S such that ltnc(n 1 , n 2 ) > 2 × ltnc(u, v) (14) Based on the comparisons were made in FindSet(u, v): ltnc(n 1 , u) ≤ ltnc(u, v) ltnc(n 2 , u) ≤ ltnc(u, v) (15) (16) In addition, the triangle inequality between latencies states that ltnc(n 1 , n 2 ) ≤ ltnc(n 1 , u) + ltnc(n 2 , u) (17) Adding up Eqs. (15), (16) and (17) leads to ltnc(n 1 , n 2 ) ≤ 2 × ltnc(u, v) (18) Overall, the output latency of FindSet(u, v) cannot be more than 2×ltnc(u, v) because Eq. (18) contradicts the Eq. (14). References 1. Abraham I, Balakrishnan M, Kuhn F, Malkhi D, RamasubramanianV, Talwar K (2007) Reconstructing approximate tree metrics. In: The26th annual ACM symposium on principles of distributed computing, PODC 2007, Portland, 12–15 August 2007, pp 43–52 2. Aceto G, Botta A, De Donato W, Pescapè A (2013) Cloud monitoring: a survey. Comput Netw 57(9):2093–2115 3. Agarwal S, Dunagan J, Jain N, Saroiu S, Wolman A (2010) Volley: automated data placement for geo-distributed cloud services. In: The 7thUSENIX symposium on networked systems design and implementation,NSDI 2010, 28–30 April 2010, USENIX Association, San Jose,pp 17–32 4. Alicherry M, Lakshman TV (2012) Network aware resource allocation indistributed clouds. In: The IEEE INFOCOM 2012, Orlando, 25–30 March 2012, pp 963–971 5. Armbrust M, Fox A, Griffith R, Joseph AD, Katz RH, Konwinski A, LeeG, Patterson DA, Rabkin A, Stoica I, Zaharia M (2010) A view of cloudcomputing. Commun ACM 53(4):50–58 6. Beloglazov A, Abawajy JH, Buyya R (2012) Energy-aware resource allocation heuristics for efficient management of data centers for cloud computing. Futur Gener Comput Syst 28(5):755–768 7. Calero JMA, Aguado JG (2015) Comparative analysis of architectures for monitoring cloud computing infrastructures. Futur Gener Comput Syst 47:16–30 123 4444 M. Malekimajd et al. 8. Chen K, Xu Y, Xi K, Chao HJ (2013) Intelligent virtual machine placement for cost efficiency in geo-distributed cloud systems. In: The IEEE international conference on communications, ICC 2013, IEEE, Budapest, 9–13 June 2013, pp 3498–3503 9. Chunlin L, Layuan L (2014) Multi-layer resource management in cloud computing. J Netw Syst Manag 22(1):100–120 10. Data center global expansion trend (2012) http://www.datacenterknowledge.com/archives/2012/05/ 15/google-data-center-faq/. Accessed 16 Oct 2015 11. Dhingra A, Paul S (2014) Green cloud: heuristic based BFO technique to optimize resource allocation. Indian J Sci Technol 7(5):685–691 12. Endo PT, de Almeida Palhares AV, Pereira NCVN, Gonçalves GE, Sadok D, Kelner J, Melander B, Mångs JE (2011) Resource allocation for distributed cloud: concepts and research challenges. IEEE Netw 25(4):42–46 13. Eriksson B, Crovella M (2013) Understanding geolocation accuracy using network geometry. In: The IEEE INFOCOM 2013, Turin, 14–19 April 2013, pp 75–79 14. Eriksson B, Dasarathy G, Barford P, Nowak R (2012) Efficient network tomography for internet topology discovery. IEEE/ACM Trans Netw 20(3):931–943 15. Gonçalves GE, Santos MA, Charamba G, Endo PT, Sadok D, Kelner J, Melander B, Mångs J (2012) D-CRAS: distributed cloud resource allocation system. In: The 2012 IEEE network operations and management symposium, IEEE, Maui, 16–20 April 2012, pp 659–662 16. Greenberg A, Hamilton J, Maltz DA, Patel P (2008) The cost of a cloud: research problems in data center networks. ACM SIGCOMM Comput Commun Rev 39(1):68–73 17. Gupta A, Kalé LV, Milojicic D, Faraboschi P, Balle SM (2013) HPC-aware VM placement in infrastructure clouds. In: 2013 IEEE international conference on cloud engineering (IC2E). IEEE, New York, pp 11–20 18. Han Y, Chan J, Alpcan T, Leckie C (2014) Virtual machine allocation policies against co-resident attacks in cloud computing. In: The IEEE international conference on communications, ICC 2014, Sydney, 10–14 June 2014, pp 786–792 19. Jennings B, Stadler R (2015) Resource management in clouds: survey and research challenges. J Netw Syst Manag 23(3):567–619. doi:10.1007/s10922-014-9307-7 20. Jiao L, Li J, Xu T, Fu X (2012) Cost optimization for online social networks on geo-distributed clouds. In: Proceedings of the 20th IEEE international conference on network protocols, ICNP 2012, IEEE, Austin, 30 October–2 November 2012, pp 1–10 21. Kesavan M, Ahmad I, Krieger O, Soundararajan R, Gavrilovska A, Schwan K (2013) Practical compute capacity management for virtualized datacenters. IEEE Trans Cloud Comput 1(1):1–14 22. Kuo J, Yang H, Tsai M (2014) Optimal approximation algorithm of virtual machine placement for data latency minimization in cloud systems. In: The IEEE INFOCOM 2014, Toronto, 27 April–2 May 2014, pp 1303–1311 23. Li J, Li D, Zheng J, Quan Y (2014) Location-aware multi-user resource allocation in distributed clouds. In: Wu J, Chen H, Wang X (eds) Advanced computer architecture. Communications in computer and information science, vol 451. Springer, Berlin, pp 152–162 24. Mell P, Grance T (2011) The nist definition of cloud computing. In: Technical report 800-145, National Institute of Standards and Technology (NIST), Gaithersburg. http://csrc.nist.gov/publications/nistpubs/ 800-145/SP800-145.pdf. Accessed 16 Oct 2015 25. Meng X, Pappas V, Zhang L (2010) Improving the scalability of data center networks with trafficaware virtual machine placement. In: The IEEE INFOCOM 2010, San Diego, 15–19 March 2010, pp 1154–1162 26. de Oliveira F, Ledoux T, Sharrock R (2013) A framework for the coordination of multiple autonomic managers in cloud environments. In: 2013 IEEE 7th international conference on self-adaptive and self-organizing systems (SASO), pp 179–188 27. Papagianni CA, Leivadeas A, Papavassiliou S, Maglaris V, Cervello-Pastor C, Monje Á (2013) On the optimal allocation of virtual resources in cloud computing networks. IEEE Trans Comput 62(6):1060– 1071 28. Ramasubramanian V, Malkhi D, Kuhn F, Balakrishnan M, Gupta A, Akella A (2009) On the treeness of internet latency and bandwidth. In: The 11th international joint conference on measurement and modeling of computer systems. ACM, Seattle, WA, pp 61–72 123 Minimizing latency in geo-distributed clouds 4445 29. Ruan L, Peng J, Xiao L, Zhu M (2013) Distributed virtual machine monitor for distributed cloud computing nodes integration. In: Park J, Arabnia H, Kim C, Shi W, Gil JM (eds) Grid and pervasive computing. Lecture notes in computer science, vol 7861. Springer, Berlin, pp 23–31 30. Rygielski P, Kounev S (2013) Network virtualization for QoS-aware resource management in cloud data centers: a survey. Praxis Informationsverarbeitung Kommun 36(1):55–64 31. Valancius V, Laoutaris N, Massoulié L, Diot C, Rodriguez P (2009) Greening the internet with nano data centers. In: The 2009 ACM conference on emerging networking experiments and technology, CoNEXT 2009, Rome, 1–4 December 2009, pp 37–48 32. Wong B, Slivkins A, Sirer EG (2005) Meridian: a lightweight network location service without virtual coordinates. In: The ACM SIGCOMM 2005 conference on applications, technologies, architectures, and protocols for computer communications, Philadelphia, 22–26 August 2005. ACM, New York, pp 85–96 33. Yao Y, Cao J, Li M (2013) Network and parallel computing. In: Lecture notes in computer science, vol 8147. Springer, Berlin, pp 71–82 34. Yi X, Liu F, Liu J, Jin H (2014) Building a network highway for big data: architecture and challenges. IEEE Netw 28(4):5–13 35. Zhang Q, Cheng L, Boutaba R (2010) Cloud computing: state-of-the-art and research challenges. J Internet Serv Appl 1(1):7–18 36. Zhu J, Kang Y, Zheng Z, Lyu MR (2012) WSP: a network coordinate based web service positioning framework for response time prediction. In: The 2012 IEEE 19th international conference on web services, IEEE, Honolulu, 24–29 June 2012, pp 90–97 123