Jointly Optimizing Job Assignment and Resource Partitioning for Improving System Throughput in Cloud Datacenters

The job placement problem has been extensively studied in the literature. Existing solutions can roughly be divided into two categories: bin packing based methods and contention-aware methods.

Bin Packing based Methods. These kinds of methods describe each job by a resource demand vector, and aim to pack the jobs among the servers such that: (1) the number of servers used is minimized or load balance is achieved, and (2) the total resource consumption of each server does not exceed server capacity [18, 19, 33, 37]. However, bin packing based methods have two drawbacks. First, they generally assume a fixed resource demand for each job, which cannot adapt to dynamic workload changes. Second, they do not consider resource fungibility (a job can have the same performance with different resource allocations), thus they cannot utilize the resource complementarity among different jobs to improve server utilization.

Contention-aware Methods. These kinds of methods strive to place jobs among servers such that the performance interference among the colocated jobs is minimized [11, 23, 30]. They generally build a prediction model to estimate performance interference among the colocated jobs, and use heuristic algorithms to search a near-optimal job placement configuration that minimizes the total performance interference. However, existing contention-aware methods generally assume the server resources are fully shared among the colocated jobs (i.e., resource partitioning is not considered), thus performance interference still exists.

Summary: Existing job placement works either ignore resource partitioning or maintain a static and suboptimal partitioning configuration.

Existing resource partitioning solutions can be divided into two categories: performance model based methods and online tuning based methods.

Performance Model based Methods. These kinds of methods generally build a performance model to evaluation partitioning configurations, and use heuristic algorithms to find the near-optimal partitioning configuration based on the performance model [14, 27, 34, 36]. However, such methods can only cope with single resource and cannot easily be applied to multiple resources. This is because: (1) building performance model is difficult for multiple resources, since the contention behavior becomes much more complex; and (2) the solution space grows exponentially when multiple resources are considered coordinately, thus using heuristic algorithm to explore the solution space is not feasible.

Online Tuning based Methods. These kinds of methods dynamically tune the partitioning configuration in an online manner according to the feedback from a real system [9, 12, 22, 28, 29]. Such methods can partition multiple resources coordinately. However, due to the lack of a precise performance model, feedback based methods have to evaluate partitioning configurations in real systems. That is rather inefficient when exploring a large solution space.

Summary: None of the existing resource partitioning solutions can achieve optimality, efficiency, and robustness simultaneously.

Given a colocation of jobs, the colocation evaluating model is to predict the performance of each job under a specific resource partitioning configuration. Note that the performances of jobs may vary dynamically over time due to workload changes. To simplify the problem, the prediction model estimates the time-averaged performance of each job, instead of considering the performance at every time point. With this model, we can roughly estimate the system throughput under any given job assignment configuration and resource partitioning configuration without evaluating them in a real system.

However, developing a precise prediction model is challenging, because the relationship between job performance and resource partitioning configuration is very complex. To address this issue, we leverage deep learning technology to build the evaluating model. In our context, the input features of the learning model include the features of jobs and the resource partitioning configuration. The features of a job consists of the performance counters of the job when it runs alone on the servers and the number of threads that it has. We use performance counters as the main features of jobs, because they are able to characterize the resource sensitivities of jobs effectively and can easily be collected without incurring too much overhead [4]. Since the performance counters change dynamically as the workload varies, we use the time-averaged values of the performance counters over the profiling period. This is reasonable because: (1) we are concerned with the average behaviors of jobs, and the average performance counters are sufficient to ensure a good estimation (validated by the results in Table 5); and (2) we want to keep a small overhead, thus we try to use short input features for the machine learning model. Modern processors provide hundreds of performance counters, but not all of them are useful in our case. To pick up the most important ones for reducing the input dimension, we use a random forest model to select the top important performance counters (see details in Section 5.2). We include the number of threads of each job in the features, because multi-threaded jobs are sensitive to the number of CPU cores allocated.

Consider a colocation \(c_i\) (i.e., the set of jobs assigned to the i-th server). Let \(|c_i|\) denote the number of jobs in \(c_i\) . For each \(1\le j\le |c_i|\) , let \(pmc_{i}^j\) denotes the time-averaged performance counters associated with the j-th job when it runs alone on the server. Let \(d_{i}^j\) denotes the number of threads of the j-th job. Let \(PMC_i=\lbrace pmc_{i}^1, pmc_{i}^2, \ldots , pmc_{i}^{|c_i|}\rbrace\) and \(D_i=\lbrace d_{i}^1, d_{i}^2, \ldots , d_{i}^{|c_i|}\rbrace\) . Let \(IPC_{i}^j\) denote the time-averaged IPC of the j-th job in \(c_i\) . The prediction model aims to predict \((IPC_{i}^1,IPC_{i}^2,\ldots ,IPC_i^{|c_i|})\) with the input \((PMC_i, D_i, p_i)\) .

The design of the colocation evaluating model is shown in Figure 2. Briefly, two fully-connected (fc) layers are used to encode the input features, and a padding mask is used to align the input vector to a fixed length for handling different colocation sizes. A multi-head attention module is used to synthesize the outputs of the fc layers. Multi-head attention [32] is a powerful mechanism to learn the dependencies among input features, which has been widely used in a variety of tasks including reading comprehension, abstractive summarization, and so on. The multi-head attention module is followed by another two fc layers, which generates the final output, i.e., the IPCs of the jobs.

As mentioned earlier, the solution space of the job assignment problem is huge. Therefore, we propose a local search based heuristic algorithm to find the near-optimal solution. The details are presented in Algorithm 1. There are two steps: in the first step (line 2 to 12), a greedy algorithm is used to quickly find an initial solution; in the second step (line 14 to 23), a local search process is conducted to improve the solution.

The basic idea of the greedy algorithm is to assign the jobs that are not likely to incur resource contention into the same colocation. Intuitively, the jobs with similar resource sensitivities are more likely to cause resource contention, because their resource demands are similar. As we use performance counters to characterize resource sensitivities, we define the similarity of resource sensitivities between two jobs as the “distance” between the performance counters of the two jobs, where the distance is defined as the Eular distance between the two vectors representing the performance counters of the two jobs. A smaller distance indicates a high similarity, and vice versa. The greedy algorithm attempts to divide the jobs into \(N_{ser}\) colocations (one for each server), and sequentially determines the jobs for each colocation. To balance the jobs among colocations, there are \(\lfloor N_{job}/N_{ser} \rfloor + 1\) jobs is each of the first \(N_{job}\%N_{ser}\) colocations, and \(\lfloor N_{job}/N_{ser} \rfloor\) jobs for each of the rest colocations. To determine the jobs for a colocation, we first randomly select a job among those that are not assigned, then, we iteratively choose the jobs until a sufficient number of jobs are selected. In each iteration, we choose the job such that: (1) the job has not been assigned; and (2) the total similarity between the job and the already selected jobs is the minimum, with ties broken randomly.

After the initial solution is obtained, we conduct a local search process to improve it. The local search process iterates for a number of rounds, and performs two local search operations in each round. The first operation is shift, which attempts to move one job from one colocation to another one. We first randomly select a colocation among the top \(x\%\) (a tunable parameter) colocations with the lowest average throughput, and randomly choose a job in the selected colocation. After that, we randomly select a target colocation among the top \(x\%\) colocations with the highest average throughput. The shift operation is accepted if the system throughput is improved after the operation is performed, and rejected otherwise. The system throughput refers to the total throughput of all the colocations, where the throughput of a colocation is defined as the total IPCs of the jobs in the colocation under the optimal partitioning configuration (i.e., the partitioning configuration that produces the maximum throughput). The second operation is swap, which attempts to exchange two jobs in two colocations, where the two jobs are randomly selected from the top \(x\%\) colocations with the lowest average throughput. Again, the operation is accepted if the system throughput is improved after it is performed, and rejected otherwise. The local search process terminates after \(rn_1\) rounds, where \(rn_1\) can be tuned according to practical requirements.

In order to estimate the throughput of a colocation, we need to find the optimal partitioning configuration for the colocation. Searching for the optimal solution is time consuming, thus we use a hill climbing algorithm to find the approximate solution. The details are presented in Algorithm 2. Initially, we assume the resources are equally partitioned among the jobs in the colocation (line 1). Then, the algorithm runs for \(rn_2\) rounds (a tunable parameter). In each round, for each resource \(r\in \mathcal {R}\) , we randomly select two jobs and attempt to shift one unit of r from one job to the other job. If the throughput is improved, we accept this operation (line 8 to 10), and reject otherwise.

Complexity. Since we only need to store the knowledge of the jobs and servers, the space complexity of the job assignment algorithm is \(O(N_{job} + N_{ser})\) . For the time complexity, the hill climbing algorithm has a complexity of \(O(rn_2|\mathcal {R}|)\) . For the job assignment algorithm, the first step (i.e., the greedy algorithm) has a complexity of \(O(N_{ser}^2)\) . The mean size of colocations is around \(N_{job}/N_{ser}\) . Therefore, the second step of the job assignment algorithm (i.e., the local search algorithm) has a complexity of \(O(rn_1(N_{job}/N_{ser})^2 x\%)\) . Therefore, the overall time complexity of the job assignment algorithm is \(O(N_{ser}^2 + rn_1 rn_2 |\mathcal {R}| (N_{job}/N_{ser})^2 x\%\) .

Reinforcement Learning (RL) is a kind of approach in which an agent interacts with the environment and learns an optimal policy by trial and error. At each time step t, the agent receives a state \(S_t\) and selects an action \(A_t\) according to a policy \(\pi (A_t|S_t)\) . The policy is the strategy that the agent employs to determine the next action based on the current state, which is a mapping from state \(S_t\) to actions \(A_t\) . After the action is performed, the agent receives a scalar reward \(w_t\) , and the environment transitions to the next state \(S_{t+1}\) . The reward is the feedback reflecting how good is the action \(A_t\) in the state \(S_t\) (a larger reward indicates a better action). This process continues until the agent finds the optimal policy, which maximizes a discounted and accumulated reward \(\sum _{t=0}^{\infty } \gamma ^t w_{t}\) ( \(\gamma \in (0,1]\) ) is the discount factor). Finding the optimal policy is crucial for RL, but is not an easy task. A recently popular approach is to use deep neural network (DNN) to approximate the policy, which we call Deep Reinforcement Learning (DRL). This is because DNN has strong fitting ability to establish the mapping from input to output without relying on domain knowledge.

We would like to say that DRL is suitable for resource partitioning. Once the DRL model is well-trained, the partitioning decision can be made very quickly, because the inference of neural network is fast. A well-trained neural network has natural generalization to the inputs which have not appeared in the training scenario, in our context, it can adapt to unseen workloads, colocations and jobs, making it highly robust to dynamic system state changes.

A DRL model consists of the following concepts: environment, action, state, reward, and policy. In the context of resource partitioning, the concepts are defined as follows:

Environment. The DRL model works on a single server, for partitioning the shared resources among the jobs colocated on the server. So, we define the server as the environment.

State. The state is defined as the server state in our context, which is also the input of the DRL model. How to define the server state is non-trivial, because (1) the server state must be able to reflect the workload characteristics of jobs and resource contention behaviors across jobs; and (2) the server state must be easy to collect since the resource partitioning decision should be generated quickly. The server state in our DRL model is composed by three parts: performance counters associated with jobs, the current partitioning configuration, and the number of threads of jobs. We include the number of threads of each job in the system state, because multi-threaded jobs are sensitive to the number of CPU cores allocated.

Consider the i-th server in the datacenter. Denote by \(c_i\) the set of jobs colocated on the i-th server. We use \(PMC_{i,t}=(pmc_{i,t}^1, \ldots , pmc_{i,t}^{|c_i|})\) to denote the performance counters for the jobs in \(c_i\) at time step t, where \(pmc_{i,t}^j\) refers to the performance counters associated with the j-th job. We use \(p_{i,t}=\lbrace p_{i,t}^1,\ldots , p_{i,t}^{|c|}\rbrace\) to denote the resource partitioning configuration of \(c_i\) at time step t, where \(p_{i,t}^j\) refers to the allocation of resources for the j-th job. So, the server state at time step t can be represented by \(S_{i,t} = \lbrace PMC_{i,t}, D_i, p_{i,t}\rbrace\) , where \(D_i\) denotes the numbers of threads of jobs.

Action. The action is the output of DRL model, which refers to the resource partitioning decision in our context. For the i-the server, we use \(A_{i,t} = \lbrace a_{i,t}^1, \ldots , a_{i,t}^{|c|}\rbrace\) to denote the action at time step t, where \(a_{i,t}^j\) refers to the resource allocation for the j-th job.

Reward. The goal of resource partitioning is to maximize the system throughput. Let \(S_{i,t}\) denote state of the i-th server at time t. Suppose an action \(A_{i,t}\) (i.e., a new resource partitioning configuration) produced by the DRL model is applied to the i-th server at time t. Let \(S_{i,t+1}\) denote the state of the i-th server after \(A_{i,t}\) is applied. The reward \(w_{i,t}\) of the DRL model for \(A_{i,t}\) is defined as the server throughput in state \(S_{i,t+1}\) .

Policy. The DRL method uses DNN to approximate the policy model. The policy model maps the input (i.e., the server state) to the output (i.e., the partitioning decision). Figure 3 shows the design of the DRL policy model. Overall, the policy model employs an encoder-decoder architecture. The encoder first transforms the input to a high-dimensional vector through a stack of fully connected (fc) layers. Then, the transformed features are fed to the decoder network through an attention layer. The decoder network adopts the Gated Recurrent Unit (GRU) to generate the partitioning decision. The GRU architecture is commonly used for sequential decision. In our context, it determines the resource allocation for the jobs one by one in a sequential manner.

DRL typically requires a large number of interactions between agent and environment to train the model. In the context of resource partitioning, a single interaction may take over 10 seconds, because we need to generate a new partitioning configuration according to the partially-trained DRL model, apply the new generated configuration to real system, observe the reward, and finally update the DRL model using the reward. Since the DRL model needs to be trained by tens of millions of interactions, the total training time is about \(O(10^5)\) hours, which is not feasible even if the training process is one-shot. So, a crucial issue of applying DRL to resource partitioning is how to reduce the training overhead.

To address this issue, we propose a prediction model to estimate the reward of a given action, without interacting with the real system. Suppose the i-th server is in state \(S_i = \lbrace PMC_i, D_i, p_i\rbrace\) at some time point, where \(PMC_i\) denotes the performance counters of jobs, \(p_i\) denotes the current resource partitioning configuration, and \(D_i\) denotes the numbers of threads of jobs. Given an action \(A_i\) (i.e., a resource partitioning decision), let \(IPC_{i}^j\) denote the IPC of the j-th job in \(c_i\) if \(A_i\) is applied to \(S_i\) . The prediction model takes \((S_i, A_i)\) as input to estimate \((IPC_i^1, IPC_{i}^2,\ldots , IPC_{i}^{|c_i|})\) , without applying \(A_i\) to the real system. The throughput of the colocation is defined as the total IPC of all the jobs, which can be computed by \(\sum _{j=1}^{|c_i|} IPC_{i}^j\) .

We leverage DNN to build the resource partitioning reward prediction model. Briefly, an fc layer is used to encode the input features at the current time step, and a multi-head attention module is used to synthesize the outputs of the fc layer. The multi-head attention module is followed by another fc layer, which generates the final output, i.e., the IPCs of the jobs at the next time step.

Training the reward prediction model still needs to interact with real systems to collect training samples. However, as the number of training samples required for training the prediction model is much less than that for training the DRL model, the prediction model can significantly reduce the training time of the DRL model (see details in Section 5.5.7).

We first show the performance of the prediction models, including the colocation evaluating model and the DRL reward prediction model. We take the mean accuracy and RMSE (root-mean-square error) as metrics to evaluate the accuracy of the placement and resource partitioning prediction model. The mean accuracy is defined by \(\frac{1}{n}\sum _{i=1}^n |y_i-y^{\prime }_i|/y_i\) , where \(y_i\) is the actual observed value and \(y^{\prime }_i\) is the predicted value. RMSE is a frequently used measure of the variance of prediction errors, which is defined by \(\sqrt {\frac{1}{n}\sum _{i=1}^n (y_i-y\prime _i)^2}\) . Generally, a smaller RMSE indicates a higher prediction accuracy.

The results for the colocation evaluating model and the DRL reward prediction model are shown in Tables 5 and 6, respectively. As can be seen, the prediction accuracy is very high (over 90%) for both of the two models over different colocation sizes. We also observe that the prediction accuracy for colocations of 4-jobs and 7-jobs is slightly worse than that for colocations of 5-jobs and 6-jobs for both of the two models. This is because: (1) the difficulty of prediction would increase as the colocation size grows, due to a higher input dimension and more complex relationship; and (2) an individual job in a smaller colocation has more resource allocation configurations and thus a more diverse performance, which also makes the prediction more difficult.

We next show how we determine the number of rounds (i.e., the parameter \(rn_2\) ) that the hill climbing algorithm (Algorithm 2) runs for finding the optimal partitioning configuration. We randomly generate 360 colocations with sizes ranging from 4-jobs to 7-jobs. Then, we run Algorithm 2 for each colocation with different numbers of rounds, and compute the mean throughput of the colocations for each colocation size for each number of rounds. Figure 5(a) shows the results. We have several observations. First, the throughput grows as the number of rounds increases for all colocation sizes, because using more iterations has more chance to find a better partitioning configuration. Second, the algorithm converges slightly slowly as the colocation size increases. This is because the solution space grows as the colocation size increases, while exploring a larger solution space is more difficult. Third, the algorithm converges at 250 rounds for all the colocation sizes. Based on the results, we set the number at 250 in the experiments.

We then show how we determine the number of rounds (i.e., the parameter \(rn_1\) ) that the job assignment algorithm (Algorithm 1) runs. We randomly generate three groups of jobs, with 100, 300, and 500 jobs, respectively, and run them on 15, 50, 90 servers. We run the job assignment algorithm with a different number of rounds for each group of jobs, and compute the system throughput for each group over each number of rounds. Figure 5(b) shows the results. We have several observations. First, the throughput grows as the number of rounds increases for all the groups of jobs, because using more rounds has more chance to find a better solution. Second, the algorithm converges slightly slowly as the number of jobs increases. This is because the solution space grows as the colocation size increases, while exploring a larger solution space is more difficult. Third, the algorithm converges at 500 rounds for all the groups. Based on the results, we set the number at 500 in the experiments.

Next, we show how we determine the parameter \(x\%\) in the job assignment algorithm (Algorithm 1). Recall that the jobs in the local search operations are selected in the top \(x\%\) colocations with the highest/lowest throughput (such colocations are more crucial for bringing throughput improvement). We randomly generate 500 jobs and divide them into 90 colocations using the job assignment algorithm. We compute the mean throughput of the colocations for running the algorithm with different \(x\%\) . Figure 5(c) shows the results. As can be seen, JointOPT achieves the best performance for \(x\% = 30\%\) . If \(x\%\) is too small, the algorithm can converge more quickly, but some important colocations may be missed. In contrast, if \(x\%\) is too large, the local search process has low efficiency, because many unimportant colocations will be measured. Based on the results, we set \(x\%=30\%\) in all the other experiments.

We next show the overall performance of JointOPT compared to the baselines. We generate 2,200 jobs and strive to assign the jobs to 400 servers. Figure 6 compares the throughput of JointOPT with the resource partitioning baselines when they are integrated with different job assignment methods. We have several observations from the results. First, for all the job assignment methods, NoPart always performs worst among the baselines. This is because NoPart incurs the largest performance interference since resource partitioning is not considered. This result confirms the importance of resource partitioning. Second, for all the job assignment methods, DRLPart always performs best among the baselines, which demonstrates the effectiveness of our DRL-based resource partitioning solution. The inefficiency of DCAPS is mainly because it partitions only one resource, while DRLPart partitions multiple resources coordinately. The problem of CLITE is mainly because it evaluates partitioning configurations in real systems, which is inefficient when exploring a large solution space.

Third, NoPart, DCAPS, and CLITE achieve higher throughput when integrated with our job assignment method compared to other job assignment methods. This is because: (1) the random and bin packing job assignment methods do not consider resource sensitivities of jobs, which may colocate the jobs with serious performance interference together; and (2) the contention-aware job assignment method does not consider the resource fungibility, which cannot fully utilize the server resources. The results demonstrate the effectiveness of our job assignment. Fourth, JointOPT always achieves the highest throughput compared to the baselines, with advantages from 13.3% to 47.7%, which demonstrates the effectiveness of optimizing job assignment and resource partitioning jointly.

Figure 8 breaks down the results according to different colocation sizes, where the y-axis represents the throughput improvement of each approach over NoPart. All the resource partitioning methods are integrated with the random job assignment method. We have three observations from the results. First, JointOPT always outperforms the other approaches for all the colocation sizes, which demonstrates the good scalability of JointOPT. Second, the advantage of JointOPT over the baselines is different across different colocation sizes. Specifically, JointOPT achieves the highest benefit for the colocation size of 5-jobs (57.4%), and less benefit for large colocation size (e.g., 35.5% for 7-jobs). This is because each job will be allocated a lesser amount of resources in a large colocation, so the jobs always have bad performance no matter how the resources are partitioned. Third, CLITE performs as bad as NoPart for colocations larger than 5-jobs. This is because the time required by CLITE to generate a decision grows exponentially as the colocation size increases. When the colocation size is larger than 5-jobs, CLITE cannot even make one partitioning decision during the examining period.

We do not consider colocations larger than 7-jobs, because (1) due to the hardware limitation of Intel’s MBA technology (the number of CLOSes in MBA is limited to 8 in our platform), our platform can only deploy fewer than 8 jobs if we want to partition the memory bandwidth; and (2) we only have 9 CPU cores to assign jobs and at least one CPU core should be allocated to each job.

In this experiment, we show how the performance of each approach is impacted by the colocation types. We say a job is sensitive to a specific resource if its performance degradation exceeds 15% when the amount of assigned resource decreases from the maximum to the minimum. If a job is sensitive to only one resource, we say it is single-sensitive (Single-S). If a job is sensitive to more than one resources, we say it is multiple-sensitive (Multi-S). If a job is not sensitive to any resource, we say it is insensitive (IN-S). We divide colocations into different types according to the types of jobs contained in each colocation. Table 7 summarizes the definition of each colocation type.

We randomly generate 100 colocations for each colocation type, with colocation size ranging from 4-jobs to 7-jobs. Figure 7 shows the throughput improvement over NoPart achieved by each approach for colocation of 4-jobs and 7-jobs. We have two observations from the results. First, the throughput improvement is more significant for colocations of Type-A, Type-B than other colocation types for most of the approaches. For example, the throughput improvement of our approach is 45.0%, 51.5%, 51.2%, and 25.6% for colocations of Type-A, Type-B, Type-C, and Type-E, respectively for colocations of 4-jobs, while it is only 15.3%, 5.0% for colocations of Type-D and Type-F. This is because the jobs in the colocations of Type-D and Type-F are less sensitive to resources, so the benefit of resource partitioning is insignificant. Second, similar to the previous results, CLITE has no benefit over NoPart for colocations of 7-jobs, which confirms that CLITE fails to handle a large colocation size due to high computational overhead.

In this experiment, we randomly generate 360 colocations with sizes from 4-jobs to 7-jobs, which contain a proportion of new jobs. We compare the performance of JointOPT with/without model refining against different proportions of new jobs. In the implementation, we refine the model once over 10% new jobs are added. Figure 9 shows the results. We observe that if model refining is not used, the performance of JointOPT gets worse as the proportion of new jobs increases, despite the performance degradation is not significant (4.2% for 30% new jobs). It implies that although JointOPT has good generalization to new jobs, it still suffers from performance loss when facing a large portion of new jobs. The performance of JointOPT with model refining consistently achieves good performance over different proportions of new jobs, which confirms the effectiveness of model refining.

Model Training. All the machine learning models are trained on an Nividia^® GeForce^® RTX 2080 Ti GPU. The training process for the colocation evaluating model takes around 50 minutes. The training process for the DRL reward prediction model takes around 3 hours. The training process for the DRL model converges in around 1,000 iterations, which takes around 16 hours. Recall that if the reward prediction model is not used, the training process of the DRL model will take around \(10^5\) hours. It implies that the reward prediction model reduces the interaction overhead by three orders of magnitude. The space consumption on model training is around 2GB, but it is offline and one-short.

Job Assignment. The inference time of the colocation evaluating model is around 300 microseconds each time. The running time of the hill climbing algorithm for finding the optimal partitioning configuration is around 0.075s for a colocation of 7 jobs. For the job assignment algorithm, the total running time is around 37.5s when there are 500 jobs. Note that the throughput-oriented jobs generally have no strict response latency requirement. Thus, the running time of the job assignment algorithm is not an issue. The space consumption on job assignment is around 6MB.

Resource Partitioning. The DRL-based model makes partitioning decisions very quickly, which takes less than 2s to make a decision using Nividia^® GeForce^® RTX 2080 Ti GPU. It is much faster than DCAPS (which consumes more than 10s due to a heuristic search using simulated annealing) and CLITE (which consumes several minutes due to high computational overhead). So, our approach is more efficient to handle fast-paced workload changes. The space consumption on the DRL-based framework is around 7MB.

Model Refining. The refining time of the machine learning models is much shorter than their training time. Specifically, it takes around 10 minutes, 0.5 hours, and 3 hours for refining the colocation evaluating model, the reward prediction model, and the DRL model, respectively. Compared to training the models from scratch, transfer learning can speed up by 5× (50 minutes vs. 10 minutes), 6× (3 hours vs. 0.5 hours), and 5× (16 hours vs. 3 hours) for the colocation evaluating model, the reward prediction model and DRL model, respectively. The space consumption on model refining is around 972kb.