Focused Layered Performance Modelling by Aggregation

LQNs are extended queueing models for contention effects in layered systems, in which a software server depends not just on its host processor but on other software servers as well. A summary of LQN concepts is found in Reference [9], based on earlier work in References [10, 31, 37]. A tutorial guide is in Reference [40]. LQN concepts are illustrated by the example in Figure 2. Software components called tasks run on hardware components called hosts. Task T has m_T servers (representing concurrent threads), host H has m_H servers (representing multiple CPUs), and we assume that the queue disciplines are FIFO at tasks and processor sharing at hosts (representing time-slicing schedulers). The operations offered by a task are called entries. Entry E requires \(d_{E}\,/ s_{H}\) seconds of service by its host (where d_E is the mean CPU demand in suitable units and s_H is the host service rate or speed factor) and may make \(y_{E,E2}\) mean calls to another entry E2. Details of an entry may be defined by a precedence graph containing activities. In practice, demands are measured on some “reference” host type and the speed factor of a given host is its execution speed relative to the reference type. The capacity of a host H is defined as \(c_{H}\,= \,m_{H}s_{H}\) and is the equivalent number of reference type CPUs.

The system's users are represented by a special task User, for which \(m_{User}\) gives the number of users. It has one entry whose execution represents a user response, including a think time \(Z_{User}\) (1,000 ms in Figure 2) and calls to system services. The throughput of the User task is the system throughput.

In this work, the model is assumed to have a single set of Users, which may, however, combine different behaviors. The tasks are assumed to be heterogeneous; replicated servers and symmetrically replicated subsystems can be simplified efficiently by a special technique summarized in Reference [9].

FSPT and SPT use the following parameters and measures derived from the LQN model:

The approximation errors of the FM are given by

Given a user think time Z, the throughput and response time are related by Littles’ result as \({R}_{User} = {N}_{User}/{\lambda }_{User} - Z\) . Because the evaluations were done with zero user think time \(Z = 0\) , it follows that the relative errors of the throughput and response time are the same, so \(Er{r}_R = {\rm{ }}Er{r}_\lambda\) . This value is denoted by \(Err\) in the rest of the article. The degree of aggregation is measured by

SPT applies to models with one load-generating User task, which may represent a combination of response types. The model is restricted to having synchronous calls with no cycles in the graph of the calls between tasks.

Construct Dependency Groups. The following notation is used for the dependency groups:

SPT aggregates the tasks outside PT in dependency groups. Task \(T_{1}\) is said to depend on the operations of another task \(T_{2}\) if it calls \(T_{2}\) directly or indirectly via another task. For each dependency subset \({{{\boldsymbol \Pi}}}_{{i}} \in {{\boldsymbol \Pi }},\) there is a distinct set of non-preserved tasks (possibly empty) that it depends on, denoted as its dependency group G_i. In SPT, processors cannot be shared between groups, and the hosts of any group G are aggregated to a single host.

Dependency groups are illustrated in Figure 3. The tasks in \({\bf PT}\,= \,\{User,\,PT_{1}\,\text{and}\,PT_{2}\}\) have bold borders_. There are four subsets \({{{\boldsymbol \Pi }}}\) _i and four dependency groups indicated by shadings. Each group gives one aggregate task with the same shading and one host, as shown in Figure 3(b).

FSPT removes the restrictions that a processor cannot be shared between groups, as described in Section 3.

Aggregation Operations. Suppose a group g of tasks with the set \({\textbf {e(g)}}\) of entries and the set \({\textbf {h(g)}}\) of hosts is aggregated into a task A with a single entry \(E(A)\) . Its demand \(D_{E(A)}\) and its mean calls per user response are sums over the values for tasks in g,

The resource multiplicities are summed,

The speed factor for \(H(A)\) is computed to give the same total host capacity,

Interactions of tasks in g with any entry outside g are aggregated to one interaction as follows:

A more detailed description of these operations is found in Reference [17].

Aggregation Properties. SPT has the following desirable properties:

Strategies to Determine the Focus. The focus set PR may be selected by the analyst, but an automated method is also useful and can be combined with analyst choices. Two strategies for determining what to automatically keep in PR were utilized, both based on the resource saturation S:

The focus also must include LQN components that involve priority queueing, execution defined by activities and second-phases, asynchronous and forwarding messages, and symmetric subsystem replication, since FSPT does not aggregate components with these properties (see Reference [9] for details).

Two non-focus tasks in different dependency groups that share a host are “entangled” by their common host dependence and cannot be aggregated. SPT in Reference [16] cannot deal with entanglement. Entanglement is illustrated in Figure 4(a).

Tasks are disentangled by giving them separate shadow hosts of reduced capacity, similarly to the shadow servers defined for classes with priorities in References [23, 34]. In those works, each priority class is provided with its own shadow server to simplify the delay calculation.

The transformation is constructed to preserve the response time to a request for the host processing provided to each task. Consider a single entangled task \(T_{i}\) on host \(H\) , and a shadow host \(H_{i}\) constructed for it with speed factor \({s}_i\) (to be determined). \(H\) is a queueing server, and the task threads are its customers, with a class for each task. \(T_{i}\) provides class \(i,\) with service time \({x}_i\) and response time \({R}_i\) . Using the analytic queueing results [20] for a processor-sharing host,On \(H\) , \(T_{i}\) has a partial saturation \({S}_i\) (which is the fraction of the capacity of \(H\) which is used by T_i) equal to its server utilization divided by \({m}_H\) , and \({S}_H = \mathop \sum_j {S}_j\) . When it is moved to \({H}_i\) , its service time is scaled by the speed ratio to \(x{'}_i\) , and it has the same arrival rate, so \(x{'}_i\) and its saturation \(S{'}_i\) are as follows:

Given that \({H}_i\) has just the one class provided by \(T_{i}\) , its response time \(R_i^{\prime}\) is given by

Setting \({R}_i = R_i^{\prime}\) and re-arranging gives the shadow server speed factor and saturation ratio as

By Equation (14), the ith shadow host capacity is the part of the original capacity not used by the other tasks. Although it is derived differently, it is the same as a shadow server created for priorities in References [23, 34].

The classes represent the tasks executing on the host. If multiple tasks have a common shadow host as in Figure 4(b), then they form a subset of classes and the same construction is applied, with S_i replaced by the sum of saturations over the subset. Formally, the tasks on host \(H\) are divided among a set of shadow hosts, with one shadow host \(h(H,\,{\textbf g})\) for each group of tasks \({\textbf g}\) that has members on \(H\) . Call those members \({\textbf{g}}'(H,{{\textbf g}})\) , and the set of their entries, \({\bf E}(H,\,{\textbf g})\) . One aggregated task \(A(H,\,{\textbf g})\) represents the tasks in \({\textbf{g}}'(H,{{\textbf g}})\) , with one entry \(e(H,\,{\textbf g})\) . The OM saturation of host H due to task T is \(S_{H,T}\) , so using Equation (14) the speed factor of the shadow host \(h(H,\,{\textbf g})\) is

where \({\bf T}(H)\) is the set of tasks with host \(H\) . The demand \(D_{e(H,g)}\) of the aggregated entry \(e(H,{\textbf g})\) is the sum of the demands of entries in \({\bf E}(H,{\textbf g})\) weighted by their speed factors (from Equation (13)),

The calls to and from \(A(H,\,{\textbf g})\) are found as in Equations (10) and (11), with the subset \(\rm{{\textbf {g}}{'}}(H,{{\textbf g}})\) in place of g.

Although the shadow server approximation is well known, it is based on open workloads, while LQN models have closed workloads. Therefore, its accuracy for disentanglement was extensively tested for closed workloads before using it. The details are in Reference [17], but, in summary, first, the calculated response times were compared to simulation results for two classes each with a closed workload of 100 customers. The queueing was simulated with one and three servers ( \(m\,= \,1\) and 3) for 1,500 cases with random parameters. The response time errors for \(m\,= \,1\) were all less than 1%, and for m = 3, less than 0.3%. Second, the errors due to disentangling an LQN model were found for 25 cases that occurred in simplifying 10 randomly generated models. Thirteen instances had 0–2% error, and the rest were all under 8%. This accuracy was judged less than ideal but adequate.

This section describes the base-case approximation errors resulting from completed FSPT with the ACCx strategy. The experiments reported here and in Reference [17] differ from the published evaluation in Reference [16] in that they include cases with entanglement. The ACCx strategy adds preserved tasks one at a time until \(\text{Err}\,<\,x\) , as illustrated in Figure 5 for an example of original size (tasks + hosts) of 31 and a 2% error target.

As more tasks are preserved, the accuracy improves but the size increases. For a given purpose the size target x can be chosen by the analyst. The nature of the accuracy/reduction tradeoff was investigated by finding the throughput error and degree of model size reduction obtained for 150 randomly generated models at two levels of traffic intensity. In addition,

To evaluate the accuracy of FSPT, 150 models were generated with random structure and parameters by using the utility program lqngen [12]. The models had three to five layers and a wide variety of calling patterns to represent calls between web services. Five model sizes with 11, 16, 21, 26, and 31 tasks (including the User task) were chosen, and 30 models were generated for each size. Another 150 models were created by doubling the number of users in each model. The ACC2 strategy was used, which increased the number of preserved tasks until \(Err\,<\,2\%\) . Figure 6 displays the tradeoff between accuracy and simplification by a scatter plot of Err against the fraction F_T of preserved tasks, for all the FMs created in the sequence of reductions. The accuracy values have great variability. Three properties stand out in Figure 6. On the left, reductions with \(F_{T}\,<\,10\%\) give a wide range of errors. For F_T > 10% and up to 50% the error is reliably less than 18%, and for \(F_{T}\,>\,50\%\) all errors were less than 10%.

Simplification also tended to be more successful for larger models, which is exactly where it can be most useful. The average \(F_{T}\) for 10% accuracy was about 42% for models with 11 tasks and increased to about 62% for models with 31 tasks. For 2% accuracy; however, all sizes of models had average \(F_{T}\) of about 40%. More details are given in Reference [17].

Figure 7 examines the tradeoff between accuracy and focused model size by a scatter plot of Err against the fraction F of size (tasks plus hosts) for all 695 FMs created in the sequence of reductions. The accuracy values have great variability for small F but FMs with F > 60% all have \(Err\,<\,20\%\) . Only 49 FMs of all sizes showed Err > 20%, and only 114 showed Err > 10%. We can conclude that a reduction to 60% in size gives a maximum error of 20% (almost always), and the great majority of all reductions down to 20% in size have error less than 10%. Since these are empirical results any given model may have, however, larger errors than this. For sensitivity analysis, further conditions must be considered as shown in Section 4.

For the time complexity of FSPT, if there are N model elements (tasks plus processors), and if several properties are bounded by constants (entries, calls and activities per task, and the number of dependency groups divided by N), then for a given set of dependency groups the aggregation operations are of order of \(N^{2}\) . If there are K preserved tasks, then to obtain the dependency groups by brute force is of order \(2^{K}\) (to obtain the powerset of PT). However, most of these have no dependency group; by working backwards from the non-preserved tasks (non-PTs) the non-empty groups can be constructed by operations of order \(K^{2}\) . The steps are as follows: construct a matrix of dependencies of PTs on non-PTs \((K^{2})\) , for each PT collect the PTs dependent on it \((K^{2})\) , and identify identical collections as the dependency sets \((K^{3})\) .

FSPT also requires a LQN solution as a starting point. If LQNS is used, then, according to Reference [10], it has a complexity of order N² per iteration, and experience with the LQNS solver suggests that the number of iterations rises only a little, if at all, with N.

Overall, if K is of the same order as N (as is assumed above), then the time complexity of FSPT is of order N³, dominated by constructing of dependency groups.

To demonstrate the application of SPT to larger models, a case with 50 tasks and 50 hosts was simplified. The process took 30 seconds, and (as described in Reference [17]) the aggregated model achieved 4.85% error with nine tasks and nine hosts, a highly successful result.

Six case studies of industrial software systems are described in Reference [17], of which three are summarized here.

(a) A Business Reporting System. The OM in Figure 1(a) was generated automatically from a design specified in the Palladio tool [28], based on an industrial system, with a deployment (a host for each task) that is artificial. The FM in Figure 1(b) preserves just two tasks of 43 and aggregates everything else. Its throughput and response time predictions across varying loads up to saturation are all within 1%. This case shows how when there is a well-defined bottleneck a small FM can give good accuracy.

(b) A Class 5 Telephony Switch. This OM of a voice telephony switch [11] shown in Figure 8(a) has 22 tasks and 19 hosts. The focused model TS-FM1 with just three preserved tasks gave 5.2% error. Preserving more tasks only slowly reduced the error, as described in Reference [17].

The focused model TS-FM2 was created with a stated saturation ratio SR = 3.33 to satisfy the Accurate Sensitivity Hypothesis and gives a “trusted range” from 0.75 times to 1.33 times the base-case throughput. Sensitivity calculations for model TS-FM2 are described in Section 4.2.

(c) An E-Commerce System. A model of a generic E-Commerce system from Reference [24] with eight tasks and six hosts was reduced to three tasks and three hosts with less than 1% error [17].

To visualize the performance of FSPT, the error Err (as a fraction) is plotted against the FM size fraction F for a sequence of focused models that preserve different numbers of elements, as in Figure 5. Small error for a small size factor indicates effective simplification. Figure 9 shows a typical case, and the best and worst cases, from the 150 random models. A point \((f,\,e)\) is called achievable if a better focused model exists with both \(Err\,<\,e\) and F < f. The fraction of the space in \((f,\,e)\) that is achievable will be defined as the effectiveness index \(Eff(\text{OM})\) for simplification of model OM:

This fraction equals the shaded area in each of Figure 9(a), (b), and (c) below.

Figure 10 shows the models for case-92 and case-11, with some saturation values labeled to explain the effectiveness values. Both models are deeply layered and have requests that cross the layers. However, case 92 has 100% saturation at t0, which depends on just one other service at t10. Other tasks have saturations below 0.4, which cannot increase because of the bottleneck. As long as tasks t0 and t10 are preserved, aggregation of the rest of the model has only a small effect on performance. However, case-11 has several medium-to-high utilization tasks along the calling path t1-t2-t4-t7, with task saturation values 0.92, 0.79, 0.24, 0.32 (under different loads these values will vary but they retain their relative magnitude). Aggregation of multiple tasks with moderate loads introduces a substantial increase in throughput due to summing their concurrency levels (this is a well-known effect of aggregating separate concurrent servers to make one multiserver).

A scatter plot of the effectiveness index against the original model size is shown in Figure 11 for the 150 random models. There is a general trend to greater effectiveness for larger models but also some outliers that are quite resistant to reduction. Case-11 above lies at the left-hand edge, and case-92 at the right-hand edge of the range of sizes shown, so model size is a partial explanation of the resistance of case-11. The industrial case study models give similar results. The Telephone system model has an excellent effectiveness index at about 0.9. However, a Linux file system model from Reference [10], shown in Figure 12, is quite resistant. It gives two FMs, with (f, e) = (8/11, 0.588) and (9/11, .0433) giving Eff = 0.179. As with case-92, it has a calling path with several nearly saturated tasks, as shown in Figure 12(a).

A first example considers the OM and FM shown in Figure 13, and some parameters of the preserved resources \({\bf PR}\,= \,\{Users,\,t\textit{01},\,t\textit{07},\,p\textit{00}\}\) .

Figure 14 shows the throughput changes \(\Delta \lambda _{{\rm{OM}}}\) and \(\Delta \lambda _{{\rm{FM}}}\) when three different parameters are varied one at a time: the demand of t01 in part (a), the demand of t07 in part (b), and the multiplicity of p00 in part(c). The throughputs for OM and FM are almost identical in all three cases. These results are encouraging but not all models give such good results.

The OM of a Class V telephone switch was described in Section 3.5 and Figure 8 above. An FM with saturation ratio SR = 3.33 was created, shown as TS-FM2 in Figure 8(c), with six tasks and five hosts. The trusted range of throughput multiples is the interval (0.75, 1.33).

A set of 1,000 sensitivity cases were calculated in which all the demand parameters of every preserved task were changed by an independent random factor that was uniformly distributed over (0.5, 1.5), giving a range of ±50% from the base case values. Figure 15 shows histograms of the errors for the cases with results within the trusted range for 300 users. For 300 users, the system is not saturated, so the throughput changes are relatively small and the errors are all less than 2.5%. For 400 users, loads are heavier, some cases have throughput changes outside the trusted range, sensitivities are larger, and 117 sensitivities have errors above 20%.

Greater insight into the importance of the trusted range is given by the plot in Figure 15 of \(Err_{{\rm{sens}}}\) ( \(\bar{a}\) , a) against the throughput multiple \(M(\bar{a},\,a)\) . For 400 users, many throughput changes fall outside the trusted range, and the relevance of the Accurate Sensitivity Hypothesis is displayed in the larger errors for many of these points. Within the trusted range, 93.4% of the results have “adequate” accuracy \((Err_{sens}\,<\,0.2)\,\) and outside it about half.

To move away from single cases and seek insight into the general validity of the sensitivity analysis, the set of 150 OMs used in Section 3.3 for assessing the base-case FMs were re-analyzed for sensitivity. Three sets of FMs were created using three different strategies:

As in Section 4.2, the demand parameters of all the preserved tasks were multiplied by a random factor uniformly distributed over (0.5, 1.5). Nine random FMs were created for each OM, giving a total of 1,350 cases for each strategy. Figure 17 shows the error histogram for each strategy; there are fewer than 1,350 points because of a few non-convergent model solutions.

The ACC2 strategy gave the worst results (Figure 17(a)) but still over 1,300 of the 1,350 points have error less than 20%. The large errors were in cases with low saturation ratios SR in the range (1.07, 1.36) and high-impact parameter changes. For SAT3.3 and SAT10 all the results have errors less than 20%. The advantage of using the SAT strategies for sensitivity analysis seems clear.

Figure 18 shows scatter plots against the throughput multiple M. For the ACC2 FMs it is clear that the largest errors are associated with M > 1. Since SR is different for every FM, a trusted range cannot be mapped. For the SAT cases, the trusted range in M can be plotted and is (0.75, 1.33) in part (b) and (0.125, 8) in part (c). SAT10 is only slightly better than SAT3.3, indicating that the extra large focus was not necessary for these parameter changes.

System scaling increases the capacity of resources to handle larger loads on the system. These experiments applied a common scale factor “scale” (ranging from 2 to 10) to the capacity of every preserved resource (tasks and hosts) and to the number of users and evaluated the throughput. Successful scaling should give an increase in throughput by a factor of f also. Figure 19(a) shows the throughputs against the scale factor, and in Figure 19(b) the points along the line show Err_sens for the successive scale factors (scale = 1, 2, \(\ldots\) ) against M, the OM throughput multiple. The FM throughput increases more than OM, because a resource that saturates in OM has been aggregated with more lightly used resources, which share the load. This model provided limited scaling, but in some cases both model throughputs scaled all the way up to 10 times.

As described above, these experiments scaled the resources of the 150 random models by factors scale from 2 to 10. Because the changes in throughput are large, the normalization of error was changed. For large deviations it is better to normalize to Δλ_OM instead of λ_OM, giving the measure \(Err_{largeSens}\) :

Using this normalization of the error, Figure 20 shows histograms of Err_largeSens that are similar to the histograms for demand sensitivities. Only the SAT10 cases are mostly accurate. The ACC2 and SAT3.33 cases have poor accuracy in many cases, because the saturation ratios of the FMs are not large enough for aggressive scaling. This issue was investigated more deeply for the ACC2 cases, which covered a wide range of ratios in an attempt to identify the inaccurate results for the analyst, which led to the Accurate Sensitivity Hypothesis.

Deeper analysis of the ACC2 case errors in Figure 20 led to the Accurate Sensitivity Hypothesis stated in Section 1. Many of the errors were associated with small values of the saturation ratio SR, which in turn indicates that some resources are close to saturation in OM but are not preserved. As the throughput increases (and M becomes large) these resources become saturated in OM but not FM, and errors become substantial. An example called “case-30” is shown in Figure 19 above.

To examine this relationship, the value of SR was found for each FM found by ACC2. Figure 21 plots M against SR separately for the “accurate” and “inaccurate” results and shows that inaccurate results emerge when M is larger than a value that is roughly a constant plus R. The sloping boundary of the grey area was chosen by eye so it contains almost all the inaccurate results and represents M = SR – 2. The points inside the grey area violate the ASH condition, and the others satisfy it and define a “trusted range” for M for a FM with a given R. There are many false negatives, showing that the hypothesis is quite conservative. There are also some false positives, showing that the trusted range does not provide a guarantee. However, there are very few false positives.

We can revisit the cases described so far to see how successful the ASH is in predicting accurate results. Table 1 summarizes the number of results that satisfy the ASH and the number of these with less than 20% error using the \(Err_{sens}\) measure for the demand studies and the \(Err_{largeSens}\) measure for the scaling studies.

For the demand sensitivity results, the trusted range was found to be reliable, but for the scaling results there are some “false positives,” results that test to be accurate but are not about 2% for the SAT10 cases and about 7% for the ACC2 cases and 7.5% for some of the telephone switch results. Most of those errors are just over 20% and few exceed 30%.

The random cases in this section apply many large parameter changes and thus place a heavy stress on the accuracy of sensitivities. Accurate sensitivities require a model with a large-enough saturation ratio R, and the Accurate Sensitivity Hypothesis provides insight to the analyst when SR is not large enough. When the ASH condition was met, over 90% of cases had errors less than 20%. The relative error was normalized to the base value of the measure for small changes (less than 100%) and to the OM change itself for larger changes.