Actor-Driven Decomposition of Microservices through Multi-level Scalability Assessment

Figure 1 illustrates the high-level building blocks of the methodology as well as the dataflow among them. White boxes represent the main activities that we present as part of our contribution. Gray boxes refer to existing practices often applied along the software life-cycle, especially in the context of microservices systems.

As shown in Figure 1, we assume the existence of functional and nonfunctional requirements, elicited for the purpose of designing and implementing the application of interest, and the domain knowledge about the actors and their role in terms of used functions provided by the application. Among the possible nonfunctional requirements (e.g., scalability, reliability, resilience, security), our methodology mainly focuses on performance and scalability.

We assume that existing methodologies, such as DDD, as well as microservices design patterns (e.g., aggregate, and subdomains), have been applied to identify a possible initial decomposition of the application into a set of coherent bounded contexts that separate the application functions according to the entity model characterizing the domain of interest. At this point, we refine⁴ the bounded contexts by ① applying the ADD strategy. We still apply well-known microservices patterns [6] (e.g., application programming interface (API) gateway, shared database, database per service) and we possibly complement them with additional patterns (e.g., command query responsibility segregation, saga, events sourcing) to take into account the actors and their role in the target system.

As outcome of ADD, engineers ② define a set of alternative deployment architectures specifying the components, their relations, and the mapping to execution units. These deployment architectures further decompose the services previously identified through DDD, with the ultimate goal of maintaining or even increasing the scale of operation. The evaluation of the deployment architectures starts with the ③ definition of the operational profile of the application to characterize the expected workload in the production environment. This profile is defined by engineers according to the set of actors identified by ADD, the operations invoked by the actors, and the expected workload intensity for each actor. This latter piece of information can be either manually defined by engineers according to an existing Service Level Agreement (SLA) or mechanically derived from execution traces collected by observing similar systems or previous versions of the system in a given observation period.

For each deployment architecture, an initial ④ resource allocation must be defined. The allocation maps the available resources to execution units taking into account the computational complexity of the functions running on such units. At this stage, we aim at ensuring an acceptable⁵ response time for the operations exposed by microservices under a baseline workload. After tuning the resource allocation, the system is ⑤ deployed in a staging (or pre-production) environment to test the application using the ⑥ multi-level scalability assessment. This latter stage automatically executes testing sessions replicating the operational setting. Then, it guides engineers in understanding and improving the scalability of the selected deployment architecture(s) at different levels of abstraction: system, component, and operation levels. The outcome of the multi-level assessment is then used to select the alternative(s) that better fits the expected operational condition. The resource allocation of the selected architecture(s) can be further tuned according to the outcome of the assessment stage. This triggers a new assessment iteration, as shown in Figure 1. The final outcome of the methodology is a deployment architecture that, at the end of the iterations, yields better scalability. With this architecture, the system is then deployed into the production environment.

To demonstrate the applicability and effectiveness of the proposed methodology, we identify the following Research Questions (RQs), which we answer in the next sections.

As anticipated in the previous sections, ADD complements and refines the results of DDD with the goal of improving scalability. According to the Scale Cube model in Section 2, ADD drives the design of a target application towards a more selective horizontal replication (x-axis scaling) through a refinement of functional partitioning and data partitioning (y-axis and z-axis scaling, respectively). While DDD usually decomposes by noun-preserving cohesion of domain entities, considering the actors (and their roles) takes a complementary perspective and identifies possible finer splitting according to usage. ADD does not prevent engineers from applying recommended practices, such as the single responsibility principle or aggregate roots. It starts from a definition of bounded contexts and considers possible decoupling based on how end-users interact with their operations. According to [28], we can see ADD as an additional tactical pattern to identify further boundaries (that do not violate the existing ones) considering non-functional requirements.

This approach has the potential of increasing the operation scale by selectively replicating finer units according to actors’ needs. The idea is to create a logical separation of the operations belonging to the same bounded context based on how the actors are going to use them. In this case, we can selectively increase/decrease computational resources to different groups of operations according to the load generated by the actors.

Figure 2 shows a high-level schema of DDD refinement through ADD. Given the bounded context \(BC_1\) in Figure 2(a), which corresponds to a coarse-grained microservice (derived from a bounded context) having its own API exposing \(o_1, \ldots , o_6\) , each actor in \(\mathcal {A} = \lbrace a_1, \ldots , a_4\rbrace\) generates a different workload whose intensity is \(\rho _{a_i}\) for all i. Each workload intensity is directed to a subset of the operations depending on the behavior model of the corresponding actor. For instance, \(\rho _{a_2}\) is directed to \(\lbrace o_3, o_4, o_5\rbrace\) , while \(\rho _{a_3}\) is directed to \(\lbrace o_5, o_6\rbrace\) . Since different workloads are handled by the same service, the whole service needs to be replicated even in cases in which only one of these workloads is heavy.

As shown in Figure 2(b), the behavior model of the actors yields the covering \(C(O) = \lbrace O_{a_1}, O_{a_2}, O_{a_3}\rbrace\) , that identifies a logical decomposition into the services \(S = \lbrace s_1, s_2, s_3\rbrace\) . This refinement of the original service can better exploit the available resources \(r_1, \ldots , r_k\) since they can be assigned to more granular services according to the generated workloads. The identification of the covering \(C(O)\) requires the following prior knowledge that architects typically have at design-time: (1) operations exported by each service (e.g., Swagger OpenAPI documentation⁸) and (2) the actors using them. The logical decomposition S induced by \(C(O)\) meets the DDD refinement guidelines suggested by tactical patterns introduced in [17, 28]:

As shown in Figure 2(b), a covering \(C(O)\) may lead to partially overlapping sets, such as \(O_{a_2}\) and \(O_{a_3}\) . In this case, alternative strategies to define services are possible. In particular, the operations in the intersection:

Binding the final decision to a single choice without proper understanding of the corresponding impact on quality attributes is dangerous and could lead to violation of the scalability requirement (Equation (1)) in production. For this reason, whenever the architects have a set of possible alternative architectures, proper and systematic assessment of them shall be carried out.

Let \(\hat{r}_i\) be the amount of resources allocated to service \(s_i\) that ensures the satisfaction of the scalability requirement in the case of reference load \(\lambda _0\) . Then, let us consider a generic load \(\lambda \gt \lambda _0\) such that \(\lambda\) generates the workload intensity \(\rho = \sum _i \rho _{a_i}\) . According to the logical decomposition described earlier, the amount of resources allocated to each service \(s_i\) shall be proportional to the workload intensity \(\rho _{a_i}\) of the corresponding actor. The mapping from available resources to services is denoted by \(\mathcal {M}\) ⁹ and defined as follows:where \(\Delta (\cdot)\) represents a scaling factor expressed as a function (either linear or nonlinear) of the relative workload intensity, which depends on the number of concurrent users.

The scaling factor function \(\Delta\) is usually implemented by means of autoscaling components that automatically adjust the amount of resources per service at runtime. Let us consider an upper-bound r in the total amount of available resources and the workload intensity \(\bar{\rho } = \sum _i \bar{\rho }_{a_i}\) such that all of the resources are saturated by the function \(\Delta\) . Under this condition, the autoscaling component usually follows the distribution of the workload intensity, that is, the amount of resources for each service \(s_i\) is proportional to the workload intensity \(\bar{\rho }_{a_i}\) . Thus, the mapping is approximately as follows:taking into account the constraints imposed by the available resources on each virtual or physical machine. These constraints could prevent the ideal mapping from being applied due to the presence of nondivisible resources.

This condition represents an important scenario that pushes microservice performance to its limit. Testing the target system under these circumstances is required to study the total amount as well as the distribution of resources that guarantee the satisfaction of the expected maximum load defined, for instance, by an SLA. Our methodology systematically replicates these circumstances in controlled experiments to study the extent to which alternative architectural options (driven by ADD) improve the scalability of the target system under different distributions of the workload intensity \(\bar{\rho }_{a_i}\) .

Even though the focus of our work is on performance and scalability, it is worth noting that the increased granularity level obtained through ADD may lead to better fault localization and higher reliability. For instance, let us consider \(BC_1\) in Figure 2(a) and \(O_{a_1}\) , \(O_{a_2}\) , \(O_{a_3}\) in Figure 2(b) obtained through DDD and ADD, respectively. While ADD yields better distribution of the workload, in the DDD version the workload of three actors ( \(\bar{\rho }_{a_1}\) , \(\bar{\rho }_{a_2}\) , and \(\bar{\rho }_{a_3}\) ) may generate a substantial amount of traffic directed to a unique service. In this case, \(BC_1\) may become a scalability bottleneck, prone to server errors due to saturation of resources, thus reducing the reliability of the system [29].

The decomposition of a bounded context into smaller pieces may lead to difficulty in splitting the database due to the high cohesion of the related schema, leading to a shared database solution that may limit scalability. Here, we propose two different approaches that can be applied in different situations according to the chracteristics of the services identified by ADD.

The objective of our assessment approach is to guide architects on deployment architecture selection, in the set of all available alternatives DA, that better fits the operational setting in terms of performance and scalability.

Figure 6 shows an overview of the main stages: ① derivation of the operational setting, ② load testing, ③ measurement framework, and ④ analysis workflow.

The operational setting derived in stage ① describes the system usage and is typically obtained from operational data generated during system execution and SUT specification [11, 30]. In stage ②, load tests are designed taking into account a target operational setting and alternative deployment architectures. The tests are then automatically executed to collect response times of invocations to each SUT operation. In stage ③, raw data are automatically processed to compute a set of metrics according to our measurement framework. In stage ④, the analysis workflow guides engineers to informed decisions by means of a cockpit that displays plots and measures tailored to the questions of interest.

In this stage, the operational setting (Section 2.3.5) is determined by combining the information on SUT specification (e.g., the allowed sequences of operations and invocations from the RESTful APIs for the workload specification model) with the historical data generated by the SUT during its operations (e.g., the frequency of occurrence of a certain behavior model). The usage profile can be manually defined or automatically derived from operational data. The manual definition of the usage profile is typically conducted by deriving user-application interaction patterns of relevant scenarios. The automatic extraction is performed on historical data and is carried out using process mining [31], or clustering algorithms, such as the WESSBAS approach [27]. For instance, for a cloud provider delivering an application as a service, the load and behavior mix represent the precondition for defining the guarantee terms of an SLA, which specifies Service Level Objects (SLOs), such as response time, against the declared preconditions [32].

The operational profile is typically determined by observing the target system during operations in a certain period of time. The load is periodically recorded and its frequency is computed. The load values in a given interval (e.g., 0–300 in Figure 7(a)) are binned to obtain the discretized distribution over a set of equally distant load values \({\lambda _1,\ldots ,\lambda _k}\in \Lambda .\) The cumulative frequency between each two consecutive loads is then assigned to the smaller one. For instance, the result of the binning process on the data in Figure 7(a) is the polygon illustrated in Figure 7(b).

As shown in Figure 8, this stage includes two load testing sessions to collect the response time of the invocation to each SUT operation. For both sessions, the usage profile is the same. The former is referred to as the baseline test session and the SUT is executed under a baseline deployment architecture \(\alpha _0\) and load \(\lambda _0\) for which the SUT is expected to operate with acceptable performance as described in Section 2.3.4. During this session, the response time of each operation is collected and the scalability threshold in Equation (1) is computed. The latter session is referred to as target test session and it tests the SUT under a target operational profile defined by the selected loads \(\Lambda\) and the alternative deployment architectures DA. For each pair \(({\lambda }, \alpha)\in \Lambda \times DA\) , a test is executed.¹⁰ During each test, all invocations to SUT operations and corresponding response times are collected. The outcome of each test is the mean response time and the invocation frequency for each SUT operation.

According to our definition of architecture in Section 2, \(\alpha\) also includes deployment aspects: deployment of servers to pods ( \(\mathit {deployment}_{sp}\) ) and deployment of pods to physical or virtual machines ( \(\mathit {deployment}_{pm}\) ). This means that the results of the tests also depend on the adopted infrastructure, that is, scalability requires the identification of proper separation boundaries in the software architecture to enable an effective exploitation of the underlying (scalable) infrastructure.

This stage is fully automated and follows the measurement framework illustrated in Figure 9. It starts with the high-level goal of assessing the SUT scalability in terms of its capability of meeting the performance requirement under increasing load. It processes the response time and invocation frequency of all operations to provide four metrics for the analysis stage: the (relative) Domain Metric, scalability footprint, scalability gap, and performance offset described in the following.

Relative Domain Metric.

The relative Domain Metric measures the overall scalability of a deployment architecture at a given load, as described in [12]. It represents the probability that the SUT with deployment architecture \(\alpha\) does not fail under a given load \(\lambda \in \Lambda\) and is computed as follows:where \(f(\cdot)\) is the discretized distribution of loads (Section 2.3.5), n is the number of operations, and \({s}_j^{\alpha }(\lambda)\) is the scalability share of \(o_j\) (Equation (3)). When no operation fails with load \(\lambda\) , \(\mathcal {DM}^{\alpha }(\lambda)\) is equal to \(f(\lambda)\) and is less than \(f(\lambda)\) otherwise. This difference measures the scalability degradation due to the failed operations with load \(\lambda\) :where \(\mathcal {S}\) is the set of indices of the operations that fail. Thus, a failing operation contributes with its scalability share (Equation (3)) to the overall scalability degradation of the SUT. At each load, performance degradation can be visualized as the gap between the plot of the relative Domain Metric (outermost polygon) and the one of the discretized distribution (inner polygon), as shown in Figure 10. Internal polygons approaching the outermost line yield scalability closer to optimal from a system-level perspective.

By applying the Bayesian rule, we can also compute the total Domain Metric, as follows: \(\mathcal {DM}^\alpha\) provides engineers with a single value that measures the overall SUT scalability with the deployment architecture \(\alpha\) [33].

Even though the total Domain Metric represents an effective instrument to decide over different deployment architectures, in some cases it may not explain subtle differences; thus, further investigation might be necessary. Figure 10 illustrates an example of such a situation. The two deployment architectures \(\alpha\) and \(\beta\) have the same total Domain Metric (0.72) but different scalability behavior over loads. To understand what is better in these cases, we need to analyze the system at a lower abstraction level. To this end, we introduce the Scalability Footprint, which represents the scalability capability per individual operation.

Scalability footprint. The scalability footprint measures the scalability level of each operation exposed by the SUT. To obtain it, we first define the Greatest Successful Load (GSL) \(\hat{\lambda }_j\) for an operation \(o_j\) as the greatest load in \(\Lambda\) for which \(o_j\) succeeds. This implies that \(o_j\) fails for all \(\lambda \gt \hat{\lambda }_j\) . When \(\hat{\lambda }_j\) equals the maximum load in \(\Lambda ,\) \(o_j\) exhibits optimal scalability. The set of GSLs for all operations is referred to as Scalability Footprint \(\hat{\Lambda }^{\alpha }\) of the SUT with deployment architecture \(\alpha\) . In some cases, the footprint represents a strong boundary: when the average response time \(\mu _j\) increases monotonically with the load, the operation \(o_j\) also succeeds for all \(\lambda \le \hat{\lambda }_j\) . In this case, the Scalability Footprint represents the boundary for which each operation always succeeds before and always fails after its GSL value.¹¹

We use the scalability footprint to compare the alternative deployment architectures in both qualitative and quantitative manners. The qualitative comparison relies on a specific visualization method, that is, a radar plot as illustrated in Figure 11. Each circle in the grid of the radar represents a load \(\lambda \in \Lambda\) . The distance between two circles is the frequency of the discretized distribution f at the greater load. Each closed polygon in the radar represents the scalability footprint of a deployment architecture. Each vertex of a polygon is the GSL of the corresponding operation. For example, the blue and pink polygons in Figure 11 represent the footprints of \(\alpha\) and \(\beta\) over the five operations \(\lbrace o_1,\ldots ,o_5\rbrace\) . Operations exposed by the same architectural component have the same font color (e.g., red and black operations in Figure 11 belong to components A and B, respectively). The vertices either reaching or exceeding the outermost grid circle indicate optimal scalability for the corresponding operation (e.g., \(o_5\) ).

The quantitative comparison between alternative deployment architectures relies on the Mann-Whitney U-test statistic [34] and the Cliff’s delta effect size to classify its magnitude [35]. The Mann-Whitney statistic \(U_{\alpha \,\beta }\) measures how many times the GSLs in \(\hat{\Lambda }^\alpha\) are greater than the GSLs in \(\hat{\Lambda }^\beta\) for the same operations. According to [34], the Mann-Whitney effect size \(u_{\alpha \,\beta }\) is then computed as follows:where \(|\cdot |\) indicates the set cardinality. To classify the effect size, the Mann-Whitney effect size is first converted to the non-parametric Cliff delta effect size d:It is then classified according to the standard categorization introduced in [36]:

The categories measure the strength of the difference between two footprints. For instance, the value \(U_{\alpha \,\beta }=13\%\) with large (L) effect size indicates that architecture \(\alpha\) is largely (L) less scalable ( \(13\%\lt 50\%\) ) than \(\beta\) .

This quantitative evaluation can be applied considering all of the operations of the SUT or just those exposed by a specific component depending on the desired granularity level.

Scalability gap and performance offset. With the aim of providing engineers with additional information at the operation level, we define two additional metrics: the scalability gap and the performance offset of failing operations. The former measures the scalability loss and the latter the performance loss due to a failing operation. The scalability gap \(\mathit {SG}_j\) of a failing operation \(o_j\) is the scalability share (Equation (2)) at the minimum \(\lambda \in \Lambda\) greater than the GLS of \(o_j\) :The performance offset \(\mathit {PO}_j\) of a failing operation \(o_j\) is the distance from the mean response time to the scalability threshold \(\Gamma ^{0}_j\) at the minimum \(\lambda \in \Lambda\) greater than the GLS of \(o_j\) :

We visualize the scalability gaps and performance offsets of failed operations by means of histograms. Figure 12 shows the scalability gap under two alternative deployment architectures \(\alpha\) and \(\beta\) . Horizontal lines represent the loads in \(\Lambda\) . A bar lying on an horizontal lines at l —say, \(l=250\) — represents the scalability gap of an operation ( \(o_5\) ) that has the GSL value equal to the maximum \(\lambda\) smaller than l ( \(\hat{\lambda }_5=200\) ). Thus, higher bars correspond to failing operations having higher negative impact on SUT scalability. We use the same representation to compare the performance offsets: higher bars correspond to failing operations having higher impact on the performance degradation of the SUT.

The analysis workflow is tailored to understand (Figure 13(a)) and then improve (Figure 13(b)) the scalability of the system. The analysis starts from the following three goals:

The decision process is then guided by questions derived from the goals. Our visualization and measurement framework is then used to answer the questions listed in Figure 13.

Understanding the target deployment architectures. Considering G1, the workflow starts by visualizing the \(\mathcal {DM}\) polygon of a deployment architecture \(\alpha\) to determine whether the architecture satisfies a scalability requirement. Specifically, the architect would like to answer Q1 and Q2 for \(\alpha\) . If the corresponding \(\mathcal {DM}\) polygon is very close to the outer one, the architect may consider \(\alpha\) as optimal. If so, the analysis ends with the decision: no change in the architecture is needed and \(\alpha\) is recommended. In case the architect observes scalability issues for some loads (e.g., \(\lambda = 150\) and architecture \(\alpha\) in Figure 10), the architect can compute the total \(\mathcal {DM}\) value and answer Q3. If the total \(\mathcal {DM}\) is far from optimal, the architect can then consider G2 and compare the polygon \(\alpha\) and the total \(\mathcal {DM}\) with alternative deployment architectures to answer Q4. For instance, \(\beta\) and \(\delta\) are two alternative deployment architectures in Figure 10. Compared to \(\alpha\) and \(\beta\) , \(\delta\) has no scalability issues up to \(\lambda = 150\) , but the overall \(\mathcal {DM}\) is lower. With this information, the architect can understand that \(\delta\) represents a worse choice.

If the \(\mathcal {DM}\) polygons and the total \(\mathcal {DM}\) value are not sufficient to identify the deployment architecture closer to optimal, the architect uses the scalability footprints and applies a pairwise comparison of the alternative deployment architectures considering the effect size and magnitude. As an example, \(\alpha\) and \(\beta\) in Figure 10 have the same total \(\mathcal {DM}\) value but different \(\mathcal {DM}\) polygons that may be equally good. In this case, differences may emerge analyzing the scalability footprints. In case multiple architectures still exhibit similar scalability, the architect can refine and improve the deployment configuration (e.g., allocation of resources to components or operations) of one or more architectures and then apply our approach to compare them, as follows.

It is worth noting that both \(\alpha\) and \(\beta\) are initial assignments that improved over one or more iterative steps. These assignments are usually based on domain knowledge (coming from the analysis of historical data [27]) or can be produced by using sampling techniques [37] driving the selection of the initial and subsequent candidate sets of architectures/configurations.

Improving the target deployment architectures. Starting from G3, the workflow considers the scalability footprints of one or more deployment architectures to answer Q5 and Q6. The architect compares the effect size and magnitude of the scalability footprints of alternative deployment architectures and selects the components and their operations that require further investigation. The scalability footprints in the radar plot show the failing operations at each load level and the overall differences for components, as for components \(\mathtt {A}\) and \(\mathtt {B}\) in Figure 11. For instance, the footprint of \(\beta\) shows worse scalability for component \(\mathtt {A}\) and better scalability for \(\mathtt {B}\) . The effect size and magnitude calculated at component level may further indicate that \(\mathtt {B}\) is largely more scalable with \(\beta\) and \(\mathtt {A}\) is less scalable with \(\beta\) but by a negligible difference with \(\alpha\) . In this case, the architect can choose \(\beta\) and change the deployment configuration for component \(\mathtt {A}\) to improve the overall scalability of the SUT.

By following the workflow, the architect analyzes each operation by means of the scalability gap and performance offset. The main objective here is to identify failing operations whose impact is higher on performance and scalability. Operations associated with the largest impact yield a severe negative effect that can be mitigated by using suitable architectural choices and resource allocation (within the limits of the physical constraints of the underlying hardware). As an example, by inspecting Figure 12, we can observe that with \(\beta\) , the operations \(o_1\) , and \(o_2\) exhibit a scalability gap at load 150, while \(o_3\) yields a scalability gap at load 200. With this information, the architect can allocate more resources to \(\mathtt {B}\) (operations \(o_1,\) \(o_2\) , \(o_3\) ). Considering instead \(\alpha\) , the operation \(o_5\) fails at load 250 with a large scalability gap, whereas all other operations fail at load 200 but with a smaller gap. This means that the operation \(o_5\) of \(\mathtt {B}\) is the most critical one since its invocation frequency is higher compared with the other operations.

The system is able to collect data through sensors installed on road networks (or by exploiting a dedicated emulator when data are only available offline for a deferred analysis) and to process those data in the cloud by exploiting technologies for distributed and high-performance processing of large amounts of data. It is an example of a large class of software systems in the context of the edge-to-cloud paradigm that today are very common due to the widespread adoption of Internet of Things (IoT) technologies for the development of smart solutions in city environments. We focus on an extract of the whole system, a specific service used for road network management. We refer the reader to [14, 38] for comprehensive descriptions.

The system handles the API requests by managing the road networks as graph-based data structures, where nodes represent road intersections, relationships (edges) between nodes represent streets, and their weights are the average travel times streamed by other platform components towards the service layer. Figure 14 shows the class diagram modeling a road network whose elements are the data entities characterizing the specific bounded context. The \(\mathtt {Street}\) s are characterized by a series of coordinates, useful for managing the geometry during the visualization on a map. Every edge is directed. Thus, a two-way street is represented as two edges with opposite orientations. To test our system with real data, the road network of the metropolitan city of Rome was loaded into the database, including 42,753 intersections and 89,251 streets, respectively, graph nodes and edges.

The application exposes the REST operations listed in Table 2, which are used by two actors.

We observed that these two actors operate on the application in different ways, by invoking two different sets of operations as reported in Table 2. This way, from the initial bounded context materialized by TrafficService, we derived two smaller microservices, named AdminService and EndUserService. In the following, we will refer to services derived from DDD as Domain-Driven Services (DDS) whereas the services derived from ADD are referred as Actor-Driven Services (ADS).

Using the Scale Cube model, in this case study we exploit the ADD strategy to guide a further decomposition of bounded contexts along the y-axis. This enables a differentiated replication of the operations according to the expected load generated by different actors onto the system.

We then decompose the database according to the CRUD decomposition introduced in Section 3.2. The details of this decomposition and the related deployment architecture are discussed in Section 5.5.

Taking available resources into consideration, we evaluated different deployment architectures with our methodology. In particular, we adopted the same deployments of application servers on software pods ( \(\mathit {deployment}_{sp}\) ) and we changed the deployments of pods to virtual machines ( \(\mathit {deployment}_{pm}\) ), as follows.

To answer our research questions (RQ1–RQ4), we designed and conducted a set of controlled experiments in two iterations. In each iteration, the three alternative software architectures (DDS, ADS-RS, ADS-CQRS) have been deployed in our testing environment. For each iteration and deployment architecture, we executed our multi-level scalability assessment (Section 4) with the two behavior mixes \(\Omega\) and \(\Omega ^{\prime }\) defined in Table 4 and increasing loads as in \(\Lambda\) (Equation (15)). In each testing session, we sampled a large amount of data to avoid the risk of obtaining results by chance. We collected \(\sim \!15k\) operation calls, excluding possible spurious data during the rump-up/ramp-down phases as recommended in [27]. Then, each session was repeated three times and the absence of statistical difference among consecutive sessions was assessed by conducting a pairwise comparison through the Mann-Whitney U Test. A total amount of \(\sim \!45k\) operation calls was sampled per each individual deployed architecture in each iteration. Overall, a total of 240 load testing sessions was executed throughout the two iterations.

In each iteration, we used the DDS as baseline deployment architecture \(\alpha _0\) executed under the baseline load \(\lambda _0=2\) from which a scalability requirement (1) was extracted. After executing the measurement workflow, we applied the analysis workflow to extract insights and possibly guide future iterations. Results from the first iteration were interpreted to plan and apply architectural changes that were assessed in the second iteration.

TrainTicket is an e-commerce application designed to manage train ticket booking for the Chinese train system. It exposes an end-point for Web browsers and a REST API gateway that conveys external requests to 41 internal services and 22 databases.

The application can be used by a number of actors that have access to the operations exposed by the API. As shown in Figure 22, the actors can be divided in three groups: (1) logged users who need authentication; (2) external users who do not need any authentication; and (3) admin users who have more privileges and can directly access to CRUD operations for management reasons. Both logged and external users are specialized into hs (high speed) and other depending on the kind of trains they search or buy (high speed or other, respectively).

In this case study, we focus on the application of our methodology to three selected bounded contexts that collectively compose the main business logic of TrainTicket:

Since we do not focus on the other (auxiliary) microservices, we considered their exposed operations (see Table 8) as a whole group hosted by a large execution unit running on a large set of resources (see Figure 25).

Taking into account the identified actors, we applied ADD to further decompose the three main bounded contexts into smaller microservices, as reported in Table 9. Figure 23 shows a schema of the ADD outputs, where the three initial bounded contexts are decomposed into smaller actor-driven microservices, following two dimensions of the Scale Cube model: functions (y-axis) and data (z-axis). For instance, \(\mathtt {Preserve}\) - \(\mathtt {hs}\) serves only \(\mathtt {Logged}\) users who are interested in high-speed trains. \(\mathtt {Travel}\) - \(\mathtt {hs}\) is another example of granular service that handles a subset of the whole travel dataset but serves both \(\mathtt {External}\) and \(\mathtt {Logged}\) users who can both search for high-speed trains.

As illustrated in Figure 23, ADD identifies finer building blocks implemented as microservices and deployed onto dedicated execution units. This enables a differentiated replication schema of the operations according to the expected load generated by different actors of the system.

To separate the inner logical levels of each microservice as well, we also decomposed the databases using the data-partitioning approach introduced in Section 3.2. The details of this decomposition and the corresponding deployment architecture are discussed in Section 6.5.

To address the research questions (RQ1–RQ4) and further generalize the previous results, we designed and conducted additional controlled experiments. We followed a number of iterations of our methodology with the same design used in the first case study (see Section 5). In each iteration, TT-DDS and TT-ADS have been deployed using the corresponding \(\mathit {deployment}_{sp}\) and \(\mathit {deployment}_{pm}\) , and we applied the behavior mix reported in Table 11 under the increasing load, according to \(\Lambda\) (Equation (16)). A total amount of \(\sim \!20k\) operation calls was sampled per each deployed architecture in each iteration. Table 11 reports the sample size of each operation per testing session. We used the starting point, TT-DDS with no business logic replication, as baseline deployment architecture \(\alpha _0\) and extracted the scalability requirement under the baseline load \(\lambda _0=5\) (i.e., the number of actors). In each iteration, the multi-level scalability assessment was executed to understand and improve the scalability of the system by refining the \(\mathit {deployment}_{pm}\) of each architectural solution.

For the sake of brevity, we do not provide the results of each iteration for this second case study. We present a summary of the final results obtained using \(\mathit {deployment}_{sp}\) in Figure 24 and \(\mathit {deployment}_{pm}\) in Figure 25. Figure 26 contains the plots obtained by applying the scalability assessment framework.

The system-level \(\mathcal {DM}\) in Figure 26(a) shows that both architectures can handle up to 100 concurrent users without failures. The total \(\mathcal {DM}\) of TT-ADS (0.657) is \(30\%\) higher compared with TT-DDS (0.941). The plot shows that TT-ADS is optimal up to \(\lambda = 200\) . Furthermore, for all \(\lambda \gt 200\) , TT-ADS is closer to optimal compared with TT-DDS.

Even though smaller services may lead to communication overhead, the finer resource management obtained through ADD yields higher scalability.

The scalability footprint in Figure 26(b) shows that TT-ADS yields a bigger area compared with TT-DDS. All of the operations but for \(p_1\) and \(t_1\) exhibit higher GSL value. Even though \(p_1\) and \(t_1\) exhibit better results in TT-DDS, the GSL values are very close to optimal also in TT-ADS. The TT-DDS polygon is also less regular than TT-ADS. This means that the components yield a very diverse scalability attitude since some of them cannot exploit the available resources (better allocated by using ADD).

Figure 26(c) and Figure 26(d) show a scalability gap and performance offset, respectively. Here, we can observe that the operation \(t_2\) is the most critical one since it exhibits the highest scalability gap. In TT-DDS, \(t_2\) fails at \(\lambda = 200\) with very high performance offset ( \(177\%\) ), whereas in TT-ADS it fails at the highest load \(\lambda = 350\) with small performance offset ( \(6\%\) ). This result is consistent with the \(\mathcal {DM}\) metric and explains the rapid degradation of TT-DDS from 150 to 200 concurrent users.

RQ1. To what extent can our scalability assessment workflow support decision-making over alternative microservices architectures at the system level?

The results of the controlled experiments show that both the domain metric and the derived scalability footprint are suitable quantitative instruments for providing engineers with the ability to compare alternative architectures at the system level. In both case studies, we consistently found that the initial microservices (identified through DDD) yield worst scalability for all of the usage profiles and deployment configurations. We also found an edge case in which the coarse-grained system-level analysis prevented us from recognizing differences between the target decomposed architectures. In the first iteration of our first case study (i.e., smart mobility application), we could not find differences between ADS-RS and ADS-CQRS under the unbalanced usage profile \(\Omega ^{\prime }\) . In this case, an analysis at finer granularity levels was required.

RQ2. To what extent can our scalability assessment workflow support resource allocation improvement at the microservice (or component) level?

According to our experience with the two case studies, the analysis workflow at component level can help achieve better performance and scalability by identifying a proper configuration in terms of software architecture, deployment, and resource allocation. For instance, during the first iteration with a smart traffic application, we identified the \(\mathtt {EndUser}\) component as the most critical one. This result guided the reallocation of the available resources to obtain scalability improvements. The component-level analysis shows that the reallocation is effective, especially with the architecture ADS-CQRS (under the unbalanced operational profile \(\Omega ^{\prime }\) ). The TrainTicket case study confirmed this result. The component-level analysis shows that the resource reallocation in TT-ADS is better compared with TT-DDS. In both case studies, the ADD yields higher scalability across operations of all components.

RQ3. Is the scalability assessment workflow able to spot scalability and performance issues at the operational level?

In both case studies, we were able to spot failing operations and measure their contribution to the overall scalability and performance degradation in each iterative step of our methodology. The results at the operational level show that the behavior of the failing operations is heterogeneous. By quantifying the scalability gap and performance offset, we were able to spot operations having a similar impact on scalability but different performance degradation. This highlights existing room for improvements by means of resource reallocation. For instance, with the smart mobility application, ADS-CQRS yields the same scalability gap for \(o_{10}\) and \(o_{11}\) (second iteration under profile \(\Omega ^{\prime }\) ). Nevertheless, \(o_{10}\) exhibits a very high performance offset compared with \(o_{11}\) .

RQ4. Can ADD practically drive further decomposition of bounded contexts yet enforce performance and scalability improvements?

The results of our controlled experiments show that ADD can be effectively used to further decompose microservices and achieve scalability improvements by taking proper architectural choices to better fit the target operational setting. The results obtained in our two case studies show that our methodology guided us towards a fine-grained decomposition yet enforced scalability improvements. In our experience, the analysis workflow provided us with decision support instruments able to compare alternative architectures and identify system components that require better resource allocation. Concerning practicability, we would like to point out that ADD can refine each bounded context that has limited dimension even though the size of the overall system is large. For instance, TrainTicket has 41 microservices, but we could selectively apply ADD to a subset of them. In this way, each bounded context can be refined by a different development and quality assurance team to split the effort and improve scalability.

External validity. External validity concerns the generalizabilty of the results. Threats in this category have been addressed by selecting two representative case studies in our target application domain, that is, performance-sensitive, service-based systems. One is representative of a wide class of modern data-intensive, edge-cloud systems whereas the other is a widespread benchmark microservices application in the e-commerce domain. We also chose a common technology stack that is modern enough to justify a decomposition through incremental refactoring actions rather than a complete rewrite and replacement. Our approach needs to work within a preproduction testing environment in which we can have control over the factors of interest. This setting is common with DevOps practices and tools in modern infrastructure that support continuous deployment [11].

Internal validity. Threats to internal validity are related to internal factors that could have influenced the results. To reduce threats in this class, we carefully designed our experimental campaign to control the factors of interest in both case studies. We controlled load, usage profiles, deployment architectures, and resources allocation. We avoided the adoption of auto-scaling mechanisms provided by our deployment infrastructure to reduce the risk of obscuring causal inferences due to uncontrolled events. This manual manipulation has been crucial to assess cause-and-effect relations between external factors and the effects measured by using the multi-level analysis. We also mitigated the effects of uncontrolled events as much as we could. The tests have been repeated multiple times and during all the tests we monitored the network to avoid anomalous traffic conditions (e.g., peaks) affecting the experiments’ results.

Conclusion validity. Since testing sessions were guided by stochastic sampling of synthetic users from multiple categories, there existed the possibility that results had been produced by chance. We addressed this threat by following the practical guidelines in [40]. We sampled a large number of invocations per individual operation by repeating each test session three times. We also conducted a pairwise comparison among samples between consecutive sessions to assess absence of statistical difference using the Mann-Whitney U-test. We also adopted this latter statistical test to compare the scalability footprints and compute the p-value with significance level \(\alpha = 0.05\) (detailed results are available in the dataset paired with this article). The Cliff delta has been adopted to measure the effect size. Finally, we use multiple visualization techniques complemented with quantitative analysis to draw our conclusions. This helped us to collectively agree on the interpretation of the analysis and collectively formulate the answers to the research questions.

Construct validity. The applicability of our multi-level analysis approach depends on the accuracy of the target operational setting under consideration, which determines the scalability requirement described in Section 4. Therefore, a careful analysis of the production usage is required. As discussed in [12], approaches to deal with this threat include: using related systems as a proxy for the SUT, conducting user surveys, and analyzing log data from an existing version of the SUT. Therefore, while some specific results about the quality of the considered architectures depend on the settings considered for the case studies, the overall methodology, decomposition strategy, and multi-level scalability assessment can be generalized.

Architectural patterns and antipatterns. A broad literature review of the challenges associated with the microservices architectural style can be found in [41]. According to the existing principles in microservices architecture, there are common mistakes occurring during the decomposition process. These pitfalls can be identified by looking at the so-called bad smells in the source code [42]. A number of bad practices, antipatterns, and their potential solutions have been collected by conducting interviews with experienced developers of cloud-native applications based on microservices [43, 44, 45]. The outcome is a taxonomy of bad practices and possible refactoring actions both from the organization perspective (e.g., organization of the development team) and from the technical perspective (e.g., code smells and communication issues). The work in [46] introduces guidelines for companies that need to migrate to microservices based on the analysis of a set of metrics they should collect before re-architecting their monolithic system. To extract these metrics, the authors conducted interviews with professionals to derive an assessment framework based on grounded theory [47]. As stated in [6, 43, 46], (anti) patterns and guidelines are system agnostic and can be adopted by companies to decide whether to adopt microservices or not and how to define migration plans by following successful stories from past experience. These approaches cannot be reasonably used to conduct a quantitative assessment of system-specific alternative architectures, which is instead the major goal of our methodology.

Automated decomposition of monolithic systems. Automated suggestions of candidate service cuts based on static analysis of a target codebase have been recently proposed [48, 49]. The authors propose the adoption of clustering techniques to extract microservices from monoliths by maximizing high cohesion and minimizing low coupling. These approaches do not take into account the impact of possible dependencies due to dynamically typed languages on architecture-level maintainability. Recent studies reveal that the impact is higher than that of explicit dependencies [50]. Other existing semi-automated approaches based on clustering recommend microservice candidates and suggest possible refactoring actions to be applied in a migration scenario [51]. Microservice candidates are selected by limiting the average development team size and the service size (lines of code) according to recent empirical studies [52]. Even though these approaches suggest possible decomposition strategies grounded on traditional architectural principles, they do not provide engineers with suitable methods to assess the given decomposition taking into account the expected operational setting and perceived quality of the product by end users. The Mono2Micro platform [53] applies semi-automated refactoring of Java monolithic applications into microservices. The approach uses static analysis if the codebase, combined with runtime dynamic analysis of the execution, traces to make recommendations in terms of groupings of classes in the monolith that can serve as starting points for the identification of bounded contexts. The Functionality-Oriented Service Candidate Identification framework [54] suggests service candidates by analyzing the execution traces using a search-based approach that optimizes three quality criteria of service candidates: independence of functionality, modularity, and independence of evolvability. CARGO [55] is a microservice partitioning and refinement tool that statically analyzes JEE applications to build a system dependency graph enriched with semantically meaningful relationships (e.g., call-return, and database transaction). This graph yields better community detection (candidate microservices) by reducing distributed database transactions.

These approaches are conceptually different from ADD since they automatically recommend candidate microservices starting from a monolithic system. Our approach can then complement these methods to refine the suggested microservices contexts taking into account the actors and their role.

Actor- and role-driven decomposition. Actor-based and role-based abstractions provide alternative perspectives to develop decomposition processes. Even though the notion of actor is an established modeling concept used to decompose and modularize complex service collaborations —for instance, in business process management [56] and multi-agent systems [57] — it received less attention for microservices systems. Mainstream approaches focus instead on the notion of bounded context. For this reason, there is a lack of knowledge regarding how actors should be systematically integrated in established decomposition frameworks as well as the impact of this integration. Workload data–aided approaches have been proposed [44], still with the ultimate goal of identifying the domain entities that may collectively represent the bounded contexts of a monolithic system. ADD is conceptually different since it assumes the existence of bounded contexts and refines them according to the actors and their role, that is, the embodiment of the participation of an actor [58].

The systematic gray literature review in [4] recognizes performance testing and scalability analysis of microservices as painful activities. Motivated by this issue, the research community conceived of novel methodologies that provide engineers with decomposition decision support based on quantitative analysis of nonfunctional qualities. The process presented in [59] evaluates a target architecture by balancing nonfunctional requirement satisfaction at the level of individual microservices as well as the overall system. The authors introduce a managing subsystem (i.e., adaptation layer) that monitors the trade-off and assists the managed microservices in achieving such a balance. However, the managing layer can introduce non-negligible overhead that could even limit performance or scalability. Other common approaches based on runtime management of resources are those enacting auto-scaling mechanisms. The work presented in [60] addresses the problem of selecting appropriate performance metrics to activate such mechanisms. The authors investigate the use of relative and absolute metrics to understand when to activate the appropriate actions. In contrast to our approach, these frameworks work at runtime along with the target system in production. In principle, they could complement our assessment methodology since they assume the existence of a microservices system, whereas we aim at guiding towards the “right” architecture before the production phase.

The conceptual framework introduced in [5] envisions an iterative technique to guide the migration steps by continuously monitoring a system in production in order to collect operational data and use performance analysis as feedback for the decomposition process. Following the same principle, our assessment methodology leverages instead the runtime evidence collected through load testing. Our scalability assessment framework is grounded on the quantitative system-level domain metric introduced in [12, 21]. This approach aims at evaluating the performance of alternative deployment configurations for microservice systems. Performance here is defined as a function of the workload intensity as originally introduced in [61]. The PPTAM software tool implements the domain metric assessment method and has been presented in [33]. The domain metric approach as well as its own modifiable software tool [62] turned out to be effective in evaluating microservice architectures based on the notion of scalability requirement [21], as a performance threshold empirically extracted by observing the responsiveness of individual services under “no load” [63, 64, 65]. A first draft of our vision has been introduced in [8]. This work describes a scalability assessment methodology able to guide the decomposition process through coarse-grained information extracted by applying the domain metric to evaluate the target system as a whole. Our assessment methodology extends this approach by introducing a multi-level analysis method to extract insights on the target application from different angles: system, service, and operation. The idea is to aid engineering decisions at different granularity levels in order to choose among alternative decomposition strategies as well as extract insights for resource allocation.