Regular Paper Evaluation of Service Oriented Architecture-Based Federated Architectures Ashraf M. Abusharekh, Lawrence E. Gloss, and Alexander H. Levis* System Architectures Laboratory, Department of Electrical and Computer Engineering, George Mason University, Fairfax VA 220304444 EVALUATION OF SOA-BASED FEDERATED ARCHITECTURES Received 28 November 2008; Revised 9 September 2009; Accepted 22 November 2009, after one or more revisions Published online in Wiley InterScience (www.interscience.wiley.com) DOI 10.1002/sys.20162 ABSTRACT This paper discusses the specifications, methods, and constructs to implement end-to-end Service Oriented Architecture (SOA)-based systems engineering across a federation of information domains. It addresses the necessity and benefits of a repeatable service design framework and its ability to consistently yield quantifiable results for SOA performance evaluation. An illustrative example of the approach is presented. © 2010 Wiley Periodicals, Inc. Syst Eng 14: Key words: zaq;2 tive, as an architectural discipline service-oriented architecture lacks a quantitative design construct. The ability to consistently “architect” and “federate” necessary and effective information sharing capabilities poses many technical and managerial challenges. There are no governing structures to consistently federate knowledge sharing across different information domains. There is also no guarantee of consistent knowledge sharing, there are no readily applicable Trust and Discovery models and standards, and there is no guarantee that different service “consumers” and “providers” can interoperate and share information across multiple information domains to achieve a Community of Interest (COI) operational objective. Unless a quantifiable framework is developed for designing service environments across a federation, the risk of “SOA stovepipes” is all too real. A working model for SOA-based systems engineering includes the key functions of design, evaluation, and deployment for single SOA’s as well as federated ones. “Service-oriented” performance can be quantified in terms of three key factors: loose coupling, service linkages with supporting applications, and SOA infrastructure specifications. Because of the dynamics of these three elements, traditional “stovepipe” architecture design, evaluation, and deployment are no longer relevant. We introduce an Enter- 1. INTRODUCTION Service-oriented design constructs are viewed by IT professionals as an effective way to align their IT architecture to emerging business opportunities. The purported advantages of Service Oriented Architecture (SOA) include lower costs, quicker integration, increased federation through common interfaces, and greater architectural adaptability. Despite the compelling potential of SOA, there currently is no repeatable design model to support “cradle to grave” SOA systems engineering across a federation of information domains. Federated domains conform to common or shared architecture standards across Communities of Interest [GIG, 2007]. While the qualitative potential of service-oriented design is attrac- *Author to whom all correspondence should be addressed (e-mail: alevis@gmu.edu; aabushar@gmu.edu). Contract grant sponsor: Office of Naval Research; contract grant no.: N00014-08-1-0319 Systems Engineering © 2010 Wiley Periodicals, Inc. 1 2 ABUSHAREKH, GLOSS, AND LEVIS Figure 1. Steps in designing DoDAF compliant architectures. prise Service Bus (ESB)-enabled, event-driven SOA for our design framework. The discrete event simulator OMNet++1 provides the behavior environment in which SOA-based performance is evaluated. Thus, a framework wherein servicebased information solutions can be evaluated across a federation is established. By demonstrating how service-oriented architectures can be evaluated in quantifiable terms, SOA architects will have a reliable tool at their disposal for evaluating design options and overall service performance. 2. INFORMATION SYSTEM ARCHITECTURES Architectural models provide insight into the structural, logical, and behavioral aspects of the architecture. Systems architecting [Maier and Rechtin, 2000; Levis, 1999; Levis and Wagenhals, 2000] is part of a larger system engineering discipline and relies on methodologies that have been developed over time. Architects can leverage several architecture design methodologies. Structured Analysis is based on functional decomposition while Object Orientation is based on the concepts of objects and classes. A third approach, the Object Process methodology, is an approach for designing information systems that integrates both structure and behavior in a single model [Dori, 2002]. In a series of papers [Bienvenu, Shin, and Levis, 2000; Wagenhals et al., 2000; Wagenhals, Haider, and Levis, 2003; Abusharekh et al., 2007; Wagenhals and Levis, 2009] Structured Analysis, Object Orientation, and, to a small extent, Service Orientation have been used to design and evaluate architectures that are compliant to the DoD Architecture Framework [DoDAF, 2007]. The end product in these cases is an executable model of the architecture derived from information contained in the framework’s artifacts. These artifacts (or products) describe the structure, data, and rules that manipulate the data to accomplish tasks. An executable model, if derived from these products in a trace1 OMNet++ (Objective Modular Network Test-bed in C++) is a publicsource, object-oriented, modular, discrete event network simulator that offers a simulation library. The library includes support for input/output, statistics, data collection, graphical presentation of simulation data, random number generators, and data structures. Systems Engineering DOI 10.1002/sys able way, can enable logical, behavioral, and performance analyses of not just one domain, but behavior across federated domains. Also, it can help verify that the combination of rules, data, and structure works (i.e., the rules are consistent and complete). Simulation runs find logical and behavioral flaws in the architecture and validate whether the architecture does what it is supposed to do. Because these models typically reference a “systems of systems” approach within a single architecture—that is, they typically define and share information across a common ontology—the models often fall far short in terms of providing behavioral insight across multiple information domains. Service-oriented design promises to solve enterprise interoperability, business and IT agility and flexibility, and resource reuse problems currently inherited from stovepipe architectures. According to Erl [2005], a “stovepipe” architecture is an architecture solely tied to one and only one application and typically relies on a finite number of resources tied to that one application. A stovepipe architecture is tightly coupled with its business rules, underlying business logic, and supporting data. SOAs [Erl, 2005; Hurwitz et al., 2007], however, utilize coarse-grained, loosely coupled, and reusable services. These services interact through the Enterprise Service Bus (ESB), which provides a highly distributed, event-driven SOA that combines Message Oriented Middleware [MOM), orchestration and process flow, intelligent routing based on content, and data transformation. Service-oriented architectures can exist without an ESB. However, using an ESB makes them more efficient, scalable and more reliable [Chappell, 2004]. SOA behavior and performance not only depend on business services, but also on the infrastructure that enables loose coupling, services implemented by other systems, and the underlying technological network supporting the SOA environment. Consequently, traditional executable models synthesized for architectures of tightly integrated systems (stovepipes) cannot capture easily the complexity and dynamism of an SOA. Figure 1 [Wagenhals and Levis, 2009] depicts one option for architecture-based systems engineering. Within this view, it is not possible to derive the executable model from the Operational Architecture View (OV) alone. Such a model can EVALUATION OF SOA-BASED FEDERATED ARCHITECTURES only address logical and behavioral issues. To address performance questions, the Systems View (SV) of the architecture is also required. The executable model also requires some additional information not typically specified in architecture descriptions. As effective as this model is, particularly in capturing both structured analysis and object-oriented based systems engineering, additional evaluation constructs are necessary when designing in a service-oriented environment. This need is particularly acute when attempting to evaluate the performance of a service oriented design across a federation of information domains. We offer an approach which ultimately supports SOA design, evaluation, and development not just within a single domain, but across a federation. Before progressing further, it is appropriate to actually define the term federation. We consider federation to mean a union of SOAs that have established an operational relationship (e.g., producer and consumer). Each SOA has its own particular ontology and associated vocabulary, and to be federated there must be a set of rules and procedures to share business services and allow for business processes across SOAs. The functions for accommodating and managing such across federation business processes include: 1. Federation level business process modeling, orchestration, and monitoring 2. Federation level messaging, routing, and event management 3. The ability to discover appropriate business services while maximizing cross federation business process efficiency 4. Orchestrating federation wide rule sets and business logic according to predefined service and performance level agreements [Khan, 2004] 5. Federation-wide security capabilities to authenticate and authorize service participants in accordance with current security technologies, policies, and standards [Nadalin et al., 2006]. This paper presents an evaluation model for service oriented architectures within a single domain—this will serve as a necessary building block for a larger discussion and demonstration of federated SOAs. Section 3 contains a discussion on SOA performance evaluation, prediction, and future trends within service orientation. Section 4 presents an approach for evaluating the performance of composite business processes that are orchestrated by an ESB-enabled event-driven SOA. The performance prediction that forms the basis for the evaluation is achieved by modeling a generic ESB-enabled SOA in a discrete event system simulator [OMNet++, 2007] and hosting business services and business processes (produced during the Architecture Design phase) in the SOA model. Section 5 presents and explains the structure of an ESB model with an illustrative example discussed in Section 6. The paper concludes with analysis and results in Section 7 and conclusions in Section 8 concerning the necessary conditions and specifications for service oriented performance prediction. 3 3. RELATED WORK Liu, Gorton, and Zhu [2007] present a performance modeling and benchmarking approach that facilitates estimating performance characteristics of the ESB and analyzing the relationship between the ESB and its composite applications. Their approach follows the general capacity planning process introduced by Menascé and Almeida [2002] and uses queuing networks for performance prediction. To simplify the analysis, this work focuses only on the performance of ESB routing and transformation; this model does not incorporate orchestration service, MOM service, or modeling of the technological network. Sloane et al. [2007] present a hybrid approach to modeling SOA systems of systems in which two separate models are developed: a Colored Petri Net (CPN) model captures internal protocols, communications, and resource consumption; a discrete event simulation system model Modeling Environment for SOA Analysis or MESA captures the interactions between nodes in a SOA environment. The models capture the behavior and performance of a Multi-Channel SOA discovery process. Although called a hybrid approach, the CPN and the MESA models are completely isolated, difficult to abstract to capture different behaviors, and they do not address completely the effect of the technological network on the behavior and performance of an SOA. Finally, the approach does not capture business processes, the very driver of employing service orientation in the first place. Shin and Levis [2003]zaq;1 combine CPN and network simulator models to gain insight into the behavior and performance characteristics of architectures. This approach models the functional and physical layers of a system as two separate executable models: the functional and physical models. The functional executable model is a CPN and the physical (communication) model is a queuing net modeled using the ns-2 network simulator [The Network Simulator, 2007]. The end product is a CPN performance prediction model constructed from a distributed simulation system (consisting of the aforementioned executable models). This distributed simulation runs offline (i.e., there is no message exchange between the two executable models during run-time). Since network delays are unknown at this point, the Occurrence Graph (OG) of the functional (CPN) executable model captures the precedence relation between events with the unknown delays. If the transition firing sequence is unknown, the designer has to examine all possible sequences as captured by the full OG of the CPN. Once the network delays are computed from the physical executable model, these delays are passed back to the functional executable model. Running the functional model after incorporating the delays allows for both logical and performance analyses. Our challenge is extending this model into an ESB-enabled service oriented architecture. We do that by using CPN models to evaluate the logic and behavior aspects of the business processes and business services. We use these CPN models as Service Processor modules in the OMNeT++ simulations. Due to the modular architecture of OMNeT++, the designer can replace the ESB model entirely or replace specific ESB services as needed to reflect his/her desired behavior. Systems Engineering DOI 10.1002/sys 4 ABUSHAREKH, GLOSS, AND LEVIS The present work provides a methodology to evaluate federated SOAs at their early design stages; the resolution of additional challenges is required to have a reliable evaluation methodology. Some of those challenges, to be addressed in our future papers, include (a) extending the ESB model to be able to run multiple business processes spanning multiple SOA implementations and (b) addressing Quality of Service management issues (including QoS negotiations and failure recovery). 4. APPROACH The structure of the SOA environment is shown in Figure 2 and is composed of three layers. The Operational layer contains the business processes, defined as a composition of single and/or composite services. The Service layer contains the business services and the ESB services. The ESB is composed of the Message Oriented Middleware (MOM), the SOA supervisor, the Orchestration service, the Registry service, and the Service Containers. Details of the ESB model will be discussed in Section 5. The Physical layer contains physical nodes (workstations, servers, network nodes, etc.) and communication networks needed for this SOA environment. Our evaluation occurs within this environment. The three main performance areas of interest are also shown in Figure 2. Layer 3 covers performance parameters and indicators for the business processes in the operational layer. The main performance metrics of interest in this area are the “creation time” defined as the time between submitting a request for a business process instance and the time the business process instance is ready to execute, and the “endto-end response time” of a business process. These parameters and indicators depend on Layers 2 and 1, which represent services and the technological network, respectively. Layer 2 contains performance parameters and indicators for services, divided into two sublayers: 2.1 (ESB performance parameters and indicators) and 2.2 (business services performance parameters and indicators). Layer 1 has parameters and indicators for the physical infrastructure, which includes bandwidth, throughput, and utilization of the various network resources. Our architecture design phase produces a set of business services and business processes to run on top of an ESB-enabled, event-driven federated SOA. The goal is to compute the creation time and end-to-end response time of business processes deployed in such an environment. These will depend on the following factors: 1. The behavior and performance of the ESB services that enable the execution of the business process (performance sublayer 2.1 across the information domains 2. The behavior and performance of the individual business services contributing to the business process (performance sublayer 2.2 3. The underlying technological network supporting the SOA (performance Layer 1) environment 4. The request load of business processes deployed on the SOA at any given time. Figure 2. Our SOA design environment. Systems Engineering DOI 10.1002/sys EVALUATION OF SOA-BASED FEDERATED ARCHITECTURES 5 Table I. The ESB Profile Parameters a Heartbeat is a periodic synchronization signal used by software and hardware. In order to capture the above factors and the related performance characteristics of the environment, five profiles are used: 1. Network profile captures the underlying technological network including the network topology and the various network resources and devices deployed to support the SOA environment. In addition, the network profile contains the configuration of the nodes hosting the ESB and business services. The main source of the network profile is the Physical view of the architecture developed during the Architecture Design Phase. Multiple network profiles can be defined to evaluate the sensi- 2. 3. 4. 5. tivity of the SOA implementation across the federation (of network profiles). ESB profile captures the processing delays of the ESB services. Table I shows the ESB profile parameters and their descriptions. Business Services Profile captures the processing delays of the Business Services. Business Processes Profile describes the business processes as a composition of business services. The main source of this profile is the Functional view of the architecture, as developed during the architecture design phase. Scenario profile is the request load of the business processes to be deployed on the SOA. Figure 3. Model construction and evaluation approach. Systems Engineering DOI 10.1002/sys 6 ABUSHAREKH, GLOSS, AND LEVIS The aforementioned profiles are used during different steps in the executable model construction and evaluation of architecture phases as shown in Figure 3. Our evaluation profiles support different phases of the overall federated SOA evaluation. Our approach assumes that an architecture has been designed and that its description consists of a set of static views describing the logic and behavior of the business services and business processes and a physical view. The approach may be described by the following steps: Step 1. Synthesize the Executable Model. Write, compile, and link C++ code. 1.1. Define, write, and build in OMNeT++ the business services modules to be hosted by the ESB-enabled SOA model. These modules are written in C++ and capture the logical and behavioral aspects of the business services. The details of the logic and behavior of these services come from the functional view of the architecture. A detailed description of the ESB model is presented in Section 5. 1.2. Define the network topology, communication resources, devices, protocols, etc. (Network profile) in OMNeT++ using the NED language and OMNeT++ configuration files. Step 2. Configure Executable Model. 2.1. Configure in detail the communication resources, devices, and protocols (network profile) and add specific details in the OMNeT++ simulation configuration files. 2.2. Configure in detail the ESB and business services. This involves: 2.2.1. Register and initialize the ESB services and configure them accordingly. 2.2.2. Registering the business services in the ESB and configuring ESB services accordingly. 2.2.3. Configure the processing delays of different ESB and business services (ESB services and Business Services Profiles) and add specific details in the OMNeT++ simulation configuration files. 2.3. Configure the business processes by registering the business processes in the ESB and configuring the ESB services accordingly. Registering business services (2.2.2) and business processes (2.3) can be done online while running the simulation. Online registration of ESB and business services will be discussed in Section 5 in detail. Step 3. Run Executable Model. The simulation model is executed and results are collected. This step needs a scenario profile to define the request loads on different business processes and within different classes of a business process. Step 4. Evaluate Results. The results of the execution are analyzed and assessed against performance requirements. Due to the centrality of the ESB as a service layer and as a mediator across the operational and physical layer, particular attention must be paid to its development. In the next Systems Engineering DOI 10.1002/sys section, the development of the ESB executable model is described in detail. 5. ESB MODEL CONSTRUCTION We developed and constructed the OMNeT++ simulation environment and the INET2 framework [OMNet++, 2007]. This environment serves as our executable model and it can be used by any interested practitioner. The SOA ESB provides business process orchestration, management of the SOA environment, Quality of Service management, and monitoring. Thus, the generic ESB model developed in OMNeT++ contains a Registry service needed to publish, discover, and consume services in the SOA. The ESB model also includes a Message Oriented Middleware (MOM) service responsible for reliable messaging between services, as well as a SOA supervisory service responsible for monitoring the status and Quality of Service of the business processes and services. Finally, the ESB model incorporates an orchestration service to execute composite processes and services and the Service Containers to manage the endpoints of services. The ESB node structure is shown in Figure 4. At the Data Link Layer it supports two interfaces, an Ethernet 802.3 interface, and a point-to-point (PPP) interface. It supports IPv4 at the Network Layer and supports TCP and UDP at the Transport Layer. At the application layer, different ESB nodes have different application-specific modules. The structure of the Application Layer of each of the ESB components contains three modules: an ESB service specific service container, the Service Processor which implements the logic and behavior of the ESB service, and a repository which is used to store information relevant to the component. Figure 5 shows the OMNeT++ implementation of the ESB MOM node which contains the application layer module “mom,” the transport layer modules “tcp” and “udp,” the network layer module “networkLayer,” and the data link modules “ppp” and “eth.” The application layer module “mom” is shown in Figure 5(b). The internal submodules of the application layer of the node are: Figure 4. ESB service node structure. 2 The INET framework is an open-source communication networks simulation package for the OMNeT++. It contains models for several network protocols including IP, TCP, UDP, Ethernet, PPP and MPLS, etc. EVALUATION OF SOA-BASED FEDERATED ARCHITECTURES 7 Figure 5. ESB MOM service node. 1. The “momApp” module, a MOM specific service container. 2. The “momProcessor” module implements the behavior of the MOM service. 3. The “repository” module, a MOM specific repository that stores message flows, MOM service requests, etc. The ESB Registry maintains all active ESB business services, and business process descriptions deployed across the SOA. The Registry’s repository tables and their description are listed in Table II. The services provided by the ESB Registry are: 1. ESB nodes and business services registration—all services are initialized with the address of the Registry. As soon as they are deployed, they start the registration process in which the service information is stored in the Registry repository and is issued a service node identifier that uniquely identifies that service node throughout the federated SOA. 2. Business process registration—the Registry maintains descriptions of all business processes that run in the SOA environment. 3. Respond-to-service-nodes-query requests—a service node can request information about other service nodes providing specific services or information about business processes from the Registry. 4. The Registry logs all requests, including registration and service node queries for performance monitoring. The ESB MOM manages and maintains all message flows between producers and consumers in the SOA. The current implementation only supports request-reply message flows. The MOM’s repository tables and their description are listed in Table III. The services provided by the ESB MOM are: 1. Configure Message Flows—a service node can request a request-reply message flow to be configured by the MOM. Table II. ESB Registry Repository Systems Engineering DOI 10.1002/sys 8 ABUSHAREKH, GLOSS, AND LEVIS Table III. ESB MOM Repository 2. The MOM logs all requests for message flows, and all routed messages requests information for performance monitoring. The ESB Orchestrator manages, executes, and maintains all business process instances deployed on the SOA. The Orchestrator’s repository tables and their description are listed in Table IV. The services provided by the ESB Orchestrator are: 1. Response to business process instantiation and execution requests—composite service nodes send requests for instantiating and executing business processes to the Orchestrator; the Orchestrator satisfies these requests and sends back the results of the execution to the requesting service nodes. The instantiation of business processes requires the Orchestrator to: (1) query the Registry for service nodes that can participate in the business process instance; (2) request message flows from the MOM; and (3) request the SOA Supervisor to monitor the execution of the business process instance. 2. Keeping logs of the service requests for performance monitoring of the instantiation and execution of business processes instances. The ESB Supervisor monitors all business service nodes and business processes instances deployed on the SOA. The Supervisor’s repository tables and their description are listed in Table V. The services provided by the ESB Supervisor are: Table IV. ESB Orchestrator Repository Systems Engineering DOI 10.1002/sys 1. Monitor business service nodes status and response times. 2. Monitor Business Process instances status and end-toend response time. Single and composite business service nodes are monitored through their Service Containers, which send performance information periodically through heart beat messages. The ESB Service Container maintains and manages the endpoints of a business service. It keeps track of the requests made to the business service and their response time and periodically sends heart beats to the SOA Supervisor, indicating its performance status. The Service Container’s repository tables and their description are listed in Table VI. The ESB Service Container: 1. Registers the Business Service Node with the hosting SOA 2. Manages Business Service Node endpoints 3. Monitors the Business Service Node performance 4. Sends periodic performance and status information to the Supervisor. The ESB model has three initialization phases: ESB initialization phase, Business Services initialization phase, and Business Processes initialization phase, all of which can be examined through simulation. The goal of the simulation is to identify the main performance drivers in these initialization phases. In the ESB initialization phase, different ESB Service nodes are activated, become aware of each other, and are ready EVALUATION OF SOA-BASED FEDERATED ARCHITECTURES 9 Table V. ESB Supervisor Repository Table VI. ESB Service Container Repository to host business services. In the Business Service Initialization phase, Business Services start to join the SOA, register with the ESB registry, and are informed of the other ESB service nodes. In the Business Processes initialization phase, Business Processes are registered by service nodes, and the ESB Orchestrator is informed and ready to host their instances. The sequence of messages exchanged during the initialization phase of the ESB model is shown in Figure 6. Each node is initialized with the ESB Registry address beforehand. As soon as a node is started, it sends a Service Registration Request message to the ESB Registry. The ESB Registry issues a unique service id, stores the services information in its Repository, and sends back to the requesting Service a Service Registration Response message containing the assigned service id and the information of all ESB nodes already registered. Upon receiving the Service Registration Response, the service node updates its id, stores the information about Figure 6. Sequence of messages during the initialization of the ESB. Systems Engineering DOI 10.1002/sys 10 ABUSHAREKH, GLOSS, AND LEVIS other ESB nodes in its repository, and acknowledges the registration by sending a Service Registration Acknowledgment message back to the ESB Registry. When the registry receives such an acknowledgment from any node, it publishes the update by sending a Service Information Update to all registered nodes informing them of the newly registered ESB service node. By default, all registering ESB nodes and business services are subscribed automatically to the ESB Service Updates topic, which publishes information regarding any change to the status of the ESB nodes. These changes include newly registered ESB services as well as failing ESB services. Business services initialization is similar to the ESB initialization except for how the information is stored in the ESB registry’s repository. For the purpose of this paper, the initialization of the ESB and business service nodes will be done online during the simulation, while the business processes initialization will be done through configuration using the business processes profile. Details of the behavior of the ESB model will be further explained with the help of an illustrative example in the next section. This example is then used to carry out computational experiments to illustrate some of the types of possible analyses. 6. ILLUSTRATIVE EXAMPLE We used a lease application approval business process to illustrate the approach outlined in the previous section. A customer submits his lease application containing his name, social security number, and the lease amount through the Lease Application service. The application gets processed, and the approval or disapproval of the application is returned to the customer through the same service (Lease Application service). The business logic (operational concept) of the lease application approval business process consists of the following: 1. All lease applications with lease amount less than $25,000 are processed by the Normal Lease service and automatically approved, and the approval is sent back to the Lease Application service. 2. For lease applications exceeding $25,000, the applicant’s credit score is first retrieved by invoking the Credit Score service. If the applicant’s credit score is above 600, the application is routed to the General Lease service for processing, and the approval is sent back to the Lease Application service; otherwise the application is processed by the Loan Review service and disapproved. 3. A lease application can take one of three paths in the business process denoted as Class 1, 2, and 3. a. In Class 1, the lease amount is greater than or equal to $25,000 and the applicant credit score is greater than 600. b. In Class 2, the lease amount is greater than or equal to $25,000 and the applicants credit score is less than or equal to 600. c. In Class 3, the lease amount is less than $25,000. The structure of the service node is almost identical to that of an ESB node except for the Service Container and the Repository, which are the same for all service nodes. The Service Processor module is the implementation of the behavior of a service node and is written in C++ for each service. The business process is shown in Figure 7. The approval process requires specific business services as described in Table VII. For each business service in Table VII, a single service node is created in the simulation. Table VIII lists the business service profile parameters and the relationship between processing nodes and any associated processing delay. The business process profile describes the business processes to be deployed on the SOA. In this example only one business process is described in the profile. The profile is used to initialize the business process tables of the ESB Registry and Orchestrator nodes. Figure 7. Lease application approval process. Systems Engineering DOI 10.1002/sys EVALUATION OF SOA-BASED FEDERATED ARCHITECTURES 11 Table VII. Business Service Descriptions The Lease Application Approval business process has been configured (registered) in the ESB Registry and the Orchestrator services. The Lease Application Service requests a business process instance by sending a Request Business Process message to the ESB Orchestrator. This message contains the name of the business process to be executed and the parameters; in this case the parameters are the customer’s ID, Name, and amount requested. The ESB Orchestrator looks up in its repository the requested business process. The business process definition stored in the Orchestrator’s repository contains the names of the services participating in the business process rather than specific service nodes. Thus, after reading the business process definition, the Orchestrator constructs a Request Service Nodes message and sends it to the Registry requesting service nodes that can participate in the business process instance. The Registry looks up the service nodes to satisfy the request in its registry and responds back to the Orchestrator with a list of service nodes that can participate in the business process instance. After receiving the service nodes, the Orchestrator requests message flow between itself and the service nodes from the MOM by sending a message to the MOM. Figure 8 shows the interactions within the SOA environment (ESB nodes and business service nodes) while executing an instance of the lease application approval business process. The MOM configures the requested message flows and responds back to the Orchestrator with a Response Message Flow message containing a list of message flow handlers. The last step in the instantiation of a business process instance is to request that the SOA supervisor monitor the business process instance by sending a Monitor Business Process Instance message to the SOA Supervisor containing the newly created business process instance handler. As soon as the Orchestrator receives an acknowledgment from the SOA Supervisor, it sends the requesting service a Business Process Instance Created message, informing it that the business process instance had been successfully created, after which the Orchestrator starts executing the business process instance by evaluating the first decision box. If the amount is less than $25,000, the Orchestrator requests the service of the Normal Lease Node by creating a Request Service message attaching the message flow id and sending it to the MOM, which in turn routes the message to the Normal Lease Node. The Normal Lease Node responds to the request and sends back a Response Service message, which indicates the approval of the application, to the MOM which reroutes the response message back to the Orchestrator. The Orchestrator creates a Response Business Process message and forwards it directly to the requesting Lease Application node. Otherwise, if the amount exceeds $25,000, the service of the Credit Score node is requested through the MOM. If the customer’s credit score Table VIII. Business Service Profile Parameters Systems Engineering DOI 10.1002/sys 12 ABUSHAREKH, GLOSS, AND LEVIS Figure 8. Sequence of messages during the execution of one business process instance. is greater than 600, the General Lease node service is requested in the same fashion, and finally an approval Response Business Process message is sent to the Lease Application node. Otherwise, if the customer’s credit score is less than or equal to 600, the Loan Review service is requested, and a disapproval Response Business Process message is sent to the Lease Application node. 7. RESULTS OF THE COMPUTATIONAL EXPERIMENT Our federated SOA model hosted the lease application approval business process using the two different network profiles: network profile A is shown in Figure 9(a), and profile B is shown in Figure 9(b). All parameters in the two network profiles are the same except for the network topologies; in particular, one router was added to network profile B with a bottleneck link (1Mbps) between routers A and B. The objective of the computational experiment was to investigate the sensitivity of performance to changes in the network topology. How do federated SOA topologies impact information sharing? In order to test the sensitivity of the end-to-end response time of the business process to changing Systems Engineering DOI 10.1002/sys network topologies, all service node processing delays in the ESB processing profile and the Business Services processing profile were set to zero. The heart beat interval of Service Containers was set to 3 s. One set of Scenario profiles was used with each network profile in which we varied the Lease Application Interval to generate request rates ranging from 100 to 850 requests/s. Table IX shows the scenario profile parameters and their description. The total number of business process instance requested in all sets was 3000 requests. In all simulations, all 3000 requests were of a single class (i.e., only one class of the business processes was requested by the Lease Application Node). Figure 10 shows the end-to-end response time of individual business process classes, under network profiles A and B, as a function of the request for service rate. This is the key result in terms of performance. The results for network profile A (single router) shown in Figure 10(a) are straightforward: The Class 1 process has the smallest end-to-end response time; the two other classes have virtually identical longer response times. Class 1 involves only one business process while Classes 2 and 3 involve two business processes. The results for network profile B [Fig. 10(b)] show the effect of the bottleneck in the link between the two routers A and B. EVALUATION OF SOA-BASED FEDERATED ARCHITECTURES 13 Figure 9. Network topologies used in the simulations Table IX. Scenario Profile Systems Engineering DOI 10.1002/sys 14 ABUSHAREKH, GLOSS, AND LEVIS Figure 10. End-to-end response time. [Color figure can be viewed in the online issue, which is available at www.interscience.wiley.com.] The bottleneck significantly increases the end-to-end response time of all the individual business process classes. Note that the response time range for network profile A is 0 to 6000 ms, while for network profile B the range is from 14,000 to 40,000 ms. In order to interpret further the results, it is necessary to decompose the end-to-end response time of a business process instance (a particular class i) into its constituent components: request time, creation time, execution time, and reply time as shown in Tstep = Tcreation + Texec + Trequest + Tresponse, (1) i where Tcreation is the time it takes to create the business process instance, Texeci is the time it takes the Orchestrator to execute the business process instance of class i, Trequest is the time it takes a business process request from the requester to reach the Orchestrator, and Tresponse is the time it takes for the response from the Orchestrator to reach the requester. The creation time Tcreation, can be further decomposed, as shown in Eq. (2), into the communication and processing delays occurring when the MOM (TcMOM), Registry (TcRegistry), and Supervisor (TcSuper) configure the newly created business process instance: Tcreation = TcMOM + TcRegistry + TcSuper, (2) where TcMOM = TMOMRequest + TMOMConfigure + TMOMResponse, (3) TcRegistry = TRegistryRequest + TRegistryQuery + TRegistryResponse, (4) Figure 11 shows the overall creation time Tcreation and its components (for nonconcurrent class execution under network profiles A and B). The business process instance creation time Tcreation clearly depends on performance Layers 1 and 2.1 of the SOA infrastructure as shown in Figure 2. One could conjecture that the creation time would be the same for all business classes because they all involve the same infrastructure entities. However, this is not true. Figure 11(a) shows Tcreation under network profile A in which business Class 1 has smaller creation time than the other two. This is due to the number of business services involved in the business class execution. In Class 1, the execution of the class imposes less service-based message traffic on the network as compared to Classes 2 and 3, which affects the creation of new business instances and additional business processes. Figure 11(b) shows Tcreation for network profile B. The effect of introducing the bottleneck link on the overall Tcreation is evident in Figure 11(b); it adds more delay and also regulates traffic between the two subnets. The effect of the latter is to produce comparable performance for all three business classes. Figure 12 shows the execution time of individual business process classes under network profiles A and B. The behavior observed in evaluating creation time occurs here too. Since execution involves the invocation of business services, Class 1 is expected to have a shorter execution time since a single service is involved while the other two classes invoke two services. The business process execution time Texeci of class i involves requesting m business services; it can be expressed in the following way: Texec = ∑ {TMOMRequest + TMOMRoute+ TMOMServiceRequest j i j TcSuper = TSuperRequest + TSuperConfigure + TSuperResponse. (5) + TService + TService MOMResponse + TMOMRoute j The processing delays shown in Eqs. (3)–(5) are captured in the ESB profile. Systems Engineering DOI 10.1002/sys j + TMOMResponse + TOrchestorProcessing } j (6) EVALUATION OF SOA-BASED FEDERATED ARCHITECTURES 15 Figure 11. Creation time. [Color figure can be viewed in the online issue, which is available at www.interscience.wiley.com.] where j = 0, 1, 2, …, m and TMOMRoute is the time it takes the MOM to route the message flow, TMOMServicejRequest is the time it takes for the MOM’s request to reach Servicej, TServicej is the time taken by the service j to process the request, TServicejMOMResponse is the time taken for the service’s response to reach the MOM, and TOrchestorProcessingj is the time it takes the Orchestrator to process the service’s response. Since the business services processing times (i.e., the operational layer processing delays) were set to zero, all the previous results reflect the delays imposed by the technological network including the network stack of the nodes hosting the service. The network topology (1 Mbps bottleneck link between Router A and Router B in network profile B) affects the business process end-to-end response time. Therefore, consideration of alternative network topologies is essential for evaluating service-oriented designs. If the Lease Review service in the lease example is an interface for a human decision maker who is supposed to setup an interview with the applicant after which he decides to approve or disapprove his application, the delay introduced by the network will be insignificant when compared to that of the human part of the process. This makes particular sense for critical systems. These results are based on hosting the business process on the previously defined ESB driven SOA. If the developer/designer is planning to deploy the business process on a different ESB, he has to develop his model of the ESB driven SOA environment. This motivated the use of the modular network simulator OMNeT++, which allows the designer to plug-in different ESB models. 8. CONCLUSION The architect has many tools and techniques available to derive architectural artifacts. These artifacts are descriptive only and present the structure, data, and rules that manipulate the data to accomplish required tasks. For complex systems, Figure 12. Execution time. [Color figure can be viewed in the online issue, which is available at www.interscience.wiley.com.] Systems Engineering DOI 10.1002/sys 16 ABUSHAREKH, GLOSS, AND LEVIS these static representations of the architecture (e.g., DoDAF) are insufficient to understand the dynamic behavior, performance, and the interaction with the environment of the systems under consideration (also their effect on other systems). The use of an executable model of the architecture enables the architect to analyze its dynamic behavior and identify logical and behavioral errors not easily seen in the static descriptions. SOA behavior and performance not only depend on the business services, but also on loose coupling, on services implemented by other systems, and on the supporting technological networks. An approach to analyze, predict, and evaluate the performance characteristics of SOA-based architecture is needed to answer questions and challenges introduced by using the SOA paradigm in designing complex, unprecedented systems. Early adopters of service-based, federated information sharing will benefit from a standards-based, technologicallyproven set of methodologies and design processes. While potentially unlimited approaches to federating different service environments exist, implementation advantages accrue to those who manage a defined, repeatable federation process built on a quantitative design framework. Extending a preexisting federation to accommodate an ever-growing set of domains and/or domain capabilities will ensure federated SOAs can constantly adapt to dynamic operations, technologies, and standards. Federation supports an advancing, dynamic set of capabilities independent of vendors and proprietary applications. In this paper we presented an approach to evaluating the end-to-end response time of business processes deployed in an ESB-enabled SOA environment. We presented an abstract ESB model used in the simulation which supports business process orchestration routing and reliable messaging. The model is capable of specifying the SOA supporting technological network to the needed level of detail. We were able to demonstrate the model using an illustrative example, and we were able to extract performance parameters from the ESB components. The example clearly demonstrated the sensitivity of end-to-end response time of a business process to the underlying technological network. We introduced an ESB-enabled, event-driven SOA for our design framework. The discrete event simulator OMNeT++ provided the behavior environment in which SOA-based performance was evaluated. Thus, we established a framework wherein service-based information solutions can be evaluated across a federation. By demonstrating how service-oriented design can be consistently quantified and measured, SOA Architects now have a reliable tool at their disposal for evaluating design options and overall service performance. Further work is needed to explore other aspects of SOAbased federated architectures, such as the effect of deploying a new SOA instance (new business process or strategy) on existing systems and infrastructures. Exploring considerations such as the implications of change in the environment or failures and changes in business logic are also design challenges which must be explored. Having established a quantifiable basis for understanding service performance characteristics within a single information domain, it is only logical to extend this to multiple domains operating within a federation. Systems Engineering DOI 10.1002/sys ACKNOWLEDGMENT This work was supported by the Office of Naval Research under Grant No. N00014-08-1-0319. REFERENCES A.M. Abusharekh, S. Kansal, A.K. Zaidi, and A.H. Levis, Modeling time in DoDAF compliant executable architectures, Proc 2007 Conf Syst Eng Res, Hoboken, NJ, 2007.zaq;3 M.P. Bienvenu, I. Shin, and A.H. Levis, C4ISR architectures III: An object-oriented approach for architecture design, Syst Eng 3(4) (2000), 288–312. D.A. Chappell, Enterprise service bus, Farnham O’Reilly, Sebastopol, CA, 2004. DoD Architecture Framework (DoDAF), Version 1.5, Department of Defense, Washington, DC, April 2007. D. Dori, Object-process methodology—a holistic systems paradigm, Springer, New York, 2002. T. Erl, Service-oriented architecture: Concepts, technology, and design, Prentice Hall, Upper Saddle River, NJ, 2005. Global Information Grid (GIG) Architecture Federation Strategy, Version 1.2, Department of Defense, CIO Office, Washington, DC, 2007 J. Hurwitz, B. Bloor, C. Baroudi, and M. Kaufman, Service oriented architecture for dummies, Wiley, Hoboken, NJ, 2007. R.N. Khan, Business process management: A practical guide, Meghan-Kiffer, Tampa, FL, 2004. D. Krafzig, K. Banke, and D. Slama, Enterprise SOA: Service-Oriented Architecture best practices, Prentice Hall, Upper Saddle River, NJ, 2004.zaq;5 A.H. Levis, “Systems architectures,” Systems engineering and management handbook, A.P. Sage and W.B. Rouse (Editors), Wiley, New York, 1999, pp. 427–453. A.H. Levis and L.W. Wagenhals, C4ISR architectures I: Developing a process for C4ISR architecture design, Syst Eng 3(4) (2000), 225–247. Y. Liu, I. Gorton, and L. Zhu, Performance prediction of service-oriented applications based on an enterprise service bus, 31st Annu Int Comput Software Appl Conf COMPSAC 2007, 2007, Vol. 1, pp. 327–334. M. Maier and E. Rechtin, The art of systems architecting, 2nd edition, CRC Press, Boca Raton, FL, 2000. D.A. Menascé and V.A.F. Almeida, Capacity planning for web services: Metrics, models, and methods, Prentice Hall, Upper Saddle River, NJ, 2002. A. Nadalin et al.zaq;4, Web Services Security: SOAP Message Security 1.1 (WS-Security 2004), OASIS Standard, http://docs.oasis-open.org/wss/2004/01/oasis-200401-wsssoap-message-security-1.1.pdf. E. Newcomer and G. Lomow, Understanding SOA with web services, Addison-Wesley Professional, Upper Saddle River, NJ, 2005.zaq;5 The Network Simulator, ns-2, Information Sciences Institute, University of Southern California, Los Angeles, CA, 2007. OMNet++ Discrete Event Simulation System, OMNet++ Community Site, http://swik.net/open source, 2007. E. Sloane, T. Way, V. Gehlot, R.A.B.R. Beck, J.A.S.J. Solderitch, and E.A.D.E. Dziembowski, A hybrid approach to modeling SOA EVALUATION OF SOA-BASED FEDERATED ARCHITECTURES systems of systems using CPN and MESA/Extend, 1st Annu IEEE Systs Conf, T. Way (Editor), 2007, pp. 1–7. L.W. Wagenhals and A.H. Levis, Service oriented architectures, the DoD architecture framework v. 1.5, and executable architectures, Syst Eng 12(4) (2009), 312–343. 17 L.W. Wagenhals, S. Haider, and A.H. Levis, Synthesizing executable models of object oriented architectures, Syst Eng 6(4) (2003), 266–300. L.W. Wagenhals, I. Shin, D. Kim, and A.H. Levis, C4ISR architectures II: A structured analysis approach for architecture design, Syst Eng 3(4) (2000), 248–287. Ashraf M. Abusharekh is currently a postdoctoral fellow at the Faculty of Computer Science at Dalhousie University, NS, Canada. He has a Bachelor’s degree in Physics from Al-Ahzhar University, Gaza (2001), and a Master’s degree in Computer Engineering from George Mason University (2004), and a Ph.D. degree in Electrical and Computer Engineering from George Mason University (2008), where he conducted research on Systems Integration and the design and analysis of Service-Oriented Architectures at the System Architectures Laboratory. Larry Gloss is a career intelligence officer, with a background in signals intelligence in the US Navy. He served in Operations Desert Shield/Storm, earning an Air Medal for reconnaissance performance and has flown over 1300 mission hours on navy reconnaissance aircraft. His assignments include the National Security Agency, National Geospatial-Intelligence Agency, National Reconnaissance Office, and on the staff of the Director of National Intelligence. He is currently pursuing a doctorate in IT and Engineering, concentrating on information sharing technologies. He holds a B.S. in Mathematics from the US Naval Academy and graduate degrees from the Naval Postgraduate School in Space Systems Operations and National Security Affairs. He is a Vice President at the National Interest Security Company. Alexander H. Levis is University Professor of Electrical, Computer, and Systems Engineering and heads the System Architectures Laboratory of the Electrical and Computer Engineering Department, George Mason University, Fairfax, VA. From 2001 to 2004 he served as the Chief Scientist of the U.S. Air Force, on leave from GMU. He was educated at MIT, where he received the B.S. (1965), M.S. (1965), M.E. (1967), and Sc.D. (1968) degrees in Mechanical Engineering with control systems as his area of specialization. He also attended Ripon College, where he received the A.B. degree (1963) in Mathematics and Physics. Dr. Levis is a Fellow of the Institute of Electrical and Electronic Engineers (IEEE) and past president of the IEEE Control Systems Society; a Fellow of the American Association for the Advancement of Science (AAAS); a Fellow of the International Council on Systems Engineering (INCOSE); and an Associate Fellow of the American Institute of Aeronautics and Astronautics (AIAA). <enote>AQ1: <enote>AQ2: <enote>AQ3: <enote>AQ4: <enote>AQ5: Please list in References Please supply Key Words Please give page numbers Please list other authors Please cite or delete listing Systems Engineering DOI 10.1002/sys