This article has been accepted for publication in a future issue of this journal, but has not been

fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JLT.2020.2990066, Journal of
Lightwave Technology
> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < 1

IT and Multi-layer Online Resource Allocation

and Offline Planning in Metropolitan Networks
Miquel Garrich, José-Luis Romero-Gázquez, Francisco-Javier Moreno-Muro, Manuel Hernández-
Bastida, María-Victoria Bueno Delgado, Anderson Bravalheri, Navdeep Uniyal, Abubakar Siddique
Muqaddas, Reza Nejabati, Ramon Casellas, Óscar González de Dios, Pablo Pavón Mariño

network functions (VNFs) on commodity hardware appliances.

Abstract—Metropolitan networks are undergoing a major In fact, a network service chain, defined as a path coupled to
technological breakthrough leveraging the capabilities of traverse a certain sequence of VNFs given by a network service,
software-defined networking (SDN) and network function unifies the concepts of SDN and NFV. These technologies
virtualization (NFV). NFV permits the deployment of virtualized
network functions (VNFs) on commodity hardware appliances
shook the telecommunications landscape with profound
which can be combined with SDN flexibility and programmability implications both in (i) the way networks are operated, i.e.
of the network infrastructure. SDN/NFV-enabled networks short-term actions, and (ii) mid-to-long term network
require decision-making in two time scales: short-term online dimensioning and planning decisions. In particular, numerous
resource allocation and mid-to-long term offline planning. In this initiatives leverage on SDN and NFV concepts for pursuing an
paper, we first tackle the dimensioning of SDN/NFV-enabled efficient use of the IT and network infrastructure [4].
metropolitan networks paying special attention to the role that
latency plays in the capacity planning. We focus on a specific use-
Encompassing the technological breakthrough of SDN/NFV in
case: the metropolitan network that covers the Murcia - Alicante metropolitan networks, the European project Metro-Haul [5]
Spanish regions. Then, we propose a latency-aware multilayer proposes scalable, dynamic and efficient metro networks to
service-chain allocation (LA-ML-SCA) algorithm to explore a efficiently interface 5G access and high-capacity
range of maximum latency requirements and their impact on the core/backbone networks. The nucleus of the Metro Haul
resources for dimensioning the metropolitan network. We observe proposal is the dynamic interconnectivity of two different types
that design costs increase for low latency requirements as more
data center facilities need to be spread to get closer to the network
of nodes, access metro edge node (AMEN) and metro core edge
edge, reducing the economies of scale on the IT infrastructure. node (MCEN), both with computational capabilities to permit
Subsequently, we review our recent joint computation of multi-site SDN/NFV functionalities to provision VNFs close to the end
VNF placement and multilayer resource allocation in the users.
deployment of a network service in a metro network. Specifically, In this context, SDN and NFV are considered two major
a set of subroutines contained in LA-ML-SCA are experimentally technology enablers for realizing 5G networks [6]. In fact, it is
validated in a network optimization-as-a-service architecture that
assists an Open-Source MANO instance, virtual infrastructure
essential to maintain high-quality service performance to end
managers and WAN controllers in a metro network test-bed. users, which are relevant to permit novel applications emerging
Index Terms— Metropolitan Networks; Network Function from Industry 4.0, vehicular networks, tactile Internet and
Virtualization, Optimization; Software-Defined Networking Internet of Things (IoT), 6-DoF (degree of freedom) virtual
reality. Special attention must be given to latency requirements
I. INTRODUCTION in these services, being critical for their performance,

T HE infrastructure in telecom operators’ central offices

(COs) has notably evolved in recent years with
advancements in software-defined networking (SDN) [2] and
successful deployment and usage. Both SDN/NFV capabilities
and latency requirements are relevant for the advent of the 5G-
era networks because they drive requirements at the
network function virtualization (NFV) [3]. SDN and NFV are control/management and operational levels [7]. Indeed,
enabling unprecedented capabilities jointly employing SDN/NFV are essential to enable “a scalable management
programmable networking and the deployment of virtualized framework enabling fast deployment of novel applications”

This work was supported in part by the Spanish Government: ONOFRE-2 josel.romero@upct.es, javier.moreno@upct.es, manuel.hernandez@upct.es,
project under Grant TEC2017-84423-C3-2-P (MINECO/AEI/FEDER, UE) and mvictoria.bueno@upct.es; pablo.pavon@upct.es).
the Go2Edge project under Grant RED2018-102585-T; and by the European A. Bravalheri, A. S. Muqaddas, N. Uniyal, and R. Nejabati, are with High
Commission: METRO-HAUL project (G.A. 761727) and INSPIRING-SNI Performance Networks Group, Department of Electrical and Electronic
project (G.A. 750611). Preliminary results contained in this paper were Engineering, University of Bristol, Bristol, BS81UB United Kingdom (e-mail:
presented at ECOC 2019 [1]. a.bravalheri@bristol.ac.uk, abubakar.muqaddas@bristol.ac.uk,
M. Garrich, J.-L. Romero-Gázquez, F.-J. Moreno-Muro, M. Hernández- navdeep.uniyal@bristol.ac.uk, reza.nejabati@bristol.ac.uk).
Bastida, M.-V. Bueno Delgado and P. Pavón Mariño are with R. Casellas is with the Centre Tecnologic de Telecomunicacions de
Telecommunication Networks Engineering Group, Department of Information Catalunya (CTTC/CERCA), Castelldefels 08860, Spain (e-mail:
and Communication Technologies, Universidad Politécnica de Cartagena; M.- ramon.casellas@cttc.es).
V. Bueno Delgado and P. Pavón Mariño are also with E-lighthouse Network Ó. González de Dios is with Telefónica GCTO, Madrid, 28050 Spain (e-
Solutions, Cartagena 30202, Spain (e-mail: miquel.garrich@upct.es, mail: oscar.gonzalezdedios@telefonica.com).

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JLT.2020.2990066, Journal of
Lightwave Technology
> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < 2

while end-to-end latency needs to be ensured below 1 ms for network that exploits the benefits of SDN and NFV.
selected flows, among other 5G key performance indicators Subsequently, Casellas et al. focus on the control aspects in
(KPIs) to be accomplished [7]. control, management, and orchestration systems for optical
In this paper, we consider both online short-term resource networks that consider multi-layer optical networks and cloud
provisioning actions and mid-to-long term offline resources for enabling 5G network slicing [12]. Recently,
planning/dimensioning decisions of SDN/NFV-enabled works in [13] and [14] focus on wide-area networks (WANs)
metropolitan networks paying special attention to the latency for applying SDN and NFV, hence enabling the IT-network
role in the latter case. We begin with a network dimensioning joint perspective in multi-layer networks. Both [13]and [14] are
use case that covers the Murcia - Alicante Spanish regions. We based on well-known protocols (e.g. PCEP) to propose an ad-
use our recently proposed NFV over IP over WDM (NIW) hoc architecture that targets latency aspects in [14].
library, an open-source modelling and evaluation framework Remarkably, Bravalheri et al. [15] report an extension of OSM
for SDN/NFV metropolitan networks proposed within the that specifically targets the WAN resource allocation jointly
Metro-Haul context. In particular, NIW is a library added to with multi-site IT infrastructure management.
Net2Plan [8] open-source network planning software, An architectural alternative is considered in the proofs of
specifically targeting the provision, design and evaluation of concept demonstrated in [16] and [17]. In particular, an
SDN/NFV networks. Leveraging NIW, we propose a latency- optimization engine based on Net2Plan enters in the scene so as
aware multilayer service chain allocation (LA-ML-SCA) to efficiently employ IT and network infrastructure in a joint
algorithm, used to explore a range of latency requisites, and we manner leveraging on algorithms commonly used for planning
observe its impact on the IT resources dimensioning. Results purposes. Specifically, [16] targets latency requirements in the
clearly show that design costs increase as the proportion of low- SDN/NFV infrastructure and [17] further elaborates on the IT
latency flows grow, as more NFV capabilities need to get closer resources in the operators’ COs.
to the network edge, and thus the economies of scale are B. Offline SDN/NFV Network Planning
degraded. Then, we review our recent NFV-IP-WDM resource 1) Latency Modelling and Assumptions.
provisioning demonstration [1], which comprises a full- Among the reviewed literature, several works assume latency
interconnected SDN-NFV test-bed with externalized models for VNFs following a queue approach. The work in [18]
intelligence hosted in a Net2plan-based network Optimization is based on a G/G/m queue model for intra-DC service chains.
as a Service (OaaS) architecture [9]. Specifically, several In this context, [19] considers a latency model at the datacenters
subroutines contained in LA-ML-SCA to iteratively provision (DCs) that accounts on an intra-datacenter CPU resource
network resources (i.e. with the goal of network planning) are sharing model among the instantiated VNFs in the DC and NIC
used to assist ETSI Open-Source MANO (OSM) [10] in multi- queues. Besides, authors in [19] disseminate radio, fixed and
virtual infrastructure managers (VIMs), multi-VNFs network aggregation-core network link congestion in end-to-end latency
service instantiation and multi-layer WAN resource allocation calculations. Few papers consider the impact of the processing
with a unified approach. overhead as the load increases in each VNF. The VNF
The remainder of the paper is organized as follows. Section II execution time in [20] depends on an approximation of the
reviews recent related works. Section III describes modelling Amdahl’s Law considering the used processors executing the
and design assumptions of the network planning use case. In VNF. Also, authors are aware of the end-to-end latency.
Section IV, we perform the offline network dimensioning that Furthermore, Savi et al. [21], assume end-to-end latency for
evaluates the latency vs. cost trade-off considering services’ services while considering CPU load balancing to calculate
maximum permitted latency to reach the first VNF of the VNF traversing time. Besides, authors provide the VNFs
service chain. Section V reviews our recent online resource sequence of the service chain to satisfy the proposed services.
allocation demonstration. Section VI concludes the paper. A different approach is considered in [22], where the end-to-
II. RELATED WORK end delay is obtained with two methods, (i) the standard
SDN and NFV are two technologies that have attracted a lot method, where latency is a function of the aggregation of the
of interest from the industry and the academia in recent years. VNF bit-rate, and is sensitive to the traffic load, and (ii) the
Here, we review related work on SDN and NFV that comprises fastpath method, that considers a constant forwarding latency.
from the optical layer up to the IT resources in the nodes’ In [23], authors consider the latency as the end-to-end delay,
location from two different perspectives: in-operation resource with the VNF processing time expressed as function of the
allocation for network control/orchestration and offline working frequency of the processor (CPU).
planning for network dimensioning that consider latency 2) IP over WDM requirements in NFV chaining
The related works reviewed above motivate the need to focus
requirements.
on service chain allocation (and VFN placement) considering
A. In-operation (online) SDN/NFV Resource Allocation the IP over WDM performance to meet the requirements of the
Recent works provide the SDN/NFV technological solutions most stringent 5G use cases. In this context, authors in [24]
for in-operation resource allocation describing network propose an algorithm which performs dynamic service-
control/orchestration architectures and protocols. For instance, chaining in an optical metro-area network which jointly
the work in [11] proposes an architectural framework for minimizes the average number of nodes required to host VNF
orchestrating mobile access networks over a multi-layer instances as well as the blocking probability. Although optical-

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JLT.2020.2990066, Journal of
Lightwave Technology
> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < 3

electrical-optical (OEO) conversion and forward error

correction (FEC) overhead penalties are mentioned in [24], no
specific values are provided nor their impact is deeply analyzed
in the joint NFV placing and IP over WDM resource allocation.
Relevantly, Pedro et al. [25] analyze the performance of three
metropolitan node architectures in terms of the number of
transceivers, capacity and latency for a 5, 10 and 15-node
metropolitan ring topology. Latency results in the range of 1 to
3 ms, which consider propagation and OEO conversion,
highlight the importance of network design in these
scenarios [25]. Indeed, FEC penalties range from 50 μs [26] up
to 150 μs [27], which combined with OEO conversion are
commonly assumed to jointly cause 100 μs of delay [25], [28].
C. Our Contribution
In this work, the novelty relies on the usage of the same
framework based on Net2plan for a twofold contribution: (i) an
offline network dimensioning with a cost model and a realistic
topology description, and (ii) an online resource allocation
based on an experimental validation. Note that other/pervious
Fig. 1. Topology of Murcia-Alicante Spanish regions under study.
work address these aspects separately. Regarding offline
network dimensioning, section II.B highlights the need for an Then, the MCENb are assigned to Murcia1 and Alicante1 nodes
analysis on the cost implications when performing VNF which are at the center of the regions’ capital cities. Finally, an
placement jointly with IP over WDM resource allocation additional MCEN is assigned to the capitals (Murcia and
considering propagation time and FEC plus OEO conversion Alicante) and the remainder 5 MCEN are distributed taking into
penalty. We report a realistic use-case of offline network account the most populated municipalities while configuring a
dimensioning in sections III and IV. Relevant for online homogeneous node type distribution across the network.
resource allocation, our work reported in section V represents a 2) Number of fibers and origin/destination
novel concept in which Net2Plan assists both OSM and WAN One hundred and two (102) bidirectional fiber links are set to
manager with the joint decision of VNF placement and network reach an average node degree of 2.87 (i.e. 102×2/71).
resource allocation. Subsequently, we jointly consider the combination of node
degree distribution (see Fig. 2) and network diameter (see
III. NFV-IP-WDM USE CASE: MURCIA-ALICANTE REGION Table I) to define the origin/destination node of each fiber.
The node degree distribution for M-A is similar to the three
In this section, we first describe the topological instances of the TIM networks. The WDM plant determines the
characteristics of the metropolitan network produced for this network diameter expressed as the longest shortest path in
use case. Then, we detail the traffic, latency, cost and service number of hops, between all possible source-destination pairs.
models used. Although small and medium TIM network instances are similar
A. Topology Description to M-A in terms of average node degree and fibers, the network
Fig. 1 illustrates the network topology under study, which is diameter diverges due to the particular geolocation and
located in the South-East of Spain. 71 nodes (31 nodes in distribution of nodes. Indeed, TIM networks present a
Murcia and 40 in Alicante) correspond to cities and towns homogeneous distribution of the nodal degree whereas M-A
considering realistic information from a nation-wide telecom concentrates the high degree nodes in two central city areas and
operator list of metropolitan COs. The nodes are connected via spreads low degree nodes toward the network edge. Thus, the
a realistic WDM topology, synthesized in a form that matches trade-off between average node degree and network diameter
the topological characteristics of reference metro networks has been assessed to form a realistic M-A network.
provided in Metro-Haul project. In particular, we consider three TABLE I
reference networks from Telecom Italia (TIM) operator [29] TOPOLOGY CHARACTERISTICS OF TIM AND M-A NETWORKS
with three sizes (see Table I). Tab TIM small TIM med TIM large M-A (ours)
1) Node categories Links 72 143 219 102
We define three node categories: AMEN, MCEN without Nodes 52 102 129 71
AMEN 44 (84.6%) 89 (87.2%) 113 (87.6 %) 62 (87.3 %)
connectivity to the core network and MCENb in which ‘b’ MCEN 6 (11.5 %) 11 (10.8%) 11 (8.5 %) 7 (9.9 %)
denotes connectivity to the backbone network. The category of MCENb 2 (3.9 %) 2 (2 %) 5 (3.9 %) 2 (2.8 %)
each node in the Murcia-Alicante (M-A) topology is defined Average node
2.77 2.80 2.75 2.87
following a similar ratio of categories in the three instances of degree
Net. diameter* 9 12 15 13
the TIM topology (see Table I). The result is that the 71 nodes AMEN: Access Core Edge Node, MCEN(c): Merto-Core Edge Node
in M-A are distributed into 62 AMEN, 7 MCEN and 2 MCENb. (connected to the core network).
* Network diameter: Max number of hops between all possible node pairs.

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JLT.2020.2990066, Journal of
Lightwave Technology
> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < 4

60 TABLE II
TIM small POPULATION CHARACTERISTICS OF THE MURCIA AND ALICANTE REGIONS
50
TIM medium
Alicante Murcia
40 Municipalities 141 45
Percentage

TIM large
Municipalities with (≥1) node 30 19
30 Murcia-Alicante (M-A) Municipalities to cover 95% of population 60 30
20 Population (in 2017)* 1,825,332 1,463,249
Population covered (95%) 1,734,066 1,390,087
10 Population not considered 91,266 73,162
0
* Governmental data of Murcia [30] and Alicante [31] regions.
1 2 3 4 5 6 7 is the fiber’s refractive index and lf is the fiber link length in
Nodal degree km. We consider a constant OEO conversion time of 0.2 ms at
Fig. 2. Nodal degree distribution of three TIM and M-A networks. each OEO hop, which includes the FEC penalty. This value is
B. Traffic Model chosen as upper bound (i.e. worst-case) that guarantees the
Here, we first describe how the population of the OEO/FEC overhead penalties (see section II.B).
municipalities is assigned to each network node. Then, we In this work, latency requirements are limited to a maximum
detail our traffic model. delay between the user and the first VNF in the service chain.
1) Population assigned to each node In each test, such latency requirements are random uniformly
The assignment of population to each network node in the picked per service chain request between [0.5, Lmax]
NIW framework is performed according to the three following milliseconds. Latency limits are enforced for reaching just the
cases. First, the population of a municipality with a single node first VNF because the delay experienced to traverse the first (as
representation is directly assigned to the node.population any) VNF largely depends on software processing performance
characteristic that defines that node. Second, in case of a as reviewed in section II. However, the LA-ML-SCA algorithm
municipality with two or more node representations in the will instantiate all the VNFs (not just the first) in the same
framework, its population is uniformly distributed among those AMEN or MCENb nodes for a given service. This is a worst-
nodes. Third, the population of a municipality with no assigned case scenario from the point of view of cost: a latency limit
node in the framework is added to the nearest node. Table II requiring instantiating the first VNF close to the user, results in
lists the number of municipalities and inhabitants in all the VNFs in the chain being instantiated in the same data
Murcia [30] and Alicante [31] regions and our assignment of center, while in a less restrictive scenario those could be left for
population into adjacent nodes without node are included a more centralized data center (e.g. in the MCENb).
(ordered from highest to lowest population) into the nearest Additionally, this modelling assumption is made to perform
nodes until 95 % of population is assigned. VNF placement decisions that aim at satisfying end-to-end low-
2) Traffic generation model latency requirements as the ones described in [24]. In particular,
Users in each node are supposed to be using 50 different [24] describes service chains for VoIP, augmented reality and
services, each one reflected by two service chain requests, smart factory with latency requirements of 5 ms, 1 ms and 1 ms,
upstream and downstream. This sums 3550 bidirectional respectively. Consequently, such low values of latency can only
service chain requests (71 nodes, by 50 services). Each service be ensured by placing all VNFs close to the user.
chain has a number of VNF instances randomly chosen between D. Cost Model
1 and 5. This modelling decision is based on the work reported In this work, we concentrate on the costs of the IT resources
in [24], which defines the characteristics for 6 realistic service required for VNF instantiation. In particular, we are interested
chains (with their number of VNFs): cloud gaming (5), in capturing how the distribution of IT resources in micro-data
augmented reality (5), VoIP (5), video streaming (5), massive centers, instead e.g. of its concentration in one or few data
IoT (3) and smart factory (2). All VNFs require 2 CPU cores, 4 centers per metro network, can impact the overall cost.
GB of RAM and 20 GB of HDD. No traffic compression is To evaluate this, we consider a cost model that reflects the
considered in any VNF. We further assume that each VNF effects of the economy of scale. Such economic laws have been
instance (and its associated resources) is devoted exclusively to shown to apply to all aspects of economy, and mean that the
the service chain it belongs. cost-per-unit-produced decreases as the number of units
Upstream service chain starts in a particular node and ends in produced increases. In our case, means i.e. that building a data
the nearest MCENb. The traffic intensity is proportional to the center of size 2Z in a node, costs less than building two data
node population, and to a random uniformly picked centers of size Z in two nodes.
multiplicative factor in the range [0.1, 1.0]. The traffic is There are a number of mathematical relations that have been
normalized so that the maximum traffic per service injected shown to model the economies of scale effects. All of them are
among all services and all nodes in the network, equals 1 Gbps. characterized by a concave (sublinear) curve describing the cost
C. Latency Assumptions and Requirements of a system, respect to its size (see [32] for details). In this
Two major latency contributions are considered: propagation paper, we choose the square-root law, which appears in many
delay in the optical fibers and OEO conversion penalty. The economic scenarios as an accurate descriptor of the economies
propagation delay (df) in the optical fiber is calculated as of scale law. According to this, we estimate the cost of a data
𝑐
𝑑𝑓 = 𝑛 · 𝑙𝑓 , where c is the speed of light (3·105 km/s), n = 1.5 center placed in a particular node as:

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JLT.2020.2990066, Journal of
Lightwave Technology
> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < 5

Algorithm 1. Latency-aware multilayer service chain allocation IV. NIW MURCIA-ALICANTE USE-CASE EVALUATION
Input: G = (N,E), Lmax In this section, we perform a dimensioning study of the
Output: VNF placement, IP links and routing, lightpaths metropolitan networks that cover Murcia and Alicante regions.
Being: G graph, N nodes, E fibers, e fiber, Lmax latency limit, l(p(r))
latency of path p of service chain request r, V(r) VNFs of r.
We present and describe the proposed LA-ML-SCA algorithm.
Begin: Then, the dimensioning study executes LA-ML-SCA and
1. Sort service chain requests in decreasing traffic computes the resulting CO and network costs. Finally,
2. Create empty list of established lighpaths: LP simulation results are presented and discussed.
3. foreach (r ϵ SCR) do:
4. Calculate shortest path p(r) in km from oriNode(r) to the nearest A. LA-ML-SCA Algorithm
core node (destNode(r)) The LA-ML-SCA algorithm is shown in Algorithm 1.
5. while (ldest(p(r)) > l(r))
receives as an input (i) an IP over WDM network modeled as a
6. Calculate ldest(p(r)) // latency to destNode(r) with OEO
conversion penalty graph G with N nodes and E fibers, (ii) with available IT
7. if (ldest(p(r)) < = l(r)) resources in the COs, (iii) the service chain requests together
8. Establish lightpath: λ ∀ e ∈ p(r) | λ(e) ∉ LP with their latency requisites. Its goal is satisfying the requests
9. Instantiate V(r) in destNode(r) by allocating service chains, creating the needed VNF instances
10. else and IP links that are realized via lightpaths over the WDM plant.
11. foreach λinter ∈ LP(ending at destNode(r)) do:
Latency-aware means that allocations are restricted to satisfy
12. Calculate l2-hop(p(r)) = l(λori*)+ l(λinter)
13. if (l2-hop(p(r)) < = l(r)) the latency requisites, as well as of course avoid IT resource
14. Establish lightpath to interNode(r): λori oversubscription and spectrum clashing.
15. Instantiate V(r) in destNode(r) The LA-ML-SCA algorithm first sorts the service chain
16. break request r according to its injected traffic (higher traffic first) in
17. end if line 1. Then, it sequentially processes each request r one by one
18. end foreach
(line 3) by calculating its shortest path p(r) from the origin node
19. if (l(λdirect*(r)) < = l(r))
20. Establish lightpath to destNode(r): λdirect to the nearest core node (line 4). The core of the LA-ML-SCA
21. Instantiate V(r) in destNode(r) algorithm (lines 5-27) is a while loop that aims at allocating all
22. break the VNFs of r in the furthest node from the user (closest to the
23. end if core, and thus more centralized). This potential allocation is
24. Calculate lpNode (r) // Calculate latency to the previous node
evaluated in three approaches. Firstly, lines 7-9 evaluate if
pNode(r) with OEO conversion in p(r) = p(r) - 1
25. Update destNode(r) = pNode(r) latency requirements are satisfied with p(r) considering OEO
26. end if and propagation. If so, the algorithm establishes a lightpath per
27. end while link in p(r), which creates a new logical IP link or increases IP
28. end foreach capacity (in case an IP link is present). Secondly, lines 11-18
𝐶𝑛𝑜𝑑𝑒 = 𝐶√𝑐1 · 𝐶𝑃𝑈 + 𝑐2 · 𝑅𝐴𝑀 + 𝑐3 · 𝐻𝐷𝐷. aim to reuse previously established lightpaths (λinter) evaluating
CPU, RAM and HDD are the total amount of CPUs (assumed potential lighpaths (λori*) in terms of total latency with two
all of the same type), RAM (in GB) and HDD (in GB) required OEO conversion. Thirdly, lines 19-23 evaluate if a direct
in the data center, respectively. Coefficients c1, c2 and c3 weight ligthpath satisfies the latency requirements.
the contribution of each resource to the total deployment of the Finally, the LA-ML-SCA algorithm iterates the three
NFV-enabled CO. In particular, we consider a common server approaches above reducing one hop the shortest path in each
size composed of 2 CPU cores, 4GB of RAM and 2,000 GB of iteration (lines 24-25). In case any approach above is satisfied,
HDD, in which the cost of these components is 70 €, 50 € and all VNFs are instantiated in the current destination node of p(r).
85 €, respectively. This permits to set c1 = 70/2 (CPU We recall that all VNFs of r are instantiated in the same node
coefficient), c2 = 50/4 (RAM coefficient) and c3 = 85/2000 for ensuring that stringent latency constraints are satisfied.
(HDD coefficient). Factor C is set to 1, reflecting that we are Additionally, this assumption is made to avoid the trivial
interested in comparing normalized cost values among the solution in which only the first VNF of r is placed in a node
different alternatives. close to the user, while all the other VNFs are consolidated in
few nodes, without caring about end-to-end latency. The output
E. The NFV over IP over WDM (NIW) Library of the LA-ML-SCA algorithm is used to compute the resources
In this use case, we use the NFV over IP over WDM (NIW) employed at all COs, number of lightpaths (i.e. transponders)
open-source library [33], with available documentation in and IP switching occurrences.
Javadoc format [34]. NIW is currently contained in Net2Plan The LA-ML-SCA algorithm has a polynomial complexity.
and shipped within its latest release [35]. NIW is developed in Its computational time is dominated by the shortest-path
the context of the Metro-Haul project [5] with the purpose of calculations which are performed for each service chain request
modeling SDN/NFV-enabled networks in a versatile r once in line 4 and iteratively three times in lines 8, 14 and 20.
framework which, based on its Excel spreadsheet importer and Note that the number of iterations that the latter three shortest-
reporting functionalities, permits the abstraction of the path calculations are performed depends on the latency
underlying representations for rapid network engineering requirements. Specifically, in case the latency requirements are
(network capacity planning and dimensioning) [33]. not satisfied, the core while loop in the LA-ML-SCA

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JLT.2020.2990066, Journal of
Lightwave Technology
> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < 6

Fig. 4. CO costs for maximum latencies allowed of 1, 15 and 60 ms.

in each network dimensioning also as a function of the
considered values of Lmax, which exhibits a similar trend as the
cost of the COs. This behavior occurs because LA-ML-SCA
Fig. 3. a) CO’s cost, b) number of transponders and c) number of tends to deploy direct lightpaths from the end user to the nearest
IP switching occurrences as function of the maximum latency MCENb (last node of the service chain) in case the latency
allowed to reach the first VNF. requirements are low, hence optical bypass is required at
algorithm iterates these three calculations with a one-hop intermediate nodes. Fig. 3c) shows the total number of IP
reduction from the destination node on each iteration. switching occurrences performed by all the 3550 service chain
Consequently, high computational time is expected for low requests. Remarkably, IP switching has the opposite trend
latency requirements. These iterations are in the order of N compared to both cost of the COs and number of transponders,
(worst case), whereas the shortest-path calculations are because IP switching is largely employed for high-latency
implemented with a complexity O(E+N log N) and exhibit a cases. In fact, high latency requirements correspond to
more uniform execution time. permissive cases and thus OEO conversions are allowed
through the service chain. In summary, a trade-off is evidenced
B. Simulation Results and Discussion because low-latency requirements devote capital expenditures
We consider a M-A network infrastructure with enough pool on IT resources at COs toward the edge jointly with
of IT resources (CPU, RAM, HP) in all COs to place all service transponders, whereas high-latency requirements migrate cost
chain requests without contention. The network size is obtained towards IP equipment.
executing the proposed LA-ML-SCA algorithm, which places Fig. 4 illustrates the per node CO cost distribution
all VNFs over the M-A network. Note that LA-ML-SCA considering Lmax values of 1 ms, 15 ms and 60 ms. Nodes in the
progressively establishes IP links (and lightpaths) to meet the x-axis are sorted in decreasing order according to their cost for
services’ latency requirements, starting with an empty network. Lmax = 1 ms, denoting MCEN and MCENb as (M) and (Mb),
We investigate the network design cost (in arbitrary units see respectively. The MCENb central nodes in the capital cities
section III.D) as a function of services’ latency requirements. In (Murcia1 and Alicante1) concentrate the largest amount of
particular, we explore a range of Lmax latency values (maximum resources. For Lmax = 1 ms, Murcia1 and Alicante1 present a 10-
latency from the user to the first VNF) in a range between 1 ms fold CO cost value compared to the remainder nodes.
up to 60 ms, which correspond to the most stringent and the Additionally, the resources not allocated in MCENb nodes are
most permissive scenarios, respectively. We recall that Lmax is distributed among various AMEN and few MCEN. Lmax values
used to define the maximum latency constraints for each service of 15 ms and 60 ms further concentrate the CO cost in the two
chain request, which are uniformly picked between [0.5, Lmax]. MCENb nodes, up to no resources needed in both MCEN and
Fig. 3a) shows the cost of COs as a function of the considered
values of Lmax. Results show an abrupt cost increase for values
below 10 ms. The cost of COs to guarantee the services’ latency
of 1 ms reaches up to four times the value of more permissive
scenarios Lmax ≥ 30 ms. Indeed, low latency values are only
attainable instantiating VNFs close to the origin nodes of the
service chains, which decentralizes the IT resources into a
larger amount of COs. Conversely, more permissive scenarios
(i.e. Lmax ≥ 30 ms) tend to concentrate the IT resources into
MCENb, which leveraging on the economy of scale, reduces
the COs’ cost. Fig. 3b) reports the number of transponders used Fig. 5. CO cost distribution between MCENb, MCEN and AMEN.

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JLT.2020.2990066, Journal of
Lightwave Technology
> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < 7

REST/API OaaS
OaaS Data
OSS server
client
REST/API REST/API
REST/API

NFV-O
REST
WIM connector /API
Or-Vi Or-Vi
Transport-API
WIM OaaS client

VIM 1 VIM 2
SDN controller SDN controller
VNF 1 (packet) (optical) VNF 2
Fig. 6 a) Total LA-ML-SCA algorithm computational time and b)
SC request resource allocation computational time as function of Network Network
interface OXC OXC interface
the maximum latency allowed to reach the first VNF.
AMEN nodes in the latter case. Relevant to this observation, Packet switch OXC Packet switch
Fig. 5 groups the CO cost distribution per node type for the
three Lmax values under study. Indeed, as anticipated in Figs. 4 Fig. 7. Functional Architecture
and 5, CO cost is concentrated in MCENb nodes for high exploiting a software layer of third-party applications to
(i.e. more permissive) Lmax values whereas low-latency values request, in this case, efficiency in IT and network resource
spread the CO cost towards nodes not connected to the allocations. The Net2Plan-based OaaS module exposes a
backbone network. REST-API, which includes calls to request the computation of
Fig. 6a) and b) report the computational times of the entire a joint VNF placement and multilayer resource allocation
LA-ML-SCA algorithm and for the allocation decision of each solution for a network service request. The VNF instantiation is
service chain request, respectively, both as a function of the communicated to OSM, while the multilayer allocation is stored
maximum latency allowed to reach the first VNF. As in a database, and is later retrieved by a API call from the WIM.
anticipated at the end of section IV.A, high computational times Secondly, the control plane is composed of:
are experienced for low-latency requirements. In particular, the i) ETSI’s NFV-Orchestrator Open-Source MANO (OSM), in
core while loop of the LA-ML-SCA is iteratively executed for charge of orchestrating network services over IP over
satisfying stringent cases of low latency. Finally, it is optical metro networks and the VNFs instantiation.
worthwhile mentioning that the total execution time in Fig. 6a) ii) Two VIMs, implemented with OpenStack clusters, which
corresponds to 3550 x (i.e. number of service chain requests) are responsible for the virtualization infrastructure by
the computational time employed for the allocation decision of hosting the virtual machines (VMs) of the VNFs.
each service chain request reported in Fig. 6b). iii) The SDN-control is split in an SDN controller per layer, one
devoted to the packet layer and the other one in charge of
V. NFV-IP-WDM EXPERIMENTAL DEMONSTRATION the optical layer.
In this section, we review our recently proposed architecture, iv) The WIM hierarchically on top of the SDN controllers.
workflow and demonstration in which Net2Plan performs a Especially relevant for SDN-NFV WAN-based work is the
network Optimization-as-a-Service (OaaS) in the joint presence of the WIM [15], because it links OSM with the
computation of multi-site VNF placement and multilayer SDN-controllers, via Transport-API. Additionally, the
resource allocation in the deployment of a network service in a WIM interconnects remote VNFs, belonging to the same
metro network [1]. Specifically, the online resource allocation network service, instantiated in different VIMs along the
performed herein includes several specific subroutines within multi-layer transport network in this WAN environment.
LA-ML-SCA to assist an Open-Source MANO instance and Moreover, the WIM includes an OaaS client to request
WAN Infrastructure Manager(s) (WIM) in an experimental optimal paths from the OaaS server.
proof of concept comprising a metro-network test-bed. Finally, the data plane considers a multi-layer transport
network, where the packet layer is represented by packet
A. SDN/NFV Architecture for Wide Area Networks switches over the optical layer that interconnect the compute
Fig. 7 shows the components comprised in the architecture nodes under the control of the VIM. The devices, both packet
for the experimental validation of IT over IP over WDM and optical layer, are abstracted by the T-API based WIM for
resource allocation. Firstly, on top of the functional schema, the configuration purposes.
role of the operator is represented by an Operations Support
System (OSS) realized with a specific extension of Net2plan. B. Workflow of the WAN-based Network Service Instantiation
The cloud/network optimization tasks are externalized in the To efficiently use both IT and network resources when
network OaaS module [9], which relies on optimization performing the network service deployment is of importance in
procedures on top of network controllers and orchestrators the context of the Metro-Haul project. To this end, the proposed

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JLT.2020.2990066, Journal of
Lightwave Technology
> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < 8

OSS OaaS OSM WIM VIMs Transport file, ready to be sent via REST/API to the OaaS server as
1 Req. VIMs’ IPs and list of NSDs inputs of the algorithm.
4. The OaaS server receives the inputs and the algorithm
2 Req. VIMs’ info. (CPU, RAM, HD) jointly solves (i) the VNFs placement (in the VIMs), (ii) the
3 Req. IT/network allocation
allocation of the flows between VNFs over the multilayer
network. Relevant for (i), NIW’s method
4 Compute IT/network allocation
Fig. 10 wNet.addVnfInstance corresponds to
5 Req. multi-site instantiation of VNFs netPlan.addResource, which is used in this
6 Req. network connectivity experimental validation. Similarly, for (ii) the network flow
Instantiate VNFs computation uses NIW library’s functionalities
7 Req. transport path details wNet.getKShortestServiceChainInIpLayer and
Fig. 10 wNet.getKShortestWdmPath, which are comprised
8 Apply multilayer tranport path
within Net2Plan’s generic method
Fig. 8. Simplified schema of the workflow. GraphUtils.getKMinimumCostServiceChains. More
architecture combines the Metro-Haul control, orchestration details regarding NIW’s functionalities can be found in [33].
and management (COM) service platform with an external Additionally, in terms of computational time, the variation
optimization engine to allow efficient use of IT/network of the LA-ML-SCA algorithm here employed is executed in
resources. The proposed architecture operates with the 4.96 ms, which corresponds to half of the lowest times
following workflow (also illustrated in Fig. 8): reported in Fig. 6b). This is because the topology employed
1. The OSS, which already contains the multi-layer transport in this experimental implementation is limited to 9 nodes
topology in an .n2p (Net2Plan) file, queries OSM for the and 13 fibers. After the OaaS computation, the VNF
VIM IP addresses and the Network Services (NSs) available placement is returned to the OSS, whilst the calculated
to be instantiated via OSM REST API requests. network allocations are stored in an internal database.
2. Once the IP addresses of the VIMs are known by the OSS, 5. Then, the OSS instance commands OSM to instantiate the
it retrieves all the IT information (CPU, RAM, HDD) of the VNFs in the VIMs selected by the algorithm hosted in the
compute nodes and updates the .n2p file. OaaS server.
3. The OSS defines service chain requests regarding the 6. OSM informs the WIM about the multi-VIM connectivity
network service to be deployed and it is included in the .n2p request.

a)

b)

Fig. 9. Net2Plan (OSS) GUI with the network service. a) Topology view and characteristics of the VNFs, and b) network service detail.

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JLT.2020.2990066, Journal of
Lightwave Technology
> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < 9

7. The WIM queries the OaaS module for the previously transport network. Note that Fig. 9b) depicts the information to
calculated path in the transport stored in the database. be stored in the database after the optimization task, origin and
8. The WIM contacts both packet and optical SDN controllers destination nodes and the set of links that compose the path
to configure the transport devices in order to provide traversing the VNFs in the correct order. Note that latency
connectivity between the VIMs enabling the VNFs values are computed also in this experimental validation in a
connection; thus satisfying the network service deployment. per-link basis (0.26 ms, 0.26 ms and 0.3 ms) and for the entire
route length (0.82 ms) as shown in Fig. 9b). These values could
C. Testbed Configuration and Experimental Outcomes
be used to validate also in this instance of Net2Plan the
The experimental setup to implement this SDN-NFV WAN subroutines employed in the offline network dimensioning part
architecture proposal is performed in two testbed islands, the of the paper (section IV). Nonetheless, the experimental
two Net2plan instances (OSS and OaaS) are located in a validation here reported targets the verification of the COM
laboratory in Cartagena (Spain) whilst the VIMs, the ETSI service platform for successfully performing the service chain
OSM, WIM, SDN controllers and the multi-layer topology are instantiation comprising VNFs in multiple VIMs and network
placed at the High Performance Networks, University of Bristol path.
(UK). Both testbeds are connected by a private VPN to provide Fig. 10 shows a capture of the REST/API GET response
control-plane visibility. The multi-layer infrastructure consists regarding used both in the queries to the OaaS performed from
of two OpenStack datacenters interconnected by a combination the OSS and the WIM. The JSON file includes the network
of packet switches and optical cross connects (OXCs) which are service name and VNF placement instantiation details for the
SDN-enabled and managed by a combination of SDN OSS (again, used in step 4 in Fig. 8) and link IDs of the path for
controllers as in the functional schema shown in Fig. 7. the multilayer resource allocation to be carried out by the WIM.
Fig. 9 shows Net2plan’s GUI with the topology that assumes After processing the reply from the OaaS module, the WIM
one datacenter placed in Bristol and the other one in London; oversees the inter-VNF connectivity over the multi-layer
where the optical network between the datacenters emulates a infrastructure.
realistic WAN domain. Fig. 9b) displays the service chain
solution, corresponding to the instantiation of a two-VNF VI. CONCLUSION
network service solved by the OaaS module (in step 4 in Fig. 8) This paper covered two aspects of SDN/NFV-enabled
with the VNF placement and the path in the multi-layer metropolitan networks: short-term resource allocation actions
and mid-to-long term planning and dimensioning decisions. We
began with the latter aspect with the capacity planning of a
metropolitan network considering the impact in the IT
infrastructural cost that latency requirements may impose, by
forcing the placement of decentralized data centers closer to the
user. To this end, we employed Net2Plan’s NIW library in a
specific metropolitan network (Murcia and Alicante Spanish
regions) proposing a Latency-Aware Multilayer Service Chain
Allocation (LA-ML-SCA) algorithm. We analyzed the impact
of latency limitations on IT resources deployment. The cost
model employed captures the economies of scale laws,
Placement reflecting the extra costs coming from decentralizing data
of VNF 1 centers. The obtained results showed that design costs increase
for low latency requirements as NFV capabilities need to get
closer to the network edge. Indeed, a drastic increase in the
Placement overall cost was observed for latency limits below 10 ms.
of VNF 2 Relevant for the SDN/NFV resource allocation aspect, we
reviewed our recent experimental validation in which Net2Plan
Network plays a network OaaS role to support OSM, VIMs and WAN
service infrastructure manger on the decision of the NFV placement
and multilayer resources to be employed.

REFERENCES
Nodes and path [1] F. J. Moreno-Muro, M. Garrich, C. San-Nicolás-Martínez, M. Hernández-
Bastida, P. Pavón-Mariño, A. Bravalheri, A. Muqaddas, N. Uniyal,
R. Nejabati, R. Casellas, and O. González de Dios, “Joint VNF and multi-
layer resource allocation with an open-source optimization-as-a-service
integration,” European Conference on Optical Communication (ECOC),
Dublin, Ireland, Sept. 2019.
[2] D. Kreutz, F. M. V. Ramos, P. E. Veríssimo, C. E. Rothenberg, S.
Fig. 10. REST-based response to a GET query capture used in two Azodolmolky and S. Uhlig, “Software-Defined Networking: A
interactions: OaaS with OSS for IT resources (step 4, Fig. 8); and Comprehensive Survey,” in Proceedings of the IEEE, vol. 103, no. 1, pp.
OaaS with WIM for network resources (step 7, Fig. 8). 14-76, Jan. 2015.

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JLT.2020.2990066, Journal of
Lightwave Technology
> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < 10

[3] M. Chiosi, et al. “Network functions virtualisation: An introduction, [25] J. Pedro, A. Eira, and N. Costa, “Metro Transport Architectures for
benefits, enablers, challenges and call for action,” in Proc. SDN OpenFlow Reliable and Ubiquitous Service Provisioning,” in proc. Asia
World Congress, Darmstadt, Germany, Oct. 2012, pp. 22–24. [Online]. Communications and Photonics Conference (ACP), Guangzhou, China,
Available: http://portal.etsi.org/NFV/NFV_White_Paper.pdf. Accessed Nov. 2018.
on: Nov. 11, 2019. [26] J. Jay, “Low signal latency in optical fiber networks,” Proc. of the 60th
[4] M. Garrich, F. J. Moreno-Muro, M. V. Bueno-Delgado, and P. Pavón- IWCS Conference, Charlotte, NC, USA, Nov. 2011.
Mariño, “Open-source Network Optimization Software in the Open [27] Optelian white paper “A Sensible Low-Latency Strategy for Optical
SDN/NFV Transport Ecosystem,” Journal of Lightwave Technology, vol. Transport Networks,” July 2014. [Online]. Available.
37, no. 1, Jan. 2019. https://www.optelian.com/wp-content/uploads/2016/10/Optelian-Low-
[5] Metro-Haul project. [Online]. Available. https://metro-haul.eu/. Accessed Latency-Strategy-for-OTN-WP.pdf
on: Nov. 11, 2019. [28] V. Bobrovs, S. Spolitis, and G. Ivanovs. “Latency causes and reduction in
[6] F. Z. Yousaf, M. Bredel, S. Schaller, and F. Schneider, “NFV and SDN - optical metro networks,” Optical Metro Networks and Short-Haul Systems
Key Technology Enablers for 5G Networks,” Journal on Selected Areas in VI. Vol. 9008. International Society for Optics and Photonics, 2014.
Communications, vol. 35, no. 11, pp. 2468–2478, Nov. 2017. [29] Metro-Haul deliverable D2.3 “Network Architecture Definition, Design
[7] 5G infrastructure public private partnership (5G PPP) key performance Methods and Performance Evaluation”. [Online]. Available.
indicators (KPIs). [Online]. Available: https://5g-ppp.eu/kpis/. Accessed https://zenodo.org/record/3496939. Accessed on: Nov. 11, 2019.
on: Nov. 11, 2019. [30] Governmental statistics, Murcia. [Online]. Available.
[8] Net2Plan. [Online]. Available. http://www.net2plan.com/. Accessed on: http://econet.carm.es/inicio/-
Nov. 11, 2019. /crem/sicrem/PU_padron/cifof10/sec1_c1.html. Accessed on: Nov. 11,
[9] M. Garrich, C. San-Nicolás, F. J. Moreno-Muro, M. V. Bueno-Delgado, 2019.
P. Pavón-Mariño, “Network Optimization as a Service with Net2plan”, [31] Governmental documents, Alicante. [Online]. Available.
European Conference on Networks and Communications (EuCNC), http://documentacion.diputacionalicante.es/censo.asp?tfm_order=DESC&
Valencia, Spain, June 2019. tfm_orderby=Pobl2017. Accessed on: Nov. 11, 2019.
[10] Open Source NFV Management and Orchestration (MANO). [Online]. [32] P. Pavón-Mariño, “Optimization of Computer Networks: Modeling and
Available: https://osm.etsi.org/ Accessed on: Nov. 11, 2019. Algorithms: A Hands-on Approach,” Hoboken, NJ, USA: Wiley, 2016.
[11] R. Martínez, A. Mayoral, R. Vilalta, R. Casellas, R. Muñoz, S. Pachnicke, [33] J. L. Romero-Gázquez, M. Garrich, F. J. Moreno-Muro, M. V. Bueno-
T. Szyrkowiec, and A. Autenrieth, “Integrated SDN/NFV orchestration for Delgado, and P. Pavón-Mariño, “NIW: A Net2Plan-based Library for NFV
the dynamic deployment of mobile virtual backhaul networks over a over IP over WDM Networks,” Int. Conference on Transparent Optical
multilayer (packet/optical) aggregation infrastructure”, Journal of Optical Networks (ICTON), Angers (France), July 2019.
Communications and Networking, vol. 9, no. 2, pp. A135-A142, Feb. 2017. [34] NIW Javadoc documentation. [Online]. Available.
[12] R. Casellas, R. Martínez, R. Vilalta, and R. Muñoz, “Control, Management, http://www.net2plan.com/documentation/current/javadoc/api/com/net2pla
and Orchestration of Optical Networks: Evolution, Trends, and n/research/niw/networkModel/package-frame.html. Accessed on: Nov. 11,
Challenges”, Journal of Lightwave Technology, vol. 36, no. 7, pp. 1390- 2019.
1402, Apr. 2018. [35] Net2Plan releases. [Online] Available:
[13] S. Fichera, R. Martínez, R. Casellas, B. Martini, R. Vilalta, R. Muñoz, and https://github.com/girtel/Net2Plan/releases. Accessed on: Nov. 11, 2019.
P. Castoldi, “Orchestrating Inter-DC Quality-Enabled VNFFG Services in
Packet / Flexi-Grid Optical Networks,” European Conference on Optical
Communication (ECOC), Rome, Italy, Sept. 2018.
[14] S. Fichera, R. Martínez, B. Martini, M. Gharbaoui, R. Casellas, R. Vilalta,
R. Muñoz, and P. Castoldi, “Latency-Aware Resource Orchestration in
SDN-based Packet Over Optical Flexi-grid Transport Network,” Journal
of Optical Communications and Networking, vol.11, no. 4 pp. B83-B96,
Apr. 2019.
[15] A. Bravalheri, et. al., “VNF Chaining across Multi-PoPs in OSM using
Transport API,” Optical Fiber Communication Conference (OFC), San
Diego, USA, Mar. 2019.
[16] F. J. Moreno-Muro, C. San-Nicolás, M. Garrich, P. Pavón-Mariño,
O. González de Dios, and R. Lopez Da Silva, “Latency-Aware
Optimization of Service Chain Allocation with Joint VNF Instantiation and
SDN Metro Network Control,” European Conference on Optical
Communication (ECOC), Rome, Italy, Sept. 2018.
[17] M. Garrich, M. Hernández-Bastida, C. San-Nicolás, F.J. Moreno-Muro,
P. Pavón-Mariño, “The Net2Plan-OpenStack Project: IT Resource
Manager for Metropolitan SDN/NFV Ecosystems”, Optical Fiber
Communication Conference (OFC), San Diego, USA, Mar. 2019.
[18] J. Prados-Garzon, P. Ameigeiras, J.J. Ramos-Munoz, et. al. “Analytical
Modeling for Virtualized Network Functions”, in Proc. of Inter. Conf.
Comm. Work. (ICC Workshop), May 2017.
[19] G. Faraci, A. Lombardo, “A simulative model of a 5G Telco Operator
Network”, Procedia Comp. Sci., vol. 110, pp 344-351, 2017.
[20] A. Alleg, T. Ahmed, M. Mosbah, et. al. “Delay-aware VNF placement and
chaining based on a flexible resource allocation approach”, in Proc. of Int.
Conf. Netw. Serv. Manag. (CNSM), Nov. 2017.
[21] M. Savi, M. Tornatore, G. Verticale, “Impact of processing costs on service
chain placement in network functions virtualization”, in Proc. of IEEE
Conf. Netw. Func. Virt. Soft. Def. Netw. (NFV-SDN), Nov. 2015.
[22] M. Gao, B. Addis, M. Bouet, S. Secci, “Optimal orchestration of virtual
network functions”, Elsevier Com. Net., vol. 142, pp. 108-127, Sept. 2018.
[23] D. Sattar, A. Mattraway, “Optimal Slice Allocation in 5G Core Networks”,
Cornell University Library, Feb. 2018.
[24] L. Askari, A. Hmaity, F. Musumeci, and M. Tornatore “Virtual-network-
function placement for dynamic service chaining in metro-area networks,”
in proc. Optical Network Design and Modeling (ONDM), Dublin, Ireland,
pp. 1-6, 2018.

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.