Ieee Performance Evaluation: Convergence of Theoreti Cal, Simulation and Experimental Results
Ieee Performance Evaluation: Convergence of Theoreti Cal, Simulation and Experimental Results
Ieee Performance Evaluation: Convergence of Theoreti Cal, Simulation and Experimental Results
i
3 4 6
7 $
-
Fig. 4 Comarison oh
/
mulated and aaaJ)tical satra-
tin througbput for a single BSS (1M bps FSS PH.
u---- -
BW --
i ~ L ~- + -
1W W Q 1& W
fW
Fig. 5 Comparison of simulated and aoalytkalsatura
tln tbrugput for a single BS versus packet si
(IMbp. DSSPH
UD,naw
I,PSIu, . ij&d%"""
---
AI Sl.lIb
ST A 11.l1b
Fig. 6 Refereact Senario ror tbe EIprimenta
Meauremenl
Not being possible to show all the resut obtained
during our reseach work, we present only one other
fige t stss the good correspondence between the
simulations r wt OPNET and !he expected !eo
retical results.
Fig. 5 collects the MAC layer satuation tughput
values of a BSS populated by 10 or 50 active stations
with a I Mbps DSSS physical layer (also in tis casc,
the physical layer has been chosen to perfonn a coher
ent comparson with the results prescnted in [6]) plot
ted versus the MAC payload size. The figure high
light a good corspondence between the to values,
with an error limited within 3% for the case of 10 ac
tive stations, and abut 4 to 8% in the 50 STA cae. As
a conclusion, the presented case studies, as well as
others not doumented here, highlight that te accu
racy of the results achievable using OPNET Modeler
simulation tool is adequte t adopt it as a tool to in
vestigate performance evaluation of 802.lIb wireless
access networks.
408
4 XQ6Om60f808$5
The last anlysis presented is based on the experimen
t approach. To perfonn the analysis, we set up an
experimentl testbed, depicted in Fig. 6, to verif te
corespondence between the teoretical goodput
achievable by a single UDP soure and the measured
one. The testbed is compsed by
a Lucent Orinoo A
)ODD Access Point a Linux baed Pentum II class
desktop PC which acts a a UDP sink, and a Linux
baed Pentiumll class laptop which acts a te UDP
Source. We chose MGEN [13} as the tafc genera
tion application, because of its wide confgurbility, in
ter of te types of statistcal properties of the gen
erated tafc (stochastic inter-ee times and
packet size). The A, connected directly t the UDP
sink, has an auto sensing 1011 OMbps auto duplex
Etheret interfce. The UDP sin is equipped with an
analogue NrC, hence t t cards autoconfgured for
a 100Mbps fll duplex connection.
4.1 Testng of trafc generator and meas
uring 1001
The first set of measurements was aimed at verifing
whether the imposed generation rate was actually pro
duced by MGEN, and at evaluatng the conditon
where the meaured rate was accurate with te equip
ments (mainly PC ad NIC) adopted in the experimen
tl scenario. To this aim, a simple pOint to point wired
conection between the UDP source a a UDP sink
has ben set up, and the packets originte at the
tansmittig applicaton have been dumped at the dat
lin layer. At the other extemity of te link, the re
ceived packet were duped at both the applicaton
and the data link layer, recording their timestmps.
The ue of a wired point-to-point connection permits
to evaluate the perfonnance of the used tools, without
the influence of the WLAN components. In order to
verif any limitaton of the generation rate due to the
application itself and the adopted devices, a piecewise
step fnction rate has been configd for generation,
raising the arrival rate of constant-sied packets at
fxed time interals. When considering large packets
(e_g. 612 Byes) we have not obsered any diference
beteen the aoWt of trafc set at generator side aod
that obsered at the receiving side until a tafic rate of
the order of the data rate of the IEEE 802-1lb tech
nology. The situation becomes ver diferent when us
ing ver small packets.
We have repeated the same test adopting the smallest
possible packet size generable by MGEN, i.e. 28
bytes, setting a nominal tansmitd dat rate which
increases of 250 kbps ever 30 seconds_ The result of
te meaurement are collected in Fig. 7; the highest
curve is the transmission rate, as it is measured at the
Data Link level of the source host whereas the other
two represent the rate measured at the receiver.
I
I
|
Fig. 7 Tansmission rat e of an UDP fow generated by
MGEN over a wired link (packet payload:28 bytes)
l
..
!a111M1mt
"t,"
--~ -, . . -"-- _ , : .. .
l
..~.-1 ....
;
.... .. - . .. -
... _.) . . ,...,,,, ... :,_, . ....- :.u,_
.`- : .. ,'-.
. . . . ~ +. ..... . :. .
.
.. J. ...L .
._... . . L ..
,
(" . .+ -
u....J ,
,
TV*I
. t=.
-..' 1 .
- .. ., , ". - , => -. . ..
Fig. 8 Comparison of tbe goodput obtained witb a
Orinoco Gold (above) and aD-Link DWL6S0 (below)
(payload=64 byt e. per packet)
Table Il: Measured Goodputs
UDP Payload
Siz [Bye]
6
576
1472
Onoo Gold
NIC[Mbpsl
0.569
3.55
6.06
DW50
NC [Mbpsl
0.59
3.7
6.2
Ihe lowest curve is obtined fom the measurements
petfmed at the Data Link level; lastly, the middle
one derives from the dump of the received packets at
the application level. It is straightforward to note that
the Data Link capture sufers of a severe limitation,
highlighted also by the number of discarded (i.e. not
dumped) frames reponed by the packet capture li
brry, i.e. libpcap. This may be due to the very high
packet rate necessar to generate a data rate of over I
Mbps using 28 byes long packets (note that each 250
kbps hig step corresponds to 1116 packets per sec
ond). Ihe d|0erence beteen the transmitted and the
received rate may be imputable to losses inside the
tansmitter network ampter, or to inaccuracies of the
dumping procedure a the receiver side. This topic is
40
being frher investigated adopting. diferent packet
generation and captre tools. However, as a result of
this preI|m|aapanalysis we can deduce that a feasible
number of measurable packets per seconds in oUr sce
nario is about 2000. Indeed, Fig. 7 h|ghlights that the
application level measurements allow to captre wit
out losses up to the secondstep.
4.2 Measurements in the WLAN sce
nario
In our experimental analysis, the laptop has been
equipped alterativelywith one of the following two
IEEE 802. lib NICs; a Lucent Orinoco Gold and a D
Link DWL650A baSed on Prism-chipset. The meas
urement campaign, frthertn evaluate the perorm
ance of the IEEE802.llb WA standard, permits to
compare the diferent performance achievable by
these to diverse couples of IEEE 802.11 b NICs and
Linux drivers. The first results seem to prove that a
limited but noticeable diference exists among these
two netork adapters. In more detail, considering the
plots repored in Fig. 8, the to graphs represent the
UDP transmitted and received rate using the two con
sidered PCMClA 802.lIb adapters in the UDP source
host. The plots are obtained varing the nominal
transmission rate of an UDP CBR fow, generated wt
MGEN, from 500 kbps t 5.5 Mbps, and measurng
tbe efective throughput at the Data Link level (at both
the tansmitter and rece|ver side) and at the applica
tion level at the receiver side, in order to verif the
presence of possibldosses inside the UP sink host.
These experiments have been caried out for diferent
UDP payload sizes (the plots of Fig. 8 refer to an UDP
payload size of 64 bytes), considering the limit of
2000 on te nominal packet generation rate found with
the analysis presented in the above paragraph.
The uppermost gaph is related to te perorce
achieve by the Lucent Orinoco Gold network
adapter, while the other one is obtained using a D
Link DWL650 based on Prism2.5chipset. The Ori
noo Gold NIC allows approach the maximum
theoretical value of 0.57 Mbps very closely; indeed,
the tsmission rate, averaged over the period 210-
330 sec, is about 0.569 Mbps. On the other hand, the
D-Link DWL650 allows to reach a slightly higher
throughput, even higher than the theoretical one
(about 0.59 Mbps in te period 60300 sec).
Note that a value of 64 bytes has been chosen instead
of 28 because of the detected instabil|t of the genera
tion process, as indicated in the subsection 4.1; in
deed, the packet rt related to the generation of a
600kbps rt is below 1200, which is lower than tbe
2200 limit obtained above. Table III sumarizes the
measured goodputs obta|ned with the three diferent
UDP payload sizes and the to considered
IEEE802.11b NICs, confrming the slightly higher
values obtained by the Prism2.5 based NIC wt re
spect to the Orinoco one.
Comparison of results
As the conclusion of our study, we compare the results
obtained considering the three approaches presented.
The results are qualitatively summarized in Fig. 9.
where we present the theoretical cure of goodput vs.
packet size when th" data rate at II Mps is consid
ered. In the same fgure. we also indicate the results
for 3 diferent packet sizes. obtained experimentally
using either the Orinoco Gold or the D-Link DWL650
NICs, or by simulation using OPNET (in this case the
analysis considers also a fourh value of packet size).
As we can observe from the picture, we have an en
couraging converence of the saturation goodput ob
tained using the diferent approaches which allows us
to assume the three approaches almost equivalent from
the results point of view. Table IV summarizes the
quantitative diference among the results prduced by
the diverse approaches. In particular, we can note the
slightly pessimistic results obtained using OPNET.
while minor differences can be noted comparing the
experimental and the theoretical ones.
.0
,$
, ..
,
I
I
Y
r
I
J.
I V
I
.'
I
10'
I
-
I I
.
I
,
, I
I
I
r
.
I ,
I
1
.....
..
IOoGdI1'1I
" E C1 J1
I
I
.
t
r
I
I
3
T 512 MD 7 10'115:IZ1401531M41721922211&
U)I"ZIlb)
Fig. 4 Comparison of analytcal, experimental and
simulated saturation god put for a .ingle UDP suurce
Table IV: Measured Goodputs
UP
Theoretic.1
Orinoo
OWL650 Simula-
Payload
Value
Gold
NIC tion
Siz
[Mbps]
NIC
[Mbps] (OPNET
[BJeJ [Mb),]
6 0.567 0.569 0.59 0.54
576 3.61 3.55 3.7 3.47
1472 6.1 t 6.06 6.2 5.94
Conclusions
The paper presents the analysis of the maximum
goodput achievable with an IEEE 802.11 b Basic Ser
vice Set, carried out following a theoretical, a simula
tive and an experimental approach. In particular, we
have taken into consideration a commercial discrete
event simulation tool, OPNET Modeler, analyzing the
802.11 model available as one of its standard libraries.
An extensive set of simulation rns has allowed us to
validate the simulation model versus theoretical re
sults known in literature, obtained using a Markovian
410
model. Such model accounts for all the exponential
backof protocol details. and is based on the assump
tion of constant and independent collision probability
of a packet transmitted by each station, regardless of
the number ofperfoned retransmissions.
Furhennore, an analytical evaluation of the saturation
goodput of a single IEEE 802.11 b tansmitter has been
conducted perfoning a timing budget of busy and
idle periods over the air interface. The most relevant
outcome of this work has been the pointing out of the
convergence towards such values of the results obtain
able with the simulative and, more considerably, ex
perimental approaches with a relative error of only
few percent, almost negligible fom an engineering
point of view. As a side result, the measurements pre
sented in Ihe paper, which have been conducted Oi our
experimental test-bed, highlight a limited, although
noticeable, diference between the maximum goodput
achievable using diferent IEEE 802.11 b compliant
wireless network interfaces, such as Ornoco and
Prism2.5 based ones.
7 Acknowledgements
This work has been carried out under the fnancial
suppor of te Italian Ministry for Education and Sci
entific Research (MIUR) in the famework of the
VICOM project.
References
[ I ] ANS!EEE SIr801.11 "Pa II: Wireless LAN Medium Access
Contol (MC) and Physicl Lye (PHY) Spc .... 1999 E.
[2] ANS!EEE St 802.llb "Par 11: Wirel ... LN Medium Access
Contol (MAC) ad Ph1"ic1 Lay (PHY) Spe.: Higher.Speed
Ph)ical uyer Ext. in the 2.4 OHz Bad", Sept. 1999
[3] http://standards.ieee.orgigetieee8021802.II.html.
[4] httpJ/grouper.ieee.orgigroups802l11IRepnstgg_update.htm
[5J hnp:l/groupr.ieee.otlgroup802l11IReptsltg_update.htm
[6] G. Bianchi, "Peronance Analysis of the IEEE 802.11 Distrib
uted Cc)rdination FunctiOll". IEEE Joural on Selecte Areas in
CQllmunications. Vol. 18, N. 3, March 200, pp. 535-541
[7) G. Anastasi. E. Borgi M. Conti. E. Gregori. "IEEE 802.1 t Ad
Ho NClwOrks; Peromlance Measu.-elllerlt:", Worksho on Mo
bile and Wireless Networks (MWN 2003) May 19. 20} Provi
dence, Rhode Island USA.
[8J D. P. Hole, f. A Tob.gi. "Capacity of M IEEE 802.llb Wireless
LAN supporing VoIP", submitted to ICC 2004
[9] G. Aatai, L Lenzini. "QoS provide by the IEEE 802.11
WLAN to advanced data applicalions a simulation study", Wire
les! Netos. Vol. 6, J.c. Bahzcr AG, Science Publish.rs. 200
rIO] OPNET Moeler web site www.opnet.corh
fll] s. Gar! a M. Kappes, "Al exprimental study oflhrougput
for VOP and Vol !raffle ;n IEEE 802.11 b networks", IEEE
Willess Communicat ions ad Nelworkmg 2003
[12] M. G. Arn. R. Aguero, L.Muioz and p, Mahonen, "Behaviour
of VOI'Based Application, over IEEE 802.1 t Wireless Net
wOTks", Pro. Qf te I Zth IEEE Interational Symposium on Per
,on.llndoo, and Mobile Radio Comm .. PIMRC 2001
[131 MGEN - htlp:llranimac.itd.nrLn.vy.miIlMGENI