2015 Fifth International Conference on Advances in Computing and Communications

A Comparative Study of OpenStack and CloudStack

Jaison Paul Mulerikkal, PhD Yedhu Sastri

Rajagiri School of Engineering and Technology Rajagiri School of Engineering and Technology
Kerala, India 682039 Kerala, India 682039
Email: jaisonmpaul@rajagiritech.ac.in Email: yedhusastri@gmail.com

Abstract—Even though of its very recent origins, Cloud There are two types of private cloud middlewares avaialble -
Computing has matured into a main stream technology over proprietary and open-source.
the past few years. Private cloud, where an organization sets up
an internal cloud infrastructure, is gaining traction these days The big players in private cloud business like VMWare,
because of its perceived security advantages. Two of the major Microsoft, HP, IBM etc offer their own enterprise class so-
open source cloud middlewares are OpenStack and CloudStack. lutions to build private cloud for an organization. But their
This paper provides a comparative study of these two cloud actual code remains proprietary which restricts the customers
middlewares regarding its implementation complexity, overall to modify the source code to customise it for their specific
stability and performance comparison using Unixbench and needs, especially for research and academic purposes.
Bonnie++ benchmarks. Experiment results show that OpenStack
perform better in most scenarios in a single node environment. On the other hand, we have couple of open-source private
OpenStack exhibits better overall stability but at the cost of cloud middlewares that have enough maturity, sophistication
increased installation complexity. and convenience to provide reasonably good cloud infrastruc-
Keywords—Private Cloud, OpenStack, CloudStack ture for interested parties. Most of them have developed in
research circles as a natural transformation from grid com-
puting platforms to cloud computing platforms, so very much
I. I NTRODUCTION suited for . OpenStack and CloudStack belong to this category.
Today most of the large and small-scale businesses are
turning towards cloud computing for better business IT solu- A. Paper Organization and Motivation
tions. The academic community is also adopting it for research
In this paper a basic study and comparison of the above two
and learning purposes. Based on the service model, cloud
popular private cloud middlewares are conducted and results
computing can be generally classified into three - private cloud,
are plotted. Both OpenStack and CloudStack are deployed in
public cloud and hybrid cloud. Private cloud refers to cloud
the same machine, one after another and their architectural and
environment with in an organization whose services are not
implementation complexities are discussed. We have also con-
accessible by the public. Only permitted members within the
ducted performance evaluation of OpenStack and CloudStack
organization can access the services of a private cloud. On
using two benchmarks - Bonnie++ and Unixbench and the
the other hand the services offered by the public cloud are
results discussed. We also make comments about their overall
supposed to be accessible to the public from anywhere in the
stability, based on our experience in working with them.
world. A mix of these two models are considered as a hybrid
cloud[1]. A performance benchmarking of OpenStack and Cloud-
Stack is done in [3]. However, it is mainly concerned about
According to the State of Cloud Report (2014) by
a comparison of cloud-management features like virtual node
RightScale, based on a survey among technical professionals
deployment time, deletion time, CPU utilization during deploy-
across a broad cross-section of business organizations, at
ment and deletion, etc. But in this paper, we are concerned
least 94% of their respondents are adopting cloud computing
more about the overall system health, once the cloud environ-
technologies. Interestingly almost 88% of them are using some
ment is deployed using the above cloud middlewares. We hope
kind of private cloud solutions for their business IT infrastruc-
that this discussion will help new cloud adopters to choose the
ture [2]. It is noted that three out of six most popular private
right cloud-middleware that gives better performance during
cloud solutions are open-source products. They are OpenStack,
run-time.
CloudStack and Eucalyptus, in that order [2]. This paper is a
study on the installation complexity, benchmark performances
and overall stability of the top two cloud middlewares in that III. C OMPARISON : A RCHITECTURE & I NSTALLATION
list - OpenStack and CloudStack. A. OpenStack

II. BACKGROUND OpenStack began as a joint project of Rackspace Hosting

and NASA in 2010. It is a collection of software pieces
A private cloud can be installed in an organization only that is aimed to provide comprehensive cloud services by
with the help of a private cloud middleware. A cloud mid- virtualizing network, storage and compute resources. Figure
dleware is a tool that helps to setup the cloud environment 1 shows an outline of OpenStack architecture [4]. There are
which acts in the middle of cloud resources and the client and mainly three nodes - Compute node for virtualization Network
provisions those resource according to the needs of the client. node for networking and Controller node for Controlling the

978-1-4673-6994-7/15 $31.00 © 2015 IEEE 81

DOI 10.1109/ICACC.2015.110
Authorized licensed use limited to: Malta College of ArtsScience&Tech. Downloaded on March 05,2024 at 09:35:40 UTC from IEEE Xplore. Restrictions apply.
 environment. Internal communication between these node are three functions - management, networking and computation
through management network. Virtual machine communication in a single machine. OpenStack community has developed
between each other and to external world is through tunnel DevStack - an opinionated script for easy installation of
network. Then communication with external world is through OpenStack on a single node. In this paper, we use single node
external network. installations of OpenStack and CloudStack for performance
evaluations.

Table 1.Comparison Table

D. Discussion
Fig. 1: OpenStack Architecture Outline
A general comparison of CloudStack (4.3.2) and Open-
Stack (Juno) are given in table 1. As mentioned earlier, the
general architecture of both CloudStack and OpenStack is a
B. CloudStack three node architecture.
CloudStack which was originally developed by Cloud.com Regarding installation complexity, OpenStack is more dif-
and later supported by Citrix. Now it is an Apache incubator ficult to deploy than that of CloudStack. This is because,
project known by name apache cloudstack. Figure 2 shows a OpenStack is a collection of different projects. So, we have to
basic outline of CloudStack architecture [5]. The three nodes install and configure each projects separately and that process
are 1) Supervisor node for managing the entire network and 2) is comparatively difficult than that of CloudStack. That is, in
VM creator for creating virtual machines and 3) Storage server CloudStack we need to bother only about configuring different
for storing disk images. Here there are no separate networks for servers, whereas in OpenStack we need to separately install
each communication. Internal communication is done through and configure the projects in different servers.
internal network and external communication is done through
normal Internet. Hypervisor is the backbone of virtualization. Both Cloud-
Stack and OpenStack have support for almost all of the leading
hypervisors. But CloudStack is much more compatible with
citrix XenServer since it was initailly owned by citirx. On the
other hand, OpenStack provides more support for container
virtualization technologies including LXC and Docker.
Both OpenStack and CloudStack warrants proper network-
ing between the servers and clients. Networking part is diffi-
cult in both CloudStack and OpenStack. Networking includes
physical switches, firewall configurations, virtual devices etc.
For storage purposes both OpenStack and CloudStack
envisages separate storage servers. OpenStack has two seperate
projects dealing with storage - namely, swift and cinder. In
cinder Logical Volume Manager(LVM) volume groups are
created for giving block storage to VM’s. Swift deals with
object storage. Swift is used for backup. In CloudStack a
separate storage server is mounted to the management server
using NFS. Regarding Operating System compatibility, both
Fig. 2: CloudStack Architecture Outline CloudStack and OpenStack support the following linux flavors:
CentOS, RHEL and Ubuntu . But OpenStack has additional
support for OpenSuse and Fedora.

C. Single Node Installation IV. P ERFORMANCE A NALYSIS

Both OpenStack and CloudStack have a basic installation Different sets of performance tests are conducted using
that can be done on a single node which can host all the single node installation of CloudStack and OpenStack on

82

Authorized licensed use limited to: Malta College of ArtsScience&Tech. Downloaded on March 05,2024 at 09:35:40 UTC from IEEE Xplore. Restrictions apply.
 the same machine first installed with CloudStack and then that of CloudStack. OpenStack has a high index score of
with OpenStack. We used a Dell Precision T1600 4 core 2002.2 than that of CloudStack which has scored 1443.7.
workstation with 8GB DDR3 memory for the purpose. Cisco
28 port gigabit managed switch is used for networking. Both
CloudStack and OpenStack uses KVM as hypervisor, in this
setup.
Two benchmark tools - Unixbench [7] and Bonnie++ [8] -
are used to conduct the experiments.

A. Unixbench Experiments
Unixbench is a collection of about 10 different tests that
are bundled together to provide a basic indicator of the perfor-
mance of a Unix-like system. This is a system benchmark, and
not a CPU, RAM or disk benchmark. That is why it is deemed
to better represent the system health of a cloud middleware. A
number of experiments as given in Table 2 are conducted and
a final index score is generated by Unixbench based on those Fig. 3: UnixBench Final Index Value
experiment results. Higher the index score, better will be the
system health.
Unixbench experiments on CloudStack were conducted B. Bonnie++ Experiments
earlier in the same lab and on the same machine and the Bonnie++ consists of a number of IO performance tests
results were published in [5]. Those results are compared that will benchmark how the file systems perform with respect
against freshly conducted experiments on the same machine, to data read and write speed. Dataset with a fixed size is
with OpenStack installation. Table 2 shows the comparison of given and multiple tests to measure the reading speed, writing
UnixBench benchmarks of CloudStack with that of OpenStack. speed, CPU utilization, execution speed etc are conducted. In
our experiments we used a dataset of 40 MB to test both
OpenStack and CloudStack environments.
Figure 4 shows the speed of writing the dataset in KB/s.
It shows that there is no much difference in writing speed
between CloudStack and OpenStack. For CloudStack it is
about 1208 KB/s and for OpenStack it is about 1181 KB/s.

Table 2.Comparison: Unixbench Results

Table 2 shows that OpenStack has higher index values Fig. 4: File Writing
than CloudStack in all the Unixbench benchmarks. Dhrystone
index value shows that OpenStack is better than CloudStack
in string handling capability. Whetstone benchmark indicates Figure 5 shows the results of testing the reading speed.
that OpenStack has better index value in performing floating CloudStack has a reading speed of 5673 KB/s and OpenStack
point operations. has a reading speed of 5956 KB/s. It clearly suggests that
OpenStack supports considerably faster reading speed than
File handling tests includes reading writing and copying CloudStack. This result corroborates the higher file handling
files using various buffer size. In this case too, OpenSatck scores we received for OpenStack in Unixbench experiments.
performs better than CloudStack.
Figure 6 shows the CPU utilization of both OpenStack
Based on these results a final index score is generated by and CloudStack while performing these experiments. It is
Unixbench for both OpenStack and CloudStack (Refer Figure calculated in percentage utilization. Both CloudStack and
3). OpenStack have comparatively high final index score than OpenStack have a CPU utilization of around 95%.

83

Authorized licensed use limited to: Malta College of ArtsScience&Tech. Downloaded on March 05,2024 at 09:35:40 UTC from IEEE Xplore. Restrictions apply.
 Fig. 5: File Reading
Fig. 7: Total Time

VII. C ONCLUSION
This paper presents the general architecture and discusses
the installation complexity of CloudStack and OpenStack
cloud middlewares. Both CloudStack and OpenStack are ar-
chitecturally similar but OpenStack seems to have higher
installation complexity. But that additional effort is compen-
sated in the better overall stability it provides against Cloud-
Stack. Both Unixbench and Bonnie++ experiments suggest
that Openstack environment gives better system performance
(after deployment) in almost all tested benchmarks. Moreover,
Fig. 6: CPU Utilization we feel that the general gravitation of major community and
enterprise initiatives towards OpenStack also makes it a more
attractive option than CloudStack to setup a private cloud in
an organization.
Figure 7 shows the round-the-clock time taken by Cloud-
Stack and OpenStack to complete all these experiments. From VIII. ACKNOWLEDGEMENT
the figure it is clear that OpenStack takes much lesser time
than CloudStack to complete these experiments. OpenStack We are extremely grateful to Rajagiri School of Engineer-
takes only 26159 micro seconds to complete the experiments ing and Technology (RSET), Kochi for supporting this research
while CloudStack takes about 44167 micro seconds. through its internal research grant titled ”OpenStack Cloud
Installation Project” at RSET.

V. OVERALL S TABILITY R EFERENCES

[1] J. Mulerikkal, P. Strazdins and B. Thekkanath, High Performance Cloud
While working with CloudStack and OpenStack, after their Computing Using an Efficient Data Service, In proceedings of IEEE Asia
installations, we felt that OpenStack is more mature and stable. Pacific Cloud Computing Congress, November 2012, Shenzhen, China.
One of the issues we faced with CloudStack was its inability to [2] State of the Cloud Report, RightScale Ltd, Whitepaper, 2014.
locate primary and secondary storage systems, once installation [3] A. Paradowski, L Liu, B. Yuan, Benchmarking the Performance of
is complete. We found that it was mainly because of the OpenStack and CloudStack, In Proceedings of IEEE 17th International
problems in NFS mounting of Storage server. Other issue was Symposium on Object/Component-Oriented Real-Time Distributed Com-
puting, 2014
the disappearance of working virtual machines after a reboot,
in CloudStack. This seems to be a much reported issue of [4] OpenStack Documentation, http://docs.openstack.org/juno/install-guide/
install/apt/content/ch overview.html
Cloudstack. [9]
[5] J. P. Martin, C. Krishna, M. J. Hareesh, S. Anish Babu, S. Cherian
and Y. Sastri, Learning Environment as a Service (LEaaS): Cloud,
IEEE Fourth International Conference on Advances in Computing and
VI. F UTURE W ORKS Communications, Kochi, 2014
[6] S. Anish Babu, M. J. Hareesh, J. P. Martin, S. Cherian and Y
This work is a progression of the work conducted in Sastri,System Performance evaluation of Para virtualization, Container
[5][6], where Unixbench benchmarks were tested for a single virtualization and Full virtualization using Xen, OpenVZ and XenServer,
IEEE Fourth International Conference on Advances in Computing and
node CloudStack environment. Now we have extended that to Communications, Kochi, 2014
OpenStack and further verified those results using Bonnie++
[7] Unixbench (Webpage), http://code.google.com/p/byte-unixbench/
performance tool. The next step will be to conduct the same
[8] Bonnie++ (Webpage), http://www.coker.com.au/bonnie++/
experiments in three node architectures of both CloudStack
[9] Virtual Machine Disappearing, Cloudstack-users Mailing List Archives
and OpenStack, which is beyond the scope of this paper. (Webpage) http://goo.gl/oBtzdf

84

Authorized licensed use limited to: Malta College of ArtsScience&Tech. Downloaded on March 05,2024 at 09:35:40 UTC from IEEE Xplore. Restrictions apply.