Abstract
This paper presents Occopus, an open-source cloud orchestration and management framework for heterogeneous multi-cloud platforms. Occopus provides a language to specify infrastructure descriptions and node definitions based on which Occopus can automatically deploy and maintain the specified virtual infrastructures in the target clouds. The paper introduces the required structure of the infrastructure descriptions and node definitions in an informal way and shows two use cases (Hadoop cluster and MICADO framework) how Occopus can be used to deploy complex virtual infrastructures. The paper also explains the architecture and implementation aspects of Occopus and describes its main distinguishing features compared to other cloud orchestrator frameworks.
Similar content being viewed by others
References
Buyya, R., Broberg, J., Goscinski, A.M.: Cloud Computing: Principles and Paradigms. Wiley, Hoboken (2011)
Sahoo, J., Mohapatra, S., Lath, R.: Virtualization: a survey on concepts, taxonomy and associated security issues. In: 2010 Second International Conference on Computer and Network Technology, IEEE, Bangkok, Thailand, pp. 222–226 (2010)
The MTA Cloud infrastructure, https://cloud.mta.hu
Mell, P., Grance, T.: The NIST definition of Cloud computing. NIST Special Publication 800-145 (Final) Technical Report. http://csrc.nist.gov/publications/nistpubs/800-145/SP800-145.pdf (2011)
The EGI Federated Cloud, https://www.egi.eu/services/cloud-compute
Chadwick, D.W., Siu, K., Lee, C., Fouillat, Y., Germonville, D.: Adding federated identity management to openstack. J Grid Comput. 12(1), 3–27 (2014)
Toraldo, G.: OpenNebula 3 Cloud Computing, Packt Publishing Ltd. ISBN1849517479, 9781849517478 (2012)
AgroDAT.hu project, http://agrodat.hu
COLA: Cloud Orchestration at the Level of Application, http://www.project-cola.eu
CloudSigma, http://www.cloudsigma.com
Amazon Web Services, https://aws.amazon.com
CloudBroker, http://cloudbroker.com
Occopus homepage, http://occopus.lpds.sztaki.hu
Petcu, D.: Consuming resources and services from multiple clouds. J of Grid Comput. 12.2, 321–345 (2014)
EGI AppDB, https://appdb.egi.eu
Heap, M.: Ansible: From Beginner to Pro, Apress, eBook ISBN 978-1-4842-1659-0, https://doi.org/10.1007/978-1-4842-1659-0, Softcover ISBN 978-1-4842-1660-6, Number of Pages XVII, 170 (2016)
Chef, https://www.chef.io/chef
Puppet, https://puppet.com
Peinl, R., Holzschuher, F., Pfitzer, F.: Docker cluster management for the cloud-survey results and own solution. J Grid Comput. 14.2, 265–282 (2016)
Netto, H.V., Lung, L.C., Correia, M., Luiz, A.F., de Souza, L.M.: State machine replication in containers managed by Kubernetes. J Syst. Archit. 73, 53–59 (2017). ISSN 1383–7621
Apache Mesos, http://mesos.apache.org
Apache jClouds, https://jclouds.apache.org
Apache libcloud, https://libcloud.apache.org
boto: A Python interface to Amazon Web Services, http://boto.cloudhackers.com
Fog, http://fog.io
Ironic Support in Red Hat OpenStack Platform, https://access.redhat.com/articles/2331251
Pham, L.M., Tchana, A., Donsez, D., De Palma, N., Zurczak, V., et al.: Roboconf: a hybrid cloud orchestrator to deploy complex applications. In: 2015 IEEE 8Th International Conference on Cloud Computing, New York, United States (2015). https://doi.org/10.1109/CLOUD.2015.56.hal-01228353
Wang, X., Liu, Z., Qi, Y., Li, J.: Livecloud: a lucid orchestrator for cloud datacenters. In: 4th IEEE International Conference on Cloud Computing Technology and Science Proceedings, Taipei, pp. 341–348 (2012). https://doi.org/10.1109/CloudCom.2012.6427544
Le, D.-H., Truong, H.-L., Copil, G., Nastic, S., Dustdar, S.: SALSA: a framework for dynamic configuration of cloud services. In: Proceedings of the 2014 IEEE 6th International Conference on Cloud Computing Technology and Science (CLOUDCOM ’14), pp. 146–153. IEEE Computer Society, Washington, DC (2014). https://doi.org/10.1109/CloudCom.2014.99
Caballer, M., Segrelles, D., Moltó, G., Blanquer, I.: A platform to deploy customized scientific virtual infrastructures on the cloud. Concurrency Computat.: Pract. Exper. 27, 4318–4329 (2015). https://doi.org/10.1002/cpe.3518
Caballer, M., Blanquer, I., Moltó, G., de Alfonso, C.: Dynamic management of virtual infrastructures. J. Grid Comput. 13(1), 53–70 (2015)
Cloudify, http://getcloudify.org
Cloudformation, https://aws.amazon.com/cloudformation
Terraform, https://www.terraform.io
Cloud-init, https://cloud-init.io
Prometheus, https://prometheus.io
Merkel, D.: Docker: lightweight Linux containers for consistent development and deployment. Linux J. 2014, 239 (2014)
Occopus Users’ Guide, http://occopus.lpds.sztaki.hu/user-guide
Jinja2, http://jinja.pocoo.org/docs/2.9
Software Design patterns, https://en.wikipedia.org/wiki/Software_design_pattern
YAML Ain’t Markup Language, http://yaml.org
Shvachko, K., Kuang, H., Radia, S., Chansler, R.: The Hadoop distributed file system. In: 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST), Incline Village, NV, USA, pp. 1–10 (2010)
Dean, J., Ghemawat, S.: Mapreduce: simplified data processing on large clusters. Commun. ACM 51(1), 107–113 (2008)
Occopus Tutorials, http://occopus.lpds.sztaki.hu/tutorials
Visti, H., Kiss, T., Terstyanszky, G., Gesmier, G., Winter, S.: MiCADO – Towards a Microservice-based Cloud Application-level Dynamic Orchestrator, Proceedings of IWSG’2016, Rome (2016)
HAProxy, http://www.haproxy.org
Occopus source code repository, https://github.com/occopus
TOSCA: Topology and orchestration specification for cloud applications version 1.0. http://docs.oasis-open.org/tosca/TOSCA/v1.0/TOSCA-v1.0.html (2013)
Acknowledgements
This work was partially funded by the European COLA - Cloud Orchestration at the Level of Application project under grant No. 731574 (H2020-ICT-2016-1), and by the European Union Horizon 2020 research and innovation program under grant No. 644179 (ENTICE), and by the National Research, Development and Innovation Fund of Hungary under grant No. VKSZ 12-1-2013-0024 (Agrodat.hu), and by the International Science & Technology Cooperation Program of China under grant No. 2015DFE12860. On behalf of the Occopus project we thank for the usage of MTA Cloud (https://cloud.mta.hu/) that significantly helped us achieving the results published in this paper. The authors are grateful to Botond Rákóczi and Enikő Nagy for their valuable help in producing the presented use cases.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Kovács, J., Kacsuk, P. Occopus: a Multi-Cloud Orchestrator to Deploy and Manage Complex Scientific Infrastructures. J Grid Computing 16, 19–37 (2018). https://doi.org/10.1007/s10723-017-9421-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10723-017-9421-3