The missing piece : when Docker networking and services finally unleashes software architecture 2.0
•Download as PPTX, PDF•
7 likes•2,096 views
Docker now provides several building blocks, combining engine, clustering, and componentization, while the new networking and service features enable many new usecases such as multi-tenancy. In this session, you will first discover the new experimental networking and service features expected soon, and then drift rapidly to software architecture, explaining how a complete Docker stack unleashes microservices paradigms. The first part of the talk will introduce what SDNs and service registries are to the audience and will cover corresponding network & service experimental features of docker accordingly, with a technical focus. For instance, it explains how to create an overlay network of top of a swarm cluster or how to publish services. The second part of the talk moves from infrastructure to application concerns, explaining that application architecture paradigms are shifting. In particular, we discuss the growing porosity of companies’s IS (especially due to massive use of cloud services) drifting security boundaries from the global IS perimeter, to the application shape. We also remind that traditional SOA patterns leveraging on buses (ie. ESBs & ETLs) are being replaced by microservices promoting more direct, full-mesh, interactions. To get the picture really complete, we’ll also rapidely remind other trends and shifts which are already covered by other docker components: scalability & resiliency to be supported by the apps themselves, fine-grained applications, or even infrastructure commoditization… Most of all, the last part depicts a concrete, state-of-the-art application, applying all the properties discussed previously, and leveraging on a multi-tenant docker full stack using new networking and services features, in addition to traditional swarm, compose, and engine components. And just because we say it doesn’t mean it’s true, we’ll be happy to demonstrate this live !
Report
Share
The missing piece : when Docker networking and services finally unleashes software architecture 2.0
- 1. The missing piece: when Docker networking unleashes software architecture 2.0 A. Blind DevOps coach Societe Generale @adrienblind L. Grangeau Solutions architect Finaxys @laurentgrangeau
- 2. Agenda 2 - Starters Docker networking & volume features discovered 4 - Dessert Taste-an-app 1 - Apetizer Back on current Docker paradigms 3 - Main course Application architecture shifts
- 3. Back on current docker paradigms
- 4. Back on Docker paradigms ‘’A universal, self-sufficient and standard artifact embedding an app module, and its subsequent infrastructure configuration’’ Immutable Versionned Light Portable Disposable Programatic Social Incremental It’s mainly focused on enclosing computing capabilities: what about storage ? Network ?
- 5. Networking & volume features discovered
- 8. Docker networking The Container Network Model (CNM) A docker container Endpoint A docker container Endpoint A docker container EndpointEndpoint Network sandbox Network sandbox Network sandbox Front network Back network
- 9. Docker networking $ docker network create mynetwork 5000dec7c180a63d87031de7e6bfcf2b25cf1e5daef6338f16fbd4 451210a938 $ docker network create –d overlay multihostnetwork e6537b859359843bc02392245ab226070f79dbf87be2d492969c84 3f89fb6de6
- 10. Docker networking $ docker network inspect mynetwork [ { "Name": "mynetwork", "Id": "5000dec7c180a63d87031de7e6bfcf2b25cf1e5daef6338f16fbd4451 210a938", "Scope": "local", "Driver": "bridge", "IPAM": { "Driver": "default", "Config": [ {} ] }, "Containers": {}, "Options": {} } ]
- 11. Docker networking Host Host Host Host SDNs SDN 1 SDN 2 SDN 3
- 13. Docker networking Docker Compose evolved to embrace new networking features $ docker-compose --x-networking --x-network-driver=overlay up $ docker-compose up
- 14. Docker volumes
- 15. Docker volumes Host file system Host file system ‘’Former data management locked in a host’’
- 16. Docker volumes Host file system Container Volume ‘’Containers mount a volume which may be backed externaly’’
- 17. Docker volumes $ docker volume create –d volplugin --name pool/name Cf872ca21d27843f6b6319ac1a34390dd38d94ed4649cd985456d5 23fb05d4cc $ docker run –d –p 8080:8080 –v pool/name:/var/jenkins_home jenkins 96aec6f4e45e050dfb4f75a1009e7f105bced5b406752e62d47061 5d07348b07
- 18. Docker volumes $ docker volume ls DRIVER VOLUME NAME local cf872ca21d27843f6b6319ac1a34390… local f19f50251f48c64a6b33a5c637c2330… $ docker volume inspect cf872ca21d27843f6b6319ac1a34390dd38d94… [ { "Name": "cf872ca21d27843f6b6319ac1a34390dd38d94…", "Driver": "local", "Mountpoint": "/mnt/sda1/var/lib/docker/volumes/[…]/_data" } ]
- 19. Take-away
- 21. Application Compute (Run containers) Storage (Volumes) ‘’Immutability of containers, resiliency & scalability led to data externalization in separate objects’’
- 22. ‘’Fine-granularity of containers led to closely interconnect them’’ Application Compute (Run containers) Storage (Volumes) Transport (Network)
- 23. ‘’The whole topology can now be described’’ Application Compute (Run containers) Storage (Volumes) Transport (Network) Topology (Compose)
- 24. ‘’Docker finally shifted to object-oriented infra. architecture’’ Application Compute (Run containers) Storage (Volumes) Transport (Network) Topology (Compose) CaaS platform (Swarm, Machine...)
- 27. Security paradigms shifts Your IT opens up • Externalization (housing, hosting) • Cloud (IaaS/PaaS/SaaS) Open up your IS • B2B, services exposition • Multi tenancy More & more breaches appears in your Great Wall of China!
- 28. Security paradigms shifts The necessary porosity of your IS requires to stick security closer to each application: sandbox your apps and expose protected interfaces! Network is part of application topology Security is an app topic, not just infra. concern Onboard security in feature teamSecDevOps
- 29. Network paradigms shifts VM VM VM VM VM VM VM VM VM Internet Internet DMZ Physical overview Logical overview Tenant #1 Tenant #2 LAN LAN DMZ1 DMZ2 Traditional networks relies a lot on low layers (L2, etc.) Application topologies are quite different from physical ones
- 30. Network paradigms shifts SDNs propose network solutions embracing cloud paradigms Massively multi-tenant Thousands tenants, massively scalable Easy & fast (de)provisioning Infra as code, API centric Infrastructure agnostic L3, does not stick with lower levels (physical designs, vlans & co) Decouple infrastructure & tenants lifecycles Cross technology, vendor agnostic
- 31. From Enterprise Services buses to full-mesh topologies ESB Service Service Service Service Service > ServiceService Service Service Service Micro services
- 32. Fine-grained, highly decoupled and atomic purpose centric services Designed for failure Multi-versioned Scalable Micro services Stateless Share-nothing Immutable Continuously delivered Distributed
- 33. Service consumer Service provider Registry 2. Find 1. Publish 3. Bind Leverage on a Service registry to discover where are services located Micro services
- 34. Resilience & scalability: apps problem now! Vertical > horizontal Apps designed for failure & scalability Data to be externalized Dumber infrastructure Structured: MongoDB, Hadoop, Cassandra, Elastic Search... Binaries: object storage with Ceph, OpenStack Swift... Helpful patterns: stateless, multi-versioning, loose coupling... Infrastructure rationalization Low-cost, poor-SLA commodity
- 35. « Organizations which design systems... are constrained to produce designs which are copies of the communication structures of these organizations ». - M. Conway, 1968 Consider shifting your organization if you wish to shift your architecture Forget about the central architects myth of organizing, integrating everything Consider changing your organization to expect changing the architecture! promote feature teams Organization
- 36. Docker suits perfectly new applications challenges Create docker networks to isolate applications Docker container properties fits micro-services challenges Resilience & scalability is mostly about multiplying containers Expect to discuss roles shift in organization
- 37. Taste-an-app
- 38. Execution platform Registry Docker-machine Docker-swarm The registrator discovers new containers and feeds the registry
- 39. Application design Provider micro serviceConsumers The python app module exposes a REST service searching information in the MongoDB The NGINX reverse proxy forward app. requests on one of the python instance registered in Consul Find
- 40. Application topology & runtime The whole application topology is stored as: docker-compose yaml file docker-compose args (aka --x-networking & --x-network-driver) You can scale up or down the python instances of the micro- service using traditionnal docker-compose scale command
- 41. Network view Only the load balancer VIP is exposed externally A WAF instance could secure this entrypoint SDN « dockerconeu15 » Host network Provider micro serviceConsumers
- 42. Network view - advanced Provider micro service Consumers SDN « front » SDN « back » Host network Back Middle Front ‘’To enhance security you may decouple each application tier’’
- 43. Zoom on the registry usages Between micro-services, consumers asks the registry where a desired micro-service is located Inside a micro-service, NGINX is made aware of the backend API instances available, via the registry At container level, the registrator enable to registers any container instances, grouped per type At infrastructure level, the registry is used by swarm (internally) to be aware of the cluster’s participants Noticed the different usages of a registry ? You may consider using different registries for each usage : for example an internal registry for the micro service internal topology
- 44. Conclusions
- 45. Docker shifted from universal containers to object-oriented infrastructure Security is an app concern Software is eating the world: application architecture is the key, infrastructure is commodity
- 46. Thank you! A. Blind @adrienblind adrien.blind@sgcib.com L. Grangeau @laurentgrangeau laurent.grangeau@gmail.com