Docker:- Application Delivery Platform Towards Edge Computing

Editor's Notes

1. We will start our presentation with the motivation towards EC Introduction about what is actually Edge computing Suitable EC Use cases Technology that may support EC Demo Conclusion
2. Currently What is IoT - embedded with electronics, software, sensors, and network connectivity, which enables these objects to collect and exchange data.
3. Deemphasizing - Centralized to decentralized - OR DC oriented to DISTRIBUTED. It provides Opportunity to Leverage resources for example compute, storage, network, sensors or actuators that is closer to user, devices, data. Other terms Mobile edge computing Virtual cloudlet Fog computing They have overlapping objectives, with slight different architecture. But still we can learn from it.
4. ETSI – European Telecommunication Standards Institute
5. Here we discuss some of the application requirements that is suitable for Edge computing. At the center is DC or Cloud The outer ring is the edge resource and all the devices such as sensors, or infrastructure that can communicate with the Edge Computing. From the application or user perspective – the latency will be lower from the extreme edge and higher when it reaches the DC. Edge can provide real time analytics while DC can provide heavy business inteligence. Lets take a look at some of the suitable application requirements Performance requirement, is there a low latency requirement that will impact where the workload should be process? For example gaming, safety or even web application. Process locally – is it better to run app or process data at the edge instead of sending huge amount of data back to DC? Data processing opportunities – if how ever you need to send huge data, does it make sense to process, compress or transmit selected data before transferring? For example, video surveillance, traffic monitoring? Distributed Application – You may one real-time analytics. Does processing at the edge is much more attractive? E.g. smart city, monitoring or even IoT A quote from CISCO which I think its true, “edge computing help ensure the right processing takes place at the right time and place”
6. Accelerate development - trying to setup developer environments, spin up new instances and make copies of production code to run locally Eliminate Environment Inconsistencies - packaging up the application with its configs and dependencies together and shipping as a container, the application will always work as designed locally, on another machine, in test or production.  https://www.docker.com/use-cases
7. Docker is a light Weight Container Technology for application It is an Open platform for developers and sysadmins to build, ship and run distributed app. Common use cases for Docker are Continuous Integration Continuous Delivery – build, deploy, test, release Infrastructure Optimization – hypervisor to container. . 
8. IF you have, website, DB, web front end, queuing or analytic DB. Every one of those can be packaged into a container image. When everything is package, you can deploy multiple environment from Development VM Or QA server Even on Contributor’s laptop.
9. Build better software SoC – Dev focus on development. Admin focus on deployment. Accelerate Development – same package/image run multiple places Eliminate env inconsistencies – Run anywhere. App work as designed locally, on another machine, in test or production.  Portability Can run on any Linux system that supports LXC (today). Raspberry Pi support Self sufficient Contains everything it needs to run – all the package, the configuration is embedded within the image itself. Can use Minimal base OS e.g. RancherOS (31MB), CoreOS which OS size is less than 50MB
10. Use the same technology for different domain. Docker Engine meets our needs Application Provisioning Simplifies distribution, installation & execution of app. Highly portable. Hardware isolated. Remote management App easy to update & remove. Self-contained, pre-configured & Easy to manage Can run on small devices Lightweight & Small footprint
11. The edge will have DC to Edge interaction. With multiple sites distributed. Whether it is App for users or businesses Smart cities IoT Video surveillance All will have a resources pool to serve the need of the Edge. After doing our evaluation on Docker, we think Docker is suitable for application delivery for Edge Computing. In the next slide we will demo on Discovery Deployment Offloading On each of the demo, we will discuss our observation results.
12. NEW Remote prep and joining. Discovery of devices Self-Organization Ability to self-organize in Ad Hoc/Infrastructure setting to discover resources (services/content/users/devices) and Communicate. Talk before demo (Bottom left) Existing node started the discovery service (Top) Mi-FOKUS to show the nodes (Bottom right) New physical host start the discovery Existing host will discover the new node Existing host run a remote command to configure the new physical host to join the Mi-FOCUS Mi-FOCUS will see the new node in seconds
13. Change to static web page. Value to demo Scenario 1 – Typical HA setup SCRIPT Deploy LB – show LB page Deploy 2 web server Show its up. Scenario 2- Rapid scaling & teardown SCRIPT As more user access We scale rapidly the web servers When not needed – we down size the web servers Value it shows *This scaling concept is achieve able in cloud – where on-demand, auto-scaling comes into the picture. But here, in Edge computing, we would like to have it even faster. As the devices at the edge might be smaller in size, and we wanted it to have space for a new application. *this shows the power of Docker to scale fast and at the same time deprovision fast. Good for edge computing. Where we can swap the applications and make for a new one. Add the step of demo Take about 1 minute Introduce the data center Talk before demo Sec 1 1 lb, 2 Mi-ESS (Bottom) LB (HA-Proxy), with configured two Mi-ESS (red) will attached the Mi-ESS later (top) container with two mi-ess (LB turn green) (New windows) – login to the Mi-ESS using the Load balancer Walk the demo with Bukh Sec 2 (bottom) Addition 6 Mi-ESS will configure in the LB (new still be red) (top) additional 6 Mi-ESS container deploy and start rapidly (LB turn green) Expect to be finished in seconds
14. How we perform the test Container BOOTTIME is captured by Get the STARTIME to run the container When the container ID returns, we know it is up. READYTIME A script will keep continuously monitor port 80. once up, get the ENDTIME
15. 1.Caveats on size 1.Developer must  adhered to the containerization technology to reap the benefit. 1.Store ONLY app on image. 2.Store DATA seperately from container. 3.Remove temporary files, caches, logs, etc before commit image into the repository. 2.Deployment 1.Create shared image layers i.e. generic OS layer, library layer, and app layer. Each layer can be shared by multiple images thus reduces the total of collective image size. 2.Caveats on start up time 1.Make the image size a small as possible to reduce image transfer time. 2.One application per container. Remove any unwanted application or service on the image during start time.
16. Benefits Lower storage usage Docker leverages a copy-on-write filesystem - AUFS. We can build image based on existing image. This allows Docker to instantiate – fast No full copies of Docker can basically use “pointers” back to existing files. Transfer only deltas change a Docker image—for example, update an application to a new version— a new layer gets built. Not replacing the whole image, only that layer is added or updated. Transfer only delta’s making distributing Docker images faster and simpler. Remote site in Malaysia – got limited or slow connection
17. 3. Caveats on Memory 1.*comparison of memory usage is based on application vs. per VM virtualization. 2.One application per container. Benefits Very low resource footprint More with less or with current infra Higher App density Almost bare metal performance Benefits Rapid deployment = rapid response to demand Higher availability due to lower Recovery Time Objective)RTO Edge Computing Provisioning Requirements Edge objectives are to improves user QoE or reduce total workload time [1]. Rapid deployment addresses the surge faster than VM based edge platform. Each edge device host multiple application. It not only require rapid deployment, but fast and clean teardown of service. Edge devices consists of low powered devices [2] low footprint & lower resource utilization is a must. Edge should deliver 99.999% Availability [3] fast start sequence contributes to a lower RTO and Higher Availability http://docs.jelastic.com/isolated-containers-migration Unnecessary OS virtualization is not required Ideally one single process per container Inside a container there will be no additional process e.g. Sshd,rsyslogd/dhclient, sshd, qmgr, kcopyd, bash, and many more.
18. Mi-Morphe Mi-Morphe performs cleansing and transformation on data ensuring accurate outcomes. Performs transformation and cleansing of unclean data (mostly due to human error) To clean data for storing or processed further in the next levels. Removing missing/undefined or duplicate data from an input file ETL (extract Transform and Load) Unclean data such as string, Boolean and more is first extracted,
19. http://www.meridianoutpost.com/resources/etools/calculators/calculator-file-download-time.php
20. http://www.meridianoutpost.com/resources/etools/calculators/calculator-file-download-time.php
21. Introduce the top is to monitor the image Bottom is monitor the container Talk along with demo (top screen) offload the mi-morphe container image to the edge (about 1 minute) (bottom screen) a new container will be launch (10sec) (new window) Login to Mi-Morphe (30 sec) ** The platform give different level of data processing, the mi-morphe can deploy anyway at the edge…. Multiple mi-morphe can be deploy then form a cluster… thus improve the processing time.
22. Mi-Morphe Mi-Morphe performs cleansing and transformation on data ensuring accurate outcomes. Performs transformation and cleansing of unclean data (mostly due to human error) To clean data for storing or processed further in the next levels. Removing missing/undefined or duplicate data from an input file ETL (extract Transform and Load) Unclean data such as string, Boolean and more is first extracted, transformed into the target format and the clean data loaded into a database Intro mi-morph Single pane comparison trad, vm, docker. Demo script 1. customer prep machine with focus 2. focus do remote installation 3. once ready user can access mi-morph to perform data cleansing.
23. Towards building Edge Computing Orchestration for our consumption. Can’t rely on existing software stacks. But some of the techniques and concepts can be adapt or reuse. CaaS – subset of IaaS Google Container Platform – Kurbenetes Rancher – orchestration of volumes/storage, networking, user access and containers. EC2 Container Service – Mesos CoreOS Nova-docker for Openstack