Running High Performance & Fault-tolerant Elasticsearch Clusters on Docker
•Download as PPTX, PDF•
8 likes•10,562 views
This document discusses running Elasticsearch clusters on Docker containers. It describes how Docker containers are more lightweight than virtual machines and have less overhead. It provides examples of running official Elasticsearch Docker images and customizing configurations. It also covers best practices for networking, storage, constraints, and high availability when running Elasticsearch on Docker.
Running High Performance & Fault-tolerant Elasticsearch Clusters on Docker
- 1. Elasticsearch & Docker Rafał Kuć – Sematext Group, Inc. @kucrafal @sematext sematext.com Running High Performance Fault Tolerant Elasticsearch Clusters On Docker
- 3. You Are Probably Familiar With That Development
- 4. You Are Probably Familiar With That Development Test
- 5. You Are Probably Familiar With That Development Test QA
- 6. You Are Probably Familiar With That Development Test QA Production enviroment
- 7. And Problems That Come With It Resources not utilized
- 8. And Problems That Come With It Resources not utilized Overprovisoned Servers
- 9. And Problems That Come With It Resources not utilized Overprovisoned Servers ≠ ≠
- 10. The solution Development Test QA Production
- 11. Why Docker? Light weight Based on Open Standards Secure
- 12. Containers vs Virtual Machines Hardware Traditional Virtual Machine
- 13. Containers vs Virtual Machines Hardware Host Operating System Traditional Virtual Machine
- 14. Containers vs Virtual Machines Hardware Host Operating System Hypervisor Traditional Virtual Machine
- 15. Containers vs Virtual Machines Hardware Host Operating System Hypervisor Guest OS Guest OS Traditional Virtual Machine
- 16. Containers vs Virtual Machines Hardware Host Operating System Hypervisor Guest OS Guest OS Libraries Libraries Traditional Virtual Machine
- 17. Containers vs Virtual Machines Hardware Host Operating System Hypervisor Guest OS Guest OS Libraries Libraries Application 1 Application 2 Traditional Virtual Machine
- 18. Containers vs Virtual Machines Hardware Host Operating System Hypervisor Guest OS Guest OS Libraries Libraries Application 1 Application 2 Hardware Host Operating System Traditional Virtual MachineContainer
- 19. Containers vs Virtual Machines Hardware Host Operating System Hypervisor Guest OS Guest OS Libraries Libraries Application 1 Application 2 Hardware Host Operating System Docker Engine Traditional Virtual MachineContainer
- 20. Containers vs Virtual Machines Hardware Host Operating System Hypervisor Guest OS Guest OS Libraries Libraries Application 1 Application 2 Hardware Host Operating System Docker Engine Libraries Libraries Application 1 Application 2 Traditional Virtual MachineContainer
- 21. Why Elasticsearch? Distributed by design http://www.dailypets.co.uk/2007/06/17/kittens-rest-at-half-time/ Indices Aggs Admin Monitor Search Index
- 22. Running Offical Elasticsearch Container $ docker run -d elasticsearch
- 23. Running Offical Elasticsearch Container $ docker run -d elasticsearch:latest
- 24. Running Offical Elasticsearch Container $ docker run -d elasticsearch:latest $ docker run -d --name es1 elasticsearch
- 25. Running Offical Elasticsearch Container $ docker run -d elasticsearch:latest $ docker run -d --name es1 elasticsearch $ docker run -d --name es1 -e ES_HEAP_SIZE=1g elasticsearch
- 26. Running Offical Elasticsearch Container $ docker run -d elasticsearch:latest $ docker run -d --name es1 elasticsearch $ docker run -d --name es1 -e ES_HEAP_SIZE=1g elasticsearch $ docker run -d --name es1 elasticsearch -Dnode.name bbuzz
- 27. Container Constraints $ docker run -d -m 2G elasticsearch $ docker run -d -m 2G --memory-swappiness=0 elasticsearch http://docs.docker.com/engine/reference/run/
- 28. Container Constraints $ docker run -d -m 2G elasticsearch $ docker run -d -m 2G --memory-swappiness=0 elasticsearch $ docker run -d --cpuset-cpus="1,3" elasticsearch http://docs.docker.com/engine/reference/run/
- 29. Container Constraints $ docker run -d -m 2G elasticsearch $ docker run -d -m 2G --memory-swappiness=0 elasticsearch $ docker run -d --cpuset-cpus="1,3" elasticsearch http://docs.docker.com/engine/reference/run/ $ docker run -d --cpu-period=50000 --cpu-quota=25000 elasticsearch
- 30. Constraints - Good Practices Limit container memory
- 31. Constraints - Good Practices Limit container memory Account for I/O cache when giving memory
- 32. Constraints - Good Practices Limit container memory Account for I/O cache when giving memory Limit amount of CPU cores
- 33. Constraints - Good Practices Limit container memory Account for I/O cache when giving memory Limit amount of CPU cores Remember about JVM GC
- 34. Creating Optimized Image Dockerfile: FROM elasticsearch ADD ./elasticsearch.yml /usr/share/elasticsearch/config/
- 35. Creating Optimized Image Dockerfile: FROM elasticsearch ADD ./elasticsearch.yml /usr/share/elasticsearch/config/ $ docker build -t bbuzz/example .
- 36. Creating Optimized Image Dockerfile: FROM elasticsearch ADD ./elasticsearch.yml /usr/share/elasticsearch/config/ $ docker build -t bbuzz/example . Sending build context to Docker daemon 5.12 kB Step 1 : FROM elasticsearch ---> 1e23f30a3667 Step 2 : ADD ./elasticsearch.yml /usr/share/elasticsearch/config/ ---> 015f12adfd2a Removing intermediate container de560c6ae0d1 Successfully built 015f12adfd2a
- 37. Dealing With Network $ docker run -d -p 9200:9200 -p 9300:9300 elasticsearch
- 38. Dealing With Network $ docker run -d -p 9200:9200 -p 9300:9300 elasticsearch $ docker run -d --link es1 elasticsearch -Ddiscovery.zen.ping.unicast.hosts=es1
- 39. Dealing With Network $ docker run -d -p 9200:9200 -p 9300:9300 elasticsearch Add network.publish_host when building own container $ docker run -d --link es1 elasticsearch -Ddiscovery.zen.ping.unicast.hosts=es1
- 40. Dealing With Network $ docker run -d -p 9200:9200 -p 9300:9300 elasticsearch Add network.publish_host when building own container Add discovery.zen.ping.unicast.hosts when building own container $ docker run -d --link es1 elasticsearch -Ddiscovery.zen.ping.unicast.hosts=es1
- 41. Network - Good Practices Separate network for Elasticsearch cluster
- 42. Network - Good Practices Separate network for Elasticsearch cluster Common host names for containers $ docker run -d -h esnode1 elasticsearch
- 43. Network - Good Practices Separate network for Elasticsearch cluster Common host names for containers $ docker run -d -h esnode1 elasticsearch Expose only needed ports
- 44. Network - Good Practices Separate network for Elasticsearch cluster Common host names for containers $ docker run -d -h esnode1 elasticsearch Expose only needed ports Elasticsearch data & client nodes point to masters only
- 45. Dealing With Storage By default in /usr/share/elasticsearch/data
- 46. Dealing With Storage By default in /usr/share/elasticsearch/data By default not persisted
- 47. Dealing With Storage By default in /usr/share/elasticsearch/data By default not persisted $ docker run -d -v /opt/elasticsearch/data:/usr/share/elasticsearch/data elasticsearch
- 48. Dealing With Storage $ docker run -d -v /opt/elasticsearch/data:/usr/share/elasticsearch/data elasticsearch By default in /usr/share/elasticsearch/data By default not persisted Use data only Docker volumes Permissions
- 49. Data Only Docker Volumes Bypasses Union File System
- 50. Data Only Docker Volumes Bypasses Union File System Can be shared between containers
- 51. Data Only Docker Volumes Bypasses Union File System Can be shared between containers Data volumes persist if the container itself is deleted
- 52. Data Only Docker Volumes Bypasses Union File System Can be shared between containers Data volumes persist if the container itself is deleted $ docker create -v /mnt/es/data:/usr/share/elasticsearch/data --name esdata elasticsearch Permissions
- 53. Data Only Docker Volumes Bypasses Union File System Can be shared between containers Data volumes persist if the container itself is deleted $ docker create -v /mnt/es/data:/usr/share/elasticsearch/data --name esdata elasticsearch $ docker run --volumes-from esdata elasticsearch
- 54. Highly Available Cluster Master only Master only Master only Data only Data only Data only Data only Data only Data only Client only Client only
- 55. Highly Available Cluster Master only Master only Master only Data only Data only Data only Data only Data only Data only Client only Client only minimum_master_nodes = N/2 + 1
- 56. Master Nodes & Docker $ docker run -d elasticsearch -Dnode.master=true -Dnode.data=false -Dnode.client=false
- 57. Client Nodes & Docker $ docker run -d elasticsearch -Dnode.master=false -Dnode.data=false -Dnode.client=true
- 58. Data Nodes & Docker $ docker run -d elasticsearch -Dnode.master=false -Dnode.data=true -Dnode.client=false
- 59. Multiple Tiers node.tag=hot node.tag=cold node.tag=cold
- 60. Multiple Tiers $ docker run -d elasticsearch -Dnode.tag=hot
- 61. Multiple Tiers curl -XPUT 'localhost:9200/data_2016-06-05' -d '{ "settings": { "index.routing.allocation.include.tag" : "hot" } }'
- 62. Multiple Tiers node.tag=hot node.tag=cold node.tag=cold data_2016-06-05 data_2016-06-05
- 63. Multiple Tiers node.tag=hot node.tag=cold node.tag=cold data_2016-06-05 data_2016-06-05
- 64. Top Metrics – Health & Shards https://sematext.com/spm/
- 65. Top Metrics - CPU https://sematext.com/spm/
- 66. Top Metrics – Memory Usage https://sematext.com/spm/
- 67. Top Metrics – I/O Usage https://sematext.com/spm/
- 68. Top Metrics – JVM Heap https://sematext.com/spm/
- 69. Top Metrics – Garbage Collector https://sematext.com/spm/
- 70. Top Metrics – Request Rate & Latency https://sematext.com/spm/
- 71. Top Metrics - Caches https://sematext.com/spm/
- 72. Top Metrics – Indexing https://sematext.com/spm/
- 73. Top Metrics – Refresh Time https://sematext.com/spm/
- 74. Top 10 Metrics – Merge Time https://sematext.com/spm/
- 76. We Are Hiring ! Dig Search ? Dig Analytics ? Dig Big Data ? Dig Performance ? Dig Logging ? Dig working with and in open – source ? We’re hiring world – wide ! http://sematext.com/about/jobs.html
Editor's Notes
- Problems with standard deployment like: Resources not utilized Need to provision machines before deployment Differences between development, test, QA and production environments Hard to scale automatically
- Problems with standard deployment like: Resources not utilized Need to provision machines before deployment Differences between development, test, QA and production environments Hard to scale automatically
- Problems with standard deployment like: Resources not utilized Need to provision machines before deployment Differences between development, test, QA and production environments Hard to scale automatically