Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
SlideShare a Scribd company logo

Cassandra on Docker @ Walmart Labs

•Download as PPTX, PDF•
•

This document discusses using Docker containers to run Cassandra clusters at Walmart. It proposes transforming existing Cassandra hardware into containers to better utilize unused compute. It also suggests building new Cassandra clusters in containers and migrating old clusters to double capacity on existing hardware and save costs. Benchmark results show Docker containers outperforming virtual machines on OpenStack and Azure in terms of reads, writes, throughput and latency for an in-house application.

1 of 8
Dense Node C* / S* / Spark on Dockers Walmart GEC-Database Engineering & Data Services Ani Mondal
NoSQL / Big Data Footprint 1500 Nodes (150 Clusters) Prod & Non- Prod Elastic Kafka
Proposed Container Cluster 1 2 3 4 5 6 7 8BM Cluster 1 2 3 4 Transform new C* hardware into containers thereby utilizing unused compute Build new clusters in containers & migrate old C* clusters into those Double the capacity on existing & newly ordered H/W for C* clusters thereby saving cost Reclaiming and reusing existing HW
App A Containers vs. VMs Hypervisor Host OS Server Guest OS Bins/ Libs App A’ Gues t OS Bins/ Libs App B Gues t OS Bins/ Libs AppA’ Docker Host OS Server Bins/Libs AppA Bins/Libs AppB AppB’ AppC AppC’ VM Container Containers are isolated, but share OS and, optionally, binaries/libraries Guest OS Guest OS Bins/ Libs App A Original App (No OS to take up space, resources, or require restart) Copy of App No OS. Can Share bins/libs App A’
Configuration Host OS Server (HP SL4540) Bins/Libs Bins/Libs Dockers Docker 1 Persistent DATA & LOGS Docker 2 Persistent DATA & LOGS OPSC Agent OPSC Agent Custom Orchestration Host Network – Dedicated NICs Docker Registry
Benchmark #s in-house application Comparison (+/-3 % Margin of Error) Bare Metal (Prod) Dockers (Prod HW) OpenStack (VM) AZURE VM Reads Total 276943 813428 999164 975179 Per Sec 4615.72 6778.57 651.77 5079.06 Avg. Latency (in ms) 0.68 0.92 9.77 1.40 Writes Total 499921 1000000 1000000 1000000 Per Sec 8332.02 8333.33 652.32 5208.3 Avg. Latency (in ms) 0.72 0.71 9.21 1.04
Patent • Kavindra Yerolkar • Jayakrishnan Parappalliyalil • Preetish Tripathi • Aniruddha Mondal
Cassandra on Docker @ Walmart Labs

More Related Content

Cassandra on Docker @ Walmart Labs

  • 1. Dense Node C* / S* / Spark on Dockers Walmart GEC-Database Engineering & Data Services Ani Mondal
  • 2. NoSQL / Big Data Footprint 1500 Nodes (150 Clusters) Prod & Non- Prod Elastic Kafka
  • 3. Proposed Container Cluster 1 2 3 4 5 6 7 8BM Cluster 1 2 3 4 Transform new C* hardware into containers thereby utilizing unused compute Build new clusters in containers & migrate old C* clusters into those Double the capacity on existing & newly ordered H/W for C* clusters thereby saving cost Reclaiming and reusing existing HW
  • 4. App A Containers vs. VMs Hypervisor Host OS Server Guest OS Bins/ Libs App A’ Gues t OS Bins/ Libs App B Gues t OS Bins/ Libs AppA’ Docker Host OS Server Bins/Libs AppA Bins/Libs AppB AppB’ AppC AppC’ VM Container Containers are isolated, but share OS and, optionally, binaries/libraries Guest OS Guest OS Bins/ Libs App A Original App (No OS to take up space, resources, or require restart) Copy of App No OS. Can Share bins/libs App A’
  • 5. Configuration Host OS Server (HP SL4540) Bins/Libs Bins/Libs Dockers Docker 1 Persistent DATA & LOGS Docker 2 Persistent DATA & LOGS OPSC Agent OPSC Agent Custom Orchestration Host Network – Dedicated NICs Docker Registry
  • 6. Benchmark #s in-house application Comparison (+/-3 % Margin of Error) Bare Metal (Prod) Dockers (Prod HW) OpenStack (VM) AZURE VM Reads Total 276943 813428 999164 975179 Per Sec 4615.72 6778.57 651.77 5079.06 Avg. Latency (in ms) 0.68 0.92 9.77 1.40 Writes Total 499921 1000000 1000000 1000000 Per Sec 8332.02 8333.33 652.32 5208.3 Avg. Latency (in ms) 0.72 0.71 9.21 1.04
  • 7. Patent • Kavindra Yerolkar • Jayakrishnan Parappalliyalil • Preetish Tripathi • Aniruddha Mondal