Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
SlideShare a Scribd company logo

Using cgroups in docker container

•Download as DOCX, PDF•
•

This document discusses experimenting with cgroups in Docker containers to isolate processes. It describes installing Docker, launching an Ubuntu container with capabilities enabled, and installing cgroup tools. It then mounts the cpuset and cpu cgroup hierarchies and creates low and high cgroups. Different CPU shares are assigned to each cgroup and processes are run in each to demonstrate the CPU isolation between cgroups.

1 of 5
Using cgroups in docker container Introduction The reason for this exercise is that I want to try out processes isolation functionality in Linux using ‘cgroups’ functionality. During this learning phase, the intention (as it is always) is to try out cgroups before actually putting them into production ready mode.This involves installing/uninstalling different cgroup packages, creating/deleting sample cgroups and testing them. I had a Ubuntu 14.04 VM installed on my Windows 10 guest OS. So, there are two options: 1. Use Ubuntu VM to experimentabout cgroups – Easiest choice but I did not chose it due to following reasons: a. It might have left some packages or cgroups installed after the experimentis over which may not be used later on. b. Effort had be put in to revert my VM to its pre-experimentstate that involved removing cgroups ,configurations and uninstalling packages etc. 2. Install Dockeron Ubuntu VM and use a Ubuntu Container for experimentwith cgroups – This choice seemedreasonable because it did not have constraints listed above. So, the rest of this article explains the steps required to install and play around with cgroups using docker containers. Pre-requisites: Docker Setupdetails: Guest OS: Windows 10 Home VM Player: VM Workstation 12 Player OS: Ubuntu 14.04 DockerVersion: 1.6.2,build 7c8fca2
2 DockerImage: Mycustom image that was created from ‘ubuntu’image present in docker hub. It contained several other standard software development packages on top, but standard image should be fine as well. Steps:  Install docker  Launch a docker container with Ubuntu image Note: Please do not forget to use options ‘--cap-add=SYS_ADMIN’ and‘--security-opt= apparmor:unconfined’ while launching the container. Otherwise, you will not be able to mount cgroups using this version of docker. Inside the launched container, following steps are performed:  Install cgroups This should create following files: Note: Install ‘htop’ utility as well. It presents a nice colorful cli based visualization over ‘top’ command. This will help to demonstrate the cgroups use as well. Please note that the directory /sys/fs/cgroup will be empty as cgroup sub-components do not automatically get mounted in this case. To mount them, following commands are required. These commands mount cpuset and cpu sub-components only. Likewise, other cgroup sub-components can also be mounted: sudo apt-get install cgroup-bin cgroup-lite libcgroup1 sudo docker run -d --cap-add=SYS_ADMIN--security-opt=apparmor:unconfined-it vne_dev_image1bash /sys/fs/cgroup /etc/init/cgroup-lite.conf /proc/cgroups sudo apt-get install htop
3 After this, following files should be seen: Now, cgroup sub-components ‘cpu’ and ‘cpu-set’ are ready. ‘cpu’ is usedto define the relative priorities to different cgroups. ‘cpu-set’is used to assign cpus to a particular cgroups. Since,my VM was using only 1 processor with 1 core, ‘cpu’ subcomponent does not offer any benefit to me.So, I used ‘cpu-set’component for rest of this article: mount -t tmpfs cgroup_root /sys/fs/cgroup mkdir /sys/fs/cgroup/cpuset mount -t cgroup cpuset -o cpuset/sys/fs/cgroup/cpuset/ mkdir /sys/fs/cgroup/cpu mount -t cgroup cpu -o cpu/sys/fs/cgroup/cpu/ ls /sys/fs/cgroup/cpuset/ cgroup.clone_children cpuset.effective_mems cpuset.memory_spread_page notify_on_release cgroup.procs cpuset.mem_exclusive cpuset.memory_spread_slab release_agent cgroup.sane_behavior cpuset.mem_hardwall cpuset.mems tasks cpuset.cpu_exclusive cpuset.memory_migrate cpuset.sched_load_balance cpuset.cpus cpuset.memory_pressure cpuset.sched_relax_domain_level cpuset.effective_cpus cpuset.memory_pressure_enabled docker/ cat /sys/fs/cgroup/cpu/ cgroup.clone_children cgroup.sane_behavior cpu.cfs_quota_us cpu.stat low/ release_agent cgroup.procs cpu.cfs_period_us cpu.shares docker/ notify_on_release tasks cpuset.cpus cpuset.memory_pressure cpuset.sched_relax_domain_level cpuset.effective_cpus cpuset.memory_pressure_enabled docker/
4 Following picture summarizes the experiment: I am going to create two cgroups ‘low’ and ‘high’;Assign different CPU weights to them; then start different processes (they will start using all processing power) in these cgroups to demonstrate the effectof process isolation. cgcreate -a root -t vnej cpu:low cgcreate -a root -t vnej cpu:high cat /sys/fs/cgroup/cpu/low/ cgroup.clone_children cpu.cfs_period_us cpu.shares notify_on_release cgroup.procs cpu.cfs_quota_us cpu.stat tasks cat /sys/fs/cgroup/cpu/high/ cgroup.clone_children cpu.cfs_period_us cpu.shares notify_on_release cgroup.procs cpu.cfs_quota_us cpu.stat tasks
5 Assign three-fourth(3/4th ) of CPU to cgroup ‘low’ Assign rest one-fourth (1/4th ) of CPU to cgroup ‘high’ start a process with cgroup ‘low’: Observe CPU usage using htop command Start another process with cgroup ‘high’ Observe CPU distribution using htop command Start another process with cgroup ‘low’: start a process with cgroup ‘high’: References: https://www.devinhoward.ca/technology/2015/feb/implementing-cgroups-ubuntu-or-debian http://stackoverflow.com/questions/27454848/cgroups-sys-fs-cgroup-is-empty-on-ubuntu http://libcg.sourceforge.net/ https://wiki.archlinux.org/index.php/cgroups cat /sys/fs/cgroup/cpu/low/cpu.shares 1024 echo768 > /sys/fs/cgroup/cpu/low/cpu.shares echo256 > /sys/fs/cgroup/cpu/high/cpu.shares cgexec -gcpu:low md5sum /dev/urandom & cgexec -gcpu:highmd5sum /dev/urandom & cgexec -gcpu:low md5sum /dev/urandom & cgexec -gcpu:highmd5sum /dev/urandom &

More Related Content

Using cgroups in docker container

  • 1. Using cgroups in docker container Introduction The reason for this exercise is that I want to try out processes isolation functionality in Linux using ‘cgroups’ functionality. During this learning phase, the intention (as it is always) is to try out cgroups before actually putting them into production ready mode.This involves installing/uninstalling different cgroup packages, creating/deleting sample cgroups and testing them. I had a Ubuntu 14.04 VM installed on my Windows 10 guest OS. So, there are two options: 1. Use Ubuntu VM to experimentabout cgroups – Easiest choice but I did not chose it due to following reasons: a. It might have left some packages or cgroups installed after the experimentis over which may not be used later on. b. Effort had be put in to revert my VM to its pre-experimentstate that involved removing cgroups ,configurations and uninstalling packages etc. 2. Install Dockeron Ubuntu VM and use a Ubuntu Container for experimentwith cgroups – This choice seemedreasonable because it did not have constraints listed above. So, the rest of this article explains the steps required to install and play around with cgroups using docker containers. Pre-requisites: Docker Setupdetails: Guest OS: Windows 10 Home VM Player: VM Workstation 12 Player OS: Ubuntu 14.04 DockerVersion: 1.6.2,build 7c8fca2
  • 2. 2 DockerImage: Mycustom image that was created from ‘ubuntu’image present in docker hub. It contained several other standard software development packages on top, but standard image should be fine as well. Steps:  Install docker  Launch a docker container with Ubuntu image Note: Please do not forget to use options ‘--cap-add=SYS_ADMIN’ and‘--security-opt= apparmor:unconfined’ while launching the container. Otherwise, you will not be able to mount cgroups using this version of docker. Inside the launched container, following steps are performed:  Install cgroups This should create following files: Note: Install ‘htop’ utility as well. It presents a nice colorful cli based visualization over ‘top’ command. This will help to demonstrate the cgroups use as well. Please note that the directory /sys/fs/cgroup will be empty as cgroup sub-components do not automatically get mounted in this case. To mount them, following commands are required. These commands mount cpuset and cpu sub-components only. Likewise, other cgroup sub-components can also be mounted: sudo apt-get install cgroup-bin cgroup-lite libcgroup1 sudo docker run -d --cap-add=SYS_ADMIN--security-opt=apparmor:unconfined-it vne_dev_image1bash /sys/fs/cgroup /etc/init/cgroup-lite.conf /proc/cgroups sudo apt-get install htop
  • 3. 3 After this, following files should be seen: Now, cgroup sub-components ‘cpu’ and ‘cpu-set’ are ready. ‘cpu’ is usedto define the relative priorities to different cgroups. ‘cpu-set’is used to assign cpus to a particular cgroups. Since,my VM was using only 1 processor with 1 core, ‘cpu’ subcomponent does not offer any benefit to me.So, I used ‘cpu-set’component for rest of this article: mount -t tmpfs cgroup_root /sys/fs/cgroup mkdir /sys/fs/cgroup/cpuset mount -t cgroup cpuset -o cpuset/sys/fs/cgroup/cpuset/ mkdir /sys/fs/cgroup/cpu mount -t cgroup cpu -o cpu/sys/fs/cgroup/cpu/ ls /sys/fs/cgroup/cpuset/ cgroup.clone_children cpuset.effective_mems cpuset.memory_spread_page notify_on_release cgroup.procs cpuset.mem_exclusive cpuset.memory_spread_slab release_agent cgroup.sane_behavior cpuset.mem_hardwall cpuset.mems tasks cpuset.cpu_exclusive cpuset.memory_migrate cpuset.sched_load_balance cpuset.cpus cpuset.memory_pressure cpuset.sched_relax_domain_level cpuset.effective_cpus cpuset.memory_pressure_enabled docker/ cat /sys/fs/cgroup/cpu/ cgroup.clone_children cgroup.sane_behavior cpu.cfs_quota_us cpu.stat low/ release_agent cgroup.procs cpu.cfs_period_us cpu.shares docker/ notify_on_release tasks cpuset.cpus cpuset.memory_pressure cpuset.sched_relax_domain_level cpuset.effective_cpus cpuset.memory_pressure_enabled docker/
  • 4. 4 Following picture summarizes the experiment: I am going to create two cgroups ‘low’ and ‘high’;Assign different CPU weights to them; then start different processes (they will start using all processing power) in these cgroups to demonstrate the effectof process isolation. cgcreate -a root -t vnej cpu:low cgcreate -a root -t vnej cpu:high cat /sys/fs/cgroup/cpu/low/ cgroup.clone_children cpu.cfs_period_us cpu.shares notify_on_release cgroup.procs cpu.cfs_quota_us cpu.stat tasks cat /sys/fs/cgroup/cpu/high/ cgroup.clone_children cpu.cfs_period_us cpu.shares notify_on_release cgroup.procs cpu.cfs_quota_us cpu.stat tasks
  • 5. 5 Assign three-fourth(3/4th ) of CPU to cgroup ‘low’ Assign rest one-fourth (1/4th ) of CPU to cgroup ‘high’ start a process with cgroup ‘low’: Observe CPU usage using htop command Start another process with cgroup ‘high’ Observe CPU distribution using htop command Start another process with cgroup ‘low’: start a process with cgroup ‘high’: References: https://www.devinhoward.ca/technology/2015/feb/implementing-cgroups-ubuntu-or-debian http://stackoverflow.com/questions/27454848/cgroups-sys-fs-cgroup-is-empty-on-ubuntu http://libcg.sourceforge.net/ https://wiki.archlinux.org/index.php/cgroups cat /sys/fs/cgroup/cpu/low/cpu.shares 1024 echo768 > /sys/fs/cgroup/cpu/low/cpu.shares echo256 > /sys/fs/cgroup/cpu/high/cpu.shares cgexec -gcpu:low md5sum /dev/urandom & cgexec -gcpu:highmd5sum /dev/urandom & cgexec -gcpu:low md5sum /dev/urandom & cgexec -gcpu:highmd5sum /dev/urandom &