Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
SlideShare a Scribd company logo

Андрей Вагин. Все что вы хотели знать о Criu, но стеснялись спросить...

•
•
WG_ Events
WG_ Events

Доклад: Все что вы хотели знать о CRIU, но стеснялись спросить... Андрей Вагин пришёл в Parallels в 2006 году, занимается разработкой ядра Linux, контейнерной системы OpenVZ и сравнительно молодого проекта CRIU. В 2008 году закончил МФТИ.

Related slideshows

Debugging with-wireshark-niels-de-vos
Debugging with-wireshark-niels-de-vosDebugging with-wireshark-niels-de-vos
Debugging with-wireshark-niels-de-vos
 
Distributed Postgres
Distributed PostgresDistributed Postgres
Distributed Postgres
 
Sdc 2012-challenges
Sdc 2012-challengesSdc 2012-challenges
Sdc 2012-challenges
 
1 of 20
Download to read offline
Andrey Vagin <avagin@openvz.org>< CRIU - Checkpoint/Restore in User-space
2 Agenda ● CRIU and use-cases ● History ● Current state ● Under the hood ● Kernel impact ● How to integrate with/into CRIU ● P.haul ● Questions
3 History ● Berkeley Lab Checkpoint/Restart (BLCR) (2003) – Load a kernel module and link with a library ● DMTCP: Distributed MultiThreaded CheckPointing (2004-2006) – Preload a library ● OpenVZ (2005) – OpenVZ kernel ● Linux Checkpoint/Restart by Oren Laadan (2008) – A non-mainline kernel ● CRIU (2011) OpenVZ 2005 BLCR 2003 Linux C/R 2008 CRIU 2011 DMTCP 2007
4 What is C/R and how can it be used? C/R is the ability to save states of processes and to restore them later. Usage scenarios: – Failure recovery – Live migration – RKU (seamless kernel update) – Rollback to the previous state – Speed up of slow-boot services – HPC issues
5 Who is CRIU user?
6 How does this work? Kernel objects Process tree crtools Image files Name-spaces Files Sockets Pipes 001101 101010 110001 011010 000011 010101 001101 101010 110001 011010 000011 010101 001101 101010 110001 011010 000011 010101 001101 101010 110001 011010 000011 010101 001101 101010 110001 011010 000011 010101 001101 101010 110001 011010 000011 010101
8 Dump ● Parasite code – Receive file descriptors – Dump memory content – Prctl(), sigaction, pending signals, timers, etc. ● Ptrace – freeze processes – Inject a parasite code ● Netlink – Get information about sockets, netns ● Procfs /proc/PID/maps, /proc/PID/map_files/, /proc/PID/status, /proc/PID/mountinfo
10 Restore ● Collect shared objects ● Restore name-spaces ● Create a process tree – Restore SID, PGID – Restore objects, which should be inherited ● Files, sockets, pipes, ... ● Restore per-task properties. ● Restore memory ● Sim! Sala bim! ● Awesome Namespaces Processes
11 sigreturn()
12 New features in a kernel ● Parasite code injection (by Tejun Heo) – Read task states, that are currently retrieved by a task only about itself ● The kcmp() system call – Helps checking which kernel objects are shared between processes ● Proc map_files directory – Find out what exact file is mapped – Mappings sharing info ● A bunch of prctl extensions – Set various private stuff on task/mm objects (c/r-only feature) ● Last-pid sysctl – Restore task with desired PID value
13 New features in a kernel ● Sockets information dumping via netlink (sock_diag) – Extendable sockets state retrieving engine ● TCP repair mode – Read intimate state of a TCP connection and reconstructs it from scratch on a freshly created socket ● Virtual net devices indexes – Allows to restore network devices in a namespace ● Socket peeking offset – Allows peeking sockets queues (reading without removing data from queue) ● Task memory tracking – incremental snapshots, online migration
14 How to integrate with CRIU ● Action scripts – block/unblock network – setup namespaces – post-dump and post-restore ● RPC, shared library ● Plugins
15 RPC and libcriu.so ● Easy to use from other languages – The protocol is based on protobuf messages ● Allow to use CRIU for unprivileged processes – CRIU still requires root privileges to run – UNIX domain sockets support passing credentials ● Self-dump – A process can request to dump itself
16 Plugins ● Unknown file types ● External dependencies – Unix sockets (dbus, journald, rsyslog, etc) – Unknown character and block devices. – External bind-mounts – External net devices – External something else
17 Community
18 In a Nutshell, CRIU... .... has had 4,375 commits made by 36 contributors representing 58,688 lines of code ... is mostly written in C with a very low number of source code comments ... has a young, but established codebase maintained by a large development team with stable Y-O-Y commits ... estimated cost $ 787,432 https://www.ohloh.net/p/criu#
19 Where is CRIU now?
20 P.haul (process hauler) - Live migration using CRIU Live migration using CRIU ● Iterative ● Optimal ● Customizable #./p.haul ovz 100 10.30.25.213 Migration succeeded total time is ~2.86 sec frozen time is ~1.99 sec ( ['0.27', '0.18', '1.55'] ) restore time is ~0.86 sec img sync time is ~0.32 sec
21 P.haul Pre-dump and sync FS Freeze, dump, sync FS Restore Kill Resume Post-dump Resume Post-resume
Thank you http://criu.org

More Related Content

Андрей Вагин. Все что вы хотели знать о Criu, но стеснялись спросить...

  • 1. Andrey Vagin <avagin@openvz.org>< CRIU - Checkpoint/Restore in User-space
  • 2. 2 Agenda ● CRIU and use-cases ● History ● Current state ● Under the hood ● Kernel impact ● How to integrate with/into CRIU ● P.haul ● Questions
  • 3. 3 History ● Berkeley Lab Checkpoint/Restart (BLCR) (2003) – Load a kernel module and link with a library ● DMTCP: Distributed MultiThreaded CheckPointing (2004-2006) – Preload a library ● OpenVZ (2005) – OpenVZ kernel ● Linux Checkpoint/Restart by Oren Laadan (2008) – A non-mainline kernel ● CRIU (2011) OpenVZ 2005 BLCR 2003 Linux C/R 2008 CRIU 2011 DMTCP 2007
  • 4. 4 What is C/R and how can it be used? C/R is the ability to save states of processes and to restore them later. Usage scenarios: – Failure recovery – Live migration – RKU (seamless kernel update) – Rollback to the previous state – Speed up of slow-boot services – HPC issues
  • 6. 6 How does this work? Kernel objects Process tree crtools Image files Name-spaces Files Sockets Pipes 001101 101010 110001 011010 000011 010101 001101 101010 110001 011010 000011 010101 001101 101010 110001 011010 000011 010101 001101 101010 110001 011010 000011 010101 001101 101010 110001 011010 000011 010101 001101 101010 110001 011010 000011 010101
  • 7. 8 Dump ● Parasite code – Receive file descriptors – Dump memory content – Prctl(), sigaction, pending signals, timers, etc. ● Ptrace – freeze processes – Inject a parasite code ● Netlink – Get information about sockets, netns ● Procfs /proc/PID/maps, /proc/PID/map_files/, /proc/PID/status, /proc/PID/mountinfo
  • 8. 10 Restore ● Collect shared objects ● Restore name-spaces ● Create a process tree – Restore SID, PGID – Restore objects, which should be inherited ● Files, sockets, pipes, ... ● Restore per-task properties. ● Restore memory ● Sim! Sala bim! ● Awesome Namespaces Processes
  • 10. 12 New features in a kernel ● Parasite code injection (by Tejun Heo) – Read task states, that are currently retrieved by a task only about itself ● The kcmp() system call – Helps checking which kernel objects are shared between processes ● Proc map_files directory – Find out what exact file is mapped – Mappings sharing info ● A bunch of prctl extensions – Set various private stuff on task/mm objects (c/r-only feature) ● Last-pid sysctl – Restore task with desired PID value
  • 11. 13 New features in a kernel ● Sockets information dumping via netlink (sock_diag) – Extendable sockets state retrieving engine ● TCP repair mode – Read intimate state of a TCP connection and reconstructs it from scratch on a freshly created socket ● Virtual net devices indexes – Allows to restore network devices in a namespace ● Socket peeking offset – Allows peeking sockets queues (reading without removing data from queue) ● Task memory tracking – incremental snapshots, online migration
  • 12. 14 How to integrate with CRIU ● Action scripts – block/unblock network – setup namespaces – post-dump and post-restore ● RPC, shared library ● Plugins
  • 13. 15 RPC and libcriu.so ● Easy to use from other languages – The protocol is based on protobuf messages ● Allow to use CRIU for unprivileged processes – CRIU still requires root privileges to run – UNIX domain sockets support passing credentials ● Self-dump – A process can request to dump itself
  • 14. 16 Plugins ● Unknown file types ● External dependencies – Unix sockets (dbus, journald, rsyslog, etc) – Unknown character and block devices. – External bind-mounts – External net devices – External something else
  • 16. 18 In a Nutshell, CRIU... .... has had 4,375 commits made by 36 contributors representing 58,688 lines of code ... is mostly written in C with a very low number of source code comments ... has a young, but established codebase maintained by a large development team with stable Y-O-Y commits ... estimated cost $ 787,432 https://www.ohloh.net/p/criu#
  • 18. 20 P.haul (process hauler) - Live migration using CRIU Live migration using CRIU ● Iterative ● Optimal ● Customizable #./p.haul ovz 100 10.30.25.213 Migration succeeded total time is ~2.86 sec frozen time is ~1.99 sec ( ['0.27', '0.18', '1.55'] ) restore time is ~0.86 sec img sync time is ~0.32 sec
  • 19. 21 P.haul Pre-dump and sync FS Freeze, dump, sync FS Restore Kill Resume Post-dump Resume Post-resume