Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
SlideShare a Scribd company logo
Container Network Interface:
Network plugins for Kubernetes
and beyond
Eugene Yakubovich
@eyakubovich
Kubernetes networking model
- IP per pod
- Pods in the cluster can be addressed by their IP
How to network containers together?
- Cloud provider integration
- AWS
- GCE
How to network containers together?
linux-bridge
macvlan
ipvlan
Open vSwitch
Weave
Project Calico
flannel
How to allocate IP addresses?
- From a fixed block on a host
- DHCP
- IPAM system backed by SQL database
- SDN assigned: e.g. Weave
How do you mix and match?
(macvlan | ipvlan) + (DHCP | host-local)
Order matters!
- macvlan + DHCP
○ Create macvlan device
○ Use the device to DHCP
○ Configure device with allocated IP
- Routed + IPAM
○ Ask IPAM for an IP
○ Create veth and routes on host and/or fabric
○ Configure device with allocated IP
Container Runtime (e.g. k8s)
veth macvlan ipvlan OVS
Container Networking Interface (CNI)
CNI
- Container can join multiple networks
- Network described by JSON config
- Plugin supports two commands
- Add container to the network
- Remove container from the network
User configures a network
$ cat /etc/cni/net.d/10-mynet.conf
{
"name": "mynet",
"type": "bridge",
"ipam": {
"type": "host-local",
"subnet": "10.10.0.0/16"
}
}
CNI: Step 1
Container runtime creates network namespace
and gives it a named handle
$ cd /var/lib/cni
$ touch myns
$ unshare -n mount --bind /proc/self/ns/net myns
CNI: Step 2
Container runtime invokes the CNI plugin
$ export CNI_COMMAND=ADD
$ export CNI_NETNS=/var/lib/cni/myns
$ export CNI_CONTAINERID=5248e9f8-3c91-11e5-...
$ export CNI_IFNAME=eth0
$ $CNI_PATH/bridge </etc/cni/net.d/10-mynet.conf
CNI: Step 3
Inside the bridge plugin (1):
$ brctl addbr mynet
$ ip link add veth123 type veth peer name $CNI_IFNAME
$ brctl addif mynet veth123
$ ip link set $CNI_IFNAME netns $CNI_IFNAME
$ ip link set veth123 up
CNI: Step 3
Inside the bridge plugin (2):
$ IPAM_PLUGIN=host-local # from network conf
$ echo $IPAM_PLUGIN
{
"ip4": {
"ip": "10.10.5.9/16",
"gateway": "10.10.0.1"
}
}
CNI: Step 3
Inside the bridge plugin (3):
# switch to container namespace
$ ip addr add 10.0.5.9/16 dev $CNI_IFNAME
# Finally, print IPAM result JSON to stdout
Kubernetes + CNI + Docker
- Kubernetes has its own network plugins
- CNI "driver" is a k8s network plugin
- Future: make CNI native plugin system
Kubernetes + CNI + Docker
- k8s starts "pause" container to create netns
- k8s invokes its plugin (CNI driver)
- k8s CNI driver executes a CNI plugin
- CNI plugin joins "pause" container to network
- Pod containers use "pause" container netns
Kubernetes + rkt
- rkt natively supports CNI
- Kubernetes delegates to rkt to invoke CNI
plugins
Get involved!
https://github.com/appc/cni
Want to work on upstream Kubernetes or
distributed systems infrastructure?
CoreOS San Francisco is hiring.
Work at CoreOS
coreos.com/careers

More Related Content

Container Network Interface: Network Plugins for Kubernetes and beyond

  • 1. Container Network Interface: Network plugins for Kubernetes and beyond Eugene Yakubovich @eyakubovich
  • 2. Kubernetes networking model - IP per pod - Pods in the cluster can be addressed by their IP
  • 3. How to network containers together? - Cloud provider integration - AWS - GCE
  • 4. How to network containers together? linux-bridge macvlan ipvlan Open vSwitch Weave Project Calico flannel
  • 5. How to allocate IP addresses? - From a fixed block on a host - DHCP - IPAM system backed by SQL database - SDN assigned: e.g. Weave
  • 6. How do you mix and match? (macvlan | ipvlan) + (DHCP | host-local)
  • 7. Order matters! - macvlan + DHCP ○ Create macvlan device ○ Use the device to DHCP ○ Configure device with allocated IP - Routed + IPAM ○ Ask IPAM for an IP ○ Create veth and routes on host and/or fabric ○ Configure device with allocated IP
  • 8. Container Runtime (e.g. k8s) veth macvlan ipvlan OVS Container Networking Interface (CNI)
  • 9. CNI - Container can join multiple networks - Network described by JSON config - Plugin supports two commands - Add container to the network - Remove container from the network
  • 10. User configures a network $ cat /etc/cni/net.d/10-mynet.conf { "name": "mynet", "type": "bridge", "ipam": { "type": "host-local", "subnet": "10.10.0.0/16" } }
  • 11. CNI: Step 1 Container runtime creates network namespace and gives it a named handle $ cd /var/lib/cni $ touch myns $ unshare -n mount --bind /proc/self/ns/net myns
  • 12. CNI: Step 2 Container runtime invokes the CNI plugin $ export CNI_COMMAND=ADD $ export CNI_NETNS=/var/lib/cni/myns $ export CNI_CONTAINERID=5248e9f8-3c91-11e5-... $ export CNI_IFNAME=eth0 $ $CNI_PATH/bridge </etc/cni/net.d/10-mynet.conf
  • 13. CNI: Step 3 Inside the bridge plugin (1): $ brctl addbr mynet $ ip link add veth123 type veth peer name $CNI_IFNAME $ brctl addif mynet veth123 $ ip link set $CNI_IFNAME netns $CNI_IFNAME $ ip link set veth123 up
  • 14. CNI: Step 3 Inside the bridge plugin (2): $ IPAM_PLUGIN=host-local # from network conf $ echo $IPAM_PLUGIN { "ip4": { "ip": "10.10.5.9/16", "gateway": "10.10.0.1" } }
  • 15. CNI: Step 3 Inside the bridge plugin (3): # switch to container namespace $ ip addr add 10.0.5.9/16 dev $CNI_IFNAME # Finally, print IPAM result JSON to stdout
  • 16. Kubernetes + CNI + Docker - Kubernetes has its own network plugins - CNI "driver" is a k8s network plugin - Future: make CNI native plugin system
  • 17. Kubernetes + CNI + Docker - k8s starts "pause" container to create netns - k8s invokes its plugin (CNI driver) - k8s CNI driver executes a CNI plugin - CNI plugin joins "pause" container to network - Pod containers use "pause" container netns
  • 18. Kubernetes + rkt - rkt natively supports CNI - Kubernetes delegates to rkt to invoke CNI plugins
  • 20. Want to work on upstream Kubernetes or distributed systems infrastructure? CoreOS San Francisco is hiring. Work at CoreOS coreos.com/careers