EdPike365

Containers and K8s

August 23, 2022

OCI/Docker | Kubernetes Orchestration | Hosting

Servers should be cattle, not pets. But it’s OK to miss “the servers of Middle Earth”.

VM Hypervisors

  • VMWare
  • KVM
  • Hyper-V

Container History

Containers appeared after invention of cgroups and namespaces. LXC was an early try.

LXC Linux Containers

  • Between OCI and fullblown VM
  • Docker was originally built on top of LXC.
  • LXD: extension of LXC

“LXD is designed for hosting virtual environments that “will typically be long running and based on a clean distribution image,” whereas “Docker focuses on ephemeral, stateless, minimal containers that won’t typically get upgraded or re-configured but instead just be replaced entirely.”

CRI, OCI

  • CRI (Container Runtime Interface) Compliant Runtimes

    • Kubernetes API that allows you to use different container runtimes (like containerd and CRI-O).
    • Defines gRPC protocol for communicating between cluster components.

  • OCI (Open Container Initiative) Specs for building images and running containers. Has 3 parts:

    • Runtime Spec
    • Image Spec
    • Distribution Spec

  • Runtime Specs

    • High Level runtimes
      • containerD: daemon that interfaces between container engine and container runtimes.
        • CRI compliant through cri plugin (this still true?)
        • Originally from Docker
      • Docker
      • CRI-O
        • Red Hat/IBM
      • podman
    • Low Level
      • runc: OCI universal OS container runtime.
        • by Docker, now part of CNCF
        • Docker and Containerd run on it
      • gvisor
      • kata

Docker

  • Docker CLI
  • Docker Desktop: Docker needs Linux, so MacOS (Unix, not Linux), needs a VM, just like Windows (except WSL2).
  • DinD
  • Docker Hub
  • VSCode DevContainers

Podman

No daemon, each container is self-sufficient. So no SPOF (single point of failure). NOT root by default. More secure than Docker.

Others

  • LXC: Canonical, pre Docker. No daemon.
  • rkt: formerly CoreOS Rocket. Can run Docker. No daemon. RedHat.

Container Image Repos

  • Docker Hub
  • GitHub Containers
  • Artifactory Docker Reg: Local repo option.
  • AWS Container Registry

Kubernetes Container Orchestration

  • Docker Compose: smaller apps

  • K8s (Kubernetes) for the big kahunas

    • Cluster Architecture

      • 1 Control Plane (see components below)
      • n Namespaces (default, plus additionals)
        • n Nodes: 1 physical or virtual host: 1 kubelet, 1 k-proxy, 1 container runtime
          • n Pods per node
            • n Containers per pod
          • services
          • deployments

    • Control Plane Components

      • kubeadm
        • HA Control Components. Options:
          1. Stacked control plane nodes
          2. External etcd cluster
        • Maps to a container runtime
          • Docker: default if it is installed
          • containerd: (docker installs containerd as well, so version conflict)
          • CRI-O
      • kube-apiserver
      • etcd (backing store) key-value store
      • kube-scheduler: watches for new pods with no assigned nodes
      • kube-controller-manager
        • Node controller
        • Job controller
        • Endpoints
        • Service Account and Token
      • cloud-controller-manager: link cluster to cloud provider’s API

    • Node Components: run on every node

      • kubelet: manages containers on this pod via PodSpec
      • kube-proxy: network proxy.
      • container runtime: Options implementing Kubernetes CRI (container runtime interface)
        1. Docker Engine
        2. containerd
        3. CRI-O
      • crictl

    • Addons

    • DNS: Cluster DNS server. Auto added to containers started by Kube.

      • Namespaces of Services
      • DNS Records
        • Services
          • A/AAAA records for normal services.
          • SRV records for headless.
        • Pods
          • A/AAAA
          • hostname and subdomain
          • setHostnameAsFQDN(boolean): fully qualified domain name -> hostname.
          • DNS Policy
          • DNS Config

    • Web UI (Dashboard)

    • Container Resource Monitoring

    • Cluster-level Logging

      • Logging agent
        • sidecar container

  • Other

    • Loft Multi-tenancy, self service.

    • KubeCtl CLI commands to talk to clusters

    • Local Kubernetes Dev

      • kind (K8s-in-Docker) Local kubernetes.
      • minikube: Local K8s. SINGLE NODE CLUSTER. Auto installs podman (check version)
        • LoadBalancer
        • Multi-cluster
        • NodePorts
        • Persistent Volumes
        • Ingress
        • Dashboard
        • Container runtime
        • API Server
        • Addons
        • NVIDIA GPU support
        • Filesystem mounts

    • Istio Service Mesh: Extends K8s

      • traffic mgt: inter service routing, failure recovery, load balancing
      • telemetry
      • security: encryption, role-based access and auth

    • Argo

      • Argo Workflows: k8s native workflow engine. Supports DAG, step-based.
      • Argo CD: Cont. delivery, UI.
      • Argo Rollouts: k8s deployment strategies. Canary, Blue-Green.
      • Argo Events: event based dependency mgt.

    • Rancher (Enterprise K8s mgt)

    • Helm

    • Tekton

    • Konfigure?

    • Open Telemetry

  • K3s (Lightweight Kubernetes) IOT, Edge, ARM, RaspPi. <50MB>

  • Local (For Devs)

Hosting Providers

Storage:

  • Longhorn (Rancher Labs)