Kubernetes

Kubernetes, often abbreviated as K8s, is an open-source container orchestration platform designed to automate the deployment, scaling, and management of containerized applications. Originally developed by Google and now maintained by the Cloud Native Computing Foundation (CNCF), Kubernetes has become the de facto standard for managing containerized workloads in production environments.

What is Kubernetes?

Kubernetes provides a platform-agnostic framework for automating the deployment, scaling, and management of containerized applications. It abstracts away the underlying infrastructure, allowing developers to focus on building and deploying applications without worrying about the underlying infrastructure complexities.

Key Concepts:

• Containerization: Kubernetes leverages containerization technology (e.g., Docker) to package applications and their dependencies into containers, ensuring consistency and portability across different environments.

• Clusters: Kubernetes organizes containerized applications into clusters, which consist of a master node (control plane) and multiple worker nodes (where containers are deployed).

• Pods: The basic unit of deployment in Kubernetes is a Pod, which represents one or more containers deployed together on the same host. Pods share network and storage resources and can be horizontally scaled.

• Services: Kubernetes abstracts away the network communication between Pods using Services. Services provide a stable IP address and DNS name to access a set of Pods, enabling load balancing and service discovery.

• Deployments: Deployments are used to manage the lifecycle of Pods and ensure that a desired number of replicas are running at all times. Deployments allow for rolling updates, rollback, and scaling of application replicas.

• Volumes: Kubernetes provides various types of storage volumes that can be attached to Pods, allowing applications to persist data beyond the Pod lifecycle.

apiVersion: v1
kind: Pod
metadata:
  name: learndeck-pod
spec:
  containers:
  - name: learndeck-container
    image: learndeck:latest
    ports:
    - containerPort: 80

Why Kubernetes is Awesome:

• Scalability: Kubernetes enables horizontal scaling of applications by automatically distributing workloads across multiple Pods and nodes, allowing applications to scale up or down based on demand.

• Resilience: Kubernetes provides built-in mechanisms for self-healing, including automatic Pod restarts, health checks, and node failover, ensuring high availability and resilience of applications.

• Portability: Kubernetes abstracts away the underlying infrastructure, making it easy to deploy and manage applications across different environments, including on-premises data centers, public clouds, and hybrid clouds.

• Flexibility: Kubernetes supports a wide range of workloads, including stateless and stateful applications, batch jobs, and microservices architectures. It provides a flexible platform for running diverse workloads in production environments.

• Ecosystem: Kubernetes has a vibrant ecosystem of tools, libraries, and extensions that extend its functionality and integrate with other cloud-native technologies, such as service mesh, monitoring, logging, and continuous integration/continuous deployment (CI/CD) tools.

Build one yourself: a 4-node Raspberry Pi cluster

Everything above is easier to believe once you have watched it happen on hardware you can touch. Four Raspberry Pis and an afternoon will get you a real, conformant Kubernetes cluster - and, more usefully, let you yank the power cable out of a node and watch the self-healing described above actually reschedule your Pods somewhere else.

To be clear about what this is: a learning and homelab cluster, not a production one. It will teach you the concepts, run your home services, and give you somewhere safe to break things. It will not survive a data centre audit.

                          your laptop
                            kubectl
                               │
                               │  :6443
   ┌───────────────────────────┴──────────────────────────────┐
   │ switch   (wired — do not try this over wifi)             │
   └─────┬───────────────┬───────────────┬───────────────┬────┘
         │               │               │               │
   ┌──────────┐    ┌──────────┐    ┌──────────┐    ┌──────────┐
   │ pi-cp    │    │ pi-w1    │    │ pi-w2    │    │ pi-w3    │
   ├──────────┤    ├──────────┤    ├──────────┤    ├──────────┤
   │ k3s      │    │ k3s      │    │ k3s      │    │ k3s      │
   │  server  │    │  agent   │    │  agent   │    │  agent   │
   │          │    │          │    │          │    │          │
   │ api      │    │ ┌──────┐ │    │ ┌──────┐ │    │ ┌──────┐ │
   │ scheduler│    │ │ pod  │ │    │ │ pod  │ │    │ │ pod  │ │
   │ ctrl-mgr │    │ └──────┘ │    │ └──────┘ │    │ └──────┘ │
   │ sqlite   │    │ ┌──────┐ │    │          │    │ ┌──────┐ │
   │          │    │ │ pod  │ │    │          │    │ │ pod  │ │
   │          │    │ └──────┘ │    │          │    │ └──────┘ │
   └──────────┘    └──────────┘    └──────────┘    └──────────┘

What you need

Give each Pi a static IP (or a DHCP reservation on your router) and a unique hostname. Kubernetes gets deeply confused by nodes that move or share names.

Why k3s rather than kubeadm

You can run upstream Kubernetes on a Pi, but on this hardware k3s is the sensible default. It is a CNCF-conformant distribution - the same API, the same kubectl, the same YAML - packaged as a single binary with the cloud-provider bloat stripped out.

Step 1: Prepare every Pi

Do this on all four. Flash 64-bit Raspberry Pi OS Lite (no desktop - it is a server), then:

# use pi-w1, pi-w2, pi-w3 on the other three
sudo hostnamectl set-hostname pi-cp

# Kubernetes needs cgroup memory accounting, which the Pi
# disables by default. Append to the END of the single line
# in that file - do not add a newline.
# on older OS versions this is /boot/cmdline.txt
sudo nano /boot/firmware/cmdline.txt
#   ... rootwait cgroup_memory=1 cgroup_enable=memory

# Kubernetes wants swap off
sudo dphys-swapfile swapoff
sudo systemctl disable dphys-swapfile

sudo reboot

That cgroup_enable=memory line is the single most common reason a Pi cluster refuses to start. If k3s installs cleanly but no Pod ever runs, this is why.

Step 2: The control plane

On pi-cp:

curl -sfL https://get.k3s.io | sh -

sudo systemctl status k3s     # should be active (running)
sudo k3s kubectl get nodes    # one node, should say Ready

# You need this token to join the workers
sudo cat /var/lib/rancher/k3s/server/node-token

Step 3: The workers

On each of pi-w1, pi-w2 and pi-w3, using the control plane's IP and the token from above:

curl -sfL https://get.k3s.io | \
  K3S_URL=https://192.168.1.10:6443 \
  K3S_TOKEN=K10abc...paste-your-token-here \
  sh -

Back on the control plane, they should appear within a few seconds:

sudo k3s kubectl get nodes -o wide

# Cosmetic, but makes `get nodes` much easier to read
for n in pi-w1 pi-w2 pi-w3; do
  sudo k3s kubectl label node $n \
    node-role.kubernetes.io/worker=worker
done

Step 4: Drive it from your laptop

You do not want to SSH into a Pi every time you run kubectl. Copy the kubeconfig across and point it at the Pi instead of localhost:

scp pi@192.168.1.10:/etc/rancher/k3s/k3s.yaml \
  ~/.kube/pi-cluster.yaml

# point it at the Pi instead of localhost
# (on Linux, drop the '' after -i)
sed -i '' 's/127.0.0.1/192.168.1.10/' ~/.kube/pi-cluster.yaml

export KUBECONFIG=~/.kube/pi-cluster.yaml
kubectl get nodes

Step 5: Deploy something, then break it

This is the part worth doing. Deploy six replicas of something small and watch the scheduler spread them across your nodes:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: learndeck
spec:
  replicas: 6
  selector:
    matchLabels:
      app: learndeck
  template:
    metadata:
      labels:
        app: learndeck
    spec:
      containers:
      - name: web
        # multi-arch image, so it runs on arm64
        image: nginx:alpine
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: learndeck
spec:
  # k3s ships a load balancer, so this just works
  type: LoadBalancer
  selector:
    app: learndeck
  ports:
  - port: 80
    targetPort: 80

kubectl apply -f learndeck.yml
# note the NODE column - they are spread across the cluster
kubectl get pods -o wide

Now pull the power on pi-w2 and keep watching:

kubectl get pods -o wide --watch

For about five minutes nothing happens - that is the node timeout, not a bug - and then the Pods that were on pi-w2 are recreated on the surviving nodes. Plug it back in and it rejoins on its own. That single experiment makes Deployments, desired state and self-healing concrete in a way no diagram can.

What k3s gives you for free

Worth knowing, because you will otherwise go looking for them:

• Traefik as an Ingress controller, so Ingress resources work out of the box
• ServiceLB, which is why type: LoadBalancer above got an IP instead of hanging in <pending> forever like it would on a bare cloud-less cluster
• CoreDNS for the in-cluster service discovery described earlier
• local-path-provisioner, a default StorageClass that puts Volumes on the node's own disk

Things that will bite you

• SD cards die. The control plane writes constantly. A cheap card will fail in weeks and the symptoms are baffling - random corruption, nodes going NotReady. Boot from an SSD, or at minimum use endurance-rated cards.
• Not every image runs on ARM. Anything without an arm64 build will crash-loop with an exec format error. Most popular images are multi-arch now; some Helm charts still are not. For your own images, docker buildx build --platform linux/arm64.
• A single-server k3s cluster is not highly available. If pi-cp dies, the cluster stops accepting changes. Real HA needs three servers with embedded etcd (--cluster-init) - which is a fine next project, but do not mistake this build for it.
• Memory disappears faster than you expect. The k3s agent and system overhead eat a chunk of a 4GB Pi before your workloads get a look in. Set resource requests on your Pods so the scheduler knows what it is working with.
• Underpowered PSUs. Undervoltage on a Pi looks exactly like flaky software. Check with dmesg | grep -i voltage before you debug anything else.

Overall, Kubernetes revolutionizes the way we deploy and manage containerized applications, providing a scalable, resilient, and portable platform for modern cloud-native architectures. Its extensibility and rich feature set make it a powerful tool for building and running applications at scale.