Final Configs
In case you missed it, Pt1 is here.
In this section we will finish with some prerequisites and configs followed by Kubernetes installation.
We will start by installing containerd.
sudo apt-get install containerd
There are other runtimes you can use instead of containerd, this post discusses various runtimes and their installation procedure.
After the containerd installation we need to update the config on it a bit.
sudo mkdir -p /etc/containerd
sudo touch /etc/containerd/config.toml
sudo chown pikube:pikube /etc/containerd/config.toml
sudo containerd config default > /etc/containerd/config.toml
sudo chown root:root /etc/containerd/config.toml
Alright, I feel the need to explain the above section a bit. Normally you would just run “sudo containerd config default > /etc/containerd/config.toml”, but I could not get it to work. I kept getting permission denied.
I even tried switching to the root user with “su -” to run the command and it was still failing. So my user on my raspberry pi is “pikube” so I changed ownership of the config.toml file to be my user. I ran the command which was failing. Then switched it back to root as the owner.
After the config is updated we still need to make one change to it. It is an important one.
sudo nano /etc/containerd/config.tomlIn the config scroll down to this section
You need to have SystemdCgroup = true
When I did not have that value set I would encounter all kind of failures of KubeIP:6443 was refused over and over again. Sometimes it would work briefly then immediately fail.
It took me a long time to figure out, and I looked through all the google results for :6443 was refused, connection refused, and anything else I could find. Finally found the solution here in a deep github issue.
Alright now that containerd is sorted. Let’s get on to Kubernetes!
We will add the GPG key and the kubernetes repo:
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add
sudo apt-add-repository "deb http://apt.kubernetes.io/ kubernetes-xenial main"
Next we will install Kubernetes
sudo apt-get install kubeadm kubelet kubectl
It is also a good idea if we set these apps to not update on their own.
sudo apt-mark hold kubelet kubeadm kubectl
Control groups(cgroups) allow Kubernetes to manage and isolate each container’s resource usage. If you want additional information on cgroups and kubernetes, check here. Fortunately configuring them in super easy with these quick lines from Pixel Robots.
cgroup="$(head -n1 /boot/firmware/cmdline.txt) cgroup_enable=cpuset cgroup_enable=memory cgroup_memory=1 swapaccount=1"
echo $cgroup | sudo tee /boot/firmware/cmdline.txt
We also need to turn off swap:
sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab
sudo swapoff -a
Let’s add some kernel modules and make sure they are always loaded.
sudo tee /etc/modules-load.d/containerd.conf <<EOF
overlay
br_netfilter
EOF
sudo modprobe overlay
sudo modprobe br_netfilter
And finish it up with some IPTables config for Kubernetes:
sudo tee /etc/sysctl.d/kubernetes.conf <<EOF net.bridge.bridge-nf-call-ip6tables = 1 net.bridge.bridge-nf-call-iptables = 1 net.ipv4.ip_forward = 1 EOF
And do a reboot for good measure
sudo reboot
Initialize Kubernetes
Time for hopefully the fun stuff. If you have been following up to this point this upcoming part is where we find out if everything is going to work for us.
While not necessary because Kubernetes will pull the images it needs on initialization, we will go ahead and pull them now.
sudo kubeadm config images pull
And finally initialize the Kubernetes cluster:
sudo kubeadm init --pod-network-cidr=10.244.0.0/16
This can take a couple minutes and you may see a lot of text, but if everything worked out, you should see this screen:
Next, run the commands Kubernetes mentions
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
It also provides a join command if you would like to join another node to the cluster. This token can timeout, but you can always generate another in the future if you need.
We should now be able to see our single node with kubectl:
kubectl get nodes -A
Finally we need to choose and setup a CNI for our Kubernetes cluster. There are several out there and I currently don’t know enough to do a proper comparison here, but you can check one out here. I personally choose Calico. Regardless of which one you choose the setup should be pretty similar though.
For Calico:
curl https://projectcalico.docs.tigera.io/manifests/calico.yaml -O
kubectl apply -f calico.yaml
That simple.
You should be able to see Calico getting setup as well
kubectl get pods -A
Now we are finally done and you have setup your own Kubernetes cluster on Raspberry Pi.
Congratulations. I hope this guide has been helpful!
References
https://pixelrobots.co.uk/2021/03/building-a-raspberry-pi-kubernetes-cluster-on-ubuntu-20-04-using-kubeadm-and-containerd-as-the-runtime/
https://www.linuxtechi.com/install-kubernetes-on-ubuntu-22-04/
https://computingforgeeks.com/install-kubernetes-cluster-ubuntu-jammy/
https://github.com/containerd/containerd/issues/4581