A practical, opinionated guide for homelab engineers who want their Pi nodes to behave like real servers
The Raspberry Pi 5 finally has enough horsepower to serve as a legitimate node in a distributed storage + Kubernetes cluster. But to get there, you need to treat it like a real machine: deterministic networking, proper cgroups, stable storage, and a clean boot environment.
This guide walks through converting a fresh Raspberry Pi OS installation into a fully functional:
- MicroK8s Kubernetes node
- MicroCeph storage node
- NFS‑capable worker
- Static‑DNS, Ethernet‑only, cluster‑friendly machine
The setup assumes your Ceph OSD lives on an external SSD connected via a USB 3 adapter, which is the most practical and performant option on a Pi 5 today.
Using Ubuntu Server as an operating system for the node is an alternative. Setup would be much simpler and require less steps, but I’ve found out Raspberry Pi OS (based on Debian) is much more stable as well as has a lower memory footprint. It’s just worth spending more time during setup phase and then benefit from a faster and more stable solution. So, Debian, here we go!
Set the Hostname
Start clean. A predictable hostname makes logs, Ceph maps, and Kubernetes node lists far easier to manage.
sudo nano /etc/hostname
sudo nano /etc/hosts
Edit both files and set desired hostname. In my case it was rpi503
Reboot if you want to be tidy.
Generate an SSH Keypair
Each node should have its own identity for cluster administration and GitOps workflows.
ssh-keygen -t ed25519 -C "rpi503"
Configure Static DNS (NetworkManager)
Raspberry Pi OS uses NetworkManager, which is great — until DHCP overwrites your DNS settings and breaks cluster resolution.
List connections:
nmcli connection show
Then enforce your DNS resolvers:
sudo nmcli connection modify "Wired connection 1" ipv4.dns "1.1.1.1 8.8.8.8"
sudo nmcli connection modify "Wired connection 1" ipv4.ignore-auto-dns yes
sudo nmcli connection up "Wired connection 1"
This ensures Kubernetes and Ceph always see the correct DNS servers.
Enable cgroups (Required for Kubernetes)
MicroK8s needs memory cgroups enabled.
Edit:
sudo nano /boot/firmware/cmdline.txt
Append:
cgroup_enable=memory cgroup_memory=1 systemd.unified_cgroup_hierarchy=1
Reboot to apply.
Disable Wi‑Fi and Force Ethernet‑Only Operation
Clusters hate unstable interfaces. Disable Wi‑Fi entirely:
sudo nano /boot/firmware/config.txt
Add:
dtoverlay=disable-wifi
Install Snap Support
Both MicroCeph and MicroK8s are delivered via Snap.
sudo apt install snapd
Reboot afterward.
Install MicroCeph
sudo snap install microcephFix your secure path so sudo can see snap binaries:
sudo visudo
Modify:
Defaults secure_path="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin"
Join the Ceph Cluster
On the existing cluster node:
sudo microceph cluster add rpi503
You’ll receive a join token.
On the new node you are configuring:
sudo microceph cluster join <token>
The node is now part of your Ceph cluster.
Install Ceph Client Tools & Enable Kernel Module
sudo apt install ceph-common
sudo modprobe ceph
Make it persistent:
sudo nano /etc/modules-load.d/ceph.conf
Add:
ceph
Add Your First OSD (USB‑Attached SSD)
Most Pi 5 Ceph deployments use an external SSD connected via USB 3. This works extremely well — as long as your USB adapter supports UAS.
Check:
lsusb -t
Look for:
Driver=uas
If it falls back to usb-storage, performance will be lower but still acceptable.
Assuming your SSD appears as /dev/sda:
sudo wipefs -a /dev/sda
sudo microceph disk add /dev/sda
Then enable Ceph services on this node:
sudo microceph enable mon -target rpi503
sudo microceph enable mgr -target rpi503
sudo microceph enable mds -target rpi503
Your Pi is now a real Ceph node with a USB‑SSD OSD.
Install MicroK8s
sudo snap install microk8s --classic
On the existing cluster node:
microk8s add-node
This will generate a cluster join command. Copy it.
On the new node:
sudo usermod -a -G microk8s <your username>
sudo chown -R pspychalski ~/.kube
newgrp microk8s
Join:
microk8s join <cluster-join-command>
The node will appear in:
microk8s kubectl get nodes
Add NFS Support
Some workloads still rely on NFS, especially legacy apps and simple PVCs.
sudo apt install nfs-common
Fix MicroK8s vs NetworkManager DNS Conflicts
MicroK8s expects /run/systemd/resolve/resolv.conf.
NetworkManager overwrites it.
Add a boot‑time fix:
crontab -e
Insert:
@reboot sudo mkdir -p /run/systemd/resolve && sudo ln -sf /etc/resolv.conf /run/systemd/resolve/resolv.conf
This ensures Kubernetes always sees your static DNS configuration.
Conclusion
Your Raspberry Pi 5 is now a fully integrated:
- Ceph storage node with a USB‑SSD OSD
- MicroK8s worker
- NFS‑capable utility node
- Static‑DNS, Ethernet‑only cluster citizen
This setup behaves like a proper datacenter node — just smaller, quieter, and far more fun to build.
Leave a Reply