MicroK8S load balancer IP broken on Ubuntu

LoadBalancer Services Broken After Ubuntu Upgrade? Here’s Why

If your pods suddenly vanish from LoadBalancer services—but only when accessed from outside the cluster—you’re not alone. Everything works perfectly from within the cluster, yet external traffic just… stops.

So what’s going on?

After a recent Ubuntu upgrade, MetalLB’s speaker pods lose the necessary permissions to manage network interfaces and announce LoadBalancer IPs via ARP. This breaks external connectivity, even though internal traffic flows without issue.

Once you inspect MetalLB logs with the following command

kubectl logs -n metallb-system -l component=speaker

You will error messages similiar to this:

{
  "caller":"announcer.go:106",
  "error":"creating ARP responder for \"eth0\": listen packet 2c:cf:67:ef:1b:4b: socket: permission denied",
  "interface":"eth0",
  "level":"error",
  "msg":"failed to create ARP responder",
  "op":"createARPResponder",
  "ts":"2025-10-01T07:13:09Z"
}

This indicates, that load balancer has no permissions to tap into the network socket and lister and respond to ARP requests. As a result, hosts that are not part of the cluster, are unable to resolve IP addresses assigned by MetalLB.

I’ve hit this situation on my local MicroK8S cluster based on Raspberry Pi 5 16GB after I upgraded the last node from Ubuntu Server 24.10 to Ubuntu Server 25.02. As long as ugrade process was ongoing (I took my time) and at least one Ubuntu Server 24.10 node was running, ARP announcement was working. The moment I upgraded the last one, it stopped out of the sudden.

The Fix

To restore ARP functionality, you need to grant MetalLB’s speaker pods privileged access:

kubectl patch daemonset speaker -n metallb-system \
  --type='json' \
  -p='[
    {
      "op": "replace",
      "path": "/spec/template/spec/containers/0/securityContext",
      "value": {
        "privileged": true
      }
    }
  ]'

And then restart load balancer with:

kubectl rollout restart daemonset speaker -n metallb-system

This patch re-enables ARP announcements and restores external LoadBalancer access.


Posted

in

by

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *