MicroK8s images prune utility for production servers

General Discussions microk8s
1

For those that are using microk8s for production, you could be interested in a prune utility that would remove images unused by current containers (like docker system prune). The microk8s ctr utility do not provide such feature.

My experience is that images are quickly stacking and I am not sure any garbage collector is taking care of that.

If you need one, check GitHub - philippelt/microk8s_prune: Containerd prune utility to cleanup loaded but unused images. This utility is targeting microk8s release but can easily be adapted to any containerd installation.

If you are aware of alternative methods, all suggestions are welcome…

2 Likes
2

Sorry for being late, but for folks in the future I’ll leave this suggestion:

  • install crictl on each node from The docs
  • run crictl -r unix:///var/snap/microk8s/common/run/containerd.sock rmi --prune
    Maybe add this to a cron task…
3 Likes
3

Franco_Martin has the right answer. Thank you! I have successfully recovered about 500GB of storage on my cluster thanks to this answer.

4

In contrast to Franco_Martin’s answer and Jamie_Carl’s affirmation, I recommend properly configuring kublet garbage collection.

The docs recommend avoiding external GC tools: “You should avoid using external garbage collection tools, as these can break the kubelet behavior and remove containers that should exist.”

5

Thank you! crictl was the solution for me;
I recovered more than 100GB of storage.
@bispaj, I spent a couple of hours trying to figure out how to get GC to work the way I wanted in MicroK8s.
The Kubernetes documentation isn’t always helpful for MicroK8s

6

Interesting, glad you found a solution. I’m mostly curious if you don’t mind sharing, how do you want GC to work?

I set the following config then restarted the kubelite daemon on each node.

sudo tee -a /var/snap/microk8s/current/args/kubelet << EOF
–image-gc-high-threshold=75
–image-gc-low-threshold=10
EOF

Simple, but it has worked flawlessly in production envs for the last year. Your GC requirements may be more complicated.

7

Well, I was looking for some kind of kubelit yaml config file, but I didn’t found!
Your example is what I was looking for.
Great, I will try to use your example!
Thank you for sharing.