This seems like a really weird decision. If base images are duplicated for every image you have, that will add up quickly.

> The containerd image store uses more disk space than the legacy storage drivers for the same images. This is because containerd stores images in both compressed and uncompressed formats, while the legacy drivers stored only the uncompressed layers.

Why ?

This means `/var/lib/docker` is no longer "hermetic": images and container snapshots are located in `/var/lib/containerd` now.

More info about the switch: https://www.docker.com/blog/docker-engine-version-29/

To configure this directory, see https://docs.docker.com/engine/storage/containerd/.

Meanwhile, the basic stuff like caching doesn't work properly.

How bad did we fall with the ship often, ship early and fix later idea? Make a major change, release it and the migration feature is experimental and not recommended.

First of all, some history; the Docker Engine was a monolith daemon that provided many services; this worked well when using Docker as a standalone solution, but when used as runtime for Kubernetes, this wasn't ideal; many components were not designed for this purpose, which meant they had to be replaced / overridden with hacks to make it work. The containerd project was created to provide a more modular runtime for the container ecosystem, providing separate subcomponents (a containerd runtime, image/content storage) for the container ecosystem to build on. It was created "from scratch" with lessons learned over the Years, providing a modern foundation.

While docker has used containerd as a runtime for many Years, it still used its own implementation for storing images ("graph-drivers"); this implementation started to show its age and had many limitations; graph-drivers have no native support for multi-platform ("multi-arch") images, no support for OCI Artifacts, and no reproducible images when pushing to different registries (among others).

Around 4 Years ago, we started to re-implement the image storage using containerd "snapshotters"; our initial goal was to provide a mostly seamless transition; add multi-arch support, but keep the UX as close as possible to the graph-drivers. Around 2 Years ago, Docker Desktop changed to using the containerd image storage (snapshotters) as a default for new installations, and Docker v29 made it the default for Linux installations.

While we kept most of the UX similar, there are some differences; when storing an image with graph-drivers, docker would pull the OCI image, extract the content (layers), and discard the (compressed) layers. While this reduced storage, it also made images non-reproducible as the image had to be re-constructed when pushing to a registry (which also resulted in slower pushes).

The containerd image storage uses a different design, where a copy of the compressed artifacts are preserved (by default); this requires more storage to keep these extra blobs, but reduces duplication and increases push performance. It was the decision containerd maintainers made early in their design process, and all containerd-based tools have used this model since the start of the containerd project.

We have a couple of roadmap items to improve this in future; some are outlined in this ticket; https://github.com/moby/moby/issues/51581, but there's other options that will become availeble through the containerd image store; support for erofs as an alternative to (tar) compressed image layers, as well as automatic garbage-collection (which would reduce the need for manually pruning content through `docker system prune` (and related commands).

(FWIW; docker still provides graph-drivers as an alternative https://docs.docker.com/engine/storage/drivers/select-storag...)

I can't believe Docker finally shit the bed. Time to replace Docker with Podman.... sigh

My reasoning is simply that I don't really want to swap out one overly complicated thing for another. I'm sure Podman is fine and amazing. But I'm just not in the IBM/Red Hat ecosystem and I have some reservations their generally a bit overly complex solutions. There's a reason IBM is involved, just saying. And as I'm not planning to use podman in production I see no reason to have it on my laptop.

As for Rancher, that seems to me a bit like moving the problem than solving it as it seems to be a for profit solution around an OSS core with its own complexity and potentially similar risks to Docker Desktop down the road.

With colima, it's all open source and easy to install/upgrade via homebrew. Nice simple wrapper around qemu. There's no UI, and I don't really miss having one. Lazydocker works fine as a TUI if you crave a UI and so do other generic docker UIs/IDEs. I mainly use docker and docker compose on the command line and that works fine for me. It has Kubernetes support as well if you need that but that's not something I use or need.

    docker system prune -a -f
    docker volume prune -a -f