However a fun fact is that you can (do not actually do this!) boot a qemu VM from /dev/sda. You have to use an overlay (eg. qemu -drive snapshot=on flag) so that qemu won't write through to /dev/sda. I use this trick in supernested, a script I wrote that runs nested within nested within nested VMs ad infinitum until your hypervisor crashes. http://git.annexia.org/?p=supernested.git;a=blob;f=run-super...

I went down a similar rabbit-hole myself, with the goal of safely replacing the Linux installation on a disk that a machine is already running from (e.g. replace a VPS's setup image with one of your own) without needing a KVM-style remote access tool to the console.

The problem there is if you directly modify the disk when a filesystem is mounted on that disk then all bets are off in terms of corruption of the filesystem that's already on there and also the filesystem(s) you're writing over the top.

My solution was to kexec into a new kernel+initramfs which has a DHCP client and cURL in it - that effectively stops any filesystem access while the image is being written over the disk, then to just reboot.

QEMU defaults to 512B sectors, which isn't true for many NVMe drives. There are some flags to change that. https://unix.stackexchange.com/a/722450

I think it should be possible to make an image with many headers at different locations, so that it works on all types of disks at once, but I don't think any tools do it for you by default.

We had a big drive with the source of truth image used to boot all our machines on it, and we added rsync to the init image. When each machine booted init would rsync everything from the storage box to the local machine. We'd keep the storage machine up to date and when we wanted to update other machines in the fleet we'd just do a reboot and it would sync up the latest files (provisioning for whatever each machine was supposed to do happened later, can't remember how that was handled now). The storage machine was running ZFS so we also took a snapshot before doing any rolling reboots, so if anything did go wrong you could just revert to the previous snapshot and reboot again as long as you didn't break the init image.

Sounds jank saying it out loud, but I don't remember it ever causing us any problems.

  After waiting for a little while, the program terminated with the following output:
  
  astrid@chungus infra  gzip -vc result/nixos.img | ssh root@myhost.example -- bash -c 'gunzip -vc > /dev/sda'
  root@myhost.example's password:
   77.8% -- replaced with stdout
  
  What happened here?

I do like this idea - with sufficient prep of the system before writing the image, namely stopping as many processes as possible especially those that might do some writing, it's a quick and dirty way to replace a stock OS with a ready-made image. Could perhaps be safer doing it twice, once into a minimal image that does very little beyond network bringup & runs ssh, followed by final OS replacement in a (more) controlled manner.

> The OS may stop you from unmounting /dev/sda1, but it won’t stop you from writing to /dev/sda1 or /dev/sda even if there’s something mounted!

Not always true. There's a kernel config option that allows it. CONFIG_BLK_DEV_WRITE_MOUNTED

They'd netboot.. not mount the disks, then download an ISO/IMG and write it directly to the primary boot disk.

If netbooting is a heavy lift, why not boot into a custom initramfs you built, with i.e. dd/curl installed, and flash the disk that way, without mounting / at all? Then kexec/chroot into it?

I'd much prefer this as a way to provision Raspberry Pis.

Stupid penguin trick I learned: Add a file inside ramdisk (i use /dev/shm) as LVM PV.

pvmove off the hard drive

Boom, now your OS lives entirely in RAM

You can now even replace the hard disk, put a new one and migrate back.

Or migrate to network storage (nbd,iSCSI etc.), re-sequence disks into whatever RAID you need, and migrate back

Need to fix /boot after that tho, and probably make sure to not have power failure in meantime

So ending up at:

> From a 292MB initramfs, we now have a 6.1MB initramfs, smaller than almost every other distro's initramfs and made entirely to run busybox wget dd.

Is pretty great achievement today - but way bigger than something that can fit on a floppy.

I had found it in a github gist when I used it but here's a similar blog post.

https://alextsang.net/articles/20191006-063049/index.html

Also, I once burned an iso straight from ftp using a fifo. I was low on disk space and really needed that CD. Worked fine because the Internet was already faster than the CDR.

[0] https://github.com/nix-community/nixos-anywhere

The neat part was the whole process kicked off when you scp'd the rootfs and inotifywait kicked off the whole process.

https://support.tools/dd-over-netcat-clone-drive-remote-back...

But I like the curl approach very much!

Well not quite direct; the bits go through your RAM in between.