From 440cb2077fdae0da287ecd2f7ee9f43aa8ab07a5 Mon Sep 17 00:00:00 2001
From: CodeGit <matloob@gmail.com>
Date: Wed, 19 Aug 2026 07:29:32 +0100
Subject: [PATCH] docs: fix QUICKSTART manual VM storage for system libvirt

Disk images under k8s's home directory aren't readable by the
libvirt-qemu user that actually runs VMs under qemu:///system. Use
libvirt's own default storage pool (vol-create-as/vol-upload) instead,
so libvirtd handles file ownership rather than k8s directly, and pin
LIBVIRT_DEFAULT_URI=qemu:///system throughout.
---
 docs/QUICKSTART.md | 61 ++++++++++++++++++++++++++++++++++++++++------
 1 file changed, 53 insertions(+), 8 deletions(-)

diff --git a/docs/QUICKSTART.md b/docs/QUICKSTART.md
index 8397160..c31b675 100644
--- a/docs/QUICKSTART.md
+++ b/docs/QUICKSTART.md
@@ -21,13 +21,49 @@ Everything below runs as `k8s` on the T630 (`sudo -iu k8s`).
 
 ## 1. Create the VM
 
-```sh
-mkdir -p ~/vms
-curl -L -o ~/vms/noble-base.img \
-  https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img
+No sudo needed anywhere in this guide — `k8s`'s `libvirt`/`kvm` group
+membership (step 2) is what authorizes talking to libvirt at all, via
+polkit. But group membership alone doesn't solve everything: system libvirt
+runs actual VMs as a separate, restricted `libvirt-qemu` user, not as
+`k8s` — so a disk image sitting under `k8s`'s home directory (mode 700 by
+default) would fail at boot, since that user can't read into `k8s`'s home
+at all. The fix is to let libvirt manage the disk storage itself, inside
+its own pool, rather than pointing at a raw path in `~` — pool operations
+go through libvirtd's API, so it's libvirtd (already running with the
+right privileges) that handles the file ownership, not `k8s` directly.
 
-# a writable overlay on top of the base image, sized up to 20G
-qemu-img create -f qcow2 -F qcow2 -b ~/vms/noble-base.img ~/vms/k3s-manual.qcow2 20G
+Make sure everything below targets the system instance — matches where
+Terraform will eventually create the real VMs too:
+
+```sh
+export LIBVIRT_DEFAULT_URI=qemu:///system
+```
+
+This only lasts for the current shell — add it to `k8s`'s `~/.bashrc` too
+(`echo 'export LIBVIRT_DEFAULT_URI=qemu:///system' >> ~/.bashrc`) so it's
+still set in step 2 and the teardown section later, even in a fresh login.
+
+Download the cloud image somewhere `k8s` can read it — this location
+itself doesn't need to be qemu-readable, since it's only ever read by
+`virsh` (running as `k8s`), never directly by the VM:
+
+```sh
+curl -L -o /tmp/noble-base.img \
+  https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img
+```
+
+Import it into libvirt's `default` storage pool (created automatically
+when `libvirt-daemon-system` was installed in step 1), then create a
+copy-on-write overlay on top of it, sized up to 20G:
+
+```sh
+virsh vol-create-as default k3s-manual-base.qcow2 \
+  --capacity "$(stat -c%s /tmp/noble-base.img)" --format qcow2
+virsh vol-upload --pool default k3s-manual-base.qcow2 /tmp/noble-base.img --sparse
+rm /tmp/noble-base.img
+
+virsh vol-create-as default k3s-manual.qcow2 20G --format qcow2 \
+  --backing-vol k3s-manual-base.qcow2 --backing-vol-format qcow2
 ```
 
 Write the cloud-init user-data — paste in the contents of
@@ -64,12 +100,16 @@ network (uses libvirt's default NAT network + DHCP). A k3s server node
 schedules workloads on itself by default, so this alone is a complete,
 usable cluster.
 
+The cloud-init user-data file itself is fine living under `~/vms` — unlike
+the disk, it's only ever read client-side by `virt-install` (as `k8s`),
+which then hands the rendered seed data to libvirtd over the API:
+
 ```sh
 virt-install \
   --name k3s-manual \
   --memory 2048 \
   --vcpus 2 \
-  --disk ~/vms/k3s-manual.qcow2 \
+  --disk vol=default/k3s-manual.qcow2 \
   --import \
   --os-variant ubuntu24.04 \
   --network network=default \
@@ -123,8 +163,13 @@ the real 3-node cluster, remove this one:
 
 ```sh
 # as k8s
+export LIBVIRT_DEFAULT_URI=qemu:///system
 virsh destroy k3s-manual              # stop it
-virsh undefine k3s-manual --remove-all-storage   # delete VM + its disk
+virsh undefine k3s-manual --remove-all-storage   # delete VM + its overlay disk
+virsh vol-delete --pool default k3s-manual-base.qcow2   # the base image isn't
+                                                          # attached to the VM
+                                                          # directly, so it
+                                                          # needs its own delete
 rm ~/.kube/config-manual
 rm -rf ~/vms
 ```