diff --git a/docs/QUICKSTART.md b/docs/QUICKSTART.md index 5718101..8251d56 100644 --- a/docs/QUICKSTART.md +++ b/docs/QUICKSTART.md @@ -32,16 +32,40 @@ its own pool, rather than pointing at a raw path in `~` — pool operations go through libvirtd's API, so it's libvirtd (already running with the right privileges) that handles the file ownership, not `k8s` directly. -Make sure everything below targets the system instance — matches where -Terraform will eventually create the real VMs too: +Every `virsh`/`virt-install` command below uses `-c qemu:///system` / +`--connect qemu:///system` explicitly, rather than relying on the +`LIBVIRT_DEFAULT_URI` env var — for a non-root user that env var is what +picks system vs. the per-user `qemu:///session` instance, and it's easy for +it to be set in one terminal and not another, silently causing commands to +operate on the wrong (session) libvirt instance where nothing you created +system-side is visible. Explicit `-c` on every command avoids that +ambiguity entirely. + +**Debian's `libvirt-daemon-system` does *not* auto-create a `default` +storage pool** the way some other distros' packaging does (it does +auto-create the `default` *network*, just left inactive) — check and fix +both before doing anything else: ```sh -export LIBVIRT_DEFAULT_URI=qemu:///system +virsh -c qemu:///system pool-list --all +virsh -c qemu:///system net-list --all ``` -This only lasts for the current shell — add it to `k8s`'s `~/.bashrc` too -(`echo 'export LIBVIRT_DEFAULT_URI=qemu:///system' >> ~/.bashrc`) so it's -still set in step 2 and the teardown section later, even in a fresh login. +If `pool-list` comes back empty: + +```sh +virsh -c qemu:///system pool-define-as default dir --target /var/lib/libvirt/images +virsh -c qemu:///system pool-build default +virsh -c qemu:///system pool-start default +virsh -c qemu:///system pool-autostart default +``` + +If `net-list` shows `default` as `inactive`: + +```sh +virsh -c qemu:///system net-start default +virsh -c qemu:///system net-autostart default +``` Download the cloud image somewhere `k8s` can read it — this location itself doesn't need to be qemu-readable, since it's only ever read by @@ -52,18 +76,19 @@ curl -L -o /tmp/noble-base.img \ https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img ``` -Import it into libvirt's `default` storage pool (created automatically -when `libvirt-daemon-system` was installed in step 1), then create a -copy-on-write overlay on top of it, sized up to 20G: +Import it into the `default` pool, then create a copy-on-write overlay on +top of it, sized up to 20G: ```sh -virsh vol-create-as default k3s-manual-base.qcow2 \ +virsh -c qemu:///system vol-create-as default k3s-manual-base.qcow2 \ --capacity "$(stat -c%s /tmp/noble-base.img)" --format qcow2 -virsh vol-upload --pool default k3s-manual-base.qcow2 /tmp/noble-base.img --sparse +virsh -c qemu:///system vol-upload --pool default k3s-manual-base.qcow2 /tmp/noble-base.img --sparse rm /tmp/noble-base.img -virsh vol-create-as default k3s-manual.qcow2 20G --format qcow2 \ +virsh -c qemu:///system vol-create-as default k3s-manual.qcow2 20G --format qcow2 \ --backing-vol k3s-manual-base.qcow2 --backing-vol-format qcow2 + +virsh -c qemu:///system vol-list --pool default # should list both volumes ``` Write the cloud-init user-data — paste in the contents of @@ -103,18 +128,30 @@ usable cluster. The cloud-init user-data file itself is fine living under `~/vms` — unlike the disk, it's only ever read client-side by `virt-install` (as `k8s`), -which then hands the rendered seed data to libvirtd over the API: +which then hands the rendered seed data to libvirtd over the API. Note +`$HOME` rather than `~` in the `--cloud-init` argument below — `~` only +expands at the start of a shell word, not after `=` inside one, so +`user-data=~/vms/...` would be passed to virt-install literally with the +tilde still in it. + +`--os-variant ubuntu22.04` below is deliberately not `ubuntu24.04` (the +actual image) — `osinfo-db` on Debian 12 may not recognize the newer +variant name yet. Check what yours knows about with +`osinfo-query os | grep -i ubuntu`; the hint mostly just tunes libvirt's +domain defaults (virtio devices, clock, etc.), so a close-enough variant +against a cloud-init `--import` like this is harmless: ```sh virt-install \ + --connect qemu:///system \ --name k3s-manual \ --memory 2048 \ --vcpus 2 \ --disk vol=default/k3s-manual.qcow2 \ --import \ - --os-variant ubuntu24.04 \ + --os-variant ubuntu22.04 \ --network network=default \ - --cloud-init user-data=~/vms/k3s-manual-user-data.yaml \ + --cloud-init user-data=$HOME/vms/k3s-manual-user-data.yaml \ --graphics none \ --noautoconsole ``` @@ -124,7 +161,7 @@ virt-install \ Give cloud-init ~2 minutes to finish installing k3s after the VM boots. ```sh -virsh domifaddr k3s-manual # note the IP under the default network +virsh -c qemu:///system domifaddr k3s-manual # note the IP under the default network ssh k3s@ sudo cat /etc/rancher/k3s/k3s.yaml \ | sed "s/127.0.0.1//" > ~/.kube/config-manual