docs: fix QUICKSTART - missing default pool/network, tilde-in-arg bug, os-variant

Debian's libvirt-daemon-system doesn't auto-create a default storage
pool (network exists but inactive) - added explicit pool-define/build/
start and net-start steps. Also fixed --cloud-init user-data=~/... not
expanding (tilde only expands at start of a shell word, not after =),
and noted why --os-variant may need to be an older release than the
actual image if osinfo-db is stale.
This commit is contained in:
CodeGit 2026-08-19 08:31:52 +01:00
parent 25eba8fd8b
commit a6123398bc

View file

@ -32,16 +32,40 @@ its own pool, rather than pointing at a raw path in `~` — pool operations
go through libvirtd's API, so it's libvirtd (already running with the
right privileges) that handles the file ownership, not `k8s` directly.
Make sure everything below targets the system instance — matches where
Terraform will eventually create the real VMs too:
Every `virsh`/`virt-install` command below uses `-c qemu:///system` /
`--connect qemu:///system` explicitly, rather than relying on the
`LIBVIRT_DEFAULT_URI` env var — for a non-root user that env var is what
picks system vs. the per-user `qemu:///session` instance, and it's easy for
it to be set in one terminal and not another, silently causing commands to
operate on the wrong (session) libvirt instance where nothing you created
system-side is visible. Explicit `-c` on every command avoids that
ambiguity entirely.
**Debian's `libvirt-daemon-system` does *not* auto-create a `default`
storage pool** the way some other distros' packaging does (it does
auto-create the `default` *network*, just left inactive) — check and fix
both before doing anything else:
```sh
export LIBVIRT_DEFAULT_URI=qemu:///system
virsh -c qemu:///system pool-list --all
virsh -c qemu:///system net-list --all
```
This only lasts for the current shell — add it to `k8s`'s `~/.bashrc` too
(`echo 'export LIBVIRT_DEFAULT_URI=qemu:///system' >> ~/.bashrc`) so it's
still set in step 2 and the teardown section later, even in a fresh login.
If `pool-list` comes back empty:
```sh
virsh -c qemu:///system pool-define-as default dir --target /var/lib/libvirt/images
virsh -c qemu:///system pool-build default
virsh -c qemu:///system pool-start default
virsh -c qemu:///system pool-autostart default
```
If `net-list` shows `default` as `inactive`:
```sh
virsh -c qemu:///system net-start default
virsh -c qemu:///system net-autostart default
```
Download the cloud image somewhere `k8s` can read it — this location
itself doesn't need to be qemu-readable, since it's only ever read by
@ -52,18 +76,19 @@ curl -L -o /tmp/noble-base.img \
https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img
```
Import it into libvirt's `default` storage pool (created automatically
when `libvirt-daemon-system` was installed in step 1), then create a
copy-on-write overlay on top of it, sized up to 20G:
Import it into the `default` pool, then create a copy-on-write overlay on
top of it, sized up to 20G:
```sh
virsh vol-create-as default k3s-manual-base.qcow2 \
virsh -c qemu:///system vol-create-as default k3s-manual-base.qcow2 \
--capacity "$(stat -c%s /tmp/noble-base.img)" --format qcow2
virsh vol-upload --pool default k3s-manual-base.qcow2 /tmp/noble-base.img --sparse
virsh -c qemu:///system vol-upload --pool default k3s-manual-base.qcow2 /tmp/noble-base.img --sparse
rm /tmp/noble-base.img
virsh vol-create-as default k3s-manual.qcow2 20G --format qcow2 \
virsh -c qemu:///system vol-create-as default k3s-manual.qcow2 20G --format qcow2 \
--backing-vol k3s-manual-base.qcow2 --backing-vol-format qcow2
virsh -c qemu:///system vol-list --pool default # should list both volumes
```
Write the cloud-init user-data — paste in the contents of
@ -103,18 +128,30 @@ usable cluster.
The cloud-init user-data file itself is fine living under `~/vms` — unlike
the disk, it's only ever read client-side by `virt-install` (as `k8s`),
which then hands the rendered seed data to libvirtd over the API:
which then hands the rendered seed data to libvirtd over the API. Note
`$HOME` rather than `~` in the `--cloud-init` argument below — `~` only
expands at the start of a shell word, not after `=` inside one, so
`user-data=~/vms/...` would be passed to virt-install literally with the
tilde still in it.
`--os-variant ubuntu22.04` below is deliberately not `ubuntu24.04` (the
actual image) — `osinfo-db` on Debian 12 may not recognize the newer
variant name yet. Check what yours knows about with
`osinfo-query os | grep -i ubuntu`; the hint mostly just tunes libvirt's
domain defaults (virtio devices, clock, etc.), so a close-enough variant
against a cloud-init `--import` like this is harmless:
```sh
virt-install \
--connect qemu:///system \
--name k3s-manual \
--memory 2048 \
--vcpus 2 \
--disk vol=default/k3s-manual.qcow2 \
--import \
--os-variant ubuntu24.04 \
--os-variant ubuntu22.04 \
--network network=default \
--cloud-init user-data=~/vms/k3s-manual-user-data.yaml \
--cloud-init user-data=$HOME/vms/k3s-manual-user-data.yaml \
--graphics none \
--noautoconsole
```
@ -124,7 +161,7 @@ virt-install \
Give cloud-init ~2 minutes to finish installing k3s after the VM boots.
```sh
virsh domifaddr k3s-manual # note the IP under the default network
virsh -c qemu:///system domifaddr k3s-manual # note the IP under the default network
ssh k3s@<VM_IP> sudo cat /etc/rancher/k3s/k3s.yaml \
| sed "s/127.0.0.1/<VM_IP>/" > ~/.kube/config-manual