docs: fix QUICKSTART - missing default pool/network, tilde-in-arg bug, os-variant

Debian's libvirt-daemon-system doesn't auto-create a default storage
pool (network exists but inactive) - added explicit pool-define/build/
start and net-start steps. Also fixed --cloud-init user-data=~/... not
expanding (tilde only expands at start of a shell word, not after =),
and noted why --os-variant may need to be an older release than the
actual image if osinfo-db is stale.
This commit is contained in:
CodeGit 2026-08-19 08:31:52 +01:00
parent 25eba8fd8b
commit a6123398bc

View file

@ -32,16 +32,40 @@ its own pool, rather than pointing at a raw path in `~` — pool operations
go through libvirtd's API, so it's libvirtd (already running with the go through libvirtd's API, so it's libvirtd (already running with the
right privileges) that handles the file ownership, not `k8s` directly. right privileges) that handles the file ownership, not `k8s` directly.
Make sure everything below targets the system instance — matches where Every `virsh`/`virt-install` command below uses `-c qemu:///system` /
Terraform will eventually create the real VMs too: `--connect qemu:///system` explicitly, rather than relying on the
`LIBVIRT_DEFAULT_URI` env var — for a non-root user that env var is what
picks system vs. the per-user `qemu:///session` instance, and it's easy for
it to be set in one terminal and not another, silently causing commands to
operate on the wrong (session) libvirt instance where nothing you created
system-side is visible. Explicit `-c` on every command avoids that
ambiguity entirely.
**Debian's `libvirt-daemon-system` does *not* auto-create a `default`
storage pool** the way some other distros' packaging does (it does
auto-create the `default` *network*, just left inactive) — check and fix
both before doing anything else:
```sh ```sh
export LIBVIRT_DEFAULT_URI=qemu:///system virsh -c qemu:///system pool-list --all
virsh -c qemu:///system net-list --all
``` ```
This only lasts for the current shell — add it to `k8s`'s `~/.bashrc` too If `pool-list` comes back empty:
(`echo 'export LIBVIRT_DEFAULT_URI=qemu:///system' >> ~/.bashrc`) so it's
still set in step 2 and the teardown section later, even in a fresh login. ```sh
virsh -c qemu:///system pool-define-as default dir --target /var/lib/libvirt/images
virsh -c qemu:///system pool-build default
virsh -c qemu:///system pool-start default
virsh -c qemu:///system pool-autostart default
```
If `net-list` shows `default` as `inactive`:
```sh
virsh -c qemu:///system net-start default
virsh -c qemu:///system net-autostart default
```
Download the cloud image somewhere `k8s` can read it — this location Download the cloud image somewhere `k8s` can read it — this location
itself doesn't need to be qemu-readable, since it's only ever read by itself doesn't need to be qemu-readable, since it's only ever read by
@ -52,18 +76,19 @@ curl -L -o /tmp/noble-base.img \
https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img
``` ```
Import it into libvirt's `default` storage pool (created automatically Import it into the `default` pool, then create a copy-on-write overlay on
when `libvirt-daemon-system` was installed in step 1), then create a top of it, sized up to 20G:
copy-on-write overlay on top of it, sized up to 20G:
```sh ```sh
virsh vol-create-as default k3s-manual-base.qcow2 \ virsh -c qemu:///system vol-create-as default k3s-manual-base.qcow2 \
--capacity "$(stat -c%s /tmp/noble-base.img)" --format qcow2 --capacity "$(stat -c%s /tmp/noble-base.img)" --format qcow2
virsh vol-upload --pool default k3s-manual-base.qcow2 /tmp/noble-base.img --sparse virsh -c qemu:///system vol-upload --pool default k3s-manual-base.qcow2 /tmp/noble-base.img --sparse
rm /tmp/noble-base.img rm /tmp/noble-base.img
virsh vol-create-as default k3s-manual.qcow2 20G --format qcow2 \ virsh -c qemu:///system vol-create-as default k3s-manual.qcow2 20G --format qcow2 \
--backing-vol k3s-manual-base.qcow2 --backing-vol-format qcow2 --backing-vol k3s-manual-base.qcow2 --backing-vol-format qcow2
virsh -c qemu:///system vol-list --pool default # should list both volumes
``` ```
Write the cloud-init user-data — paste in the contents of Write the cloud-init user-data — paste in the contents of
@ -103,18 +128,30 @@ usable cluster.
The cloud-init user-data file itself is fine living under `~/vms` — unlike The cloud-init user-data file itself is fine living under `~/vms` — unlike
the disk, it's only ever read client-side by `virt-install` (as `k8s`), the disk, it's only ever read client-side by `virt-install` (as `k8s`),
which then hands the rendered seed data to libvirtd over the API: which then hands the rendered seed data to libvirtd over the API. Note
`$HOME` rather than `~` in the `--cloud-init` argument below — `~` only
expands at the start of a shell word, not after `=` inside one, so
`user-data=~/vms/...` would be passed to virt-install literally with the
tilde still in it.
`--os-variant ubuntu22.04` below is deliberately not `ubuntu24.04` (the
actual image) — `osinfo-db` on Debian 12 may not recognize the newer
variant name yet. Check what yours knows about with
`osinfo-query os | grep -i ubuntu`; the hint mostly just tunes libvirt's
domain defaults (virtio devices, clock, etc.), so a close-enough variant
against a cloud-init `--import` like this is harmless:
```sh ```sh
virt-install \ virt-install \
--connect qemu:///system \
--name k3s-manual \ --name k3s-manual \
--memory 2048 \ --memory 2048 \
--vcpus 2 \ --vcpus 2 \
--disk vol=default/k3s-manual.qcow2 \ --disk vol=default/k3s-manual.qcow2 \
--import \ --import \
--os-variant ubuntu24.04 \ --os-variant ubuntu22.04 \
--network network=default \ --network network=default \
--cloud-init user-data=~/vms/k3s-manual-user-data.yaml \ --cloud-init user-data=$HOME/vms/k3s-manual-user-data.yaml \
--graphics none \ --graphics none \
--noautoconsole --noautoconsole
``` ```
@ -124,7 +161,7 @@ virt-install \
Give cloud-init ~2 minutes to finish installing k3s after the VM boots. Give cloud-init ~2 minutes to finish installing k3s after the VM boots.
```sh ```sh
virsh domifaddr k3s-manual # note the IP under the default network virsh -c qemu:///system domifaddr k3s-manual # note the IP under the default network
ssh k3s@<VM_IP> sudo cat /etc/rancher/k3s/k3s.yaml \ ssh k3s@<VM_IP> sudo cat /etc/rancher/k3s/k3s.yaml \
| sed "s/127.0.0.1/<VM_IP>/" > ~/.kube/config-manual | sed "s/127.0.0.1/<VM_IP>/" > ~/.kube/config-manual