docs: fix QUICKSTART - missing default pool/network, tilde-in-arg bug, os-variant
Debian's libvirt-daemon-system doesn't auto-create a default storage pool (network exists but inactive) - added explicit pool-define/build/ start and net-start steps. Also fixed --cloud-init user-data=~/... not expanding (tilde only expands at start of a shell word, not after =), and noted why --os-variant may need to be an older release than the actual image if osinfo-db is stale.
This commit is contained in:
parent
25eba8fd8b
commit
a6123398bc
1 changed files with 53 additions and 16 deletions
|
|
@ -32,16 +32,40 @@ its own pool, rather than pointing at a raw path in `~` — pool operations
|
||||||
go through libvirtd's API, so it's libvirtd (already running with the
|
go through libvirtd's API, so it's libvirtd (already running with the
|
||||||
right privileges) that handles the file ownership, not `k8s` directly.
|
right privileges) that handles the file ownership, not `k8s` directly.
|
||||||
|
|
||||||
Make sure everything below targets the system instance — matches where
|
Every `virsh`/`virt-install` command below uses `-c qemu:///system` /
|
||||||
Terraform will eventually create the real VMs too:
|
`--connect qemu:///system` explicitly, rather than relying on the
|
||||||
|
`LIBVIRT_DEFAULT_URI` env var — for a non-root user that env var is what
|
||||||
|
picks system vs. the per-user `qemu:///session` instance, and it's easy for
|
||||||
|
it to be set in one terminal and not another, silently causing commands to
|
||||||
|
operate on the wrong (session) libvirt instance where nothing you created
|
||||||
|
system-side is visible. Explicit `-c` on every command avoids that
|
||||||
|
ambiguity entirely.
|
||||||
|
|
||||||
|
**Debian's `libvirt-daemon-system` does *not* auto-create a `default`
|
||||||
|
storage pool** the way some other distros' packaging does (it does
|
||||||
|
auto-create the `default` *network*, just left inactive) — check and fix
|
||||||
|
both before doing anything else:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
export LIBVIRT_DEFAULT_URI=qemu:///system
|
virsh -c qemu:///system pool-list --all
|
||||||
|
virsh -c qemu:///system net-list --all
|
||||||
```
|
```
|
||||||
|
|
||||||
This only lasts for the current shell — add it to `k8s`'s `~/.bashrc` too
|
If `pool-list` comes back empty:
|
||||||
(`echo 'export LIBVIRT_DEFAULT_URI=qemu:///system' >> ~/.bashrc`) so it's
|
|
||||||
still set in step 2 and the teardown section later, even in a fresh login.
|
```sh
|
||||||
|
virsh -c qemu:///system pool-define-as default dir --target /var/lib/libvirt/images
|
||||||
|
virsh -c qemu:///system pool-build default
|
||||||
|
virsh -c qemu:///system pool-start default
|
||||||
|
virsh -c qemu:///system pool-autostart default
|
||||||
|
```
|
||||||
|
|
||||||
|
If `net-list` shows `default` as `inactive`:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
virsh -c qemu:///system net-start default
|
||||||
|
virsh -c qemu:///system net-autostart default
|
||||||
|
```
|
||||||
|
|
||||||
Download the cloud image somewhere `k8s` can read it — this location
|
Download the cloud image somewhere `k8s` can read it — this location
|
||||||
itself doesn't need to be qemu-readable, since it's only ever read by
|
itself doesn't need to be qemu-readable, since it's only ever read by
|
||||||
|
|
@ -52,18 +76,19 @@ curl -L -o /tmp/noble-base.img \
|
||||||
https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img
|
https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img
|
||||||
```
|
```
|
||||||
|
|
||||||
Import it into libvirt's `default` storage pool (created automatically
|
Import it into the `default` pool, then create a copy-on-write overlay on
|
||||||
when `libvirt-daemon-system` was installed in step 1), then create a
|
top of it, sized up to 20G:
|
||||||
copy-on-write overlay on top of it, sized up to 20G:
|
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
virsh vol-create-as default k3s-manual-base.qcow2 \
|
virsh -c qemu:///system vol-create-as default k3s-manual-base.qcow2 \
|
||||||
--capacity "$(stat -c%s /tmp/noble-base.img)" --format qcow2
|
--capacity "$(stat -c%s /tmp/noble-base.img)" --format qcow2
|
||||||
virsh vol-upload --pool default k3s-manual-base.qcow2 /tmp/noble-base.img --sparse
|
virsh -c qemu:///system vol-upload --pool default k3s-manual-base.qcow2 /tmp/noble-base.img --sparse
|
||||||
rm /tmp/noble-base.img
|
rm /tmp/noble-base.img
|
||||||
|
|
||||||
virsh vol-create-as default k3s-manual.qcow2 20G --format qcow2 \
|
virsh -c qemu:///system vol-create-as default k3s-manual.qcow2 20G --format qcow2 \
|
||||||
--backing-vol k3s-manual-base.qcow2 --backing-vol-format qcow2
|
--backing-vol k3s-manual-base.qcow2 --backing-vol-format qcow2
|
||||||
|
|
||||||
|
virsh -c qemu:///system vol-list --pool default # should list both volumes
|
||||||
```
|
```
|
||||||
|
|
||||||
Write the cloud-init user-data — paste in the contents of
|
Write the cloud-init user-data — paste in the contents of
|
||||||
|
|
@ -103,18 +128,30 @@ usable cluster.
|
||||||
|
|
||||||
The cloud-init user-data file itself is fine living under `~/vms` — unlike
|
The cloud-init user-data file itself is fine living under `~/vms` — unlike
|
||||||
the disk, it's only ever read client-side by `virt-install` (as `k8s`),
|
the disk, it's only ever read client-side by `virt-install` (as `k8s`),
|
||||||
which then hands the rendered seed data to libvirtd over the API:
|
which then hands the rendered seed data to libvirtd over the API. Note
|
||||||
|
`$HOME` rather than `~` in the `--cloud-init` argument below — `~` only
|
||||||
|
expands at the start of a shell word, not after `=` inside one, so
|
||||||
|
`user-data=~/vms/...` would be passed to virt-install literally with the
|
||||||
|
tilde still in it.
|
||||||
|
|
||||||
|
`--os-variant ubuntu22.04` below is deliberately not `ubuntu24.04` (the
|
||||||
|
actual image) — `osinfo-db` on Debian 12 may not recognize the newer
|
||||||
|
variant name yet. Check what yours knows about with
|
||||||
|
`osinfo-query os | grep -i ubuntu`; the hint mostly just tunes libvirt's
|
||||||
|
domain defaults (virtio devices, clock, etc.), so a close-enough variant
|
||||||
|
against a cloud-init `--import` like this is harmless:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
virt-install \
|
virt-install \
|
||||||
|
--connect qemu:///system \
|
||||||
--name k3s-manual \
|
--name k3s-manual \
|
||||||
--memory 2048 \
|
--memory 2048 \
|
||||||
--vcpus 2 \
|
--vcpus 2 \
|
||||||
--disk vol=default/k3s-manual.qcow2 \
|
--disk vol=default/k3s-manual.qcow2 \
|
||||||
--import \
|
--import \
|
||||||
--os-variant ubuntu24.04 \
|
--os-variant ubuntu22.04 \
|
||||||
--network network=default \
|
--network network=default \
|
||||||
--cloud-init user-data=~/vms/k3s-manual-user-data.yaml \
|
--cloud-init user-data=$HOME/vms/k3s-manual-user-data.yaml \
|
||||||
--graphics none \
|
--graphics none \
|
||||||
--noautoconsole
|
--noautoconsole
|
||||||
```
|
```
|
||||||
|
|
@ -124,7 +161,7 @@ virt-install \
|
||||||
Give cloud-init ~2 minutes to finish installing k3s after the VM boots.
|
Give cloud-init ~2 minutes to finish installing k3s after the VM boots.
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
virsh domifaddr k3s-manual # note the IP under the default network
|
virsh -c qemu:///system domifaddr k3s-manual # note the IP under the default network
|
||||||
|
|
||||||
ssh k3s@<VM_IP> sudo cat /etc/rancher/k3s/k3s.yaml \
|
ssh k3s@<VM_IP> sudo cat /etc/rancher/k3s/k3s.yaml \
|
||||||
| sed "s/127.0.0.1/<VM_IP>/" > ~/.kube/config-manual
|
| sed "s/127.0.0.1/<VM_IP>/" > ~/.kube/config-manual
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue