docs: fix QUICKSTART manual VM storage for system libvirt
Disk images under k8s's home directory aren't readable by the libvirt-qemu user that actually runs VMs under qemu:///system. Use libvirt's own default storage pool (vol-create-as/vol-upload) instead, so libvirtd handles file ownership rather than k8s directly, and pin LIBVIRT_DEFAULT_URI=qemu:///system throughout.
This commit is contained in:
parent
a3cfba3d6a
commit
440cb2077f
1 changed files with 53 additions and 8 deletions
|
|
@ -21,13 +21,49 @@ Everything below runs as `k8s` on the T630 (`sudo -iu k8s`).
|
|||
|
||||
## 1. Create the VM
|
||||
|
||||
```sh
|
||||
mkdir -p ~/vms
|
||||
curl -L -o ~/vms/noble-base.img \
|
||||
https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img
|
||||
No sudo needed anywhere in this guide — `k8s`'s `libvirt`/`kvm` group
|
||||
membership (step 2) is what authorizes talking to libvirt at all, via
|
||||
polkit. But group membership alone doesn't solve everything: system libvirt
|
||||
runs actual VMs as a separate, restricted `libvirt-qemu` user, not as
|
||||
`k8s` — so a disk image sitting under `k8s`'s home directory (mode 700 by
|
||||
default) would fail at boot, since that user can't read into `k8s`'s home
|
||||
at all. The fix is to let libvirt manage the disk storage itself, inside
|
||||
its own pool, rather than pointing at a raw path in `~` — pool operations
|
||||
go through libvirtd's API, so it's libvirtd (already running with the
|
||||
right privileges) that handles the file ownership, not `k8s` directly.
|
||||
|
||||
# a writable overlay on top of the base image, sized up to 20G
|
||||
qemu-img create -f qcow2 -F qcow2 -b ~/vms/noble-base.img ~/vms/k3s-manual.qcow2 20G
|
||||
Make sure everything below targets the system instance — matches where
|
||||
Terraform will eventually create the real VMs too:
|
||||
|
||||
```sh
|
||||
export LIBVIRT_DEFAULT_URI=qemu:///system
|
||||
```
|
||||
|
||||
This only lasts for the current shell — add it to `k8s`'s `~/.bashrc` too
|
||||
(`echo 'export LIBVIRT_DEFAULT_URI=qemu:///system' >> ~/.bashrc`) so it's
|
||||
still set in step 2 and the teardown section later, even in a fresh login.
|
||||
|
||||
Download the cloud image somewhere `k8s` can read it — this location
|
||||
itself doesn't need to be qemu-readable, since it's only ever read by
|
||||
`virsh` (running as `k8s`), never directly by the VM:
|
||||
|
||||
```sh
|
||||
curl -L -o /tmp/noble-base.img \
|
||||
https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img
|
||||
```
|
||||
|
||||
Import it into libvirt's `default` storage pool (created automatically
|
||||
when `libvirt-daemon-system` was installed in step 1), then create a
|
||||
copy-on-write overlay on top of it, sized up to 20G:
|
||||
|
||||
```sh
|
||||
virsh vol-create-as default k3s-manual-base.qcow2 \
|
||||
--capacity "$(stat -c%s /tmp/noble-base.img)" --format qcow2
|
||||
virsh vol-upload --pool default k3s-manual-base.qcow2 /tmp/noble-base.img --sparse
|
||||
rm /tmp/noble-base.img
|
||||
|
||||
virsh vol-create-as default k3s-manual.qcow2 20G --format qcow2 \
|
||||
--backing-vol k3s-manual-base.qcow2 --backing-vol-format qcow2
|
||||
```
|
||||
|
||||
Write the cloud-init user-data — paste in the contents of
|
||||
|
|
@ -64,12 +100,16 @@ network (uses libvirt's default NAT network + DHCP). A k3s server node
|
|||
schedules workloads on itself by default, so this alone is a complete,
|
||||
usable cluster.
|
||||
|
||||
The cloud-init user-data file itself is fine living under `~/vms` — unlike
|
||||
the disk, it's only ever read client-side by `virt-install` (as `k8s`),
|
||||
which then hands the rendered seed data to libvirtd over the API:
|
||||
|
||||
```sh
|
||||
virt-install \
|
||||
--name k3s-manual \
|
||||
--memory 2048 \
|
||||
--vcpus 2 \
|
||||
--disk ~/vms/k3s-manual.qcow2 \
|
||||
--disk vol=default/k3s-manual.qcow2 \
|
||||
--import \
|
||||
--os-variant ubuntu24.04 \
|
||||
--network network=default \
|
||||
|
|
@ -123,8 +163,13 @@ the real 3-node cluster, remove this one:
|
|||
|
||||
```sh
|
||||
# as k8s
|
||||
export LIBVIRT_DEFAULT_URI=qemu:///system
|
||||
virsh destroy k3s-manual # stop it
|
||||
virsh undefine k3s-manual --remove-all-storage # delete VM + its disk
|
||||
virsh undefine k3s-manual --remove-all-storage # delete VM + its overlay disk
|
||||
virsh vol-delete --pool default k3s-manual-base.qcow2 # the base image isn't
|
||||
# attached to the VM
|
||||
# directly, so it
|
||||
# needs its own delete
|
||||
rm ~/.kube/config-manual
|
||||
rm -rf ~/vms
|
||||
```
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue