# Quickstart: manual cluster (no Terraform) Gets a real k3s cluster running today, by hand, so you can learn/practice Flux and GitOps immediately instead of waiting on the Terraform/libvirt provider work in `docs/SETUP.md` to be sorted out. One throwaway VM, created directly with `virt-install` rather than the `dmacvicar/libvirt` Terraform provider. When the Terraform track is ready, tear this down (last section) and provision the "real" 3-node cluster properly instead. Shares steps 1–4 of `docs/SETUP.md` as prerequisites — do those first if you haven't: - Step 1: KVM/libvirt installed on the T630. - Step 2: the unprivileged `k8s` user exists, with `~/.ssh/k3s_homelab` generated. - Step 3: `openssl rand -hex 32` isn't needed here (no agents joining, so no cluster token) — skip it. - Step 4: `cloud-demo` pushed to Forgejo, with the `flux-write` token generated. Everything below runs as `k8s` on the T630 (`sudo -iu k8s`). ## 1. Create the VM No sudo needed anywhere in this guide — `k8s`'s `libvirt`/`kvm` group membership (step 2) is what authorizes talking to libvirt at all, via polkit. But group membership alone doesn't solve everything: system libvirt runs actual VMs as a separate, restricted `libvirt-qemu` user, not as `k8s` — so a disk image sitting under `k8s`'s home directory (mode 700 by default) would fail at boot, since that user can't read into `k8s`'s home at all. The fix is to let libvirt manage the disk storage itself, inside its own pool, rather than pointing at a raw path in `~` — pool operations go through libvirtd's API, so it's libvirtd (already running with the right privileges) that handles the file ownership, not `k8s` directly. Make sure everything below targets the system instance — matches where Terraform will eventually create the real VMs too: ```sh export LIBVIRT_DEFAULT_URI=qemu:///system ``` This only lasts for the current shell — add it to `k8s`'s `~/.bashrc` too (`echo 'export LIBVIRT_DEFAULT_URI=qemu:///system' >> ~/.bashrc`) so it's still set in step 2 and the teardown section later, even in a fresh login. Download the cloud image somewhere `k8s` can read it — this location itself doesn't need to be qemu-readable, since it's only ever read by `virsh` (running as `k8s`), never directly by the VM: ```sh curl -L -o /tmp/noble-base.img \ https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img ``` Import it into libvirt's `default` storage pool (created automatically when `libvirt-daemon-system` was installed in step 1), then create a copy-on-write overlay on top of it, sized up to 20G: ```sh virsh vol-create-as default k3s-manual-base.qcow2 \ --capacity "$(stat -c%s /tmp/noble-base.img)" --format qcow2 virsh vol-upload --pool default k3s-manual-base.qcow2 /tmp/noble-base.img --sparse rm /tmp/noble-base.img virsh vol-create-as default k3s-manual.qcow2 20G --format qcow2 \ --backing-vol k3s-manual-base.qcow2 --backing-vol-format qcow2 ``` Write the cloud-init user-data — paste in the contents of `~/.ssh/k3s_homelab.pub` where marked: ```sh cat > ~/vms/k3s-manual-user-data.yaml <<'EOF' #cloud-config hostname: k3s-manual manage_etc_hosts: true users: - name: k3s groups: sudo shell: /bin/bash sudo: ALL=(ALL) NOPASSWD:ALL ssh_authorized_keys: - PASTE ~/.ssh/k3s_homelab.pub CONTENTS HERE package_update: true packages: - curl runcmd: - curl -sfL https://get.k3s.io | sh -s - server - mkdir -p /home/k3s/.kube - k3s kubectl config view --raw > /home/k3s/.kube/config - chown -R k3s:k3s /home/k3s/.kube EOF ``` This is a single, standalone server — no join token, no agents, no custom network (uses libvirt's default NAT network + DHCP). A k3s server node schedules workloads on itself by default, so this alone is a complete, usable cluster. The cloud-init user-data file itself is fine living under `~/vms` — unlike the disk, it's only ever read client-side by `virt-install` (as `k8s`), which then hands the rendered seed data to libvirtd over the API: ```sh virt-install \ --name k3s-manual \ --memory 2048 \ --vcpus 2 \ --disk vol=default/k3s-manual.qcow2 \ --import \ --os-variant ubuntu24.04 \ --network network=default \ --cloud-init user-data=~/vms/k3s-manual-user-data.yaml \ --graphics none \ --noautoconsole ``` ## 2. Find its IP and get kubectl talking to it Give cloud-init ~2 minutes to finish installing k3s after the VM boots. ```sh virsh domifaddr k3s-manual # note the IP under the default network ssh k3s@ sudo cat /etc/rancher/k3s/k3s.yaml \ | sed "s/127.0.0.1//" > ~/.kube/config-manual export KUBECONFIG=~/.kube/config-manual kubectl get nodes # expect 1 Ready node ``` ## 3. Bootstrap Flux against Forgejo Same repo, same token, same target path as the full guide would use — nothing about this is cluster-specific: ```sh flux bootstrap git \ --url=https://git.boglabob.com/codegit/cloud-demo \ --branch=main \ --path=clusters/homelab \ --username=codegit \ --password= \ --token-auth \ --kubeconfig ~/.kube/config-manual ``` ## 4. Continue with the shared steps From here, `docs/SETUP.md` steps 8 onward apply exactly as written, regardless of how the cluster was created — verifying the podinfo GitOps loop, registering the Forgejo Actions runner, exposing apps through Caddy, the Dashboard, remote kubectl access. Just use `~/.kube/config-manual` as the kubeconfig throughout instead of the Terraform-provisioned one. ## Tearing this down Once the Terraform track in `docs/SETUP.md` is ready and you `tofu apply` the real 3-node cluster, remove this one: ```sh # as k8s export LIBVIRT_DEFAULT_URI=qemu:///system virsh destroy k3s-manual # stop it virsh undefine k3s-manual --remove-all-storage # delete VM + its overlay disk virsh vol-delete --pool default k3s-manual-base.qcow2 # the base image isn't # attached to the VM # directly, so it # needs its own delete rm ~/.kube/config-manual rm -rf ~/vms ``` Nothing else needs cleaning up — Flux's state lived entirely inside that VM's cluster and goes away with it. The Forgejo repo, deploy tokens, and the Forgejo Actions runner registration are all cluster-independent and carry over to the real cluster unchanged; just re-run `flux bootstrap` against its kubeconfig once it's up.