diff --git a/docs/QUICKSTART.md b/docs/QUICKSTART.md index ad81923..3292f7f 100644 --- a/docs/QUICKSTART.md +++ b/docs/QUICKSTART.md @@ -1,98 +1,209 @@ # Quickstart: manual cluster (no Terraform) -Gets a real k3s cluster running today, by hand, so you can learn/practice -Flux and GitOps immediately instead of waiting on the Terraform/libvirt -provider work in `docs/SETUP.md` to be sorted out. One throwaway VM, created -directly with `virt-install` rather than the `dmacvicar/libvirt` Terraform -provider. When the Terraform track is ready, tear this down (last section) -and provision the "real" 3-node cluster properly instead. +A real k3s cluster, built by hand, so Flux/GitOps can be learned right away +instead of waiting on the Terraform/libvirt provider work in `docs/SETUP.md` +to get sorted (that provider did a breaking rewrite between 0.8.x and +0.9.x, and the HCL needs writing against the real schema, not memory). -Shares steps 1–4 of `docs/SETUP.md` as prerequisites — do those first if you -haven't: -- Step 1: KVM/libvirt installed on the T630. -- Step 2: the unprivileged `k8s` user exists, with `~/.ssh/id_ed25519` - generated. -- Step 3: `openssl rand -hex 32` isn't needed here (no agents joining, so no - cluster token) — skip it. -- Step 4: `cloud-demo` pushed to Forgejo, with the `flux-write` token - generated. +This is one throwaway VM, created directly with `virt-install` — no +Terraform involved at all. Everything here explains the *why*, not just +the *what*: read each section before running its commands, and where a +config file is being built (the cloud-init data, the Caddy block), treat +the snippets as pieces to assemble into your own file, not something to +paste wholesale. + +Shares steps 1–4 of `docs/SETUP.md` as prerequisites: +- Step 1: KVM/libvirt packages installed on the T630. +- Step 2: the unprivileged `k8s` user exists, in the `libvirt`/`kvm` groups, + with an SSH keypair at `~/.ssh/id_ed25519`. +- Step 3: skip — no join token needed here (see "one node is enough" below). +- Step 4: `cloud-demo` pushed to Forgejo, with a `k8s-readonly` and a + `flux-write` token generated. Everything below runs as `k8s` on the T630 (`sudo -iu k8s`). -## 1. Create the VM +--- -No sudo needed anywhere in this guide — `k8s`'s `libvirt`/`kvm` group -membership (step 2) is what authorizes talking to libvirt at all, via -polkit. But group membership alone doesn't solve everything: system libvirt -runs actual VMs as a separate, restricted `libvirt-qemu` user, not as -`k8s` — so a disk image sitting under `k8s`'s home directory (mode 700 by -default) would fail at boot, since that user can't read into `k8s`'s home -at all. The fix is to let libvirt manage the disk storage itself, inside -its own pool, rather than pointing at a raw path in `~` — pool operations -go through libvirtd's API, so it's libvirtd (already running with the -right privileges) that handles the file ownership, not `k8s` directly. +## Why one node is enough -Every `virsh`/`virt-install` command below uses `-c qemu:///system` / -`--connect qemu:///system` explicitly, rather than relying on the -`LIBVIRT_DEFAULT_URI` env var — for a non-root user that env var is what -picks system vs. the per-user `qemu:///session` instance, and it's easy for -it to be set in one terminal and not another, silently causing commands to -operate on the wrong (session) libvirt instance where nothing you created -system-side is visible. Explicit `-c` on every command avoids that -ambiguity entirely. +A k3s **server** node runs the control plane (API server, scheduler, +etcd/SQLite) *and* schedules ordinary workloads onto itself unless you +explicitly disable that. So a single server, with no agents, is already a +complete, working cluster — nothing here needs a join token or a second +VM. The 3-node design in `docs/SETUP.md` exists to mirror a more +realistic multi-node cluster once Terraform can build it repeatedly and +disposably; for learning Flux and GitOps, that extra shape doesn't buy +you anything yet. -**Debian's `libvirt-daemon-system` does *not* auto-create a `default` -storage pool** the way some other distros' packaging does (it does -auto-create the `default` *network*, just left inactive) — check and fix -both before doing anything else: +## Two libvirt connections, and why it matters + +libvirt isn't one daemon with one namespace of VMs — from a client's +perspective there are (at least) two separate connections: + +- `qemu:///system` — the shared, host-wide instance. VMs here can use + privileged networking (bridges, NAT with DHCP), and management access is + gated by group membership (`libvirt`/`kvm`) checked via polkit — which is + exactly what step 2's `usermod -aG libvirt,kvm k8s` set up. No sudo + needed for any command below; that group membership *is* the + authorization. +- `qemu:///session` — a private, per-user instance with no special + privileges, and critically, its own separate storage pools and networks + that don't overlap with the system instance at all. + +For a non-root user, `virsh`/`virt-install` **default to `session`** unless +told otherwise. This matters a lot in practice: it's easy to set +`LIBVIRT_DEFAULT_URI=qemu:///system` in one terminal, run a command in a +different terminal where it isn't set, and have that command silently +create something under `session` instead — where it's invisible to +everything else you're doing. Every command below uses `-c qemu:///system` +/ `--connect qemu:///system` explicitly for exactly this reason, rather +than relying on the environment variable. + +## Storage: why a raw path in `~` doesn't work + +The most natural first instinct is to put a VM's disk file somewhere in +`k8s`'s home directory and point `--disk` at it directly. That fails +non-obviously: under `qemu:///system`, the actual QEMU process backing a +VM doesn't run as `k8s` — it runs as a separate, restricted `libvirt-qemu` +user (a deliberate security boundary, so a compromised VM process has its +own limited identity rather than the identity of whoever created it). +`k8s`'s home directory defaults to mode `700` — readable only by `k8s` — +so `libvirt-qemu` can't read into it at all, and the VM fails at boot. + +The fix is to let libvirt manage the storage itself, in a **pool**. A pool +is just a named, libvirt-tracked location for disk images (a directory, in +the simplest case). The key property: creating or writing a volume inside +a pool goes through libvirtd's API, not through `k8s`'s own filesystem +permissions — so it's libvirtd (already running with the right privileges) +that handles ownership correctly, regardless of what user asked for it. + +Check what pools/networks already exist: ```sh virsh -c qemu:///system pool-list --all virsh -c qemu:///system net-list --all ``` -If `pool-list` comes back empty: +On this box, `net-list` showed a `default` network already defined (just +inactive), but `pool-list` came back completely empty — **Debian's +`libvirt-daemon-system` package does not auto-create a `default` storage +pool**, unlike some other distros' packaging. Both need fixing before +anything else: ```sh +# only if pool-list was empty virsh -c qemu:///system pool-define-as default dir --target /var/lib/libvirt/images virsh -c qemu:///system pool-build default virsh -c qemu:///system pool-start default virsh -c qemu:///system pool-autostart default -``` -If `net-list` shows `default` as `inactive`: - -```sh +# only if net-list showed 'default' as inactive virsh -c qemu:///system net-start default virsh -c qemu:///system net-autostart default ``` -Download the cloud image somewhere `k8s` can read it — this location -itself doesn't need to be qemu-readable, since it's only ever read by -`virsh` (running as `k8s`), never directly by the VM: +`pool-autostart`/`net-autostart` mean both come back up automatically after +a host reboot — without it, they'd need manually starting again every time. + +## Building the base image and the VM's own disk + +Downloading the cloud image doesn't need to go through the pool — this +copy is only ever read by `virsh` itself (running as `k8s`), never +directly by the VM, so an ordinary temp location is fine: ```sh curl -L -o /tmp/noble-base.img \ https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img ``` -Import it into the `default` pool, then create a copy-on-write overlay on -top of it, sized up to 20G: +Getting it *into* the pool, though, does need to go through libvirt's API +— `vol-create-as` allocates an empty volume of a given size inside the +pool, and `vol-upload` streams a local file's bytes into it: ```sh virsh -c qemu:///system vol-create-as default k3s-manual-base.qcow2 \ --capacity "$(stat -c%s /tmp/noble-base.img)" --format qcow2 virsh -c qemu:///system vol-upload --pool default k3s-manual-base.qcow2 /tmp/noble-base.img --sparse rm /tmp/noble-base.img +``` +The VM itself shouldn't boot directly off this base image — if it did, +every write the OS makes (logs, package installs, k3s's own state) would +permanently modify the one shared base file, corrupting it for any future +VM built from the same base. Instead, create a **copy-on-write overlay**: +a second volume that starts out empty and only stores the *differences* +from its backing volume. Reads that haven't been changed transparently +fall through to the base image; writes go into the overlay. This is the +same relationship a Docker image layer has to its base layer. + +```sh virsh -c qemu:///system vol-create-as default k3s-manual.qcow2 20G --format qcow2 \ --backing-vol k3s-manual-base.qcow2 --backing-vol-format qcow2 -virsh -c qemu:///system vol-list --pool default # should list both volumes +virsh -c qemu:///system vol-list --pool default # should list both volumes now ``` -Write the cloud-init user-data — paste in the contents of -`~/.ssh/id_ed25519.pub` where marked: +## cloud-init: how a stock image becomes *this* VM + +The base image is a generic Ubuntu install — it has no idea it's about to +become a k3s node, and has no user account you could log into. **cloud-init** +is the standard mechanism cloud images use to configure themselves on +first boot, driven by data supplied externally rather than baked into the +image. `virt-install`'s `--cloud-init` flag builds a small ISO (the +"NoCloud" datasource) containing that data and attaches it to the VM; +cloud-init, already installed in the image, detects it automatically at +boot. + +Two separate pieces of data go in, and they answer different questions: + +- **`user-data`** — *what should exist on this machine*: users, packages, + commands to run. Written as `#cloud-config` YAML. +- **`network-config`** — *how should this machine's network be set up*. + This quickstart skips it entirely and lets the VM get an address via + DHCP from the `default` network instead (the Terraform track, by + contrast, uses this for static IPs, since it manages its own isolated + network). + +Build `user-data` up piece by piece. Start with identity: + +```yaml +#cloud-config +hostname: k3s-manual +manage_etc_hosts: true +``` + +Then the one thing you actually need to log in and administer this box — +a user, with your public key rather than a password (cloud images have no +default password, and SSH password auth is normally disabled anyway), and +passwordless sudo so you're not stuck typing a password you never set: + +```yaml +users: + - name: k3s + groups: sudo + shell: /bin/bash + sudo: ALL=(ALL) NOPASSWD:ALL + ssh_authorized_keys: + - +``` + +Then the actual payload — install k3s, and stage a copy of its +auto-generated kubeconfig somewhere the `k3s` user can read (by default +it's only readable by root): + +```yaml +package_update: true +packages: + - curl + +runcmd: + - curl -sfL https://get.k3s.io | sh -s - server + - mkdir -p /home/k3s/.kube + - k3s kubectl config view --raw > /home/k3s/.kube/config + - chown -R k3s:k3s /home/k3s/.kube +``` + +Assemble those three pieces into one file: ```sh mkdir -p ~/vms @@ -121,25 +232,41 @@ runcmd: EOF ``` -This is a single, standalone server — no join token, no agents, no custom -network (uses libvirt's default NAT network + DHCP). A k3s server node -schedules workloads on itself by default, so this alone is a complete, -usable cluster. +Unlike the disk, this file is fine sitting under `~/vms` — it's only ever +read client-side by `virt-install` (as `k8s`), which hands the resulting +seed data to libvirtd over the API; `libvirt-qemu` never touches it +directly. -The cloud-init user-data file itself is fine living under `~/vms` — unlike -the disk, it's only ever read client-side by `virt-install` (as `k8s`), -which then hands the rendered seed data to libvirtd over the API. Note -`$HOME` rather than `~` in the `--cloud-init` argument below — `~` only -expands at the start of a shell word, not after `=` inside one, so -`user-data=~/vms/...` would be passed to virt-install literally with the -tilde still in it. +## Creating the VM -`--os-variant ubuntu22.04` below is deliberately not `ubuntu24.04` (the -actual image) — `osinfo-db` on Debian 12 may not recognize the newer -variant name yet. Check what yours knows about with -`osinfo-query os | grep -i ubuntu`; the hint mostly just tunes libvirt's -domain defaults (virtio devices, clock, etc.), so a close-enough variant -against a cloud-init `--import` like this is harmless: +Each `virt-install` flag is answering a specific question: + +| Flag | Answers | +|---|---| +| `--connect qemu:///system` | which libvirt instance (see above) | +| `--name` | the domain's name, used everywhere else (`virsh`, `domifaddr`, teardown) | +| `--memory` / `--vcpus` | resource allocation — kept small deliberately | +| `--disk vol=default/k3s-manual.qcow2` | use the pool-managed overlay, not a raw path | +| `--import` | boot the disk as-is rather than running an OS installer against it | +| `--os-variant` | a hint for libvirt's own defaults (virtio devices, clock behavior) — not what OS actually gets installed | +| `--network network=default` | attach to the NAT network from earlier | +| `--cloud-init user-data=...` | the file just built | +| `--graphics none` | no VNC/spice display — this is a headless server VM | +| `--noautoconsole` | don't attach to its console interactively after creation | + +Two gotchas worth knowing before running this: + +- **`$HOME`, not `~`, in the `--cloud-init` argument.** Bash only expands + `~` at the very start of a word; `user-data=~/vms/...` is *inside* a + word (after `=`), so the tilde would be passed through literally and + `virt-install` would fail looking for a file called `~`. `$HOME` + expands regardless of position. +- **`--os-variant` may need to be an older release than the actual image.** + `osinfo-db` (the database `virt-install` validates this against) can lag + behind real Ubuntu releases — if `ubuntu24.04` comes back "unknown", check + what's actually available with `osinfo-query os | grep -i ubuntu` and use + the newest one it recognizes. Since `--import` just boots the disk as-is, + this hint doesn't change what's actually installed. ```sh virt-install \ @@ -156,16 +283,50 @@ virt-install \ --noautoconsole ``` -## 2. Find its IP and get kubectl talking to it - -Give cloud-init ~2 minutes to finish installing k3s after the VM boots. +## Finding the VM and connecting ```sh -virsh -c qemu:///system domifaddr k3s-manual # note the IP under the default network +virsh -c qemu:///system domifaddr k3s-manual ``` -If you haven't installed `kubectl` on the T630 yet (as `k8s`, no sudo -needed): +This prints something like `192.168.122.67/24` — the `/24` is CIDR +notation for the subnet mask (`255.255.255.0`), describing the *network* +this address belongs to, not part of the address itself. Use just the +plain IP (`192.168.122.67`) to actually connect. + +Give cloud-init a couple of minutes after `Domain creation completed` +before it's reachable — it's installing k3s in the background. + +This address comes from the `default` network's DHCP server, leased +against the VM's MAC address (shown in the same `domifaddr` output). DHCP +leases are "sticky" in practice — the VM will keep asking for and getting +the same address on renewal — but that's not the same as *guaranteed* +fixed. Pin it explicitly if you don't want to risk it changing later: + +```sh +virsh -c qemu:///system net-update default add ip-dhcp-host \ + "" --live --config +``` + +## kubectl: what's actually in a kubeconfig + +A kubeconfig isn't a password — it's mutual TLS: a cluster CA certificate +(so your client trusts the API server's identity) plus a client +certificate and private key (so the API server trusts yours). k3s +generates one for itself pointed at `127.0.0.1`, which only works from +inside the VM — the `sed` below swaps that for the VM's real address so +it works from the T630 instead: + +```sh +mkdir -p ~/.kube +ssh k3s@ sudo cat /etc/rancher/k3s/k3s.yaml \ + | sed "s/127.0.0.1//" > ~/.kube/config-manual +``` + +`kubectl` itself isn't installed anywhere yet. Rather than a system +package (which would need `sudo`, which `k8s` doesn't have), grab the +official binary release straight from Kubernetes' own distribution point +and drop it somewhere already on `k8s`'s `PATH`: ```sh curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl" @@ -174,20 +335,24 @@ mkdir -p ~/.local/bin mv kubectl ~/.local/bin/ echo 'export PATH="$HOME/.local/bin:$PATH"' >> ~/.bashrc export PATH="$HOME/.local/bin:$PATH" -``` - -```sh -mkdir -p ~/.kube -ssh k3s@ sudo cat /etc/rancher/k3s/k3s.yaml \ - | sed "s/127.0.0.1//" > ~/.kube/config-manual export KUBECONFIG=~/.kube/config-manual kubectl get nodes # expect 1 Ready node ``` -## 3. Bootstrap Flux against Forgejo +## Flux: what's actually happening, mechanism by mechanism -If you haven't installed the Flux CLI yet (no sudo needed): +"Flux" isn't one program watching your repo by magic — it's a handful of +ordinary Kubernetes controllers (just Deployments, like anything else +you'd run on the cluster), each one understanding a couple of **Custom +Resource Definitions** — CRDs extend the Kubernetes API with new object +kinds, the same way `Deployment` or `Service` are built-in kinds. +`GitRepository` and `Kustomization` are two such kinds Flux adds. Nothing +about any of this is special-cased outside the normal Kubernetes API — +it's the exact same "define an object, a controller notices it and acts" +loop that runs the whole rest of Kubernetes. + +Install the CLI the same no-sudo, direct-binary-release way as `kubectl`: ```sh FLUX_VERSION=$(curl -s https://api.github.com/repos/fluxcd/flux2/releases/latest | grep tag_name | cut -d '"' -f4 | sed 's/^v//') @@ -197,8 +362,11 @@ rm /tmp/flux.tar.gz flux --version ``` -Same repo, same token, same target path as the full guide would use — -nothing about this is cluster-specific: +Forgejo isn't a Flux-native provider the way GitHub/GitLab are, so this +uses the generic git bootstrap — over HTTPS with the `flux-write` token +from step 4, not SSH (Forgejo's git-SSH port turned out not to be reliably +reachable from either this desktop or the T630 — see the earlier +troubleshooting in this project's history): ```sh flux check --pre --kubeconfig ~/.kube/config-manual @@ -213,33 +381,136 @@ flux bootstrap git \ --kubeconfig ~/.kube/config-manual ``` -## 4. Continue with the shared steps +That one command did five genuinely separate things. Go look at each — +these commands work against the cluster you already bootstrapped: -From here, `docs/SETUP.md` steps 8–10 and 12–13 apply exactly as written, -regardless of how the cluster was created — verifying the podinfo GitOps -loop, registering the Forgejo Actions runner, the Dashboard, remote kubectl -access. Just use `~/.kube/config-manual` as the kubeconfig throughout -instead of the Terraform-provisioned one. - -**Step 11 (exposing apps through Caddy) does not apply as-written**, -though — it assumes the Terraform track's static `10.20.30.x` addressing. -This VM is on libvirt's `default` network instead, using DHCP: +**1. It installed the controllers** — plain Kubernetes Deployments, no +different in kind from anything else running on the cluster: ```sh -virsh -c qemu:///system domifaddr k3s-manual # note the IP and MAC address +kubectl -n flux-system get deployments ``` -DHCP leases are "sticky" to a MAC address in practice, but not guaranteed -fixed the way the Terraform track's static IPs are — pin it explicitly so -it can't change later: +`source-controller`, `kustomize-controller`, `helm-controller`, +`notification-controller`. Alongside them, it registered the CRDs those +controllers understand: ```sh -virsh -c qemu:///system net-update default add ip-dhcp-host \ - "" --live --config +kubectl get crds | grep fluxcd ``` -Then in Caddy's config, use that IP directly rather than -`docs/Caddyfile.example`'s multi-node placeholder blocks: +**2. It created a `GitRepository` object** — this is the whole "what repo +am I watching" declaration, and nothing more. `source-controller`'s job is +entirely mechanical: every `interval` (default 1m), do a real `git fetch` +against `.spec.url`/`.spec.ref`; if the commit SHA changed, package that +tree into a `.tar.gz`, and record its location in `.status.artifact`. +That's the entire job — it doesn't know or care what's *in* the repo. + +```sh +kubectl -n flux-system get gitrepository flux-system -o yaml +``` + +Look at `.status.conditions` and `.status.artifact` — that's the result of +an actual git fetch that already happened, not a static config. + +**3. It created a `Kustomization` object** (confusingly, the same name as +the `kustomization.yaml` files already sitting in `apps/podinfo/` etc. — +related but not identical). `kustomize-controller`'s job, on its own +interval: fetch the artifact `source-controller` produced, run the real +`kustomize` tool against `.spec.path` inside it (the exact same tool a +plain `kustomize build apps/podinfo` would run locally against those +`kustomization.yaml` files), and apply the resulting objects via the +Kubernetes API — the automated equivalent of you running +`kubectl apply -f <(kustomize build apps/podinfo)` yourself, on a timer, +forever. `prune: true` (set on all the `Kustomization` objects in this +repo) means it also *deletes* anything it previously created that's no +longer present in the current git state — that's what makes it +self-healing rather than just "apply once." + +```sh +kubectl -n flux-system get kustomization +``` + +**This is also why `apps/podinfo`, `apps/hello-app`, and +`apps/kubernetes-dashboard` started deploying without you ever running +`kubectl apply` on them.** Bootstrap's own `Kustomization` watches +`clusters/homelab` with `prune: true`. `clusters/homelab/apps.yaml` +(already sitting in the repo, hand-written earlier in this project) itself +just *defines more `Kustomization` objects*, one per app — so the first +one picks it up as part of its own normal reconcile, creates those three +child `Kustomization`s, and each of *those* then does its own +fetch-and-apply against its own app directory. It's the same mechanism +recursing, not a special case. + +**4. It stored your credential as a Kubernetes Secret** — `--password` +here is the `flux-write` token, not an account password. Bootstrap +base64-wraps it into a Secret that matches what the `GitRepository`'s +`.spec.secretRef` points at: + +```sh +kubectl -n flux-system get secret flux-system -o yaml +``` + +The `password` field is base64 (`| base64 -d` to read it) — this is +exactly the same credential `git clone https://user:token@host/repo` would +use, just read by `source-controller` on every fetch instead of typed by +you once. It's never written to `k8s`'s own filesystem. + +**5. It committed that config back into the repo itself** — `git pull` in +your desktop clone and look at `clusters/homelab/flux-system/`. The +`GitRepository`/`Kustomization` objects you just inspected live in the +cluster *because* those exact YAML files are committed there — so +rebuilding this cluster from scratch would mean running `flux bootstrap` +again (or even just `kubectl apply -f clusters/homelab/flux-system/`) and +landing in the identical state. The fact that Flux watches this repo is +itself declared *in* this repo. + +If you want to watch a reconcile happen live rather than just inspect the +end state: + +```sh +kubectl -n flux-system logs deploy/source-controller -f +``` + +then in another terminal, make any commit and push it — you'll see the +next fetch pick it up within the interval. + +## Verifying the GitOps loop actually works + +```sh +flux get kustomizations --watch +``` + +Watch until `podinfo`, `hello-app`, and `kubernetes-dashboard` all show +`Ready: True`, then confirm pods actually landed: + +```sh +kubectl -n podinfo get pods +kubectl -n hello-app get pods +kubectl -n kubernetes-dashboard get pods +``` + +If that's all healthy, the entire chain — Forgejo repo → Flux → this +cluster — is working end to end with nothing manually `kubectl apply`'d. + +## Exposing podinfo/hello-app through Caddy + +k3s's bundled ingress controller (Traefik) is already listening on this +node's own IP, port 80, routing by the `Host:` header from each app's +`Ingress` resource (already defined in `apps/podinfo` and +`apps/hello-app`). Caddy just needs to forward matching requests there. + +Two DNS records, as CNAMEs pointed at `git.boglabob.com` rather than +duplicating its IP directly — CNAME means "this name is an alias for that +one," so there's one place (that record) to update if the underlying IP +ever changes, instead of several: + +``` +podinfo.boglabob.com CNAME git.boglabob.com +hello.boglabob.com CNAME git.boglabob.com +``` + +Then in Caddy's own config, using the VM's pinned IP from earlier: ``` podinfo.boglabob.com { @@ -251,26 +522,45 @@ hello.boglabob.com { } ``` +```sh +podman exec caddy reload --config /etc/caddy/Caddyfile +``` + +The Dashboard and the k3s API server are deliberately **not** here — +`docs/SETUP.md` steps 12–13 cover why (both are cluster-admin-capable, and +exposing either publicly is the exact pattern behind real breaches like +Tesla's 2018 incident) and how to reach them instead (`kubectl +port-forward`, and LAN/tunnel-only kubectl access). Both apply to this +cluster exactly as written there. + +## Registering the Forgejo Actions runner + +`docs/SETUP.md` step 9 applies as written — it's about the `k8s` user and +rootless Podman, not about which cluster exists. Worth understanding +before running it: the runner would normally get root-equivalent power +over its host via a mounted `docker.sock`; instead it runs as `k8s` itself +using rootless Podman's own API socket, and `build-hello-app.yml` builds +images with `kaniko` (no daemon, no elevated privileges needed at all) — +so nothing in that pipeline ever touches `sudo`. + ## Tearing this down -Once the Terraform track in `docs/SETUP.md` is ready and you `tofu apply` -the real 3-node cluster, remove this one: +Once the Terraform track is ready and `tofu apply` brings up the real +3-node cluster, remove this one: ```sh -# as k8s -export LIBVIRT_DEFAULT_URI=qemu:///system -virsh destroy k3s-manual # stop it -virsh undefine k3s-manual --remove-all-storage # delete VM + its overlay disk -virsh vol-delete --pool default k3s-manual-base.qcow2 # the base image isn't - # attached to the VM - # directly, so it - # needs its own delete +virsh -c qemu:///system destroy k3s-manual # stop it +virsh -c qemu:///system undefine k3s-manual --remove-all-storage # VM + overlay disk +virsh -c qemu:///system vol-delete --pool default k3s-manual-base.qcow2 # base image isn't + # attached to the VM + # directly, needs its + # own delete rm ~/.kube/config-manual rm -rf ~/vms ``` -Nothing else needs cleaning up — Flux's state lived entirely inside that -VM's cluster and goes away with it. The Forgejo repo, deploy tokens, and -the Forgejo Actions runner registration are all cluster-independent and -carry over to the real cluster unchanged; just re-run `flux bootstrap` -against its kubeconfig once it's up. +Nothing else needs cleaning up — Flux's own state lived entirely inside +that VM's cluster and goes away with it. The Forgejo repo, both tokens, +and the Forgejo Actions runner registration are all cluster-independent +and carry over unchanged; point `flux bootstrap` at the new cluster's +kubeconfig once it exists.