# Quickstart: manual cluster (no Terraform)

Gets a real k3s cluster running today, by hand, so you can learn/practice
Flux and GitOps immediately instead of waiting on the Terraform/libvirt
provider work in `docs/SETUP.md` to be sorted out. One throwaway VM, created
directly with `virt-install` rather than the `dmacvicar/libvirt` Terraform
provider. When the Terraform track is ready, tear this down (last section)
and provision the "real" 3-node cluster properly instead.

Shares steps 1–4 of `docs/SETUP.md` as prerequisites — do those first if you
haven't:
- Step 1: KVM/libvirt installed on the T630.
- Step 2: the unprivileged `k8s` user exists, with `~/.ssh/k3s_homelab`
  generated.
- Step 3: `openssl rand -hex 32` isn't needed here (no agents joining, so no
  cluster token) — skip it.
- Step 4: `cloud-demo` pushed to Forgejo, with the `flux-write` token
  generated.

Everything below runs as `k8s` on the T630 (`sudo -iu k8s`).

## 1. Create the VM

No sudo needed anywhere in this guide — `k8s`'s `libvirt`/`kvm` group
membership (step 2) is what authorizes talking to libvirt at all, via
polkit. But group membership alone doesn't solve everything: system libvirt
runs actual VMs as a separate, restricted `libvirt-qemu` user, not as
`k8s` — so a disk image sitting under `k8s`'s home directory (mode 700 by
default) would fail at boot, since that user can't read into `k8s`'s home
at all. The fix is to let libvirt manage the disk storage itself, inside
its own pool, rather than pointing at a raw path in `~` — pool operations
go through libvirtd's API, so it's libvirtd (already running with the
right privileges) that handles the file ownership, not `k8s` directly.

Every `virsh`/`virt-install` command below uses `-c qemu:///system` /
`--connect qemu:///system` explicitly, rather than relying on the
`LIBVIRT_DEFAULT_URI` env var — for a non-root user that env var is what
picks system vs. the per-user `qemu:///session` instance, and it's easy for
it to be set in one terminal and not another, silently causing commands to
operate on the wrong (session) libvirt instance where nothing you created
system-side is visible. Explicit `-c` on every command avoids that
ambiguity entirely.

**Debian's `libvirt-daemon-system` does *not* auto-create a `default`
storage pool** the way some other distros' packaging does (it does
auto-create the `default` *network*, just left inactive) — check and fix
both before doing anything else:

```sh
virsh -c qemu:///system pool-list --all
virsh -c qemu:///system net-list --all
```

If `pool-list` comes back empty:

```sh
virsh -c qemu:///system pool-define-as default dir --target /var/lib/libvirt/images
virsh -c qemu:///system pool-build default
virsh -c qemu:///system pool-start default
virsh -c qemu:///system pool-autostart default
```

If `net-list` shows `default` as `inactive`:

```sh
virsh -c qemu:///system net-start default
virsh -c qemu:///system net-autostart default
```

Download the cloud image somewhere `k8s` can read it — this location
itself doesn't need to be qemu-readable, since it's only ever read by
`virsh` (running as `k8s`), never directly by the VM:

```sh
curl -L -o /tmp/noble-base.img \
  https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img
```

Import it into the `default` pool, then create a copy-on-write overlay on
top of it, sized up to 20G:

```sh
virsh -c qemu:///system vol-create-as default k3s-manual-base.qcow2 \
  --capacity "$(stat -c%s /tmp/noble-base.img)" --format qcow2
virsh -c qemu:///system vol-upload --pool default k3s-manual-base.qcow2 /tmp/noble-base.img --sparse
rm /tmp/noble-base.img

virsh -c qemu:///system vol-create-as default k3s-manual.qcow2 20G --format qcow2 \
  --backing-vol k3s-manual-base.qcow2 --backing-vol-format qcow2

virsh -c qemu:///system vol-list --pool default   # should list both volumes
```

Write the cloud-init user-data — paste in the contents of
`~/.ssh/k3s_homelab.pub` where marked:

```sh
mkdir -p ~/vms
cat > ~/vms/k3s-manual-user-data.yaml <<'EOF'
#cloud-config
hostname: k3s-manual
manage_etc_hosts: true

users:
  - name: k3s
    groups: sudo
    shell: /bin/bash
    sudo: ALL=(ALL) NOPASSWD:ALL
    ssh_authorized_keys:
      - PASTE ~/.ssh/k3s_homelab.pub CONTENTS HERE

package_update: true
packages:
  - curl

runcmd:
  - curl -sfL https://get.k3s.io | sh -s - server
  - mkdir -p /home/k3s/.kube
  - k3s kubectl config view --raw > /home/k3s/.kube/config
  - chown -R k3s:k3s /home/k3s/.kube
EOF
```

This is a single, standalone server — no join token, no agents, no custom
network (uses libvirt's default NAT network + DHCP). A k3s server node
schedules workloads on itself by default, so this alone is a complete,
usable cluster.

The cloud-init user-data file itself is fine living under `~/vms` — unlike
the disk, it's only ever read client-side by `virt-install` (as `k8s`),
which then hands the rendered seed data to libvirtd over the API. Note
`$HOME` rather than `~` in the `--cloud-init` argument below — `~` only
expands at the start of a shell word, not after `=` inside one, so
`user-data=~/vms/...` would be passed to virt-install literally with the
tilde still in it.

`--os-variant ubuntu22.04` below is deliberately not `ubuntu24.04` (the
actual image) — `osinfo-db` on Debian 12 may not recognize the newer
variant name yet. Check what yours knows about with
`osinfo-query os | grep -i ubuntu`; the hint mostly just tunes libvirt's
domain defaults (virtio devices, clock, etc.), so a close-enough variant
against a cloud-init `--import` like this is harmless:

```sh
virt-install \
  --connect qemu:///system \
  --name k3s-manual \
  --memory 2048 \
  --vcpus 2 \
  --disk vol=default/k3s-manual.qcow2 \
  --import \
  --os-variant ubuntu22.04 \
  --network network=default \
  --cloud-init user-data=$HOME/vms/k3s-manual-user-data.yaml \
  --graphics none \
  --noautoconsole
```

## 2. Find its IP and get kubectl talking to it

Give cloud-init ~2 minutes to finish installing k3s after the VM boots.

```sh
virsh -c qemu:///system domifaddr k3s-manual   # note the IP under the default network

ssh k3s@<VM_IP> sudo cat /etc/rancher/k3s/k3s.yaml \
  | sed "s/127.0.0.1/<VM_IP>/" > ~/.kube/config-manual

export KUBECONFIG=~/.kube/config-manual
kubectl get nodes   # expect 1 Ready node
```

## 3. Bootstrap Flux against Forgejo

Same repo, same token, same target path as the full guide would use —
nothing about this is cluster-specific:

```sh
flux bootstrap git \
  --url=https://git.boglabob.com/codegit/cloud-demo \
  --branch=main \
  --path=clusters/homelab \
  --username=codegit \
  --password=<FLUX_WRITE_TOKEN> \
  --token-auth \
  --kubeconfig ~/.kube/config-manual
```

## 4. Continue with the shared steps

From here, `docs/SETUP.md` steps 8 onward apply exactly as written,
regardless of how the cluster was created — verifying the podinfo GitOps
loop, registering the Forgejo Actions runner, exposing apps through Caddy,
the Dashboard, remote kubectl access. Just use `~/.kube/config-manual` as
the kubeconfig throughout instead of the Terraform-provisioned one.

## Tearing this down

Once the Terraform track in `docs/SETUP.md` is ready and you `tofu apply`
the real 3-node cluster, remove this one:

```sh
# as k8s
export LIBVIRT_DEFAULT_URI=qemu:///system
virsh destroy k3s-manual              # stop it
virsh undefine k3s-manual --remove-all-storage   # delete VM + its overlay disk
virsh vol-delete --pool default k3s-manual-base.qcow2   # the base image isn't
                                                          # attached to the VM
                                                          # directly, so it
                                                          # needs its own delete
rm ~/.kube/config-manual
rm -rf ~/vms
```

Nothing else needs cleaning up — Flux's state lived entirely inside that
VM's cluster and goes away with it. The Forgejo repo, deploy tokens, and
the Forgejo Actions runner registration are all cluster-independent and
carry over to the real cluster unchanged; just re-run `flux bootstrap`
against its kubeconfig once it's up.