Updated project to improve guidance
Some checks failed
terraform / validate (push) Failing after 37s

This commit is contained in:
CodeGit 2026-09-03 19:06:27 +01:00
parent 63008b3ab9
commit 4ea6d8b9e5
26 changed files with 1091 additions and 695 deletions

132
docs/01-bootstrap.md Normal file
View file

@ -0,0 +1,132 @@
# Stage 1: Bootstrap
This project is a four-stage tutorial, each stage building on the last:
1. **`docs/01-bootstrap.md`** (this doc) — one-time host setup: KVM/libvirt,
an unprivileged user to run everything as, and pushing this repo to
Forgejo. Nothing here is specific to k3s, Flux, or Terraform — every
later stage depends on it.
2. **`docs/02-k3s.md`** — build one VM by hand and get a real k3s cluster
running on it. No Flux yet, no Terraform — just a working cluster and a
kubeconfig that talks to it.
3. **`docs/03-flux.md`** — bootstrap Flux against that cluster and deploy
this repo's apps through it, including what Helm chart management looks
like under GitOps (podinfo, headlamp) versus plain manifests
(hello-app), and the CI loop that builds and auto-deploys hello-app.
4. **`docs/04-tofu.md`** — graduate from the one hand-built VM to a proper
3-node cluster provisioned by Terraform/OpenTofu, and point the same
Flux config at it.
Each stage says exactly which earlier steps it actually depends on, rather
than assuming you need everything done up front — stage 2, for instance,
only needs steps 1-2 below.
Assumes: the T630 is an existing Debian box already running other
self-hosted services — this project installs alongside those as ordinary
packages (`qemu-kvm`/`libvirt`), not a hypervisor OS replacing Debian.
Forgejo is already running and reachable at `https://git.boglabob.com`, and
you can point DNS records under `boglabob.com` at hosts on your network
(directly, or via whatever reverse proxy/tunnel already gets
`git.boglabob.com` there).
---
## 1. Install KVM/libvirt on the T630
Ordinary packages, no reboot into an installer, nothing else on the box is
touched:
```sh
# on the T630
sudo apt update
sudo apt install -y qemu-kvm libvirt-daemon-system libvirt-clients virtinst
# confirm hardware virtualization is available (T630's Xeons support it)
sudo kvm-ok
```
## 2. Create the unprivileged 'k8s' user
One dedicated, no-sudo user for everything this project touches: driving
`virsh`/`kubectl`/`flux`/`tofu` against libvirt, and running the Forgejo
Actions runner later (`docs/03-flux.md` step 6). It needs group membership
to talk to libvirt — that's a one-time root action; nothing it does
afterwards needs `sudo`.
```sh
sudo useradd -m -s /bin/bash k8s # one-time, needs root to create the user itself
sudo usermod -aG libvirt,kvm k8s
sudo loginctl enable-linger k8s # lets its services keep running after logout
# as k8s, from here on (sudo -iu, not su -, since k8s has no password set):
sudo -iu k8s
ssh-keygen -t ed25519 -C "k3s-homelab" -f ~/.ssh/id_ed25519 # only needed if you'll SSH in as k8s day-to-day
virsh -c qemu:///system list --all # sanity check: should run with no permission error, no sudo
```
Do the rest of this project logged in as `k8s` on the T630 itself (`ssh
k8s@t630`) — VM IPs live on a private libvirt network that's only directly
reachable from the T630, so this is the simplest place to run
`kubectl`/`flux`/`tofu` from. (If you'd rather drive Terraform from your
own workstation instead once you reach stage 4, see the `libvirt_uri`
comment in `terraform/terraform.tfvars.example` — you'll then need an SSH
tunnel for kubectl/flux to reach node IPs.)
## 3. Generate the secret k3s needs
```sh
openssl rand -hex 32 # -> k3s_token
```
This is the shared token agent nodes use to join a k3s server — irrelevant
for stage 2's single-node VM (a lone server needs no one to join it), but
generate it now while you're doing one-time setup; stage 4's
Terraform-provisioned multi-node cluster is what actually uses it.
## 4. Push this repo to Forgejo
Doing this before any cluster exists (rather than after) means `k8s` can
get the repo with a plain `git clone` later, instead of needing a one-off
copy handed to it — and any future change to this repo just needs a
`git pull` on the T630. Repo/owner used throughout this project:
`codegit/cloud-demo` (already baked into `apps/hello-app/deployment.yaml`
and `image-automation.yaml`'s image references — no placeholder-swapping
needed).
1. On Forgejo (`https://git.boglabob.com`), as `codegit`: **+ → New
Repository** → name `cloud-demo`. Leave it empty — don't initialize with
a README/`.gitignore`/license, since this repo already has its own.
Visibility (public/private) is your call; either works, since access for
`k8s`/Flux/CI goes through the tokens below regardless.
2. Locally, wherever you're editing this repo (`maq`):
```sh
git init # if not already
git add .
git commit -m "initial scaffold"
git remote add origin https://git.boglabob.com/codegit/cloud-demo.git
git push -u origin main
```
3. Generate two access tokens (`Settings → Applications → Generate New
Token`), scoped as narrowly as Forgejo's token UI allows to repository
read/write:
- **`k8s-readonly`** — read-only. Used only for `k8s`'s own manual
`git clone`/`pull` on the T630 — never leaves that box, isn't used by
anything automated. Not needed until `docs/04-tofu.md` (that's the
first stage that clones this repo onto the T630 rather than editing
it from your workstation).
- **`flux-write`** — read/write. Used once, as a `flux bootstrap`
argument (`docs/03-flux.md` step 2); Flux stores it as a Kubernetes
Secret inside the cluster from then on (`ImageUpdateAutomation`'s
commits back, in the hello-app section of that stage, reuse that same
in-cluster Secret) — it's never written to `k8s`'s filesystem at all.
Using HTTPS tokens instead of `k8s`'s SSH key (`~/.ssh/id_ed25519`, from
step 2) sidesteps an open question: Forgejo's git-SSH port isn't
reachable from this desktop through your router (see the SSH
troubleshooting earlier in this project's history), and whether it's
reachable from `k8s` on the T630 itself was never actually confirmed
either. HTTPS (443, via Caddy) is already proven to work, so both
tokens use that instead. Copy both token values now — Forgejo only
shows them once.
Next: `docs/02-k3s.md` — you only need steps 1-2 above to start it.