Updated project to improve guidance
Some checks failed
terraform / validate (push) Failing after 37s
Some checks failed
terraform / validate (push) Failing after 37s
This commit is contained in:
parent
63008b3ab9
commit
4ea6d8b9e5
26 changed files with 1091 additions and 695 deletions
132
docs/01-bootstrap.md
Normal file
132
docs/01-bootstrap.md
Normal file
|
|
@ -0,0 +1,132 @@
|
|||
# Stage 1: Bootstrap
|
||||
|
||||
This project is a four-stage tutorial, each stage building on the last:
|
||||
|
||||
1. **`docs/01-bootstrap.md`** (this doc) — one-time host setup: KVM/libvirt,
|
||||
an unprivileged user to run everything as, and pushing this repo to
|
||||
Forgejo. Nothing here is specific to k3s, Flux, or Terraform — every
|
||||
later stage depends on it.
|
||||
2. **`docs/02-k3s.md`** — build one VM by hand and get a real k3s cluster
|
||||
running on it. No Flux yet, no Terraform — just a working cluster and a
|
||||
kubeconfig that talks to it.
|
||||
3. **`docs/03-flux.md`** — bootstrap Flux against that cluster and deploy
|
||||
this repo's apps through it, including what Helm chart management looks
|
||||
like under GitOps (podinfo, headlamp) versus plain manifests
|
||||
(hello-app), and the CI loop that builds and auto-deploys hello-app.
|
||||
4. **`docs/04-tofu.md`** — graduate from the one hand-built VM to a proper
|
||||
3-node cluster provisioned by Terraform/OpenTofu, and point the same
|
||||
Flux config at it.
|
||||
|
||||
Each stage says exactly which earlier steps it actually depends on, rather
|
||||
than assuming you need everything done up front — stage 2, for instance,
|
||||
only needs steps 1-2 below.
|
||||
|
||||
Assumes: the T630 is an existing Debian box already running other
|
||||
self-hosted services — this project installs alongside those as ordinary
|
||||
packages (`qemu-kvm`/`libvirt`), not a hypervisor OS replacing Debian.
|
||||
Forgejo is already running and reachable at `https://git.boglabob.com`, and
|
||||
you can point DNS records under `boglabob.com` at hosts on your network
|
||||
(directly, or via whatever reverse proxy/tunnel already gets
|
||||
`git.boglabob.com` there).
|
||||
|
||||
---
|
||||
|
||||
## 1. Install KVM/libvirt on the T630
|
||||
|
||||
Ordinary packages, no reboot into an installer, nothing else on the box is
|
||||
touched:
|
||||
|
||||
```sh
|
||||
# on the T630
|
||||
sudo apt update
|
||||
sudo apt install -y qemu-kvm libvirt-daemon-system libvirt-clients virtinst
|
||||
|
||||
# confirm hardware virtualization is available (T630's Xeons support it)
|
||||
sudo kvm-ok
|
||||
```
|
||||
|
||||
## 2. Create the unprivileged 'k8s' user
|
||||
|
||||
One dedicated, no-sudo user for everything this project touches: driving
|
||||
`virsh`/`kubectl`/`flux`/`tofu` against libvirt, and running the Forgejo
|
||||
Actions runner later (`docs/03-flux.md` step 6). It needs group membership
|
||||
to talk to libvirt — that's a one-time root action; nothing it does
|
||||
afterwards needs `sudo`.
|
||||
|
||||
```sh
|
||||
sudo useradd -m -s /bin/bash k8s # one-time, needs root to create the user itself
|
||||
sudo usermod -aG libvirt,kvm k8s
|
||||
sudo loginctl enable-linger k8s # lets its services keep running after logout
|
||||
|
||||
# as k8s, from here on (sudo -iu, not su -, since k8s has no password set):
|
||||
sudo -iu k8s
|
||||
ssh-keygen -t ed25519 -C "k3s-homelab" -f ~/.ssh/id_ed25519 # only needed if you'll SSH in as k8s day-to-day
|
||||
virsh -c qemu:///system list --all # sanity check: should run with no permission error, no sudo
|
||||
```
|
||||
|
||||
Do the rest of this project logged in as `k8s` on the T630 itself (`ssh
|
||||
k8s@t630`) — VM IPs live on a private libvirt network that's only directly
|
||||
reachable from the T630, so this is the simplest place to run
|
||||
`kubectl`/`flux`/`tofu` from. (If you'd rather drive Terraform from your
|
||||
own workstation instead once you reach stage 4, see the `libvirt_uri`
|
||||
comment in `terraform/terraform.tfvars.example` — you'll then need an SSH
|
||||
tunnel for kubectl/flux to reach node IPs.)
|
||||
|
||||
## 3. Generate the secret k3s needs
|
||||
|
||||
```sh
|
||||
openssl rand -hex 32 # -> k3s_token
|
||||
```
|
||||
|
||||
This is the shared token agent nodes use to join a k3s server — irrelevant
|
||||
for stage 2's single-node VM (a lone server needs no one to join it), but
|
||||
generate it now while you're doing one-time setup; stage 4's
|
||||
Terraform-provisioned multi-node cluster is what actually uses it.
|
||||
|
||||
## 4. Push this repo to Forgejo
|
||||
|
||||
Doing this before any cluster exists (rather than after) means `k8s` can
|
||||
get the repo with a plain `git clone` later, instead of needing a one-off
|
||||
copy handed to it — and any future change to this repo just needs a
|
||||
`git pull` on the T630. Repo/owner used throughout this project:
|
||||
`codegit/cloud-demo` (already baked into `apps/hello-app/deployment.yaml`
|
||||
and `image-automation.yaml`'s image references — no placeholder-swapping
|
||||
needed).
|
||||
|
||||
1. On Forgejo (`https://git.boglabob.com`), as `codegit`: **+ → New
|
||||
Repository** → name `cloud-demo`. Leave it empty — don't initialize with
|
||||
a README/`.gitignore`/license, since this repo already has its own.
|
||||
Visibility (public/private) is your call; either works, since access for
|
||||
`k8s`/Flux/CI goes through the tokens below regardless.
|
||||
2. Locally, wherever you're editing this repo (`maq`):
|
||||
```sh
|
||||
git init # if not already
|
||||
git add .
|
||||
git commit -m "initial scaffold"
|
||||
git remote add origin https://git.boglabob.com/codegit/cloud-demo.git
|
||||
git push -u origin main
|
||||
```
|
||||
3. Generate two access tokens (`Settings → Applications → Generate New
|
||||
Token`), scoped as narrowly as Forgejo's token UI allows to repository
|
||||
read/write:
|
||||
- **`k8s-readonly`** — read-only. Used only for `k8s`'s own manual
|
||||
`git clone`/`pull` on the T630 — never leaves that box, isn't used by
|
||||
anything automated. Not needed until `docs/04-tofu.md` (that's the
|
||||
first stage that clones this repo onto the T630 rather than editing
|
||||
it from your workstation).
|
||||
- **`flux-write`** — read/write. Used once, as a `flux bootstrap`
|
||||
argument (`docs/03-flux.md` step 2); Flux stores it as a Kubernetes
|
||||
Secret inside the cluster from then on (`ImageUpdateAutomation`'s
|
||||
commits back, in the hello-app section of that stage, reuse that same
|
||||
in-cluster Secret) — it's never written to `k8s`'s filesystem at all.
|
||||
|
||||
Using HTTPS tokens instead of `k8s`'s SSH key (`~/.ssh/id_ed25519`, from
|
||||
step 2) sidesteps an open question: Forgejo's git-SSH port isn't
|
||||
reachable from this desktop through your router (see the SSH
|
||||
troubleshooting earlier in this project's history), and whether it's
|
||||
reachable from `k8s` on the T630 itself was never actually confirmed
|
||||
either. HTTPS (443, via Caddy) is already proven to work, so both
|
||||
tokens use that instead. Copy both token values now — Forgejo only
|
||||
shows them once.
|
||||
|
||||
Next: `docs/02-k3s.md` — you only need steps 1-2 above to start it.
|
||||
Loading…
Add table
Add a link
Reference in a new issue