132 lines
6.1 KiB
Markdown
132 lines
6.1 KiB
Markdown
# Stage 1: Bootstrap
|
|
|
|
This project is a four-stage tutorial, each stage building on the last:
|
|
|
|
1. **`docs/01-bootstrap.md`** (this doc) — one-time host setup: KVM/libvirt,
|
|
an unprivileged user to run everything as, and pushing this repo to
|
|
Forgejo. Nothing here is specific to k3s, Flux, or Terraform — every
|
|
later stage depends on it.
|
|
2. **`docs/02-k3s.md`** — build one VM by hand and get a real k3s cluster
|
|
running on it. No Flux yet, no Terraform — just a working cluster and a
|
|
kubeconfig that talks to it.
|
|
3. **`docs/03-flux.md`** — bootstrap Flux against that cluster and deploy
|
|
this repo's apps through it, including what Helm chart management looks
|
|
like under GitOps (podinfo, headlamp) versus plain manifests
|
|
(hello-app), and the CI loop that builds and auto-deploys hello-app.
|
|
4. **`docs/04-tofu.md`** — graduate from the one hand-built VM to a proper
|
|
3-node cluster provisioned by Terraform/OpenTofu, and point the same
|
|
Flux config at it.
|
|
|
|
Each stage says exactly which earlier steps it actually depends on, rather
|
|
than assuming you need everything done up front — stage 2, for instance,
|
|
only needs steps 1-2 below.
|
|
|
|
Assumes: the T630 is an existing Debian box already running other
|
|
self-hosted services — this project installs alongside those as ordinary
|
|
packages (`qemu-kvm`/`libvirt`), not a hypervisor OS replacing Debian.
|
|
Forgejo is already running and reachable at `https://git.boglabob.com`, and
|
|
you can point DNS records under `boglabob.com` at hosts on your network
|
|
(directly, or via whatever reverse proxy/tunnel already gets
|
|
`git.boglabob.com` there).
|
|
|
|
---
|
|
|
|
## 1. Install KVM/libvirt on the T630
|
|
|
|
Ordinary packages, no reboot into an installer, nothing else on the box is
|
|
touched:
|
|
|
|
```sh
|
|
# on the T630
|
|
sudo apt update
|
|
sudo apt install -y qemu-kvm libvirt-daemon-system libvirt-clients virtinst
|
|
|
|
# confirm hardware virtualization is available (T630's Xeons support it)
|
|
sudo kvm-ok
|
|
```
|
|
|
|
## 2. Create the unprivileged 'k8s' user
|
|
|
|
One dedicated, no-sudo user for everything this project touches: driving
|
|
`virsh`/`kubectl`/`flux`/`tofu` against libvirt, and running the Forgejo
|
|
Actions runner later (`docs/03-flux.md` step 6). It needs group membership
|
|
to talk to libvirt — that's a one-time root action; nothing it does
|
|
afterwards needs `sudo`.
|
|
|
|
```sh
|
|
sudo useradd -m -s /bin/bash k8s # one-time, needs root to create the user itself
|
|
sudo usermod -aG libvirt,kvm k8s
|
|
sudo loginctl enable-linger k8s # lets its services keep running after logout
|
|
|
|
# as k8s, from here on (sudo -iu, not su -, since k8s has no password set):
|
|
sudo -iu k8s
|
|
ssh-keygen -t ed25519 -C "k3s-homelab" -f ~/.ssh/id_ed25519 # only needed if you'll SSH in as k8s day-to-day
|
|
virsh -c qemu:///system list --all # sanity check: should run with no permission error, no sudo
|
|
```
|
|
|
|
Do the rest of this project logged in as `k8s` on the T630 itself (`ssh
|
|
k8s@t630`) — VM IPs live on a private libvirt network that's only directly
|
|
reachable from the T630, so this is the simplest place to run
|
|
`kubectl`/`flux`/`tofu` from. (If you'd rather drive Terraform from your
|
|
own workstation instead once you reach stage 4, see the `libvirt_uri`
|
|
comment in `terraform/terraform.tfvars.example` — you'll then need an SSH
|
|
tunnel for kubectl/flux to reach node IPs.)
|
|
|
|
## 3. Generate the secret k3s needs
|
|
|
|
```sh
|
|
openssl rand -hex 32 # -> k3s_token
|
|
```
|
|
|
|
This is the shared token agent nodes use to join a k3s server — irrelevant
|
|
for stage 2's single-node VM (a lone server needs no one to join it), but
|
|
generate it now while you're doing one-time setup; stage 4's
|
|
Terraform-provisioned multi-node cluster is what actually uses it.
|
|
|
|
## 4. Push this repo to Forgejo
|
|
|
|
Doing this before any cluster exists (rather than after) means `k8s` can
|
|
get the repo with a plain `git clone` later, instead of needing a one-off
|
|
copy handed to it — and any future change to this repo just needs a
|
|
`git pull` on the T630. Repo/owner used throughout this project:
|
|
`codegit/cloud-demo` (already baked into `apps/hello-app/deployment.yaml`
|
|
and `image-automation.yaml`'s image references — no placeholder-swapping
|
|
needed).
|
|
|
|
1. On Forgejo (`https://git.boglabob.com`), as `codegit`: **+ → New
|
|
Repository** → name `cloud-demo`. Leave it empty — don't initialize with
|
|
a README/`.gitignore`/license, since this repo already has its own.
|
|
Visibility (public/private) is your call; either works, since access for
|
|
`k8s`/Flux/CI goes through the tokens below regardless.
|
|
2. Locally, wherever you're editing this repo (`maq`):
|
|
```sh
|
|
git init # if not already
|
|
git add .
|
|
git commit -m "initial scaffold"
|
|
git remote add origin https://git.boglabob.com/codegit/cloud-demo.git
|
|
git push -u origin main
|
|
```
|
|
3. Generate two access tokens (`Settings → Applications → Generate New
|
|
Token`), scoped as narrowly as Forgejo's token UI allows to repository
|
|
read/write:
|
|
- **`k8s-readonly`** — read-only. Used only for `k8s`'s own manual
|
|
`git clone`/`pull` on the T630 — never leaves that box, isn't used by
|
|
anything automated. Not needed until `docs/04-tofu.md` (that's the
|
|
first stage that clones this repo onto the T630 rather than editing
|
|
it from your workstation).
|
|
- **`flux-write`** — read/write. Used once, as a `flux bootstrap`
|
|
argument (`docs/03-flux.md` step 2); Flux stores it as a Kubernetes
|
|
Secret inside the cluster from then on (`ImageUpdateAutomation`'s
|
|
commits back, in the hello-app section of that stage, reuse that same
|
|
in-cluster Secret) — it's never written to `k8s`'s filesystem at all.
|
|
|
|
Using HTTPS tokens instead of `k8s`'s SSH key (`~/.ssh/id_ed25519`, from
|
|
step 2) sidesteps an open question: Forgejo's git-SSH port isn't
|
|
reachable from this desktop through your router (see the SSH
|
|
troubleshooting earlier in this project's history), and whether it's
|
|
reachable from `k8s` on the T630 itself was never actually confirmed
|
|
either. HTTPS (443, via Caddy) is already proven to work, so both
|
|
tokens use that instead. Copy both token values now — Forgejo only
|
|
shows them once.
|
|
|
|
Next: `docs/02-k3s.md` — you only need steps 1-2 above to start it.
|