docs: document homelab services, deployment, and repository review

This commit is contained in:
forust committed 2026-10-06 16:09:50 +02:00
1 parent cc9c3dea88
commit 3c4732ae20
44 files changed
+1354 -298

No files matched your search

+88
View File
@@ -0,0 +1,88 @@
# Build and deployment workflows
Gitea Actions checks this repository, builds its custom images, and deploys
selected services to the workstation. Workflows use the self-hosted runner labels
`linux`, `arch`, and `homelab`; deployment jobs also require `prod`.
## Checks
`ci.yaml` runs Compose validation, actionlint, ShellCheck, Prettier, Ruff,
yamllint, hadolint, and kubeconform. Tool versions are pinned in
`workflows/tool-versions.env` and installed by `install-ci-tools.sh`.
Compose CI checks structure without resolving local environment files or paths.
On the reviewed main commit it only discovers standard filenames; the
`fix/deploy-validation` branch adds the manual Compose entry points too.
Kubeconform validates known resource schemas. Unknown CRDs are skipped. On main,
CI also attempts server-side dry-runs for marked services; these require an
existing namespace and contact the cluster's admission webhooks. A cluster that
is unreachable produces a warning and skips that CI pass. Deploy validation has
its own dry-run stage.
`renovate-ci.yaml` validates Renovate settings and checks that its generated
ConfigMap matches `renovate/renovate.json`.
## Image builds
CI builds changed custom images for `errorpages`, both `homepages` variants, and
the two `edu_master` Python services. Main builds publish `main`, `prod`, and a
commit tag. Dev builds publish `dev`. Build jobs wait for the lint and manifest
checks.
Kubernetes deployment resolves the lab's own registry images to digests, preferring
commit-specific tags. Third-party image versions remain declared in the manifests.
## Deploy selection
`workflows/deploy-lib.sh` owns the stage logic; `ssh-run.sh` invokes it on the
workstation through SSH. Kubernetes selection uses `k8s/active`; Compose selection
uses an `active` file beside a standard `compose.yaml` or `compose.yml`.
Kustomize overlays are supported, although the current tree primarily contains
plain manifests.
Secret files, examples, Helm values, and patch files are excluded from plain
manifest selection. Create local Kubernetes Secrets separately in their target
namespaces. The Helm table lists Prometheus, Loki, Alloy, and Reloader, with each
release controlled by its configured marker. Other charts need separate setup.
## Trigger and required settings
Automatic deployment follows a successful main CI run when the repository Actions
variable `AUTODEPLOY` is `true`. The manual deploy workflow bypasses that switch
and targets the fetched main branch when no validated commit SHA is provided.
A manual dispatch does not prove that this commit passed CI.
Configure the Actions secrets `DEPLOY_HOST`, `DEPLOY_USER`, `DEPLOY_SSH_KEY`, and,
where needed, `DEPLOY_PORT` and `DEPLOY_PATH`. Registry publishing uses
`REGISTRY_USERNAME` and `REGISTRY_PASSWORD`. The remote user needs access to Git,
Docker, kubectl, Helm, jq, and the state directory used for snapshots.
Keep `APPLY_PRUNE` false on the reviewed implementation: its per-file prune loop
is unsafe. `fix/deploy-prune-guard` rejects that option before changes are applied.
Preflight fetches and resets the remote checkout. It refuses when tracked files
have local changes; ignored local env and Secret files stay in place. Do not use
a development checkout with uncommitted tracked changes as the deployment target.
## Stages and recovery
1. Preflight fetches the target commit and checks the remote working tree.
2. Validate selects services, parses Compose, performs Kubernetes dry-runs, and
checks referenced Secrets.
3. Apply Kubernetes records a workload snapshot, upgrades selected Helm releases,
applies resources, and refreshes owned custom images.
4. Apply Compose recreates marked stacks and checks container state.
5. Verify Kubernetes checks changed workloads and attempts rollback for failures.
6. Smoke probes public routes after verification.
The two apply jobs share a remote lock. Workflow concurrency queues deployments
rather than interrupting an older apply. Snapshots live under
`$XDG_STATE_HOME/homelab-deploy`, or `~/.local/state/homelab-deploy` by default.
They contain the pre-apply workload data and commit identifier.
Rollback uses workload revisions. It does not restore ConfigMaps, Secrets,
database schemas, or data. Helm-owned workloads are handled through the Helm
upgrade's rollback path; the generic rollback skips them. Compose has no automatic
rollback. See the [review](../docs/repository-review.md) for remaining recovery
limitations, including SSH retries and serial rollback timing.