Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
d1423c4192 |
No files matched your search
@@ -1,50 +0,0 @@
|
|||||||
# EDU ownership handoff
|
|
||||||
|
|
||||||
## Status
|
|
||||||
|
|
||||||
The EDU ownership handoff is complete. The homelab repository no longer owns
|
|
||||||
EDU workloads, images, routes, alerts, or deployment selection. The EDU
|
|
||||||
repository is the only deployment owner: [forust/edu-master](https://git.forust.xyz/forust/edu-master).
|
|
||||||
|
|
||||||
Homelab PRs #99 and #105 are merged. PR #105 removed the EDU subtree and its
|
|
||||||
build, deploy, rollback, verification, route-probe, and registry references.
|
|
||||||
It also added the serial image build matrix for the homelab services. This
|
|
||||||
handoff record is the only remaining EDU-specific file in homelab Git.
|
|
||||||
|
|
||||||
The dedicated workstation checkout is `/srv/edu-master`, at release
|
|
||||||
`4f2b2a0e37dc11ac2c75441a15076c178e219d37`. It contains `k8s/active`; root
|
|
||||||
`active` is absent. The old untracked `/srv/homelab/edu_master` checkout was
|
|
||||||
moved outside the homelab repository to
|
|
||||||
`/srv/edu-master-legacy-archive-20261007/edu_master`. Its private files remain
|
|
||||||
mode `0600` inside an archive directory with mode `0700`. The homelab deploy
|
|
||||||
checkout has no EDU marker or tracked EDU application/deployment files.
|
|
||||||
`AUTODEPLOY=false` remains in place for homelab deployment.
|
|
||||||
|
|
||||||
## Release evidence
|
|
||||||
|
|
||||||
EDU PR #4 merged after its review and CI checks. Main-push CI run 1652 passed
|
|
||||||
all validation and both image builds. Deploy run 1653 passed for the exact main
|
|
||||||
SHA above.
|
|
||||||
|
|
||||||
The workstation rollout completed for both Deployments. The deployment
|
|
||||||
verified `/health` and `/live` with HTTP 200, Redis AUTH, session TTL of 1058
|
|
||||||
seconds, a delivery backlog of zero, and all nine EDU vmalert rules with
|
|
||||||
matching expressions and healthy evaluation.
|
|
||||||
|
|
||||||
The images now run by digest:
|
|
||||||
|
|
||||||
- Session keeper: `sha256:998dea51aa3015fd9cabefb0f53b030157a650c3bef72e02fe84f17d5762613d`
|
|
||||||
- Webinar checker: `sha256:92f3c1fa2bb7f9b4680a9fc76a5b33dfbea8ef3dd9c6490ebc45876fd4c54461`
|
|
||||||
|
|
||||||
Redis StatefulSet was unchanged. PVC `redis-data-pvc` remains bound to PV
|
|
||||||
`pvc-a4f2a79a-363a-4c12-ae91-92cdfc2a0d2e` with capacity 1 GiB. The existing
|
|
||||||
runtime Secret and Fernet key were preserved during the handoff. Notification
|
|
||||||
delivery was verified before closeout, as confirmed by the operator. The
|
|
||||||
deployment did not record downtime.
|
|
||||||
|
|
||||||
The release rollback snapshot is
|
|
||||||
`/home/forust/.local/state/edu-master-deploy/20261007T180541Z-4f2b2a0e37dc11ac2c75441a15076c178e219d37`.
|
|
||||||
The handoff data snapshot remains at
|
|
||||||
`/home/forust/.local/state/edu-master-deploy/handoff-20261007T080838Z`.
|
|
||||||
Both snapshots are outside Git. Do not restore old Redis data unless recovery
|
|
||||||
requires it. Never delete or recreate the Redis PVC.
|
|
||||||
@@ -7,5 +7,4 @@ self-hosted-runner:
|
|||||||
labels:
|
labels:
|
||||||
- arch
|
- arch
|
||||||
- homelab
|
- homelab
|
||||||
- homelab-pr
|
|
||||||
- prod
|
- prod
|
||||||
+5
-42
@@ -1,20 +1,8 @@
|
|||||||
# Homelab CI/CD
|
# Homelab CI/CD
|
||||||
|
|
||||||
The native Gitea runners run on **vps**; production runs on **workstation**.
|
The native Gitea runner runs on **vps**; production runs on **workstation**.
|
||||||
Main-branch checks and image builds use `homelab:host`. Pull request and
|
Jobs run on `homelab:host`, one at a time. No job images or Kubernetes credentials
|
||||||
non-main checks use `homelab-pr:host` under a separate account without Docker
|
are needed on the VPS. Builds use one pinned BuildKit helper container. CI and deploy are separate workflows.
|
||||||
access. The `homelab-pr` runner is registered at User scope for `forust`, so
|
|
||||||
any repository under that account can schedule jobs that request this label.
|
|
||||||
Each runner accepts one job at a time; the build waits for every check to pass.
|
|
||||||
CI and deploy runs also show a summary with
|
|
||||||
the release SHA, image build or reuse results, deploy mode, selected services,
|
|
||||||
and image digests. Failed runs keep a summary of completed image builds, stage
|
|
||||||
results, apply results, and recorded Kubernetes recovery. The final deploy
|
|
||||||
summary is in the smoke job; earlier jobs show the state observed at that time.
|
|
||||||
Apply success is separate from health and recovery. Update the installed
|
|
||||||
workstation controller with `setup-workstation.sh` when no deploy is running.
|
|
||||||
No job images or Kubernetes credentials are needed on the VPS. Builds use one
|
|
||||||
pinned BuildKit helper container. CI and deploy are separate workflows.
|
|
||||||
|
|
||||||
## Runner installation
|
## Runner installation
|
||||||
|
|
||||||
@@ -40,32 +28,6 @@ pushes directly to the registry, and caps retained local cache at 1 GiB with a
|
|||||||
2 GiB free-space target. This is not a hard limit on peak build disk usage.
|
2 GiB free-space target. This is not a hard limit on peak build disk usage.
|
||||||
Nothing runs `docker system prune`, removes unrelated images, or deletes volumes.
|
Nothing runs `docker system prune`, removes unrelated images, or deletes volumes.
|
||||||
|
|
||||||
### Pull request runner
|
|
||||||
|
|
||||||
Install the unprivileged host runner on the VPS:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
sudo bash .gitea/runner/setup-pr-runner.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
Get a registration token from the user Actions runner settings. Run the
|
|
||||||
installer in a terminal. It asks for the token without echoing it, registers the
|
|
||||||
runner as `homelab-pr` with label `homelab-pr:host`, then enables the service.
|
|
||||||
The work directory is `/var/lib/gitea-pr-runner`. Confirm that Gitea lists the
|
|
||||||
runner as User scope before merging the workflow change. An unmatched label can
|
|
||||||
fall back to the default job image.
|
|
||||||
|
|
||||||
Renovate PR validation uses `pull_request_target`, which reads the workflow from
|
|
||||||
the base branch. It checks out the PR head only after runner selection and runs
|
|
||||||
that code on `homelab-pr`. Keep this workflow read-only and do not add secrets.
|
|
||||||
|
|
||||||
The PR runner has a separate home and tool cache. Do not add it to the `docker`
|
|
||||||
group or give it access to `/var/run/docker.sock`. It runs repository code from
|
|
||||||
pull requests, so keep its registration and permissions separate from the
|
|
||||||
trusted `homelab` runner. This separates users and host permissions, but both
|
|
||||||
runners still share the VPS kernel and network. Use a disposable VM if PRs from
|
|
||||||
untrusted external authors must be fully isolated.
|
|
||||||
|
|
||||||
## Workstation setup
|
## Workstation setup
|
||||||
|
|
||||||
As the existing SSH deploy user on workstation:
|
As the existing SSH deploy user on workstation:
|
||||||
@@ -96,7 +58,8 @@ The deploy user's existing Docker registry authentication remains necessary.
|
|||||||
|
|
||||||
CI publishes `release-<full SHA>` as a Gitea artifact with all three owned image
|
CI publishes `release-<full SHA>` as a Gitea artifact with all three owned image
|
||||||
digests and build input fingerprints. Unchanged images are reused only from a
|
digests and build input fingerprints. Unchanged images are reused only from a
|
||||||
successful main CI artifact, never from `:prod`. Expired artifacts cause CI to
|
successful main CI artifact, never from `:prod`. EDU images remain pinned to the
|
||||||
|
digests released by their application repository. Expired artifacts cause CI to
|
||||||
rebuild images; they block deployment until CI is rerun.
|
rebuild images; they block deployment until CI is rerun.
|
||||||
|
|
||||||
Run deploy from main with `deploy_ref=main` or a checked SHA:
|
Run deploy from main with `deploy_ref=main` or a checked SHA:
|
||||||
|
|||||||
@@ -1,8 +0,0 @@
|
|||||||
runner:
|
|
||||||
file: /var/lib/gitea-pr-runner/.runner
|
|
||||||
capacity: 1
|
|
||||||
timeout: 5h
|
|
||||||
labels:
|
|
||||||
- homelab-pr:host
|
|
||||||
cache:
|
|
||||||
enabled: false
|
|
||||||
@@ -1,27 +0,0 @@
|
|||||||
[Unit]
|
|
||||||
Description=Gitea Actions untrusted pull request runner
|
|
||||||
After=network-online.target
|
|
||||||
Wants=network-online.target
|
|
||||||
|
|
||||||
[Service]
|
|
||||||
User=gitea-pr-runner
|
|
||||||
Group=gitea-pr-runner
|
|
||||||
WorkingDirectory=/var/lib/gitea-pr-runner
|
|
||||||
Environment=HOME=/var/lib/gitea-pr-runner
|
|
||||||
Environment=PATH=/var/lib/gitea-pr-runner/.cache/homelab-ci/bin:/usr/local/bin:/usr/bin:/bin
|
|
||||||
ExecStart=/usr/local/bin/gitea-runner daemon --config /etc/gitea-pr-runner/config.yaml
|
|
||||||
Restart=on-failure
|
|
||||||
RestartSec=5
|
|
||||||
NoNewPrivileges=yes
|
|
||||||
PrivateTmp=yes
|
|
||||||
ProtectSystem=full
|
|
||||||
ProtectHome=yes
|
|
||||||
ProtectKernelTunables=yes
|
|
||||||
ProtectKernelModules=yes
|
|
||||||
ProtectControlGroups=yes
|
|
||||||
RestrictSUIDSGID=yes
|
|
||||||
LockPersonality=yes
|
|
||||||
UMask=0077
|
|
||||||
|
|
||||||
[Install]
|
|
||||||
WantedBy=multi-user.target
|
|
||||||
@@ -1,56 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
# Install a native runner for untrusted PR jobs without Docker access.
|
|
||||||
set -euo pipefail
|
|
||||||
here="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
||||||
[ "$(id -u)" -eq 0 ] || { echo 'Run with sudo on the runner host' >&2; exit 1; }
|
|
||||||
for tool in cp cut date getent id install runuser systemctl useradd; do
|
|
||||||
command -v "$tool" >/dev/null || { echo "Install missing prerequisite: $tool" >&2; exit 1; }
|
|
||||||
done
|
|
||||||
command -v /usr/local/bin/gitea-runner >/dev/null || {
|
|
||||||
echo 'Install gitea-runner 3.0.2 at /usr/local/bin/gitea-runner first' >&2
|
|
||||||
exit 1
|
|
||||||
}
|
|
||||||
|
|
||||||
id gitea-pr-runner >/dev/null 2>&1 || \
|
|
||||||
useradd --system --create-home --home-dir /var/lib/gitea-pr-runner --shell /usr/bin/bash gitea-pr-runner
|
|
||||||
runner_home="$(getent passwd gitea-pr-runner | cut -d: -f6)"
|
|
||||||
[ "$runner_home" = /var/lib/gitea-pr-runner ] || {
|
|
||||||
echo 'Unexpected PR runner home; inspect the existing service first' >&2
|
|
||||||
exit 1
|
|
||||||
}
|
|
||||||
case " $(id -nG gitea-pr-runner) " in
|
|
||||||
*' docker '*)
|
|
||||||
echo 'The PR runner account must not belong to the docker group' >&2
|
|
||||||
exit 1
|
|
||||||
;;
|
|
||||||
esac
|
|
||||||
|
|
||||||
install -d -m 0755 /etc/gitea-pr-runner
|
|
||||||
stamp="$(date -u +%Y%m%dT%H%M%SZ)"
|
|
||||||
for existing in /etc/gitea-pr-runner/config.yaml /etc/systemd/system/gitea-pr-runner.service; do
|
|
||||||
[ ! -f "$existing" ] || cp -p "$existing" "$existing.before-$stamp"
|
|
||||||
done
|
|
||||||
install -m 0644 "$here/pr-config.yaml" /etc/gitea-pr-runner/config.yaml
|
|
||||||
install -m 0644 "$here/pr-runner.service" /etc/systemd/system/gitea-pr-runner.service
|
|
||||||
|
|
||||||
if [ ! -f /var/lib/gitea-pr-runner/.runner ]; then
|
|
||||||
read -r -s -p 'Enter the Gitea repository runner registration token: ' runner_token
|
|
||||||
printf '\n'
|
|
||||||
[ -n "$runner_token" ] || { echo 'Runner token is required' >&2; exit 1; }
|
|
||||||
export GITEA_RUNNER_REGISTRATION_TOKEN="$runner_token"
|
|
||||||
unset runner_token
|
|
||||||
runuser --preserve-environment -u gitea-pr-runner -- \
|
|
||||||
/usr/local/bin/gitea-runner register \
|
|
||||||
--config /etc/gitea-pr-runner/config.yaml \
|
|
||||||
--instance https://gitea.forust.xyz \
|
|
||||||
--name homelab-pr \
|
|
||||||
--labels homelab-pr:host \
|
|
||||||
--no-interactive
|
|
||||||
unset GITEA_RUNNER_REGISTRATION_TOKEN
|
|
||||||
fi
|
|
||||||
chmod 0600 /var/lib/gitea-pr-runner/.runner
|
|
||||||
|
|
||||||
systemctl daemon-reload
|
|
||||||
systemctl enable --now gitea-pr-runner.service
|
|
||||||
systemctl restart gitea-pr-runner.service
|
|
||||||
echo "PR runner ready. Configuration backups: *.before-$stamp"
|
|
||||||
+16
-364
@@ -12,14 +12,18 @@ concurrency:
|
|||||||
group: ci-${{ github.ref }}
|
group: ci-${{ github.ref }}
|
||||||
cancel-in-progress: ${{ github.ref != 'refs/heads/main' }}
|
cancel-in-progress: ${{ github.ref != 'refs/heads/main' }}
|
||||||
jobs:
|
jobs:
|
||||||
compose:
|
checks:
|
||||||
name: Compose
|
runs-on: homelab
|
||||||
runs-on: ${{ github.ref == 'refs/heads/main' && 'homelab' || 'homelab-pr' }}
|
timeout-minutes: 30
|
||||||
timeout-minutes: 15
|
|
||||||
steps:
|
steps:
|
||||||
- name: Checkout repository
|
- name: Checkout repository
|
||||||
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262
|
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262
|
||||||
id: source
|
- name: Prepare pinned tools
|
||||||
|
shell: bash
|
||||||
|
run: |
|
||||||
|
set -euo pipefail
|
||||||
|
tools_dir="$(bash .gitea/workflows/install-ci-tools.sh)"
|
||||||
|
echo "$tools_dir" >> "$GITHUB_PATH"
|
||||||
- name: Validate Compose files
|
- name: Validate Compose files
|
||||||
shell: bash
|
shell: bash
|
||||||
run: |
|
run: |
|
||||||
@@ -48,73 +52,11 @@ jobs:
|
|||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
echo "checked ${#files[@]} Compose file(s)"
|
echo "checked ${#files[@]} Compose file(s)"
|
||||||
id: check
|
|
||||||
- name: Write the job result
|
|
||||||
if: always()
|
|
||||||
env:
|
|
||||||
SUMMARY_CHECK: Compose
|
|
||||||
SUMMARY_RESULT: ${{ job.status }}
|
|
||||||
SUMMARY_FAILED_STEP:
|
|
||||||
${{ steps.check.conclusion == 'failure' && 'Check or image build' || steps.source.conclusion == 'failure'
|
|
||||||
&& 'Source checkout' || '' }}
|
|
||||||
shell: bash
|
|
||||||
run: |
|
|
||||||
if [ -f .gitea/workflows/release.py ]; then
|
|
||||||
python3 .gitea/workflows/release.py check-summary
|
|
||||||
elif [ -n "${GITHUB_STEP_SUMMARY:-}" ]; then
|
|
||||||
printf '## %s\n\n- Result: **%s**\n- Failed step: %s\n' "$SUMMARY_CHECK" "$SUMMARY_RESULT" "$SUMMARY_FAILED_STEP" >>"$GITHUB_STEP_SUMMARY" || true
|
|
||||||
fi
|
|
||||||
workflows:
|
|
||||||
name: Workflows
|
|
||||||
runs-on: ${{ github.ref == 'refs/heads/main' && 'homelab' || 'homelab-pr' }}
|
|
||||||
timeout-minutes: 15
|
|
||||||
steps:
|
|
||||||
- name: Checkout repository
|
|
||||||
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262
|
|
||||||
id: source
|
|
||||||
- name: Prepare pinned tools
|
|
||||||
shell: bash
|
|
||||||
run: |
|
|
||||||
set -euo pipefail
|
|
||||||
tools_dir="$(bash .gitea/workflows/install-ci-tools.sh actionlint shellcheck)"
|
|
||||||
echo "$tools_dir" >> "$GITHUB_PATH"
|
|
||||||
id: tools
|
|
||||||
- name: Lint Gitea Actions workflows with actionlint
|
- name: Lint Gitea Actions workflows with actionlint
|
||||||
shell: bash
|
shell: bash
|
||||||
run: |
|
run: |
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
actionlint -config-file .gitea/actionlint.yaml -color .gitea/workflows/*.yaml
|
actionlint -config-file .gitea/actionlint.yaml -color .gitea/workflows/*.yaml
|
||||||
id: check
|
|
||||||
- name: Write the job result
|
|
||||||
if: always()
|
|
||||||
env:
|
|
||||||
SUMMARY_CHECK: Workflows
|
|
||||||
SUMMARY_RESULT: ${{ job.status }}
|
|
||||||
SUMMARY_FAILED_STEP:
|
|
||||||
${{ steps.check.conclusion == 'failure' && 'Check or image build' || steps.tools.conclusion == 'failure'
|
|
||||||
&& 'Tool setup' || steps.source.conclusion == 'failure' && 'Source checkout' || '' }}
|
|
||||||
shell: bash
|
|
||||||
run: |
|
|
||||||
if [ -f .gitea/workflows/release.py ]; then
|
|
||||||
python3 .gitea/workflows/release.py check-summary
|
|
||||||
elif [ -n "${GITHUB_STEP_SUMMARY:-}" ]; then
|
|
||||||
printf '## %s\n\n- Result: **%s**\n- Failed step: %s\n' "$SUMMARY_CHECK" "$SUMMARY_RESULT" "$SUMMARY_FAILED_STEP" >>"$GITHUB_STEP_SUMMARY" || true
|
|
||||||
fi
|
|
||||||
shell:
|
|
||||||
name: Shell
|
|
||||||
runs-on: ${{ github.ref == 'refs/heads/main' && 'homelab' || 'homelab-pr' }}
|
|
||||||
timeout-minutes: 15
|
|
||||||
steps:
|
|
||||||
- name: Checkout repository
|
|
||||||
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262
|
|
||||||
id: source
|
|
||||||
- name: Prepare pinned tools
|
|
||||||
shell: bash
|
|
||||||
run: |
|
|
||||||
set -euo pipefail
|
|
||||||
tools_dir="$(bash .gitea/workflows/install-ci-tools.sh shellcheck jq)"
|
|
||||||
echo "$tools_dir" >> "$GITHUB_PATH"
|
|
||||||
id: tools
|
|
||||||
- name: Lint shell scripts with ShellCheck
|
- name: Lint shell scripts with ShellCheck
|
||||||
shell: bash
|
shell: bash
|
||||||
run: |
|
run: |
|
||||||
@@ -128,37 +70,6 @@ jobs:
|
|||||||
fi
|
fi
|
||||||
shellcheck --external-sources --source-path=SCRIPTDIR --severity=style "${scripts[@]}"
|
shellcheck --external-sources --source-path=SCRIPTDIR --severity=style "${scripts[@]}"
|
||||||
bash .gitea/tests/deploy-validation.sh
|
bash .gitea/tests/deploy-validation.sh
|
||||||
id: check
|
|
||||||
- name: Write the job result
|
|
||||||
if: always()
|
|
||||||
env:
|
|
||||||
SUMMARY_CHECK: Shell
|
|
||||||
SUMMARY_RESULT: ${{ job.status }}
|
|
||||||
SUMMARY_FAILED_STEP:
|
|
||||||
${{ steps.check.conclusion == 'failure' && 'Check or image build' || steps.tools.conclusion == 'failure'
|
|
||||||
&& 'Tool setup' || steps.source.conclusion == 'failure' && 'Source checkout' || '' }}
|
|
||||||
shell: bash
|
|
||||||
run: |
|
|
||||||
if [ -f .gitea/workflows/release.py ]; then
|
|
||||||
python3 .gitea/workflows/release.py check-summary
|
|
||||||
elif [ -n "${GITHUB_STEP_SUMMARY:-}" ]; then
|
|
||||||
printf '## %s\n\n- Result: **%s**\n- Failed step: %s\n' "$SUMMARY_CHECK" "$SUMMARY_RESULT" "$SUMMARY_FAILED_STEP" >>"$GITHUB_STEP_SUMMARY" || true
|
|
||||||
fi
|
|
||||||
formatting:
|
|
||||||
name: Formatting
|
|
||||||
runs-on: ${{ github.ref == 'refs/heads/main' && 'homelab' || 'homelab-pr' }}
|
|
||||||
timeout-minutes: 15
|
|
||||||
steps:
|
|
||||||
- name: Checkout repository
|
|
||||||
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262
|
|
||||||
id: source
|
|
||||||
- name: Prepare pinned tools
|
|
||||||
shell: bash
|
|
||||||
run: |
|
|
||||||
set -euo pipefail
|
|
||||||
tools_dir="$(bash .gitea/workflows/install-ci-tools.sh prettier)"
|
|
||||||
echo "$tools_dir" >> "$GITHUB_PATH"
|
|
||||||
id: tools
|
|
||||||
- name: Check formatting with Prettier
|
- name: Check formatting with Prettier
|
||||||
shell: bash
|
shell: bash
|
||||||
run: |
|
run: |
|
||||||
@@ -176,37 +87,6 @@ jobs:
|
|||||||
fi
|
fi
|
||||||
|
|
||||||
prettier --check --ignore-unknown "${prettier_files[@]}"
|
prettier --check --ignore-unknown "${prettier_files[@]}"
|
||||||
id: check
|
|
||||||
- name: Write the job result
|
|
||||||
if: always()
|
|
||||||
env:
|
|
||||||
SUMMARY_CHECK: Formatting
|
|
||||||
SUMMARY_RESULT: ${{ job.status }}
|
|
||||||
SUMMARY_FAILED_STEP:
|
|
||||||
${{ steps.check.conclusion == 'failure' && 'Check or image build' || steps.tools.conclusion == 'failure'
|
|
||||||
&& 'Tool setup' || steps.source.conclusion == 'failure' && 'Source checkout' || '' }}
|
|
||||||
shell: bash
|
|
||||||
run: |
|
|
||||||
if [ -f .gitea/workflows/release.py ]; then
|
|
||||||
python3 .gitea/workflows/release.py check-summary
|
|
||||||
elif [ -n "${GITHUB_STEP_SUMMARY:-}" ]; then
|
|
||||||
printf '## %s\n\n- Result: **%s**\n- Failed step: %s\n' "$SUMMARY_CHECK" "$SUMMARY_RESULT" "$SUMMARY_FAILED_STEP" >>"$GITHUB_STEP_SUMMARY" || true
|
|
||||||
fi
|
|
||||||
python:
|
|
||||||
name: Python and tests
|
|
||||||
runs-on: ${{ github.ref == 'refs/heads/main' && 'homelab' || 'homelab-pr' }}
|
|
||||||
timeout-minutes: 15
|
|
||||||
steps:
|
|
||||||
- name: Checkout repository
|
|
||||||
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262
|
|
||||||
id: source
|
|
||||||
- name: Prepare pinned tools
|
|
||||||
shell: bash
|
|
||||||
run: |
|
|
||||||
set -euo pipefail
|
|
||||||
tools_dir="$(bash .gitea/workflows/install-ci-tools.sh ruff jq)"
|
|
||||||
echo "$tools_dir" >> "$GITHUB_PATH"
|
|
||||||
id: tools
|
|
||||||
- name: Lint and format-check Python with Ruff
|
- name: Lint and format-check Python with Ruff
|
||||||
shell: bash
|
shell: bash
|
||||||
run: |
|
run: |
|
||||||
@@ -214,37 +94,6 @@ jobs:
|
|||||||
ruff check . .gitea/workflows
|
ruff check . .gitea/workflows
|
||||||
ruff format --check . .gitea/workflows
|
ruff format --check . .gitea/workflows
|
||||||
python3 -m unittest discover -s tests -v
|
python3 -m unittest discover -s tests -v
|
||||||
id: check
|
|
||||||
- name: Write the job result
|
|
||||||
if: always()
|
|
||||||
env:
|
|
||||||
SUMMARY_CHECK: Python and tests
|
|
||||||
SUMMARY_RESULT: ${{ job.status }}
|
|
||||||
SUMMARY_FAILED_STEP:
|
|
||||||
${{ steps.check.conclusion == 'failure' && 'Check or image build' || steps.tools.conclusion == 'failure'
|
|
||||||
&& 'Tool setup' || steps.source.conclusion == 'failure' && 'Source checkout' || '' }}
|
|
||||||
shell: bash
|
|
||||||
run: |
|
|
||||||
if [ -f .gitea/workflows/release.py ]; then
|
|
||||||
python3 .gitea/workflows/release.py check-summary
|
|
||||||
elif [ -n "${GITHUB_STEP_SUMMARY:-}" ]; then
|
|
||||||
printf '## %s\n\n- Result: **%s**\n- Failed step: %s\n' "$SUMMARY_CHECK" "$SUMMARY_RESULT" "$SUMMARY_FAILED_STEP" >>"$GITHUB_STEP_SUMMARY" || true
|
|
||||||
fi
|
|
||||||
yaml:
|
|
||||||
name: YAML
|
|
||||||
runs-on: ${{ github.ref == 'refs/heads/main' && 'homelab' || 'homelab-pr' }}
|
|
||||||
timeout-minutes: 15
|
|
||||||
steps:
|
|
||||||
- name: Checkout repository
|
|
||||||
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262
|
|
||||||
id: source
|
|
||||||
- name: Prepare pinned tools
|
|
||||||
shell: bash
|
|
||||||
run: |
|
|
||||||
set -euo pipefail
|
|
||||||
tools_dir="$(bash .gitea/workflows/install-ci-tools.sh yamllint)"
|
|
||||||
echo "$tools_dir" >> "$GITHUB_PATH"
|
|
||||||
id: tools
|
|
||||||
- name: Lint YAML syntax
|
- name: Lint YAML syntax
|
||||||
shell: bash
|
shell: bash
|
||||||
run: |
|
run: |
|
||||||
@@ -262,37 +111,6 @@ jobs:
|
|||||||
fi
|
fi
|
||||||
|
|
||||||
yamllint -c .yamllint "${yaml_files[@]}"
|
yamllint -c .yamllint "${yaml_files[@]}"
|
||||||
id: check
|
|
||||||
- name: Write the job result
|
|
||||||
if: always()
|
|
||||||
env:
|
|
||||||
SUMMARY_CHECK: YAML
|
|
||||||
SUMMARY_RESULT: ${{ job.status }}
|
|
||||||
SUMMARY_FAILED_STEP:
|
|
||||||
${{ steps.check.conclusion == 'failure' && 'Check or image build' || steps.tools.conclusion == 'failure'
|
|
||||||
&& 'Tool setup' || steps.source.conclusion == 'failure' && 'Source checkout' || '' }}
|
|
||||||
shell: bash
|
|
||||||
run: |
|
|
||||||
if [ -f .gitea/workflows/release.py ]; then
|
|
||||||
python3 .gitea/workflows/release.py check-summary
|
|
||||||
elif [ -n "${GITHUB_STEP_SUMMARY:-}" ]; then
|
|
||||||
printf '## %s\n\n- Result: **%s**\n- Failed step: %s\n' "$SUMMARY_CHECK" "$SUMMARY_RESULT" "$SUMMARY_FAILED_STEP" >>"$GITHUB_STEP_SUMMARY" || true
|
|
||||||
fi
|
|
||||||
dockerfiles:
|
|
||||||
name: Dockerfiles
|
|
||||||
runs-on: ${{ github.ref == 'refs/heads/main' && 'homelab' || 'homelab-pr' }}
|
|
||||||
timeout-minutes: 15
|
|
||||||
steps:
|
|
||||||
- name: Checkout repository
|
|
||||||
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262
|
|
||||||
id: source
|
|
||||||
- name: Prepare pinned tools
|
|
||||||
shell: bash
|
|
||||||
run: |
|
|
||||||
set -euo pipefail
|
|
||||||
tools_dir="$(bash .gitea/workflows/install-ci-tools.sh hadolint)"
|
|
||||||
echo "$tools_dir" >> "$GITHUB_PATH"
|
|
||||||
id: tools
|
|
||||||
- name: Lint Dockerfiles
|
- name: Lint Dockerfiles
|
||||||
shell: bash
|
shell: bash
|
||||||
run: |
|
run: |
|
||||||
@@ -308,37 +126,6 @@ jobs:
|
|||||||
fi
|
fi
|
||||||
|
|
||||||
hadolint -c .hadolint.yaml "${dockerfiles[@]}"
|
hadolint -c .hadolint.yaml "${dockerfiles[@]}"
|
||||||
id: check
|
|
||||||
- name: Write the job result
|
|
||||||
if: always()
|
|
||||||
env:
|
|
||||||
SUMMARY_CHECK: Dockerfiles
|
|
||||||
SUMMARY_RESULT: ${{ job.status }}
|
|
||||||
SUMMARY_FAILED_STEP:
|
|
||||||
${{ steps.check.conclusion == 'failure' && 'Check or image build' || steps.tools.conclusion == 'failure'
|
|
||||||
&& 'Tool setup' || steps.source.conclusion == 'failure' && 'Source checkout' || '' }}
|
|
||||||
shell: bash
|
|
||||||
run: |
|
|
||||||
if [ -f .gitea/workflows/release.py ]; then
|
|
||||||
python3 .gitea/workflows/release.py check-summary
|
|
||||||
elif [ -n "${GITHUB_STEP_SUMMARY:-}" ]; then
|
|
||||||
printf '## %s\n\n- Result: **%s**\n- Failed step: %s\n' "$SUMMARY_CHECK" "$SUMMARY_RESULT" "$SUMMARY_FAILED_STEP" >>"$GITHUB_STEP_SUMMARY" || true
|
|
||||||
fi
|
|
||||||
kubernetes:
|
|
||||||
name: Kubernetes
|
|
||||||
runs-on: ${{ github.ref == 'refs/heads/main' && 'homelab' || 'homelab-pr' }}
|
|
||||||
timeout-minutes: 15
|
|
||||||
steps:
|
|
||||||
- name: Checkout repository
|
|
||||||
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262
|
|
||||||
id: source
|
|
||||||
- name: Prepare pinned tools
|
|
||||||
shell: bash
|
|
||||||
run: |
|
|
||||||
set -euo pipefail
|
|
||||||
tools_dir="$(bash .gitea/workflows/install-ci-tools.sh kubeconform)"
|
|
||||||
echo "$tools_dir" >> "$GITHUB_PATH"
|
|
||||||
id: tools
|
|
||||||
- name: Validate Kubernetes manifests against JSON schemas
|
- name: Validate Kubernetes manifests against JSON schemas
|
||||||
shell: bash
|
shell: bash
|
||||||
run: |
|
run: |
|
||||||
@@ -359,162 +146,27 @@ jobs:
|
|||||||
-ignore-missing-schemas \
|
-ignore-missing-schemas \
|
||||||
-summary \
|
-summary \
|
||||||
"${manifests[@]}"
|
"${manifests[@]}"
|
||||||
id: check
|
build:
|
||||||
- name: Write the job result
|
needs:
|
||||||
if: always()
|
- checks
|
||||||
env:
|
|
||||||
SUMMARY_CHECK: Kubernetes
|
|
||||||
SUMMARY_RESULT: ${{ job.status }}
|
|
||||||
SUMMARY_FAILED_STEP:
|
|
||||||
${{ steps.check.conclusion == 'failure' && 'Check or image build' || steps.tools.conclusion == 'failure'
|
|
||||||
&& 'Tool setup' || steps.source.conclusion == 'failure' && 'Source checkout' || '' }}
|
|
||||||
shell: bash
|
|
||||||
run: |
|
|
||||||
if [ -f .gitea/workflows/release.py ]; then
|
|
||||||
python3 .gitea/workflows/release.py check-summary
|
|
||||||
elif [ -n "${GITHUB_STEP_SUMMARY:-}" ]; then
|
|
||||||
printf '## %s\n\n- Result: **%s**\n- Failed step: %s\n' "$SUMMARY_CHECK" "$SUMMARY_RESULT" "$SUMMARY_FAILED_STEP" >>"$GITHUB_STEP_SUMMARY" || true
|
|
||||||
fi
|
|
||||||
image-plan:
|
|
||||||
needs: [compose, workflows, shell, formatting, python, yaml, dockerfiles, kubernetes]
|
|
||||||
if: github.event_name != 'pull_request' && github.ref == 'refs/heads/main'
|
if: github.event_name != 'pull_request' && github.ref == 'refs/heads/main'
|
||||||
runs-on: homelab
|
runs-on: homelab
|
||||||
timeout-minutes: 10
|
timeout-minutes: 60
|
||||||
outputs:
|
|
||||||
matrix: ${{ steps.plan.outputs.matrix }}
|
|
||||||
steps:
|
steps:
|
||||||
- name: Checkout repository
|
- name: Checkout repository
|
||||||
id: source
|
|
||||||
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262
|
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262
|
||||||
with:
|
with:
|
||||||
fetch-depth: 0
|
fetch-depth: 0
|
||||||
- name: Detect build inputs against successful CI
|
- name: Build changed images and write release
|
||||||
id: plan
|
|
||||||
env:
|
env:
|
||||||
GITEA_TOKEN: ${{ github.token }}
|
GITEA_TOKEN: ${{ github.token }}
|
||||||
run: python3 .gitea/workflows/release.py prepare --output build-plan.json
|
|
||||||
- name: Store the image plan
|
|
||||||
id: artifact
|
|
||||||
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
|
||||||
with:
|
|
||||||
name: build-plan
|
|
||||||
path: build-plan.json
|
|
||||||
if-no-files-found: error
|
|
||||||
retention-days: 30
|
|
||||||
|
|
||||||
- name: Write the plan result
|
|
||||||
if: always()
|
|
||||||
env:
|
|
||||||
SUMMARY_CHECK: Image plan
|
|
||||||
SUMMARY_RESULT: ${{ job.status }}
|
|
||||||
SUMMARY_FAILED_STEP: >-
|
|
||||||
${{ steps.plan.conclusion == 'failure' && 'Build input detection' ||
|
|
||||||
steps.artifact.conclusion == 'failure' && 'Plan upload' ||
|
|
||||||
steps.source.conclusion == 'failure' && 'Source checkout' || '' }}
|
|
||||||
shell: bash
|
|
||||||
run: |
|
|
||||||
if [ -f .gitea/workflows/release.py ]; then
|
|
||||||
python3 .gitea/workflows/release.py check-summary
|
|
||||||
elif [ -n "${GITHUB_STEP_SUMMARY:-}" ]; then
|
|
||||||
printf '## Image plan\n\nResult: %s\n' "$SUMMARY_RESULT" >>"$GITHUB_STEP_SUMMARY" || true
|
|
||||||
fi
|
|
||||||
|
|
||||||
images:
|
|
||||||
name: Image (${{ matrix.name }})
|
|
||||||
needs: [image-plan]
|
|
||||||
if: needs.image-plan.result == 'success'
|
|
||||||
runs-on: homelab
|
|
||||||
timeout-minutes: 60
|
|
||||||
strategy:
|
|
||||||
max-parallel: 1
|
|
||||||
fail-fast: false
|
|
||||||
matrix: ${{ fromJSON(needs.image-plan.outputs.matrix || '{"include":[{"name":"inactive"}]}') }}
|
|
||||||
steps:
|
|
||||||
- name: Checkout repository
|
|
||||||
id: source
|
|
||||||
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262
|
|
||||||
- name: Download the checked image plan
|
|
||||||
id: inputs
|
|
||||||
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
|
|
||||||
with:
|
|
||||||
name: build-plan
|
|
||||||
- name: Build or reuse this image
|
|
||||||
id: check
|
|
||||||
env:
|
|
||||||
IMAGE_NAME: ${{ matrix.name }}
|
|
||||||
REGISTRY_USERNAME: ${{ secrets.REGISTRY_USERNAME }}
|
REGISTRY_USERNAME: ${{ secrets.REGISTRY_USERNAME }}
|
||||||
REGISTRY_PASSWORD: ${{ secrets.REGISTRY_PASSWORD }}
|
REGISTRY_PASSWORD: ${{ secrets.REGISTRY_PASSWORD }}
|
||||||
run: python3 .gitea/workflows/release.py image --image "$IMAGE_NAME" --output image.json
|
run: python3 .gitea/workflows/release.py build
|
||||||
- name: Store the image result
|
|
||||||
id: artifact
|
|
||||||
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
|
||||||
with:
|
|
||||||
name: image-${{ matrix.name }}
|
|
||||||
path: image.json
|
|
||||||
if-no-files-found: error
|
|
||||||
retention-days: 30
|
|
||||||
- name: Write the job result
|
|
||||||
if: always()
|
|
||||||
env:
|
|
||||||
SUMMARY_CHECK: Image (${{ matrix.name }})
|
|
||||||
SUMMARY_RESULT: ${{ job.status }}
|
|
||||||
SUMMARY_FAILED_STEP: >-
|
|
||||||
${{ steps.check.conclusion == 'failure' && 'Build or tag images' ||
|
|
||||||
steps.artifact.conclusion == 'failure' && 'Artifact upload' ||
|
|
||||||
steps.inputs.conclusion == 'failure' && 'Artifact download' ||
|
|
||||||
steps.source.conclusion == 'failure' && 'Source checkout' || '' }}
|
|
||||||
shell: bash
|
|
||||||
run: |
|
|
||||||
if [ -f .gitea/workflows/release.py ]; then
|
|
||||||
python3 .gitea/workflows/release.py check-summary
|
|
||||||
elif [ -n "${GITHUB_STEP_SUMMARY:-}" ]; then
|
|
||||||
printf '## %s\n\n- Result: **%s**\n- Failed step: %s\n' "$SUMMARY_CHECK" "$SUMMARY_RESULT" "$SUMMARY_FAILED_STEP" >>"$GITHUB_STEP_SUMMARY" || true
|
|
||||||
fi
|
|
||||||
|
|
||||||
# Retain the build job name required by the immutable release deployment gate.
|
|
||||||
build:
|
|
||||||
needs: [image-plan, images]
|
|
||||||
runs-on: homelab
|
|
||||||
timeout-minutes: 15
|
|
||||||
steps:
|
|
||||||
- name: Checkout repository
|
|
||||||
id: source
|
|
||||||
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262
|
|
||||||
- name: Download all image results
|
|
||||||
id: inputs
|
|
||||||
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
|
|
||||||
with:
|
|
||||||
path: artifacts
|
|
||||||
- name: Pin SHA tags and write the complete release
|
|
||||||
id: check
|
|
||||||
env:
|
|
||||||
REGISTRY_USERNAME: ${{ secrets.REGISTRY_USERNAME }}
|
|
||||||
REGISTRY_PASSWORD: ${{ secrets.REGISTRY_PASSWORD }}
|
|
||||||
run: >-
|
|
||||||
python3 .gitea/workflows/release.py finalize
|
|
||||||
--plan artifacts/build-plan/build-plan.json
|
|
||||||
- name: Store commit release
|
- name: Store commit release
|
||||||
id: artifact
|
uses: actions/upload-artifact@c6a366c94c3e0affe28c06c8df20a878f24da3cf
|
||||||
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
|
||||||
with:
|
with:
|
||||||
name: release-${{ github.sha }}
|
name: release-${{ github.sha }}
|
||||||
path: release.json
|
path: release.json
|
||||||
if-no-files-found: error
|
if-no-files-found: error
|
||||||
retention-days: 30
|
retention-days: 30
|
||||||
- name: Write the job result
|
|
||||||
if: always()
|
|
||||||
env:
|
|
||||||
SUMMARY_CHECK: Image release and SHA tags
|
|
||||||
SUMMARY_RESULT: ${{ job.status }}
|
|
||||||
SUMMARY_FAILED_STEP: >-
|
|
||||||
${{ steps.check.conclusion == 'failure' && 'Build or tag images' ||
|
|
||||||
steps.artifact.conclusion == 'failure' && 'Artifact upload' ||
|
|
||||||
steps.inputs.conclusion == 'failure' && 'Artifact download' ||
|
|
||||||
steps.source.conclusion == 'failure' && 'Source checkout' || '' }}
|
|
||||||
shell: bash
|
|
||||||
run: |
|
|
||||||
if [ -f .gitea/workflows/release.py ]; then
|
|
||||||
python3 .gitea/workflows/release.py check-summary
|
|
||||||
elif [ -n "${GITHUB_STEP_SUMMARY:-}" ]; then
|
|
||||||
printf '## %s\n\n- Result: **%s**\n- Failed step: %s\n' "$SUMMARY_CHECK" "$SUMMARY_RESULT" "$SUMMARY_FAILED_STEP" >>"$GITHUB_STEP_SUMMARY" || true
|
|
||||||
fi
|
|
||||||
@@ -45,17 +45,12 @@ def prepare(source_file):
|
|||||||
before = json.loads(json.dumps(config))
|
before = json.loads(json.dumps(config))
|
||||||
for service, settings in config['services'].items():
|
for service, settings in config['services'].items():
|
||||||
reference = settings.get('image')
|
reference = settings.get('image')
|
||||||
nextcloud_aio_master = project == 'nextcloud' and service == 'nextcloud-aio-mastercontainer'
|
|
||||||
if not reference or settings.get('build'):
|
if not reference or settings.get('build'):
|
||||||
raise ValueError(f'{project}/{service}: Compose deploy requires a published image')
|
raise ValueError(f'{project}/{service}: Compose deploy requires a published image')
|
||||||
image_repo = reference.split('@')[0].rsplit('/', 1)
|
image_repo = reference.split('@')[0].rsplit('/', 1)
|
||||||
image_repo[-1] = image_repo[-1].split(':')[0]
|
image_repo[-1] = image_repo[-1].split(':')[0]
|
||||||
image_repo = '/'.join(image_repo)
|
image_repo = '/'.join(image_repo)
|
||||||
# Nextcloud AIO validates the mastercontainer image reference and rejects
|
if image_repo in release['images']:
|
||||||
# a digest. Keep its configured tag so AIO can start and manage its stack.
|
|
||||||
if nextcloud_aio_master:
|
|
||||||
pinned = reference
|
|
||||||
elif image_repo in release['images']:
|
|
||||||
pinned = image_repo + '@' + release['images'][image_repo]
|
pinned = image_repo + '@' + release['images'][image_repo]
|
||||||
elif os.environ.get('REFRESH_IMAGES') != 'true' and reference in locks:
|
elif os.environ.get('REFRESH_IMAGES') != 'true' and reference in locks:
|
||||||
pinned = locks[reference]
|
pinned = locks[reference]
|
||||||
@@ -80,11 +75,6 @@ def prepare(source_file):
|
|||||||
actual.add(next((d for d in digests or [] if d.split('@')[0] == image_repo), image_id))
|
actual.add(next((d for d in digests or [] if d.split('@')[0] == image_repo), image_id))
|
||||||
if len(actual) > 1:
|
if len(actual) > 1:
|
||||||
raise ValueError(f'{project}/{service}: mixed running images, cannot capture one recovery config')
|
raise ValueError(f'{project}/{service}: mixed running images, cannot capture one recovery config')
|
||||||
# AIO also rejects a digest in its recovery config. Preserve its tag in
|
|
||||||
# both deploy and recovery files.
|
|
||||||
if nextcloud_aio_master:
|
|
||||||
before['services'][service]['image'] = reference
|
|
||||||
else:
|
|
||||||
before['services'][service]['image'] = next(iter(actual)) if actual else reference
|
before['services'][service]['image'] = next(iter(actual)) if actual else reference
|
||||||
for name, data in (('compose', config), ('compose-before', before)):
|
for name, data in (('compose', config), ('compose-before', before)):
|
||||||
folder = directory / name
|
folder = directory / name
|
||||||
|
|||||||
@@ -141,8 +141,7 @@ def make_plan(directory):
|
|||||||
planner = load_module('deploy_plan', source / '.gitea/workflows/deploy-plan.py')
|
planner = load_module('deploy_plan', source / '.gitea/workflows/deploy-plan.py')
|
||||||
request = json.loads((directory / 'request.json').read_text())
|
request = json.loads((directory / 'request.json').read_text())
|
||||||
previous = json.loads((STATE / 'last-success.json').read_text()) if (STATE / 'last-success.json').exists() else None
|
previous = json.loads((STATE / 'last-success.json').read_text()) if (STATE / 'last-success.json').exists() else None
|
||||||
# Helm 4 lists every release status by default and removed the --all flag.
|
helm = json.loads(command('helm', 'list', '--all', '-A', '-o', 'json'))
|
||||||
helm = json.loads(command('helm', 'list', '-A', '-o', 'json'))
|
|
||||||
plan = planner.make_plan(source, CONFIG_REPO, request['release'], previous, request['mode'], helm)
|
plan = planner.make_plan(source, CONFIG_REPO, request['release'], previous, request['mode'], helm)
|
||||||
if request['refresh_images']:
|
if request['refresh_images']:
|
||||||
plan['selected']['compose'] = plan['active']['compose']
|
plan['selected']['compose'] = plan['active']['compose']
|
||||||
@@ -213,17 +212,14 @@ def execute(run_id):
|
|||||||
raise ValueError(f'Interrupted deploy {other.name}; run recover first')
|
raise ValueError(f'Interrupted deploy {other.name}; run recover first')
|
||||||
status['state'] = 'running'
|
status['state'] = 'running'
|
||||||
atomic_json(directory / 'status.json', status)
|
atomic_json(directory / 'status.json', status)
|
||||||
phase = 'plan'
|
|
||||||
try:
|
try:
|
||||||
plan = make_plan(directory)
|
plan = make_plan(directory)
|
||||||
print(
|
print(
|
||||||
json.dumps({'selected': plan['selected'], 'helm': plan['helm'], 'manual_removals': plan['removed']}),
|
json.dumps({'selected': plan['selected'], 'helm': plan['helm'], 'manual_removals': plan['removed']}),
|
||||||
flush=True,
|
flush=True,
|
||||||
)
|
)
|
||||||
phase = 'doctor'
|
|
||||||
if not stage(directory, 'doctor', 600):
|
if not stage(directory, 'doctor', 600):
|
||||||
raise RuntimeError('Preflight failed')
|
raise RuntimeError('Preflight failed')
|
||||||
phase = 'validate'
|
|
||||||
if not stage(directory, 'validate', 1200):
|
if not stage(directory, 'validate', 1200):
|
||||||
raise RuntimeError('Validation failed')
|
raise RuntimeError('Validation failed')
|
||||||
if json.loads((directory / 'request.json').read_text())['mode'] == 'plan':
|
if json.loads((directory / 'request.json').read_text())['mode'] == 'plan':
|
||||||
@@ -232,7 +228,6 @@ def execute(run_id):
|
|||||||
atomic_json(directory / 'status.json', status)
|
atomic_json(directory / 'status.json', status)
|
||||||
return
|
return
|
||||||
# Budget includes both rollout checks and rollback waves, plus API overhead.
|
# Budget includes both rollout checks and rollback waves, plus API overhead.
|
||||||
phase = 'Recovery budget'
|
|
||||||
count = int(
|
count = int(
|
||||||
command(
|
command(
|
||||||
'bash',
|
'bash',
|
||||||
@@ -244,24 +239,14 @@ def execute(run_id):
|
|||||||
verify_budget = max(600, 2 * math.ceil(count / 4) * 300 + 120)
|
verify_budget = max(600, 2 * math.ceil(count / 4) * 300 + 120)
|
||||||
if verify_budget > 7200:
|
if verify_budget > 7200:
|
||||||
raise ValueError('More than two hours of recovery required; split this deploy')
|
raise ValueError('More than two hours of recovery required; split this deploy')
|
||||||
phase = 'apply-k8s'
|
|
||||||
k8s_ok = stage(directory, 'apply-k8s', 2700)
|
k8s_ok = stage(directory, 'apply-k8s', 2700)
|
||||||
phase = 'apply-compose'
|
|
||||||
compose_ok = stage(directory, 'apply-compose', 1800) if k8s_ok else False
|
compose_ok = stage(directory, 'apply-compose', 1800) if k8s_ok else False
|
||||||
phase = 'verify-k8s'
|
|
||||||
verify_ok = stage(directory, 'verify-k8s', verify_budget)
|
verify_ok = stage(directory, 'verify-k8s', verify_budget)
|
||||||
phase = 'smoke'
|
|
||||||
smoke_ok = stage(directory, 'smoke', 600)
|
smoke_ok = stage(directory, 'smoke', 600)
|
||||||
if not all((k8s_ok, compose_ok, verify_ok, smoke_ok)):
|
if not all((k8s_ok, compose_ok, verify_ok, smoke_ok)):
|
||||||
raise RuntimeError('Deploy failed; inspect stage logs and recovery report')
|
raise RuntimeError('Deploy failed; inspect stage logs and recovery report')
|
||||||
phase = 'Save the successful baseline'
|
|
||||||
finish_success(directory, plan)
|
finish_success(directory, plan)
|
||||||
except Exception as error:
|
except Exception as error:
|
||||||
status = json.loads((directory / 'status.json').read_text())
|
|
||||||
status['failure_stage'] = next(
|
|
||||||
(name for name, result in status['stages'].items() if result.get('result') == 'failure'), phase
|
|
||||||
)
|
|
||||||
atomic_json(directory / 'status.json', status)
|
|
||||||
with (directory / 'controller.log').open('a') as stream:
|
with (directory / 'controller.log').open('a') as stream:
|
||||||
stream.write(f'{error}\n')
|
stream.write(f'{error}\n')
|
||||||
recover(directory)
|
recover(directory)
|
||||||
@@ -309,93 +294,10 @@ def follow(run_id, phase):
|
|||||||
time.sleep(3)
|
time.sleep(3)
|
||||||
|
|
||||||
|
|
||||||
def summary(run_id):
|
|
||||||
directory = run_directory(run_id)
|
|
||||||
request = json.loads((directory / 'request.json').read_text())
|
|
||||||
release = request['release']
|
|
||||||
plan_file = directory / 'plan.json'
|
|
||||||
lines = [
|
|
||||||
f'## Deploy `{release["sha"]}`',
|
|
||||||
'',
|
|
||||||
f'- Mode: `{request["mode"]}`',
|
|
||||||
f'- Refresh third-party images: `{request["refresh_images"]}`',
|
|
||||||
]
|
|
||||||
status = json.loads((directory / 'status.json').read_text())
|
|
||||||
if status.get('failure_stage'):
|
|
||||||
lines.append(f'- Failed stage: **{status["failure_stage"]}**')
|
|
||||||
lines.extend(
|
|
||||||
[
|
|
||||||
'',
|
|
||||||
f'- Observed run state: **{status["state"]}**',
|
|
||||||
'',
|
|
||||||
'### Stage results',
|
|
||||||
'| Stage | Result | Exit code |',
|
|
||||||
'| --- | --- | --- |',
|
|
||||||
]
|
|
||||||
)
|
|
||||||
for name in ('doctor', 'validate', 'apply-k8s', 'apply-compose', 'verify-k8s', 'smoke'):
|
|
||||||
stage_result = status['stages'].get(name, {})
|
|
||||||
lines.append(f'| {name} | {stage_result.get("result", "not started")} | {stage_result.get("exit_code", "—")} |')
|
|
||||||
lines.extend(['', '### Apply and Helm recovery results'])
|
|
||||||
events_file = directory / 'apply-events.jsonl'
|
|
||||||
events = []
|
|
||||||
if events_file.exists():
|
|
||||||
for line in events_file.read_text().splitlines():
|
|
||||||
try:
|
|
||||||
events.append(json.loads(line))
|
|
||||||
except json.JSONDecodeError:
|
|
||||||
lines.append('- An operation record is incomplete. Check the stage log.')
|
|
||||||
latest = {(event['action'], event['target']): event['result'] for event in events}
|
|
||||||
lines.extend(f'- `{action}` `{target}`: **{result}**' for (action, target), result in latest.items())
|
|
||||||
if not latest:
|
|
||||||
lines.append('- No apply results were recorded.')
|
|
||||||
lines.append('- A completed apply does not confirm health. See verification and smoke results.')
|
|
||||||
lines.extend(['', '### Kubernetes recovery'])
|
|
||||||
pointer = directory / 'snapshot/current'
|
|
||||||
failed = Path(pointer.read_text().strip()) / 'failed-workloads' if pointer.exists() else None
|
|
||||||
if failed and failed.exists():
|
|
||||||
contents = failed.read_text()
|
|
||||||
counts = dict(re.findall(r'^(ROLLED_BACK|UNRECOVERED)=([0-9]+)$', contents, re.MULTILINE))
|
|
||||||
if not contents.strip():
|
|
||||||
lines.append('- No failed workloads were recorded. See the verification result above.')
|
|
||||||
elif counts:
|
|
||||||
lines.append(f'- Workloads restored: **{counts.get("ROLLED_BACK", "unknown")}**')
|
|
||||||
lines.append(f'- Workloads that need manual recovery: **{counts.get("UNRECOVERED", "unknown")}**')
|
|
||||||
else:
|
|
||||||
lines.append('- Rollback has no recorded result yet. Check the verification log.')
|
|
||||||
else:
|
|
||||||
lines.append('- No workload rollback was recorded. This does not confirm health.')
|
|
||||||
lines.append('- Compose requires manual recovery. Use the saved command in the apply log.')
|
|
||||||
if not plan_file.exists():
|
|
||||||
lines.extend(['', 'Plan was not created. Check the controller log.'])
|
|
||||||
print('\n'.join(lines))
|
|
||||||
return
|
|
||||||
plan = json.loads(plan_file.read_text())
|
|
||||||
lines.extend(['', '### Selected services'])
|
|
||||||
count = 0
|
|
||||||
for kind, services in plan['selected'].items():
|
|
||||||
for service in services:
|
|
||||||
lines.append(f'- `{kind}`: `{service}`')
|
|
||||||
count += 1
|
|
||||||
if not count:
|
|
||||||
lines.append('- None')
|
|
||||||
lines.extend(['', '### Selected Helm releases'])
|
|
||||||
lines.extend(f'- `{release}`' for release in plan.get('helm', []))
|
|
||||||
if not plan.get('helm'):
|
|
||||||
lines.append('- None')
|
|
||||||
lines.extend(['', '### Images pinned in the checked release'])
|
|
||||||
lines.extend(f'- `{image}@{digest}`' for image, digest in sorted(release['images'].items()))
|
|
||||||
lines.extend(['', '### Removed resources requiring manual review'])
|
|
||||||
lines.extend(f'- `{item}`' for item in plan.get('removed', []))
|
|
||||||
if not plan.get('removed'):
|
|
||||||
lines.append('- None')
|
|
||||||
print('\n'.join(lines))
|
|
||||||
|
|
||||||
|
|
||||||
def main():
|
def main():
|
||||||
os.umask(0o077)
|
os.umask(0o077)
|
||||||
parser = argparse.ArgumentParser(description=__doc__)
|
parser = argparse.ArgumentParser(description=__doc__)
|
||||||
parser.add_argument('action', choices=('start', 'execute', 'recover', 'status', 'follow', 'summary'))
|
parser.add_argument('action', choices=('start', 'execute', 'recover', 'status', 'follow'))
|
||||||
parser.add_argument('run_id')
|
parser.add_argument('run_id')
|
||||||
parser.add_argument('phase', nargs='?', choices=('apply', 'verify', 'smoke'))
|
parser.add_argument('phase', nargs='?', choices=('apply', 'verify', 'smoke'))
|
||||||
parser.add_argument('--retry', action='store_true', help='Retry failed recovery checks; never repeat apply')
|
parser.add_argument('--retry', action='store_true', help='Retry failed recovery checks; never repeat apply')
|
||||||
@@ -413,8 +315,6 @@ def main():
|
|||||||
if (directory / 'plan.json').exists():
|
if (directory / 'plan.json').exists():
|
||||||
plan = json.loads((directory / 'plan.json').read_text())
|
plan = json.loads((directory / 'plan.json').read_text())
|
||||||
print(json.dumps({k: plan[k] for k in ('sha', 'selected', 'helm', 'removed')}, indent=2))
|
print(json.dumps({k: plan[k] for k in ('sha', 'selected', 'helm', 'removed')}, indent=2))
|
||||||
elif args.action == 'summary':
|
|
||||||
summary(args.run_id)
|
|
||||||
elif not follow(args.run_id, args.phase):
|
elif not follow(args.run_id, args.phase):
|
||||||
sys.exit(1)
|
sys.exit(1)
|
||||||
|
|
||||||
|
|||||||
@@ -27,15 +27,6 @@ log() {
|
|||||||
echo "== $* =="
|
echo "== $* =="
|
||||||
}
|
}
|
||||||
|
|
||||||
# Store operation results without command output or local configuration values.
|
|
||||||
record_apply() {
|
|
||||||
[ -n "${RUN_DIR:-}" ] || return 0
|
|
||||||
jq -cn --arg action "$1" --arg target "$2" --arg result "$3" \
|
|
||||||
'{action: $action, target: $target, result: $result}' >>"$RUN_DIR/apply-events.jsonl" \
|
|
||||||
|| echo 'WARNING: cannot record an apply result' >&2
|
|
||||||
return 0
|
|
||||||
}
|
|
||||||
|
|
||||||
warn() {
|
warn() {
|
||||||
echo "WARNING: $*" >&2
|
echo "WARNING: $*" >&2
|
||||||
}
|
}
|
||||||
@@ -180,8 +171,7 @@ save_snapshot() {
|
|||||||
| select(any(.metadata.ownerReferences[]?; .uid == $w.metadata.uid))
|
| select(any(.metadata.ownerReferences[]?; .uid == $w.metadata.uid))
|
||||||
| select($w.kind != "StatefulSet" or .metadata.name == $w.status.currentRevision) | .revision] | max // 0) end)
|
| select($w.kind != "StatefulSet" or .metadata.name == $w.status.currentRevision) | .revision] | max // 0) end)
|
||||||
}]' "$dir/workloads.json" >"$dir/revisions.json" || return 1
|
}]' "$dir/workloads.json" >"$dir/revisions.json" || return 1
|
||||||
# Helm 4 lists every release status by default and removed the --all flag.
|
releases="$(helm list --all -A -o json)" || return 1
|
||||||
releases="$(helm list -A -o json)" || return 1
|
|
||||||
for entry in "${HELM_RELEASES[@]}"; do
|
for entry in "${HELM_RELEASES[@]}"; do
|
||||||
IFS='|' read -r release _ namespace _ _ _ <<<"$entry"
|
IFS='|' read -r release _ namespace _ _ _ <<<"$entry"
|
||||||
if ! jq -e --arg r "$release" --arg n "$namespace" \
|
if ! jq -e --arg r "$release" --arg n "$namespace" \
|
||||||
@@ -330,7 +320,7 @@ rollback_workloads() {
|
|||||||
# written straight into a `helm upgrade` command would never be updated: these
|
# written straight into a `helm upgrade` command would never be updated: these
|
||||||
# have to be declared as custom.regex managers in renovate/renovate.json.
|
# have to be declared as custom.regex managers in renovate/renovate.json.
|
||||||
HELM_RELEASES=(
|
HELM_RELEASES=(
|
||||||
"prometheus-stack|prometheus-community/kube-prometheus-stack|prometheus|86.2.3|prometheus-stack/k8s/grafana-values.yaml|prometheus-stack/k8s/active"
|
"prometheus-stack|prometheus-community/kube-prometheus-stack|prometheus|86.3.2|prometheus-stack/k8s/grafana-values.yaml|prometheus-stack/k8s/active"
|
||||||
"victoria-operator|victoriametrics/victoria-metrics-operator|prometheus|0.68.1|prometheus-stack/k8s/victoria-operator-values.yaml|prometheus-stack/k8s/active"
|
"victoria-operator|victoriametrics/victoria-metrics-operator|prometheus|0.68.1|prometheus-stack/k8s/victoria-operator-values.yaml|prometheus-stack/k8s/active"
|
||||||
"loki|grafana/loki|prometheus|7.3.0|loki/k8s/loki-values.yaml|loki/k8s/active"
|
"loki|grafana/loki|prometheus|7.3.0|loki/k8s/loki-values.yaml|loki/k8s/active"
|
||||||
"alloy|grafana/alloy|prometheus|1.12.1|loki/k8s/alloy-values.yaml|loki/k8s/active"
|
"alloy|grafana/alloy|prometheus|1.12.1|loki/k8s/alloy-values.yaml|loki/k8s/active"
|
||||||
@@ -386,19 +376,15 @@ recover_pending_release() {
|
|||||||
echo "ERROR: no captured Helm revision for $release; manual recovery required"
|
echo "ERROR: no captured Helm revision for $release; manual recovery required"
|
||||||
return 1
|
return 1
|
||||||
fi
|
fi
|
||||||
record_apply helm-rollback "$namespace/$release" started
|
|
||||||
if ! helm rollback "$release" "$revision" -n "$namespace" --wait --timeout 10m; then
|
if ! helm rollback "$release" "$revision" -n "$namespace" --wait --timeout 10m; then
|
||||||
record_apply helm-rollback "$namespace/$release" failure
|
|
||||||
echo "WARN: helm rollback of $release did not complete"
|
echo "WARN: helm rollback of $release did not complete"
|
||||||
return 1
|
return 1
|
||||||
fi
|
fi
|
||||||
status="$(helm_release_status "$release" "$namespace")" || return 1
|
status="$(helm_release_status "$release" "$namespace")" || return 1
|
||||||
if [ "$status" != "deployed" ]; then
|
if [ "$status" != "deployed" ]; then
|
||||||
record_apply helm-rollback "$namespace/$release" failure
|
|
||||||
echo "WARN: $release is $status after rollback"
|
echo "WARN: $release is $status after rollback"
|
||||||
return 1
|
return 1
|
||||||
fi
|
fi
|
||||||
record_apply helm-rollback "$namespace/$release" success
|
|
||||||
;;
|
;;
|
||||||
esac
|
esac
|
||||||
return 0
|
return 0
|
||||||
@@ -462,13 +448,11 @@ upgrade_helm_releases() {
|
|||||||
# --rollback-on-failure (+ --wait) rolls the release back when the upgrade
|
# --rollback-on-failure (+ --wait) rolls the release back when the upgrade
|
||||||
# times out or the workloads it touches never become ready, so a bad chart
|
# times out or the workloads it touches never become ready, so a bad chart
|
||||||
# bump is not left half applied. (--atomic was this combo; deprecated.)
|
# bump is not left half applied. (--atomic was this combo; deprecated.)
|
||||||
record_apply helm-upgrade "$namespace/$release" started
|
|
||||||
if ! helm upgrade --install "$release" "$chart" \
|
if ! helm upgrade --install "$release" "$chart" \
|
||||||
--namespace "$namespace" \
|
--namespace "$namespace" \
|
||||||
--version "$version" \
|
--version "$version" \
|
||||||
--values "$values" \
|
--values "$values" \
|
||||||
--wait --rollback-on-failure --cleanup-on-fail --timeout 10m; then
|
--wait --rollback-on-failure --cleanup-on-fail --timeout 10m; then
|
||||||
record_apply helm-upgrade "$namespace/$release" failure
|
|
||||||
echo "WARN: upgrade of $release failed, checking release state"
|
echo "WARN: upgrade of $release failed, checking release state"
|
||||||
# --rollback-on-failure already attempted its own rollback; finish the job when that
|
# --rollback-on-failure already attempted its own rollback; finish the job when that
|
||||||
# rollback never completed, otherwise the release stays pending-* and
|
# rollback never completed, otherwise the release stays pending-* and
|
||||||
@@ -478,10 +462,8 @@ upgrade_helm_releases() {
|
|||||||
else
|
else
|
||||||
echo "ERROR: upgrade of $release failed (release is back on its previous revision)."
|
echo "ERROR: upgrade of $release failed (release is back on its previous revision)."
|
||||||
fi
|
fi
|
||||||
record_apply helm-recovery-state "$namespace/$release" "$(helm_release_status "$release" "$namespace" || echo unknown)"
|
|
||||||
return 1
|
return 1
|
||||||
fi
|
fi
|
||||||
record_apply helm-upgrade "$namespace/$release" success
|
|
||||||
done
|
done
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -649,12 +631,7 @@ stage_apply_k8s() {
|
|||||||
if [ "${#ns_files[@]}" -gt 0 ]; then
|
if [ "${#ns_files[@]}" -gt 0 ]; then
|
||||||
log "Applying namespaces (${#ns_files[@]} files)"
|
log "Applying namespaces (${#ns_files[@]} files)"
|
||||||
for m in "${ns_files[@]}"; do
|
for m in "${ns_files[@]}"; do
|
||||||
record_apply kubectl "${m#"$REPO"/}" started
|
kubectl apply -f "$m"
|
||||||
if ! kubectl apply -f "$m"; then
|
|
||||||
record_apply kubectl "${m#"$REPO"/}" failure
|
|
||||||
return 1
|
|
||||||
fi
|
|
||||||
record_apply kubectl "${m#"$REPO"/}" success
|
|
||||||
done
|
done
|
||||||
fi
|
fi
|
||||||
if selected_service k8s prometheus-stack && [ -f "$REPO/prometheus-stack/k8s/active" ]; then
|
if selected_service k8s prometheus-stack && [ -f "$REPO/prometheus-stack/k8s/active" ]; then
|
||||||
@@ -669,24 +646,18 @@ stage_apply_k8s() {
|
|||||||
log "Applying resources (${#other_files[@]} files, our images pinned to digests)"
|
log "Applying resources (${#other_files[@]} files, our images pinned to digests)"
|
||||||
for m in "${other_files[@]}"; do
|
for m in "${other_files[@]}"; do
|
||||||
log "Applying ${m#"$REPO"/}"
|
log "Applying ${m#"$REPO"/}"
|
||||||
record_apply kubectl "${m#"$REPO"/}" started
|
|
||||||
if ! render_pinned <"$m" | kubectl apply -f -; then
|
if ! render_pinned <"$m" | kubectl apply -f -; then
|
||||||
record_apply kubectl "${m#"$REPO"/}" failure
|
|
||||||
echo "ERROR: apply failed for ${m#"$REPO"/}" >&2
|
echo "ERROR: apply failed for ${m#"$REPO"/}" >&2
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
record_apply kubectl "${m#"$REPO"/}" success
|
|
||||||
done
|
done
|
||||||
fi
|
fi
|
||||||
for k in ${KUSTOMIZE_APPS[@]+"${KUSTOMIZE_APPS[@]}"}; do
|
for k in ${KUSTOMIZE_APPS[@]+"${KUSTOMIZE_APPS[@]}"}; do
|
||||||
log "Applying kustomize app: ${k#"$REPO"/} (our images pinned to digests)"
|
log "Applying kustomize app: ${k#"$REPO"/} (our images pinned to digests)"
|
||||||
record_apply kustomize "${k#"$REPO"/}" started
|
|
||||||
if ! kubectl kustomize "$k" | render_pinned | kubectl apply -f -; then
|
if ! kubectl kustomize "$k" | render_pinned | kubectl apply -f -; then
|
||||||
record_apply kustomize "${k#"$REPO"/}" failure
|
|
||||||
echo "ERROR: apply failed for kustomize app ${k#"$REPO"/}" >&2
|
echo "ERROR: apply failed for kustomize app ${k#"$REPO"/}" >&2
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
record_apply kustomize "${k#"$REPO"/}" success
|
|
||||||
done
|
done
|
||||||
|
|
||||||
# No verification here on purpose. This stage may be killed at any point by
|
# No verification here on purpose. This stage may be killed at any point by
|
||||||
@@ -988,12 +959,7 @@ stage_apply_compose() {
|
|||||||
local cf
|
local cf
|
||||||
for cf in "${COMPOSE_STACKS[@]}"; do
|
for cf in "${COMPOSE_STACKS[@]}"; do
|
||||||
log "Applying Compose ${cf#"$REPO"/}"
|
log "Applying Compose ${cf#"$REPO"/}"
|
||||||
record_apply compose "${cf#"$REPO"/}" started
|
compose "$cf" up -d --wait --wait-timeout 180 --pull missing --remove-orphans
|
||||||
if ! compose "$cf" up -d --wait --wait-timeout 180 --pull missing --remove-orphans; then
|
|
||||||
record_apply compose "${cf#"$REPO"/}" failure
|
|
||||||
return 1
|
|
||||||
fi
|
|
||||||
record_apply compose "${cf#"$REPO"/}" success
|
|
||||||
verify_compose_stack "$cf"
|
verify_compose_stack "$cf"
|
||||||
done
|
done
|
||||||
echo "Compose recovery files: $RUN_DIR/compose-before (manual recovery only)"
|
echo "Compose recovery files: $RUN_DIR/compose-before (manual recovery only)"
|
||||||
|
|||||||
@@ -64,23 +64,11 @@ jobs:
|
|||||||
run: python3 .gitea/workflows/release.py gate --ref "$DEPLOY_REF" --event-sha "$EVENT_SHA"
|
run: python3 .gitea/workflows/release.py gate --ref "$DEPLOY_REF" --event-sha "$EVENT_SHA"
|
||||||
- name: Submit durable deploy to workstation
|
- name: Submit durable deploy to workstation
|
||||||
run: bash .gitea/workflows/ssh-run.sh start
|
run: bash .gitea/workflows/ssh-run.sh start
|
||||||
- name: Write the request result
|
|
||||||
if: always()
|
|
||||||
env:
|
|
||||||
REQUEST_RESULT: ${{ job.status }}
|
|
||||||
CHECKED_SHA: ${{ steps.release.outputs.sha }}
|
|
||||||
run: |
|
|
||||||
if [ -n "${GITHUB_STEP_SUMMARY:-}" ]; then
|
|
||||||
printf '## Deploy request\n\n- Result: **%s**\n- Checked commit: %s\n- Mode: %s\n' "$REQUEST_RESULT" "${CHECKED_SHA:-not checked}" "$DEPLOY_MODE" >>"$GITHUB_STEP_SUMMARY"
|
|
||||||
if [ "$REQUEST_RESULT" != success ]; then
|
|
||||||
echo 'Open the failed step log. If SSH submission failed, check the remote controller state.' >>"$GITHUB_STEP_SUMMARY"
|
|
||||||
fi
|
|
||||||
fi
|
|
||||||
|
|
||||||
apply:
|
apply:
|
||||||
needs: [gate]
|
needs: [gate]
|
||||||
runs-on: homelab
|
runs-on: homelab
|
||||||
timeout-minutes: 120
|
timeout-minutes: 100
|
||||||
steps:
|
steps:
|
||||||
- name: Checkout checked commit
|
- name: Checkout checked commit
|
||||||
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
|
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
|
||||||
@@ -88,14 +76,6 @@ jobs:
|
|||||||
ref: ${{ needs.gate.outputs.sha }}
|
ref: ${{ needs.gate.outputs.sha }}
|
||||||
- name: Follow validation and sequential Kubernetes / Compose apply
|
- name: Follow validation and sequential Kubernetes / Compose apply
|
||||||
run: bash .gitea/workflows/ssh-run.sh apply
|
run: bash .gitea/workflows/ssh-run.sh apply
|
||||||
- name: Write the deploy result
|
|
||||||
if: always()
|
|
||||||
run: |
|
|
||||||
if [ -f .gitea/workflows/ssh-run.sh ]; then
|
|
||||||
bash .gitea/workflows/ssh-run.sh summary
|
|
||||||
elif [ -n "${GITHUB_STEP_SUMMARY:-}" ]; then
|
|
||||||
echo 'Source checkout failed. The remote deploy state is unknown. Check the job log.' >>"$GITHUB_STEP_SUMMARY"
|
|
||||||
fi
|
|
||||||
|
|
||||||
verify:
|
verify:
|
||||||
needs: [gate, apply]
|
needs: [gate, apply]
|
||||||
@@ -109,14 +89,6 @@ jobs:
|
|||||||
ref: ${{ needs.gate.outputs.sha }}
|
ref: ${{ needs.gate.outputs.sha }}
|
||||||
- name: Follow workload verification and recovery
|
- name: Follow workload verification and recovery
|
||||||
run: bash .gitea/workflows/ssh-run.sh verify
|
run: bash .gitea/workflows/ssh-run.sh verify
|
||||||
- name: Write the deploy result
|
|
||||||
if: always()
|
|
||||||
run: |
|
|
||||||
if [ -f .gitea/workflows/ssh-run.sh ]; then
|
|
||||||
bash .gitea/workflows/ssh-run.sh summary
|
|
||||||
elif [ -n "${GITHUB_STEP_SUMMARY:-}" ]; then
|
|
||||||
echo 'Source checkout failed. The remote deploy state is unknown. Check the job log.' >>"$GITHUB_STEP_SUMMARY"
|
|
||||||
fi
|
|
||||||
|
|
||||||
smoke:
|
smoke:
|
||||||
needs: [gate, verify]
|
needs: [gate, verify]
|
||||||
@@ -130,11 +102,3 @@ jobs:
|
|||||||
ref: ${{ needs.gate.outputs.sha }}
|
ref: ${{ needs.gate.outputs.sha }}
|
||||||
- name: Follow public route checks
|
- name: Follow public route checks
|
||||||
run: bash .gitea/workflows/ssh-run.sh smoke
|
run: bash .gitea/workflows/ssh-run.sh smoke
|
||||||
- name: Write the deploy result
|
|
||||||
if: always()
|
|
||||||
run: |
|
|
||||||
if [ -f .gitea/workflows/ssh-run.sh ]; then
|
|
||||||
bash .gitea/workflows/ssh-run.sh summary
|
|
||||||
elif [ -n "${GITHUB_STEP_SUMMARY:-}" ]; then
|
|
||||||
echo 'Source checkout failed. The remote deploy state is unknown. Check the job log.' >>"$GITHUB_STEP_SUMMARY"
|
|
||||||
fi
|
|
||||||
+18
-184
@@ -26,6 +26,9 @@ IMAGES = {
|
|||||||
'xdfnx-homepage': ('homepages', 'homepages/Dockerfile.xdfnx'),
|
'xdfnx-homepage': ('homepages', 'homepages/Dockerfile.xdfnx'),
|
||||||
}
|
}
|
||||||
|
|
||||||
|
# These images are released by the EDU application repository.
|
||||||
|
EXTERNAL_IMAGES = {'gcr.forust.xyz/forust/session-keeper', 'gcr.forust.xyz/forust/webinar-checker'}
|
||||||
|
|
||||||
|
|
||||||
def command(*args, **kwargs):
|
def command(*args, **kwargs):
|
||||||
"""Arguments are passed directly to the executable, never to a shell."""
|
"""Arguments are passed directly to the executable, never to a shell."""
|
||||||
@@ -160,7 +163,7 @@ def gate(output, requested_ref, event_sha):
|
|||||||
print(f'CI gate accepted {sha}')
|
print(f'CI gate accepted {sha}')
|
||||||
|
|
||||||
|
|
||||||
def prepare_images(output):
|
def build(output):
|
||||||
sha = command('git', 'rev-parse', 'HEAD')
|
sha = command('git', 'rev-parse', 'HEAD')
|
||||||
if sha != os.environ['GITHUB_SHA'] or not SHA.fullmatch(sha):
|
if sha != os.environ['GITHUB_SHA'] or not SHA.fullmatch(sha):
|
||||||
raise ValueError('Build checkout does not match GITHUB_SHA')
|
raise ValueError('Build checkout does not match GITHUB_SHA')
|
||||||
@@ -175,61 +178,11 @@ def prepare_images(output):
|
|||||||
except ValueError:
|
except ValueError:
|
||||||
# Expired artifacts only cost a rebuild; mutable tags are never a fallback.
|
# Expired artifacts only cost a rebuild; mutable tags are never a fallback.
|
||||||
continue
|
continue
|
||||||
targets = []
|
|
||||||
for name, (context, dockerfile) in IMAGES.items():
|
|
||||||
image = f'gcr.forust.xyz/forust/{name}'
|
|
||||||
inputs = fingerprint(context, dockerfile)
|
|
||||||
old_digest = (previous or {}).get('images', {}).get(image)
|
|
||||||
targets.append(
|
|
||||||
{
|
|
||||||
'name': name,
|
|
||||||
'image': image,
|
|
||||||
'context': context,
|
|
||||||
'dockerfile': dockerfile,
|
|
||||||
'inputs': inputs,
|
|
||||||
'reuse_digest': old_digest if (previous or {}).get('inputs', {}).get(image) == inputs else None,
|
|
||||||
}
|
|
||||||
)
|
|
||||||
output.write_text(json.dumps({'sha': sha, 'targets': targets}, indent=2) + '\n')
|
|
||||||
if os.environ.get('GITHUB_OUTPUT'):
|
|
||||||
with Path(os.environ['GITHUB_OUTPUT']).open('a') as stream:
|
|
||||||
stream.write('matrix=' + json.dumps({'include': targets}, separators=(',', ':')) + '\n')
|
|
||||||
print(f'Prepared {len(targets)} image jobs; {sum(t["reuse_digest"] is None for t in targets)} require builds')
|
|
||||||
|
|
||||||
|
|
||||||
def checked_plan(path):
|
|
||||||
data = json.loads(path.read_text())
|
|
||||||
sha = command('git', 'rev-parse', 'HEAD')
|
|
||||||
if data.get('sha') != sha or sha != os.environ['GITHUB_SHA'] or not SHA.fullmatch(sha):
|
|
||||||
raise ValueError('Image plan does not match the checked source commit')
|
|
||||||
targets = data.get('targets', [])
|
|
||||||
if sorted(t['name'] for t in targets) != sorted(IMAGES):
|
|
||||||
raise ValueError('Image plan must contain each owned image once')
|
|
||||||
for target in targets:
|
|
||||||
name = target['name']
|
|
||||||
context, dockerfile = IMAGES[name]
|
|
||||||
if (target['context'], target['dockerfile'], target['image']) != (
|
|
||||||
context,
|
|
||||||
dockerfile,
|
|
||||||
f'gcr.forust.xyz/forust/{name}',
|
|
||||||
) or target['inputs'] != fingerprint(context, dockerfile):
|
|
||||||
raise ValueError('Image plan has invalid build inputs')
|
|
||||||
if target['reuse_digest'] is not None and not DIGEST.fullmatch(target['reuse_digest']):
|
|
||||||
raise ValueError('Image plan has an invalid reuse digest')
|
|
||||||
return data
|
|
||||||
|
|
||||||
|
|
||||||
def build_images(output, report, name, plan):
|
|
||||||
data = checked_plan(plan)
|
|
||||||
sha = data['sha']
|
|
||||||
target = next(t for t in data['targets'] if t['name'] == name)
|
|
||||||
context, dockerfile = IMAGES[name]
|
|
||||||
docker_config = tempfile.mkdtemp(prefix='homelab-registry-')
|
docker_config = tempfile.mkdtemp(prefix='homelab-registry-')
|
||||||
builder_config = Path.home() / '.cache/homelab-ci/buildx'
|
builder_config = Path.home() / '.cache/homelab-ci/buildx'
|
||||||
builder_config.mkdir(parents=True, exist_ok=True)
|
builder_config.mkdir(parents=True, exist_ok=True)
|
||||||
env = {**os.environ, 'DOCKER_CONFIG': docker_config, 'BUILDX_CONFIG': str(builder_config)}
|
env = {**os.environ, 'DOCKER_CONFIG': docker_config, 'BUILDX_CONFIG': str(builder_config)}
|
||||||
try:
|
try:
|
||||||
report['phase'] = 'Registry login'
|
|
||||||
subprocess.run( # noqa: S603, S607
|
subprocess.run( # noqa: S603, S607
|
||||||
[
|
[
|
||||||
shutil.which('docker') or '/usr/bin/docker',
|
shutil.which('docker') or '/usr/bin/docker',
|
||||||
@@ -244,7 +197,6 @@ def build_images(output, report, name, plan):
|
|||||||
check=True,
|
check=True,
|
||||||
env=env,
|
env=env,
|
||||||
)
|
)
|
||||||
report['phase'] = 'Prepare the builder'
|
|
||||||
builder = 'homelab-ci'
|
builder = 'homelab-ci'
|
||||||
versions = dict(
|
versions = dict(
|
||||||
re.findall(r'^([A-Z_]+)="([^"\n]+)"$', Path('.gitea/workflows/tool-versions.env').read_text(), re.MULTILINE)
|
re.findall(r'^([A-Z_]+)="([^"\n]+)"$', Path('.gitea/workflows/tool-versions.env').read_text(), re.MULTILINE)
|
||||||
@@ -279,14 +231,12 @@ def build_images(output, report, name, plan):
|
|||||||
)
|
)
|
||||||
signature.write_text(image + '\n')
|
signature.write_text(image + '\n')
|
||||||
release = {'version': 1, 'sha': sha, 'images': {}, 'inputs': {}}
|
release = {'version': 1, 'sha': sha, 'images': {}, 'inputs': {}}
|
||||||
report['images'] = release['images']
|
for name, (context, dockerfile) in IMAGES.items():
|
||||||
report['phase'] = f'Build or reuse {name}'
|
|
||||||
report['current'] = name
|
|
||||||
image = f'gcr.forust.xyz/forust/{name}'
|
image = f'gcr.forust.xyz/forust/{name}'
|
||||||
inputs = target['inputs']
|
inputs = fingerprint(context, dockerfile)
|
||||||
old_digest = target['reuse_digest']
|
old_digest = (previous or {}).get('images', {}).get(image)
|
||||||
exists = False
|
exists = False
|
||||||
if old_digest:
|
if old_digest and previous['inputs'].get(image) == inputs:
|
||||||
exists = (
|
exists = (
|
||||||
subprocess.run( # noqa: S603, S607
|
subprocess.run( # noqa: S603, S607
|
||||||
[
|
[
|
||||||
@@ -305,7 +255,6 @@ def build_images(output, report, name, plan):
|
|||||||
if exists:
|
if exists:
|
||||||
print(f'Reuse {name}: inputs unchanged')
|
print(f'Reuse {name}: inputs unchanged')
|
||||||
digest = old_digest
|
digest = old_digest
|
||||||
report['reused'].append(name)
|
|
||||||
else:
|
else:
|
||||||
print(f'Build {name}', flush=True)
|
print(f'Build {name}', flush=True)
|
||||||
metadata = Path(docker_config) / 'metadata.json'
|
metadata = Path(docker_config) / 'metadata.json'
|
||||||
@@ -315,6 +264,7 @@ def build_images(output, report, name, plan):
|
|||||||
'build',
|
'build',
|
||||||
'--builder',
|
'--builder',
|
||||||
builder,
|
builder,
|
||||||
|
'--push',
|
||||||
'--platform',
|
'--platform',
|
||||||
'linux/amd64',
|
'linux/amd64',
|
||||||
'--provenance=false',
|
'--provenance=false',
|
||||||
@@ -322,8 +272,8 @@ def build_images(output, report, name, plan):
|
|||||||
f'type=registry,ref={image}:buildcache',
|
f'type=registry,ref={image}:buildcache',
|
||||||
'--cache-to',
|
'--cache-to',
|
||||||
f'type=registry,ref={image}:buildcache,mode=max',
|
f'type=registry,ref={image}:buildcache,mode=max',
|
||||||
'--output',
|
'--tag',
|
||||||
f'type=image,name={image},push-by-digest=true,name-canonical=true,push=true',
|
f'{image}:sha-{sha}',
|
||||||
'--metadata-file',
|
'--metadata-file',
|
||||||
str(metadata),
|
str(metadata),
|
||||||
'--file',
|
'--file',
|
||||||
@@ -332,14 +282,10 @@ def build_images(output, report, name, plan):
|
|||||||
env=env,
|
env=env,
|
||||||
)
|
)
|
||||||
digest = json.loads(metadata.read_text())['containerimage.digest']
|
digest = json.loads(metadata.read_text())['containerimage.digest']
|
||||||
report['built'].append(name)
|
|
||||||
release['images'][image] = digest
|
release['images'][image] = digest
|
||||||
release['inputs'][image] = inputs
|
release['inputs'][image] = inputs
|
||||||
if not DIGEST.fullmatch(digest):
|
validate_release(release, sha)
|
||||||
raise ValueError('Image job returned an invalid digest')
|
|
||||||
output.write_text(json.dumps(release, indent=2) + '\n')
|
output.write_text(json.dumps(release, indent=2) + '\n')
|
||||||
report['current'] = None
|
|
||||||
report['phase'] = 'Release file saved'
|
|
||||||
finally:
|
finally:
|
||||||
# Cleanup errors must neither leak credentials nor mask the original build error.
|
# Cleanup errors must neither leak credentials nor mask the original build error.
|
||||||
try:
|
try:
|
||||||
@@ -363,59 +309,6 @@ def build_images(output, report, name, plan):
|
|||||||
shutil.rmtree(docker_config)
|
shutil.rmtree(docker_config)
|
||||||
|
|
||||||
|
|
||||||
def write_summary(lines):
|
|
||||||
path = os.environ.get('GITHUB_STEP_SUMMARY')
|
|
||||||
if path:
|
|
||||||
try:
|
|
||||||
with Path(path).open('a') as stream:
|
|
||||||
stream.write('\n'.join(lines) + '\n\n')
|
|
||||||
except OSError:
|
|
||||||
print('WARNING: cannot write the job summary')
|
|
||||||
|
|
||||||
|
|
||||||
def check_summary():
|
|
||||||
lines = [
|
|
||||||
f'## {os.environ["SUMMARY_CHECK"]}',
|
|
||||||
'',
|
|
||||||
f'- Commit: `{os.environ.get("GITHUB_SHA", "unknown")}`',
|
|
||||||
f'- Result: **{os.environ["SUMMARY_RESULT"]}**',
|
|
||||||
]
|
|
||||||
if os.environ.get('SUMMARY_FAILED_STEP'):
|
|
||||||
lines.append(f'- Failed step: {os.environ["SUMMARY_FAILED_STEP"]}')
|
|
||||||
if os.environ['SUMMARY_RESULT'] != 'success':
|
|
||||||
lines.append('- Open the failed step log for the error details.')
|
|
||||||
write_summary(lines)
|
|
||||||
|
|
||||||
|
|
||||||
def build(output, name, plan):
|
|
||||||
report = {'phase': 'Check the source commit', 'current': None, 'built': [], 'reused': [], 'images': {}}
|
|
||||||
result = 'failure'
|
|
||||||
try:
|
|
||||||
build_images(output, report, name, plan)
|
|
||||||
result = 'success'
|
|
||||||
finally:
|
|
||||||
lines = [
|
|
||||||
f'## Image release `{os.environ.get("GITHUB_SHA", "unknown")}`',
|
|
||||||
'',
|
|
||||||
f'- Result: **{result}**',
|
|
||||||
f'- Last stage: {report["phase"]}',
|
|
||||||
]
|
|
||||||
if result == 'failure':
|
|
||||||
lines.append('- No release from this build can be deployed. Open the failed step log.')
|
|
||||||
if report['current']:
|
|
||||||
lines.append(f'- Image at the failure: `{report["current"]}`')
|
|
||||||
for title, key in (('Built', 'built'), ('Reused from successful CI', 'reused')):
|
|
||||||
lines.extend(['', f'### {title}'])
|
|
||||||
lines.extend(f'- `{name}`' for name in report[key])
|
|
||||||
if not report[key]:
|
|
||||||
lines.append('- None')
|
|
||||||
lines.extend(['', '### Completed image digests'])
|
|
||||||
lines.extend(f'- `{image}@{digest}`' for image, digest in report['images'].items())
|
|
||||||
if not report['images']:
|
|
||||||
lines.append('- None')
|
|
||||||
write_summary(lines)
|
|
||||||
|
|
||||||
|
|
||||||
def render(stream, destination):
|
def render(stream, destination):
|
||||||
release = validate_release(json.loads(Path(os.environ['RELEASE_FILE']).read_text()), os.environ['DEPLOY_SHA'])
|
release = validate_release(json.loads(Path(os.environ['RELEASE_FILE']).read_text()), os.environ['DEPLOY_SHA'])
|
||||||
image_line = re.compile(
|
image_line = re.compile(
|
||||||
@@ -426,6 +319,9 @@ def render(stream, destination):
|
|||||||
match = image_line.fullmatch(line.rstrip('\n'))
|
match = image_line.fullmatch(line.rstrip('\n'))
|
||||||
if match:
|
if match:
|
||||||
prefix, quote, image, tail = match.groups()
|
prefix, quote, image, tail = match.groups()
|
||||||
|
if image in EXTERNAL_IMAGES and f'{image}@sha256:' in line:
|
||||||
|
rendered.append(line)
|
||||||
|
continue
|
||||||
if image not in release['images']:
|
if image not in release['images']:
|
||||||
raise ValueError(f'Owned image missing from checked release: {image}')
|
raise ValueError(f'Owned image missing from checked release: {image}')
|
||||||
line = f'{prefix}{quote}{image}@{release["images"][image]}{quote}{tail}\n'
|
line = f'{prefix}{quote}{image}@{release["images"][image]}{quote}{tail}\n'
|
||||||
@@ -435,81 +331,19 @@ def render(stream, destination):
|
|||||||
destination.writelines(rendered)
|
destination.writelines(rendered)
|
||||||
|
|
||||||
|
|
||||||
def finalize_images(output, fragments, plan):
|
|
||||||
data = checked_plan(plan)
|
|
||||||
sha = data['sha']
|
|
||||||
release = {'version': 1, 'sha': sha, 'images': {}, 'inputs': {}}
|
|
||||||
for name in IMAGES:
|
|
||||||
fragment = json.loads((fragments / f'image-{name}' / 'image.json').read_text())
|
|
||||||
image = f'gcr.forust.xyz/forust/{name}'
|
|
||||||
if fragment.get('sha') != sha or fragment.get('version') != 1 or set(fragment.get('images', {})) != {image}:
|
|
||||||
raise ValueError('Image job artifact is missing or belongs to another commit')
|
|
||||||
target = next(t for t in data['targets'] if t['name'] == name)
|
|
||||||
if fragment.get('inputs') != {image: target['inputs']}:
|
|
||||||
raise ValueError('Image artifact does not match the build plan')
|
|
||||||
release['images'].update(fragment['images'])
|
|
||||||
release['inputs'].update(fragment['inputs'])
|
|
||||||
validate_release(release, sha)
|
|
||||||
# Only a complete set of successful image jobs can publish the release tags.
|
|
||||||
docker_config = tempfile.mkdtemp(prefix='homelab-registry-')
|
|
||||||
env = {**os.environ, 'DOCKER_CONFIG': docker_config}
|
|
||||||
try:
|
|
||||||
subprocess.run( # noqa: S603, S607
|
|
||||||
[
|
|
||||||
shutil.which('docker') or '/usr/bin/docker',
|
|
||||||
'login',
|
|
||||||
'gcr.forust.xyz',
|
|
||||||
'-u',
|
|
||||||
os.environ['REGISTRY_USERNAME'],
|
|
||||||
'--password-stdin',
|
|
||||||
],
|
|
||||||
input=os.environ['REGISTRY_PASSWORD'],
|
|
||||||
text=True,
|
|
||||||
check=True,
|
|
||||||
env=env,
|
|
||||||
)
|
|
||||||
for image, digest in release['images'].items():
|
|
||||||
command(
|
|
||||||
'docker',
|
|
||||||
'buildx',
|
|
||||||
'imagetools',
|
|
||||||
'create',
|
|
||||||
'--prefer-index=false',
|
|
||||||
'--tag',
|
|
||||||
f'{image}:sha-{sha}',
|
|
||||||
f'{image}@{digest}',
|
|
||||||
env=env,
|
|
||||||
timeout=90,
|
|
||||||
)
|
|
||||||
output.write_text(json.dumps(release, indent=2) + '\n')
|
|
||||||
finally:
|
|
||||||
shutil.rmtree(docker_config)
|
|
||||||
|
|
||||||
|
|
||||||
def main():
|
def main():
|
||||||
parser = argparse.ArgumentParser(description=__doc__)
|
parser = argparse.ArgumentParser(description=__doc__)
|
||||||
parser.add_argument('action', choices=('prepare', 'image', 'finalize', 'gate', 'render', 'check-summary'))
|
parser.add_argument('action', choices=('build', 'gate', 'render'))
|
||||||
parser.add_argument('--output', type=Path, default=Path('release.json'))
|
parser.add_argument('--output', type=Path, default=Path('release.json'))
|
||||||
parser.add_argument('--ref', default='main')
|
parser.add_argument('--ref', default='main')
|
||||||
parser.add_argument('--event-sha', default='')
|
parser.add_argument('--event-sha', default='')
|
||||||
parser.add_argument('--image', choices=IMAGES)
|
|
||||||
parser.add_argument('--plan', type=Path, default=Path('build-plan.json'))
|
|
||||||
parser.add_argument('--fragments', type=Path, default=Path('artifacts'))
|
|
||||||
args = parser.parse_args()
|
args = parser.parse_args()
|
||||||
if args.action == 'check-summary':
|
if args.action == 'render':
|
||||||
check_summary()
|
|
||||||
elif args.action == 'render':
|
|
||||||
render(sys.stdin, sys.stdout)
|
render(sys.stdin, sys.stdout)
|
||||||
elif args.action == 'gate':
|
elif args.action == 'gate':
|
||||||
gate(args.output, args.ref, args.event_sha)
|
gate(args.output, args.ref, args.event_sha)
|
||||||
elif args.action == 'prepare':
|
|
||||||
prepare_images(args.output)
|
|
||||||
elif args.action == 'image':
|
|
||||||
if not args.image:
|
|
||||||
parser.error('--image is required')
|
|
||||||
build(args.output, args.image, args.plan)
|
|
||||||
else:
|
else:
|
||||||
finalize_images(args.output, args.fragments, args.plan)
|
build(args.output)
|
||||||
|
|
||||||
|
|
||||||
if __name__ == '__main__':
|
if __name__ == '__main__':
|
||||||
|
|||||||
@@ -1,9 +1,7 @@
|
|||||||
name: renovate-ci
|
name: renovate-ci
|
||||||
|
|
||||||
on:
|
on:
|
||||||
# Read the workflow from the trusted base branch. PR code runs only on the
|
pull_request:
|
||||||
# unprivileged runner selected below.
|
|
||||||
pull_request_target:
|
|
||||||
paths:
|
paths:
|
||||||
- "renovate/**"
|
- "renovate/**"
|
||||||
- ".gitea/workflows/renovate-ci.yaml"
|
- ".gitea/workflows/renovate-ci.yaml"
|
||||||
@@ -28,47 +26,37 @@ permissions:
|
|||||||
|
|
||||||
jobs:
|
jobs:
|
||||||
validate-renovate:
|
validate-renovate:
|
||||||
runs-on: ${{ github.event_name == 'push' && github.ref == 'refs/heads/main' && 'homelab' || 'homelab-pr' }}
|
runs-on: homelab
|
||||||
timeout-minutes: 20
|
timeout-minutes: 20
|
||||||
steps:
|
steps:
|
||||||
- name: Checkout repository
|
- name: Checkout repository
|
||||||
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
|
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
|
||||||
with:
|
|
||||||
ref: ${{ github.event_name == 'pull_request_target' && github.event.pull_request.head.sha || github.sha }}
|
|
||||||
|
|
||||||
# renovate/k8s/cronjob.yaml is the single source of truth for the version.
|
# renovate/k8s/cronjob.yaml is the single source of truth for the image tag,
|
||||||
- name: Resolve the deployed Renovate version
|
# so the same version that runs in the cluster is the one validated here.
|
||||||
|
- name: Resolve the deployed Renovate image
|
||||||
id: image
|
id: image
|
||||||
shell: bash
|
shell: bash
|
||||||
run: |
|
run: |
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
image="$(sed -n 's|.*image:[[:space:]]*\(renovate/renovate:[^[:space:]]*\).*|\1|p' \
|
image="$(sed -n 's|.*image:[[:space:]]*\(renovate/renovate:[^[:space:]]*\).*|\1|p' \
|
||||||
renovate/k8s/cronjob.yaml | head -1)"
|
renovate/k8s/cronjob.yaml | head -1)"
|
||||||
if [[ ! "$image" =~ ^renovate/renovate:([0-9]+\.[0-9]+\.[0-9]+)$ ]]; then
|
if [ -z "$image" ]; then
|
||||||
echo "::error::expected a pinned renovate/renovate semantic version in renovate/k8s/cronjob.yaml"
|
echo "::error::no renovate/renovate image found in renovate/k8s/cronjob.yaml"
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
version="${BASH_REMATCH[1]}"
|
echo "using $image"
|
||||||
echo "using Renovate $version"
|
echo "image=$image" >> "$GITHUB_OUTPUT"
|
||||||
printf 'version=%s\n' "$version" >> "$GITHUB_OUTPUT"
|
|
||||||
|
|
||||||
- name: Prepare pinned validation tools
|
|
||||||
shell: bash
|
|
||||||
run: |
|
|
||||||
set -euo pipefail
|
|
||||||
tools_dir="$(bash .gitea/workflows/install-ci-tools.sh kubeconform node)"
|
|
||||||
echo "$tools_dir" >> "$GITHUB_PATH"
|
|
||||||
|
|
||||||
- name: Validate Renovate repository config
|
- name: Validate Renovate repository config
|
||||||
shell: bash
|
shell: bash
|
||||||
env:
|
|
||||||
RENOVATE_VERSION: ${{ steps.image.outputs.version }}
|
|
||||||
run: |
|
run: |
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
npm_cache="$(mktemp -d "${RUNNER_TEMP:-/tmp}/renovate-npm-cache.XXXXXXXX")"
|
docker run --rm \
|
||||||
trap 'rm -rf "$npm_cache"' EXIT
|
-v "$PWD/renovate:/opt/renovate:ro" \
|
||||||
NPM_CONFIG_CACHE="$npm_cache" RENOVATE_CONFIG_FILE="$PWD/renovate/renovate.json" \
|
-e RENOVATE_CONFIG_FILE=/opt/renovate/renovate.json \
|
||||||
npm exec --yes --package="renovate@${RENOVATE_VERSION}" -- renovate-config-validator
|
"${{ steps.image.outputs.image }}" \
|
||||||
|
renovate-config-validator /opt/renovate/renovate.json
|
||||||
|
|
||||||
# The CronJob cannot read the repository, so renovate/k8s/configmap.yaml
|
# The CronJob cannot read the repository, so renovate/k8s/configmap.yaml
|
||||||
# carries an inlined copy of the config. Fail if it no longer matches.
|
# carries an inlined copy of the config. Fail if it no longer matches.
|
||||||
@@ -82,6 +70,8 @@ jobs:
|
|||||||
shell: bash
|
shell: bash
|
||||||
run: |
|
run: |
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
tools_dir="$(bash .gitea/workflows/install-ci-tools.sh kubeconform)"
|
||||||
|
export PATH="$tools_dir:$PATH"
|
||||||
kubeconform \
|
kubeconform \
|
||||||
-strict \
|
-strict \
|
||||||
-ignore-missing-schemas \
|
-ignore-missing-schemas \
|
||||||
|
|||||||
@@ -32,14 +32,11 @@ concurrency:
|
|||||||
|
|
||||||
jobs:
|
jobs:
|
||||||
run-renovate:
|
run-renovate:
|
||||||
if: github.ref == 'refs/heads/main'
|
|
||||||
runs-on: homelab
|
runs-on: homelab
|
||||||
timeout-minutes: 60
|
timeout-minutes: 60
|
||||||
steps:
|
steps:
|
||||||
- name: Checkout repository
|
- name: Checkout repository
|
||||||
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
|
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
|
||||||
with:
|
|
||||||
ref: refs/heads/main
|
|
||||||
|
|
||||||
# renovate/k8s/cronjob.yaml is the single source of truth for the image tag.
|
# renovate/k8s/cronjob.yaml is the single source of truth for the image tag.
|
||||||
# Reading it here means this workflow validates and runs the exact version
|
# Reading it here means this workflow validates and runs the exact version
|
||||||
@@ -51,23 +48,21 @@ jobs:
|
|||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
image="$(sed -n 's|.*image:[[:space:]]*\(renovate/renovate:[^[:space:]]*\).*|\1|p' \
|
image="$(sed -n 's|.*image:[[:space:]]*\(renovate/renovate:[^[:space:]]*\).*|\1|p' \
|
||||||
renovate/k8s/cronjob.yaml | head -1)"
|
renovate/k8s/cronjob.yaml | head -1)"
|
||||||
if [[ ! "$image" =~ ^renovate/renovate:[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
|
if [ -z "$image" ]; then
|
||||||
echo "::error::expected a pinned renovate/renovate semantic version in renovate/k8s/cronjob.yaml"
|
echo "::error::no renovate/renovate image found in renovate/k8s/cronjob.yaml"
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
echo "using $image"
|
echo "using $image"
|
||||||
printf 'image=%s\n' "$image" >> "$GITHUB_OUTPUT"
|
echo "image=$image" >> "$GITHUB_OUTPUT"
|
||||||
|
|
||||||
- name: Validate Renovate config
|
- name: Validate Renovate config
|
||||||
shell: bash
|
shell: bash
|
||||||
env:
|
|
||||||
RENOVATE_IMAGE: ${{ steps.image.outputs.image }}
|
|
||||||
run: |
|
run: |
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
docker run --rm \
|
docker run --rm \
|
||||||
-v "$PWD/renovate/renovate.json:/opt/renovate/renovate.json:ro" \
|
-v "$PWD/renovate/renovate.json:/opt/renovate/renovate.json:ro" \
|
||||||
-e RENOVATE_CONFIG_FILE=/opt/renovate/renovate.json \
|
-e RENOVATE_CONFIG_FILE=/opt/renovate/renovate.json \
|
||||||
"$RENOVATE_IMAGE" \
|
"${{ steps.image.outputs.image }}" \
|
||||||
renovate-config-validator
|
renovate-config-validator
|
||||||
|
|
||||||
- name: Run Renovate
|
- name: Run Renovate
|
||||||
@@ -78,7 +73,6 @@ jobs:
|
|||||||
RENOVATE_REPOSITORIES: ${{ inputs.repositories }}
|
RENOVATE_REPOSITORIES: ${{ inputs.repositories }}
|
||||||
RENOVATE_DRY_RUN: ${{ inputs.dry_run && 'full' || '' }}
|
RENOVATE_DRY_RUN: ${{ inputs.dry_run && 'full' || '' }}
|
||||||
LOG_LEVEL: ${{ inputs.log_level }}
|
LOG_LEVEL: ${{ inputs.log_level }}
|
||||||
RENOVATE_IMAGE: ${{ steps.image.outputs.image }}
|
|
||||||
run: |
|
run: |
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
@@ -95,4 +89,4 @@ jobs:
|
|||||||
-e RENOVATE_CONFIG_FILE=/opt/renovate/renovate.json \
|
-e RENOVATE_CONFIG_FILE=/opt/renovate/renovate.json \
|
||||||
-e RENOVATE_BASE_DIR=/tmp/renovate \
|
-e RENOVATE_BASE_DIR=/tmp/renovate \
|
||||||
-e LOG_LEVEL="${LOG_LEVEL:-info}" \
|
-e LOG_LEVEL="${LOG_LEVEL:-info}" \
|
||||||
"$RENOVATE_IMAGE"
|
"${{ steps.image.outputs.image }}"
|
||||||
@@ -20,7 +20,7 @@ ssh_opts=(-i "$key_dir/key" -p "${DEPLOY_PORT:-22}" -o BatchMode=yes -o StrictHo
|
|||||||
-o "UserKnownHostsFile=$key_dir/known_hosts" -o ConnectTimeout=15
|
-o "UserKnownHostsFile=$key_dir/known_hosts" -o ConnectTimeout=15
|
||||||
-o ServerAliveInterval=15 -o ServerAliveCountMax=4)
|
-o ServerAliveInterval=15 -o ServerAliveCountMax=4)
|
||||||
controller=.local/lib/homelab-deploy/controller.py
|
controller=.local/lib/homelab-deploy/controller.py
|
||||||
case "${1:?start, apply, verify, smoke or summary required}" in
|
case "${1:?start, apply, verify or smoke required}" in
|
||||||
start)
|
start)
|
||||||
python3 - <<'PY' >"$key_dir/request.json"
|
python3 - <<'PY' >"$key_dir/request.json"
|
||||||
import json
|
import json
|
||||||
@@ -41,30 +41,16 @@ PY
|
|||||||
exit "$rc"
|
exit "$rc"
|
||||||
;;
|
;;
|
||||||
apply|verify|smoke)
|
apply|verify|smoke)
|
||||||
result=0
|
|
||||||
for attempt in 1 2 3; do
|
for attempt in 1 2 3; do
|
||||||
rc=0
|
rc=0
|
||||||
# shellcheck disable=SC2029 # The run ID and operation are validated local arguments, not remote variables.
|
# shellcheck disable=SC2029 # The run ID and operation are validated local arguments, not remote variables.
|
||||||
ssh "${ssh_opts[@]}" "$DEPLOY_USER@$DEPLOY_HOST" python3 "$controller" follow "$DEPLOY_RUN_ID" "$1" || rc=$?
|
ssh "${ssh_opts[@]}" "$DEPLOY_USER@$DEPLOY_HOST" python3 "$controller" follow "$DEPLOY_RUN_ID" "$1" || rc=$?
|
||||||
[ "$rc" -eq 0 ] && break
|
[ "$rc" -eq 0 ] && exit 0
|
||||||
[ "$rc" -eq 255 ] || { result="$rc"; break; }
|
[ "$rc" -eq 255 ] || exit "$rc"
|
||||||
echo "SSH disconnected; reconnecting to the existing deploy ($attempt/3)"
|
echo "SSH disconnected; reconnecting to the existing deploy ($attempt/3)"
|
||||||
if [ "$attempt" -eq 3 ]; then result=255; break; fi
|
|
||||||
sleep 5
|
sleep 5
|
||||||
done
|
done
|
||||||
exit "$result"
|
exit "$rc"
|
||||||
;;
|
|
||||||
summary)
|
|
||||||
if [ -n "${GITHUB_STEP_SUMMARY:-}" ]; then
|
|
||||||
rc=0
|
|
||||||
# shellcheck disable=SC2029 # The run ID is validated above.
|
|
||||||
ssh "${ssh_opts[@]}" "$DEPLOY_USER@$DEPLOY_HOST" python3 "$controller" summary "$DEPLOY_RUN_ID" >"$key_dir/deploy-summary.md" || rc=$?
|
|
||||||
if [ "$rc" -eq 0 ]; then
|
|
||||||
cat "$key_dir/deploy-summary.md" >>"$GITHUB_STEP_SUMMARY" || echo "WARNING: cannot write the deploy summary"
|
|
||||||
else
|
|
||||||
echo 'Deploy summary is unavailable. The SSH connection failed or the controller did not respond. Check the job log.' >>"$GITHUB_STEP_SUMMARY" || true
|
|
||||||
fi
|
|
||||||
fi
|
|
||||||
;;
|
;;
|
||||||
*) echo "Unknown SSH operation: $1" >&2; exit 1 ;;
|
*) echo "Unknown SSH operation: $1" >&2; exit 1 ;;
|
||||||
esac
|
esac
|
||||||
@@ -94,6 +94,7 @@ replacements.txt
|
|||||||
.idea
|
.idea
|
||||||
|
|
||||||
# Temp files
|
# Temp files
|
||||||
|
edu_master/temp/
|
||||||
temp/*
|
temp/*
|
||||||
# Local-only tooling scratch space (pinned CI tools, verification scripts)
|
# Local-only tooling scratch space (pinned CI tools, verification scripts)
|
||||||
tmp/
|
tmp/
|
||||||
|
|||||||
@@ -0,0 +1,14 @@
|
|||||||
|
EDU_LOGIN=your_edu_login_here
|
||||||
|
EDU_PASSWORD=your_edu_password_here
|
||||||
|
EDU_URL_LOGIN=https://edu.edu.vn.ua/user/login
|
||||||
|
EDU_URL_VERIFY=https://edu.edu.vn.ua/course/userlist
|
||||||
|
PHPSESSID_INTERVAL=10
|
||||||
|
USER_AGENT="Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/142.0.0.0 Safari/537.36"
|
||||||
|
WEBINAR_URL=https://edu.edu.vn.ua/webinar/useractive
|
||||||
|
WEBINAR_CHECK_INTERVAL=60
|
||||||
|
REDIS_HOST=redis
|
||||||
|
REDIS_PORT=6379
|
||||||
|
PLAYWRIGHT_WS=ws://playwright-service:3000/ws
|
||||||
|
TZ=Europe/Kyiv
|
||||||
|
WEBINAR_TELEGRAM_TOKEN=your_telegram_bot_token_here
|
||||||
|
WEBINAR_ADMIN_ID=123456789
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
1.56.0
|
||||||
@@ -0,0 +1,14 @@
|
|||||||
|
# EDU deployment ownership
|
||||||
|
|
||||||
|
Application source and release builds: `forust/edu-master`.
|
||||||
|
The homelab pipeline deploys `edu_master/k8s` and preserves explicit image digests.
|
||||||
|
The application copies in this directory are legacy and are not build inputs.
|
||||||
|
Do not publish EDU `prod` images from homelab or resolve releases from moving tags.
|
||||||
|
|
||||||
|
For an EDU release, validate both images, select their digests in the keeper and
|
||||||
|
checker manifests, and run the existing homelab validation/apply/verification
|
||||||
|
helpers against this service. Keep the existing Secret and Redis PVC.
|
||||||
|
Coordinate Redis authentication changes with both clients and all init/probes;
|
||||||
|
keep a pre-rollout Redis backup and both previous compatible image references.
|
||||||
|
The current HTTP checker does not depend on Playwright; check other consumers
|
||||||
|
before removing the separate browser service.
|
||||||
@@ -0,0 +1,49 @@
|
|||||||
|
services:
|
||||||
|
redis:
|
||||||
|
image: redis:8.10.2-alpine
|
||||||
|
restart: unless-stopped
|
||||||
|
volumes:
|
||||||
|
- redis-data:/data
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD", "redis-cli", "ping"]
|
||||||
|
interval: 5s
|
||||||
|
timeout: 3s
|
||||||
|
retries: 5
|
||||||
|
|
||||||
|
playwright-service:
|
||||||
|
image: mcr.microsoft.com/playwright:v1.56.0-jammy
|
||||||
|
restart: unless-stopped
|
||||||
|
command: npx -y playwright@1.56.0 run-server --port 3000 --path /ws
|
||||||
|
|
||||||
|
session-keeper:
|
||||||
|
build: ./phpsessid-bot
|
||||||
|
image: gcr.forust.xyz/forust/session-keeper:prod
|
||||||
|
pull_policy: build
|
||||||
|
env_file: .env
|
||||||
|
restart: unless-stopped
|
||||||
|
depends_on:
|
||||||
|
redis:
|
||||||
|
condition: service_healthy
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD-SHELL", "redis-cli -h redis EXISTS EDU_PHPSESSID | grep -q 1"]
|
||||||
|
interval: 30s
|
||||||
|
timeout: 5s
|
||||||
|
retries: 10
|
||||||
|
start_period: 60s
|
||||||
|
|
||||||
|
webinar-checker:
|
||||||
|
build: ./webinar-checker
|
||||||
|
image: gcr.forust.xyz/forust/webinar-checker:prod
|
||||||
|
pull_policy: build
|
||||||
|
env_file: .env
|
||||||
|
restart: unless-stopped
|
||||||
|
depends_on:
|
||||||
|
redis:
|
||||||
|
condition: service_healthy
|
||||||
|
session-keeper:
|
||||||
|
condition: service_healthy
|
||||||
|
playwright-service:
|
||||||
|
condition: service_started
|
||||||
|
|
||||||
|
volumes:
|
||||||
|
redis-data:
|
||||||
Whitespace-only changes.
@@ -0,0 +1,115 @@
|
|||||||
|
apiVersion: monitoring.coreos.com/v1
|
||||||
|
kind: PrometheusRule
|
||||||
|
metadata:
|
||||||
|
name: edu-master-webinar
|
||||||
|
namespace: edu-master
|
||||||
|
labels:
|
||||||
|
release: prometheus-stack
|
||||||
|
spec:
|
||||||
|
groups:
|
||||||
|
- name: edu_master.webinar
|
||||||
|
rules:
|
||||||
|
# No successful webinar check for 5m (~2-3 missed 2-min checks).
|
||||||
|
# Catches: playwright hangs/timeouts, version skew, site changes, hung job.
|
||||||
|
# The last_success > 0 guard is mandatory: checker.py initialises
|
||||||
|
# last_success to 0, so without it `time() - 0` equals the current epoch
|
||||||
|
# and humanizeDuration renders ~20722d on every pod restart. Keep the
|
||||||
|
# duration expression on the left so $value stays the real gap.
|
||||||
|
- alert: WebinarCheckerNoSuccessfulCheck
|
||||||
|
expr: |
|
||||||
|
((time() - webinar_check_last_success_timestamp_seconds) > 300)
|
||||||
|
and (webinar_check_last_success_timestamp_seconds > 0)
|
||||||
|
and (webinar_check_last_run_timestamp_seconds > 0)
|
||||||
|
for: 2m
|
||||||
|
labels:
|
||||||
|
severity: critical
|
||||||
|
annotations:
|
||||||
|
summary: "Webinar checker has no successful check for 5m"
|
||||||
|
description: "edu-master/webinar-checker: last successful webinar check was {{ $value | humanizeDuration }} ago. Checks are failing or hanging (see consecutive failures alert). Notifications about new webinars are NOT being sent."
|
||||||
|
|
||||||
|
# Checks are running but none has ever succeeded since pod start.
|
||||||
|
# Split out from the rule above so a zeroed gauge never feeds
|
||||||
|
# humanizeDuration.
|
||||||
|
- alert: WebinarCheckerNeverSucceeded
|
||||||
|
expr: |
|
||||||
|
(webinar_check_last_success_timestamp_seconds == 0)
|
||||||
|
and (webinar_check_last_run_timestamp_seconds > 0)
|
||||||
|
for: 10m
|
||||||
|
labels:
|
||||||
|
severity: critical
|
||||||
|
annotations:
|
||||||
|
summary: "Webinar checker has never completed a successful check"
|
||||||
|
description: 'edu-master/webinar-checker: checks have been running for 10m but not one has ever succeeded since the pod started, so every check is failing. Check pod logs (Loki: {namespace="edu-master", container="webinar-checker"}).'
|
||||||
|
|
||||||
|
# Fast path: 3 consecutive failures (~6+ min at 2-min interval).
|
||||||
|
- alert: WebinarCheckerConsecutiveFailures
|
||||||
|
expr: |
|
||||||
|
webinar_check_consecutive_failures >= 3
|
||||||
|
for: 5m
|
||||||
|
labels:
|
||||||
|
severity: critical
|
||||||
|
annotations:
|
||||||
|
summary: "Webinar checker failing consecutively"
|
||||||
|
description: 'edu-master/webinar-checker: {{ $value }} consecutive webinar check failures (timeout / http error / page error). Check pod logs (Loki: {namespace="edu-master", container="webinar-checker"}).'
|
||||||
|
|
||||||
|
- alert: WebinarCheckerNeverStarted
|
||||||
|
expr: |
|
||||||
|
(time() - edu_process_start > 120)
|
||||||
|
and (webinar_check_last_run_timestamp_seconds == 0)
|
||||||
|
for: 2m
|
||||||
|
labels:
|
||||||
|
severity: critical
|
||||||
|
annotations:
|
||||||
|
summary: "Webinar checker job has not started"
|
||||||
|
description: "The process exposes metrics but its webinar job has never started."
|
||||||
|
|
||||||
|
- alert: WebinarDeliveryPending
|
||||||
|
expr: edu_delivery_pending > 0
|
||||||
|
for: 5m
|
||||||
|
labels:
|
||||||
|
severity: warning
|
||||||
|
annotations:
|
||||||
|
summary: "Webinar notifications await delivery"
|
||||||
|
description: "Telegram delivery has pending recipients. Check delivery failures and retry status."
|
||||||
|
|
||||||
|
- alert: EduRedisUnavailable
|
||||||
|
expr: edu_redis_connected == 0
|
||||||
|
for: 2m
|
||||||
|
labels:
|
||||||
|
severity: critical
|
||||||
|
annotations:
|
||||||
|
summary: "EDU checker cannot reach Redis"
|
||||||
|
description: "Redis health checks are failing; checker commands and delivery may be unavailable."
|
||||||
|
|
||||||
|
# Metrics endpoint not scraped for 10m: pod down, metrics server dead, or ServiceMonitor broken.
|
||||||
|
- alert: WebinarCheckerScrapeDown
|
||||||
|
expr: |
|
||||||
|
absent(webinar_check_last_run_timestamp_seconds) == 1
|
||||||
|
for: 10m
|
||||||
|
labels:
|
||||||
|
severity: critical
|
||||||
|
annotations:
|
||||||
|
summary: "Webinar checker metrics missing"
|
||||||
|
description: "edu-master/webinar-checker: no metrics series for 10m. Pod may be down, metrics server dead, or ServiceMonitor/Service broken. Webinar checks are unobserved."
|
||||||
|
|
||||||
|
# EDU session lost: session-keeper down or credentials expired. Without PHPSESSID every check is skipped.
|
||||||
|
- alert: EduPhpsessidMissing
|
||||||
|
expr: |
|
||||||
|
edu_phpsessid_present == 0
|
||||||
|
for: 10m
|
||||||
|
labels:
|
||||||
|
severity: critical
|
||||||
|
annotations:
|
||||||
|
summary: "EDU_PHPSESSID missing"
|
||||||
|
description: "edu-master: EDU_PHPSESSID absent from redis for 10m. Webinar/diari/schedule checks are all skipped. Check session-keeper logs and EDU credentials."
|
||||||
|
|
||||||
|
# Hard deps: checker deployment unavailable.
|
||||||
|
- alert: WebinarCheckerDeploymentDown
|
||||||
|
expr: |
|
||||||
|
kube_deployment_status_replicas_unavailable{deployment="webinar-checker", namespace="edu-master"} > 0
|
||||||
|
for: 10m
|
||||||
|
labels:
|
||||||
|
severity: critical
|
||||||
|
annotations:
|
||||||
|
summary: "Webinar checker deployment unavailable"
|
||||||
|
description: "edu-master/webinar-checker deployment has {{ $value }} unavailable replica(s) for 10m."
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
apiVersion: v1
|
apiVersion: v1
|
||||||
kind: Namespace
|
kind: Namespace
|
||||||
metadata:
|
metadata:
|
||||||
name: paperless
|
name: edu-master
|
||||||
@@ -0,0 +1,69 @@
|
|||||||
|
apiVersion: apps/v1
|
||||||
|
kind: Deployment
|
||||||
|
metadata:
|
||||||
|
name: playwright-service
|
||||||
|
namespace: edu-master
|
||||||
|
labels:
|
||||||
|
app: edu-master-playwright
|
||||||
|
spec:
|
||||||
|
replicas: 1
|
||||||
|
selector:
|
||||||
|
matchLabels:
|
||||||
|
app: edu-master-playwright
|
||||||
|
strategy:
|
||||||
|
type: Recreate
|
||||||
|
template:
|
||||||
|
metadata:
|
||||||
|
labels:
|
||||||
|
app: edu-master-playwright
|
||||||
|
spec:
|
||||||
|
containers:
|
||||||
|
- name: playwright
|
||||||
|
# renovate: datasource=docker depName=mcr.microsoft.com/playwright versioning=docker
|
||||||
|
image: mcr.microsoft.com/playwright:v1.56.0-jammy
|
||||||
|
imagePullPolicy: IfNotPresent
|
||||||
|
# p95 412M, max 478M over 7 days, no limit before. Request is set at p95
|
||||||
|
# so the pod is not an eviction candidate; the limit stays above 2x the
|
||||||
|
# request because browser page lifetimes are unpredictable.
|
||||||
|
resources:
|
||||||
|
requests:
|
||||||
|
cpu: "200m"
|
||||||
|
memory: "416Mi"
|
||||||
|
limits:
|
||||||
|
memory: "1Gi"
|
||||||
|
command:
|
||||||
|
- npx
|
||||||
|
- -y
|
||||||
|
- playwright@1.56.0
|
||||||
|
- run-server
|
||||||
|
- --port
|
||||||
|
- "3000"
|
||||||
|
- --path
|
||||||
|
- /ws
|
||||||
|
ports:
|
||||||
|
- containerPort: 3000
|
||||||
|
readinessProbe:
|
||||||
|
tcpSocket:
|
||||||
|
port: 3000
|
||||||
|
initialDelaySeconds: 5
|
||||||
|
periodSeconds: 10
|
||||||
|
timeoutSeconds: 3
|
||||||
|
livenessProbe:
|
||||||
|
tcpSocket:
|
||||||
|
port: 3000
|
||||||
|
initialDelaySeconds: 15
|
||||||
|
periodSeconds: 20
|
||||||
|
timeoutSeconds: 3
|
||||||
|
---
|
||||||
|
apiVersion: v1
|
||||||
|
kind: Service
|
||||||
|
metadata:
|
||||||
|
name: playwright-service
|
||||||
|
namespace: edu-master
|
||||||
|
spec:
|
||||||
|
selector:
|
||||||
|
app: edu-master-playwright
|
||||||
|
ports:
|
||||||
|
- name: ws
|
||||||
|
port: 3000
|
||||||
|
targetPort: 3000
|
||||||
@@ -0,0 +1,22 @@
|
|||||||
|
apiVersion: networking.k8s.io/v1
|
||||||
|
kind: NetworkPolicy
|
||||||
|
metadata:
|
||||||
|
name: redis-clients-only
|
||||||
|
namespace: edu-master
|
||||||
|
spec:
|
||||||
|
podSelector:
|
||||||
|
matchLabels:
|
||||||
|
app: edu-master-redis
|
||||||
|
policyTypes:
|
||||||
|
- Ingress
|
||||||
|
ingress:
|
||||||
|
- from:
|
||||||
|
- podSelector:
|
||||||
|
matchLabels:
|
||||||
|
app: edu-master-session-keeper
|
||||||
|
- podSelector:
|
||||||
|
matchLabels:
|
||||||
|
app: edu-master-webinar-checker
|
||||||
|
ports:
|
||||||
|
- protocol: TCP
|
||||||
|
port: 6379
|
||||||
@@ -0,0 +1,96 @@
|
|||||||
|
apiVersion: apps/v1
|
||||||
|
kind: StatefulSet
|
||||||
|
metadata:
|
||||||
|
name: redis
|
||||||
|
namespace: edu-master
|
||||||
|
labels:
|
||||||
|
app: edu-master-redis
|
||||||
|
spec:
|
||||||
|
serviceName: redis
|
||||||
|
replicas: 1
|
||||||
|
selector:
|
||||||
|
matchLabels:
|
||||||
|
app: edu-master-redis
|
||||||
|
template:
|
||||||
|
metadata:
|
||||||
|
labels:
|
||||||
|
app: edu-master-redis
|
||||||
|
spec:
|
||||||
|
containers:
|
||||||
|
- name: redis
|
||||||
|
image: redis:8.10.2-alpine
|
||||||
|
imagePullPolicy: IfNotPresent
|
||||||
|
env:
|
||||||
|
- name: REDIS_PASSWORD
|
||||||
|
valueFrom:
|
||||||
|
secretKeyRef:
|
||||||
|
name: edu-master-secrets
|
||||||
|
key: REDIS_PASSWORD
|
||||||
|
- name: REDISCLI_AUTH
|
||||||
|
valueFrom:
|
||||||
|
secretKeyRef:
|
||||||
|
name: edu-master-secrets
|
||||||
|
key: REDIS_PASSWORD
|
||||||
|
command:
|
||||||
|
- /bin/sh
|
||||||
|
- -ec
|
||||||
|
- |
|
||||||
|
case "$REDIS_PASSWORD" in *[!0-9a-fA-F]*|'') echo 'REDIS_PASSWORD must be 64 hex characters' >&2; exit 1;; esac
|
||||||
|
[ "${#REDIS_PASSWORD}" -eq 64 ] || { echo 'REDIS_PASSWORD must be 64 hex characters' >&2; exit 1; }
|
||||||
|
umask 077
|
||||||
|
printf 'requirepass "%s"\n' "$REDIS_PASSWORD" > /tmp/redis-auth.conf
|
||||||
|
chown redis:redis /tmp/redis-auth.conf
|
||||||
|
exec docker-entrypoint.sh redis-server /tmp/redis-auth.conf
|
||||||
|
ports:
|
||||||
|
- containerPort: 6379
|
||||||
|
volumeMounts:
|
||||||
|
- name: redis-data
|
||||||
|
mountPath: /data
|
||||||
|
resources:
|
||||||
|
requests:
|
||||||
|
cpu: 25m
|
||||||
|
memory: 32Mi
|
||||||
|
limits:
|
||||||
|
cpu: 250m
|
||||||
|
memory: 128Mi
|
||||||
|
readinessProbe:
|
||||||
|
exec:
|
||||||
|
command: ["redis-cli", "ping"]
|
||||||
|
initialDelaySeconds: 5
|
||||||
|
periodSeconds: 5
|
||||||
|
timeoutSeconds: 3
|
||||||
|
livenessProbe:
|
||||||
|
exec:
|
||||||
|
command: ["redis-cli", "ping"]
|
||||||
|
initialDelaySeconds: 10
|
||||||
|
periodSeconds: 10
|
||||||
|
timeoutSeconds: 3
|
||||||
|
volumes:
|
||||||
|
- name: redis-data
|
||||||
|
persistentVolumeClaim:
|
||||||
|
claimName: redis-data-pvc
|
||||||
|
---
|
||||||
|
apiVersion: v1
|
||||||
|
kind: PersistentVolumeClaim
|
||||||
|
metadata:
|
||||||
|
name: redis-data-pvc
|
||||||
|
namespace: edu-master
|
||||||
|
spec:
|
||||||
|
accessModes:
|
||||||
|
- ReadWriteOnce
|
||||||
|
resources:
|
||||||
|
requests:
|
||||||
|
storage: 1Gi
|
||||||
|
---
|
||||||
|
apiVersion: v1
|
||||||
|
kind: Service
|
||||||
|
metadata:
|
||||||
|
name: redis
|
||||||
|
namespace: edu-master
|
||||||
|
spec:
|
||||||
|
selector:
|
||||||
|
app: edu-master-redis
|
||||||
|
ports:
|
||||||
|
- name: redis
|
||||||
|
port: 6379
|
||||||
|
targetPort: 6379
|
||||||
@@ -0,0 +1,50 @@
|
|||||||
|
# One-time Job to migrate redis state from docker compose to k8s (maintenance window).
|
||||||
|
# The .example file is not applied by the deploy pipeline (mask *.example.yaml).
|
||||||
|
#
|
||||||
|
# Runbook:
|
||||||
|
# 1. docker compose -f <repo>/edu_master/compose.yaml stop # SIGTERM -> redis will flush dump.rdb
|
||||||
|
# 2. docker run --rm -v edu_master_redis-data:/data \
|
||||||
|
# -v /tmp/edu-master-backup:/backup \
|
||||||
|
# redis:alpine sh -c "cp /data/dump.rdb /backup/ && ls -la /backup"
|
||||||
|
# 3. kubectl apply -f edu_master/k8s/namespace.yaml
|
||||||
|
# 4. kubectl apply -f <only the PVC from redis.yaml> # seed must come BEFORE redis pod starts
|
||||||
|
# 5. kubectl apply -f edu_master/k8s/restore-seed-job.yaml.example
|
||||||
|
# kubectl wait --for=condition=complete job/redis-restore-seed -n edu-master --timeout=120s
|
||||||
|
# 6. kubectl delete job redis-restore-seed -n edu-master
|
||||||
|
# 7. kubectl apply -f edu_master/k8s/ -R # apply remaining manifests
|
||||||
|
apiVersion: batch/v1
|
||||||
|
kind: Job
|
||||||
|
metadata:
|
||||||
|
name: redis-restore-seed
|
||||||
|
namespace: edu-master
|
||||||
|
spec:
|
||||||
|
backoffLimit: 2
|
||||||
|
ttlSecondsAfterFinished: 3600
|
||||||
|
template:
|
||||||
|
spec:
|
||||||
|
restartPolicy: Never
|
||||||
|
containers:
|
||||||
|
- name: seed
|
||||||
|
image: redis:alpine
|
||||||
|
command:
|
||||||
|
- /bin/sh
|
||||||
|
- -ec
|
||||||
|
- |
|
||||||
|
ls -la /backup
|
||||||
|
cp /backup/dump.rdb /data/dump.rdb
|
||||||
|
chmod 644 /data/dump.rdb
|
||||||
|
ls -la /data
|
||||||
|
volumeMounts:
|
||||||
|
- name: redis-data
|
||||||
|
mountPath: /data
|
||||||
|
- name: backup
|
||||||
|
mountPath: /backup
|
||||||
|
readOnly: true
|
||||||
|
volumes:
|
||||||
|
- name: redis-data
|
||||||
|
persistentVolumeClaim:
|
||||||
|
claimName: redis-data-pvc
|
||||||
|
- name: backup
|
||||||
|
hostPath:
|
||||||
|
path: /tmp/edu-master-backup
|
||||||
|
type: DirectoryOrCreate
|
||||||
@@ -0,0 +1,29 @@
|
|||||||
|
apiVersion: v1
|
||||||
|
kind: Secret
|
||||||
|
metadata:
|
||||||
|
name: edu-master-secrets
|
||||||
|
namespace: edu-master
|
||||||
|
type: Opaque
|
||||||
|
stringData:
|
||||||
|
# Session keeper credentials
|
||||||
|
KEEPER_LOGIN: ""
|
||||||
|
KEEPER_PASSWORD: ""
|
||||||
|
KEEPER_INTERVAL: "10"
|
||||||
|
# EDU links
|
||||||
|
EDU_URL_BASE: "https://edu.edu.vn.ua"
|
||||||
|
EDU_URL_LOGIN: "/user/login"
|
||||||
|
EDU_URL_COURSES: "/course/userlist"
|
||||||
|
EDU_URL_WEBINAR: "/webinar/useractive"
|
||||||
|
# Playwright
|
||||||
|
USER_AGENT: ""
|
||||||
|
PLAYWRIGHT_WS: "ws://playwright-service:3000/ws"
|
||||||
|
# Webinar-checker
|
||||||
|
WEBINAR_TELEGRAM_TOKEN: ""
|
||||||
|
WEBINAR_ADMIN_ID: ""
|
||||||
|
WEBINAR_CHECK_INTERVAL: "60"
|
||||||
|
# Prometheus metrics endpoint (scraped via ServiceMonitor, alerts in k8s/alerts.yaml)
|
||||||
|
METRICS_PORT: "8000"
|
||||||
|
# Database
|
||||||
|
REDIS_HOST: "redis"
|
||||||
|
REDIS_PORT: "6379"
|
||||||
|
TZ: "Europe/Kyiv"
|
||||||
@@ -0,0 +1,15 @@
|
|||||||
|
apiVersion: v1
|
||||||
|
kind: Service
|
||||||
|
metadata:
|
||||||
|
name: webinar-checker
|
||||||
|
namespace: edu-master
|
||||||
|
labels:
|
||||||
|
app: edu-master-webinar-checker
|
||||||
|
spec:
|
||||||
|
selector:
|
||||||
|
app: edu-master-webinar-checker
|
||||||
|
ports:
|
||||||
|
- name: metrics
|
||||||
|
port: 8000
|
||||||
|
targetPort: metrics
|
||||||
|
protocol: TCP
|
||||||
@@ -1,14 +1,14 @@
|
|||||||
apiVersion: monitoring.coreos.com/v1
|
apiVersion: monitoring.coreos.com/v1
|
||||||
kind: ServiceMonitor
|
kind: ServiceMonitor
|
||||||
metadata:
|
metadata:
|
||||||
name: netbird-server
|
name: webinar-checker
|
||||||
namespace: netbird
|
namespace: edu-master
|
||||||
labels:
|
labels:
|
||||||
release: prometheus-stack
|
release: prometheus-stack
|
||||||
spec:
|
spec:
|
||||||
selector:
|
selector:
|
||||||
matchLabels:
|
matchLabels:
|
||||||
app: netbird-server
|
app: edu-master-webinar-checker
|
||||||
endpoints:
|
endpoints:
|
||||||
- port: metrics
|
- port: metrics
|
||||||
path: /metrics
|
path: /metrics
|
||||||
@@ -0,0 +1,69 @@
|
|||||||
|
apiVersion: apps/v1
|
||||||
|
kind: Deployment
|
||||||
|
metadata:
|
||||||
|
annotations:
|
||||||
|
reloader.stakater.com/auto: "true"
|
||||||
|
name: session-keeper
|
||||||
|
namespace: edu-master
|
||||||
|
labels:
|
||||||
|
app: edu-master-session-keeper
|
||||||
|
spec:
|
||||||
|
replicas: 1
|
||||||
|
selector:
|
||||||
|
matchLabels:
|
||||||
|
app: edu-master-session-keeper
|
||||||
|
strategy:
|
||||||
|
type: Recreate
|
||||||
|
template:
|
||||||
|
metadata:
|
||||||
|
annotations:
|
||||||
|
edu.forust.xyz/source-commit: "90829d6c8080b9928f9da23587678e640939e10a"
|
||||||
|
labels:
|
||||||
|
app: edu-master-session-keeper
|
||||||
|
spec:
|
||||||
|
initContainers:
|
||||||
|
- name: wait-redis
|
||||||
|
image: redis:8.10.2-alpine
|
||||||
|
env:
|
||||||
|
- name: REDISCLI_AUTH
|
||||||
|
valueFrom:
|
||||||
|
secretKeyRef:
|
||||||
|
name: edu-master-secrets
|
||||||
|
key: REDIS_PASSWORD
|
||||||
|
command:
|
||||||
|
- /bin/sh
|
||||||
|
- -ec
|
||||||
|
- |
|
||||||
|
i=0
|
||||||
|
until redis-cli -h redis ping | grep -q PONG; do
|
||||||
|
i=$((i+1))
|
||||||
|
[ "$i" -ge 300 ] && echo "TIMEOUT: redis not ready" && exit 1
|
||||||
|
sleep 2
|
||||||
|
done
|
||||||
|
echo "redis is ready"
|
||||||
|
containers:
|
||||||
|
- name: session-keeper
|
||||||
|
image: gcr.forust.xyz/forust/session-keeper@sha256:49285e87cc5bc4cf4ffe190813d87927916c2df8a206daac0aeb7d227c636450
|
||||||
|
envFrom:
|
||||||
|
- secretRef:
|
||||||
|
name: edu-master-secrets
|
||||||
|
env:
|
||||||
|
- name: REDISCLI_AUTH
|
||||||
|
valueFrom:
|
||||||
|
secretKeyRef:
|
||||||
|
name: edu-master-secrets
|
||||||
|
key: REDIS_PASSWORD
|
||||||
|
resources:
|
||||||
|
requests:
|
||||||
|
cpu: 25m
|
||||||
|
memory: 32Mi
|
||||||
|
limits:
|
||||||
|
cpu: 250m
|
||||||
|
memory: 128Mi
|
||||||
|
readinessProbe:
|
||||||
|
exec:
|
||||||
|
command: ["/bin/sh", "-ec", "redis-cli -h redis EXISTS EDU_PHPSESSID | grep -q 1"]
|
||||||
|
initialDelaySeconds: 15
|
||||||
|
periodSeconds: 30
|
||||||
|
timeoutSeconds: 5
|
||||||
|
failureThreshold: 10
|
||||||
@@ -0,0 +1,87 @@
|
|||||||
|
apiVersion: apps/v1
|
||||||
|
kind: Deployment
|
||||||
|
metadata:
|
||||||
|
annotations:
|
||||||
|
reloader.stakater.com/auto: "true"
|
||||||
|
name: webinar-checker
|
||||||
|
namespace: edu-master
|
||||||
|
labels:
|
||||||
|
app: edu-master-webinar-checker
|
||||||
|
spec:
|
||||||
|
replicas: 1
|
||||||
|
selector:
|
||||||
|
matchLabels:
|
||||||
|
app: edu-master-webinar-checker
|
||||||
|
strategy:
|
||||||
|
type: Recreate
|
||||||
|
template:
|
||||||
|
metadata:
|
||||||
|
annotations:
|
||||||
|
edu.forust.xyz/source-commit: "90829d6c8080b9928f9da23587678e640939e10a"
|
||||||
|
labels:
|
||||||
|
app: edu-master-webinar-checker
|
||||||
|
spec:
|
||||||
|
# Enforces dependency order like compose depends_on:
|
||||||
|
# redis healthy -> session-keeper healthy (EXISTS EDU_PHPSESSID)
|
||||||
|
initContainers:
|
||||||
|
- name: wait-deps
|
||||||
|
image: redis:8.10.2-alpine
|
||||||
|
env:
|
||||||
|
- name: REDISCLI_AUTH
|
||||||
|
valueFrom:
|
||||||
|
secretKeyRef:
|
||||||
|
name: edu-master-secrets
|
||||||
|
key: REDIS_PASSWORD
|
||||||
|
command:
|
||||||
|
- /bin/sh
|
||||||
|
- -ec
|
||||||
|
- |
|
||||||
|
i=0
|
||||||
|
until redis-cli -h redis ping | grep -q PONG; do
|
||||||
|
i=$((i+1))
|
||||||
|
[ "$i" -ge 300 ] && echo "TIMEOUT: redis not ready" && exit 1
|
||||||
|
sleep 2
|
||||||
|
done
|
||||||
|
echo "redis ok"
|
||||||
|
until [ "$(redis-cli -h redis EXISTS EDU_PHPSESSID)" = "1" ]; do
|
||||||
|
i=$((i+1))
|
||||||
|
[ "$i" -ge 300 ] && echo "TIMEOUT: no PHPSESSID (session-keeper down?)" && exit 1
|
||||||
|
sleep 2
|
||||||
|
done
|
||||||
|
echo "PHPSESSID ok"
|
||||||
|
containers:
|
||||||
|
- name: webinar-checker
|
||||||
|
image: gcr.forust.xyz/forust/webinar-checker@sha256:66c146f7b43cb9f0dc31ba9aa36d217e01df42ddafba5971b79c12ec215b2c01
|
||||||
|
ports:
|
||||||
|
- name: metrics
|
||||||
|
containerPort: 8000
|
||||||
|
protocol: TCP
|
||||||
|
readinessProbe:
|
||||||
|
httpGet:
|
||||||
|
path: /health
|
||||||
|
port: metrics
|
||||||
|
periodSeconds: 10
|
||||||
|
timeoutSeconds: 3
|
||||||
|
failureThreshold: 12
|
||||||
|
initialDelaySeconds: 10
|
||||||
|
livenessProbe:
|
||||||
|
httpGet:
|
||||||
|
path: /live
|
||||||
|
port: metrics
|
||||||
|
initialDelaySeconds: 60
|
||||||
|
periodSeconds: 15
|
||||||
|
timeoutSeconds: 3
|
||||||
|
failureThreshold: 4
|
||||||
|
envFrom:
|
||||||
|
- secretRef:
|
||||||
|
name: edu-master-secrets
|
||||||
|
env:
|
||||||
|
- name: TZ
|
||||||
|
value: "Europe/Kyiv"
|
||||||
|
resources:
|
||||||
|
requests:
|
||||||
|
cpu: "50m"
|
||||||
|
memory: "192Mi"
|
||||||
|
limits:
|
||||||
|
cpu: "600m"
|
||||||
|
memory: "384Mi"
|
||||||
@@ -0,0 +1,15 @@
|
|||||||
|
FROM python:3.14-slim
|
||||||
|
|
||||||
|
WORKDIR /app
|
||||||
|
|
||||||
|
# Install system dependencies
|
||||||
|
RUN apt-get update && apt-get install -y --no-install-recommends redis-tools && rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
# Install dependencies
|
||||||
|
RUN pip install --no-cache-dir requests==2.32.3 redis==5.2.1
|
||||||
|
|
||||||
|
# Copy application code
|
||||||
|
COPY . .
|
||||||
|
|
||||||
|
# Run the bot
|
||||||
|
CMD ["python", "bot.py"]
|
||||||
@@ -0,0 +1,132 @@
|
|||||||
|
import logging
|
||||||
|
import os
|
||||||
|
import time
|
||||||
|
from datetime import datetime
|
||||||
|
|
||||||
|
import redis
|
||||||
|
import requests
|
||||||
|
|
||||||
|
# Configure logging
|
||||||
|
logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s')
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
# Load configuration (adapted to .env keys)
|
||||||
|
def _env(key, default=None):
|
||||||
|
v = os.getenv(key, default)
|
||||||
|
if isinstance(v, str) and len(v) >= 2 and ((v[0] == '"' and v[-1] == '"') or (v[0] == "'" and v[-1] == "'")):
|
||||||
|
return v[1:-1]
|
||||||
|
return v
|
||||||
|
|
||||||
|
|
||||||
|
LOGIN = _env('KEEPER_LOGIN')
|
||||||
|
PASSWORD = _env('KEEPER_PASSWORD')
|
||||||
|
|
||||||
|
EDU_BASE = _env('EDU_URL_BASE', 'https://edu.edu.vn.ua')
|
||||||
|
EDU_LOGIN_PATH = _env('EDU_URL_LOGIN', '/user/login')
|
||||||
|
EDU_COURSES_PATH = _env('EDU_URL_COURSES', '/course/userlist')
|
||||||
|
URL_LOGIN = f'{EDU_BASE.rstrip("/")}/{EDU_LOGIN_PATH.lstrip("/")}'
|
||||||
|
URL_VERIFY = f'{EDU_BASE.rstrip("/")}/{EDU_COURSES_PATH.lstrip("/")}'
|
||||||
|
|
||||||
|
INTERVAL = int(_env('KEEPER_INTERVAL', 10))
|
||||||
|
USER_AGENT = _env(
|
||||||
|
'USER_AGENT',
|
||||||
|
'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/142.0.0.0 Safari/537.36',
|
||||||
|
)
|
||||||
|
REDIS_HOST = _env('REDIS_HOST', 'redis')
|
||||||
|
REDIS_PORT = int(_env('REDIS_PORT', 6379))
|
||||||
|
|
||||||
|
SUCCESS_FILE = '/tmp/last_success' # noqa: S108
|
||||||
|
|
||||||
|
|
||||||
|
def touch_success_file():
|
||||||
|
"""Updates the timestamp of the success file for healthchecks."""
|
||||||
|
try:
|
||||||
|
with open(SUCCESS_FILE, 'w') as f:
|
||||||
|
f.write(str(datetime.now().timestamp()))
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f'Failed to touch success file: {e}')
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
logger.info('Starting Session Keeper Bot')
|
||||||
|
|
||||||
|
# Connect to Redis
|
||||||
|
try:
|
||||||
|
redis_client = redis.Redis(host=REDIS_HOST, port=REDIS_PORT, decode_responses=True)
|
||||||
|
redis_client.ping()
|
||||||
|
logger.info(f'Connected to Redis at {REDIS_HOST}:{REDIS_PORT}')
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f'Failed to connect to Redis: {e}')
|
||||||
|
return
|
||||||
|
|
||||||
|
session = requests.Session()
|
||||||
|
|
||||||
|
# Set headers
|
||||||
|
headers = {
|
||||||
|
'User-Agent': USER_AGENT,
|
||||||
|
'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7',
|
||||||
|
'Accept-Language': 'en-US,en;q=0.9',
|
||||||
|
'Cache-Control': 'max-age=0',
|
||||||
|
'Upgrade-Insecure-Requests': '1',
|
||||||
|
'Sec-Fetch-Site': 'same-origin',
|
||||||
|
'Sec-Fetch-Mode': 'navigate',
|
||||||
|
'Sec-Fetch-User': '?1',
|
||||||
|
'Sec-Fetch-Dest': 'document',
|
||||||
|
'Sec-Ch-Ua': '"Not_A Brand";v="99", "Chromium";v="142"',
|
||||||
|
'Sec-Ch-Ua-Mobile': '?0',
|
||||||
|
'Sec-Ch-Ua-Platform': '"Linux"',
|
||||||
|
'Accept-Encoding': 'gzip, deflate, br',
|
||||||
|
'Priority': 'u=0, i',
|
||||||
|
}
|
||||||
|
session.headers.update(headers)
|
||||||
|
|
||||||
|
while True:
|
||||||
|
try:
|
||||||
|
logger.info('Attempting login...')
|
||||||
|
|
||||||
|
# Login payload
|
||||||
|
payload = {'login': LOGIN, 'password': PASSWORD}
|
||||||
|
|
||||||
|
# Perform Login
|
||||||
|
# Note: The user request shows a POST to /user/login with form data
|
||||||
|
# We need to make sure we handle the PHPSESSID correctly.
|
||||||
|
# If we already have a PHPSESSID, requests will send it.
|
||||||
|
|
||||||
|
login_response = session.post(URL_LOGIN, data=payload, allow_redirects=True)
|
||||||
|
|
||||||
|
logger.info(f'Login Response Status: {login_response.status_code}')
|
||||||
|
logger.info(f'Cookies after login: {session.cookies.get_dict()}')
|
||||||
|
|
||||||
|
# Verify Session
|
||||||
|
logger.info('Verifying session...')
|
||||||
|
verify_response = session.get(URL_VERIFY, allow_redirects=False)
|
||||||
|
|
||||||
|
logger.info(f'Verify Response Status: {verify_response.status_code}')
|
||||||
|
|
||||||
|
if verify_response.status_code == 200:
|
||||||
|
logger.info('Session verification SUCCESS (200 OK).')
|
||||||
|
touch_success_file()
|
||||||
|
|
||||||
|
# Save PHPSESSID to Redis
|
||||||
|
phpsessid = session.cookies.get('PHPSESSID')
|
||||||
|
if phpsessid:
|
||||||
|
try:
|
||||||
|
redis_client.set('EDU_PHPSESSID', phpsessid)
|
||||||
|
logger.info(f'Saved PHPSESSID to Redis: {phpsessid}')
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f'Failed to save PHPSESSID to Redis: {e}')
|
||||||
|
elif verify_response.status_code == 302:
|
||||||
|
logger.warning('Session verification FAILED (302 Redirect). Session might be invalid.')
|
||||||
|
else:
|
||||||
|
logger.warning(f'Session verification returned unexpected status: {verify_response.status_code}')
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f'An error occurred: {e}')
|
||||||
|
|
||||||
|
logger.info(f'Sleeping for {INTERVAL} minutes...')
|
||||||
|
time.sleep(INTERVAL * 60)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
main()
|
||||||
@@ -0,0 +1,13 @@
|
|||||||
|
FROM python:3.14-slim
|
||||||
|
|
||||||
|
WORKDIR /app
|
||||||
|
|
||||||
|
# renovate: datasource=pypi depName=playwright versioning=pep440
|
||||||
|
ARG PLAYWRIGHT_VERSION=1.56.0
|
||||||
|
|
||||||
|
# Install dependencies - PLAYWRIGHT_VERSION is single-source, renovate updates ARG above and all other places via regexManagers
|
||||||
|
RUN pip install --no-cache-dir pip==25.0.1 && pip install --no-cache-dir playwright==${PLAYWRIGHT_VERSION} redis==5.2.1 requests==2.32.3 "python-telegram-bot[job-queue]==21.10"
|
||||||
|
|
||||||
|
COPY checker.py .
|
||||||
|
|
||||||
|
CMD ["python", "checker.py"]
|
||||||
File diff suppressed because it is too large.
Load diff
@@ -20,8 +20,6 @@ data:
|
|||||||
|
|
||||||
GITEA__mailer__ENABLED: "false"
|
GITEA__mailer__ENABLED: "false"
|
||||||
|
|
||||||
GITEA__metrics__ENABLED: "true"
|
|
||||||
|
|
||||||
# No code/issue search needed: bleve reindexes the whole issue index on
|
# No code/issue search needed: bleve reindexes the whole issue index on
|
||||||
# every pod restart (cron.rebuild_issue_indexer RUN_AT_START) and hammers
|
# every pod restart (cron.rebuild_issue_indexer RUN_AT_START) and hammers
|
||||||
# the rotational disk for an hour. "db" serves issue search from postgres.
|
# the rotational disk for an hour. "db" serves issue search from postgres.
|
||||||
|
|||||||
@@ -3,8 +3,6 @@ kind: Service
|
|||||||
metadata:
|
metadata:
|
||||||
name: gitea-service
|
name: gitea-service
|
||||||
namespace: gitea
|
namespace: gitea
|
||||||
labels:
|
|
||||||
app: gitea
|
|
||||||
spec:
|
spec:
|
||||||
selector:
|
selector:
|
||||||
app: gitea
|
app: gitea
|
||||||
|
|||||||
@@ -7,8 +7,7 @@ spec:
|
|||||||
entryPoints:
|
entryPoints:
|
||||||
- websecure
|
- websecure
|
||||||
routes:
|
routes:
|
||||||
# Metrics are scraped directly through the cluster Service.
|
- match: Host(`gitea.forust.xyz`) || Host(`git.forust.xyz`)
|
||||||
- match: (Host(`gitea.forust.xyz`) || Host(`git.forust.xyz`)) && !PathPrefix(`/metrics`)
|
|
||||||
kind: Rule
|
kind: Rule
|
||||||
services:
|
services:
|
||||||
- name: gitea-service
|
- name: gitea-service
|
||||||
|
|||||||
@@ -1,16 +0,0 @@
|
|||||||
apiVersion: monitoring.coreos.com/v1
|
|
||||||
kind: ServiceMonitor
|
|
||||||
metadata:
|
|
||||||
name: gitea
|
|
||||||
namespace: gitea
|
|
||||||
labels:
|
|
||||||
release: prometheus-stack
|
|
||||||
spec:
|
|
||||||
selector:
|
|
||||||
matchLabels:
|
|
||||||
app: gitea
|
|
||||||
endpoints:
|
|
||||||
- port: http
|
|
||||||
path: /metrics
|
|
||||||
interval: 30s
|
|
||||||
scrapeTimeout: 10s
|
|
||||||
@@ -7,7 +7,7 @@
|
|||||||
# # Dev server_url
|
# # Dev server_url
|
||||||
# server_url: https://hs.dev_internal_domain.internal
|
# server_url: https://hs.dev_internal_domain.internal
|
||||||
listen_addr: 0.0.0.0:8080
|
listen_addr: 0.0.0.0:8080
|
||||||
metrics_listen_addr: 0.0.0.0:9090
|
metrics_listen_addr: 127.0.0.1:9090
|
||||||
grpc_listen_addr: 127.0.0.1:50443
|
grpc_listen_addr: 127.0.0.1:50443
|
||||||
grpc_allow_insecure: false
|
grpc_allow_insecure: false
|
||||||
noise:
|
noise:
|
||||||
|
|||||||
@@ -3,8 +3,6 @@ kind: Service
|
|||||||
metadata:
|
metadata:
|
||||||
name: headscale-server-external
|
name: headscale-server-external
|
||||||
namespace: headscale
|
namespace: headscale
|
||||||
labels:
|
|
||||||
app: headscale
|
|
||||||
spec:
|
spec:
|
||||||
ports:
|
ports:
|
||||||
- port: 8080
|
- port: 8080
|
||||||
|
|||||||
@@ -1,18 +0,0 @@
|
|||||||
apiVersion: operator.victoriametrics.com/v1beta1
|
|
||||||
kind: VMServiceScrape
|
|
||||||
metadata:
|
|
||||||
name: headscale
|
|
||||||
namespace: headscale
|
|
||||||
labels:
|
|
||||||
release: prometheus-stack
|
|
||||||
spec:
|
|
||||||
# The external Service has a manually managed EndpointSlice, not Endpoints.
|
|
||||||
discoveryRole: endpointslice
|
|
||||||
selector:
|
|
||||||
matchLabels:
|
|
||||||
app: headscale
|
|
||||||
endpoints:
|
|
||||||
- port: metrics
|
|
||||||
path: /metrics
|
|
||||||
interval: 30s
|
|
||||||
scrapeTimeout: 10s
|
|
||||||
@@ -6,10 +6,6 @@ metadata:
|
|||||||
data:
|
data:
|
||||||
TZ: "Europe/Bratislava"
|
TZ: "Europe/Bratislava"
|
||||||
|
|
||||||
IMMICH_TELEMETRY_INCLUDE: "all"
|
|
||||||
IMMICH_API_METRICS_PORT: "8081"
|
|
||||||
IMMICH_MICROSERVICES_METRICS_PORT: "8082"
|
|
||||||
|
|
||||||
# The database in this namespace, not the shared one in the database
|
# The database in this namespace, not the shared one in the database
|
||||||
# namespace: v3 needs VectorChord, and only the dedicated image carries it.
|
# namespace: v3 needs VectorChord, and only the dedicated image carries it.
|
||||||
DB_HOSTNAME: "immich-postgres"
|
DB_HOSTNAME: "immich-postgres"
|
||||||
|
|||||||
@@ -3,8 +3,6 @@ kind: Service
|
|||||||
metadata:
|
metadata:
|
||||||
name: immich-service
|
name: immich-service
|
||||||
namespace: immich
|
namespace: immich
|
||||||
labels:
|
|
||||||
app: immich
|
|
||||||
spec:
|
spec:
|
||||||
selector:
|
selector:
|
||||||
app: immich
|
app: immich
|
||||||
@@ -12,12 +10,6 @@ spec:
|
|||||||
- name: http
|
- name: http
|
||||||
port: 2283
|
port: 2283
|
||||||
targetPort: 2283
|
targetPort: 2283
|
||||||
- name: api-metrics
|
|
||||||
port: 8081
|
|
||||||
targetPort: api-metrics
|
|
||||||
- name: worker-metrics
|
|
||||||
port: 8082
|
|
||||||
targetPort: worker-metrics
|
|
||||||
---
|
---
|
||||||
apiVersion: apps/v1
|
apiVersion: apps/v1
|
||||||
kind: Deployment
|
kind: Deployment
|
||||||
@@ -49,10 +41,6 @@ spec:
|
|||||||
ports:
|
ports:
|
||||||
- name: http
|
- name: http
|
||||||
containerPort: 2283
|
containerPort: 2283
|
||||||
- name: api-metrics
|
|
||||||
containerPort: 8081
|
|
||||||
- name: worker-metrics
|
|
||||||
containerPort: 8082
|
|
||||||
volumeMounts:
|
volumeMounts:
|
||||||
- name: immich-data
|
- name: immich-data
|
||||||
mountPath: /data
|
mountPath: /data
|
||||||
|
|||||||
@@ -1,20 +0,0 @@
|
|||||||
apiVersion: monitoring.coreos.com/v1
|
|
||||||
kind: ServiceMonitor
|
|
||||||
metadata:
|
|
||||||
name: immich
|
|
||||||
namespace: immich
|
|
||||||
labels:
|
|
||||||
release: prometheus-stack
|
|
||||||
spec:
|
|
||||||
selector:
|
|
||||||
matchLabels:
|
|
||||||
app: immich
|
|
||||||
endpoints:
|
|
||||||
- port: api-metrics
|
|
||||||
path: /metrics
|
|
||||||
interval: 30s
|
|
||||||
scrapeTimeout: 10s
|
|
||||||
- port: worker-metrics
|
|
||||||
path: /metrics
|
|
||||||
interval: 30s
|
|
||||||
scrapeTimeout: 10s
|
|
||||||
@@ -3,8 +3,6 @@ kind: Service
|
|||||||
metadata:
|
metadata:
|
||||||
name: netbird-server-service
|
name: netbird-server-service
|
||||||
namespace: netbird
|
namespace: netbird
|
||||||
labels:
|
|
||||||
app: netbird-server
|
|
||||||
spec:
|
spec:
|
||||||
selector:
|
selector:
|
||||||
app: netbird-server
|
app: netbird-server
|
||||||
@@ -13,10 +11,6 @@ spec:
|
|||||||
name: http
|
name: http
|
||||||
targetPort: 80
|
targetPort: 80
|
||||||
protocol: TCP
|
protocol: TCP
|
||||||
- port: 9090
|
|
||||||
name: metrics
|
|
||||||
targetPort: metrics
|
|
||||||
protocol: TCP
|
|
||||||
- port: 3478
|
- port: 3478
|
||||||
name: stun
|
name: stun
|
||||||
targetPort: 3478
|
targetPort: 3478
|
||||||
@@ -65,9 +59,6 @@ spec:
|
|||||||
- containerPort: 80
|
- containerPort: 80
|
||||||
name: http
|
name: http
|
||||||
protocol: TCP
|
protocol: TCP
|
||||||
- containerPort: 9090
|
|
||||||
name: metrics
|
|
||||||
protocol: TCP
|
|
||||||
- containerPort: 3478
|
- containerPort: 3478
|
||||||
name: stun
|
name: stun
|
||||||
protocol: UDP
|
protocol: UDP
|
||||||
|
|||||||
@@ -1,16 +0,0 @@
|
|||||||
PAPERLESS_URL=https://papers.forust.xyz
|
|
||||||
PAPERLESS_ALLOWED_HOSTS=papers.forust.xyz,papers.workstation.internal
|
|
||||||
PAPERLESS_CSRF_TRUSTED_ORIGINS=https://papers.forust.xyz,https://papers.workstation.internal
|
|
||||||
PAPERLESS_TIME_ZONE=Europe/Bratislava
|
|
||||||
PAPERLESS_REDIS=redis://valkey:6379
|
|
||||||
PAPERLESS_DBENGINE=postgresql
|
|
||||||
PAPERLESS_DBHOST=homelab-postgres
|
|
||||||
PAPERLESS_DBNAME=paperless
|
|
||||||
PAPERLESS_DBUSER=paperless
|
|
||||||
PAPERLESS_DBPASS=<SET_THE_SAME_VALUE_AS_SHARED_POSTGRES_PAPERLESS_DB_PASSWORD>
|
|
||||||
PAPERLESS_OCR_LANGUAGE=rus+eng
|
|
||||||
PAPERLESS_OCR_LANGUAGES=rus
|
|
||||||
PAPERLESS_TASK_WORKERS=1
|
|
||||||
PAPERLESS_ADMIN_USER=admin
|
|
||||||
PAPERLESS_SECRET_KEY=<GENERATE_WITH_python3_-c_import_secrets;_print(secrets.token_urlsafe(64))>
|
|
||||||
PAPERLESS_ADMIN_PASSWORD=<SET_A_LONG_UNIQUE_PASSWORD>
|
|
||||||
@@ -1,79 +0,0 @@
|
|||||||
# Paperless-ngx
|
|
||||||
|
|
||||||
Paperless-ngx runs in the `paperless` namespace. It uses the shared PostgreSQL
|
|
||||||
service in the `database` namespace and Valkey for its task queue. The document
|
|
||||||
library, exports, and consume folder are stored on the `local-path-retain`
|
|
||||||
volume. The PVC size is fixed at 50 GiB because this storage class does not
|
|
||||||
support volume expansion.
|
|
||||||
|
|
||||||
The local route is `https://papers.workstation.internal`; the public route is
|
|
||||||
`https://papers.forust.xyz`. Both use TLS. Paperless keeps its own login and
|
|
||||||
password authentication. OCR is configured for Russian and English documents.
|
|
||||||
Keep `papers.forust.xyz` in the existing Cloudflare DDNS `DOMAINS` setting so
|
|
||||||
the public record follows the workstation address.
|
|
||||||
|
|
||||||
## Compose alternative
|
|
||||||
|
|
||||||
`compose.yaml` is an alternative to the active Kubernetes deployment. Do not
|
|
||||||
run both at the same time: they use the same Paperless database and route
|
|
||||||
names, but have separate document volumes.
|
|
||||||
|
|
||||||
The Compose variant uses the shared Compose PostgreSQL service on the
|
|
||||||
`homelab-database` Docker network. It does not start a PostgreSQL container.
|
|
||||||
The shared Compose database must be running and must have the `paperless`
|
|
||||||
database and role. Set `PAPERLESS_DBPASS` to the same password as
|
|
||||||
`PAPERLESS_DB_PASSWORD` in the shared PostgreSQL configuration.
|
|
||||||
|
|
||||||
To prepare and start the Compose variant:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
cp paperless/.env.example paperless/.env
|
|
||||||
cd paperless
|
|
||||||
docker compose -f compose.yaml config --quiet
|
|
||||||
docker compose -f compose.yaml up -d
|
|
||||||
```
|
|
||||||
|
|
||||||
Create unique values for `PAPERLESS_SECRET_KEY` and
|
|
||||||
`PAPERLESS_ADMIN_PASSWORD` in `.env`. This directory has no Compose `active`
|
|
||||||
marker, so the repository deploy workflow does not start this alternative.
|
|
||||||
Stop the Kubernetes Paperless deployment before switching to Compose. Back up
|
|
||||||
and migrate the media files as well as the database; the Compose named volumes
|
|
||||||
are separate from the Kubernetes PVC.
|
|
||||||
|
|
||||||
## Prepare the secret
|
|
||||||
|
|
||||||
Create `k8s/secrets.yaml` on the workstation from
|
|
||||||
`k8s/secrets.yaml.example`. Set a unique random `PAPERLESS_SECRET_KEY`, a long
|
|
||||||
`PAPERLESS_ADMIN_PASSWORD`, and `PAPERLESS_DB_PASSWORD`.
|
|
||||||
|
|
||||||
Add the same `PAPERLESS_DB_PASSWORD` value to the local
|
|
||||||
`postgres/k8s/secrets.yaml` file. Keep both secret files out of Git. The
|
|
||||||
database bootstrap Job creates the `paperless` role and database from the
|
|
||||||
shared PostgreSQL secret. The job runs in the `database` namespace and needs
|
|
||||||
that namespace's existing `postgres-shared-secrets` Secret.
|
|
||||||
|
|
||||||
For example, generate a key with:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
python3 -c 'import secrets; print(secrets.token_urlsafe(64))'
|
|
||||||
```
|
|
||||||
|
|
||||||
Then apply the secret before enabling the service:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
kubectl apply -f paperless/k8s/namespace.yaml
|
|
||||||
kubectl apply -f postgres/k8s/secrets.yaml
|
|
||||||
kubectl apply -f paperless/k8s/secrets.yaml
|
|
||||||
```
|
|
||||||
|
|
||||||
The normal deploy workflow applies the remaining manifests when
|
|
||||||
`paperless/k8s/active` is present. Verify the rollout and ingress after deploy:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
kubectl -n paperless rollout status deployment/paperless
|
|
||||||
kubectl -n paperless get pods,pvc,services
|
|
||||||
```
|
|
||||||
|
|
||||||
Back up the `paperless-data` PVC and the shared PostgreSQL database. The PVC
|
|
||||||
contains the originals, archived PDFs, and export/consume folders. Valkey has
|
|
||||||
no persistent volume; queued tasks are recreated after a restart.
|
|
||||||
@@ -1,77 +0,0 @@
|
|||||||
services:
|
|
||||||
paperless:
|
|
||||||
image: ghcr.io/paperless-ngx/paperless-ngx:3.2.1
|
|
||||||
container_name: paperless
|
|
||||||
restart: unless-stopped
|
|
||||||
env_file:
|
|
||||||
- .env
|
|
||||||
depends_on:
|
|
||||||
valkey:
|
|
||||||
condition: service_healthy
|
|
||||||
deploy:
|
|
||||||
resources:
|
|
||||||
limits:
|
|
||||||
cpus: "2.0"
|
|
||||||
memory: 2G
|
|
||||||
reservations:
|
|
||||||
cpus: "0.10"
|
|
||||||
memory: 512M
|
|
||||||
volumes:
|
|
||||||
- paperless-data:/usr/src/paperless/data
|
|
||||||
- paperless-media:/usr/src/paperless/media
|
|
||||||
- paperless-export:/usr/src/paperless/export
|
|
||||||
- paperless-consume:/usr/src/paperless/consume
|
|
||||||
networks:
|
|
||||||
- default
|
|
||||||
- proxy
|
|
||||||
- database
|
|
||||||
labels:
|
|
||||||
- "traefik.enable=true"
|
|
||||||
- "traefik.docker.network=proxy"
|
|
||||||
- "traefik.http.services.paperless-compose.loadbalancer.server.port=8000"
|
|
||||||
- "traefik.http.routers.paperless-compose.rule=Host(`papers.forust.xyz`)"
|
|
||||||
- "traefik.http.routers.paperless-compose.entrypoints=websecure"
|
|
||||||
- "traefik.http.routers.paperless-compose.tls.certresolver=letsencrypt"
|
|
||||||
- "traefik.http.routers.paperless-compose-local.rule=Host(`papers.workstation.internal`)"
|
|
||||||
- "traefik.http.routers.paperless-compose-local.entrypoints=websecure"
|
|
||||||
- "traefik.http.routers.paperless-compose-local.tls=true"
|
|
||||||
|
|
||||||
valkey:
|
|
||||||
image: valkey/valkey:9.0.3-alpine
|
|
||||||
container_name: paperless-valkey
|
|
||||||
restart: unless-stopped
|
|
||||||
command:
|
|
||||||
- valkey-server
|
|
||||||
- --save
|
|
||||||
- ""
|
|
||||||
- --appendonly
|
|
||||||
- "no"
|
|
||||||
healthcheck:
|
|
||||||
test: ["CMD", "valkey-cli", "ping"]
|
|
||||||
interval: 10s
|
|
||||||
timeout: 5s
|
|
||||||
retries: 5
|
|
||||||
deploy:
|
|
||||||
resources:
|
|
||||||
limits:
|
|
||||||
cpus: "0.25"
|
|
||||||
memory: 256M
|
|
||||||
reservations:
|
|
||||||
cpus: "0.025"
|
|
||||||
memory: 64M
|
|
||||||
networks:
|
|
||||||
- default
|
|
||||||
|
|
||||||
volumes:
|
|
||||||
paperless-data:
|
|
||||||
paperless-media:
|
|
||||||
paperless-export:
|
|
||||||
paperless-consume:
|
|
||||||
|
|
||||||
networks:
|
|
||||||
default:
|
|
||||||
proxy:
|
|
||||||
external: true
|
|
||||||
database:
|
|
||||||
name: homelab-database
|
|
||||||
external: true
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
|
|
||||||
@@ -1,28 +0,0 @@
|
|||||||
apiVersion: cert-manager.io/v1
|
|
||||||
kind: Certificate
|
|
||||||
metadata:
|
|
||||||
name: paperless-prod-tls
|
|
||||||
namespace: paperless
|
|
||||||
spec:
|
|
||||||
secretName: paperless-prod-tls
|
|
||||||
dnsNames:
|
|
||||||
- papers.forust.xyz
|
|
||||||
issuerRef:
|
|
||||||
name: letsencrypt-prod
|
|
||||||
kind: ClusterIssuer
|
|
||||||
---
|
|
||||||
apiVersion: cert-manager.io/v1
|
|
||||||
kind: Certificate
|
|
||||||
metadata:
|
|
||||||
name: internal-wildcard-tls
|
|
||||||
namespace: paperless
|
|
||||||
spec:
|
|
||||||
secretName: internal-wildcard-tls
|
|
||||||
dnsNames:
|
|
||||||
- "*.workstation.internal"
|
|
||||||
- "*.gigaforust.internal"
|
|
||||||
- workstation.internal
|
|
||||||
- gigaforust.internal
|
|
||||||
issuerRef:
|
|
||||||
name: internal-ca
|
|
||||||
kind: ClusterIssuer
|
|
||||||
@@ -1,58 +0,0 @@
|
|||||||
apiVersion: batch/v1
|
|
||||||
kind: Job
|
|
||||||
metadata:
|
|
||||||
name: paperless-database-init
|
|
||||||
namespace: database
|
|
||||||
spec:
|
|
||||||
backoffLimit: 5
|
|
||||||
template:
|
|
||||||
metadata:
|
|
||||||
labels:
|
|
||||||
app.kubernetes.io/name: paperless-database-init
|
|
||||||
spec:
|
|
||||||
restartPolicy: OnFailure
|
|
||||||
containers:
|
|
||||||
- name: create-database
|
|
||||||
image: postgres:17.11-alpine
|
|
||||||
command:
|
|
||||||
- /bin/sh
|
|
||||||
- -ec
|
|
||||||
- |
|
|
||||||
PGPASSWORD="$POSTGRES_ADMIN_PASSWORD" psql \
|
|
||||||
--host postgres \
|
|
||||||
--username postgres \
|
|
||||||
--dbname postgres \
|
|
||||||
--set ON_ERROR_STOP=1 \
|
|
||||||
--set paperless_password="$PAPERLESS_DB_PASSWORD" <<'SQL'
|
|
||||||
SELECT format(
|
|
||||||
'CREATE ROLE paperless LOGIN PASSWORD %L',
|
|
||||||
:'paperless_password'
|
|
||||||
)
|
|
||||||
WHERE NOT EXISTS (
|
|
||||||
SELECT FROM pg_roles WHERE rolname = 'paperless'
|
|
||||||
)
|
|
||||||
\gexec
|
|
||||||
SELECT format('CREATE DATABASE paperless OWNER paperless')
|
|
||||||
WHERE NOT EXISTS (
|
|
||||||
SELECT FROM pg_database WHERE datname = 'paperless'
|
|
||||||
)
|
|
||||||
\gexec
|
|
||||||
SQL
|
|
||||||
env:
|
|
||||||
- name: POSTGRES_ADMIN_PASSWORD
|
|
||||||
valueFrom:
|
|
||||||
secretKeyRef:
|
|
||||||
name: postgres-shared-secrets
|
|
||||||
key: POSTGRES_ADMIN_PASSWORD
|
|
||||||
- name: PAPERLESS_DB_PASSWORD
|
|
||||||
valueFrom:
|
|
||||||
secretKeyRef:
|
|
||||||
name: postgres-shared-secrets
|
|
||||||
key: PAPERLESS_DB_PASSWORD
|
|
||||||
resources:
|
|
||||||
requests:
|
|
||||||
cpu: 10m
|
|
||||||
memory: 32Mi
|
|
||||||
limits:
|
|
||||||
cpu: 100m
|
|
||||||
memory: 128Mi
|
|
||||||
@@ -1,33 +0,0 @@
|
|||||||
apiVersion: traefik.io/v1alpha1
|
|
||||||
kind: IngressRoute
|
|
||||||
metadata:
|
|
||||||
name: paperless-prod
|
|
||||||
namespace: paperless
|
|
||||||
spec:
|
|
||||||
entryPoints:
|
|
||||||
- websecure
|
|
||||||
routes:
|
|
||||||
- match: Host(`papers.forust.xyz`)
|
|
||||||
kind: Rule
|
|
||||||
services:
|
|
||||||
- name: paperless
|
|
||||||
port: 8000
|
|
||||||
tls:
|
|
||||||
secretName: paperless-prod-tls
|
|
||||||
---
|
|
||||||
apiVersion: traefik.io/v1alpha1
|
|
||||||
kind: IngressRoute
|
|
||||||
metadata:
|
|
||||||
name: paperless-local
|
|
||||||
namespace: paperless
|
|
||||||
spec:
|
|
||||||
entryPoints:
|
|
||||||
- websecure
|
|
||||||
routes:
|
|
||||||
- match: Host(`papers.workstation.internal`)
|
|
||||||
kind: Rule
|
|
||||||
services:
|
|
||||||
- name: paperless
|
|
||||||
port: 8000
|
|
||||||
tls:
|
|
||||||
secretName: internal-wildcard-tls
|
|
||||||
@@ -1,39 +0,0 @@
|
|||||||
apiVersion: networking.k8s.io/v1
|
|
||||||
kind: NetworkPolicy
|
|
||||||
metadata:
|
|
||||||
name: paperless-ingress
|
|
||||||
namespace: paperless
|
|
||||||
spec:
|
|
||||||
podSelector:
|
|
||||||
matchLabels:
|
|
||||||
app.kubernetes.io/name: paperless
|
|
||||||
policyTypes:
|
|
||||||
- Ingress
|
|
||||||
ingress:
|
|
||||||
- from:
|
|
||||||
- namespaceSelector:
|
|
||||||
matchLabels:
|
|
||||||
kubernetes.io/metadata.name: traefik
|
|
||||||
ports:
|
|
||||||
- protocol: TCP
|
|
||||||
port: 8000
|
|
||||||
---
|
|
||||||
apiVersion: networking.k8s.io/v1
|
|
||||||
kind: NetworkPolicy
|
|
||||||
metadata:
|
|
||||||
name: paperless-valkey-ingress
|
|
||||||
namespace: paperless
|
|
||||||
spec:
|
|
||||||
podSelector:
|
|
||||||
matchLabels:
|
|
||||||
app.kubernetes.io/name: paperless-valkey
|
|
||||||
policyTypes:
|
|
||||||
- Ingress
|
|
||||||
ingress:
|
|
||||||
- from:
|
|
||||||
- podSelector:
|
|
||||||
matchLabels:
|
|
||||||
app.kubernetes.io/name: paperless
|
|
||||||
ports:
|
|
||||||
- protocol: TCP
|
|
||||||
port: 6379
|
|
||||||
@@ -1,210 +0,0 @@
|
|||||||
apiVersion: v1
|
|
||||||
kind: Service
|
|
||||||
metadata:
|
|
||||||
name: paperless
|
|
||||||
namespace: paperless
|
|
||||||
labels:
|
|
||||||
app.kubernetes.io/name: paperless
|
|
||||||
spec:
|
|
||||||
selector:
|
|
||||||
app.kubernetes.io/name: paperless
|
|
||||||
ports:
|
|
||||||
- name: http
|
|
||||||
port: 8000
|
|
||||||
targetPort: http
|
|
||||||
---
|
|
||||||
apiVersion: apps/v1
|
|
||||||
kind: Deployment
|
|
||||||
metadata:
|
|
||||||
name: paperless
|
|
||||||
namespace: paperless
|
|
||||||
labels:
|
|
||||||
app.kubernetes.io/name: paperless
|
|
||||||
spec:
|
|
||||||
replicas: 1
|
|
||||||
strategy:
|
|
||||||
type: Recreate
|
|
||||||
selector:
|
|
||||||
matchLabels:
|
|
||||||
app.kubernetes.io/name: paperless
|
|
||||||
template:
|
|
||||||
metadata:
|
|
||||||
labels:
|
|
||||||
app.kubernetes.io/name: paperless
|
|
||||||
spec:
|
|
||||||
enableServiceLinks: false
|
|
||||||
containers:
|
|
||||||
- name: paperless
|
|
||||||
image: ghcr.io/paperless-ngx/paperless-ngx:3.2.1
|
|
||||||
ports:
|
|
||||||
- name: http
|
|
||||||
containerPort: 8000
|
|
||||||
env:
|
|
||||||
- name: PAPERLESS_URL
|
|
||||||
value: https://papers.forust.xyz
|
|
||||||
- name: PAPERLESS_ALLOWED_HOSTS
|
|
||||||
value: papers.forust.xyz,papers.workstation.internal
|
|
||||||
- name: PAPERLESS_CSRF_TRUSTED_ORIGINS
|
|
||||||
value: https://papers.forust.xyz,https://papers.workstation.internal
|
|
||||||
- name: PAPERLESS_TIME_ZONE
|
|
||||||
value: Europe/Bratislava
|
|
||||||
- name: PAPERLESS_REDIS
|
|
||||||
value: redis://paperless-valkey:6379
|
|
||||||
- name: PAPERLESS_DBENGINE
|
|
||||||
value: postgresql
|
|
||||||
- name: PAPERLESS_DBHOST
|
|
||||||
value: postgres.database.svc.cluster.local
|
|
||||||
- name: PAPERLESS_DBNAME
|
|
||||||
value: paperless
|
|
||||||
- name: PAPERLESS_DBUSER
|
|
||||||
value: paperless
|
|
||||||
- name: PAPERLESS_DBPASS
|
|
||||||
valueFrom:
|
|
||||||
secretKeyRef:
|
|
||||||
name: paperless-secrets
|
|
||||||
key: PAPERLESS_DB_PASSWORD
|
|
||||||
- name: PAPERLESS_OCR_LANGUAGE
|
|
||||||
value: rus+eng
|
|
||||||
- name: PAPERLESS_OCR_LANGUAGES
|
|
||||||
value: rus
|
|
||||||
- name: PAPERLESS_TASK_WORKERS
|
|
||||||
value: "1"
|
|
||||||
- name: PAPERLESS_ADMIN_USER
|
|
||||||
value: admin
|
|
||||||
- name: PAPERLESS_SECRET_KEY
|
|
||||||
valueFrom:
|
|
||||||
secretKeyRef:
|
|
||||||
name: paperless-secrets
|
|
||||||
key: PAPERLESS_SECRET_KEY
|
|
||||||
- name: PAPERLESS_ADMIN_PASSWORD
|
|
||||||
valueFrom:
|
|
||||||
secretKeyRef:
|
|
||||||
name: paperless-secrets
|
|
||||||
key: PAPERLESS_ADMIN_PASSWORD
|
|
||||||
volumeMounts:
|
|
||||||
- name: documents
|
|
||||||
mountPath: /usr/src/paperless/data
|
|
||||||
subPath: data
|
|
||||||
- name: documents
|
|
||||||
mountPath: /usr/src/paperless/media
|
|
||||||
subPath: media
|
|
||||||
- name: documents
|
|
||||||
mountPath: /usr/src/paperless/export
|
|
||||||
subPath: export
|
|
||||||
- name: documents
|
|
||||||
mountPath: /usr/src/paperless/consume
|
|
||||||
subPath: consume
|
|
||||||
startupProbe:
|
|
||||||
httpGet:
|
|
||||||
path: /
|
|
||||||
port: http
|
|
||||||
httpHeaders:
|
|
||||||
- name: Host
|
|
||||||
value: papers.workstation.internal
|
|
||||||
failureThreshold: 60
|
|
||||||
periodSeconds: 10
|
|
||||||
timeoutSeconds: 5
|
|
||||||
readinessProbe:
|
|
||||||
httpGet:
|
|
||||||
path: /
|
|
||||||
port: http
|
|
||||||
httpHeaders:
|
|
||||||
- name: Host
|
|
||||||
value: papers.workstation.internal
|
|
||||||
periodSeconds: 10
|
|
||||||
timeoutSeconds: 5
|
|
||||||
livenessProbe:
|
|
||||||
httpGet:
|
|
||||||
path: /
|
|
||||||
port: http
|
|
||||||
httpHeaders:
|
|
||||||
- name: Host
|
|
||||||
value: papers.workstation.internal
|
|
||||||
periodSeconds: 30
|
|
||||||
timeoutSeconds: 5
|
|
||||||
resources:
|
|
||||||
requests:
|
|
||||||
cpu: 100m
|
|
||||||
memory: 512Mi
|
|
||||||
limits:
|
|
||||||
cpu: "2"
|
|
||||||
memory: 2Gi
|
|
||||||
volumes:
|
|
||||||
- name: documents
|
|
||||||
persistentVolumeClaim:
|
|
||||||
claimName: paperless-data
|
|
||||||
---
|
|
||||||
apiVersion: v1
|
|
||||||
kind: PersistentVolumeClaim
|
|
||||||
metadata:
|
|
||||||
name: paperless-data
|
|
||||||
namespace: paperless
|
|
||||||
spec:
|
|
||||||
accessModes:
|
|
||||||
- ReadWriteOnce
|
|
||||||
storageClassName: local-path-retain
|
|
||||||
resources:
|
|
||||||
requests:
|
|
||||||
storage: 50Gi
|
|
||||||
---
|
|
||||||
apiVersion: v1
|
|
||||||
kind: Service
|
|
||||||
metadata:
|
|
||||||
name: paperless-valkey
|
|
||||||
namespace: paperless
|
|
||||||
labels:
|
|
||||||
app.kubernetes.io/name: paperless-valkey
|
|
||||||
spec:
|
|
||||||
selector:
|
|
||||||
app.kubernetes.io/name: paperless-valkey
|
|
||||||
ports:
|
|
||||||
- name: redis
|
|
||||||
port: 6379
|
|
||||||
targetPort: redis
|
|
||||||
---
|
|
||||||
apiVersion: apps/v1
|
|
||||||
kind: Deployment
|
|
||||||
metadata:
|
|
||||||
name: paperless-valkey
|
|
||||||
namespace: paperless
|
|
||||||
labels:
|
|
||||||
app.kubernetes.io/name: paperless-valkey
|
|
||||||
spec:
|
|
||||||
replicas: 1
|
|
||||||
strategy:
|
|
||||||
type: Recreate
|
|
||||||
selector:
|
|
||||||
matchLabels:
|
|
||||||
app.kubernetes.io/name: paperless-valkey
|
|
||||||
template:
|
|
||||||
metadata:
|
|
||||||
labels:
|
|
||||||
app.kubernetes.io/name: paperless-valkey
|
|
||||||
spec:
|
|
||||||
containers:
|
|
||||||
- name: valkey
|
|
||||||
image: valkey/valkey:9.0.3-alpine
|
|
||||||
args:
|
|
||||||
- valkey-server
|
|
||||||
- --save
|
|
||||||
- ""
|
|
||||||
- --appendonly
|
|
||||||
- "no"
|
|
||||||
ports:
|
|
||||||
- name: redis
|
|
||||||
containerPort: 6379
|
|
||||||
readinessProbe:
|
|
||||||
exec:
|
|
||||||
command: ["valkey-cli", "ping"]
|
|
||||||
periodSeconds: 10
|
|
||||||
livenessProbe:
|
|
||||||
exec:
|
|
||||||
command: ["valkey-cli", "ping"]
|
|
||||||
periodSeconds: 30
|
|
||||||
resources:
|
|
||||||
requests:
|
|
||||||
cpu: 25m
|
|
||||||
memory: 64Mi
|
|
||||||
limits:
|
|
||||||
cpu: 250m
|
|
||||||
memory: 256Mi
|
|
||||||
@@ -1,10 +0,0 @@
|
|||||||
apiVersion: v1
|
|
||||||
kind: Secret
|
|
||||||
metadata:
|
|
||||||
name: paperless-secrets
|
|
||||||
namespace: paperless
|
|
||||||
type: Opaque
|
|
||||||
stringData:
|
|
||||||
PAPERLESS_SECRET_KEY: "<GENERATE_WITH_python3_-c_import_secrets;_print(secrets.token_urlsafe(64))>"
|
|
||||||
PAPERLESS_ADMIN_PASSWORD: "<SET_A_LONG_UNIQUE_PASSWORD>"
|
|
||||||
PAPERLESS_DB_PASSWORD: "<SET_THE_SAME_VALUE_AS_database_PAPERLESS_DB_PASSWORD>"
|
|
||||||
@@ -4,5 +4,4 @@ GITEA_DB_PASSWORD=
|
|||||||
NETBOX_DB_PASSWORD=
|
NETBOX_DB_PASSWORD=
|
||||||
NETRONOME_DB_PASSWORD=
|
NETRONOME_DB_PASSWORD=
|
||||||
PENPOT_DB_PASSWORD=
|
PENPOT_DB_PASSWORD=
|
||||||
PAPERLESS_DB_PASSWORD=
|
|
||||||
STATUSPAGE_DB_PASSWORD=
|
STATUSPAGE_DB_PASSWORD=
|
||||||
+5
-6
@@ -1,20 +1,19 @@
|
|||||||
# Shared PostgreSQL
|
# Shared PostgreSQL
|
||||||
|
|
||||||
This directory contains the shared PostgreSQL 17 deployment for Authentik,
|
This directory contains the shared PostgreSQL 17 deployment for Authentik,
|
||||||
Gitea, NetBox, Netronome, Paperless-ngx, and Statuspage. It creates one database
|
Gitea, NetBox, Netronome, and Statuspage. It creates one database and one login role
|
||||||
and one login role per service. Per-service standalone databases were removed
|
per service. Per-service standalone databases were removed after the
|
||||||
after the migration (Sep 2026); Penpot stays on its own compose PostgreSQL
|
migration (Sep 2026); Penpot stays on its own compose PostgreSQL (archived,
|
||||||
(archived, not part of the shared instance).
|
not part of the shared instance).
|
||||||
|
|
||||||
## Compatibility baseline
|
## Compatibility baseline
|
||||||
|
|
||||||
| Service | Current application | Shared PostgreSQL 17 |
|
| Service | Current application | Shared PostgreSQL 17 |
|
||||||
| ------------- | ------------------- | -------------------------------------- |
|
| ---------- | ------------------- | -------------------------------------- |
|
||||||
| Authentik | 2025.10.x | Supported (Authentik requires 14+) |
|
| Authentik | 2025.10.x | Supported (Authentik requires 14+) |
|
||||||
| Gitea | 1.27.3 | Supported (Gitea requires 12+) |
|
| Gitea | 1.27.3 | Supported (Gitea requires 12+) |
|
||||||
| NetBox | 4.7.x | Supported (NetBox 4.x requires 13+) |
|
| NetBox | 4.7.x | Supported (NetBox 4.x requires 13+) |
|
||||||
| Netronome | 0.14.0 | Supported (upstream's example uses 17) |
|
| Netronome | 0.14.0 | Supported (upstream's example uses 17) |
|
||||||
| Paperless-ngx | 3.2.1 | Supported |
|
|
||||||
| Statuspage | custom | Supported |
|
| Statuspage | custom | Supported |
|
||||||
|
|
||||||
A major-version change must use a logical dump/restore; changing only the
|
A major-version change must use a logical dump/restore; changing only the
|
||||||
|
|||||||
@@ -6,7 +6,6 @@ set -euo pipefail
|
|||||||
: "${NETBOX_DB_PASSWORD:?NETBOX_DB_PASSWORD is required}"
|
: "${NETBOX_DB_PASSWORD:?NETBOX_DB_PASSWORD is required}"
|
||||||
: "${NETRONOME_DB_PASSWORD:?NETRONOME_DB_PASSWORD is required}"
|
: "${NETRONOME_DB_PASSWORD:?NETRONOME_DB_PASSWORD is required}"
|
||||||
: "${PENPOT_DB_PASSWORD:?PENPOT_DB_PASSWORD is required}"
|
: "${PENPOT_DB_PASSWORD:?PENPOT_DB_PASSWORD is required}"
|
||||||
: "${PAPERLESS_DB_PASSWORD:?PAPERLESS_DB_PASSWORD is required}"
|
|
||||||
: "${STATUSPAGE_DB_PASSWORD:?STATUSPAGE_DB_PASSWORD is required}"
|
: "${STATUSPAGE_DB_PASSWORD:?STATUSPAGE_DB_PASSWORD is required}"
|
||||||
|
|
||||||
create_role_and_database() {
|
create_role_and_database() {
|
||||||
@@ -28,5 +27,4 @@ create_role_and_database gitea gitea "$GITEA_DB_PASSWORD"
|
|||||||
create_role_and_database netbox netbox "$NETBOX_DB_PASSWORD"
|
create_role_and_database netbox netbox "$NETBOX_DB_PASSWORD"
|
||||||
create_role_and_database netronome netronome "$NETRONOME_DB_PASSWORD"
|
create_role_and_database netronome netronome "$NETRONOME_DB_PASSWORD"
|
||||||
create_role_and_database penpot penpot "$PENPOT_DB_PASSWORD"
|
create_role_and_database penpot penpot "$PENPOT_DB_PASSWORD"
|
||||||
create_role_and_database paperless paperless "$PAPERLESS_DB_PASSWORD"
|
|
||||||
create_role_and_database statuspage statuspage "$STATUSPAGE_DB_PASSWORD"
|
create_role_and_database statuspage statuspage "$STATUSPAGE_DB_PASSWORD"
|
||||||
@@ -26,18 +26,6 @@ spec:
|
|||||||
- namespaceSelector:
|
- namespaceSelector:
|
||||||
matchLabels:
|
matchLabels:
|
||||||
kubernetes.io/metadata.name: penpot
|
kubernetes.io/metadata.name: penpot
|
||||||
- namespaceSelector:
|
|
||||||
matchLabels:
|
|
||||||
kubernetes.io/metadata.name: paperless
|
|
||||||
podSelector:
|
|
||||||
matchLabels:
|
|
||||||
app.kubernetes.io/name: paperless
|
|
||||||
- namespaceSelector:
|
|
||||||
matchLabels:
|
|
||||||
kubernetes.io/metadata.name: database
|
|
||||||
podSelector:
|
|
||||||
matchLabels:
|
|
||||||
app.kubernetes.io/name: paperless-database-init
|
|
||||||
- namespaceSelector:
|
- namespaceSelector:
|
||||||
matchLabels:
|
matchLabels:
|
||||||
kubernetes.io/metadata.name: statuspage
|
kubernetes.io/metadata.name: statuspage
|
||||||
|
|||||||
@@ -126,7 +126,6 @@ data:
|
|||||||
: "${NETBOX_DB_PASSWORD:?NETBOX_DB_PASSWORD is required}"
|
: "${NETBOX_DB_PASSWORD:?NETBOX_DB_PASSWORD is required}"
|
||||||
: "${NETRONOME_DB_PASSWORD:?NETRONOME_DB_PASSWORD is required}"
|
: "${NETRONOME_DB_PASSWORD:?NETRONOME_DB_PASSWORD is required}"
|
||||||
: "${PENPOT_DB_PASSWORD:?PENPOT_DB_PASSWORD is required}"
|
: "${PENPOT_DB_PASSWORD:?PENPOT_DB_PASSWORD is required}"
|
||||||
: "${PAPERLESS_DB_PASSWORD:?PAPERLESS_DB_PASSWORD is required}"
|
|
||||||
: "${STATUSPAGE_DB_PASSWORD:?STATUSPAGE_DB_PASSWORD is required}"
|
: "${STATUSPAGE_DB_PASSWORD:?STATUSPAGE_DB_PASSWORD is required}"
|
||||||
|
|
||||||
create_role_and_database() {
|
create_role_and_database() {
|
||||||
@@ -148,5 +147,4 @@ data:
|
|||||||
create_role_and_database netbox netbox "$NETBOX_DB_PASSWORD"
|
create_role_and_database netbox netbox "$NETBOX_DB_PASSWORD"
|
||||||
create_role_and_database netronome netronome "$NETRONOME_DB_PASSWORD"
|
create_role_and_database netronome netronome "$NETRONOME_DB_PASSWORD"
|
||||||
create_role_and_database penpot penpot "$PENPOT_DB_PASSWORD"
|
create_role_and_database penpot penpot "$PENPOT_DB_PASSWORD"
|
||||||
create_role_and_database paperless paperless "$PAPERLESS_DB_PASSWORD"
|
|
||||||
create_role_and_database statuspage statuspage "$STATUSPAGE_DB_PASSWORD"
|
create_role_and_database statuspage statuspage "$STATUSPAGE_DB_PASSWORD"
|
||||||
@@ -11,5 +11,4 @@ stringData:
|
|||||||
NETBOX_DB_PASSWORD: ""
|
NETBOX_DB_PASSWORD: ""
|
||||||
NETRONOME_DB_PASSWORD: ""
|
NETRONOME_DB_PASSWORD: ""
|
||||||
PENPOT_DB_PASSWORD: ""
|
PENPOT_DB_PASSWORD: ""
|
||||||
PAPERLESS_DB_PASSWORD: ""
|
|
||||||
STATUSPAGE_DB_PASSWORD: ""
|
STATUSPAGE_DB_PASSWORD: ""
|
||||||
@@ -15,29 +15,3 @@ The VictoriaMetrics Operator chart and its CRDs are installed before the
|
|||||||
Kubernetes manifests by the normal deploy workflow. On a cluster where the
|
Kubernetes manifests by the normal deploy workflow. On a cluster where the
|
||||||
operator CRDs are not installed yet, CI skips the server-side dry-run of the
|
operator CRDs are not installed yet, CI skips the server-side dry-run of the
|
||||||
`VMAgent` resource; the deploy installs the chart before applying that resource.
|
`VMAgent` resource; the deploy installs the chart before applying that resource.
|
||||||
|
|
||||||
## Application metrics
|
|
||||||
|
|
||||||
The application ServiceMonitors use a 30s interval and a 10s timeout:
|
|
||||||
|
|
||||||
- Headscale: the external Service points to the Compose host on port 19090.
|
|
||||||
A VMServiceScrape uses EndpointSlice discovery for this manually managed target.
|
|
||||||
The Compose configuration must bind metrics to `0.0.0.0:9090`.
|
|
||||||
- NetBird: the combined server exports `/metrics` on port 9090. The existing
|
|
||||||
`server.metricsPort` setting enables the listener.
|
|
||||||
- Gitea: `GITEA__metrics__ENABLED` enables `/metrics` on the HTTP port. The public
|
|
||||||
ingress excludes this path. The monitor uses the internal Service directly.
|
|
||||||
- Immich: `IMMICH_TELEMETRY_INCLUDE=all` enables API and worker metrics on ports
|
|
||||||
8081 and 8082. The monitor scrapes both ports on each server replica.
|
|
||||||
|
|
||||||
Deploy through the existing CI and deploy workflow. Gitea and Immich reload their
|
|
||||||
ConfigMap changes through Reloader. Check the VMAgent targets after deployment
|
|
||||||
and query `up{scraper="victoria",namespace=~"netbird|gitea|immich|headscale"}` in
|
|
||||||
VictoriaMetrics. All targets should report 1.
|
|
||||||
|
|
||||||
For rollback, revert the application metrics changes, run CI, and deploy the
|
|
||||||
revert. Remove the three application ServiceMonitors and the Headscale VMServiceScrape explicitly: the deployment
|
|
||||||
workflow applies manifests and does not prune removed resources.
|
|
||||||
|
|
||||||
For Headscale rollback, remove its VMServiceScrape and Service label, restore the
|
|
||||||
previous Compose metrics bind address, and restart only the Headscale service.
|
|
||||||
@@ -29,6 +29,31 @@ data:
|
|||||||
"managerFilePatterns": ["/k8s/.+\\.ya?ml$/"]
|
"managerFilePatterns": ["/k8s/.+\\.ya?ml$/"]
|
||||||
},
|
},
|
||||||
"customManagers": [
|
"customManagers": [
|
||||||
|
{
|
||||||
|
"customType": "regex",
|
||||||
|
"description": "singlesource: playwright npm version pinned in npx command (k8s + compose)",
|
||||||
|
"managerFilePatterns": ["edu_master/k8s/playwright.yaml", "edu_master/compose.yaml"],
|
||||||
|
"matchStrings": ["playwright@(?<currentValue>\\d+\\.\\d+\\.\\d+)"],
|
||||||
|
"datasourceTemplate": "npm",
|
||||||
|
"depNameTemplate": "playwright"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"customType": "regex",
|
||||||
|
"description": "singlesource: PLAYWRIGHT_VERSION file",
|
||||||
|
"managerFilePatterns": ["edu_master/PLAYWRIGHT_VERSION"],
|
||||||
|
"matchStrings": ["^(?<currentValue>\\d+\\.\\d+\\.\\d+)(?:\\r?\\n)?$"],
|
||||||
|
"datasourceTemplate": "pypi",
|
||||||
|
"depNameTemplate": "playwright"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"customType": "regex",
|
||||||
|
"description": "singlesource: playwright Python client version pinned in Dockerfile ARG",
|
||||||
|
"managerFilePatterns": ["edu_master/webinar-checker/Dockerfile"],
|
||||||
|
"matchStrings": ["(?:^|\\n)ARG PLAYWRIGHT_VERSION=(?<currentValue>\\d+\\.\\d+\\.\\d+)(?:\\r?\\n|$)"],
|
||||||
|
"datasourceTemplate": "pypi",
|
||||||
|
"depNameTemplate": "playwright",
|
||||||
|
"versioningTemplate": "pep440"
|
||||||
|
},
|
||||||
{
|
{
|
||||||
"customType": "regex",
|
"customType": "regex",
|
||||||
"description": "kube-prometheus-stack chart version pinned in the deploy workflow",
|
"description": "kube-prometheus-stack chart version pinned in the deploy workflow",
|
||||||
@@ -187,6 +212,17 @@ data:
|
|||||||
"matchPackageNames": ["/gcr\\.forust\\.xyz\\/forust\\/.+/"],
|
"matchPackageNames": ["/gcr\\.forust\\.xyz\\/forust\\/.+/"],
|
||||||
"enabled": false
|
"enabled": false
|
||||||
},
|
},
|
||||||
|
{
|
||||||
|
"description": "singlesource playwright - use whichever version is found, keep docker+pypi+npm in sync",
|
||||||
|
"matchPackageNames": ["playwright", "mcr.microsoft.com/playwright"],
|
||||||
|
"groupName": "playwright singlesource",
|
||||||
|
"groupSlug": "playwright"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"description": "playwright must not automerge - version skew breaks the WS handshake (checker.py:1523 vs playwright.yaml:20)",
|
||||||
|
"matchPackageNames": ["playwright", "mcr.microsoft.com/playwright"],
|
||||||
|
"automerge": false
|
||||||
|
},
|
||||||
{
|
{
|
||||||
"description": "Renovate updates itself in lockstep across the CronJob and the Compose file",
|
"description": "Renovate updates itself in lockstep across the CronJob and the Compose file",
|
||||||
"matchPackageNames": ["renovate/renovate"],
|
"matchPackageNames": ["renovate/renovate"],
|
||||||
|
|||||||
@@ -18,6 +18,31 @@
|
|||||||
"managerFilePatterns": ["/k8s/.+\\.ya?ml$/"]
|
"managerFilePatterns": ["/k8s/.+\\.ya?ml$/"]
|
||||||
},
|
},
|
||||||
"customManagers": [
|
"customManagers": [
|
||||||
|
{
|
||||||
|
"customType": "regex",
|
||||||
|
"description": "singlesource: playwright npm version pinned in npx command (k8s + compose)",
|
||||||
|
"managerFilePatterns": ["edu_master/k8s/playwright.yaml", "edu_master/compose.yaml"],
|
||||||
|
"matchStrings": ["playwright@(?<currentValue>\\d+\\.\\d+\\.\\d+)"],
|
||||||
|
"datasourceTemplate": "npm",
|
||||||
|
"depNameTemplate": "playwright"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"customType": "regex",
|
||||||
|
"description": "singlesource: PLAYWRIGHT_VERSION file",
|
||||||
|
"managerFilePatterns": ["edu_master/PLAYWRIGHT_VERSION"],
|
||||||
|
"matchStrings": ["^(?<currentValue>\\d+\\.\\d+\\.\\d+)(?:\\r?\\n)?$"],
|
||||||
|
"datasourceTemplate": "pypi",
|
||||||
|
"depNameTemplate": "playwright"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"customType": "regex",
|
||||||
|
"description": "singlesource: playwright Python client version pinned in Dockerfile ARG",
|
||||||
|
"managerFilePatterns": ["edu_master/webinar-checker/Dockerfile"],
|
||||||
|
"matchStrings": ["(?:^|\\n)ARG PLAYWRIGHT_VERSION=(?<currentValue>\\d+\\.\\d+\\.\\d+)(?:\\r?\\n|$)"],
|
||||||
|
"datasourceTemplate": "pypi",
|
||||||
|
"depNameTemplate": "playwright",
|
||||||
|
"versioningTemplate": "pep440"
|
||||||
|
},
|
||||||
{
|
{
|
||||||
"customType": "regex",
|
"customType": "regex",
|
||||||
"description": "kube-prometheus-stack chart version pinned in the deploy workflow",
|
"description": "kube-prometheus-stack chart version pinned in the deploy workflow",
|
||||||
@@ -176,6 +201,17 @@
|
|||||||
"matchPackageNames": ["/gcr\\.forust\\.xyz\\/forust\\/.+/"],
|
"matchPackageNames": ["/gcr\\.forust\\.xyz\\/forust\\/.+/"],
|
||||||
"enabled": false
|
"enabled": false
|
||||||
},
|
},
|
||||||
|
{
|
||||||
|
"description": "singlesource playwright - use whichever version is found, keep docker+pypi+npm in sync",
|
||||||
|
"matchPackageNames": ["playwright", "mcr.microsoft.com/playwright"],
|
||||||
|
"groupName": "playwright singlesource",
|
||||||
|
"groupSlug": "playwright"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"description": "playwright must not automerge - version skew breaks the WS handshake (checker.py:1523 vs playwright.yaml:20)",
|
||||||
|
"matchPackageNames": ["playwright", "mcr.microsoft.com/playwright"],
|
||||||
|
"automerge": false
|
||||||
|
},
|
||||||
{
|
{
|
||||||
"description": "Renovate updates itself in lockstep across the CronJob and the Compose file",
|
"description": "Renovate updates itself in lockstep across the CronJob and the Compose file",
|
||||||
"matchPackageNames": ["renovate/renovate"],
|
"matchPackageNames": ["renovate/renovate"],
|
||||||
|
|||||||
Whitespace-only changes.
Whitespace-only changes.
@@ -40,6 +40,19 @@ class ArtifactTests(unittest.TestCase):
|
|||||||
release_module.render(io.StringIO(f' image: "{image}:prod" # note\n'), result)
|
release_module.render(io.StringIO(f' image: "{image}:prod" # note\n'), result)
|
||||||
self.assertEqual(result.getvalue(), f' image: "{image}@sha256:{"b" * 64}" # note\n')
|
self.assertEqual(result.getvalue(), f' image: "{image}@sha256:{"b" * 64}" # note\n')
|
||||||
|
|
||||||
|
def test_edu_release_digest_is_preserved(self):
|
||||||
|
with tempfile.TemporaryDirectory() as scratch:
|
||||||
|
path = Path(scratch) / 'release.json'
|
||||||
|
path.write_text(json.dumps(release()))
|
||||||
|
image = 'gcr.forust.xyz/forust/session-keeper'
|
||||||
|
line = f'image: {image}@sha256:{"e" * 64}\n'
|
||||||
|
result = io.StringIO()
|
||||||
|
with patch.dict(os.environ, {'RELEASE_FILE': str(path), 'DEPLOY_SHA': 'a' * 40}):
|
||||||
|
release_module.render(io.StringIO(line), result)
|
||||||
|
self.assertEqual(result.getvalue(), line)
|
||||||
|
with self.assertRaises(ValueError):
|
||||||
|
release_module.render(io.StringIO(f'image: {image}:prod\n'), io.StringIO())
|
||||||
|
|
||||||
def test_unknown_image_cannot_emit_partial_manifest(self):
|
def test_unknown_image_cannot_emit_partial_manifest(self):
|
||||||
with tempfile.TemporaryDirectory() as scratch:
|
with tempfile.TemporaryDirectory() as scratch:
|
||||||
path = Path(scratch) / 'release.json'
|
path = Path(scratch) / 'release.json'
|
||||||
@@ -94,13 +107,10 @@ class ArtifactTests(unittest.TestCase):
|
|||||||
patch.object(release_module, 'command', side_effect=fake_command),
|
patch.object(release_module, 'command', side_effect=fake_command),
|
||||||
patch.object(subprocess, 'run', return_value=subprocess.CompletedProcess([], 0)),
|
patch.object(subprocess, 'run', return_value=subprocess.CompletedProcess([], 0)),
|
||||||
):
|
):
|
||||||
plan = root / 'plan.json'
|
release_module.build(root / 'release.json')
|
||||||
release_module.prepare_images(plan)
|
|
||||||
for name in release_module.IMAGES:
|
|
||||||
release_module.build(root / f'{name}.json', name, plan)
|
|
||||||
self.assertEqual(built, ['errorpages/Dockerfile'])
|
self.assertEqual(built, ['errorpages/Dockerfile'])
|
||||||
self.assertTrue(all(not directory.exists() for directory in auth_directories))
|
self.assertTrue(all(not directory.exists() for directory in auth_directories))
|
||||||
self.assertEqual(json.loads((root / 'error-pages.json').read_text())['sha'], 'e' * 40)
|
self.assertEqual(json.loads((root / 'release.json').read_text())['sha'], 'e' * 40)
|
||||||
|
|
||||||
|
|
||||||
class DurableRunTests(unittest.TestCase):
|
class DurableRunTests(unittest.TestCase):
|
||||||
@@ -203,64 +213,6 @@ class DurableRunTests(unittest.TestCase):
|
|||||||
self.assertEqual(json.loads((directory / 'status.json').read_text())['state'], 'failure')
|
self.assertEqual(json.loads((directory / 'status.json').read_text())['state'], 'failure')
|
||||||
|
|
||||||
|
|
||||||
class FailureSummaryTests(unittest.TestCase):
|
|
||||||
def test_build_failure_keeps_progress_and_does_not_expose_exception_text(self):
|
|
||||||
with tempfile.TemporaryDirectory() as scratch:
|
|
||||||
summary = Path(scratch) / 'summary.md'
|
|
||||||
|
|
||||||
def failed_build(_output, report, _name, _plan):
|
|
||||||
report.update(phase='Build or reuse xdfnx-homepage', current='xdfnx-homepage', built=['error-pages'])
|
|
||||||
report['images']['gcr.forust.xyz/forust/error-pages'] = 'sha256:' + 'b' * 64
|
|
||||||
raise RuntimeError('private value must not appear in the summary')
|
|
||||||
|
|
||||||
with (
|
|
||||||
patch.dict(os.environ, {'GITHUB_STEP_SUMMARY': str(summary), 'GITHUB_SHA': 'a' * 40}),
|
|
||||||
patch.object(release_module, 'build_images', side_effect=failed_build),
|
|
||||||
self.assertRaises(RuntimeError),
|
|
||||||
):
|
|
||||||
release_module.build(Path(scratch) / 'release.json', 'xdfnx-homepage', Path('plan.json'))
|
|
||||||
content = summary.read_text()
|
|
||||||
self.assertIn('**failure**', content)
|
|
||||||
self.assertIn('error-pages', content)
|
|
||||||
self.assertIn('xdfnx-homepage', content)
|
|
||||||
self.assertNotIn('private value', content)
|
|
||||||
|
|
||||||
def test_deploy_failure_reports_completed_apply_and_rollback_result(self):
|
|
||||||
with tempfile.TemporaryDirectory() as scratch:
|
|
||||||
state = Path(scratch)
|
|
||||||
directory = state / 'runs/123-1'
|
|
||||||
snapshot = directory / 'snapshot/before'
|
|
||||||
snapshot.mkdir(parents=True)
|
|
||||||
(directory / 'snapshot/current').write_text(str(snapshot))
|
|
||||||
(snapshot / 'failed-workloads').write_text('deployment app api\nROLLED_BACK=1\nUNRECOVERED=0\n')
|
|
||||||
controller.atomic_json(
|
|
||||||
directory / 'request.json', {'release': release(), 'mode': 'changed', 'refresh_images': False}
|
|
||||||
)
|
|
||||||
controller.atomic_json(
|
|
||||||
directory / 'status.json',
|
|
||||||
{
|
|
||||||
'state': 'failure',
|
|
||||||
'stages': {
|
|
||||||
'apply-k8s': {'result': 'success', 'exit_code': 0},
|
|
||||||
'verify-k8s': {'result': 'failure', 'exit_code': 1},
|
|
||||||
},
|
|
||||||
},
|
|
||||||
)
|
|
||||||
controller.atomic_json(directory / 'plan.json', {'selected': {'k8s': ['app'], 'compose': []}})
|
|
||||||
(directory / 'apply-events.jsonl').write_text(
|
|
||||||
json.dumps({'action': 'kubectl', 'target': 'app/k8s/api.yaml', 'result': 'success'}) + '\n'
|
|
||||||
)
|
|
||||||
output = io.StringIO()
|
|
||||||
with patch.object(controller, 'STATE', state), patch('sys.stdout', output):
|
|
||||||
controller.summary('123-1')
|
|
||||||
content = output.getvalue()
|
|
||||||
self.assertIn('verify-k8s | failure | 1', content)
|
|
||||||
self.assertIn('app/k8s/api.yaml', content)
|
|
||||||
self.assertIn('Workloads restored: **1**', content)
|
|
||||||
self.assertIn('manual recovery: **0**', content)
|
|
||||||
self.assertIn('Compose requires manual recovery', content)
|
|
||||||
|
|
||||||
|
|
||||||
class InstallerTests(unittest.TestCase):
|
class InstallerTests(unittest.TestCase):
|
||||||
def test_version_comparison_is_exact_without_network_or_host_packages(self):
|
def test_version_comparison_is_exact_without_network_or_host_packages(self):
|
||||||
with tempfile.TemporaryDirectory() as scratch:
|
with tempfile.TemporaryDirectory() as scratch:
|
||||||
|
|||||||
@@ -1,144 +0,0 @@
|
|||||||
"""Matrix release contracts and failure gates without a registry."""
|
|
||||||
|
|
||||||
import json
|
|
||||||
import os
|
|
||||||
import tempfile
|
|
||||||
import unittest
|
|
||||||
from pathlib import Path
|
|
||||||
from unittest.mock import Mock, call, patch
|
|
||||||
|
|
||||||
from test_cicd import release, release_module
|
|
||||||
|
|
||||||
|
|
||||||
def plan_data(changed):
|
|
||||||
targets = []
|
|
||||||
for name, (context, dockerfile) in release_module.IMAGES.items():
|
|
||||||
targets.append(
|
|
||||||
{
|
|
||||||
'name': name,
|
|
||||||
'image': f'gcr.forust.xyz/forust/{name}',
|
|
||||||
'context': context,
|
|
||||||
'dockerfile': dockerfile,
|
|
||||||
'inputs': ('d' if name in changed else 'c') * 64,
|
|
||||||
'reuse_digest': None if name in changed else 'sha256:' + 'b' * 64,
|
|
||||||
}
|
|
||||||
)
|
|
||||||
return {'sha': 'a' * 40, 'targets': targets}
|
|
||||||
|
|
||||||
|
|
||||||
class MatrixTests(unittest.TestCase):
|
|
||||||
def test_no_change_one_image_all_images_and_missing_baseline(self):
|
|
||||||
for changed in (set(), {'error-pages'}, set(release_module.IMAGES)):
|
|
||||||
with self.subTest(changed=changed), tempfile.TemporaryDirectory() as scratch:
|
|
||||||
api = Mock()
|
|
||||||
api.successful_runs.return_value = iter([{'id': 1}])
|
|
||||||
api.release.return_value = release()
|
|
||||||
expected = plan_data(changed)
|
|
||||||
fingerprints = {t['dockerfile']: t['inputs'] for t in expected['targets']}
|
|
||||||
output = Path(scratch) / 'plan.json'
|
|
||||||
with (
|
|
||||||
patch.dict(os.environ, {'GITHUB_SHA': 'a' * 40, 'GITHUB_RUN_ID': '2'}),
|
|
||||||
patch.object(release_module, 'command', return_value='a' * 40),
|
|
||||||
patch.object(release_module, 'Gitea', return_value=api),
|
|
||||||
patch.object(
|
|
||||||
release_module, 'fingerprint', side_effect=lambda _c, f, mapping=fingerprints: mapping[f]
|
|
||||||
),
|
|
||||||
):
|
|
||||||
release_module.prepare_images(output)
|
|
||||||
self.assertEqual(json.loads(output.read_text()), expected)
|
|
||||||
api.successful_runs.return_value = iter([])
|
|
||||||
with (
|
|
||||||
tempfile.TemporaryDirectory() as scratch,
|
|
||||||
patch.dict(os.environ, {'GITHUB_SHA': 'a' * 40}),
|
|
||||||
patch.object(release_module, 'command', return_value='a' * 40),
|
|
||||||
patch.object(release_module, 'Gitea', return_value=api),
|
|
||||||
patch.object(release_module, 'fingerprint', return_value='c' * 64),
|
|
||||||
):
|
|
||||||
output = Path(scratch) / 'plan.json'
|
|
||||||
release_module.prepare_images(output)
|
|
||||||
self.assertTrue(all(t['reuse_digest'] is None for t in json.loads(output.read_text())['targets']))
|
|
||||||
|
|
||||||
def test_incomplete_or_wrong_sha_fragments_cannot_publish_tags(self):
|
|
||||||
for wrong_sha in (False, True):
|
|
||||||
with self.subTest(wrong_sha=wrong_sha), tempfile.TemporaryDirectory() as scratch:
|
|
||||||
root = Path(scratch)
|
|
||||||
plan = root / 'plan.json'
|
|
||||||
plan.write_text(json.dumps(plan_data(set())))
|
|
||||||
for name in release_module.IMAGES:
|
|
||||||
if name == 'xdfnx-homepage' and not wrong_sha:
|
|
||||||
continue
|
|
||||||
folder = root / f'image-{name}'
|
|
||||||
folder.mkdir()
|
|
||||||
image = f'gcr.forust.xyz/forust/{name}'
|
|
||||||
folder.joinpath('image.json').write_text(
|
|
||||||
json.dumps(
|
|
||||||
{
|
|
||||||
'version': 1,
|
|
||||||
'sha': ('e' if wrong_sha else 'a') * 40,
|
|
||||||
'images': {image: 'sha256:' + 'b' * 64},
|
|
||||||
'inputs': {image: 'c' * 64},
|
|
||||||
}
|
|
||||||
)
|
|
||||||
)
|
|
||||||
with (
|
|
||||||
patch.object(release_module, 'checked_plan', return_value=plan_data(set())),
|
|
||||||
patch.object(release_module.subprocess, 'run') as execute,
|
|
||||||
self.assertRaises((ValueError, FileNotFoundError)),
|
|
||||||
):
|
|
||||||
release_module.finalize_images(root / 'release.json', root, plan)
|
|
||||||
execute.assert_not_called()
|
|
||||||
self.assertFalse((root / 'release.json').exists())
|
|
||||||
|
|
||||||
def test_manifest_only_release_pins_each_successful_digest(self):
|
|
||||||
with tempfile.TemporaryDirectory() as scratch:
|
|
||||||
root = Path(scratch)
|
|
||||||
data = plan_data(set())
|
|
||||||
for target in data['targets']:
|
|
||||||
folder = root / f'image-{target["name"]}'
|
|
||||||
folder.mkdir()
|
|
||||||
image = target['image']
|
|
||||||
folder.joinpath('image.json').write_text(
|
|
||||||
json.dumps(
|
|
||||||
{
|
|
||||||
'version': 1,
|
|
||||||
'sha': data['sha'],
|
|
||||||
'images': {image: target['reuse_digest']},
|
|
||||||
'inputs': {image: target['inputs']},
|
|
||||||
}
|
|
||||||
)
|
|
||||||
)
|
|
||||||
with (
|
|
||||||
patch.object(release_module, 'checked_plan', return_value=data),
|
|
||||||
patch.dict(os.environ, {'REGISTRY_USERNAME': 'test', 'REGISTRY_PASSWORD': 'placeholder'}),
|
|
||||||
patch.object(release_module.subprocess, 'run'),
|
|
||||||
patch.object(release_module, 'command') as execute,
|
|
||||||
):
|
|
||||||
release_module.finalize_images(root / 'release.json', root, root / 'plan.json')
|
|
||||||
self.assertEqual(
|
|
||||||
[entry.args for entry in execute.call_args_list],
|
|
||||||
[
|
|
||||||
call(
|
|
||||||
'docker',
|
|
||||||
'buildx',
|
|
||||||
'imagetools',
|
|
||||||
'create',
|
|
||||||
'--prefer-index=false',
|
|
||||||
'--tag',
|
|
||||||
f'{t["image"]}:sha-{data["sha"]}',
|
|
||||||
f'{t["image"]}@{t["reuse_digest"]}',
|
|
||||||
).args
|
|
||||||
for t in data['targets']
|
|
||||||
],
|
|
||||||
)
|
|
||||||
self.assertEqual(json.loads((root / 'release.json').read_text()), release())
|
|
||||||
|
|
||||||
def test_checkout_mismatch_cannot_build(self):
|
|
||||||
with tempfile.TemporaryDirectory() as scratch:
|
|
||||||
plan = Path(scratch) / 'plan.json'
|
|
||||||
plan.write_text(json.dumps(plan_data(set())))
|
|
||||||
with (
|
|
||||||
patch.dict(os.environ, {'GITHUB_SHA': 'e' * 40}),
|
|
||||||
patch.object(release_module, 'command', return_value='a' * 40),
|
|
||||||
self.assertRaisesRegex(ValueError, 'source commit'),
|
|
||||||
):
|
|
||||||
release_module.checked_plan(plan)
|
|
||||||
Reference in new issue
Block a user