diff --git a/.gitignore b/.gitignore index cbee9c6..b09dea4 100644 --- a/.gitignore +++ b/.gitignore @@ -21,6 +21,9 @@ checkmk/checkmk/* downtify/Downtify_downloads headscale/config/* headscale/data/* +# NetBird local hostnames and generated secrets +netbird/.env +netbird/secrets/ searxng/core-config/* # Steaming services files diff --git a/netbird/.env.example b/netbird/.env.example new file mode 100644 index 0000000..fbd3ccc --- /dev/null +++ b/netbird/.env.example @@ -0,0 +1,13 @@ +# Public hostname advertised to NetBird clients and used for TLS/OAuth. +NETBIRD_DOMAIN=nb.forust.xyz + +# Internal-only aliases routed by the existing Traefik instance. +NETBIRD_LOCAL_DOMAIN=netbird.workstation.internal +NETBIRD_DEV_DOMAIN=netbird.gigaforust.internal + +NETBIRD_PROXY_SUBNET=auto + +NETBIRD_CLIENT_HOSTNAME=hostname + +# Add a dashboard-generated setup key before starting client.compose.yaml. +# NB_SETUP_KEY= diff --git a/netbird/README.md b/netbird/README.md new file mode 100644 index 0000000..f5ef834 --- /dev/null +++ b/netbird/README.md @@ -0,0 +1,123 @@ +# NetBird + +Self-hosted NetBird with the combined management, signal, relay, and STUN server. The dashboard and server run behind the repository's existing external Traefik instance on the Docker `proxy` network. Only STUN UDP `3478` is published directly. + +The deployment uses SQLite for a single-instance homelab server. The persistent `netbird_data` volume and the datastore encryption key are both required to recover the installation. + +## Files + +- `compose.yaml`: dashboard and combined server; selected by the marker-driven deploy workflow through `active`. +- `config.template.yaml`: non-secret server configuration rendered at startup. +- `entrypoint.sh`: injects Docker secrets into an in-memory runtime configuration. +- `client.compose.yaml`: optional host-network peer using a dashboard-generated setup key. +- `.env`: ignored local hostnames, the detected Traefik Docker-network subnet, and optional client setup key. +- `secrets/`: ignored relay secret and datastore encryption key. + +## First deployment + +Run these commands on the Docker host before merging the activating branch. The deploy preflight resets tracked files but preserves ignored local state. + +```bash +cd /srv/homelab/netbird +./setup.sh +$EDITOR .env +docker compose config --quiet +docker compose up -d +``` + +Review the values in `.env` before starting. The example public hostname is `netbird.forust.xyz`; change it if a different public domain was selected. `setup.sh` replaces `NETBIRD_PROXY_SUBNET=auto` with the first IPv4 subnet of the external Docker `proxy` network. Keep that value synchronized with the network; set an explicit CIDR instead if the network is managed elsewhere. + +`setup.sh` is idempotent and never replaces existing secrets. Do not delete or regenerate `secrets/datastore-encryption-key` after the first successful start unless all encrypted setup keys and API tokens are intentionally being invalidated. + +## Network prerequisites + +- Point the public hostname directly to the Docker host. Do not proxy UDP `3478` through Cloudflare or another CDN. +- Allow inbound TCP `80`, TCP `443`, and UDP `3478` through the host firewall and upstream router. +- Ensure the external `proxy` Docker network exists and Traefik uses its `websecure` entrypoint and `letsencrypt` resolver. `NETBIRD_PROXY_SUBNET` must describe that network; it is used to trust only forwarded client addresses from Traefik. +- Ensure the internal names in `.env` resolve where the local and development aliases are needed. +- Keep Traefik's `websecure` read timeout disabled for long-lived gRPC and WebSocket sessions. This repository configures `--entrypoints.websecure.transport.respondingTimeouts.readTimeout=0` in `traefik/compose.yaml`. + +After startup, verify OIDC discovery through the public TLS endpoint: + +```bash +curl -fsS "https://${NETBIRD_DOMAIN}/oauth2/.well-known/openid-configuration" +``` + +Open `https://${NETBIRD_DOMAIN}` immediately and complete the initial owner setup. Treat the initial setup flow as public until the owner exists. + +## Optional host client + +The client intentionally lives in a separate Compose project. Normal server deploys use `--remove-orphans`, so keeping the client in the server project would cause it to be removed. + +1. Create a reusable or ephemeral setup key in the NetBird dashboard. +2. Put `NB_SETUP_KEY=` in the ignored `netbird/.env` file. +3. Set `NETBIRD_CLIENT_HOSTNAME` to this machine's desired peer name. +4. Start and inspect the client: + +```bash +cd /srv/homelab/netbird +docker compose -f client.compose.yaml config --quiet +docker compose -f client.compose.yaml up -d +docker compose -f client.compose.yaml exec netbird-client netbird status +``` + +The client uses host networking and requires `NET_ADMIN`, `SYS_ADMIN`, `SYS_RESOURCE`, and `/dev/net/tun`. Remove it without affecting the server stack: + +```bash +docker compose -f client.compose.yaml down +``` + +## Operations + +Inspect status and logs: + +```bash +docker compose ps +docker compose logs --tail=200 netbird-server dashboard +``` + +Stop or remove containers without deleting data: + +```bash +docker compose down +``` + +Do not add `-v` to `docker compose down`; it would delete the NetBird datastore. + +## Backup and restore + +Back up both the persistent volume and the ignored secret files. For a consistent SQLite backup, briefly stop the server first and store the resulting archive and `datastore-encryption-key` in an encrypted backup: + +```bash +cd /srv/homelab/netbird +mkdir -p backups +docker compose stop netbird-server +docker run --rm \ + -v netbird_data:/data:ro \ + -v "$PWD/backups:/backup" \ + busybox:1.37.0 \ + tar -C /data -czf "/backup/netbird-data-$(date -u +%Y%m%dT%H%M%SZ).tar.gz" . +docker compose start netbird-server +``` + +Also securely back up: + +- `secrets/datastore-encryption-key` — required to decrypt stored secrets. +- `secrets/relay-auth-secret` — keeps issued relay credentials valid across restoration. +- `netbird/.env` — optional, but it records the public and internal hostnames. + +Test a restore in an isolated Docker host before relying on a backup. + +## Upgrade + +1. Take and verify a backup. +2. Review NetBird release notes for server, client, and dashboard compatibility. +3. Update the pinned tags in `compose.yaml`; update `client.compose.yaml` separately when deploying the client. +4. Pull and recreate the selected services: + +```bash +docker compose pull +docker compose up -d +``` + +The image tags are intentionally pinned instead of using `latest`, matching this repository's pull-on-deploy policy. diff --git a/netbird/client.compose.yaml b/netbird/client.compose.yaml new file mode 100644 index 0000000..aa3d783 --- /dev/null +++ b/netbird/client.compose.yaml @@ -0,0 +1,31 @@ +name: netbird-client + +services: + netbird-client: + image: netbirdio/netbird:0.79.0 + container_name: netbird-client + hostname: "${NETBIRD_CLIENT_HOSTNAME:?Set NETBIRD_CLIENT_HOSTNAME in netbird/.env}" + restart: unless-stopped + cap_add: + - NET_ADMIN + - SYS_ADMIN + - SYS_RESOURCE + devices: + - /dev/net/tun + network_mode: host + environment: + NB_SETUP_KEY: "${NB_SETUP_KEY:?Set NB_SETUP_KEY in netbird/.env after creating a peer setup key}" + NB_MANAGEMENT_URL: "https://${NETBIRD_DOMAIN:?Set NETBIRD_DOMAIN in netbird/.env}" + volumes: + - netbird-client:/var/lib/netbird + healthcheck: + test: ["CMD", "/usr/local/bin/netbird", "status", "--check", "live"] + interval: 30s + timeout: 5s + retries: 5 + start_period: 30s + stop_grace_period: 30s + +volumes: + netbird-client: + name: netbird-client diff --git a/netbird/compose.yaml b/netbird/compose.yaml new file mode 100644 index 0000000..9f55ca7 --- /dev/null +++ b/netbird/compose.yaml @@ -0,0 +1,152 @@ +name: netbird + +services: + netbird-server: + image: netbirdio/netbird-server:0.79.0 + container_name: netbird-server + restart: unless-stopped + environment: + NETBIRD_DOMAIN: "${NETBIRD_DOMAIN:?Set NETBIRD_DOMAIN in netbird/.env}" + NETBIRD_PROXY_SUBNET: "${NETBIRD_PROXY_SUBNET:?Set NETBIRD_PROXY_SUBNET in netbird/.env (run setup.sh)}" + entrypoint: + - /bin/sh + - /opt/netbird/entrypoint.sh + command: + - --config + - /run/netbird/config.yaml + ports: + - "3478:3478/udp" + volumes: + - netbird_data:/var/lib/netbird + - ./config.template.yaml:/opt/netbird/config.template.yaml:ro + - ./entrypoint.sh:/opt/netbird/entrypoint.sh:ro + secrets: + - relay_auth_secret + - datastore_encryption_key + tmpfs: + - /run/netbird:mode=0700 + healthcheck: + test: + - CMD + - bash + - -ec + - exec 3<>/dev/tcp/127.0.0.1/80 + interval: 30s + timeout: 5s + retries: 5 + start_period: 30s + stop_grace_period: 30s + labels: + - "traefik.enable=true" + - "traefik.http.services.netbird-server.loadbalancer.server.port=80" + - "traefik.http.services.netbird-server-h2c.loadbalancer.server.port=80" + - "traefik.http.services.netbird-server-h2c.loadbalancer.server.scheme=h2c" + + # gRPC routers + # Prod Router + - "traefik.http.routers.netbird-grpc.rule=Host(`${NETBIRD_DOMAIN:?Set NETBIRD_DOMAIN in netbird/.env}`) && (PathPrefix(`/signalexchange.SignalExchange/`) || PathPrefix(`/management.ManagementService/`) || PathPrefix(`/management.ProxyService/`))" + - "traefik.http.routers.netbird-grpc.entrypoints=websecure" + - "traefik.http.routers.netbird-grpc.service=netbird-server-h2c" + - "traefik.http.routers.netbird-grpc.priority=100" + - "traefik.http.routers.netbird-grpc.tls=true" + - "traefik.http.routers.netbird-grpc.tls.certresolver=letsencrypt" + # Local Router + - "traefik.http.routers.netbird-grpc-local.rule=Host(`${NETBIRD_LOCAL_DOMAIN:?Set NETBIRD_LOCAL_DOMAIN in netbird/.env}`) && (PathPrefix(`/signalexchange.SignalExchange/`) || PathPrefix(`/management.ManagementService/`) || PathPrefix(`/management.ProxyService/`))" + - "traefik.http.routers.netbird-grpc-local.entrypoints=websecure" + - "traefik.http.routers.netbird-grpc-local.service=netbird-server-h2c" + - "traefik.http.routers.netbird-grpc-local.priority=100" + - "traefik.http.routers.netbird-grpc-local.tls=true" + # Dev Router + - "traefik.http.routers.netbird-grpc-dev.rule=Host(`${NETBIRD_DEV_DOMAIN:?Set NETBIRD_DEV_DOMAIN in netbird/.env}`) && (PathPrefix(`/signalexchange.SignalExchange/`) || PathPrefix(`/management.ManagementService/`) || PathPrefix(`/management.ProxyService/`))" + - "traefik.http.routers.netbird-grpc-dev.entrypoints=websecure" + - "traefik.http.routers.netbird-grpc-dev.service=netbird-server-h2c" + - "traefik.http.routers.netbird-grpc-dev.priority=100" + - "traefik.http.routers.netbird-grpc-dev.tls=true" + + # Backend routers + # Prod Router + - "traefik.http.routers.netbird-backend.rule=Host(`${NETBIRD_DOMAIN:?Set NETBIRD_DOMAIN in netbird/.env}`) && (PathPrefix(`/relay`) || PathPrefix(`/ws-proxy/`) || PathPrefix(`/api`) || PathPrefix(`/oauth2`))" + - "traefik.http.routers.netbird-backend.entrypoints=websecure" + - "traefik.http.routers.netbird-backend.service=netbird-server" + - "traefik.http.routers.netbird-backend.priority=100" + - "traefik.http.routers.netbird-backend.tls=true" + - "traefik.http.routers.netbird-backend.tls.certresolver=letsencrypt" + # Local Router + - "traefik.http.routers.netbird-backend-local.rule=Host(`${NETBIRD_LOCAL_DOMAIN:?Set NETBIRD_LOCAL_DOMAIN in netbird/.env}`) && (PathPrefix(`/relay`) || PathPrefix(`/ws-proxy/`) || PathPrefix(`/api`) || PathPrefix(`/oauth2`))" + - "traefik.http.routers.netbird-backend-local.entrypoints=websecure" + - "traefik.http.routers.netbird-backend-local.service=netbird-server" + - "traefik.http.routers.netbird-backend-local.priority=100" + - "traefik.http.routers.netbird-backend-local.tls=true" + # Dev Router + - "traefik.http.routers.netbird-backend-dev.rule=Host(`${NETBIRD_DEV_DOMAIN:?Set NETBIRD_DEV_DOMAIN in netbird/.env}`) && (PathPrefix(`/relay`) || PathPrefix(`/ws-proxy/`) || PathPrefix(`/api`) || PathPrefix(`/oauth2`))" + - "traefik.http.routers.netbird-backend-dev.entrypoints=websecure" + - "traefik.http.routers.netbird-backend-dev.service=netbird-server" + - "traefik.http.routers.netbird-backend-dev.priority=100" + - "traefik.http.routers.netbird-backend-dev.tls=true" + networks: + - proxy + + dashboard: + image: netbirdio/dashboard:v2.90.10 + container_name: netbird-dashboard + restart: unless-stopped + environment: + NETBIRD_MGMT_API_ENDPOINT: "https://${NETBIRD_DOMAIN:?Set NETBIRD_DOMAIN in netbird/.env}" + NETBIRD_MGMT_GRPC_API_ENDPOINT: "https://${NETBIRD_DOMAIN:?Set NETBIRD_DOMAIN in netbird/.env}" + AUTH_AUDIENCE: netbird-dashboard + AUTH_CLIENT_ID: netbird-dashboard + AUTH_CLIENT_SECRET: "" + AUTH_AUTHORITY: "https://${NETBIRD_DOMAIN:?Set NETBIRD_DOMAIN in netbird/.env}/oauth2" + AUTH_SUPPORTED_SCOPES: openid profile email groups + AUTH_REDIRECT_URI: /nb-auth + AUTH_SILENT_REDIRECT_URI: /nb-silent-auth + USE_AUTH0: "false" + LETSENCRYPT_DOMAIN: none + depends_on: + netbird-server: + condition: service_healthy + healthcheck: + test: ["CMD", "curl", "--fail", "--silent", "--show-error", "http://127.0.0.1/"] + interval: 30s + timeout: 5s + retries: 5 + start_period: 15s + labels: + - "traefik.enable=true" + - "traefik.http.services.netbird-dashboard.loadbalancer.server.port=80" + # Dashboard catch-all routers + # Prod Router + - "traefik.http.routers.netbird-dashboard.rule=Host(`${NETBIRD_DOMAIN:?Set NETBIRD_DOMAIN in netbird/.env}`)" + - "traefik.http.routers.netbird-dashboard.entrypoints=websecure" + - "traefik.http.routers.netbird-dashboard.service=netbird-dashboard" + - "traefik.http.routers.netbird-dashboard.priority=1" + - "traefik.http.routers.netbird-dashboard.tls=true" + - "traefik.http.routers.netbird-dashboard.tls.certresolver=letsencrypt" + # Local Router + - "traefik.http.routers.netbird-dashboard-local.rule=Host(`${NETBIRD_LOCAL_DOMAIN:?Set NETBIRD_LOCAL_DOMAIN in netbird/.env}`)" + - "traefik.http.routers.netbird-dashboard-local.entrypoints=websecure" + - "traefik.http.routers.netbird-dashboard-local.service=netbird-dashboard" + - "traefik.http.routers.netbird-dashboard-local.priority=1" + - "traefik.http.routers.netbird-dashboard-local.tls=true" + # Dev Router + - "traefik.http.routers.netbird-dashboard-dev.rule=Host(`${NETBIRD_DEV_DOMAIN:?Set NETBIRD_DEV_DOMAIN in netbird/.env}`)" + - "traefik.http.routers.netbird-dashboard-dev.entrypoints=websecure" + - "traefik.http.routers.netbird-dashboard-dev.service=netbird-dashboard" + - "traefik.http.routers.netbird-dashboard-dev.priority=1" + - "traefik.http.routers.netbird-dashboard-dev.tls=true" + networks: + - proxy + +networks: + proxy: + external: true + +volumes: + netbird_data: + name: netbird_data + +secrets: + relay_auth_secret: + file: ./secrets/relay-auth-secret + datastore_encryption_key: + file: ./secrets/datastore-encryption-key diff --git a/netbird/config.template.yaml b/netbird/config.template.yaml new file mode 100644 index 0000000..1ec383c --- /dev/null +++ b/netbird/config.template.yaml @@ -0,0 +1,26 @@ +server: + listenAddress: ":80" + exposedAddress: "https://__NETBIRD_DOMAIN__:443" + stunPorts: + - 3478 + metricsPort: 9090 + healthcheckAddress: ":9000" + logLevel: info + logFile: console + authSecret: "__NETBIRD_AUTH_SECRET__" + dataDir: "/var/lib/netbird" + disableAnonymousMetrics: true + auth: + issuer: "https://__NETBIRD_DOMAIN__/oauth2" + signKeyRefreshEnabled: true + dashboardRedirectURIs: + - "https://__NETBIRD_DOMAIN__/nb-auth" + - "https://__NETBIRD_DOMAIN__/nb-silent-auth" + reverseProxy: + trustedHTTPProxies: + - "__NETBIRD_PROXY_SUBNET__" + trustedPeers: + - "__NETBIRD_PROXY_SUBNET__" + store: + engine: sqlite + encryptionKey: "__NETBIRD_ENCRYPTION_KEY__" diff --git a/netbird/k8s/certificates.yaml b/netbird/k8s/certificates.yaml new file mode 100644 index 0000000..5a2afca --- /dev/null +++ b/netbird/k8s/certificates.yaml @@ -0,0 +1,28 @@ +apiVersion: cert-manager.io/v1 +kind: Certificate +metadata: + name: netbird-prod-tls + namespace: netbird +spec: + secretName: netbird-prod-tls + dnsNames: + - nb.forust.xyz + issuerRef: + name: letsencrypt-prod + kind: ClusterIssuer +--- +apiVersion: cert-manager.io/v1 +kind: Certificate +metadata: + name: internal-wildcard-tls + namespace: netbird +spec: + secretName: internal-wildcard-tls + dnsNames: + - "*.workstation.internal" + - "*.gigaforust.internal" + - workstation.internal + - gigaforust.internal + issuerRef: + name: internal-ca + kind: ClusterIssuer diff --git a/netbird/k8s/config.yaml b/netbird/k8s/config.yaml new file mode 100644 index 0000000..ae150b9 --- /dev/null +++ b/netbird/k8s/config.yaml @@ -0,0 +1,160 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: netbird-config + namespace: netbird +data: + # Public hostname, rendered into the server config by entrypoint.sh. + NETBIRD_DOMAIN: "nb.forust.xyz" + NETBIRD_PROXY_SUBNET: "10.244.0.0/16" + + NETBIRD_MGMT_API_ENDPOINT: "https://nb.forust.xyz" + NETBIRD_MGMT_GRPC_API_ENDPOINT: "https://nb.forust.xyz" + AUTH_AUDIENCE: "netbird-dashboard" + AUTH_CLIENT_ID: "netbird-dashboard" + AUTH_CLIENT_SECRET: "" + AUTH_AUTHORITY: "https://nb.forust.xyz/oauth2" + AUTH_SUPPORTED_SCOPES: "openid profile email groups" + AUTH_REDIRECT_URI: "/nb-auth" + AUTH_SILENT_REDIRECT_URI: "/nb-silent-auth" + USE_AUTH0: "false" + LETSENCRYPT_DOMAIN: "none" + + config.template.yaml: | + server: + listenAddress: ":80" + exposedAddress: "https://__NETBIRD_DOMAIN__:443" + stunPorts: + - 3478 + metricsPort: 9090 + healthcheckAddress: ":9000" + logLevel: info + logFile: console + authSecret: "__NETBIRD_AUTH_SECRET__" + dataDir: "/var/lib/netbird" + disableAnonymousMetrics: true + auth: + issuer: "https://__NETBIRD_DOMAIN__/oauth2" + signKeyRefreshEnabled: true + dashboardRedirectURIs: + - "https://__NETBIRD_DOMAIN__/nb-auth" + - "https://__NETBIRD_DOMAIN__/nb-silent-auth" + reverseProxy: + trustedHTTPProxies: + - "__NETBIRD_PROXY_SUBNET__" + trustedPeers: + - "__NETBIRD_PROXY_SUBNET__" + store: + engine: sqlite + encryptionKey: "__NETBIRD_ENCRYPTION_KEY__" + + entrypoint.sh: | + #!/bin/sh + set -eu + + umask 077 + + TEMPLATE_PATH=/opt/netbird/config.template.yaml + RENDERED_PATH=/run/netbird/config.yaml + RELAY_SECRET_PATH=/run/secrets/relay_auth_secret + ENCRYPTION_KEY_PATH=/run/secrets/datastore_encryption_key + + is_valid_proxy_subnet() { + candidate="$1" + case "$candidate" in + 0.0.0.0/0) + return 1 + ;; + */*) + address="${candidate%%/*}" + prefix="${candidate#*/}" + ;; + *) + return 1 + ;; + esac + + case "$prefix" in + 0|[1-9]|[1-2][0-9]|3[0-2]) ;; + *) + return 1 + ;; + esac + + old_ifs="$IFS" + IFS=. + # shellcheck disable=SC2086 + set -- $address + IFS="$old_ifs" + [ "$#" -eq 4 ] || return 1 + + for octet do + case "$octet" in + 0|[1-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5]) ;; + *) + return 1 + ;; + esac + done + } + + read_secret() { + secret_path="$1" + + if [ ! -r "$secret_path" ]; then + echo "Required secret is not readable: $secret_path" >&2 + exit 1 + fi + + secret_value="$(cat "$secret_path")" + if [ -z "$secret_value" ]; then + echo "Required secret is empty: $secret_path" >&2 + exit 1 + fi + + printf '%s' "$secret_value" + } + + if [ -z "${NETBIRD_DOMAIN:-}" ]; then + echo "NETBIRD_DOMAIN must be set" >&2 + exit 1 + fi + + case "$NETBIRD_DOMAIN" in + *[!A-Za-z0-9.-]*) + echo "NETBIRD_DOMAIN contains unsupported characters" >&2 + exit 1 + ;; + esac + + if [ -z "${NETBIRD_PROXY_SUBNET:-}" ] || [ "$NETBIRD_PROXY_SUBNET" = "auto" ]; then + echo "NETBIRD_PROXY_SUBNET must be an explicit IPv4 CIDR; run netbird/setup.sh first" >&2 + exit 1 + fi + if ! is_valid_proxy_subnet "$NETBIRD_PROXY_SUBNET"; then + echo "NETBIRD_PROXY_SUBNET must be a non-default IPv4 CIDR, for example 172.20.0.0/16" >&2 + exit 1 + fi + + if [ "$#" -ne 2 ] || [ "$1" != "--config" ] || [ "$2" != "$RENDERED_PATH" ]; then + echo "Expected: --config $RENDERED_PATH" >&2 + exit 1 + fi + + relay_secret="$(read_secret "$RELAY_SECRET_PATH")" + encryption_key="$(read_secret "$ENCRYPTION_KEY_PATH")" + + mkdir -p "$(dirname "$RENDERED_PATH")" + sed \ + -e "s|__NETBIRD_DOMAIN__|${NETBIRD_DOMAIN}|g" \ + -e "s|__NETBIRD_AUTH_SECRET__|${relay_secret}|g" \ + -e "s|__NETBIRD_ENCRYPTION_KEY__|${encryption_key}|g" \ + -e "s|__NETBIRD_PROXY_SUBNET__|${NETBIRD_PROXY_SUBNET}|g" \ + "$TEMPLATE_PATH" >"$RENDERED_PATH" + + if grep -q '__NETBIRD_' "$RENDERED_PATH"; then + echo "Rendered NetBird configuration still contains unresolved placeholders" >&2 + exit 1 + fi + + exec /go/bin/netbird-server "$@" diff --git a/netbird/k8s/ingress.yaml b/netbird/k8s/ingress.yaml new file mode 100644 index 0000000..a1d762e --- /dev/null +++ b/netbird/k8s/ingress.yaml @@ -0,0 +1,83 @@ +apiVersion: traefik.io/v1alpha1 +kind: IngressRoute +metadata: + name: netbird-prod + namespace: netbird +spec: + entryPoints: + - websecure + routes: + - match: Host(`nb.forust.xyz`) && (PathPrefix(`/signalexchange.SignalExchange/`) || PathPrefix(`/management.ManagementService/`) || PathPrefix(`/management.ProxyService/`)) + kind: Rule + priority: 100 + middlewares: + - name: crowdsec-bouncer + namespace: crowdsec + services: + - name: netbird-server-service + port: 80 + scheme: h2c + - match: Host(`nb.forust.xyz`) && (PathPrefix(`/relay`) || PathPrefix(`/ws-proxy/`) || PathPrefix(`/api`) || PathPrefix(`/oauth2`)) + kind: Rule + priority: 100 + middlewares: + - name: crowdsec-bouncer + namespace: crowdsec + services: + - name: netbird-server-service + port: 80 + - match: Host(`nb.forust.xyz`) + kind: Rule + priority: 1 + middlewares: + - name: crowdsec-bouncer + namespace: crowdsec + services: + - name: netbird-dashboard-service + port: 80 + tls: + secretName: netbird-prod-tls +--- +apiVersion: traefik.io/v1alpha1 +kind: IngressRoute +metadata: + name: netbird-local + namespace: netbird +spec: + entryPoints: + - websecure + routes: + - match: (Host(`netbird.workstation.internal`) || Host(`netbird.gigaforust.internal`)) && (PathPrefix(`/signalexchange.SignalExchange/`) || PathPrefix(`/management.ManagementService/`) || PathPrefix(`/management.ProxyService/`)) + kind: Rule + priority: 100 + services: + - name: netbird-server-service + port: 80 + scheme: h2c + - match: (Host(`netbird.workstation.internal`) || Host(`netbird.gigaforust.internal`)) && (PathPrefix(`/relay`) || PathPrefix(`/ws-proxy/`) || PathPrefix(`/api`) || PathPrefix(`/oauth2`)) + kind: Rule + priority: 100 + services: + - name: netbird-server-service + port: 80 + - match: Host(`netbird.workstation.internal`) || Host(`netbird.gigaforust.internal`) + kind: Rule + priority: 1 + services: + - name: netbird-dashboard-service + port: 80 + tls: + secretName: internal-wildcard-tls +--- +apiVersion: traefik.io/v1alpha1 +kind: IngressRouteUDP +metadata: + name: netbird-stun + namespace: netbird +spec: + entryPoints: + - netbird-stun + routes: + - services: + - name: netbird-server-service + port: 3478 diff --git a/netbird/k8s/namespace.yaml b/netbird/k8s/namespace.yaml new file mode 100644 index 0000000..db05a13 --- /dev/null +++ b/netbird/k8s/namespace.yaml @@ -0,0 +1,4 @@ +apiVersion: v1 +kind: Namespace +metadata: + name: netbird diff --git a/netbird/k8s/netbird.yaml b/netbird/k8s/netbird.yaml new file mode 100644 index 0000000..7ae1fc4 --- /dev/null +++ b/netbird/k8s/netbird.yaml @@ -0,0 +1,181 @@ +apiVersion: v1 +kind: Service +metadata: + name: netbird-server-service + namespace: netbird +spec: + selector: + app: netbird-server + ports: + - port: 80 + name: http + targetPort: 80 + protocol: TCP + - port: 3478 + name: stun + targetPort: 3478 + protocol: UDP +--- +apiVersion: v1 +kind: Service +metadata: + name: netbird-dashboard-service + namespace: netbird +spec: + selector: + app: netbird-dashboard + ports: + - port: 80 + name: http + targetPort: 80 +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: netbird-server-deployment + namespace: netbird +spec: + replicas: 1 + selector: + matchLabels: + app: netbird-server + template: + metadata: + labels: + app: netbird-server + spec: + containers: + - name: netbird-server + image: netbirdio/netbird-server:0.79.0 + command: ["/bin/sh", "/opt/netbird/entrypoint.sh", "--config", "/run/netbird/config.yaml"] + envFrom: + - configMapRef: + name: netbird-config + ports: + - containerPort: 80 + name: http + protocol: TCP + - containerPort: 3478 + name: stun + protocol: UDP + volumeMounts: + - name: netbird-data + mountPath: /var/lib/netbird + - name: netbird-files + mountPath: /opt/netbird + readOnly: true + - name: netbird-secrets + mountPath: /run/secrets/relay_auth_secret + subPath: relay_auth_secret + readOnly: true + - name: netbird-secrets + mountPath: /run/secrets/datastore_encryption_key + subPath: datastore_encryption_key + readOnly: true + - name: netbird-run + mountPath: /run/netbird + readinessProbe: + tcpSocket: + port: 80 + initialDelaySeconds: 30 + periodSeconds: 30 + timeoutSeconds: 5 + failureThreshold: 5 + livenessProbe: + tcpSocket: + port: 80 + initialDelaySeconds: 60 + periodSeconds: 30 + timeoutSeconds: 5 + failureThreshold: 5 + resources: + requests: + memory: "256Mi" + cpu: "250m" + limits: + memory: "1Gi" + cpu: "1000m" + volumes: + - name: netbird-data + persistentVolumeClaim: + claimName: netbird-pvc + - name: netbird-files + configMap: + name: netbird-config + defaultMode: 0755 + items: + - key: config.template.yaml + path: config.template.yaml + - key: entrypoint.sh + path: entrypoint.sh + - name: netbird-secrets + secret: + secretName: netbird-secrets + items: + - key: relay_auth_secret + path: relay_auth_secret + - key: datastore_encryption_key + path: datastore_encryption_key + - name: netbird-run + emptyDir: + medium: Memory +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: netbird-dashboard-deployment + namespace: netbird +spec: + replicas: 1 + selector: + matchLabels: + app: netbird-dashboard + template: + metadata: + labels: + app: netbird-dashboard + spec: + containers: + - name: dashboard + image: netbirdio/dashboard:v2.90.10 + envFrom: + - configMapRef: + name: netbird-config + ports: + - containerPort: 80 + name: http + readinessProbe: + httpGet: + path: / + port: 80 + initialDelaySeconds: 15 + periodSeconds: 30 + timeoutSeconds: 5 + failureThreshold: 5 + livenessProbe: + httpGet: + path: / + port: 80 + initialDelaySeconds: 30 + periodSeconds: 30 + timeoutSeconds: 5 + failureThreshold: 5 + resources: + requests: + memory: "64Mi" + cpu: "50m" + limits: + memory: "256Mi" + cpu: "300m" +--- +apiVersion: v1 +kind: PersistentVolumeClaim +metadata: + name: netbird-pvc + namespace: netbird +spec: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 2Gi diff --git a/netbird/k8s/secrets.yaml.example b/netbird/k8s/secrets.yaml.example new file mode 100644 index 0000000..699f9ef --- /dev/null +++ b/netbird/k8s/secrets.yaml.example @@ -0,0 +1,11 @@ +apiVersion: v1 +kind: Secret +metadata: + name: netbird-secrets + namespace: netbird +type: Opaque +stringData: + # hex, 64 chars: openssl rand -hex 32 + relay_auth_secret: "REPLACE_ME" + # base64, 44 chars: openssl rand -base64 32 + datastore_encryption_key: "REPLACE_ME" diff --git a/traefik/compose.yaml b/traefik/compose.yaml index 247d12a..05b8a7c 100644 --- a/traefik/compose.yaml +++ b/traefik/compose.yaml @@ -20,6 +20,7 @@ services: - "--entryPoints.web.http.redirections.entryPoint.scheme=https" - "--entryPoints.web.http.redirections.entryPoint.to=websecure" - "--entryPoints.websecure.address=:443" + - "--entrypoints.websecure.transport.respondingTimeouts.readTimeout=0" - "--entryPoints.websecure.http.middlewares=error-pages@docker" - "--entryPoints.websecure.http.tls=true" - "--entryPoints.ssh.address=:2221" diff --git a/traefik/k8s/traefik-values.yaml b/traefik/k8s/traefik-values.yaml index 9fc2e63..566e8a8 100644 --- a/traefik/k8s/traefik-values.yaml +++ b/traefik/k8s/traefik-values.yaml @@ -86,6 +86,12 @@ ports: protocol: UDP expose: default: true + netbird-stun: + port: 3478 + exposedPort: 3478 + protocol: UDP + expose: + default: true checkmk-agent: port: 8000 protocol: TCP