diff --git a/docs/CURRENT_INFRA_HANDOFF.md b/docs/CURRENT_INFRA_HANDOFF.md index 368b95e..02df799 100644 --- a/docs/CURRENT_INFRA_HANDOFF.md +++ b/docs/CURRENT_INFRA_HANDOFF.md @@ -210,10 +210,10 @@ https://ops-agents.nodedc.ru Synology reverse proxy should route: ```text -HTTPS ops-agents.nodedc.ru:443 -> HTTP 127.0.0.1:18190 +HTTPS ops-agents.nodedc.ru:443 -> HTTP 172.22.0.222:18190 ``` -Do not use host port `18090` for this module: `18090` is the Tasker proxy port. Gateway container port remains `4100`; host port is controlled by `HOST_PORT=18190`. +Do not use host port `18090` for this module: `18090` is the Tasker proxy port. Gateway container port remains `4100`; host bind/port are controlled by `HOST_BIND=172.22.0.222` and `HOST_PORT=18190`. Main services: @@ -246,10 +246,23 @@ From macOS with `/Volumes/docker` mounted: cd /Users/dcconstructions/Downloads/mnt/NODEDC/platform NAS_ROOT=/Volumes/docker/nodedc-platform \ LAUNCHER_REPO=/Users/dcconstructions/Downloads/mnt/data/nodedc_launcher \ +TASKER_REPO=/Users/dcconstructions/Downloads/mnt/data/dc_taskmanager/NODEDC_TASKMANAGER \ +TASKER_CHANGED_BASE=533f8c6 \ +GATEWAY_REPO=/Users/dcconstructions/Downloads/mnt/data/NODEDC_TASKMANAGER_CODEXAPI \ ./infra/synology/deploy-current.sh ``` -The script syncs platform deploy files and prints the Docker commands that must be run on Synology, because Docker there requires interactive `sudo`. +The script syncs platform deploy files plus optional Launcher, Tasker, and Ops Agents Gateway source copies. It prints the Docker commands that must be run on Synology, because Docker there requires interactive `sudo`. + +Tasker full source sync over SMB is intentionally not the default. Use `TASKER_CHANGED_BASE=` for normal deploys, or `TASKER_SYNC_SOURCE=1` only when a full source refresh is required. + +The script does not overwrite live secret files: + +```text +/volume1/docker/nodedc-platform/platform/.env.synology +/volume1/docker/nodedc-platform/tasker/plane-app/.env.synology +/volume1/docker/nodedc-platform/ops-agents/.env +``` Launcher image build is done from the Launcher repo when frontend/backend code changes: @@ -327,7 +340,7 @@ Replace `web` with `admin`, `api`, or another service only when that service ima Tasker backend must receive the Gateway internal URL/token before Codex/Ops Agent UI can create agents in production: ```env -PLANE_NODEDC_AGENT_GATEWAY_URL=http://127.0.0.1:18190 +PLANE_NODEDC_AGENT_GATEWAY_URL=http://172.22.0.222:18190 PLANE_NODEDC_AGENT_GATEWAY_TOKEN= ``` @@ -351,8 +364,8 @@ Required Synology `.env` values are documented in: Minimum checks: ```bash -curl -fsS http://127.0.0.1:18190/healthz -curl -fsS http://127.0.0.1:18190/readyz +curl -fsS http://172.22.0.222:18190/healthz +curl -fsS http://172.22.0.222:18190/readyz curl -fsS -i https://ops-agents.nodedc.ru/mcp | head ``` diff --git a/infra/synology/README.md b/infra/synology/README.md index 7da5a0a..b36be70 100644 --- a/infra/synology/README.md +++ b/infra/synology/README.md @@ -47,7 +47,7 @@ http://task.nas.nodedc:18090 - `deploy-current.sh` синхронизирует compose, Caddyfile, Authentik templates и опционально Launcher source в NAS mount. - `backup-current.sh` делает snapshot Launcher runtime/uploads/Auth templates/config и готовит команду `pg_dump` для Authentik Postgres. - Tasker поднимается отдельным compose из `NODEDC_TASKMANAGER/plane-app/docker-compose.yaml` на порту `18090`. -- Ops Agents Gateway поднимается отдельным compose из `NODEDC_TASKMANAGER_CODEXAPI/docker-compose.synology.yml` на `127.0.0.1:18190`; Synology reverse proxy должен вести `ops-agents.nodedc.ru` на этот порт, а не на `18090`. +- Ops Agents Gateway поднимается отдельным compose из `NODEDC_TASKMANAGER_CODEXAPI/docker-compose.synology.yml` на `172.22.0.222:18190`; Synology reverse proxy должен вести `ops-agents.nodedc.ru` на этот порт, а не на `18090`. ## Синхронизация текущего состояния @@ -57,11 +57,33 @@ http://task.nas.nodedc:18090 cd /Users/dcconstructions/Downloads/mnt/NODEDC/platform NAS_ROOT=/Volumes/docker/nodedc-platform \ LAUNCHER_REPO=/Users/dcconstructions/Downloads/mnt/data/nodedc_launcher \ +TASKER_REPO=/Users/dcconstructions/Downloads/mnt/data/dc_taskmanager/NODEDC_TASKMANAGER \ +TASKER_CHANGED_BASE=533f8c6 \ +GATEWAY_REPO=/Users/dcconstructions/Downloads/mnt/data/NODEDC_TASKMANAGER_CODEXAPI \ ./infra/synology/deploy-current.sh ``` Скрипт не запускает Docker сам: на NAS `sudo` интерактивный, поэтому команды применения печатаются в конце. +Что синхронизируется: + +- Platform compose/Caddy/Auth templates. +- Launcher source в `/volume1/docker/nodedc-platform/launcher/source`. +- Tasker `plane-app/docker-compose.yaml` и, если задан `TASKER_CHANGED_BASE`, только изменённые source-файлы из диапазона `TASKER_CHANGED_BASE..HEAD`. +- Ops Agents Gateway source в `/volume1/docker/nodedc-platform/ops-agents`. + +Полный sync Tasker source по SMB тяжёлый для Plane fork. Использовать его только осознанно: + +```bash +TASKER_SYNC_SOURCE=1 ./infra/synology/deploy-current.sh +``` + +Секретные runtime env-файлы не перетираются: + +- `/volume1/docker/nodedc-platform/platform/.env.synology` +- `/volume1/docker/nodedc-platform/tasker/plane-app/.env.synology` +- `/volume1/docker/nodedc-platform/ops-agents/.env` + ## Backup текущего состояния С Mac, при смонтированном `/Volumes/docker`: @@ -79,6 +101,18 @@ NAS_ROOT=/Volumes/docker/nodedc-platform ./infra/synology/backup-current.sh bash /volume1/docker/nodedc-platform/backups/platform-current-YYYYMMDD-HHMMSS/run-authentik-db-dump-on-synology.sh ``` +Если планируются изменения Tasker backend/schema, дополнительно выполнить: + +```bash +bash /volume1/docker/nodedc-platform/backups/platform-current-YYYYMMDD-HHMMSS/run-tasker-db-dump-on-synology.sh +``` + +Если Ops Agents Gateway уже был запущен и там есть production tokens/grants/audit, дополнительно выполнить: + +```bash +bash /volume1/docker/nodedc-platform/backups/platform-current-YYYYMMDD-HHMMSS/run-ops-agents-db-dump-on-synology.sh +``` + ## Что нужно перед запуском - Собрать или загрузить `linux/amd64` images: diff --git a/infra/synology/backup-current.sh b/infra/synology/backup-current.sh index be09d8b..72c0440 100755 --- a/infra/synology/backup-current.sh +++ b/infra/synology/backup-current.sh @@ -3,6 +3,8 @@ set -euo pipefail NAS_ROOT="${NAS_ROOT:-/Volumes/docker/nodedc-platform}" NAS_PLATFORM_DIR="${NAS_PLATFORM_DIR:-/volume1/docker/nodedc-platform/platform}" +NAS_TASKER_DIR="${NAS_TASKER_DIR:-/volume1/docker/nodedc-platform/tasker/plane-app}" +NAS_GATEWAY_DIR="${NAS_GATEWAY_DIR:-/volume1/docker/nodedc-platform/ops-agents}" BACKUP_ROOT="${BACKUP_ROOT:-${NAS_ROOT}/backups}" TIMESTAMP="${TIMESTAMP:-$(date +%Y%m%d-%H%M%S)}" BACKUP_DIR="${BACKUP_DIR:-${BACKUP_ROOT}/platform-current-${TIMESTAMP}}" @@ -12,7 +14,9 @@ ENV_FILE="${ENV_FILE:-${NAS_PLATFORM_DIR}/.env.synology}" mkdir -p "${BACKUP_DIR}/files/platform" \ "${BACKUP_DIR}/files/launcher" \ - "${BACKUP_DIR}/files/authentik" + "${BACKUP_DIR}/files/authentik" \ + "${BACKUP_DIR}/files/tasker" \ + "${BACKUP_DIR}/files/ops-agents" rsync_dir() { local source="$1" @@ -46,6 +50,14 @@ rsync_file "${NAS_ROOT}/platform/.env.synology.example" "${BACKUP_DIR}/files/pla rsync_file "${NAS_ROOT}/platform/docker-compose.platform-http.yml" "${BACKUP_DIR}/files/platform/" rsync_file "${NAS_ROOT}/platform/Caddyfile.http" "${BACKUP_DIR}/files/platform/" +rsync_file "${NAS_ROOT}/tasker/plane-app/.env.synology" "${BACKUP_DIR}/files/tasker/" +rsync_file "${NAS_ROOT}/tasker/plane-app/docker-compose.yaml" "${BACKUP_DIR}/files/tasker/" +rsync_file "${NAS_ROOT}/tasker/plane-app/docker-compose.synology.override.yml" "${BACKUP_DIR}/files/tasker/" + +rsync_file "${NAS_ROOT}/ops-agents/.env" "${BACKUP_DIR}/files/ops-agents/" +rsync_file "${NAS_ROOT}/ops-agents/.env.synology.example" "${BACKUP_DIR}/files/ops-agents/" +rsync_file "${NAS_ROOT}/ops-agents/docker-compose.synology.yml" "${BACKUP_DIR}/files/ops-agents/" + cat > "${BACKUP_DIR}/manifest.txt" < "${BACKUP_DIR}/run-tasker-db-dump-on-synology.sh" < "\${BACKUP_DIR}/tasker-postgres.dump" + +sudo "${DOCKER_BIN}" "\${compose_args[@]}" \\ + exec -T plane-db \\ + sh -lc 'pg_restore --list /dev/stdin >/dev/null' \\ + < "\${BACKUP_DIR}/tasker-postgres.dump" + +echo "tasker-db-dump-ok: \${BACKUP_DIR}/tasker-postgres.dump" +EOF +chmod +x "${BACKUP_DIR}/run-tasker-db-dump-on-synology.sh" + +cat > "${BACKUP_DIR}/run-ops-agents-db-dump-on-synology.sh" < "\${BACKUP_DIR}/ops-agents-postgres.dump" + +sudo "${DOCKER_BIN}" compose \\ + --env-file .env \\ + -f docker-compose.synology.yml \\ + exec -T postgres \\ + sh -lc 'pg_restore --list /dev/stdin >/dev/null' \\ + < "\${BACKUP_DIR}/ops-agents-postgres.dump" + +echo "ops-agents-db-dump-ok: \${BACKUP_DIR}/ops-agents-postgres.dump" +EOF +chmod +x "${BACKUP_DIR}/run-ops-agents-db-dump-on-synology.sh" + find "${BACKUP_DIR}" -name @eaDir -prune -o -type d -exec chmod 700 {} \; find "${BACKUP_DIR}" -name @eaDir -prune -o -type f -exec chmod 600 {} \; chmod 700 "${BACKUP_DIR}/run-authentik-db-dump-on-synology.sh" +chmod 700 "${BACKUP_DIR}/run-tasker-db-dump-on-synology.sh" +chmod 700 "${BACKUP_DIR}/run-ops-agents-db-dump-on-synology.sh" if [[ -x "${DOCKER_BIN}" && "${NAS_ROOT}" == /volume1/* ]]; then "${BACKUP_DIR}/run-authentik-db-dump-on-synology.sh" @@ -108,5 +177,11 @@ DB dump was not run from this host. Run on Synology: bash /volume1/docker/nodedc-platform/backups/$(basename "${BACKUP_DIR}")/run-authentik-db-dump-on-synology.sh + +# If Tasker database exists and backend/schema changes are planned: +bash /volume1/docker/nodedc-platform/backups/$(basename "${BACKUP_DIR}")/run-tasker-db-dump-on-synology.sh + +# If Ops Agents Gateway was already deployed: +bash /volume1/docker/nodedc-platform/backups/$(basename "${BACKUP_DIR}")/run-ops-agents-db-dump-on-synology.sh EOF fi diff --git a/infra/synology/deploy-current.sh b/infra/synology/deploy-current.sh index 8a04087..601fec9 100755 --- a/infra/synology/deploy-current.sh +++ b/infra/synology/deploy-current.sh @@ -5,6 +5,10 @@ SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)" PLATFORM_REPO="$(cd -- "${SCRIPT_DIR}/../.." && pwd)" NAS_ROOT="${NAS_ROOT:-/Volumes/docker/nodedc-platform}" LAUNCHER_REPO="${LAUNCHER_REPO:-}" +TASKER_REPO="${TASKER_REPO:-}" +TASKER_SYNC_SOURCE="${TASKER_SYNC_SOURCE:-0}" +TASKER_CHANGED_BASE="${TASKER_CHANGED_BASE:-}" +GATEWAY_REPO="${GATEWAY_REPO:-}" if [[ ! -d "${NAS_ROOT}" ]]; then echo "NAS_ROOT not found: ${NAS_ROOT}" >&2 @@ -50,6 +54,102 @@ else echo "LAUNCHER_REPO is not set; launcher source was not synced." fi +if [[ -n "${TASKER_REPO}" ]]; then + if [[ ! -d "${TASKER_REPO}/plane-src" || ! -d "${TASKER_REPO}/plane-app" ]]; then + echo "TASKER_REPO must point to NODEDC_TASKMANAGER with plane-src and plane-app: ${TASKER_REPO}" >&2 + exit 1 + fi + + mkdir -p "${NAS_ROOT}/tasker/plane-src" "${NAS_ROOT}/tasker/plane-app" + + rsync -av \ + "${TASKER_REPO}/plane-app/docker-compose.yaml" \ + "${NAS_ROOT}/tasker/plane-app/docker-compose.yaml" + + if [[ -n "${TASKER_CHANGED_BASE}" ]]; then + echo "Syncing Tasker changed files from ${TASKER_CHANGED_BASE}..HEAD" + git -C "${TASKER_REPO}" diff --name-only "${TASKER_CHANGED_BASE}..HEAD" -- plane-src plane-app/docker-compose.yaml | + while IFS= read -r changed_file; do + [[ -z "${changed_file}" ]] && continue + + case "${changed_file}" in + plane-src/*) + source_path="${TASKER_REPO}/${changed_file}" + target_path="${NAS_ROOT}/tasker/plane-src/${changed_file#plane-src/}" + ;; + plane-app/docker-compose.yaml) + source_path="${TASKER_REPO}/${changed_file}" + target_path="${NAS_ROOT}/tasker/plane-app/docker-compose.yaml" + ;; + *) + continue + ;; + esac + + if [[ -f "${source_path}" ]]; then + mkdir -p "$(dirname -- "${target_path}")" + rsync -av "${source_path}" "${target_path}" + else + echo "skip deleted Tasker file in changed sync: ${changed_file}" + fi + done + elif [[ "${TASKER_SYNC_SOURCE}" == "1" ]]; then + echo "TASKER_SYNC_SOURCE=1: syncing full Tasker source without delete" + rsync -av \ + --exclude='.git/' \ + --exclude='node_modules/' \ + --exclude='.pnpm-store/' \ + --exclude='.turbo/' \ + --exclude='.cache/' \ + --exclude='.react-router/' \ + --exclude='build/' \ + --exclude='dist/' \ + --exclude='coverage/' \ + --exclude='playwright-report/' \ + --exclude='test-results/' \ + --exclude='__pycache__/' \ + --exclude='@eaDir/' \ + --exclude='.DS_Store' \ + --exclude='.env' \ + --exclude='.env.*' \ + --exclude='*.pyc' \ + "${TASKER_REPO}/plane-src/" \ + "${NAS_ROOT}/tasker/plane-src/" + else + echo "TASKER_REPO set; synced compose only. Set TASKER_CHANGED_BASE= or TASKER_SYNC_SOURCE=1 to sync source." + fi + + if [[ -f "${TASKER_REPO}/plane-app/docker-compose.synology.override.yml" ]]; then + rsync -av \ + "${TASKER_REPO}/plane-app/docker-compose.synology.override.yml" \ + "${NAS_ROOT}/tasker/plane-app/docker-compose.synology.override.yml" + else + echo "TASKER_REPO has no plane-app/docker-compose.synology.override.yml; existing NAS override, if any, was preserved." + fi +else + echo "TASKER_REPO is not set; tasker source and compose were not synced." +fi + +if [[ -n "${GATEWAY_REPO}" ]]; then + if [[ ! -f "${GATEWAY_REPO}/docker-compose.synology.yml" || ! -f "${GATEWAY_REPO}/.env.synology.example" ]]; then + echo "GATEWAY_REPO must point to NODEDC_TASKMANAGER_CODEXAPI with docker-compose.synology.yml: ${GATEWAY_REPO}" >&2 + exit 1 + fi + + mkdir -p "${NAS_ROOT}/ops-agents" + rsync -av --delete \ + --exclude='.git/' \ + --exclude='node_modules/' \ + --exclude='dist/' \ + --include='.env.synology.example' \ + --exclude='.env' \ + --exclude='.env.*' \ + "${GATEWAY_REPO}/" \ + "${NAS_ROOT}/ops-agents/" +else + echo "GATEWAY_REPO is not set; Ops Agents Gateway source was not synced." +fi + cat <<'EOF' Synced files to NAS mount. Run on Synology to apply runtime changes: @@ -75,4 +175,29 @@ sudo /usr/local/bin/docker exec nodedc-platform-launcher-1 sh -lc \ curl -k -sS --compressed https://id.nodedc.ru/if/flow/default-authentication-flow/ \ | grep -aE 'hub.nodedc.ru|launcher.local|getLauncherBaseUrl|Запросить доступ' + +Optional Tasker apply after TASKER_REPO sync: + +cd /volume1/docker/nodedc-platform/tasker/plane-src +BUILD_BACKEND=1 BUILD_WEB=1 BUILD_ADMIN=0 sh rebuild-nas-legacy.sh + +Optional Ops Agents Gateway first deploy after GATEWAY_REPO sync: + +cd /volume1/docker/nodedc-platform/ops-agents +cp -n .env.synology.example .env +# Edit .env before first start: +# - NODEDC_AGENT_GATEWAY_INTERNAL_TOKEN +# - NODEDC_INTERNAL_ACCESS_TOKEN +# - POSTGRES_PASSWORD + +sudo /usr/local/bin/docker compose \ + --env-file .env \ + -f docker-compose.synology.yml \ + up -d --build + +Verify Ops Agents Gateway: + +curl -fsS http://172.22.0.222:18190/healthz +curl -fsS http://172.22.0.222:18190/readyz +curl -fsS -i http://172.22.0.222:18190/mcp | head EOF