SILVER
/
NODEDC_PLATFORM
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
NODEDC_PLATFORM/infra/deploy-runner/README.md

9.2 KiB
Raw Blame History

NODE.DC deploy runner

This directory stores the versioned source for the Synology canonical deploy runner.

Live runner:

/usr/local/sbin/nodedc-deploy

Synology staging candidate:

/volume1/docker/nodedc-deploy/runner-install/nodedc-deploy

The runner accepts data-only app-overlay artifacts from:

/volume1/docker/nodedc-deploy/inbox

Supported components in this source:

  • engine
  • launcher
  • platform
  • tasker
  • ops-agents
  • bim-viewer
  • n8n-private-extension
  • module-foundry
  • proxy-contur
  • dc-amd-proxy

n8n-private-extension is a staging-only trust boundary for reviewed offline n8n private-node releases. Its artifact may contain exactly one digest-bound n8n-nodes-ndc release with package.tgz, release.json and rollback.json. The runner validates the inner npm tarball, rejects lifecycle scripts and runtime dependencies, refuses to overwrite an existing release, and seals the installed release root-owned/read-only under:

/volume1/docker/nodedc-platform/n8n-private-extensions/releases/n8n-nodes-ndc/<version>-<sha256-prefix>

This component has no Compose file, service, container mutation or activation side effect. In particular, staging does not make the node visible to n8n. Activation remains an Engine-owned change: mount the reviewed immutable release at /home/node/.n8n/nodes/node_modules/n8n-nodes-ndc, atomically switch between verified releases, restart every n8n process, and accept only after MCP exposes the package-qualified n8n-nodes-ndc.* schemas. The Platform runner cannot cross that boundary and never runs npm install in a live container.

Build a verified offline release artifact:

node infra/deploy-runner/build-n8n-private-extension-artifact.mjs \
  n8n-nodes-ndc-release-YYYYMMDD-NNN

The builder is byte-reproducible and accepts exactly the three reviewed NDC runtime types:

  • n8n-nodes-ndc.ndcDataProductPublish
  • n8n-nodes-ndc.ndcDataProductRead
  • n8n-nodes-ndc.ndcFoundryBinding

Their three opaque capability credential schemas are ndcDataProductWriterApi, ndcDataProductReaderApi and ndcFoundryBindingApi. A node description containing usableAsTool is rejected because n8n 2.3.2 would synthesize an additional *Tool runtime type and violate the exact-three activation contract. Run the positive and negative release-policy suite before publishing an artifact:

PYTHONDONTWRITEBYTECODE=1 \
  python3 infra/deploy-runner/test_n8n_private_extension.py

Release/rollback manifests use schema v2. Before a first activation, the Engine-owned activator must verify and record the current inactive state. That verified_inactive state is an allowed rollback baseline when no previous verified immutable release exists; later upgrades prefer the previous verified release. Rollback never deletes or mutates a staged release.

The historical 0.1.0 release remains immutable and must not be overwritten. Release 0.1.1-994756958861518e is retained as rejected/inactive: its three node descriptions used usableAsTool, so n8n 2.3.2 exposed six NDC runtime types instead of the required three. It must not be activated, overwritten or deleted.

The corrected candidate is package version 0.1.2, built with patch id n8n-nodes-ndc-release-20260716-003. It receives a new digest-bound release directory and remains inert after staging; only a separately reviewed Engine-owned activator may select it after exact MCP schema acceptance.

The paired Engine activation is built by build-engine-n8n-private-extension-artifact.mjs. It deliberately does not copy the dirty Engine docker-compose.yml and does not build an image. Instead it installs a narrowly scoped Compose override plus a strict transition descriptor. On apply, the runner validates the staged release again, verifies that the running n8n container and the NAS-local 2.3.2 tag resolve to the same immutable image ID, and extracts the package into the root-owned, read-only Engine release tree:

/volume2/nodedc-demo/n8n-private-extensions/releases/n8n-nodes-ndc/0.1.2-05e4b38b14b4a019/package

The override sets N8N_USER_FOLDER=/home/node, which is required because the actual Engine service runs as root while the canonical community package path is below /home/node/.n8n. It enables loading but disables reinstall, mounts only the exact release read-only, and uses both Compose pull_policy: never and docker compose up --pull never. No registry access, lifecycle script, database installed_packages row or custom-extension loader is involved.

The runner pins the exact Engine service topology observed in source and rejects an added worker/webhook generation. Only the single actual n8n service is force-recreated with --no-deps; the Postgres service, .n8n data, encryption key and credentials remain intact. The apply gate verifies readiness, the running image/version, sealed mount, loader environment, package-loader node/credential sets, scoped loader logs, restart stability and content-exact pinned Engine MCP catalogs. The runner pins both the complete 434/385 inactive baseline and the reviewed 437/388 activation catalogs, so a same-count substitution of any built-in schema is rejected. Any gate failure after mutation automatically restores the pre-apply catalogs/descriptor and force-recreates the previous verified runtime. Staged, sealed and failed releases are retained. The separate rollback artifact returns the first activation to the verified inactive 434-node/385-credential catalog baseline.

Run both policy suites before publishing the Engine pair:

PYTHONDONTWRITEBYTECODE=1 \
  python3 infra/deploy-runner/test_n8n_private_extension.py
PYTHONDONTWRITEBYTECODE=1 \
  python3 infra/deploy-runner/test_engine_n8n_private_extension.py

For platform artifacts, the allowlist includes the versioned Ontology Core, the legacy Gelios experiment and the provider-neutral External Data Plane sources. An External Data Plane artifact builds only its image and starts external-data-plane-postgres plus external-data-plane; it never contains a provider credential, provider endpoint, collection schedule or command transport. Database credentials remain root-owned live .env.synology configuration and must not reuse NODEDC_INTERNAL_ACCESS_TOKEN.

The External Data Plane writer-provisioner credential is different: on the first relevant Platform apply, the root-owned runner creates /volume1/docker/nodedc-platform/secrets/external-data-plane-provisioner/token atomically in a dedicated UID/GID 11006 directory (directory 0500, token 0400). It is never an .env value or an artifact member; Compose mounts it read-only only into External Data Plane and, when implemented, its dedicated Engine provisioner running under the same restricted identity.

module-foundry is an independent, authenticated application component. Its artifact contains source and compose infrastructure only; its live /volume1/docker/nodedc-platform/module-foundry/source/.env is root-owned and never enters an artifact. The component reuses the existing internal platform credential for Launcher handoff validation and requires that runtime configuration before its first apply.

The Foundry ↔ Map Gateway signing key is not an application or .env setting. On the first relevant platform or module-foundry apply, the root-owned runner creates /volume1/docker/nodedc-platform/secrets/map-gateway-admin-secret atomically (root:gid 1000, mode 0640). Both containers receive that file only as a read-only mount. The value is never printed, backed up with source, included in an artifact, or administered through Foundry.

proxy-contur is the canonical VPN egress for selected Map Gateway provider hosts. Its existing root-owned PROXY_TOKEN is copied by the runner into /volume1/docker/nodedc-platform/secrets/map-egress-proxy-token with root:gid 1000, mode 0640, then mounted read-only only into Map Gateway. The value is neither printed nor contained in an artifact, Foundry setting, or browser response. Apply the proxy-contur artifact before the Platform Map Gateway artifact: it creates the private nodedc-map-egress Docker network.

dc-amd-proxy is the separate, staged connector for the neighbouring AMD VPN machine. Its active artifact attaches only to the private nodedc-map-egress network, exposes a narrow NAS-LAN pairing port, and has no direct provider egress. The runner preserves a 0700, service-user-owned runtime directory for the one-time paired connector credential and synchronizes the existing private Map Gateway egress credential as a read-only file. Neither value is ever placed in an artifact, .env, browser response, or runner output. The separate Platform switch is applied only after the connector and pairing are verified; it does not alter NAS routes, VPN, DNS, or Tailscale.

Install or update the root-owned live runner on Synology:

sudo install -o root -g root -m 0755 \
  /volume1/docker/nodedc-deploy/runner-install/nodedc-deploy \
  /usr/local/sbin/nodedc-deploy
sudo /usr/local/sbin/nodedc-deploy verify-install

Normal service deploys must still use explicit artifacts:

sudo /usr/local/sbin/nodedc-deploy plan /volume1/docker/nodedc-deploy/inbox/<artifact>.tgz
sudo /usr/local/sbin/nodedc-deploy apply /volume1/docker/nodedc-deploy/inbox/<artifact>.tgz