NODEDC_PLATFORM/infra/deploy-runner/README.md

# NODE.DC deploy runner

This directory stores the versioned source for the Synology canonical deploy runner.

Live runner:

```text
/usr/local/sbin/nodedc-deploy
```

Synology staging candidate:

```text
/volume1/docker/nodedc-deploy/runner-install/nodedc-deploy
```

The runner accepts data-only app-overlay artifacts from:

```text
/volume1/docker/nodedc-deploy/inbox
```

Supported components in this source:

- `engine`
- `launcher`
- `platform`
- `tasker`
- `ops-agents`
- `bim-viewer`
- `n8n-private-extension`
- `module-foundry`
- `proxy-contur`
- `dc-amd-proxy`

`n8n-private-extension` is a staging-only trust boundary for reviewed offline
n8n private-node releases. Its artifact may contain exactly one digest-bound
`n8n-nodes-ndc` release with `package.tgz`, `release.json` and
`rollback.json`. The runner validates the inner npm tarball, rejects lifecycle
scripts and runtime dependencies, refuses to overwrite an existing release,
and seals the installed release root-owned/read-only under:

```text
/volume1/docker/nodedc-platform/n8n-private-extensions/releases/n8n-nodes-ndc/<version>-<sha256-prefix>
```

This component has no Compose file, service, container mutation or activation
side effect. In particular, staging does **not** make the node visible to n8n.
Activation remains an Engine-owned change: mount the reviewed immutable release
at `/home/node/.n8n/nodes/node_modules/n8n-nodes-ndc`, atomically switch between
verified releases, restart every n8n process, and accept only after MCP exposes
the package-qualified `n8n-nodes-ndc.*` schemas. The Platform runner cannot
cross that boundary and never runs `npm install` in a live container.

Build a verified offline release artifact:

```bash
node infra/deploy-runner/build-n8n-private-extension-artifact.mjs \
  n8n-nodes-ndc-release-YYYYMMDD-NNN
```

The builder is byte-reproducible and accepts exactly the three reviewed NDC
runtime types:

- `n8n-nodes-ndc.ndcDataProductPublish`
- `n8n-nodes-ndc.ndcDataProductRead`
- `n8n-nodes-ndc.ndcFoundryBinding`

Their three opaque capability credential schemas are
`ndcDataProductWriterApi`, `ndcDataProductReaderApi` and
`ndcFoundryBindingApi`. A node description containing `usableAsTool` is
rejected because n8n 2.3.2 would synthesize an additional `*Tool` runtime type
and violate the exact-three activation contract. Run the positive and negative
release-policy suite before publishing an artifact:

```bash
PYTHONDONTWRITEBYTECODE=1 \
  python3 infra/deploy-runner/test_n8n_private_extension.py
```

Release/rollback manifests use schema v2. Before a first activation, the
Engine-owned activator must verify and record the current inactive state. That
`verified_inactive` state is an allowed rollback baseline when no previous
verified immutable release exists; later upgrades prefer the previous verified
release. Rollback never deletes or mutates a staged release.

The historical `0.1.0` release remains immutable and must not be overwritten.
Release `0.1.1-994756958861518e` is retained as rejected/inactive: its three
node descriptions used `usableAsTool`, so n8n 2.3.2 exposed six NDC runtime
types instead of the required three. It must not be activated, overwritten or
deleted.

The corrected candidate is package version `0.1.2`, built with patch id
`n8n-nodes-ndc-release-20260716-003`. It receives a new digest-bound release
directory and remains inert after staging; only a separately reviewed
Engine-owned activator may select it after exact MCP schema acceptance.

The paired Engine activation is built by
`build-engine-n8n-private-extension-artifact.mjs`. It deliberately does not
copy from or write generated files into the dirty Engine worktree, and it does
not build an image. A fresh transition id is mandatory and previously issued
ids are rejected:

```bash
node infra/deploy-runner/build-engine-n8n-private-extension-artifact.mjs 20260717-004
```

The builder emits a narrowly scoped Compose override plus a strict transition
descriptor. On apply, the runner validates the staged release again, verifies
that the running n8n container and the NAS-local `2.3.2` tag resolve to the
same immutable image ID, and extracts the package into the root-owned,
read-only Engine release tree:

```text
/volume2/nodedc-demo/n8n-private-extensions/releases/n8n-nodes-ndc/0.1.2-05e4b38b14b4a019/package
```

The override sets `N8N_USER_FOLDER=/home/node`, which is required because the
actual Engine service runs as root while the canonical community package path
is below `/home/node/.n8n`. The live runtime contract remains the successfully
deployed generation-003 contract and does not set `NODE_PATH`. Only the
runner's isolated `node -e` package-loader probe temporarily initializes the
dependency tree bundled inside the exact n8n base image; this reproduces n8n's
own loader without redefining the live Compose state. The override enables
loading but disables reinstall, mounts only the exact release read-only, and
uses both Compose `pull_policy: never` and `docker compose up --pull never`.
No registry access, lifecycle script, database `installed_packages` row or
custom-extension loader is involved.

The runner pins the exact Engine service topology observed in source and
rejects an added worker/webhook generation. Only the single actual `n8n`
service is force-recreated with `--no-deps`; the
Postgres service, `.n8n` data, encryption key and credentials remain intact.
The apply gate verifies readiness, the running image/version, sealed mount,
loader environment, package-loader node/credential sets, scoped loader logs,
restart stability and content-exact pinned Engine MCP catalogs. The runner pins
both the complete 434/385 inactive baseline and the reviewed 437/388
activation catalogs, so a same-count substitution of any built-in schema is
rejected. Any gate failure after
mutation automatically restores the pre-apply catalogs/descriptor and
force-recreates the previous verified runtime. Staged, sealed and failed
releases are retained. The separate rollback artifact returns the first
activation to the verified inactive 434-node/385-credential catalog baseline.

Run both policy suites before publishing the Engine pair:

```bash
PYTHONDONTWRITEBYTECODE=1 \
  python3 infra/deploy-runner/test_n8n_private_extension.py
PYTHONDONTWRITEBYTECODE=1 \
  python3 infra/deploy-runner/test_engine_n8n_private_extension.py
```

For `platform` artifacts, the allowlist includes the versioned Ontology Core,
the frozen legacy Gelios compatibility service, and the provider-neutral
External Data Plane sources. The Gelios service remains reproducible only to
protect its existing database/workflow; it is not a template for a provider
integration. An External Data Plane artifact builds only its image and
force-recreates only `external-data-plane`; the already healthy
`external-data-plane-postgres` container and its Timescale volume are an
independent deploy prerequisite and are never selected by an EDP application
artifact. The reviewed Compose source pins the Timescale image, named volume
and target, internal database-only network, absence of database host ports,
healthy dependency, localhost-only EDP bind and the two read-only
provisioner/trust mounts in the reviewed Compose source. The runner does not
reinterpret version-dependent `docker compose config` JSON as a second deploy
schema. Its canonical enforcement remains the artifact/path allowlist plus
hard-coded build command, selected service set, runtime-secret preparation and
health acceptance. Post-apply acceptance also requires
`database=ready`; a first-rollout failure removes only the candidate EDP
container without volumes and restores the source overlay. It never contains a
provider credential, provider endpoint,
collection schedule or command
transport. Its contract payload is an exact provider-neutral runtime subset;
`providers/*`, mappings, fixtures and tests are excluded and do not trigger an
EDP rebuild. Database credentials remain root-owned live `.env.synology`
configuration and must not reuse `NODEDC_INTERNAL_ACCESS_TOKEN`.

On the first relevant Platform apply, the root-owned runner creates
`/volume1/docker/nodedc-platform/secrets/external-data-plane-provisioner/token`
atomically in a dedicated UID/GID `11006` directory (directory `0500`, token
`0400`). It is never an `.env` value or an artifact member and is mounted
read-only only into External Data Plane.

Manual one-time binding issuance and digest-only managed writer ensure are
independently disabled by default. The target control-plane operation generates
and stores the capability inside native NDC L2 Credentials and sends only its
digest to EDP; users and MCP consumers receive only an opaque compatible
reference/status. The runner-owned bearer above authenticates only legacy
plaintext issuance and is intentionally rejected by managed ensure/revoke.
Managed requests use a deployment Ed25519 Engine service key; EDP mounts only
the public-key trust directory read-only. On every reviewed Engine apply and on
an EDP runtime apply, this runner creates or validates one matching Ed25519 pair:
the Engine-only private key is `root:root 0400`, while the EDP trust copy is
`root:11006 0440`. Public-only crash state, key mismatch, a non-Ed25519 key,
symlinks and permissive modes fail closed. `plan` discloses both paths without
printing key material. The private key must not be broadened into an L2 graph,
MCP surface, artifact or shared-token boundary.

The reviewed Engine source candidate has dedicated server-derived MCP
plan/apply handling for the exact `ndcDataProductWriterApi` + Data Product
Publish tuple. It is separate from the generic HTTP safe-ref path and accepts no
caller-provided provider/scope/credential identity, capability, generation or
service URL. The production managed flag remains false during staging. In the
explicitly confirmed activation window it is set to true immediately before
the Platform EDP artifact; runner acceptance then requires EDP `/healthz` to
report managed provisioning `enabled`. At that point the old Engine still has
no signer mount, so the endpoint remains usable only after the separately
accepted Engine artifact. Root/UI transfer is emergency
self-hosted diagnostics only, not the user journey or acceptance
path. A provider-specific daemon remains forbidden.

Build the narrow Engine source artifact with:

```bash
node infra/deploy-runner/build-engine-data-product-publish-grant-artifact.mjs \
  engine-data-product-publish-grant-YYYYMMDD-NNN
```

The builder includes exactly the pinned provider-security catalog, the
Publish-grant service, Engine Agent scope/gateway wiring, the existing n8n
adapter and the reviewed additive backend runtime overlay. It deliberately
excludes base Compose, frontend/dist, runtime data, tests, native credentials,
the NDC L2 process and the legacy generic credential-sink route/core. The
runner fixes the runtime action to `nodedc-backend` with `--no-deps` and
`--pull never`; n8n, nginx app, database services and volumes are not selected.
Acceptance requires backend health, the active immutable backend identity and
the exact additive mount inventory. Any failure restores the touched source and
recreates only the previous verified backend runtime.

Existing Engine Agents created before Publish-grant support store the original
nine scopes as the complete developer profile. They must not require a new
agent, setup command or device credential when the profile gains a server-owned
capability. Build the compatibility artifact that introduces the durable named
`full-developer` profile with:

```bash
node infra/deploy-runner/build-engine-agent-full-grant-migration-artifact.mjs \
  engine-agent-full-grant-migration-YYYYMMDD-NNN
```

This follow-up slice contains exactly
`nodedc-source/server/engineAgents/store.js`. The runner pins its predecessor to
the successfully applied Publish generation, requires the installed Publish
overlay and active immutable backend, recreates only `nodedc-backend`, and
proves the exact candidate SHA, named profile and current expanded runtime
scope view. Store schema v1 is migrated atomically to v2 only when a grant
contains the complete legacy nine-scope developer bundle. From then on the
profile name is the authorization authority and its scope list is derived on
every read, so capabilities deliberately added to `full-developer` immediately
apply to existing full grants without token or store migrations. Partial grants
become `custom` and remain exact; they are never elevated. The artifact does not
touch UI, n8n, credentials, agent tokens, workflow graphs, databases or runtime
payloads.

## Engine L2 node-intelligence transition

The node-intelligence transition is an additive Engine-owned deployment domain.
It does not merge Ontology, provider APIs or Ops into the Engine MCP. It pins the
reviewed upstream `n8n-mcp` implementation at package `2.33.2`, commit
`974a9fb3492fe2c4984ee0549085d531cdc6242a`, and exposes only the safe NDC L2
projection through the existing Engine Agent gateway. Upstream management and
write tools are not forwarded.

Production never clones, pulls, installs or builds this dependency. The builder
saves the reviewed `linux/amd64` image once, embeds that exact archive in a
data-only activation artifact and records both the archive and image-config
digests. The runner validates every inner blob, revision label, entrypoint,
command and platform before an offline `docker image load`; Compose then uses
the fixed tag with `pull_policy: never` and `--pull never`.

Build a fresh activation/rollback pair:

```bash
node infra/deploy-runner/build-engine-node-intelligence-artifacts.mjs \
  YYYYMMDD-NNN
```

The activation exact set is:

- `nodedc-source/server/nodeIntelligence`
- `nodedc-source/server/routes/engineAgentGateway.js`
- `nodedc-source/services/node-intelligence`

The runtime adds only `nodedc-node-intelligence` and recreates the existing
`nodedc-backend`; n8n, UI, databases, L1, provider services and volumes are not
selected. The sidecar has no host port, runs as `11007:11007`, has a read-only
root filesystem, drops all capabilities and receives no Engine or provider
credential. Its independent MCP bearer is created by the root-owned runner as a
read-only file and mounted only into the sidecar and backend. It never appears
in an artifact, shared environment file, log or MCP response.

Acceptance requires the exact image/container/mount/security inventory,
immutable backend identity and live authenticated `get_node`, `validate_node`
and `validate_workflow` calls. Any apply failure restores source and runtime
automatically. The separate rollback artifact first removes only the sidecar
without volumes, restores the pinned inactive gateway, and recreates only the
verified backend. Loaded images and failed candidate source are retained for
audit rather than destructively deleted.

Stage the reviewed runner under a unique candidate name first:

```text
/volume1/docker/nodedc-deploy/runner-install/candidates/
  nodedc-deploy.engine-node-intelligence-YYYYMMDD-NNN
```

Do not overwrite the canonical staging candidate while another deploy may be in
flight. After the no-lock/no-process gate, promote the exact candidate in a
standalone root step, run `verify-install`, then invoke a fresh process for
activation `plan` and only then `apply`. The generated plan/apply runbook carries
the runner, activation and rollback SHA-256 values and is staged beside the
unique runner candidate.

`module-foundry` is an independent, authenticated application component. Its
artifact contains source and compose infrastructure only; its live
`/volume1/docker/nodedc-platform/module-foundry/source/.env` is root-owned and
never enters an artifact. The component reuses the existing internal platform
credential for Launcher handoff validation and requires that runtime
configuration before its first `apply`.

The Foundry ↔ Map Gateway signing key is not an application or `.env` setting.
On the first relevant `platform` or `module-foundry` apply, the root-owned
runner creates `/volume1/docker/nodedc-platform/secrets/map-gateway-admin-secret`
atomically (root:gid 1000, mode `0640`). Both containers receive that file only
as a read-only mount. The value is never printed, backed up with source,
included in an artifact, or administered through Foundry.

`proxy-contur` is the canonical VPN egress for selected Map Gateway provider
hosts. Its existing root-owned `PROXY_TOKEN` is copied by the runner into
`/volume1/docker/nodedc-platform/secrets/map-egress-proxy-token` with
`root:gid 1000`, mode `0640`, then mounted read-only only into Map Gateway.
The value is neither printed nor contained in an artifact, Foundry setting, or
browser response. Apply the `proxy-contur` artifact before the Platform Map
Gateway artifact: it creates the private `nodedc-map-egress` Docker network.

`dc-amd-proxy` is the separate, staged connector for the neighbouring AMD VPN
machine. Its active artifact attaches only to the private `nodedc-map-egress`
network, exposes a narrow NAS-LAN pairing port, and has no direct provider
egress. The runner preserves a `0700`, service-user-owned runtime directory
for the one-time paired connector credential and synchronizes the existing
private Map Gateway egress credential as a read-only file. Neither value is
ever placed in an artifact, `.env`, browser response, or runner output. The
separate Platform switch is applied only after the connector and pairing are
verified; it does not alter NAS routes, VPN, DNS, or Tailscale.

Install or update the root-owned live runner on Synology:

```bash
# First verify that no deploy process is active and state/deploy.lock is absent.
sudo install -o root -g root -m 0755 \
  /volume1/docker/nodedc-deploy/runner-install/nodedc-deploy \
  /usr/local/sbin/nodedc-deploy
sudo /usr/local/sbin/nodedc-deploy verify-install
```

Runner promotion is a standalone admin step, never an app-overlay artifact and
never part of a running apply. A Python process already executing the old file
keeps the old code in memory; always invoke a fresh verified process afterward.

Normal service deploys must still use explicit artifacts:

```bash
sudo /usr/local/sbin/nodedc-deploy plan /volume1/docker/nodedc-deploy/inbox/<artifact>.tgz
sudo /usr/local/sbin/nodedc-deploy apply /volume1/docker/nodedc-deploy/inbox/<artifact>.tgz
```