feat(platform): add managed data product history plane

This commit is contained in:
Codex
2026-07-18 14:38:06 +03:00
parent 02816c4352
commit 3c5d8f6cef
34 changed files with 2091 additions and 163 deletions
+32 -11
View File
@@ -90,10 +90,11 @@ node descriptions used `usableAsTool`, so n8n 2.3.2 exposed six NDC runtime
types instead of the required three. It must not be activated, overwritten or
deleted.
The corrected candidate is package version `0.1.2`, built with patch id
`n8n-nodes-ndc-release-20260716-003`. It receives a new digest-bound release
directory and remains inert after staging; only a separately reviewed
Engine-owned activator may select it after exact MCP schema acceptance.
The active predecessor is package version `0.1.2` at immutable release
`0.1.2-05e4b38b14b4a019`. History Read is package version `0.1.3`, built with
patch id `n8n-nodes-ndc-history-read-20260717-004` and immutable release
`0.1.3-3354149b5245e39a`. Staging remains inert; only the separately reviewed
Engine-owned transition may select it after exact MCP schema acceptance.
The paired Engine activation is built by
`build-engine-n8n-private-extension-artifact.mjs`. It deliberately does not
@@ -102,7 +103,7 @@ not build an image. A fresh transition id is mandatory and previously issued
ids are rejected:
```bash
node infra/deploy-runner/build-engine-n8n-private-extension-artifact.mjs 20260717-004
node infra/deploy-runner/build-engine-n8n-private-extension-artifact.mjs 20260717-005
```
The builder emits a narrowly scoped Compose override plus a strict transition
@@ -112,7 +113,7 @@ same immutable image ID, and extracts the package into the root-owned,
read-only Engine release tree:
```text
/volume2/nodedc-demo/n8n-private-extensions/releases/n8n-nodes-ndc/0.1.2-05e4b38b14b4a019/package
/volume2/nodedc-demo/n8n-private-extensions/releases/n8n-nodes-ndc/0.1.3-3354149b5245e39a/package
```
The override sets `N8N_USER_FOLDER=/home/node`, which is required because the
@@ -134,13 +135,16 @@ Postgres service, `.n8n` data, encryption key and credentials remain intact.
The apply gate verifies readiness, the running image/version, sealed mount,
loader environment, package-loader node/credential sets, scoped loader logs,
restart stability and content-exact pinned Engine MCP catalogs. The runner pins
both the complete 434/385 inactive baseline and the reviewed 437/388
activation catalogs, so a same-count substitution of any built-in schema is
rejected. Any gate failure after
the complete 434/385 inactive baseline and a digest registry for every
reviewed 437/388 active release, so a same-count substitution of any built-in
or private schema is rejected. Upgrade `0.1.2 -> 0.1.3` is accepted only when
the verified live predecessor, descriptor, package mount and catalog all agree.
Any gate failure after
mutation automatically restores the pre-apply catalogs/descriptor and
force-recreates the previous verified runtime. Staged, sealed and failed
releases are retained. The separate rollback artifact returns the first
activation to the verified inactive 434-node/385-credential catalog baseline.
releases are retained. The paired rollback artifact returns `0.1.3` to the
verified immutable `0.1.2` release and its exact catalogs; it does not invent
an inactive baseline for an already-active upgrade.
Run both policy suites before publishing the Engine pair:
@@ -151,6 +155,23 @@ PYTHONDONTWRITEBYTECODE=1 \
python3 infra/deploy-runner/test_engine_n8n_private_extension.py
```
The source-less Engine MCP control-plane update is a separate exact slice:
```bash
node infra/deploy-runner/build-engine-mcp-control-plane-artifact.mjs 20260717-001
```
Its six-entry registry contains only the Engine Agent gateway, verified graph
patch route, Codex installer source/package and the active node-intelligence
descriptor with the new gateway digest. It force-recreates only the existing
`nodedc-backend`. The node-intelligence image/service, n8n, L1, credentials,
databases and volumes are outside the slice. The runner proves the installed
predecessor digests, exact installer archive/source equality, bounded change
session policy, no-effect/post-write graph barriers, active immutable backend
runtime and descriptor equality. The ordinary overlay backup is the automatic
rollback source; rollback restores the predecessor gateway and descriptor
together before recreating the same backend service.
For `platform` artifacts, the allowlist includes the versioned Ontology Core,
the frozen legacy Gelios compatibility service, and the provider-neutral
External Data Plane sources. The Gelios service remains reproducible only to
@@ -1,7 +1,7 @@
#!/usr/bin/env node
import { createHash } from 'node:crypto'
import { spawnSync } from 'node:child_process'
import { cp, lstat, mkdir, mkdtemp, readFile, rm, writeFile } from 'node:fs/promises'
import { lstat, mkdir, mkdtemp, readFile, rm, writeFile } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { dirname, join, resolve } from 'node:path'
import { fileURLToPath } from 'node:url'
@@ -15,6 +15,7 @@ const [patchId = '', ...extra] = process.argv.slice(2)
const storeRelativePath = 'nodedc-source/server/engineAgents/store.js'
const predecessorSha256 = '52daa43499d6d9a97fe7ffa891edb9212b7791e733f91dd3ca686d42739b7e9a'
const targetSha256 = '2e62654c2dc12905efcc83a9dff45a818dd7b47924a10600160835c4416540e9'
const readerExtendedSourceSha256 = 'debd351fe0b8c72b33b8c79909b8f339cbaf061b970576f3ae72df52ebaa211f'
const previouslyIssuedPatchIds = new Set([
'engine-agent-full-grant-migration-20260717-001',
])
@@ -29,7 +30,7 @@ if (previouslyIssuedPatchIds.has(patchId)) {
const source = join(engineRoot, storeRelativePath)
const sourceInfo = await lstat(source)
if (sourceInfo.isSymbolicLink() || !sourceInfo.isFile()) throw new Error('engine_agent_store_source_unsafe')
const sourceBytes = await readFile(source)
const sourceBytes = materializeFrozenTarget(await readFile(source))
if (digest(sourceBytes) !== targetSha256) throw new Error('engine_agent_store_target_sha256_mismatch')
const sourceText = sourceBytes.toString('utf8')
for (const required of [
@@ -56,7 +57,7 @@ const stage = await mkdtemp(join(tmpdir(), 'nodedc-engine-agent-grant-migration-
try {
const destination = join(stage, 'payload', storeRelativePath)
await mkdir(dirname(destination), { recursive: true })
await cp(source, destination, { force: false, verbatimSymlinks: true })
await writeFile(destination, sourceBytes)
await writeFile(
join(stage, 'manifest.env'),
`id=${patchId}\ncomponent=engine\ntype=app-overlay\n`,
@@ -119,3 +120,22 @@ function run(command, args) {
function digest(bytes) {
return createHash('sha256').update(bytes).digest('hex')
}
function materializeFrozenTarget(bytes) {
const sourceSha256 = digest(bytes)
if (sourceSha256 === targetSha256) return bytes
if (sourceSha256 !== readerExtendedSourceSha256) {
throw new Error('engine_agent_store_target_sha256_mismatch')
}
const text = bytes.toString('utf8')
const readerScopes = [
" 'engine:l2:data-product-read-grant:plan',\n",
" 'engine:l2:data-product-read-grant:write',\n",
]
let frozen = text
for (const scope of readerScopes) {
if (!frozen.includes(scope)) throw new Error('engine_agent_store_reader_scope_boundary_missing')
frozen = frozen.replace(scope, '')
}
return Buffer.from(frozen, 'utf8')
}
@@ -45,7 +45,7 @@ const credentialSinkPredecessorSha256 = Object.freeze({
'nodedc-source/server/credentialSink/vendor/engine-credential-sink.mjs': 'b4800eead9bf94793ff1280d06e34aad6b37ef055a9d7f8d83d7fb5f9bd66d8b',
'nodedc-source/server/index.js': 'b2b790b02839570d967a2ca68b00e2724485a99389ac9b3a589a1b22302a36b8',
'nodedc-source/server/routes/engineCredentialSink.js': '9cbb69dbc8cbe6181cd5b0170fe9c4d717b0a173866ca98cba3e3766c0bab94e',
'nodedc-source/server/routes/ndcAgentMcp.js': 'fbb3342b1a617b956d5b3a6a40d5111aa107c1176b4ff89c37b104995bada081',
'nodedc-source/server/routes/ndcAgentMcp.js': '534e2c85e1faecc72a00da7ad32d0584ee09b6bd89eeec2221c3b23d80e1e962',
'nodedc-source/services/backend/credential-sink/docker-compose.immutable-runtime.yml': '944fa64b08255eb8207b93fd327aebb98ecd9400d37d25fcfa8e3a040ee44afe',
})
const publishGrantRuntimeOverride = [
@@ -0,0 +1,177 @@
#!/usr/bin/env node
import { createHash } from "node:crypto";
import { spawnSync } from "node:child_process";
import { cp, lstat, mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { dirname, join, resolve } from "node:path";
import { fileURLToPath } from "node:url";
const here = dirname(fileURLToPath(import.meta.url));
const platformRoot = resolve(here, "../..");
const engineRoot = resolve(
process.env.NODEDC_ENGINE_SOURCE_ROOT || resolve(platformRoot, "../NODEDC_ENGINE_INFRA"),
);
const canonicalArtifactRoot = resolve(here, "../deploy-artifacts");
const artifactRoot = resolve(process.env.NODEDC_DEPLOY_ARTIFACT_DIR || canonicalArtifactRoot);
const [transitionId = "20260718-003", ...extra] = process.argv.slice(2);
if (extra.length || !/^\d{8}-[0-9]{3}$/.test(transitionId)) {
throw new Error("usage: build-engine-mcp-control-plane-artifact.mjs [YYYYMMDD-NNN]");
}
const id = `engine-mcp-control-plane-${transitionId}`;
const target = join(artifactRoot, `nodedc-${id}.tgz`);
const nodeIntelligenceArtifact = join(
canonicalArtifactRoot,
"nodedc-engine-node-intelligence-20260717-001.tgz",
);
const nodeIntelligenceArtifactSha256 = "d126afaa0c714fef26362aad4749e3f4647f551d6eb975f0133cdf9ff2e6fc4f";
const descriptorRel = "nodedc-source/services/node-intelligence/activation.json";
const gatewayRel = "nodedc-source/server/routes/engineAgentGateway.js";
const upstreamProjectionRel = "nodedc-source/server/nodeIntelligence/upstreamProjection.js";
const files = [
"nodedc-source/server/assets/engine-agent-npm/bin/nodedc-engine-codex-agent.mjs",
"nodedc-source/server/assets/engine-agent-npm/package.json",
"nodedc-source/server/assets/nodedc-engine-codex-agent-0.1.4.tgz",
"nodedc-source/server/dataProductPublishGrant/providerCatalog.js",
"nodedc-source/server/dataProductPublishGrant/signedDataPlaneClient.js",
"nodedc-source/server/dataProductReadGrant/acceptance.js",
"nodedc-source/server/dataProductReadGrant/service.js",
"nodedc-source/server/dataProductReadGrant/store.js",
"nodedc-source/server/engineAgents/store.js",
upstreamProjectionRel,
gatewayRel,
"nodedc-source/server/routes/n8n.js",
"nodedc-source/server/routes/ndcAgentMcp.js",
"nodedc-source/services/backend/data-product-read-grant/docker-compose.immutable-runtime.yml",
"nodedc-source/server/dataProductPublishGrant/service.js",
"nodedc-source/server/dataProductPublishGrant/store.js",
descriptorRel,
];
const expectedSha256 = new Map([
[files[0], "521098de69fe288a56bd4158c849c1055ba24be08e19f7a4111618d0e8138445"],
[files[1], "cbe113e9b10bb84b9ccbffa3e261908e58a8430c11abe2cf4fd5304741b04597"],
[files[2], "e74c0136d346f904b42589d75cb11a952b4d2f21153f1b057fba28e9acf4f95f"],
[files[3], "901b8fad80018ce177b34ced804b39cb140a47e831414057f484296b373c651d"],
[files[4], "c2a13d5eb49937fe70ec6451d0465cac54c19c7fae44448db099079473ec02fc"],
[files[5], "202dbe575b2f2e1c584aa7e6e99e38fa84f12496feb454e3dbd3f98e2d0396dd"],
[files[6], "65b8cdd6b603333bac1feb303e9791feb1e9da39dc9b5bfc3df3961273b0052e"],
[files[7], "c3fcdc59a794f57d92e3d2ac713ee28341347cebd25364c4060beaa3dc2151de"],
[files[8], "debd351fe0b8c72b33b8c79909b8f339cbaf061b970576f3ae72df52ebaa211f"],
[files[9], "761a874b102a938bc6018159ddacdaac71ad6ae08e9f0f8d7f3b58a0165a5131"],
[files[10], "96c726dab5cf1341f74e6e1095d518058ca320e0dd5738e25bdbe75db1f4fc15"],
[files[11], "f293a7794405badbabd2bf7ef088f96fe1167d9e249f05cbfc8af280a0d3e8f8"],
[files[12], "534e2c85e1faecc72a00da7ad32d0584ee09b6bd89eeec2221c3b23d80e1e962"],
[files[13], "952df0cb2ac477e644f5f3a9d64b4872ce1da33356eeab0bec17dcb2931cf653"],
[files[14], "ded3832c9677345a988448ae8e69cef254fd9bf39d2de20cffa295c1feedcd7c"],
[files[15], "a92303b2732e21f68c1ac732fa26e4983cbadf3515741cee983b916a283d754c"],
]);
await assertFresh(target);
assertSha(await readFile(nodeIntelligenceArtifact), nodeIntelligenceArtifactSha256, "node intelligence artifact");
const stage = await mkdtemp(join(tmpdir(), "nodedc-engine-mcp-control-plane-"));
const payload = join(stage, "payload");
try {
await mkdir(payload, { recursive: true });
for (const rel of files.slice(0, -1)) {
const source = join(engineRoot, rel);
const stat = await lstat(source);
if (!stat.isFile() || stat.isSymbolicLink()) throw new Error(`source_boundary_invalid:${rel}`);
assertSha(await readFile(source), expectedSha256.get(rel), rel);
await mkdir(dirname(join(payload, rel)), { recursive: true });
await cp(source, join(payload, rel), { force: false });
}
const descriptorText = extractMember(
nodeIntelligenceArtifact,
`payload/${descriptorRel}`,
);
const descriptor = JSON.parse(descriptorText);
if (
descriptor?.action !== "activate"
|| descriptor?.releaseId !== "2.33.2-974a9fb3492f"
|| descriptor?.source?.gatewaySha256 !== "9642c76fd5765a8a20629c8d09e16579a1c3b2cfb7bdbab38cf55af469d8908f"
) throw new Error("node_intelligence_predecessor_descriptor_mismatch");
descriptor.source.gatewaySha256 = expectedSha256.get(gatewayRel);
descriptor.source.upstreamProjectionSha256 = expectedSha256.get(upstreamProjectionRel);
await mkdir(dirname(join(payload, descriptorRel)), { recursive: true });
await writeFile(join(payload, descriptorRel), `${JSON.stringify(descriptor, null, 2)}\n`, "utf8");
await writeFile(join(stage, "manifest.env"), `id=${id}\ncomponent=engine\ntype=app-overlay\n`, "utf8");
await writeFile(join(stage, "files.txt"), `${files.join("\n")}\n`, "utf8");
await mkdir(artifactRoot, { recursive: true });
run("python3", ["-c", canonicalTarScript(), target, stage]);
const artifactSha256 = sha(await readFile(target));
console.log(JSON.stringify({
ok: true,
id,
artifact: target,
artifactSha256,
services: ["nodedc-backend"],
predecessorGatewaySha256: "9642c76fd5765a8a20629c8d09e16579a1c3b2cfb7bdbab38cf55af469d8908f",
targetGatewaySha256: expectedSha256.get(gatewayRel),
mcpVersion: "0.5.0",
installerVersion: "0.1.4",
files,
}, null, 2));
} finally {
await rm(stage, { recursive: true, force: true });
}
async function assertFresh(path) {
try {
await lstat(path);
} catch (error) {
if (error?.code === "ENOENT") return;
throw error;
}
throw new Error("artifact_already_exists");
}
function extractMember(archive, member) {
const script = [
"import pathlib,sys,tarfile",
"p=pathlib.Path(sys.argv[1]); name=sys.argv[2]",
"with tarfile.open(p,'r:gz') as t:",
" m=t.getmember(name)",
" if not m.isfile(): raise SystemExit('member-not-file')",
" f=t.extractfile(m)",
" if f is None: raise SystemExit('member-unreadable')",
" sys.stdout.buffer.write(f.read())",
].join("\n");
return run("python3", ["-c", script, archive, member]).stdout;
}
function canonicalTarScript() {
return [
"import gzip,io,pathlib,sys,tarfile",
"root=pathlib.Path(sys.argv[2])",
"with open(sys.argv[1],'wb') as out:",
" with gzip.GzipFile(filename='',mode='wb',fileobj=out,compresslevel=9,mtime=0) as gz:",
" with tarfile.open(fileobj=gz,mode='w',format=tarfile.PAX_FORMAT) as tar:",
" for top in ('manifest.env','files.txt','payload'):",
" p=root/top; paths=[p]+(sorted(p.rglob('*')) if p.is_dir() else [])",
" for x in paths:",
" info=tar.gettarinfo(str(x),arcname=x.relative_to(root).as_posix())",
" info.uid=info.gid=0; info.uname=info.gname='root'; info.mtime=0; info.mode=0o755 if info.isdir() else 0o644",
" with (open(x,'rb') if info.isfile() else io.BytesIO()) as src: tar.addfile(info,src if info.isfile() else None)",
].join("\n");
}
function assertSha(bytes, expected, label) {
const actual = sha(bytes);
if (!expected || actual !== expected) throw new Error(`${label.replaceAll(" ", "_")}_sha256_mismatch:${actual}`);
}
function sha(bytes) {
return createHash("sha256").update(bytes).digest("hex");
}
function run(command, args) {
const result = spawnSync(command, args, {
encoding: "utf8",
maxBuffer: 128 * 1024 * 1024,
stdio: ["ignore", "pipe", "pipe"],
});
if (result.status !== 0) throw new Error(`${command}_failed:${result.stderr || result.stdout}`);
return result;
}
@@ -13,18 +13,19 @@ const engineRoot = resolve(platformRoot, "../NODEDC_ENGINE_INFRA");
const artifactRoot = resolve(process.env.NODEDC_DEPLOY_ARTIFACT_DIR || join(here, "../deploy-artifacts"));
const stageArtifact = resolve(
process.env.NODEDC_N8N_EXTENSION_STAGE_ARTIFACT
|| join(artifactRoot, "nodedc-n8n-private-extension-n8n-nodes-ndc-release-20260716-003.tgz"),
|| join(artifactRoot, "nodedc-n8n-private-extension-n8n-nodes-ndc-history-read-20260717-004.tgz"),
);
const stageArtifactSha256 = "4601c16c57e5182996adf18d0163837511b27ab3f7cfdf97eac679418ee2d078";
const releaseId = "0.1.2-05e4b38b14b4a019";
const packageVersion = "0.1.2";
const packageSha256 = "05e4b38b14b4a019ce1f6eee27b9e320094cb3560903bd68074966b3a1267af5";
const stageArtifactSha256 = "a78f8250704e4893eab727cdb862b69ce45a84310fbb73ecd3dfd968b72d12ee";
const releaseId = "0.1.3-3354149b5245e39a";
const packageVersion = "0.1.3";
const packageSha256 = "3354149b5245e39a10f6f03a4b43796602d7e57ea1f91dcc6cb6774ead97501d";
const n8nVersion = "2.3.2";
const baseImage = "docker.n8n.io/n8nio/n8n:2.3.2";
const architecture = "amd64";
const generatedAt = "2026-07-15T21:51:41.000Z";
const previouslyIssuedTransitionIds = new Set(["20260715-002", "20260716-003"]);
const generatedAt = "2026-07-17T21:00:00.000Z";
const predecessorGitRevision = "96ad46e3c62943f818233481957c62e5088ae35c";
const previouslyIssuedTransitionIds = new Set(["20260715-002", "20260716-003", "20260717-004"]);
const transitionId = readTransitionId(process.argv.slice(2), process.env.NODEDC_N8N_TRANSITION_ID);
const activationId = `engine-n8n-private-extension-${transitionId}`;
const rollbackId = `engine-n8n-private-extension-rollback-${transitionId}`;
@@ -39,7 +40,6 @@ const metaRel = `${schemaRoot}/meta.json`;
const iconRoot = "nodedc-source/server/assets/n8n/icons";
const iconRel = `${iconRoot}/ndc.svg`;
const darkIconRel = `${iconRoot}/ndc.dark.svg`;
const sealedReleaseRelativePath = `n8n-private-extensions/releases/n8n-nodes-ndc/${releaseId}/package`;
const runtimePackagePath = "/home/node/.n8n/nodes/node_modules/n8n-nodes-ndc";
const expectedNodeTypes = [
@@ -111,10 +111,23 @@ try {
assertExact(privateNodes.map((item) => item.name), expectedNodeTypes, "node types");
assertExact(privateCredentials.map((item) => item.name), expectedCredentialTypes, "credential types");
const baselineNodes = JSON.parse(gitFile(nodesCatalogRel));
const baselineCredentials = JSON.parse(gitFile(credentialsCatalogRel));
const baselineMeta = JSON.parse(gitFile(metaRel));
assertBaselineCatalogs(baselineNodes, baselineCredentials, baselineMeta);
const predecessorDescriptor = JSON.parse(gitFile(descriptorRel));
const predecessorNodes = JSON.parse(gitFile(nodesCatalogRel));
const predecessorCredentials = JSON.parse(gitFile(credentialsCatalogRel));
const predecessorMeta = JSON.parse(gitFile(metaRel));
assertPredecessorCatalogs(
predecessorDescriptor,
predecessorNodes,
predecessorCredentials,
predecessorMeta,
);
const baselineNodes = predecessorNodes.filter(
(item) => !String(item?.name || "").startsWith("n8n-nodes-ndc."),
);
const baselineCredentials = predecessorCredentials.filter(
(item) => !expectedCredentialTypes.includes(String(item?.name || "")),
);
assertBaselineCatalogs(baselineNodes, baselineCredentials);
const activeNodes = [...baselineNodes, ...privateNodes];
const activeCredentials = [...baselineCredentials, ...privateCredentials];
const activeMeta = {
@@ -125,9 +138,20 @@ try {
credentialCount: activeCredentials.length,
};
const activationDescriptor = descriptor("activate", expectedNodeTypes, expectedCredentialTypes, "verified_inactive");
const rollbackDescriptor = descriptor("rollback-inactive", [], [], releaseId);
const override = composeOverride();
const target = {
releaseId,
packageVersion,
packageSha256,
};
const predecessor = {
releaseId: predecessorDescriptor.releaseId,
packageVersion: predecessorDescriptor.packageVersion,
packageSha256: predecessorDescriptor.packageSha256,
};
const activationDescriptor = descriptor(target, predecessor.releaseId, predecessor.releaseId);
const rollbackDescriptor = descriptor(predecessor, releaseId, releaseId);
const override = composeOverride(target);
const rollbackOverride = composeOverride(predecessor);
const generatedRoot = join(work, "generated-engine-payload");
await writeJson(join(generatedRoot, nodesCatalogRel), activeNodes);
@@ -154,13 +178,16 @@ try {
}
});
const rollbackEntries = [descriptorRel, nodesCatalogRel, credentialsCatalogRel, metaRel];
const rollbackEntries = [...activationEntries];
const rollbackArtifact = await buildArtifact(work, rollbackId, rollbackEntries, async (payload) => {
await writeJson(join(payload, descriptorRel), rollbackDescriptor);
await mkdir(join(payload, schemaRoot), { recursive: true });
await writeFile(join(payload, nodesCatalogRel), `${JSON.stringify(baselineNodes, null, 2)}\n`, "utf8");
await writeFile(join(payload, credentialsCatalogRel), `${JSON.stringify(baselineCredentials, null, 2)}\n`, "utf8");
await writeFile(join(payload, metaRel), `${JSON.stringify(baselineMeta, null, 2)}\n`, "utf8");
await writeFile(join(payload, overrideRel), rollbackOverride, "utf8");
await writeJson(join(payload, nodesCatalogRel), predecessorNodes);
await writeJson(join(payload, credentialsCatalogRel), predecessorCredentials);
await writeJson(join(payload, metaRel), predecessorMeta);
await mkdir(join(payload, iconRoot), { recursive: true });
await cp(join(engineRoot, iconRel), join(payload, iconRel), { force: false });
await cp(join(engineRoot, darkIconRel), join(payload, darkIconRel), { force: false });
});
console.log(JSON.stringify({
@@ -177,25 +204,25 @@ try {
await rm(work, { recursive: true, force: true });
}
function descriptor(action, nodeTypes, credentialTypes, expectedCurrent) {
function descriptor(target, expectedCurrent, rollbackBaseline) {
return {
schemaVersion: "nodedc.engine-n8n-private-extension-transition/v1",
action,
releaseId,
packageVersion,
packageSha256,
action: "activate",
releaseId: target.releaseId,
packageVersion: target.packageVersion,
packageSha256: target.packageSha256,
n8nVersion,
baseImage,
baseImageArchitecture: architecture,
baseImageIdentityPolicy: "running-container-and-local-tag-must-match",
sealedReleaseRelativePath,
sealedReleaseRelativePath: `n8n-private-extensions/releases/n8n-nodes-ndc/${target.releaseId}/package`,
composeOverride: overrideRel,
runtimePackagePath,
topologyServices: ["n8n"],
expectedCurrent,
expectedNodeTypes: nodeTypes,
expectedCredentialTypes: credentialTypes,
rollbackBaseline: "verified_inactive",
expectedNodeTypes,
expectedCredentialTypes,
rollbackBaseline,
};
}
@@ -235,8 +262,9 @@ async function assertArtifactTargetFresh(path) {
throw new Error("transition_artifact_already_exists");
}
function composeOverride() {
function composeOverride(target) {
const health = "const http=require('http');const req=http.get('http://127.0.0.1:5678/healthz/readiness',r=>{r.resume();process.exit(r.statusCode===200?0:1)});req.on('error',()=>process.exit(1));req.setTimeout(4000,()=>{req.destroy();process.exit(1)});";
const sealedReleaseRelativePath = `n8n-private-extensions/releases/n8n-nodes-ndc/${target.releaseId}/package`;
return [
"services:",
" n8n:",
@@ -257,8 +285,8 @@ function composeOverride() {
" retries: 30",
" start_period: 30s",
" labels:",
` nodedc.n8n-private-extension.release: ${releaseId}`,
` nodedc.n8n-private-extension.package-sha256: ${packageSha256}`,
` nodedc.n8n-private-extension.release: ${target.releaseId}`,
` nodedc.n8n-private-extension.package-sha256: ${target.packageSha256}`,
"",
].join("\n");
}
@@ -297,7 +325,34 @@ function assertPackage(value) {
assertExact(value.n8n?.credentials, credentialModules.map(([path]) => path), "package credentials");
}
function assertBaselineCatalogs(nodes, credentials, meta) {
function assertPredecessorCatalogs(descriptorValue, nodes, credentials, meta) {
if (descriptorValue?.action !== "activate"
|| descriptorValue?.releaseId !== "0.1.2-05e4b38b14b4a019"
|| descriptorValue?.packageVersion !== "0.1.2"
|| descriptorValue?.packageSha256 !== "05e4b38b14b4a019ce1f6eee27b9e320094cb3560903bd68074966b3a1267af5") {
throw new Error("predecessor_descriptor_mismatch");
}
if (!Array.isArray(nodes) || nodes.length !== 437
|| !Array.isArray(credentials) || credentials.length !== 388
|| meta?.n8nVersion !== n8nVersion
|| meta?.source !== "n8n-core+n8n-nodes-ndc@0.1.2"
|| meta?.nodeCount !== 437
|| meta?.credentialCount !== 388) {
throw new Error("predecessor_catalog_mismatch");
}
assertExact(
nodes.filter((item) => String(item?.name || "").startsWith("n8n-nodes-ndc.")).map((item) => item.name),
expectedNodeTypes,
"predecessor node types",
);
assertExact(
credentials.filter((item) => expectedCredentialTypes.includes(String(item?.name || ""))).map((item) => item.name),
expectedCredentialTypes,
"predecessor credential types",
);
}
function assertBaselineCatalogs(nodes, credentials) {
if (!Array.isArray(nodes) || nodes.length !== 434 || nodes.some((item) => String(item?.name || "").startsWith("n8n-nodes-ndc."))) {
throw new Error("baseline_node_catalog_mismatch");
}
@@ -305,9 +360,6 @@ function assertBaselineCatalogs(nodes, credentials, meta) {
|| credentials.some((item) => expectedCredentialTypes.includes(String(item?.name || "")))) {
throw new Error("baseline_credential_catalog_mismatch");
}
if (meta?.n8nVersion !== n8nVersion || meta?.nodeCount !== 434 || meta?.credentialCount !== 385) {
throw new Error("baseline_meta_mismatch");
}
}
function assertEngineBaseline(compose) {
@@ -330,7 +382,7 @@ function assertSha(bytes, expected, label) {
}
function gitFile(rel) {
return run("git", ["show", `HEAD:${rel}`], engineRoot).stdout;
return run("git", ["show", `${predecessorGitRevision}:${rel}`], engineRoot).stdout;
}
async function writeJson(path, value) {
@@ -23,6 +23,7 @@ const files = [
["packages/external-provider-contract/src/data-plane.mjs", "platform/packages/external-provider-contract/src/data-plane.mjs"],
["packages/external-provider-contract/src/data-product.mjs", "platform/packages/external-provider-contract/src/data-product.mjs"],
["packages/external-provider-contract/src/intake-batch.mjs", "platform/packages/external-provider-contract/src/intake-batch.mjs"],
["packages/external-provider-contract/src/index.mjs", "platform/packages/external-provider-contract/src/index.mjs"],
["packages/external-provider-contract/src/sensitive-field-policy.mjs", "platform/packages/external-provider-contract/src/sensitive-field-policy.mjs"],
];
const ignoredBasenames = new Set([".DS_Store", ".git", "node_modules"]);
@@ -84,6 +85,21 @@ async function assertSourceBoundary() {
compose.includes("private-key.pem")
|| compose.includes("ENGINE_EDP_MANAGED_PROVISIONER_PRIVATE_KEY_FILE")
) throw new Error("platform_artifact_private_key_boundary_violation");
const server = await readFile(
resolve(platformRoot, "services/external-data-plane/src/server.mjs"),
"utf8",
);
for (const marker of [
'managedWriterBindingLifetime: config.managedProvisionerApiEnabled ? "explicit-revoke" : "disabled"',
'managedWriterBindings: "digest+idempotent-generation+explicit-revoke"',
'app.put("/internal/data-plane/v1/reader-bindings/by-key/:bindingKey"',
'managedReaderBindingProvisioning: config.managedProvisionerApiEnabled',
'managedReaderBindingLifetime: config.managedProvisionerApiEnabled ? "explicit-revoke" : "disabled"',
'managedReaderBindings: "digest+idempotent-generation+explicit-revoke"',
'where id = $1 and binding_key is null and active = true',
]) {
if (!server.includes(marker)) throw new Error(`managed_reader_boundary_missing:${marker}`);
}
}
function canonicalTarScript() {
@@ -10,10 +10,10 @@ const scriptDir = dirname(fileURLToPath(import.meta.url));
const platformRoot = resolve(scriptDir, "../..");
const workspaceRoot = resolve(platformRoot, "..");
const foundryRoot = resolve(workspaceRoot, "NODEDC_DESIGN_GUIDELINE");
const artifactDir = resolve(scriptDir, "../deploy-artifacts");
const patchId = process.argv[2] || "module-foundry-bootstrap-20260714-001";
const artifactDir = resolve(process.env.NODEDC_DEPLOY_ARTIFACT_DIR || resolve(scriptDir, "../deploy-artifacts"));
const [patchId = "module-foundry-bootstrap-20260714-001", ...extra] = process.argv.slice(2);
if (!/^[A-Za-z0-9._-]{1,96}$/.test(patchId)) {
if (extra.length || !/^[A-Za-z0-9._-]{1,96}$/.test(patchId)) {
throw new Error("patch_id_must_contain_only_letters_digits_dot_underscore_hyphen");
}
@@ -48,13 +48,9 @@ try {
await writeFile(join(stage, "files.txt"), `${files.join("\n")}\n`, "utf8");
await mkdir(artifactDir, { recursive: true });
const tar = spawnSync("python3", ["-c", [
"import sys, tarfile",
"with tarfile.open(sys.argv[1], 'w:gz', format=tarfile.PAX_FORMAT) as archive:",
" [archive.add(name, arcname=name, recursive=True) for name in ('manifest.env', 'files.txt', 'payload')]",
].join("\n"), target], {
cwd: stage,
const tar = spawnSync("python3", ["-c", canonicalTarScript(), target, stage], {
encoding: "utf8",
maxBuffer: 128 * 1024 * 1024,
});
if (tar.status !== 0) throw new Error(`tar_failed:${tar.stderr || tar.stdout}`);
@@ -64,6 +60,22 @@ try {
await rm(stage, { recursive: true, force: true });
}
function canonicalTarScript() {
return [
"import gzip,io,pathlib,sys,tarfile",
"root=pathlib.Path(sys.argv[2])",
"with open(sys.argv[1],'wb') as out:",
" with gzip.GzipFile(filename='',mode='wb',fileobj=out,compresslevel=9,mtime=0) as gz:",
" with tarfile.open(fileobj=gz,mode='w',format=tarfile.PAX_FORMAT) as tar:",
" for top in ('manifest.env','files.txt','payload'):",
" p=root/top; paths=[p]+(sorted(p.rglob('*')) if p.is_dir() else [])",
" for x in paths:",
" info=tar.gettarinfo(str(x),arcname=x.relative_to(root).as_posix())",
" info.uid=info.gid=0; info.uname=info.gname='root'; info.mtime=0; info.mode=0o755 if info.isdir() else 0o644",
" with (open(x,'rb') if info.isfile() else io.BytesIO()) as src: tar.addfile(info,src if info.isfile() else None)",
].join("\n");
}
async function copySafe(source, destination) {
const sourceStat = await lstat(source);
if (sourceStat.isSymbolicLink()) throw new Error(`source_symlink_rejected:${relative(foundryRoot, source)}`);
@@ -12,8 +12,8 @@ const platformRoot = resolve(scriptDir, "../..");
const packageRoot = resolve(platformRoot, "packages/n8n-nodes-ndc");
const artifactDir = resolve(process.env.NODEDC_DEPLOY_ARTIFACT_DIR || resolve(scriptDir, "../deploy-artifacts"));
const requireModule = createRequire(import.meta.url);
const expectedPackageVersion = "0.1.2";
const [patchId = "n8n-nodes-ndc-release-20260716-003", ...extra] = process.argv.slice(2);
const expectedPackageVersion = "0.1.3";
const [patchId = "n8n-nodes-ndc-history-read-20260717-004", ...extra] = process.argv.slice(2);
const expectedRuntimeNodes = [
{
file: "dist/nodes/NdcDataProductPublish/NdcDataProductPublish.node.js",
+473 -25
View File
@@ -54,6 +54,7 @@ ENGINE_CREDENTIAL_BACKEND_ROOTFS_FILE = ENGINE_CREDENTIAL_BACKEND_RUNTIME_DIR /
ENGINE_CREDENTIAL_BACKEND_METADATA_FILE = ENGINE_CREDENTIAL_BACKEND_RUNTIME_DIR / "runtime-metadata.json"
ENGINE_CREDENTIAL_BACKEND_ACTIVATION_FILE = ENGINE_CREDENTIAL_BACKEND_RUNTIME_DIR / "activation.ready"
ENGINE_DATA_PRODUCT_PUBLISH_GRANT_OVERRIDE_REL = "nodedc-source/services/backend/data-product-publish-grant/docker-compose.immutable-runtime.yml"
ENGINE_DATA_PRODUCT_READ_GRANT_OVERRIDE_REL = "nodedc-source/services/backend/data-product-read-grant/docker-compose.immutable-runtime.yml"
ENGINE_AGENT_FULL_GRANT_MIGRATION_STORE_REL = "nodedc-source/server/engineAgents/store.js"
ENGINE_AGENT_FULL_GRANT_MIGRATION_PREDECESSOR_SHA256 = "52daa43499d6d9a97fe7ffa891edb9212b7791e733f91dd3ca686d42739b7e9a"
ENGINE_AGENT_FULL_GRANT_MIGRATION_TARGET_SHA256 = "2e62654c2dc12905efcc83a9dff45a818dd7b47924a10600160835c4416540e9"
@@ -115,13 +116,74 @@ ENGINE_NODE_INTELLIGENCE_ROLLBACK_ENTRIES = (
ENGINE_NODE_INTELLIGENCE_GATEWAY_REL,
ENGINE_NODE_INTELLIGENCE_SERVICE_ROOT_REL,
)
ENGINE_MCP_CONTROL_PLANE_DESCRIPTOR_REL = ENGINE_NODE_INTELLIGENCE_DESCRIPTOR_REL
ENGINE_MCP_CONTROL_PLANE_ARTIFACT_ENTRIES = (
"nodedc-source/server/assets/engine-agent-npm/bin/nodedc-engine-codex-agent.mjs",
"nodedc-source/server/assets/engine-agent-npm/package.json",
"nodedc-source/server/assets/nodedc-engine-codex-agent-0.1.4.tgz",
"nodedc-source/server/dataProductPublishGrant/providerCatalog.js",
"nodedc-source/server/dataProductPublishGrant/signedDataPlaneClient.js",
"nodedc-source/server/dataProductReadGrant/acceptance.js",
"nodedc-source/server/dataProductReadGrant/service.js",
"nodedc-source/server/dataProductReadGrant/store.js",
"nodedc-source/server/engineAgents/store.js",
"nodedc-source/server/nodeIntelligence/upstreamProjection.js",
ENGINE_NODE_INTELLIGENCE_GATEWAY_REL,
"nodedc-source/server/routes/n8n.js",
"nodedc-source/server/routes/ndcAgentMcp.js",
ENGINE_DATA_PRODUCT_READ_GRANT_OVERRIDE_REL,
"nodedc-source/server/dataProductPublishGrant/service.js",
"nodedc-source/server/dataProductPublishGrant/store.js",
ENGINE_MCP_CONTROL_PLANE_DESCRIPTOR_REL,
)
ENGINE_MCP_CONTROL_PLANE_PREDECESSOR_SHA256 = {
ENGINE_NODE_INTELLIGENCE_GATEWAY_REL: "604a75ad3b9b463cea5e578760c8a23592e2e00b8acfc0099730a347e27bf4e8",
"nodedc-source/server/routes/ndcAgentMcp.js": "534e2c85e1faecc72a00da7ad32d0584ee09b6bd89eeec2221c3b23d80e1e962",
"nodedc-source/server/routes/n8n.js": "de6e3c76740af86e2eaeedede140b5ee54eb526f03db270324d7a4fe513f6817",
"nodedc-source/server/dataProductPublishGrant/providerCatalog.js": "a96d3695fdb3ed7012b9041182f37b09e32d1dfa0dbb391d8db2d7d25a39d64d",
"nodedc-source/server/dataProductPublishGrant/signedDataPlaneClient.js": "1a26728ac69c5ef3fdce1ce1154a6ce330f72286b6e07825796a246070d15bbf",
"nodedc-source/server/engineAgents/store.js": "2e62654c2dc12905efcc83a9dff45a818dd7b47924a10600160835c4416540e9",
"nodedc-source/server/nodeIntelligence/upstreamProjection.js": "946353ed8582c5d96e08361603425cc45eaf231f730ae46a75028d3bef352ee9",
"nodedc-source/server/assets/engine-agent-npm/bin/nodedc-engine-codex-agent.mjs": "7d3d7a9bc48614e6cd238199caa384073096b7ec8273703833681151b464ef17",
"nodedc-source/server/assets/engine-agent-npm/package.json": "9f299b20db4855b1bb4a9bd2e1095dfd059cd90ec82e936884a380aa41249e28",
"nodedc-source/server/dataProductPublishGrant/service.js": "85ded82d707e0ef5b1ad739b5a9be24c1432e2dd9e5051108739f09daf20f47c",
"nodedc-source/server/dataProductPublishGrant/store.js": "ef2a5e6bfad9f0b1710db006a0d30d01cd9bcaf06524b0e9f975dd771efe3952",
}
ENGINE_MCP_CONTROL_PLANE_TARGET_SHA256 = {
ENGINE_NODE_INTELLIGENCE_GATEWAY_REL: "96c726dab5cf1341f74e6e1095d518058ca320e0dd5738e25bdbe75db1f4fc15",
"nodedc-source/server/routes/ndcAgentMcp.js": "534e2c85e1faecc72a00da7ad32d0584ee09b6bd89eeec2221c3b23d80e1e962",
"nodedc-source/server/routes/n8n.js": "f293a7794405badbabd2bf7ef088f96fe1167d9e249f05cbfc8af280a0d3e8f8",
"nodedc-source/server/dataProductPublishGrant/providerCatalog.js": "901b8fad80018ce177b34ced804b39cb140a47e831414057f484296b373c651d",
"nodedc-source/server/dataProductPublishGrant/signedDataPlaneClient.js": "c2a13d5eb49937fe70ec6451d0465cac54c19c7fae44448db099079473ec02fc",
"nodedc-source/server/dataProductReadGrant/acceptance.js": "202dbe575b2f2e1c584aa7e6e99e38fa84f12496feb454e3dbd3f98e2d0396dd",
"nodedc-source/server/dataProductReadGrant/service.js": "65b8cdd6b603333bac1feb303e9791feb1e9da39dc9b5bfc3df3961273b0052e",
"nodedc-source/server/dataProductReadGrant/store.js": "c3fcdc59a794f57d92e3d2ac713ee28341347cebd25364c4060beaa3dc2151de",
"nodedc-source/server/engineAgents/store.js": "debd351fe0b8c72b33b8c79909b8f339cbaf061b970576f3ae72df52ebaa211f",
"nodedc-source/server/nodeIntelligence/upstreamProjection.js": "761a874b102a938bc6018159ddacdaac71ad6ae08e9f0f8d7f3b58a0165a5131",
"nodedc-source/server/assets/engine-agent-npm/bin/nodedc-engine-codex-agent.mjs": "521098de69fe288a56bd4158c849c1055ba24be08e19f7a4111618d0e8138445",
"nodedc-source/server/assets/engine-agent-npm/package.json": "cbe113e9b10bb84b9ccbffa3e261908e58a8430c11abe2cf4fd5304741b04597",
"nodedc-source/server/assets/nodedc-engine-codex-agent-0.1.4.tgz": "e74c0136d346f904b42589d75cb11a952b4d2f21153f1b057fba28e9acf4f95f",
ENGINE_DATA_PRODUCT_READ_GRANT_OVERRIDE_REL: "952df0cb2ac477e644f5f3a9d64b4872ce1da33356eeab0bec17dcb2931cf653",
"nodedc-source/server/dataProductPublishGrant/service.js": "ded3832c9677345a988448ae8e69cef254fd9bf39d2de20cffa295c1feedcd7c",
"nodedc-source/server/dataProductPublishGrant/store.js": "a92303b2732e21f68c1ac732fa26e4983cbadf3515741cee983b916a283d754c",
}
ENGINE_MCP_CONTROL_PLANE_NEW_PATHS = (
"nodedc-source/server/assets/nodedc-engine-codex-agent-0.1.4.tgz",
"nodedc-source/server/dataProductReadGrant/acceptance.js",
"nodedc-source/server/dataProductReadGrant/service.js",
"nodedc-source/server/dataProductReadGrant/store.js",
ENGINE_DATA_PRODUCT_READ_GRANT_OVERRIDE_REL,
)
ENGINE_CONTROL_PLANE_STATE_REL = "nodedc-control-plane"
ENGINE_PUBLISH_GRANT_STATE_REL = f"{ENGINE_CONTROL_PLANE_STATE_REL}/publish-grants"
ENGINE_READ_GRANT_STATE_REL = f"{ENGINE_CONTROL_PLANE_STATE_REL}/read-grants"
ENGINE_CONTROL_PLANE_STATE_PATH = Path("/volume2/nodedc-demo/nodedc-control-plane")
ENGINE_PUBLISH_GRANT_STATE_PATH = ENGINE_CONTROL_PLANE_STATE_PATH / "publish-grants"
ENGINE_READ_GRANT_STATE_PATH = ENGINE_CONTROL_PLANE_STATE_PATH / "read-grants"
ENGINE_EDP_PRIVATE_KEY_CONTAINER_PATH = "/run/nodedc-secrets/engine-edp-managed-provisioner-private.pem"
ENGINE_CONTROL_PLANE_CONTAINER_PATH = "/var/lib/nodedc-control-plane"
ENGINE_PUBLISH_GRANT_CONTAINER_PATH = "/var/lib/nodedc-control-plane/publish-grants"
ENGINE_READ_GRANT_CONTAINER_PATH = "/var/lib/nodedc-control-plane/read-grants"
EXTERNAL_DATA_PLANE_MANAGED_TRUST_CONTAINER_PATH = "/run/nodedc-trust/engine-managed-provisioner"
EXTERNAL_DATA_PLANE_INTERNAL_URL = "http://external-data-plane:18106"
PLATFORM_EXTERNAL_DATA_PLANE_COMPOSE_REL = "platform/docker-compose.external-data-plane.yml"
@@ -135,9 +197,18 @@ ENGINE_N8N_RUNTIME_PACKAGE_PATH = "/home/node/.n8n/nodes/node_modules/n8n-nodes-
ENGINE_N8N_NODE_MODULES_PATH = "/usr/local/lib/node_modules/n8n/node_modules"
ENGINE_N8N_ICON_SHA256 = "1c928fc996d8b82121e8f8e327d74b1dd21556c106de99c5e8d317d926c0ba17"
ENGINE_N8N_DARK_ICON_SHA256 = "ef3b8551da4ce04527736405afa9a220a2c76d047bd18f6f7124b9adb4216b0c"
ENGINE_N8N_ACTIVATION_NODES_CATALOG_JSON_SHA256 = "230f712230f7ee8debaa4b44a4358cc19c205bc43305d8ed89d5e3daac466506"
ENGINE_N8N_ACTIVATION_CREDENTIALS_CATALOG_JSON_SHA256 = "a26824ffc0e15db857c792153de3417febc0dbba4880380d6c8cd544b3c80f4d"
ENGINE_N8N_ACTIVATION_META_JSON_SHA256 = "caf7635420c61333e65f68350f5567217aed96fb51354b38764bcac2c24e7966"
ENGINE_N8N_RELEASE_CATALOG_JSON_SHA256 = {
"0.1.2-05e4b38b14b4a019": {
"nodes": "230f712230f7ee8debaa4b44a4358cc19c205bc43305d8ed89d5e3daac466506",
"credentials": "a26824ffc0e15db857c792153de3417febc0dbba4880380d6c8cd544b3c80f4d",
"meta": "caf7635420c61333e65f68350f5567217aed96fb51354b38764bcac2c24e7966",
},
"0.1.3-3354149b5245e39a": {
"nodes": "b0a5215699a6e691b8cfc505ab457d5632ef6d83adb3537520d0b60760ba16b2",
"credentials": "a26824ffc0e15db857c792153de3417febc0dbba4880380d6c8cd544b3c80f4d",
"meta": "b8aa60c0d919573626fa0694a1e4ae9ede015325a5d312e3e05d5ac293084aa7",
},
}
ENGINE_N8N_INACTIVE_NODES_CATALOG_JSON_SHA256 = "b70d9d8130d498c55de46a5d0758c844f1242b70803457b2291ab3a9de8056f2"
ENGINE_N8N_INACTIVE_CREDENTIALS_CATALOG_JSON_SHA256 = "680e9f52aac791efbd38e3bd99bd51ef5cded9756d867897c6c2755850e87b50"
ENGINE_N8N_INACTIVE_META_JSON_SHA256 = "0ac555d7ab31cb0ca042db4f474e83cfefbf20b5bbb2e1509fead27ed21e8acb"
@@ -215,10 +286,10 @@ ENGINE_DATA_PRODUCT_PUBLISH_GRANT_PREDECESSOR_SHA256 = {
"nodedc-source/server/credentialSink/store.js": "c78dc285a973b6acd8a2330f0310935ad08720b2905454492f292d235abf12c0",
"nodedc-source/server/credentialSink/vendor/engine-credential-sink.mjs": "b4800eead9bf94793ff1280d06e34aad6b37ef055a9d7f8d83d7fb5f9bd66d8b",
"nodedc-source/server/index.js": "b2b790b02839570d967a2ca68b00e2724485a99389ac9b3a589a1b22302a36b8",
"nodedc-source/server/routes/engineAgentGateway.js": "e3450a4e1d5318dbac37627b67804d62804e27e2032c9e90478a1e739cea6d3d",
"nodedc-source/server/routes/engineAgentGateway.js": "604a75ad3b9b463cea5e578760c8a23592e2e00b8acfc0099730a347e27bf4e8",
"nodedc-source/server/routes/engineCredentialSink.js": "9cbb69dbc8cbe6181cd5b0170fe9c4d717b0a173866ca98cba3e3766c0bab94e",
"nodedc-source/server/routes/n8n.js": "783d822e2457d82e890f43bc00c7e33822077dc2841f0511a89ecb210fd36d48",
"nodedc-source/server/routes/ndcAgentMcp.js": "fbb3342b1a617b956d5b3a6a40d5111aa107c1176b4ff89c37b104995bada081",
"nodedc-source/server/routes/ndcAgentMcp.js": "534e2c85e1faecc72a00da7ad32d0584ee09b6bd89eeec2221c3b23d80e1e962",
ENGINE_CREDENTIAL_BACKEND_OVERRIDE_TEMPLATE_REL: "944fa64b08255eb8207b93fd327aebb98ecd9400d37d25fcfa8e3a040ee44afe",
}
ENGINE_CREDENTIAL_SINK_CONTRACT_SHA256 = "b4800eead9bf94793ff1280d06e34aad6b37ef055a9d7f8d83d7fb5f9bd66d8b"
@@ -1048,7 +1119,7 @@ def ensure_engine_edp_managed_provisioner_keypair():
return "created"
def ensure_engine_publish_grant_private_state():
def ensure_engine_data_product_grant_private_state(include_reader=False):
# Compose must never create this bind source on our behalf: Docker's normal
# 0755 directory would either expose control-plane metadata or be rejected
# by the Engine's fail-closed PublishGrantStore.
@@ -1060,10 +1131,16 @@ def ensure_engine_publish_grant_private_state():
if stat.S_ISLNK(engine_root_stat.st_mode) or not stat.S_ISDIR(engine_root_stat.st_mode):
die("Engine root is unsafe before private state preparation")
for state_path, label in (
state_paths = [
(engine_root / ENGINE_CONTROL_PLANE_STATE_REL, "Engine control-plane state"),
(engine_root / ENGINE_PUBLISH_GRANT_STATE_REL, "Engine Publish grant state"),
):
]
if include_reader:
state_paths.append((
engine_root / ENGINE_READ_GRANT_STATE_REL,
"Engine Read grant state",
))
for state_path, label in state_paths:
try:
state_stat = state_path.lstat()
except FileNotFoundError:
@@ -1079,6 +1156,11 @@ def ensure_engine_publish_grant_private_state():
die(f"{label} ownership or mode is unsafe: {state_path}")
def ensure_engine_publish_grant_private_state():
# Frozen compatibility seam for the established Publish transition.
return ensure_engine_data_product_grant_private_state(include_reader=False)
def fsync_directory(path):
descriptor = os.open(str(path), os.O_RDONLY | os.O_DIRECTORY)
try:
@@ -1565,6 +1647,8 @@ def allowed_payload_path(component, rel):
return True
if rel == ENGINE_DATA_PRODUCT_PUBLISH_GRANT_OVERRIDE_REL:
return True
if rel == ENGINE_DATA_PRODUCT_READ_GRANT_OVERRIDE_REL:
return True
if rel == ENGINE_NODE_INTELLIGENCE_SERVICE_ROOT_REL or rel.startswith(
ENGINE_NODE_INTELLIGENCE_SERVICE_ROOT_REL + "/"
):
@@ -2158,14 +2242,21 @@ def read_engine_n8n_transition_descriptor(path, label="Engine n8n transition des
"composeOverride": ENGINE_N8N_COMPOSE_OVERRIDE_REL,
"runtimePackagePath": ENGINE_N8N_RUNTIME_PACKAGE_PATH,
"topologyServices": ["n8n"],
"rollbackBaseline": "verified_inactive",
}
for key, expected in exact_common.items():
if descriptor.get(key) != expected:
die(f"Engine n8n transition {key} mismatch")
if action == "activate":
if descriptor.get("expectedCurrent") != "verified_inactive":
die("Engine n8n activation must start from the verified inactive baseline")
expected_current = descriptor.get("expectedCurrent")
if (
expected_current != "verified_inactive"
and expected_current not in ENGINE_N8N_RELEASE_CATALOG_JSON_SHA256
):
die("Engine n8n activation expected-current release is not registered")
if expected_current == release_id:
die("Engine n8n activation target already equals expected current")
if descriptor.get("rollbackBaseline") != expected_current:
die("Engine n8n activation rollback baseline mismatch")
if descriptor.get("expectedNodeTypes") != list(ENGINE_N8N_NODE_TYPES):
die("Engine n8n activation node type set mismatch")
if descriptor.get("expectedCredentialTypes") != list(ENGINE_N8N_CREDENTIAL_TYPES):
@@ -2173,6 +2264,8 @@ def read_engine_n8n_transition_descriptor(path, label="Engine n8n transition des
else:
if descriptor.get("expectedCurrent") != release_id:
die("Engine n8n rollback expected-current release mismatch")
if descriptor.get("rollbackBaseline") != "verified_inactive":
die("Engine n8n inactive rollback baseline mismatch")
if descriptor.get("expectedNodeTypes") != [] or descriptor.get("expectedCredentialTypes") != []:
die("Engine n8n rollback must select the inactive catalog")
return descriptor
@@ -2275,10 +2368,13 @@ def validate_engine_n8n_catalog_payload(payload_dir, descriptor):
die("Engine n8n light icon sha256 mismatch")
if sha256_file(payload_dir / ENGINE_N8N_DARK_ICON_REL) != ENGINE_N8N_DARK_ICON_SHA256:
die("Engine n8n dark icon sha256 mismatch")
release_hashes = ENGINE_N8N_RELEASE_CATALOG_JSON_SHA256.get(descriptor["releaseId"])
if release_hashes is None:
die("Engine n8n activation release catalog is not registered")
expected_catalog_hashes = (
(nodes, ENGINE_N8N_ACTIVATION_NODES_CATALOG_JSON_SHA256, "node"),
(credentials, ENGINE_N8N_ACTIVATION_CREDENTIALS_CATALOG_JSON_SHA256, "credential"),
(meta, ENGINE_N8N_ACTIVATION_META_JSON_SHA256, "metadata"),
(nodes, release_hashes["nodes"], "node"),
(credentials, release_hashes["credentials"], "credential"),
(meta, release_hashes["meta"], "metadata"),
)
else:
expected_catalog_hashes = (
@@ -2683,10 +2779,18 @@ def validate_engine_node_intelligence_transition(payload_dir, entries):
"engine_get_node_guidance",
"engine_validate_node_configuration",
"engine_validate_l2_deep",
"const ENGINE_AGENT_MCP_VERSION = '0.3.0'",
):
if required not in gateway_text:
die(f"Engine node-intelligence gateway contract missing: {required}")
if not any(
version in gateway_text
for version in (
"const ENGINE_AGENT_MCP_VERSION = '0.3.0'",
"const ENGINE_AGENT_MCP_VERSION = '0.4.0'",
"const ENGINE_AGENT_MCP_VERSION = '0.5.0'",
)
):
die("Engine node-intelligence gateway MCP version is not registered")
validate_engine_node_intelligence_image_archive(
payload_dir / ENGINE_NODE_INTELLIGENCE_IMAGE_ARCHIVE_REL,
descriptor,
@@ -2705,6 +2809,245 @@ def validate_engine_node_intelligence_transition(payload_dir, entries):
return descriptor
def is_engine_mcp_control_plane_slice(component, entries):
return (
component == "engine"
and entries is not None
and tuple(entries) == ENGINE_MCP_CONTROL_PLANE_ARTIFACT_ENTRIES
)
def validate_engine_mcp_control_plane_payload(payload_dir, entries):
if tuple(entries) != ENGINE_MCP_CONTROL_PLANE_ARTIFACT_ENTRIES:
die("Engine MCP control-plane files.txt exact set/order mismatch")
for rel, expected_sha256 in ENGINE_MCP_CONTROL_PLANE_TARGET_SHA256.items():
path = payload_dir / rel
if path.is_symlink() or not path.is_file() or sha256_file(path) != expected_sha256:
die(f"Engine MCP control-plane target sha256 mismatch: {rel}")
gateway = (payload_dir / ENGINE_NODE_INTELLIGENCE_GATEWAY_REL).read_text(encoding="utf-8")
graph_route = (payload_dir / "nodedc-source/server/routes/ndcAgentMcp.js").read_text(
encoding="utf-8"
)
installer = (
payload_dir
/ "nodedc-source/server/assets/engine-agent-npm/bin/nodedc-engine-codex-agent.mjs"
).read_text(encoding="utf-8")
package = read_strict_json(
payload_dir / "nodedc-source/server/assets/engine-agent-npm/package.json",
"Engine MCP Codex installer package",
)
if package.get("name") != "@nodedc/engine-codex-agent" or package.get("version") != "0.1.4":
die("Engine MCP Codex installer package identity mismatch")
if any(
marker not in gateway
for marker in (
"const ENGINE_AGENT_MCP_VERSION = '0.5.0'",
"engine_open_l2_change_session",
"engine_get_l2_change_session",
"engine_close_l2_change_session",
"never_infer_expiry",
"three_identical_failures_without_new_evidence",
"engine_plan_data_product_read_grant",
"engine_apply_data_product_read_grant",
"engine_accept_data_product_read_grant",
"engine_rollback_data_product_read_grant",
)
):
die("Engine MCP bounded-change-session contract is incomplete")
if any(
marker not in graph_route
for marker in (
"update_node_no_effect",
"subworkflow_post_write_equality_failed",
"verifiedWrite: true",
"graphDigest: expectedDigest",
)
):
die("Engine MCP graph post-write equality contract is incomplete")
if any(
marker not in installer
for marker in (
"Do not interrupt the user after ordinary recoverable errors",
"Never infer that a provider or managed capability token expired",
"Three identical failures with no new evidence",
"engine_plan_data_product_*_grant",
"durable until an explicit rollback or revoke",
)
):
die("Engine MCP Codex bounded-autonomy policy is incomplete")
combined = "\n".join((gateway, graph_route, installer)).lower()
if "gelios" in combined or "robot2b" in combined:
die("provider identity is forbidden in Engine MCP control-plane source")
archive_path = payload_dir / "nodedc-source/server/assets/nodedc-engine-codex-agent-0.1.4.tgz"
expected_members = {
"package/bin/nodedc-engine-codex-agent.mjs": (
payload_dir
/ "nodedc-source/server/assets/engine-agent-npm/bin/nodedc-engine-codex-agent.mjs"
).read_bytes(),
"package/package.json": (
payload_dir / "nodedc-source/server/assets/engine-agent-npm/package.json"
).read_bytes(),
}
observed_members = {}
try:
with tarfile.open(archive_path, "r:gz") as archive:
for member in archive:
if not (member.isfile() or member.isdir()) or member.name.startswith("/"):
die("Engine MCP Codex installer archive member is unsafe")
if member.isfile():
source = archive.extractfile(member)
if source is None:
die("Engine MCP Codex installer archive member is unreadable")
observed_members[member.name] = source.read(MAX_FILE_BYTES + 1)
except (OSError, tarfile.TarError):
die("Engine MCP Codex installer archive is invalid")
if observed_members != expected_members:
die("Engine MCP Codex installer archive/source equality mismatch")
n8n_route = (payload_dir / "nodedc-source/server/routes/n8n.js").read_text(
encoding="utf-8"
)
if "engineDataProductReadGrantN8nAdapter" not in n8n_route:
die("Engine MCP managed reader native adapter is missing")
if "ndcDataProductReaderApi" not in n8n_route or "ndc_edprb_" not in n8n_route:
die("Engine MCP managed reader credential contract is incomplete")
if "read_grant_must_be_durable" not in n8n_route:
die("Engine MCP managed reader durable lifecycle is missing")
reader_service = (
payload_dir / "nodedc-source/server/dataProductReadGrant/service.js"
).read_text(encoding="utf-8")
if (
"readerGrantLifetime: 'explicit-revoke'" not in reader_service
or "expiresAt: null" not in reader_service
or "ENGINE_READ_GRANT_TTL_DAYS" in reader_service
):
die("Engine MCP managed reader explicit-revoke lifecycle is incomplete")
publish_service = (
payload_dir / "nodedc-source/server/dataProductPublishGrant/service.js"
).read_text(encoding="utf-8")
if (
"writerGrantLifetime: 'explicit-revoke'" not in publish_service
or "migrateCurrentGrantToDurable" not in publish_service
or "expiresAt: null" not in publish_service
or "ENGINE_PUBLISH_GRANT_TTL_DAYS" in publish_service
):
die("Engine MCP managed writer explicit-revoke lifecycle is incomplete")
reader_override = payload_dir / ENGINE_DATA_PRODUCT_READ_GRANT_OVERRIDE_REL
try:
reader_override_text = reader_override.read_text(encoding="utf-8")
except (OSError, UnicodeDecodeError):
die("Engine MCP managed reader runtime override is unreadable")
if reader_override_text != expected_engine_data_product_read_grant_override():
die("Engine MCP managed reader runtime override mismatch")
agent_store = (payload_dir / "nodedc-source/server/engineAgents/store.js").read_text(
encoding="utf-8"
)
for scope in (
"'engine:l2:data-product-read-grant:plan'",
"'engine:l2:data-product-read-grant:write'",
):
if scope not in agent_store:
die(f"Engine MCP managed reader scope is missing: {scope}")
descriptor = read_engine_node_intelligence_descriptor(
payload_dir / ENGINE_MCP_CONTROL_PLANE_DESCRIPTOR_REL,
"Engine MCP control-plane node-intelligence descriptor",
)
if (
descriptor.get("action") != "activate"
or descriptor["source"].get("gatewaySha256")
!= ENGINE_MCP_CONTROL_PLANE_TARGET_SHA256[ENGINE_NODE_INTELLIGENCE_GATEWAY_REL]
or descriptor["source"].get("upstreamProjectionSha256")
!= ENGINE_MCP_CONTROL_PLANE_TARGET_SHA256[
"nodedc-source/server/nodeIntelligence/upstreamProjection.js"
]
):
die("Engine MCP control-plane descriptor target mismatch")
return descriptor
def preflight_engine_mcp_control_plane_predecessor(payload_dir):
candidate = validate_engine_mcp_control_plane_payload(
payload_dir,
ENGINE_MCP_CONTROL_PLANE_ARTIFACT_ENTRIES,
)
installed = current_engine_node_intelligence_descriptor()
validate_installed_engine_node_intelligence_source(installed)
if installed is None or installed.get("action") != "activate":
die("Engine MCP control-plane requires active node intelligence")
expected_candidate = json.loads(json.dumps(installed))
expected_candidate["source"]["gatewaySha256"] = (
ENGINE_MCP_CONTROL_PLANE_TARGET_SHA256[ENGINE_NODE_INTELLIGENCE_GATEWAY_REL]
)
expected_candidate["source"]["upstreamProjectionSha256"] = (
ENGINE_MCP_CONTROL_PLANE_TARGET_SHA256[
"nodedc-source/server/nodeIntelligence/upstreamProjection.js"
]
)
if candidate != expected_candidate:
die("Engine MCP control-plane descriptor crosses node-intelligence source boundary")
root = component_root("engine")
for rel, expected_sha256 in ENGINE_MCP_CONTROL_PLANE_PREDECESSOR_SHA256.items():
path = root / rel
if path.is_symlink() or not path.is_file() or sha256_file(path) != expected_sha256:
die(f"Engine MCP control-plane predecessor drift detected: {rel}")
for rel in ENGINE_MCP_CONTROL_PLANE_NEW_PATHS:
path = root / rel
if path.exists() or path.is_symlink():
die(f"Engine MCP control-plane new path already exists: {rel}")
backend = preflight_engine_credential_backend_runtime()
if backend["mode"] != "verified-derived-retry":
die("Engine MCP control-plane requires the active immutable backend runtime")
return {
"descriptor": candidate,
"predecessor_gateway_sha256": installed["source"]["gatewaySha256"],
"target_gateway_sha256": candidate["source"]["gatewaySha256"],
"backend_mode": backend["mode"],
}
def accept_engine_mcp_control_plane_runtime():
root = component_root("engine")
descriptor = validate_engine_mcp_control_plane_payload(
root,
ENGINE_MCP_CONTROL_PLANE_ARTIFACT_ENTRIES,
)
installed = current_engine_node_intelligence_descriptor()
if installed != descriptor:
die("Engine MCP control-plane installed descriptor equality failed")
validate_installed_engine_node_intelligence_source(installed)
backend = preflight_engine_credential_backend_runtime()
if backend["mode"] != "verified-derived-retry":
die("Engine MCP control-plane backend runtime acceptance failed")
live = run_engine_backend_probe(
(
"node",
"--input-type=module",
"-e",
"""
const module=await import('file:///app/server/routes/engineAgentGateway.js');
const store=await import('file:///app/server/engineAgents/store.js');
const names=new Set(module.engineAgentTools.map((tool)=>tool.name));
const required=['engine_plan_data_product_read_grant','engine_apply_data_product_read_grant','engine_accept_data_product_read_grant','engine_rollback_data_product_read_grant'];
const scopes=['engine:l2:data-product-read-grant:plan','engine:l2:data-product-read-grant:write'];
if(!required.every((name)=>names.has(name))||!scopes.every((scope)=>store.ENGINE_AGENT_SCOPES.includes(scope)))process.exit(2);
process.stdout.write('engine-mcp-reader-grant:0.5.0:'+required.length+':'+scopes.length);
""".strip(),
),
"Engine MCP managed reader capability",
container_id=engine_backend_container_id(),
)
if live != "engine-mcp-reader-grant:0.5.0:4:2":
die("Engine MCP managed reader live capability acceptance mismatch")
return {
"gateway_sha256": descriptor["source"]["gatewaySha256"],
"backend_mode": backend["mode"],
"reader_grant": live,
}
def validate_no_lifecycle_scripts(payload_dir, label):
forbidden = ("preinstall", "install", "postinstall", "prepare", "prepack", "postpack")
for package_path in payload_dir.rglob("package.json"):
@@ -2800,6 +3143,22 @@ def expected_engine_data_product_publish_grant_override():
))
def expected_engine_data_product_read_grant_override():
return "\n".join((
"services:",
" nodedc-backend:",
" environment:",
f" ENGINE_CONTROL_PLANE_READ_GRANT_ROOT: {ENGINE_READ_GRANT_CONTAINER_PATH}",
" volumes:",
" - type: bind",
f" source: {ENGINE_READ_GRANT_STATE_PATH}",
f" target: {ENGINE_READ_GRANT_CONTAINER_PATH}",
" bind:",
" create_host_path: false",
"",
))
def validate_engine_credential_sink_slice(payload_dir, entries):
if tuple(entries) != ENGINE_CREDENTIAL_SINK_ARTIFACT_ENTRIES:
die("Engine credential sink files.txt exact set/order mismatch")
@@ -3103,20 +3462,30 @@ def load_artifact(artifact, work_dir):
or rel.startswith(ENGINE_NODE_INTELLIGENCE_SERVICE_ROOT_REL + "/")
for rel in entries
)
if touches_node_intelligence and not is_engine_node_intelligence_transition(
manifest["component"], entries
if (
touches_node_intelligence
and not is_engine_node_intelligence_transition(manifest["component"], entries)
and not is_engine_mcp_control_plane_slice(manifest["component"], entries)
):
die("Engine node-intelligence payload requires the canonical transition")
if is_engine_node_intelligence_transition(manifest["component"], entries):
validate_engine_node_intelligence_transition(payload_dir, entries)
if is_engine_mcp_control_plane_slice(manifest["component"], entries):
validate_engine_mcp_control_plane_payload(payload_dir, entries)
if touches_engine_credential_sink(manifest["component"], entries):
validate_engine_credential_sink_slice(payload_dir, entries)
if (
touches_engine_data_product_publish_grant(entries)
or ENGINE_DATA_PRODUCT_PUBLISH_GRANT_OVERRIDE_REL in entries
not is_engine_mcp_control_plane_slice(manifest["component"], entries)
and (
touches_engine_data_product_publish_grant(entries)
or ENGINE_DATA_PRODUCT_PUBLISH_GRANT_OVERRIDE_REL in entries
)
):
validate_engine_data_product_publish_grant_slice(payload_dir, entries)
elif ENGINE_AGENT_FULL_GRANT_MIGRATION_STORE_REL in entries:
elif (
not is_engine_mcp_control_plane_slice(manifest["component"], entries)
and ENGINE_AGENT_FULL_GRANT_MIGRATION_STORE_REL in entries
):
validate_engine_agent_full_grant_migration_slice(payload_dir, entries)
return manifest, entries, payload_dir
@@ -3202,6 +3571,7 @@ def touches_external_data_plane_files(entries):
"platform/packages/external-provider-contract/src/data-plane.mjs",
"platform/packages/external-provider-contract/src/data-product.mjs",
"platform/packages/external-provider-contract/src/intake-batch.mjs",
"platform/packages/external-provider-contract/src/index.mjs",
"platform/packages/external-provider-contract/src/sensitive-field-policy.mjs",
}
return any(
@@ -3462,6 +3832,12 @@ def is_platform_provider_catalog_only(entries):
def component_services(component, entries=None):
if is_engine_mcp_control_plane_slice(component, entries):
# This slice updates only the existing Engine backend control plane.
# Node intelligence keeps the same immutable sidecar image and n8n/L1
# retain their current generations.
return ("nodedc-backend",)
if is_engine_node_intelligence_transition(component, entries):
if tuple(entries) == ENGINE_NODE_INTELLIGENCE_ARTIFACT_ENTRIES:
return (ENGINE_NODE_INTELLIGENCE_SERVICE, "nodedc-backend")
@@ -3653,6 +4029,18 @@ def component_compose_files(component, allow_prepared_engine_backend=False):
# Canonical order is immutable: base, active L2 extension, credential
# sink, then the additive Publish grant overlay.
files.append(publish_grant_override)
read_grant_override = root / ENGINE_DATA_PRODUCT_READ_GRANT_OVERRIDE_REL
if read_grant_override.exists() or read_grant_override.is_symlink():
if read_grant_override.is_symlink() or not read_grant_override.is_file():
die("installed Engine data product read grant override is unsafe")
try:
read_grant_override_text = read_grant_override.read_text(encoding="utf-8")
except (OSError, UnicodeDecodeError):
die("installed Engine data product read grant override cannot be read")
if read_grant_override_text != expected_engine_data_product_read_grant_override():
die("installed Engine data product read grant override drift detected")
# Read authority is additive and ordered after the writer/private-key overlay.
files.append(read_grant_override)
node_intelligence_descriptor = current_engine_node_intelligence_descriptor()
if (
node_intelligence_descriptor is not None
@@ -4164,6 +4552,22 @@ def validate_engine_backend_mounts_for_installed_runtime(
str(ENGINE_EDP_MANAGED_PROVISIONER_PRIVATE_KEY_FILE),
),
}
read_override = component_root("engine") / ENGINE_DATA_PRODUCT_READ_GRANT_OVERRIDE_REL
read_installed = read_override.exists() or read_override.is_symlink()
if read_installed:
if read_override.is_symlink() or not read_override.is_file():
die("installed Engine data product read grant override is unsafe")
try:
read_override_text = read_override.read_text(encoding="utf-8")
except (OSError, UnicodeDecodeError):
die("installed Engine data product read grant override cannot be read")
if read_override_text != expected_engine_data_product_read_grant_override():
die("installed Engine data product read grant override drift detected")
expected_publish_mounts[ENGINE_READ_GRANT_CONTAINER_PATH] = (
"bind",
True,
str(ENGINE_READ_GRANT_STATE_PATH),
)
mounts = container.get("Mounts")
if not isinstance(mounts, list):
die("Engine backend mount inventory is missing")
@@ -5306,10 +5710,6 @@ def preflight_engine_n8n_transition(descriptor, enforce_expected_current):
"Engine n8n transition stale current state: "
f"expected={descriptor['expectedCurrent']} actual={current_state}"
)
if current_descriptor and current_state != "verified_inactive":
if current_descriptor.get("packageSha256") != descriptor["packageSha256"]:
if enforce_expected_current:
die("Engine n8n current release package sha256 mismatch")
return {
"base_image_id": base_image["id"],
"base_image_architecture": base_image["architecture"],
@@ -5558,6 +5958,7 @@ def plan_artifact(artifact):
sha = sha256_file(artifact)
publish_grant_preflight = None
node_intelligence_preflight = None
mcp_control_plane_preflight = None
with tempfile.TemporaryDirectory(prefix="plan-", dir=TMP_DIR) as tmp:
manifest, entries, payload_dir = load_artifact(artifact, Path(tmp))
transition_descriptor = None
@@ -5578,6 +5979,10 @@ def plan_artifact(artifact):
node_intelligence_preflight = preflight_engine_node_intelligence_predecessor(
payload_dir
)
if is_engine_mcp_control_plane_slice(manifest["component"], entries):
mcp_control_plane_preflight = preflight_engine_mcp_control_plane_predecessor(
payload_dir
)
component = manifest["component"]
root = component_root(component)
@@ -5644,6 +6049,22 @@ def plan_artifact(artifact):
print("node_intelligence_host_ports=none")
print(f"predecessor_gateway_sha256={node_intelligence_preflight['gateway_sha256']}")
print(f"backend_current_barrier={node_intelligence_preflight['backend_mode']}")
if mcp_control_plane_preflight:
print("engine_mcp_transition=managed-reader-grant+validator-reconciliation")
print("engine_mcp_version=0.5.0")
print("engine_mcp_installer=0.1.4")
print("engine_mcp_reader_grant=plan+apply+accept+rollback")
print(
"predecessor_gateway_sha256="
f"{mcp_control_plane_preflight['predecessor_gateway_sha256']}"
)
print(
"target_gateway_sha256="
f"{mcp_control_plane_preflight['target_gateway_sha256']}"
)
print(f"backend_current_barrier={mcp_control_plane_preflight['backend_mode']}")
print("node_intelligence_image=preserved")
print("n8n_l1=untouched")
if transition_descriptor:
print(f"n8n_transition={transition_descriptor['action']}")
print(f"n8n_version={transition_descriptor['n8nVersion']}")
@@ -5665,7 +6086,7 @@ def plan_artifact(artifact):
print("n8n_database=preserved:n8n-postgres")
print(f"n8n_expected_node_types={','.join(transition_descriptor['expectedNodeTypes'])}")
print(f"n8n_expected_credential_types={','.join(transition_descriptor['expectedCredentialTypes'])}")
print("n8n_rollback_baseline=verified_inactive")
print(f"n8n_rollback_baseline={transition_descriptor['rollbackBaseline']}")
touches_map_gateway = component == "platform" and any(rel == "platform/services/map-gateway" or rel.startswith("platform/services/map-gateway/") for rel in entries)
touches_external_data_plane = component == "platform" and touches_external_data_plane_files(entries)
if component == "module-foundry" or touches_map_gateway:
@@ -6387,6 +6808,10 @@ def prepare_component_runtime(component, entries=None):
ensure_engine_edp_managed_provisioner_keypair()
ensure_engine_publish_grant_private_state()
if is_engine_mcp_control_plane_slice(component, entries):
ensure_engine_edp_managed_provisioner_keypair()
ensure_engine_data_product_grant_private_state(include_reader=True)
if is_engine_n8n_transition(component, entries):
descriptor = current_engine_n8n_transition_descriptor()
if descriptor is None:
@@ -6553,6 +6978,20 @@ def external_data_plane_healthcheck(require_managed=True):
}
if require_managed:
expected_json["managedWriterBindingProvisioning"] = "enabled"
server_source = (
component_root("platform")
/ "platform/services/external-data-plane/src/server.mjs"
)
try:
server_text = server_source.read_text(encoding="utf-8")
except (OSError, UnicodeDecodeError):
server_text = ""
if 'managedReaderBindingProvisioning: config.managedProvisionerApiEnabled' in server_text:
expected_json["managedReaderBindingProvisioning"] = "enabled"
if 'managedWriterBindingLifetime: config.managedProvisionerApiEnabled ? "explicit-revoke" : "disabled"' in server_text:
expected_json["managedWriterBindingLifetime"] = "explicit-revoke"
if 'managedReaderBindingLifetime: config.managedProvisionerApiEnabled ? "explicit-revoke" : "disabled"' in server_text:
expected_json["managedReaderBindingLifetime"] = "explicit-revoke"
return {
"url": "http://127.0.0.1:18106/healthz",
"expected_json": expected_json,
@@ -6576,6 +7015,7 @@ def component_healthchecks(component, entries=None, services=None):
(
is_engine_data_product_publish_grant_slice(component, entries)
or is_engine_agent_full_grant_migration_slice(component, entries)
or is_engine_mcp_control_plane_slice(component, entries)
)
and services is not None
):
@@ -6888,10 +7328,12 @@ def run_healthchecks(component, entries=None, services=None):
component,
entries,
)
touches_mcp_control_plane = is_engine_mcp_control_plane_slice(component, entries)
if (
touches_engine_credential_sink(component, entries)
or touches_publish_grant
or touches_agent_grant_migration
or touches_mcp_control_plane
):
# The HTTP endpoint can become ready before Docker publishes the first
# successful health probe. Wait for the Compose health barrier before
@@ -6903,12 +7345,15 @@ def run_healthchecks(component, entries=None, services=None):
touches_engine_credential_sink(component, entries)
or touches_publish_grant
or touches_agent_grant_migration
or touches_mcp_control_plane
):
runtime = preflight_engine_credential_backend_runtime()
if runtime["mode"] != "verified-derived-retry":
die("Engine backend immutable runtime activation acceptance failed")
if touches_agent_grant_migration:
accept_engine_agent_full_grant_migration_runtime()
if touches_mcp_control_plane:
accept_engine_mcp_control_plane_runtime()
container_name = COMPONENTS[component].get("health_container")
if container_name:
healthcheck_container(container_name)
@@ -6983,6 +7428,8 @@ def apply_artifact(artifact):
payload_dir
)
node_intelligence_descriptor = node_intelligence_preflight["descriptor"]
if is_engine_mcp_control_plane_slice(component, entries):
preflight_engine_mcp_control_plane_predecessor(payload_dir)
if not artifact_only and not compose_root.is_dir():
if bootstrap_root and is_relative_to(compose_root.resolve(strict=False), root.resolve(strict=False)):
compose_root.mkdir(parents=True, exist_ok=True)
@@ -7159,6 +7606,7 @@ def apply_artifact(artifact):
(
is_engine_data_product_publish_grant_slice(component, entries)
or is_engine_agent_full_grant_migration_slice(component, entries)
or is_engine_mcp_control_plane_slice(component, entries)
)
and services is not None
):
@@ -0,0 +1,198 @@
#!/usr/bin/env python3
import hashlib
import importlib.machinery
import importlib.util
import json
import os
import subprocess
import tarfile
import tempfile
import unittest
from unittest import mock
from pathlib import Path
SCRIPT_DIR = Path(__file__).resolve().parent
PLATFORM_ROOT = SCRIPT_DIR.parent.parent
RUNNER_PATH = SCRIPT_DIR / "nodedc-deploy"
BUILDER_PATH = SCRIPT_DIR / "build-engine-mcp-control-plane-artifact.mjs"
TRANSITION_ID = "20260718-003"
STAGE_ARTIFACT = (
PLATFORM_ROOT
/ "infra/deploy-artifacts"
/ "nodedc-engine-mcp-control-plane-20260718-003.tgz"
)
STAGE_ARTIFACT_SHA256 = "249aef9527666c562e9648b15737e66cc5c1dc7c0788b58ea714da270b5eb4ba"
def load_runner():
loader = importlib.machinery.SourceFileLoader("nodedc_engine_mcp_runner", str(RUNNER_PATH))
spec = importlib.util.spec_from_loader(loader.name, loader)
module = importlib.util.module_from_spec(spec)
loader.exec_module(module)
return module
RUNNER = load_runner()
class EngineMcpControlPlaneTest(unittest.TestCase):
@classmethod
def setUpClass(cls):
cls.temporary = tempfile.TemporaryDirectory(prefix="nodedc-engine-mcp-control-plane-")
cls.root = Path(cls.temporary.name)
stage_bytes = STAGE_ARTIFACT.read_bytes()
if hashlib.sha256(stage_bytes).hexdigest() != STAGE_ARTIFACT_SHA256:
raise RuntimeError("engine_mcp_control_plane_stage_artifact_sha256_mismatch")
source_stage = cls.root / "immutable-source"
source_stage.mkdir()
RUNNER.safe_extract(STAGE_ARTIFACT, source_stage)
cls.engine_source_root = source_stage / "payload"
cls.results = []
for index in range(2):
output = cls.root / f"build-{index}"
output.mkdir()
env = os.environ.copy()
env["NODEDC_DEPLOY_ARTIFACT_DIR"] = str(output)
env["NODEDC_ENGINE_SOURCE_ROOT"] = str(cls.engine_source_root)
result = subprocess.run(
["node", str(BUILDER_PATH), TRANSITION_ID],
cwd=PLATFORM_ROOT,
env=env,
check=True,
capture_output=True,
text=True,
)
cls.results.append(json.loads(result.stdout))
@classmethod
def tearDownClass(cls):
cls.temporary.cleanup()
def artifact(self, index=0):
return Path(self.results[index]["artifact"])
def extract(self, artifact, directory):
RUNNER.safe_extract(artifact, directory)
return directory / "payload", RUNNER.parse_files_list(directory / "files.txt")
def test_artifact_is_byte_reproducible_and_canonical(self):
first = self.artifact(0).read_bytes()
second = self.artifact(1).read_bytes()
self.assertEqual(first, second)
self.assertEqual(first, STAGE_ARTIFACT.read_bytes())
self.assertEqual(first[4:8], b"\0\0\0\0")
self.assertEqual(self.results[0]["artifactSha256"], hashlib.sha256(first).hexdigest())
with tarfile.open(self.artifact(0), "r:gz") as archive:
for member in archive:
self.assertTrue(member.isfile() or member.isdir())
self.assertFalse(Path(member.name).name.startswith("._"))
def test_runner_selects_only_existing_backend(self):
with tempfile.TemporaryDirectory() as directory:
manifest, entries, payload = RUNNER.load_artifact(
self.artifact(0),
Path(directory),
)
descriptor = RUNNER.validate_engine_mcp_control_plane_payload(payload, entries)
override_text = (
payload
/ RUNNER.ENGINE_DATA_PRODUCT_READ_GRANT_OVERRIDE_REL
).read_text(encoding="utf-8")
self.assertEqual(manifest["component"], "engine")
self.assertEqual(tuple(entries), RUNNER.ENGINE_MCP_CONTROL_PLANE_ARTIFACT_ENTRIES)
self.assertEqual(RUNNER.component_services("engine", entries), ("nodedc-backend",))
self.assertEqual(descriptor["action"], "activate")
self.assertEqual(
descriptor["source"]["gatewaySha256"],
RUNNER.ENGINE_MCP_CONTROL_PLANE_TARGET_SHA256[
RUNNER.ENGINE_NODE_INTELLIGENCE_GATEWAY_REL
],
)
self.assertEqual(override_text, RUNNER.expected_engine_data_product_read_grant_override())
def test_runtime_preparation_adds_only_private_reader_state(self):
with (
mock.patch.object(
RUNNER,
"ensure_engine_edp_managed_provisioner_keypair",
) as keypair,
mock.patch.object(
RUNNER,
"ensure_engine_data_product_grant_private_state",
) as private_state,
):
RUNNER.prepare_component_runtime(
"engine",
RUNNER.ENGINE_MCP_CONTROL_PLANE_ARTIFACT_ENTRIES,
)
keypair.assert_called_once_with()
private_state.assert_called_once_with(include_reader=True)
def test_candidate_preserves_node_intelligence_identity_except_exact_control_plane_digests(self):
with tempfile.TemporaryDirectory() as directory:
work = Path(directory)
payload, entries = self.extract(self.artifact(0), work)
candidate = RUNNER.validate_engine_mcp_control_plane_payload(payload, entries)
self.assertEqual(candidate["releaseId"], RUNNER.ENGINE_NODE_INTELLIGENCE_RELEASE_ID)
self.assertEqual(candidate["upstream"]["commit"], RUNNER.ENGINE_NODE_INTELLIGENCE_UPSTREAM_COMMIT)
self.assertEqual(candidate["image"]["tag"], RUNNER.ENGINE_NODE_INTELLIGENCE_IMAGE)
self.assertEqual(candidate["expectedCurrent"], "inactive")
self.assertEqual(
candidate["source"]["upstreamProjectionSha256"],
RUNNER.ENGINE_MCP_CONTROL_PLANE_TARGET_SHA256[
"nodedc-source/server/nodeIntelligence/upstreamProjection.js"
],
)
def test_descriptor_boundary_and_source_digest_are_enforced(self):
with tempfile.TemporaryDirectory() as directory:
work = Path(directory)
payload, entries = self.extract(self.artifact(0), work)
descriptor_path = payload / RUNNER.ENGINE_MCP_CONTROL_PLANE_DESCRIPTOR_REL
descriptor = json.loads(descriptor_path.read_text(encoding="utf-8"))
descriptor["source"]["catalogSha256"] = "0" * 64
descriptor_path.write_text(json.dumps(descriptor), encoding="utf-8")
# Payload validation permits only a structurally valid descriptor;
# predecessor equality is the state-aware barrier for untouched
# node-intelligence source identities.
with self.assertRaisesRegex(
RUNNER.DeployError,
"descriptor crosses node-intelligence source boundary",
):
installed = json.loads(json.dumps(descriptor))
installed["source"]["catalogSha256"] = "1" * 64
installed["source"]["gatewaySha256"] = (
RUNNER.ENGINE_MCP_CONTROL_PLANE_PREDECESSOR_SHA256[
RUNNER.ENGINE_NODE_INTELLIGENCE_GATEWAY_REL
]
)
from unittest import mock
with (
mock.patch.object(
RUNNER,
"current_engine_node_intelligence_descriptor",
return_value=installed,
),
mock.patch.object(RUNNER, "validate_installed_engine_node_intelligence_source"),
):
RUNNER.preflight_engine_mcp_control_plane_predecessor(payload)
def test_existing_artifact_is_not_overwritten(self):
env = os.environ.copy()
env["NODEDC_DEPLOY_ARTIFACT_DIR"] = str(self.artifact(0).parent)
env["NODEDC_ENGINE_SOURCE_ROOT"] = str(self.engine_source_root)
result = subprocess.run(
["node", str(BUILDER_PATH), TRANSITION_ID],
cwd=PLATFORM_ROOT,
env=env,
check=False,
capture_output=True,
text=True,
)
self.assertNotEqual(result.returncode, 0)
self.assertIn("artifact_already_exists", result.stderr)
if __name__ == "__main__":
unittest.main(verbosity=2)
@@ -21,11 +21,12 @@ BUILDER_PATH = SCRIPT_DIR / "build-engine-n8n-private-extension-artifact.mjs"
STAGE_ARTIFACT = (
PLATFORM_ROOT
/ "infra/deploy-artifacts"
/ "nodedc-n8n-private-extension-n8n-nodes-ndc-release-20260716-003.tgz"
/ "nodedc-n8n-private-extension-n8n-nodes-ndc-history-read-20260717-004.tgz"
)
TRANSITION_ID = "20260717-004"
SEALED_RELEASE_ID = "0.1.2-05e4b38b14b4a019"
SEALED_PACKAGE_SHA256 = "05e4b38b14b4a019ce1f6eee27b9e320094cb3560903bd68074966b3a1267af5"
TRANSITION_ID = "20260717-005"
SEALED_RELEASE_ID = "0.1.3-3354149b5245e39a"
SEALED_PACKAGE_SHA256 = "3354149b5245e39a10f6f03a4b43796602d7e57ea1f91dcc6cb6774ead97501d"
PREDECESSOR_RELEASE_ID = "0.1.2-05e4b38b14b4a019"
BUILDER_ENGINE_PATHS = (
"nodedc-source/server/assets/n8n/schema/v2.3.2/nodes.catalog.json",
"nodedc-source/server/assets/n8n/schema/v2.3.2/credentials.catalog.json",
@@ -50,7 +51,7 @@ PRIVATE_EXTENSION_CANON_FUNCTION_SHA256 = {
"engine_n8n_package_loader_probe_script": "2b3b3c96fad6e5e48ebea74426b21bc7eea6b6fe0e786b537da3366d18aedd9b",
"engine_n8n_private_loader_catalog": "a2ae389b4ef215900cf967b863d01056bd2a8eec55554566d0f6005e4e0bcbac",
"validate_engine_n8n_base_compose_source": "adacc5b20e52684cbc427362ddba5a6d2e13ef2091a89859e2b8f7bbf2e20458",
"preflight_engine_n8n_transition": "da3cec853e3e3e20df4d1e301aab98b77815f93ec3d68b0b61c84c6e6a3fb335",
"preflight_engine_n8n_transition": "3d574acb081fb752e45cc0c3286d3ae4596dcee83a85e2c3fee81ac65ec796e0",
"accept_engine_n8n_runtime": "1a2614a465161e3833e84aca402817682da1a538bdb2aecc4af66b665497338d",
}
@@ -138,7 +139,7 @@ class EngineN8nPrivateExtensionTest(unittest.TestCase):
self.assertEqual(first[4:8], b"\0\0\0\0")
self.assertEqual(first[3] & 0x08, 0)
def test_successful_generation_003_runner_functions_are_frozen(self):
def test_state_aware_upgrade_runner_functions_are_frozen(self):
actual = {
name: hashlib.sha256(inspect.getsource(getattr(RUNNER, name)).encode("utf-8")).hexdigest()
for name in PRIVATE_EXTENSION_CANON_FUNCTION_SHA256
@@ -164,8 +165,8 @@ class EngineN8nPrivateExtensionTest(unittest.TestCase):
def test_transition_id_rejects_missing_invalid_and_previously_issued_values(self):
cases = (
([], "transition_id_required"),
(["20260716-003"], "transition_id_already_issued"),
(["engine-n8n-private-extension-20260717-004"], "transition_id_invalid"),
(["20260717-004"], "transition_id_already_issued"),
(["engine-n8n-private-extension-20260717-005"], "transition_id_invalid"),
(["20260230-004"], "transition_id_invalid"),
(["20260717-000"], "transition_id_invalid"),
([TRANSITION_ID, "unexpected-second-id"], "transition_id_argument_count_invalid"),
@@ -202,7 +203,7 @@ class EngineN8nPrivateExtensionTest(unittest.TestCase):
self.assertIn("transition_artifact_already_exists", result.stderr)
def test_activation_and_rollback_pass_strict_runner_policy(self):
expected = {"activation": ("activate", 7), "rollback": ("rollback-inactive", 4)}
expected = {"activation": ("activate", 7), "rollback": ("activate", 7)}
for kind, (action, entry_count) in expected.items():
with self.subTest(kind=kind), tempfile.TemporaryDirectory() as directory:
manifest, entries, payload = RUNNER.load_artifact(
@@ -234,7 +235,7 @@ class EngineN8nPrivateExtensionTest(unittest.TestCase):
self.assertTrue(all("usableAsTool" not in item for item in private_nodes))
self.assertEqual((len(nodes), len(credentials)), (437, 388))
def test_rollback_catalog_restores_verified_inactive_baseline(self):
def test_rollback_catalog_restores_verified_predecessor_release(self):
with tempfile.TemporaryDirectory() as directory:
_manifest, _entries, payload = RUNNER.load_artifact(
self.artifact(0, "rollback"),
@@ -242,9 +243,20 @@ class EngineN8nPrivateExtensionTest(unittest.TestCase):
)
nodes = json.loads((payload / RUNNER.ENGINE_N8N_NODES_CATALOG_REL).read_text())
credentials = json.loads((payload / RUNNER.ENGINE_N8N_CREDENTIALS_CATALOG_REL).read_text())
self.assertEqual([item for item in nodes if item.get("name", "").startswith("n8n-nodes-ndc.")], [])
self.assertEqual([item for item in credentials if item.get("name") in EXPECTED_CREDENTIALS], [])
self.assertEqual((len(nodes), len(credentials)), (434, 385))
descriptor = RUNNER.read_engine_n8n_transition_descriptor(
payload / RUNNER.ENGINE_N8N_TRANSITION_DESCRIPTOR_REL
)
self.assertEqual(descriptor["releaseId"], PREDECESSOR_RELEASE_ID)
self.assertEqual(descriptor["expectedCurrent"], SEALED_RELEASE_ID)
self.assertEqual(
[item.get("name") for item in nodes if item.get("name", "").startswith("n8n-nodes-ndc.")],
EXPECTED_NODES,
)
self.assertEqual(
[item.get("name") for item in credentials if item.get("name") in EXPECTED_CREDENTIALS],
EXPECTED_CREDENTIALS,
)
self.assertEqual((len(nodes), len(credentials)), (437, 388))
def test_compose_override_is_runner_derived_and_offline(self):
with tempfile.TemporaryDirectory() as directory:
@@ -259,7 +271,7 @@ class EngineN8nPrivateExtensionTest(unittest.TestCase):
self.assertEqual(override, RUNNER.expected_engine_n8n_compose_override(descriptor))
self.assertEqual(
hashlib.sha256(override.encode("utf-8")).hexdigest(),
"a29e2f92b87dda59da2b4e3ce250bc9705ed9fa551c0fbe4e95464fca5caa3e1",
"256142c14ae295bf9be4d74fdc8dcd6e134e4ded8a264f15e66b38e85503b589",
)
self.assertIn("pull_policy: never", override)
self.assertIn("N8N_USER_FOLDER: /home/node", override)
@@ -268,7 +280,7 @@ class EngineN8nPrivateExtensionTest(unittest.TestCase):
self.assertNotIn("N8N_CUSTOM_EXTENSIONS", override)
self.assertNotIn("build:", override)
def test_generation_003_override_is_accepted_as_installed_canon(self):
def test_upgrade_override_is_accepted_as_installed_canon(self):
original_root = RUNNER.COMPONENTS["engine"]["payload_root"]
original_compose = RUNNER.COMPONENTS["engine"]["compose_root"]
try:
@@ -551,7 +563,7 @@ class EngineN8nPrivateExtensionTest(unittest.TestCase):
def test_sealed_release_exact_set_includes_implicit_directories(self):
release_relative = Path(
"payload/releases/n8n-nodes-ndc/0.1.2-05e4b38b14b4a019"
"payload/releases/n8n-nodes-ndc/0.1.3-3354149b5245e39a"
)
with tempfile.TemporaryDirectory() as directory:
work = Path(directory)
@@ -19,6 +19,7 @@ EXPECTED_ENTRIES = [
"platform/packages/external-provider-contract/src/data-plane.mjs",
"platform/packages/external-provider-contract/src/data-product.mjs",
"platform/packages/external-provider-contract/src/intake-batch.mjs",
"platform/packages/external-provider-contract/src/index.mjs",
"platform/packages/external-provider-contract/src/sensitive-field-policy.mjs",
]
@@ -17,7 +17,7 @@ SCRIPT_DIR = Path(__file__).resolve().parent
PLATFORM_ROOT = SCRIPT_DIR.parent.parent
RUNNER_PATH = SCRIPT_DIR / "nodedc-deploy"
BUILDER_PATH = SCRIPT_DIR / "build-n8n-private-extension-artifact.mjs"
PATCH_ID = "n8n-nodes-ndc-release-20260716-003"
PATCH_ID = "n8n-nodes-ndc-history-read-20260717-004"
EXPECTED_NODES = [
"dist/nodes/NdcDataProductPublish/NdcDataProductPublish.node.js",
"dist/nodes/NdcDataProductRead/NdcDataProductRead.node.js",
@@ -106,7 +106,7 @@ class N8nPrivateExtensionPolicyTest(unittest.TestCase):
manifest, entries, _payload = RUNNER.load_artifact(self.artifacts[0], Path(directory))
self.assertEqual(manifest["component"], "n8n-private-extension")
self.assertEqual(len(entries), 1)
self.assertRegex(entries[0], r"^releases/n8n-nodes-ndc/0\.1\.2-[a-f0-9]{16}$")
self.assertRegex(entries[0], r"^releases/n8n-nodes-ndc/0\.1\.3-[a-f0-9]{16}$")
with tarfile.open(self.artifacts[0], "r:gz") as archive:
names = archive.getnames()
@@ -118,7 +118,7 @@ class N8nPrivateExtensionPolicyTest(unittest.TestCase):
with tarfile.open(fileobj=io.BytesIO(package), mode="r:gz") as archive:
package_json_member = archive.getmember("package/package.json")
package_json = json.loads(archive.extractfile(package_json_member).read())
self.assertEqual(package_json["version"], "0.1.2")
self.assertEqual(package_json["version"], "0.1.3")
self.assertEqual(package_json["n8n"]["nodes"], EXPECTED_NODES)
self.assertEqual(package_json["n8n"]["credentials"], EXPECTED_CREDENTIALS)
for node_path in EXPECTED_NODES:
@@ -154,7 +154,7 @@ class N8nPrivateExtensionPolicyTest(unittest.TestCase):
"requiresPreActivationVerification": True,
}
self.assertEqual(release["schemaVersion"], "nodedc.n8n-private-extension-release/v2")
self.assertEqual(release["package"]["version"], "0.1.2")
self.assertEqual(release["package"]["version"], "0.1.3")
self.assertEqual(release["activation"]["rollbackBaselinePolicy"], expected_policy)
self.assertEqual(rollback["schemaVersion"], "nodedc.n8n-private-extension-rollback/v2")
self.assertEqual(rollback["baselinePolicy"], expected_policy)