feat(platform): add managed data product history plane

This commit is contained in:
Codex 2026-07-18 14:38:06 +03:00
parent 02816c4352
commit 3c5d8f6cef
34 changed files with 2091 additions and 163 deletions

View File

@ -90,10 +90,11 @@ node descriptions used `usableAsTool`, so n8n 2.3.2 exposed six NDC runtime
types instead of the required three. It must not be activated, overwritten or types instead of the required three. It must not be activated, overwritten or
deleted. deleted.
The corrected candidate is package version `0.1.2`, built with patch id The active predecessor is package version `0.1.2` at immutable release
`n8n-nodes-ndc-release-20260716-003`. It receives a new digest-bound release `0.1.2-05e4b38b14b4a019`. History Read is package version `0.1.3`, built with
directory and remains inert after staging; only a separately reviewed patch id `n8n-nodes-ndc-history-read-20260717-004` and immutable release
Engine-owned activator may select it after exact MCP schema acceptance. `0.1.3-3354149b5245e39a`. Staging remains inert; only the separately reviewed
Engine-owned transition may select it after exact MCP schema acceptance.
The paired Engine activation is built by The paired Engine activation is built by
`build-engine-n8n-private-extension-artifact.mjs`. It deliberately does not `build-engine-n8n-private-extension-artifact.mjs`. It deliberately does not
@ -102,7 +103,7 @@ not build an image. A fresh transition id is mandatory and previously issued
ids are rejected: ids are rejected:
```bash ```bash
node infra/deploy-runner/build-engine-n8n-private-extension-artifact.mjs 20260717-004 node infra/deploy-runner/build-engine-n8n-private-extension-artifact.mjs 20260717-005
``` ```
The builder emits a narrowly scoped Compose override plus a strict transition The builder emits a narrowly scoped Compose override plus a strict transition
@ -112,7 +113,7 @@ same immutable image ID, and extracts the package into the root-owned,
read-only Engine release tree: read-only Engine release tree:
```text ```text
/volume2/nodedc-demo/n8n-private-extensions/releases/n8n-nodes-ndc/0.1.2-05e4b38b14b4a019/package /volume2/nodedc-demo/n8n-private-extensions/releases/n8n-nodes-ndc/0.1.3-3354149b5245e39a/package
``` ```
The override sets `N8N_USER_FOLDER=/home/node`, which is required because the The override sets `N8N_USER_FOLDER=/home/node`, which is required because the
@ -134,13 +135,16 @@ Postgres service, `.n8n` data, encryption key and credentials remain intact.
The apply gate verifies readiness, the running image/version, sealed mount, The apply gate verifies readiness, the running image/version, sealed mount,
loader environment, package-loader node/credential sets, scoped loader logs, loader environment, package-loader node/credential sets, scoped loader logs,
restart stability and content-exact pinned Engine MCP catalogs. The runner pins restart stability and content-exact pinned Engine MCP catalogs. The runner pins
both the complete 434/385 inactive baseline and the reviewed 437/388 the complete 434/385 inactive baseline and a digest registry for every
activation catalogs, so a same-count substitution of any built-in schema is reviewed 437/388 active release, so a same-count substitution of any built-in
rejected. Any gate failure after or private schema is rejected. Upgrade `0.1.2 -> 0.1.3` is accepted only when
the verified live predecessor, descriptor, package mount and catalog all agree.
Any gate failure after
mutation automatically restores the pre-apply catalogs/descriptor and mutation automatically restores the pre-apply catalogs/descriptor and
force-recreates the previous verified runtime. Staged, sealed and failed force-recreates the previous verified runtime. Staged, sealed and failed
releases are retained. The separate rollback artifact returns the first releases are retained. The paired rollback artifact returns `0.1.3` to the
activation to the verified inactive 434-node/385-credential catalog baseline. verified immutable `0.1.2` release and its exact catalogs; it does not invent
an inactive baseline for an already-active upgrade.
Run both policy suites before publishing the Engine pair: Run both policy suites before publishing the Engine pair:
@ -151,6 +155,23 @@ PYTHONDONTWRITEBYTECODE=1 \
python3 infra/deploy-runner/test_engine_n8n_private_extension.py python3 infra/deploy-runner/test_engine_n8n_private_extension.py
``` ```
The source-less Engine MCP control-plane update is a separate exact slice:
```bash
node infra/deploy-runner/build-engine-mcp-control-plane-artifact.mjs 20260717-001
```
Its six-entry registry contains only the Engine Agent gateway, verified graph
patch route, Codex installer source/package and the active node-intelligence
descriptor with the new gateway digest. It force-recreates only the existing
`nodedc-backend`. The node-intelligence image/service, n8n, L1, credentials,
databases and volumes are outside the slice. The runner proves the installed
predecessor digests, exact installer archive/source equality, bounded change
session policy, no-effect/post-write graph barriers, active immutable backend
runtime and descriptor equality. The ordinary overlay backup is the automatic
rollback source; rollback restores the predecessor gateway and descriptor
together before recreating the same backend service.
For `platform` artifacts, the allowlist includes the versioned Ontology Core, For `platform` artifacts, the allowlist includes the versioned Ontology Core,
the frozen legacy Gelios compatibility service, and the provider-neutral the frozen legacy Gelios compatibility service, and the provider-neutral
External Data Plane sources. The Gelios service remains reproducible only to External Data Plane sources. The Gelios service remains reproducible only to

View File

@ -1,7 +1,7 @@
#!/usr/bin/env node #!/usr/bin/env node
import { createHash } from 'node:crypto' import { createHash } from 'node:crypto'
import { spawnSync } from 'node:child_process' import { spawnSync } from 'node:child_process'
import { cp, lstat, mkdir, mkdtemp, readFile, rm, writeFile } from 'node:fs/promises' import { lstat, mkdir, mkdtemp, readFile, rm, writeFile } from 'node:fs/promises'
import { tmpdir } from 'node:os' import { tmpdir } from 'node:os'
import { dirname, join, resolve } from 'node:path' import { dirname, join, resolve } from 'node:path'
import { fileURLToPath } from 'node:url' import { fileURLToPath } from 'node:url'
@ -15,6 +15,7 @@ const [patchId = '', ...extra] = process.argv.slice(2)
const storeRelativePath = 'nodedc-source/server/engineAgents/store.js' const storeRelativePath = 'nodedc-source/server/engineAgents/store.js'
const predecessorSha256 = '52daa43499d6d9a97fe7ffa891edb9212b7791e733f91dd3ca686d42739b7e9a' const predecessorSha256 = '52daa43499d6d9a97fe7ffa891edb9212b7791e733f91dd3ca686d42739b7e9a'
const targetSha256 = '2e62654c2dc12905efcc83a9dff45a818dd7b47924a10600160835c4416540e9' const targetSha256 = '2e62654c2dc12905efcc83a9dff45a818dd7b47924a10600160835c4416540e9'
const readerExtendedSourceSha256 = 'debd351fe0b8c72b33b8c79909b8f339cbaf061b970576f3ae72df52ebaa211f'
const previouslyIssuedPatchIds = new Set([ const previouslyIssuedPatchIds = new Set([
'engine-agent-full-grant-migration-20260717-001', 'engine-agent-full-grant-migration-20260717-001',
]) ])
@ -29,7 +30,7 @@ if (previouslyIssuedPatchIds.has(patchId)) {
const source = join(engineRoot, storeRelativePath) const source = join(engineRoot, storeRelativePath)
const sourceInfo = await lstat(source) const sourceInfo = await lstat(source)
if (sourceInfo.isSymbolicLink() || !sourceInfo.isFile()) throw new Error('engine_agent_store_source_unsafe') if (sourceInfo.isSymbolicLink() || !sourceInfo.isFile()) throw new Error('engine_agent_store_source_unsafe')
const sourceBytes = await readFile(source) const sourceBytes = materializeFrozenTarget(await readFile(source))
if (digest(sourceBytes) !== targetSha256) throw new Error('engine_agent_store_target_sha256_mismatch') if (digest(sourceBytes) !== targetSha256) throw new Error('engine_agent_store_target_sha256_mismatch')
const sourceText = sourceBytes.toString('utf8') const sourceText = sourceBytes.toString('utf8')
for (const required of [ for (const required of [
@ -56,7 +57,7 @@ const stage = await mkdtemp(join(tmpdir(), 'nodedc-engine-agent-grant-migration-
try { try {
const destination = join(stage, 'payload', storeRelativePath) const destination = join(stage, 'payload', storeRelativePath)
await mkdir(dirname(destination), { recursive: true }) await mkdir(dirname(destination), { recursive: true })
await cp(source, destination, { force: false, verbatimSymlinks: true }) await writeFile(destination, sourceBytes)
await writeFile( await writeFile(
join(stage, 'manifest.env'), join(stage, 'manifest.env'),
`id=${patchId}\ncomponent=engine\ntype=app-overlay\n`, `id=${patchId}\ncomponent=engine\ntype=app-overlay\n`,
@ -119,3 +120,22 @@ function run(command, args) {
function digest(bytes) { function digest(bytes) {
return createHash('sha256').update(bytes).digest('hex') return createHash('sha256').update(bytes).digest('hex')
} }
function materializeFrozenTarget(bytes) {
const sourceSha256 = digest(bytes)
if (sourceSha256 === targetSha256) return bytes
if (sourceSha256 !== readerExtendedSourceSha256) {
throw new Error('engine_agent_store_target_sha256_mismatch')
}
const text = bytes.toString('utf8')
const readerScopes = [
" 'engine:l2:data-product-read-grant:plan',\n",
" 'engine:l2:data-product-read-grant:write',\n",
]
let frozen = text
for (const scope of readerScopes) {
if (!frozen.includes(scope)) throw new Error('engine_agent_store_reader_scope_boundary_missing')
frozen = frozen.replace(scope, '')
}
return Buffer.from(frozen, 'utf8')
}

View File

@ -45,7 +45,7 @@ const credentialSinkPredecessorSha256 = Object.freeze({
'nodedc-source/server/credentialSink/vendor/engine-credential-sink.mjs': 'b4800eead9bf94793ff1280d06e34aad6b37ef055a9d7f8d83d7fb5f9bd66d8b', 'nodedc-source/server/credentialSink/vendor/engine-credential-sink.mjs': 'b4800eead9bf94793ff1280d06e34aad6b37ef055a9d7f8d83d7fb5f9bd66d8b',
'nodedc-source/server/index.js': 'b2b790b02839570d967a2ca68b00e2724485a99389ac9b3a589a1b22302a36b8', 'nodedc-source/server/index.js': 'b2b790b02839570d967a2ca68b00e2724485a99389ac9b3a589a1b22302a36b8',
'nodedc-source/server/routes/engineCredentialSink.js': '9cbb69dbc8cbe6181cd5b0170fe9c4d717b0a173866ca98cba3e3766c0bab94e', 'nodedc-source/server/routes/engineCredentialSink.js': '9cbb69dbc8cbe6181cd5b0170fe9c4d717b0a173866ca98cba3e3766c0bab94e',
'nodedc-source/server/routes/ndcAgentMcp.js': 'fbb3342b1a617b956d5b3a6a40d5111aa107c1176b4ff89c37b104995bada081', 'nodedc-source/server/routes/ndcAgentMcp.js': '534e2c85e1faecc72a00da7ad32d0584ee09b6bd89eeec2221c3b23d80e1e962',
'nodedc-source/services/backend/credential-sink/docker-compose.immutable-runtime.yml': '944fa64b08255eb8207b93fd327aebb98ecd9400d37d25fcfa8e3a040ee44afe', 'nodedc-source/services/backend/credential-sink/docker-compose.immutable-runtime.yml': '944fa64b08255eb8207b93fd327aebb98ecd9400d37d25fcfa8e3a040ee44afe',
}) })
const publishGrantRuntimeOverride = [ const publishGrantRuntimeOverride = [

View File

@ -0,0 +1,177 @@
#!/usr/bin/env node
import { createHash } from "node:crypto";
import { spawnSync } from "node:child_process";
import { cp, lstat, mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { dirname, join, resolve } from "node:path";
import { fileURLToPath } from "node:url";
const here = dirname(fileURLToPath(import.meta.url));
const platformRoot = resolve(here, "../..");
const engineRoot = resolve(
process.env.NODEDC_ENGINE_SOURCE_ROOT || resolve(platformRoot, "../NODEDC_ENGINE_INFRA"),
);
const canonicalArtifactRoot = resolve(here, "../deploy-artifacts");
const artifactRoot = resolve(process.env.NODEDC_DEPLOY_ARTIFACT_DIR || canonicalArtifactRoot);
const [transitionId = "20260718-003", ...extra] = process.argv.slice(2);
if (extra.length || !/^\d{8}-[0-9]{3}$/.test(transitionId)) {
throw new Error("usage: build-engine-mcp-control-plane-artifact.mjs [YYYYMMDD-NNN]");
}
const id = `engine-mcp-control-plane-${transitionId}`;
const target = join(artifactRoot, `nodedc-${id}.tgz`);
const nodeIntelligenceArtifact = join(
canonicalArtifactRoot,
"nodedc-engine-node-intelligence-20260717-001.tgz",
);
const nodeIntelligenceArtifactSha256 = "d126afaa0c714fef26362aad4749e3f4647f551d6eb975f0133cdf9ff2e6fc4f";
const descriptorRel = "nodedc-source/services/node-intelligence/activation.json";
const gatewayRel = "nodedc-source/server/routes/engineAgentGateway.js";
const upstreamProjectionRel = "nodedc-source/server/nodeIntelligence/upstreamProjection.js";
const files = [
"nodedc-source/server/assets/engine-agent-npm/bin/nodedc-engine-codex-agent.mjs",
"nodedc-source/server/assets/engine-agent-npm/package.json",
"nodedc-source/server/assets/nodedc-engine-codex-agent-0.1.4.tgz",
"nodedc-source/server/dataProductPublishGrant/providerCatalog.js",
"nodedc-source/server/dataProductPublishGrant/signedDataPlaneClient.js",
"nodedc-source/server/dataProductReadGrant/acceptance.js",
"nodedc-source/server/dataProductReadGrant/service.js",
"nodedc-source/server/dataProductReadGrant/store.js",
"nodedc-source/server/engineAgents/store.js",
upstreamProjectionRel,
gatewayRel,
"nodedc-source/server/routes/n8n.js",
"nodedc-source/server/routes/ndcAgentMcp.js",
"nodedc-source/services/backend/data-product-read-grant/docker-compose.immutable-runtime.yml",
"nodedc-source/server/dataProductPublishGrant/service.js",
"nodedc-source/server/dataProductPublishGrant/store.js",
descriptorRel,
];
const expectedSha256 = new Map([
[files[0], "521098de69fe288a56bd4158c849c1055ba24be08e19f7a4111618d0e8138445"],
[files[1], "cbe113e9b10bb84b9ccbffa3e261908e58a8430c11abe2cf4fd5304741b04597"],
[files[2], "e74c0136d346f904b42589d75cb11a952b4d2f21153f1b057fba28e9acf4f95f"],
[files[3], "901b8fad80018ce177b34ced804b39cb140a47e831414057f484296b373c651d"],
[files[4], "c2a13d5eb49937fe70ec6451d0465cac54c19c7fae44448db099079473ec02fc"],
[files[5], "202dbe575b2f2e1c584aa7e6e99e38fa84f12496feb454e3dbd3f98e2d0396dd"],
[files[6], "65b8cdd6b603333bac1feb303e9791feb1e9da39dc9b5bfc3df3961273b0052e"],
[files[7], "c3fcdc59a794f57d92e3d2ac713ee28341347cebd25364c4060beaa3dc2151de"],
[files[8], "debd351fe0b8c72b33b8c79909b8f339cbaf061b970576f3ae72df52ebaa211f"],
[files[9], "761a874b102a938bc6018159ddacdaac71ad6ae08e9f0f8d7f3b58a0165a5131"],
[files[10], "96c726dab5cf1341f74e6e1095d518058ca320e0dd5738e25bdbe75db1f4fc15"],
[files[11], "f293a7794405badbabd2bf7ef088f96fe1167d9e249f05cbfc8af280a0d3e8f8"],
[files[12], "534e2c85e1faecc72a00da7ad32d0584ee09b6bd89eeec2221c3b23d80e1e962"],
[files[13], "952df0cb2ac477e644f5f3a9d64b4872ce1da33356eeab0bec17dcb2931cf653"],
[files[14], "ded3832c9677345a988448ae8e69cef254fd9bf39d2de20cffa295c1feedcd7c"],
[files[15], "a92303b2732e21f68c1ac732fa26e4983cbadf3515741cee983b916a283d754c"],
]);
await assertFresh(target);
assertSha(await readFile(nodeIntelligenceArtifact), nodeIntelligenceArtifactSha256, "node intelligence artifact");
const stage = await mkdtemp(join(tmpdir(), "nodedc-engine-mcp-control-plane-"));
const payload = join(stage, "payload");
try {
await mkdir(payload, { recursive: true });
for (const rel of files.slice(0, -1)) {
const source = join(engineRoot, rel);
const stat = await lstat(source);
if (!stat.isFile() || stat.isSymbolicLink()) throw new Error(`source_boundary_invalid:${rel}`);
assertSha(await readFile(source), expectedSha256.get(rel), rel);
await mkdir(dirname(join(payload, rel)), { recursive: true });
await cp(source, join(payload, rel), { force: false });
}
const descriptorText = extractMember(
nodeIntelligenceArtifact,
`payload/${descriptorRel}`,
);
const descriptor = JSON.parse(descriptorText);
if (
descriptor?.action !== "activate"
|| descriptor?.releaseId !== "2.33.2-974a9fb3492f"
|| descriptor?.source?.gatewaySha256 !== "9642c76fd5765a8a20629c8d09e16579a1c3b2cfb7bdbab38cf55af469d8908f"
) throw new Error("node_intelligence_predecessor_descriptor_mismatch");
descriptor.source.gatewaySha256 = expectedSha256.get(gatewayRel);
descriptor.source.upstreamProjectionSha256 = expectedSha256.get(upstreamProjectionRel);
await mkdir(dirname(join(payload, descriptorRel)), { recursive: true });
await writeFile(join(payload, descriptorRel), `${JSON.stringify(descriptor, null, 2)}\n`, "utf8");
await writeFile(join(stage, "manifest.env"), `id=${id}\ncomponent=engine\ntype=app-overlay\n`, "utf8");
await writeFile(join(stage, "files.txt"), `${files.join("\n")}\n`, "utf8");
await mkdir(artifactRoot, { recursive: true });
run("python3", ["-c", canonicalTarScript(), target, stage]);
const artifactSha256 = sha(await readFile(target));
console.log(JSON.stringify({
ok: true,
id,
artifact: target,
artifactSha256,
services: ["nodedc-backend"],
predecessorGatewaySha256: "9642c76fd5765a8a20629c8d09e16579a1c3b2cfb7bdbab38cf55af469d8908f",
targetGatewaySha256: expectedSha256.get(gatewayRel),
mcpVersion: "0.5.0",
installerVersion: "0.1.4",
files,
}, null, 2));
} finally {
await rm(stage, { recursive: true, force: true });
}
async function assertFresh(path) {
try {
await lstat(path);
} catch (error) {
if (error?.code === "ENOENT") return;
throw error;
}
throw new Error("artifact_already_exists");
}
function extractMember(archive, member) {
const script = [
"import pathlib,sys,tarfile",
"p=pathlib.Path(sys.argv[1]); name=sys.argv[2]",
"with tarfile.open(p,'r:gz') as t:",
" m=t.getmember(name)",
" if not m.isfile(): raise SystemExit('member-not-file')",
" f=t.extractfile(m)",
" if f is None: raise SystemExit('member-unreadable')",
" sys.stdout.buffer.write(f.read())",
].join("\n");
return run("python3", ["-c", script, archive, member]).stdout;
}
function canonicalTarScript() {
return [
"import gzip,io,pathlib,sys,tarfile",
"root=pathlib.Path(sys.argv[2])",
"with open(sys.argv[1],'wb') as out:",
" with gzip.GzipFile(filename='',mode='wb',fileobj=out,compresslevel=9,mtime=0) as gz:",
" with tarfile.open(fileobj=gz,mode='w',format=tarfile.PAX_FORMAT) as tar:",
" for top in ('manifest.env','files.txt','payload'):",
" p=root/top; paths=[p]+(sorted(p.rglob('*')) if p.is_dir() else [])",
" for x in paths:",
" info=tar.gettarinfo(str(x),arcname=x.relative_to(root).as_posix())",
" info.uid=info.gid=0; info.uname=info.gname='root'; info.mtime=0; info.mode=0o755 if info.isdir() else 0o644",
" with (open(x,'rb') if info.isfile() else io.BytesIO()) as src: tar.addfile(info,src if info.isfile() else None)",
].join("\n");
}
function assertSha(bytes, expected, label) {
const actual = sha(bytes);
if (!expected || actual !== expected) throw new Error(`${label.replaceAll(" ", "_")}_sha256_mismatch:${actual}`);
}
function sha(bytes) {
return createHash("sha256").update(bytes).digest("hex");
}
function run(command, args) {
const result = spawnSync(command, args, {
encoding: "utf8",
maxBuffer: 128 * 1024 * 1024,
stdio: ["ignore", "pipe", "pipe"],
});
if (result.status !== 0) throw new Error(`${command}_failed:${result.stderr || result.stdout}`);
return result;
}

View File

@ -13,18 +13,19 @@ const engineRoot = resolve(platformRoot, "../NODEDC_ENGINE_INFRA");
const artifactRoot = resolve(process.env.NODEDC_DEPLOY_ARTIFACT_DIR || join(here, "../deploy-artifacts")); const artifactRoot = resolve(process.env.NODEDC_DEPLOY_ARTIFACT_DIR || join(here, "../deploy-artifacts"));
const stageArtifact = resolve( const stageArtifact = resolve(
process.env.NODEDC_N8N_EXTENSION_STAGE_ARTIFACT process.env.NODEDC_N8N_EXTENSION_STAGE_ARTIFACT
|| join(artifactRoot, "nodedc-n8n-private-extension-n8n-nodes-ndc-release-20260716-003.tgz"), || join(artifactRoot, "nodedc-n8n-private-extension-n8n-nodes-ndc-history-read-20260717-004.tgz"),
); );
const stageArtifactSha256 = "4601c16c57e5182996adf18d0163837511b27ab3f7cfdf97eac679418ee2d078"; const stageArtifactSha256 = "a78f8250704e4893eab727cdb862b69ce45a84310fbb73ecd3dfd968b72d12ee";
const releaseId = "0.1.2-05e4b38b14b4a019"; const releaseId = "0.1.3-3354149b5245e39a";
const packageVersion = "0.1.2"; const packageVersion = "0.1.3";
const packageSha256 = "05e4b38b14b4a019ce1f6eee27b9e320094cb3560903bd68074966b3a1267af5"; const packageSha256 = "3354149b5245e39a10f6f03a4b43796602d7e57ea1f91dcc6cb6774ead97501d";
const n8nVersion = "2.3.2"; const n8nVersion = "2.3.2";
const baseImage = "docker.n8n.io/n8nio/n8n:2.3.2"; const baseImage = "docker.n8n.io/n8nio/n8n:2.3.2";
const architecture = "amd64"; const architecture = "amd64";
const generatedAt = "2026-07-15T21:51:41.000Z"; const generatedAt = "2026-07-17T21:00:00.000Z";
const previouslyIssuedTransitionIds = new Set(["20260715-002", "20260716-003"]); const predecessorGitRevision = "96ad46e3c62943f818233481957c62e5088ae35c";
const previouslyIssuedTransitionIds = new Set(["20260715-002", "20260716-003", "20260717-004"]);
const transitionId = readTransitionId(process.argv.slice(2), process.env.NODEDC_N8N_TRANSITION_ID); const transitionId = readTransitionId(process.argv.slice(2), process.env.NODEDC_N8N_TRANSITION_ID);
const activationId = `engine-n8n-private-extension-${transitionId}`; const activationId = `engine-n8n-private-extension-${transitionId}`;
const rollbackId = `engine-n8n-private-extension-rollback-${transitionId}`; const rollbackId = `engine-n8n-private-extension-rollback-${transitionId}`;
@ -39,7 +40,6 @@ const metaRel = `${schemaRoot}/meta.json`;
const iconRoot = "nodedc-source/server/assets/n8n/icons"; const iconRoot = "nodedc-source/server/assets/n8n/icons";
const iconRel = `${iconRoot}/ndc.svg`; const iconRel = `${iconRoot}/ndc.svg`;
const darkIconRel = `${iconRoot}/ndc.dark.svg`; const darkIconRel = `${iconRoot}/ndc.dark.svg`;
const sealedReleaseRelativePath = `n8n-private-extensions/releases/n8n-nodes-ndc/${releaseId}/package`;
const runtimePackagePath = "/home/node/.n8n/nodes/node_modules/n8n-nodes-ndc"; const runtimePackagePath = "/home/node/.n8n/nodes/node_modules/n8n-nodes-ndc";
const expectedNodeTypes = [ const expectedNodeTypes = [
@ -111,10 +111,23 @@ try {
assertExact(privateNodes.map((item) => item.name), expectedNodeTypes, "node types"); assertExact(privateNodes.map((item) => item.name), expectedNodeTypes, "node types");
assertExact(privateCredentials.map((item) => item.name), expectedCredentialTypes, "credential types"); assertExact(privateCredentials.map((item) => item.name), expectedCredentialTypes, "credential types");
const baselineNodes = JSON.parse(gitFile(nodesCatalogRel)); const predecessorDescriptor = JSON.parse(gitFile(descriptorRel));
const baselineCredentials = JSON.parse(gitFile(credentialsCatalogRel)); const predecessorNodes = JSON.parse(gitFile(nodesCatalogRel));
const baselineMeta = JSON.parse(gitFile(metaRel)); const predecessorCredentials = JSON.parse(gitFile(credentialsCatalogRel));
assertBaselineCatalogs(baselineNodes, baselineCredentials, baselineMeta); const predecessorMeta = JSON.parse(gitFile(metaRel));
assertPredecessorCatalogs(
predecessorDescriptor,
predecessorNodes,
predecessorCredentials,
predecessorMeta,
);
const baselineNodes = predecessorNodes.filter(
(item) => !String(item?.name || "").startsWith("n8n-nodes-ndc."),
);
const baselineCredentials = predecessorCredentials.filter(
(item) => !expectedCredentialTypes.includes(String(item?.name || "")),
);
assertBaselineCatalogs(baselineNodes, baselineCredentials);
const activeNodes = [...baselineNodes, ...privateNodes]; const activeNodes = [...baselineNodes, ...privateNodes];
const activeCredentials = [...baselineCredentials, ...privateCredentials]; const activeCredentials = [...baselineCredentials, ...privateCredentials];
const activeMeta = { const activeMeta = {
@ -125,9 +138,20 @@ try {
credentialCount: activeCredentials.length, credentialCount: activeCredentials.length,
}; };
const activationDescriptor = descriptor("activate", expectedNodeTypes, expectedCredentialTypes, "verified_inactive"); const target = {
const rollbackDescriptor = descriptor("rollback-inactive", [], [], releaseId); releaseId,
const override = composeOverride(); packageVersion,
packageSha256,
};
const predecessor = {
releaseId: predecessorDescriptor.releaseId,
packageVersion: predecessorDescriptor.packageVersion,
packageSha256: predecessorDescriptor.packageSha256,
};
const activationDescriptor = descriptor(target, predecessor.releaseId, predecessor.releaseId);
const rollbackDescriptor = descriptor(predecessor, releaseId, releaseId);
const override = composeOverride(target);
const rollbackOverride = composeOverride(predecessor);
const generatedRoot = join(work, "generated-engine-payload"); const generatedRoot = join(work, "generated-engine-payload");
await writeJson(join(generatedRoot, nodesCatalogRel), activeNodes); await writeJson(join(generatedRoot, nodesCatalogRel), activeNodes);
@ -154,13 +178,16 @@ try {
} }
}); });
const rollbackEntries = [descriptorRel, nodesCatalogRel, credentialsCatalogRel, metaRel]; const rollbackEntries = [...activationEntries];
const rollbackArtifact = await buildArtifact(work, rollbackId, rollbackEntries, async (payload) => { const rollbackArtifact = await buildArtifact(work, rollbackId, rollbackEntries, async (payload) => {
await writeJson(join(payload, descriptorRel), rollbackDescriptor); await writeJson(join(payload, descriptorRel), rollbackDescriptor);
await mkdir(join(payload, schemaRoot), { recursive: true }); await writeFile(join(payload, overrideRel), rollbackOverride, "utf8");
await writeFile(join(payload, nodesCatalogRel), `${JSON.stringify(baselineNodes, null, 2)}\n`, "utf8"); await writeJson(join(payload, nodesCatalogRel), predecessorNodes);
await writeFile(join(payload, credentialsCatalogRel), `${JSON.stringify(baselineCredentials, null, 2)}\n`, "utf8"); await writeJson(join(payload, credentialsCatalogRel), predecessorCredentials);
await writeFile(join(payload, metaRel), `${JSON.stringify(baselineMeta, null, 2)}\n`, "utf8"); await writeJson(join(payload, metaRel), predecessorMeta);
await mkdir(join(payload, iconRoot), { recursive: true });
await cp(join(engineRoot, iconRel), join(payload, iconRel), { force: false });
await cp(join(engineRoot, darkIconRel), join(payload, darkIconRel), { force: false });
}); });
console.log(JSON.stringify({ console.log(JSON.stringify({
@ -177,25 +204,25 @@ try {
await rm(work, { recursive: true, force: true }); await rm(work, { recursive: true, force: true });
} }
function descriptor(action, nodeTypes, credentialTypes, expectedCurrent) { function descriptor(target, expectedCurrent, rollbackBaseline) {
return { return {
schemaVersion: "nodedc.engine-n8n-private-extension-transition/v1", schemaVersion: "nodedc.engine-n8n-private-extension-transition/v1",
action, action: "activate",
releaseId, releaseId: target.releaseId,
packageVersion, packageVersion: target.packageVersion,
packageSha256, packageSha256: target.packageSha256,
n8nVersion, n8nVersion,
baseImage, baseImage,
baseImageArchitecture: architecture, baseImageArchitecture: architecture,
baseImageIdentityPolicy: "running-container-and-local-tag-must-match", baseImageIdentityPolicy: "running-container-and-local-tag-must-match",
sealedReleaseRelativePath, sealedReleaseRelativePath: `n8n-private-extensions/releases/n8n-nodes-ndc/${target.releaseId}/package`,
composeOverride: overrideRel, composeOverride: overrideRel,
runtimePackagePath, runtimePackagePath,
topologyServices: ["n8n"], topologyServices: ["n8n"],
expectedCurrent, expectedCurrent,
expectedNodeTypes: nodeTypes, expectedNodeTypes,
expectedCredentialTypes: credentialTypes, expectedCredentialTypes,
rollbackBaseline: "verified_inactive", rollbackBaseline,
}; };
} }
@ -235,8 +262,9 @@ async function assertArtifactTargetFresh(path) {
throw new Error("transition_artifact_already_exists"); throw new Error("transition_artifact_already_exists");
} }
function composeOverride() { function composeOverride(target) {
const health = "const http=require('http');const req=http.get('http://127.0.0.1:5678/healthz/readiness',r=>{r.resume();process.exit(r.statusCode===200?0:1)});req.on('error',()=>process.exit(1));req.setTimeout(4000,()=>{req.destroy();process.exit(1)});"; const health = "const http=require('http');const req=http.get('http://127.0.0.1:5678/healthz/readiness',r=>{r.resume();process.exit(r.statusCode===200?0:1)});req.on('error',()=>process.exit(1));req.setTimeout(4000,()=>{req.destroy();process.exit(1)});";
const sealedReleaseRelativePath = `n8n-private-extensions/releases/n8n-nodes-ndc/${target.releaseId}/package`;
return [ return [
"services:", "services:",
" n8n:", " n8n:",
@ -257,8 +285,8 @@ function composeOverride() {
" retries: 30", " retries: 30",
" start_period: 30s", " start_period: 30s",
" labels:", " labels:",
` nodedc.n8n-private-extension.release: ${releaseId}`, ` nodedc.n8n-private-extension.release: ${target.releaseId}`,
` nodedc.n8n-private-extension.package-sha256: ${packageSha256}`, ` nodedc.n8n-private-extension.package-sha256: ${target.packageSha256}`,
"", "",
].join("\n"); ].join("\n");
} }
@ -297,7 +325,34 @@ function assertPackage(value) {
assertExact(value.n8n?.credentials, credentialModules.map(([path]) => path), "package credentials"); assertExact(value.n8n?.credentials, credentialModules.map(([path]) => path), "package credentials");
} }
function assertBaselineCatalogs(nodes, credentials, meta) { function assertPredecessorCatalogs(descriptorValue, nodes, credentials, meta) {
if (descriptorValue?.action !== "activate"
|| descriptorValue?.releaseId !== "0.1.2-05e4b38b14b4a019"
|| descriptorValue?.packageVersion !== "0.1.2"
|| descriptorValue?.packageSha256 !== "05e4b38b14b4a019ce1f6eee27b9e320094cb3560903bd68074966b3a1267af5") {
throw new Error("predecessor_descriptor_mismatch");
}
if (!Array.isArray(nodes) || nodes.length !== 437
|| !Array.isArray(credentials) || credentials.length !== 388
|| meta?.n8nVersion !== n8nVersion
|| meta?.source !== "n8n-core+n8n-nodes-ndc@0.1.2"
|| meta?.nodeCount !== 437
|| meta?.credentialCount !== 388) {
throw new Error("predecessor_catalog_mismatch");
}
assertExact(
nodes.filter((item) => String(item?.name || "").startsWith("n8n-nodes-ndc.")).map((item) => item.name),
expectedNodeTypes,
"predecessor node types",
);
assertExact(
credentials.filter((item) => expectedCredentialTypes.includes(String(item?.name || ""))).map((item) => item.name),
expectedCredentialTypes,
"predecessor credential types",
);
}
function assertBaselineCatalogs(nodes, credentials) {
if (!Array.isArray(nodes) || nodes.length !== 434 || nodes.some((item) => String(item?.name || "").startsWith("n8n-nodes-ndc."))) { if (!Array.isArray(nodes) || nodes.length !== 434 || nodes.some((item) => String(item?.name || "").startsWith("n8n-nodes-ndc."))) {
throw new Error("baseline_node_catalog_mismatch"); throw new Error("baseline_node_catalog_mismatch");
} }
@ -305,9 +360,6 @@ function assertBaselineCatalogs(nodes, credentials, meta) {
|| credentials.some((item) => expectedCredentialTypes.includes(String(item?.name || "")))) { || credentials.some((item) => expectedCredentialTypes.includes(String(item?.name || "")))) {
throw new Error("baseline_credential_catalog_mismatch"); throw new Error("baseline_credential_catalog_mismatch");
} }
if (meta?.n8nVersion !== n8nVersion || meta?.nodeCount !== 434 || meta?.credentialCount !== 385) {
throw new Error("baseline_meta_mismatch");
}
} }
function assertEngineBaseline(compose) { function assertEngineBaseline(compose) {
@ -330,7 +382,7 @@ function assertSha(bytes, expected, label) {
} }
function gitFile(rel) { function gitFile(rel) {
return run("git", ["show", `HEAD:${rel}`], engineRoot).stdout; return run("git", ["show", `${predecessorGitRevision}:${rel}`], engineRoot).stdout;
} }
async function writeJson(path, value) { async function writeJson(path, value) {

View File

@ -23,6 +23,7 @@ const files = [
["packages/external-provider-contract/src/data-plane.mjs", "platform/packages/external-provider-contract/src/data-plane.mjs"], ["packages/external-provider-contract/src/data-plane.mjs", "platform/packages/external-provider-contract/src/data-plane.mjs"],
["packages/external-provider-contract/src/data-product.mjs", "platform/packages/external-provider-contract/src/data-product.mjs"], ["packages/external-provider-contract/src/data-product.mjs", "platform/packages/external-provider-contract/src/data-product.mjs"],
["packages/external-provider-contract/src/intake-batch.mjs", "platform/packages/external-provider-contract/src/intake-batch.mjs"], ["packages/external-provider-contract/src/intake-batch.mjs", "platform/packages/external-provider-contract/src/intake-batch.mjs"],
["packages/external-provider-contract/src/index.mjs", "platform/packages/external-provider-contract/src/index.mjs"],
["packages/external-provider-contract/src/sensitive-field-policy.mjs", "platform/packages/external-provider-contract/src/sensitive-field-policy.mjs"], ["packages/external-provider-contract/src/sensitive-field-policy.mjs", "platform/packages/external-provider-contract/src/sensitive-field-policy.mjs"],
]; ];
const ignoredBasenames = new Set([".DS_Store", ".git", "node_modules"]); const ignoredBasenames = new Set([".DS_Store", ".git", "node_modules"]);
@ -84,6 +85,21 @@ async function assertSourceBoundary() {
compose.includes("private-key.pem") compose.includes("private-key.pem")
|| compose.includes("ENGINE_EDP_MANAGED_PROVISIONER_PRIVATE_KEY_FILE") || compose.includes("ENGINE_EDP_MANAGED_PROVISIONER_PRIVATE_KEY_FILE")
) throw new Error("platform_artifact_private_key_boundary_violation"); ) throw new Error("platform_artifact_private_key_boundary_violation");
const server = await readFile(
resolve(platformRoot, "services/external-data-plane/src/server.mjs"),
"utf8",
);
for (const marker of [
'managedWriterBindingLifetime: config.managedProvisionerApiEnabled ? "explicit-revoke" : "disabled"',
'managedWriterBindings: "digest+idempotent-generation+explicit-revoke"',
'app.put("/internal/data-plane/v1/reader-bindings/by-key/:bindingKey"',
'managedReaderBindingProvisioning: config.managedProvisionerApiEnabled',
'managedReaderBindingLifetime: config.managedProvisionerApiEnabled ? "explicit-revoke" : "disabled"',
'managedReaderBindings: "digest+idempotent-generation+explicit-revoke"',
'where id = $1 and binding_key is null and active = true',
]) {
if (!server.includes(marker)) throw new Error(`managed_reader_boundary_missing:${marker}`);
}
} }
function canonicalTarScript() { function canonicalTarScript() {

View File

@ -10,10 +10,10 @@ const scriptDir = dirname(fileURLToPath(import.meta.url));
const platformRoot = resolve(scriptDir, "../.."); const platformRoot = resolve(scriptDir, "../..");
const workspaceRoot = resolve(platformRoot, ".."); const workspaceRoot = resolve(platformRoot, "..");
const foundryRoot = resolve(workspaceRoot, "NODEDC_DESIGN_GUIDELINE"); const foundryRoot = resolve(workspaceRoot, "NODEDC_DESIGN_GUIDELINE");
const artifactDir = resolve(scriptDir, "../deploy-artifacts"); const artifactDir = resolve(process.env.NODEDC_DEPLOY_ARTIFACT_DIR || resolve(scriptDir, "../deploy-artifacts"));
const patchId = process.argv[2] || "module-foundry-bootstrap-20260714-001"; const [patchId = "module-foundry-bootstrap-20260714-001", ...extra] = process.argv.slice(2);
if (!/^[A-Za-z0-9._-]{1,96}$/.test(patchId)) { if (extra.length || !/^[A-Za-z0-9._-]{1,96}$/.test(patchId)) {
throw new Error("patch_id_must_contain_only_letters_digits_dot_underscore_hyphen"); throw new Error("patch_id_must_contain_only_letters_digits_dot_underscore_hyphen");
} }
@ -48,13 +48,9 @@ try {
await writeFile(join(stage, "files.txt"), `${files.join("\n")}\n`, "utf8"); await writeFile(join(stage, "files.txt"), `${files.join("\n")}\n`, "utf8");
await mkdir(artifactDir, { recursive: true }); await mkdir(artifactDir, { recursive: true });
const tar = spawnSync("python3", ["-c", [ const tar = spawnSync("python3", ["-c", canonicalTarScript(), target, stage], {
"import sys, tarfile",
"with tarfile.open(sys.argv[1], 'w:gz', format=tarfile.PAX_FORMAT) as archive:",
" [archive.add(name, arcname=name, recursive=True) for name in ('manifest.env', 'files.txt', 'payload')]",
].join("\n"), target], {
cwd: stage,
encoding: "utf8", encoding: "utf8",
maxBuffer: 128 * 1024 * 1024,
}); });
if (tar.status !== 0) throw new Error(`tar_failed:${tar.stderr || tar.stdout}`); if (tar.status !== 0) throw new Error(`tar_failed:${tar.stderr || tar.stdout}`);
@ -64,6 +60,22 @@ try {
await rm(stage, { recursive: true, force: true }); await rm(stage, { recursive: true, force: true });
} }
function canonicalTarScript() {
return [
"import gzip,io,pathlib,sys,tarfile",
"root=pathlib.Path(sys.argv[2])",
"with open(sys.argv[1],'wb') as out:",
" with gzip.GzipFile(filename='',mode='wb',fileobj=out,compresslevel=9,mtime=0) as gz:",
" with tarfile.open(fileobj=gz,mode='w',format=tarfile.PAX_FORMAT) as tar:",
" for top in ('manifest.env','files.txt','payload'):",
" p=root/top; paths=[p]+(sorted(p.rglob('*')) if p.is_dir() else [])",
" for x in paths:",
" info=tar.gettarinfo(str(x),arcname=x.relative_to(root).as_posix())",
" info.uid=info.gid=0; info.uname=info.gname='root'; info.mtime=0; info.mode=0o755 if info.isdir() else 0o644",
" with (open(x,'rb') if info.isfile() else io.BytesIO()) as src: tar.addfile(info,src if info.isfile() else None)",
].join("\n");
}
async function copySafe(source, destination) { async function copySafe(source, destination) {
const sourceStat = await lstat(source); const sourceStat = await lstat(source);
if (sourceStat.isSymbolicLink()) throw new Error(`source_symlink_rejected:${relative(foundryRoot, source)}`); if (sourceStat.isSymbolicLink()) throw new Error(`source_symlink_rejected:${relative(foundryRoot, source)}`);

View File

@ -12,8 +12,8 @@ const platformRoot = resolve(scriptDir, "../..");
const packageRoot = resolve(platformRoot, "packages/n8n-nodes-ndc"); const packageRoot = resolve(platformRoot, "packages/n8n-nodes-ndc");
const artifactDir = resolve(process.env.NODEDC_DEPLOY_ARTIFACT_DIR || resolve(scriptDir, "../deploy-artifacts")); const artifactDir = resolve(process.env.NODEDC_DEPLOY_ARTIFACT_DIR || resolve(scriptDir, "../deploy-artifacts"));
const requireModule = createRequire(import.meta.url); const requireModule = createRequire(import.meta.url);
const expectedPackageVersion = "0.1.2"; const expectedPackageVersion = "0.1.3";
const [patchId = "n8n-nodes-ndc-release-20260716-003", ...extra] = process.argv.slice(2); const [patchId = "n8n-nodes-ndc-history-read-20260717-004", ...extra] = process.argv.slice(2);
const expectedRuntimeNodes = [ const expectedRuntimeNodes = [
{ {
file: "dist/nodes/NdcDataProductPublish/NdcDataProductPublish.node.js", file: "dist/nodes/NdcDataProductPublish/NdcDataProductPublish.node.js",

View File

@ -54,6 +54,7 @@ ENGINE_CREDENTIAL_BACKEND_ROOTFS_FILE = ENGINE_CREDENTIAL_BACKEND_RUNTIME_DIR /
ENGINE_CREDENTIAL_BACKEND_METADATA_FILE = ENGINE_CREDENTIAL_BACKEND_RUNTIME_DIR / "runtime-metadata.json" ENGINE_CREDENTIAL_BACKEND_METADATA_FILE = ENGINE_CREDENTIAL_BACKEND_RUNTIME_DIR / "runtime-metadata.json"
ENGINE_CREDENTIAL_BACKEND_ACTIVATION_FILE = ENGINE_CREDENTIAL_BACKEND_RUNTIME_DIR / "activation.ready" ENGINE_CREDENTIAL_BACKEND_ACTIVATION_FILE = ENGINE_CREDENTIAL_BACKEND_RUNTIME_DIR / "activation.ready"
ENGINE_DATA_PRODUCT_PUBLISH_GRANT_OVERRIDE_REL = "nodedc-source/services/backend/data-product-publish-grant/docker-compose.immutable-runtime.yml" ENGINE_DATA_PRODUCT_PUBLISH_GRANT_OVERRIDE_REL = "nodedc-source/services/backend/data-product-publish-grant/docker-compose.immutable-runtime.yml"
ENGINE_DATA_PRODUCT_READ_GRANT_OVERRIDE_REL = "nodedc-source/services/backend/data-product-read-grant/docker-compose.immutable-runtime.yml"
ENGINE_AGENT_FULL_GRANT_MIGRATION_STORE_REL = "nodedc-source/server/engineAgents/store.js" ENGINE_AGENT_FULL_GRANT_MIGRATION_STORE_REL = "nodedc-source/server/engineAgents/store.js"
ENGINE_AGENT_FULL_GRANT_MIGRATION_PREDECESSOR_SHA256 = "52daa43499d6d9a97fe7ffa891edb9212b7791e733f91dd3ca686d42739b7e9a" ENGINE_AGENT_FULL_GRANT_MIGRATION_PREDECESSOR_SHA256 = "52daa43499d6d9a97fe7ffa891edb9212b7791e733f91dd3ca686d42739b7e9a"
ENGINE_AGENT_FULL_GRANT_MIGRATION_TARGET_SHA256 = "2e62654c2dc12905efcc83a9dff45a818dd7b47924a10600160835c4416540e9" ENGINE_AGENT_FULL_GRANT_MIGRATION_TARGET_SHA256 = "2e62654c2dc12905efcc83a9dff45a818dd7b47924a10600160835c4416540e9"
@ -115,13 +116,74 @@ ENGINE_NODE_INTELLIGENCE_ROLLBACK_ENTRIES = (
ENGINE_NODE_INTELLIGENCE_GATEWAY_REL, ENGINE_NODE_INTELLIGENCE_GATEWAY_REL,
ENGINE_NODE_INTELLIGENCE_SERVICE_ROOT_REL, ENGINE_NODE_INTELLIGENCE_SERVICE_ROOT_REL,
) )
ENGINE_MCP_CONTROL_PLANE_DESCRIPTOR_REL = ENGINE_NODE_INTELLIGENCE_DESCRIPTOR_REL
ENGINE_MCP_CONTROL_PLANE_ARTIFACT_ENTRIES = (
"nodedc-source/server/assets/engine-agent-npm/bin/nodedc-engine-codex-agent.mjs",
"nodedc-source/server/assets/engine-agent-npm/package.json",
"nodedc-source/server/assets/nodedc-engine-codex-agent-0.1.4.tgz",
"nodedc-source/server/dataProductPublishGrant/providerCatalog.js",
"nodedc-source/server/dataProductPublishGrant/signedDataPlaneClient.js",
"nodedc-source/server/dataProductReadGrant/acceptance.js",
"nodedc-source/server/dataProductReadGrant/service.js",
"nodedc-source/server/dataProductReadGrant/store.js",
"nodedc-source/server/engineAgents/store.js",
"nodedc-source/server/nodeIntelligence/upstreamProjection.js",
ENGINE_NODE_INTELLIGENCE_GATEWAY_REL,
"nodedc-source/server/routes/n8n.js",
"nodedc-source/server/routes/ndcAgentMcp.js",
ENGINE_DATA_PRODUCT_READ_GRANT_OVERRIDE_REL,
"nodedc-source/server/dataProductPublishGrant/service.js",
"nodedc-source/server/dataProductPublishGrant/store.js",
ENGINE_MCP_CONTROL_PLANE_DESCRIPTOR_REL,
)
ENGINE_MCP_CONTROL_PLANE_PREDECESSOR_SHA256 = {
ENGINE_NODE_INTELLIGENCE_GATEWAY_REL: "604a75ad3b9b463cea5e578760c8a23592e2e00b8acfc0099730a347e27bf4e8",
"nodedc-source/server/routes/ndcAgentMcp.js": "534e2c85e1faecc72a00da7ad32d0584ee09b6bd89eeec2221c3b23d80e1e962",
"nodedc-source/server/routes/n8n.js": "de6e3c76740af86e2eaeedede140b5ee54eb526f03db270324d7a4fe513f6817",
"nodedc-source/server/dataProductPublishGrant/providerCatalog.js": "a96d3695fdb3ed7012b9041182f37b09e32d1dfa0dbb391d8db2d7d25a39d64d",
"nodedc-source/server/dataProductPublishGrant/signedDataPlaneClient.js": "1a26728ac69c5ef3fdce1ce1154a6ce330f72286b6e07825796a246070d15bbf",
"nodedc-source/server/engineAgents/store.js": "2e62654c2dc12905efcc83a9dff45a818dd7b47924a10600160835c4416540e9",
"nodedc-source/server/nodeIntelligence/upstreamProjection.js": "946353ed8582c5d96e08361603425cc45eaf231f730ae46a75028d3bef352ee9",
"nodedc-source/server/assets/engine-agent-npm/bin/nodedc-engine-codex-agent.mjs": "7d3d7a9bc48614e6cd238199caa384073096b7ec8273703833681151b464ef17",
"nodedc-source/server/assets/engine-agent-npm/package.json": "9f299b20db4855b1bb4a9bd2e1095dfd059cd90ec82e936884a380aa41249e28",
"nodedc-source/server/dataProductPublishGrant/service.js": "85ded82d707e0ef5b1ad739b5a9be24c1432e2dd9e5051108739f09daf20f47c",
"nodedc-source/server/dataProductPublishGrant/store.js": "ef2a5e6bfad9f0b1710db006a0d30d01cd9bcaf06524b0e9f975dd771efe3952",
}
ENGINE_MCP_CONTROL_PLANE_TARGET_SHA256 = {
ENGINE_NODE_INTELLIGENCE_GATEWAY_REL: "96c726dab5cf1341f74e6e1095d518058ca320e0dd5738e25bdbe75db1f4fc15",
"nodedc-source/server/routes/ndcAgentMcp.js": "534e2c85e1faecc72a00da7ad32d0584ee09b6bd89eeec2221c3b23d80e1e962",
"nodedc-source/server/routes/n8n.js": "f293a7794405badbabd2bf7ef088f96fe1167d9e249f05cbfc8af280a0d3e8f8",
"nodedc-source/server/dataProductPublishGrant/providerCatalog.js": "901b8fad80018ce177b34ced804b39cb140a47e831414057f484296b373c651d",
"nodedc-source/server/dataProductPublishGrant/signedDataPlaneClient.js": "c2a13d5eb49937fe70ec6451d0465cac54c19c7fae44448db099079473ec02fc",
"nodedc-source/server/dataProductReadGrant/acceptance.js": "202dbe575b2f2e1c584aa7e6e99e38fa84f12496feb454e3dbd3f98e2d0396dd",
"nodedc-source/server/dataProductReadGrant/service.js": "65b8cdd6b603333bac1feb303e9791feb1e9da39dc9b5bfc3df3961273b0052e",
"nodedc-source/server/dataProductReadGrant/store.js": "c3fcdc59a794f57d92e3d2ac713ee28341347cebd25364c4060beaa3dc2151de",
"nodedc-source/server/engineAgents/store.js": "debd351fe0b8c72b33b8c79909b8f339cbaf061b970576f3ae72df52ebaa211f",
"nodedc-source/server/nodeIntelligence/upstreamProjection.js": "761a874b102a938bc6018159ddacdaac71ad6ae08e9f0f8d7f3b58a0165a5131",
"nodedc-source/server/assets/engine-agent-npm/bin/nodedc-engine-codex-agent.mjs": "521098de69fe288a56bd4158c849c1055ba24be08e19f7a4111618d0e8138445",
"nodedc-source/server/assets/engine-agent-npm/package.json": "cbe113e9b10bb84b9ccbffa3e261908e58a8430c11abe2cf4fd5304741b04597",
"nodedc-source/server/assets/nodedc-engine-codex-agent-0.1.4.tgz": "e74c0136d346f904b42589d75cb11a952b4d2f21153f1b057fba28e9acf4f95f",
ENGINE_DATA_PRODUCT_READ_GRANT_OVERRIDE_REL: "952df0cb2ac477e644f5f3a9d64b4872ce1da33356eeab0bec17dcb2931cf653",
"nodedc-source/server/dataProductPublishGrant/service.js": "ded3832c9677345a988448ae8e69cef254fd9bf39d2de20cffa295c1feedcd7c",
"nodedc-source/server/dataProductPublishGrant/store.js": "a92303b2732e21f68c1ac732fa26e4983cbadf3515741cee983b916a283d754c",
}
ENGINE_MCP_CONTROL_PLANE_NEW_PATHS = (
"nodedc-source/server/assets/nodedc-engine-codex-agent-0.1.4.tgz",
"nodedc-source/server/dataProductReadGrant/acceptance.js",
"nodedc-source/server/dataProductReadGrant/service.js",
"nodedc-source/server/dataProductReadGrant/store.js",
ENGINE_DATA_PRODUCT_READ_GRANT_OVERRIDE_REL,
)
ENGINE_CONTROL_PLANE_STATE_REL = "nodedc-control-plane" ENGINE_CONTROL_PLANE_STATE_REL = "nodedc-control-plane"
ENGINE_PUBLISH_GRANT_STATE_REL = f"{ENGINE_CONTROL_PLANE_STATE_REL}/publish-grants" ENGINE_PUBLISH_GRANT_STATE_REL = f"{ENGINE_CONTROL_PLANE_STATE_REL}/publish-grants"
ENGINE_READ_GRANT_STATE_REL = f"{ENGINE_CONTROL_PLANE_STATE_REL}/read-grants"
ENGINE_CONTROL_PLANE_STATE_PATH = Path("/volume2/nodedc-demo/nodedc-control-plane") ENGINE_CONTROL_PLANE_STATE_PATH = Path("/volume2/nodedc-demo/nodedc-control-plane")
ENGINE_PUBLISH_GRANT_STATE_PATH = ENGINE_CONTROL_PLANE_STATE_PATH / "publish-grants" ENGINE_PUBLISH_GRANT_STATE_PATH = ENGINE_CONTROL_PLANE_STATE_PATH / "publish-grants"
ENGINE_READ_GRANT_STATE_PATH = ENGINE_CONTROL_PLANE_STATE_PATH / "read-grants"
ENGINE_EDP_PRIVATE_KEY_CONTAINER_PATH = "/run/nodedc-secrets/engine-edp-managed-provisioner-private.pem" ENGINE_EDP_PRIVATE_KEY_CONTAINER_PATH = "/run/nodedc-secrets/engine-edp-managed-provisioner-private.pem"
ENGINE_CONTROL_PLANE_CONTAINER_PATH = "/var/lib/nodedc-control-plane" ENGINE_CONTROL_PLANE_CONTAINER_PATH = "/var/lib/nodedc-control-plane"
ENGINE_PUBLISH_GRANT_CONTAINER_PATH = "/var/lib/nodedc-control-plane/publish-grants" ENGINE_PUBLISH_GRANT_CONTAINER_PATH = "/var/lib/nodedc-control-plane/publish-grants"
ENGINE_READ_GRANT_CONTAINER_PATH = "/var/lib/nodedc-control-plane/read-grants"
EXTERNAL_DATA_PLANE_MANAGED_TRUST_CONTAINER_PATH = "/run/nodedc-trust/engine-managed-provisioner" EXTERNAL_DATA_PLANE_MANAGED_TRUST_CONTAINER_PATH = "/run/nodedc-trust/engine-managed-provisioner"
EXTERNAL_DATA_PLANE_INTERNAL_URL = "http://external-data-plane:18106" EXTERNAL_DATA_PLANE_INTERNAL_URL = "http://external-data-plane:18106"
PLATFORM_EXTERNAL_DATA_PLANE_COMPOSE_REL = "platform/docker-compose.external-data-plane.yml" PLATFORM_EXTERNAL_DATA_PLANE_COMPOSE_REL = "platform/docker-compose.external-data-plane.yml"
@ -135,9 +197,18 @@ ENGINE_N8N_RUNTIME_PACKAGE_PATH = "/home/node/.n8n/nodes/node_modules/n8n-nodes-
ENGINE_N8N_NODE_MODULES_PATH = "/usr/local/lib/node_modules/n8n/node_modules" ENGINE_N8N_NODE_MODULES_PATH = "/usr/local/lib/node_modules/n8n/node_modules"
ENGINE_N8N_ICON_SHA256 = "1c928fc996d8b82121e8f8e327d74b1dd21556c106de99c5e8d317d926c0ba17" ENGINE_N8N_ICON_SHA256 = "1c928fc996d8b82121e8f8e327d74b1dd21556c106de99c5e8d317d926c0ba17"
ENGINE_N8N_DARK_ICON_SHA256 = "ef3b8551da4ce04527736405afa9a220a2c76d047bd18f6f7124b9adb4216b0c" ENGINE_N8N_DARK_ICON_SHA256 = "ef3b8551da4ce04527736405afa9a220a2c76d047bd18f6f7124b9adb4216b0c"
ENGINE_N8N_ACTIVATION_NODES_CATALOG_JSON_SHA256 = "230f712230f7ee8debaa4b44a4358cc19c205bc43305d8ed89d5e3daac466506" ENGINE_N8N_RELEASE_CATALOG_JSON_SHA256 = {
ENGINE_N8N_ACTIVATION_CREDENTIALS_CATALOG_JSON_SHA256 = "a26824ffc0e15db857c792153de3417febc0dbba4880380d6c8cd544b3c80f4d" "0.1.2-05e4b38b14b4a019": {
ENGINE_N8N_ACTIVATION_META_JSON_SHA256 = "caf7635420c61333e65f68350f5567217aed96fb51354b38764bcac2c24e7966" "nodes": "230f712230f7ee8debaa4b44a4358cc19c205bc43305d8ed89d5e3daac466506",
"credentials": "a26824ffc0e15db857c792153de3417febc0dbba4880380d6c8cd544b3c80f4d",
"meta": "caf7635420c61333e65f68350f5567217aed96fb51354b38764bcac2c24e7966",
},
"0.1.3-3354149b5245e39a": {
"nodes": "b0a5215699a6e691b8cfc505ab457d5632ef6d83adb3537520d0b60760ba16b2",
"credentials": "a26824ffc0e15db857c792153de3417febc0dbba4880380d6c8cd544b3c80f4d",
"meta": "b8aa60c0d919573626fa0694a1e4ae9ede015325a5d312e3e05d5ac293084aa7",
},
}
ENGINE_N8N_INACTIVE_NODES_CATALOG_JSON_SHA256 = "b70d9d8130d498c55de46a5d0758c844f1242b70803457b2291ab3a9de8056f2" ENGINE_N8N_INACTIVE_NODES_CATALOG_JSON_SHA256 = "b70d9d8130d498c55de46a5d0758c844f1242b70803457b2291ab3a9de8056f2"
ENGINE_N8N_INACTIVE_CREDENTIALS_CATALOG_JSON_SHA256 = "680e9f52aac791efbd38e3bd99bd51ef5cded9756d867897c6c2755850e87b50" ENGINE_N8N_INACTIVE_CREDENTIALS_CATALOG_JSON_SHA256 = "680e9f52aac791efbd38e3bd99bd51ef5cded9756d867897c6c2755850e87b50"
ENGINE_N8N_INACTIVE_META_JSON_SHA256 = "0ac555d7ab31cb0ca042db4f474e83cfefbf20b5bbb2e1509fead27ed21e8acb" ENGINE_N8N_INACTIVE_META_JSON_SHA256 = "0ac555d7ab31cb0ca042db4f474e83cfefbf20b5bbb2e1509fead27ed21e8acb"
@ -215,10 +286,10 @@ ENGINE_DATA_PRODUCT_PUBLISH_GRANT_PREDECESSOR_SHA256 = {
"nodedc-source/server/credentialSink/store.js": "c78dc285a973b6acd8a2330f0310935ad08720b2905454492f292d235abf12c0", "nodedc-source/server/credentialSink/store.js": "c78dc285a973b6acd8a2330f0310935ad08720b2905454492f292d235abf12c0",
"nodedc-source/server/credentialSink/vendor/engine-credential-sink.mjs": "b4800eead9bf94793ff1280d06e34aad6b37ef055a9d7f8d83d7fb5f9bd66d8b", "nodedc-source/server/credentialSink/vendor/engine-credential-sink.mjs": "b4800eead9bf94793ff1280d06e34aad6b37ef055a9d7f8d83d7fb5f9bd66d8b",
"nodedc-source/server/index.js": "b2b790b02839570d967a2ca68b00e2724485a99389ac9b3a589a1b22302a36b8", "nodedc-source/server/index.js": "b2b790b02839570d967a2ca68b00e2724485a99389ac9b3a589a1b22302a36b8",
"nodedc-source/server/routes/engineAgentGateway.js": "e3450a4e1d5318dbac37627b67804d62804e27e2032c9e90478a1e739cea6d3d", "nodedc-source/server/routes/engineAgentGateway.js": "604a75ad3b9b463cea5e578760c8a23592e2e00b8acfc0099730a347e27bf4e8",
"nodedc-source/server/routes/engineCredentialSink.js": "9cbb69dbc8cbe6181cd5b0170fe9c4d717b0a173866ca98cba3e3766c0bab94e", "nodedc-source/server/routes/engineCredentialSink.js": "9cbb69dbc8cbe6181cd5b0170fe9c4d717b0a173866ca98cba3e3766c0bab94e",
"nodedc-source/server/routes/n8n.js": "783d822e2457d82e890f43bc00c7e33822077dc2841f0511a89ecb210fd36d48", "nodedc-source/server/routes/n8n.js": "783d822e2457d82e890f43bc00c7e33822077dc2841f0511a89ecb210fd36d48",
"nodedc-source/server/routes/ndcAgentMcp.js": "fbb3342b1a617b956d5b3a6a40d5111aa107c1176b4ff89c37b104995bada081", "nodedc-source/server/routes/ndcAgentMcp.js": "534e2c85e1faecc72a00da7ad32d0584ee09b6bd89eeec2221c3b23d80e1e962",
ENGINE_CREDENTIAL_BACKEND_OVERRIDE_TEMPLATE_REL: "944fa64b08255eb8207b93fd327aebb98ecd9400d37d25fcfa8e3a040ee44afe", ENGINE_CREDENTIAL_BACKEND_OVERRIDE_TEMPLATE_REL: "944fa64b08255eb8207b93fd327aebb98ecd9400d37d25fcfa8e3a040ee44afe",
} }
ENGINE_CREDENTIAL_SINK_CONTRACT_SHA256 = "b4800eead9bf94793ff1280d06e34aad6b37ef055a9d7f8d83d7fb5f9bd66d8b" ENGINE_CREDENTIAL_SINK_CONTRACT_SHA256 = "b4800eead9bf94793ff1280d06e34aad6b37ef055a9d7f8d83d7fb5f9bd66d8b"
@ -1048,7 +1119,7 @@ def ensure_engine_edp_managed_provisioner_keypair():
return "created" return "created"
def ensure_engine_publish_grant_private_state(): def ensure_engine_data_product_grant_private_state(include_reader=False):
# Compose must never create this bind source on our behalf: Docker's normal # Compose must never create this bind source on our behalf: Docker's normal
# 0755 directory would either expose control-plane metadata or be rejected # 0755 directory would either expose control-plane metadata or be rejected
# by the Engine's fail-closed PublishGrantStore. # by the Engine's fail-closed PublishGrantStore.
@ -1060,10 +1131,16 @@ def ensure_engine_publish_grant_private_state():
if stat.S_ISLNK(engine_root_stat.st_mode) or not stat.S_ISDIR(engine_root_stat.st_mode): if stat.S_ISLNK(engine_root_stat.st_mode) or not stat.S_ISDIR(engine_root_stat.st_mode):
die("Engine root is unsafe before private state preparation") die("Engine root is unsafe before private state preparation")
for state_path, label in ( state_paths = [
(engine_root / ENGINE_CONTROL_PLANE_STATE_REL, "Engine control-plane state"), (engine_root / ENGINE_CONTROL_PLANE_STATE_REL, "Engine control-plane state"),
(engine_root / ENGINE_PUBLISH_GRANT_STATE_REL, "Engine Publish grant state"), (engine_root / ENGINE_PUBLISH_GRANT_STATE_REL, "Engine Publish grant state"),
): ]
if include_reader:
state_paths.append((
engine_root / ENGINE_READ_GRANT_STATE_REL,
"Engine Read grant state",
))
for state_path, label in state_paths:
try: try:
state_stat = state_path.lstat() state_stat = state_path.lstat()
except FileNotFoundError: except FileNotFoundError:
@ -1079,6 +1156,11 @@ def ensure_engine_publish_grant_private_state():
die(f"{label} ownership or mode is unsafe: {state_path}") die(f"{label} ownership or mode is unsafe: {state_path}")
def ensure_engine_publish_grant_private_state():
# Frozen compatibility seam for the established Publish transition.
return ensure_engine_data_product_grant_private_state(include_reader=False)
def fsync_directory(path): def fsync_directory(path):
descriptor = os.open(str(path), os.O_RDONLY | os.O_DIRECTORY) descriptor = os.open(str(path), os.O_RDONLY | os.O_DIRECTORY)
try: try:
@ -1565,6 +1647,8 @@ def allowed_payload_path(component, rel):
return True return True
if rel == ENGINE_DATA_PRODUCT_PUBLISH_GRANT_OVERRIDE_REL: if rel == ENGINE_DATA_PRODUCT_PUBLISH_GRANT_OVERRIDE_REL:
return True return True
if rel == ENGINE_DATA_PRODUCT_READ_GRANT_OVERRIDE_REL:
return True
if rel == ENGINE_NODE_INTELLIGENCE_SERVICE_ROOT_REL or rel.startswith( if rel == ENGINE_NODE_INTELLIGENCE_SERVICE_ROOT_REL or rel.startswith(
ENGINE_NODE_INTELLIGENCE_SERVICE_ROOT_REL + "/" ENGINE_NODE_INTELLIGENCE_SERVICE_ROOT_REL + "/"
): ):
@ -2158,14 +2242,21 @@ def read_engine_n8n_transition_descriptor(path, label="Engine n8n transition des
"composeOverride": ENGINE_N8N_COMPOSE_OVERRIDE_REL, "composeOverride": ENGINE_N8N_COMPOSE_OVERRIDE_REL,
"runtimePackagePath": ENGINE_N8N_RUNTIME_PACKAGE_PATH, "runtimePackagePath": ENGINE_N8N_RUNTIME_PACKAGE_PATH,
"topologyServices": ["n8n"], "topologyServices": ["n8n"],
"rollbackBaseline": "verified_inactive",
} }
for key, expected in exact_common.items(): for key, expected in exact_common.items():
if descriptor.get(key) != expected: if descriptor.get(key) != expected:
die(f"Engine n8n transition {key} mismatch") die(f"Engine n8n transition {key} mismatch")
if action == "activate": if action == "activate":
if descriptor.get("expectedCurrent") != "verified_inactive": expected_current = descriptor.get("expectedCurrent")
die("Engine n8n activation must start from the verified inactive baseline") if (
expected_current != "verified_inactive"
and expected_current not in ENGINE_N8N_RELEASE_CATALOG_JSON_SHA256
):
die("Engine n8n activation expected-current release is not registered")
if expected_current == release_id:
die("Engine n8n activation target already equals expected current")
if descriptor.get("rollbackBaseline") != expected_current:
die("Engine n8n activation rollback baseline mismatch")
if descriptor.get("expectedNodeTypes") != list(ENGINE_N8N_NODE_TYPES): if descriptor.get("expectedNodeTypes") != list(ENGINE_N8N_NODE_TYPES):
die("Engine n8n activation node type set mismatch") die("Engine n8n activation node type set mismatch")
if descriptor.get("expectedCredentialTypes") != list(ENGINE_N8N_CREDENTIAL_TYPES): if descriptor.get("expectedCredentialTypes") != list(ENGINE_N8N_CREDENTIAL_TYPES):
@ -2173,6 +2264,8 @@ def read_engine_n8n_transition_descriptor(path, label="Engine n8n transition des
else: else:
if descriptor.get("expectedCurrent") != release_id: if descriptor.get("expectedCurrent") != release_id:
die("Engine n8n rollback expected-current release mismatch") die("Engine n8n rollback expected-current release mismatch")
if descriptor.get("rollbackBaseline") != "verified_inactive":
die("Engine n8n inactive rollback baseline mismatch")
if descriptor.get("expectedNodeTypes") != [] or descriptor.get("expectedCredentialTypes") != []: if descriptor.get("expectedNodeTypes") != [] or descriptor.get("expectedCredentialTypes") != []:
die("Engine n8n rollback must select the inactive catalog") die("Engine n8n rollback must select the inactive catalog")
return descriptor return descriptor
@ -2275,10 +2368,13 @@ def validate_engine_n8n_catalog_payload(payload_dir, descriptor):
die("Engine n8n light icon sha256 mismatch") die("Engine n8n light icon sha256 mismatch")
if sha256_file(payload_dir / ENGINE_N8N_DARK_ICON_REL) != ENGINE_N8N_DARK_ICON_SHA256: if sha256_file(payload_dir / ENGINE_N8N_DARK_ICON_REL) != ENGINE_N8N_DARK_ICON_SHA256:
die("Engine n8n dark icon sha256 mismatch") die("Engine n8n dark icon sha256 mismatch")
release_hashes = ENGINE_N8N_RELEASE_CATALOG_JSON_SHA256.get(descriptor["releaseId"])
if release_hashes is None:
die("Engine n8n activation release catalog is not registered")
expected_catalog_hashes = ( expected_catalog_hashes = (
(nodes, ENGINE_N8N_ACTIVATION_NODES_CATALOG_JSON_SHA256, "node"), (nodes, release_hashes["nodes"], "node"),
(credentials, ENGINE_N8N_ACTIVATION_CREDENTIALS_CATALOG_JSON_SHA256, "credential"), (credentials, release_hashes["credentials"], "credential"),
(meta, ENGINE_N8N_ACTIVATION_META_JSON_SHA256, "metadata"), (meta, release_hashes["meta"], "metadata"),
) )
else: else:
expected_catalog_hashes = ( expected_catalog_hashes = (
@ -2683,10 +2779,18 @@ def validate_engine_node_intelligence_transition(payload_dir, entries):
"engine_get_node_guidance", "engine_get_node_guidance",
"engine_validate_node_configuration", "engine_validate_node_configuration",
"engine_validate_l2_deep", "engine_validate_l2_deep",
"const ENGINE_AGENT_MCP_VERSION = '0.3.0'",
): ):
if required not in gateway_text: if required not in gateway_text:
die(f"Engine node-intelligence gateway contract missing: {required}") die(f"Engine node-intelligence gateway contract missing: {required}")
if not any(
version in gateway_text
for version in (
"const ENGINE_AGENT_MCP_VERSION = '0.3.0'",
"const ENGINE_AGENT_MCP_VERSION = '0.4.0'",
"const ENGINE_AGENT_MCP_VERSION = '0.5.0'",
)
):
die("Engine node-intelligence gateway MCP version is not registered")
validate_engine_node_intelligence_image_archive( validate_engine_node_intelligence_image_archive(
payload_dir / ENGINE_NODE_INTELLIGENCE_IMAGE_ARCHIVE_REL, payload_dir / ENGINE_NODE_INTELLIGENCE_IMAGE_ARCHIVE_REL,
descriptor, descriptor,
@ -2705,6 +2809,245 @@ def validate_engine_node_intelligence_transition(payload_dir, entries):
return descriptor return descriptor
def is_engine_mcp_control_plane_slice(component, entries):
return (
component == "engine"
and entries is not None
and tuple(entries) == ENGINE_MCP_CONTROL_PLANE_ARTIFACT_ENTRIES
)
def validate_engine_mcp_control_plane_payload(payload_dir, entries):
if tuple(entries) != ENGINE_MCP_CONTROL_PLANE_ARTIFACT_ENTRIES:
die("Engine MCP control-plane files.txt exact set/order mismatch")
for rel, expected_sha256 in ENGINE_MCP_CONTROL_PLANE_TARGET_SHA256.items():
path = payload_dir / rel
if path.is_symlink() or not path.is_file() or sha256_file(path) != expected_sha256:
die(f"Engine MCP control-plane target sha256 mismatch: {rel}")
gateway = (payload_dir / ENGINE_NODE_INTELLIGENCE_GATEWAY_REL).read_text(encoding="utf-8")
graph_route = (payload_dir / "nodedc-source/server/routes/ndcAgentMcp.js").read_text(
encoding="utf-8"
)
installer = (
payload_dir
/ "nodedc-source/server/assets/engine-agent-npm/bin/nodedc-engine-codex-agent.mjs"
).read_text(encoding="utf-8")
package = read_strict_json(
payload_dir / "nodedc-source/server/assets/engine-agent-npm/package.json",
"Engine MCP Codex installer package",
)
if package.get("name") != "@nodedc/engine-codex-agent" or package.get("version") != "0.1.4":
die("Engine MCP Codex installer package identity mismatch")
if any(
marker not in gateway
for marker in (
"const ENGINE_AGENT_MCP_VERSION = '0.5.0'",
"engine_open_l2_change_session",
"engine_get_l2_change_session",
"engine_close_l2_change_session",
"never_infer_expiry",
"three_identical_failures_without_new_evidence",
"engine_plan_data_product_read_grant",
"engine_apply_data_product_read_grant",
"engine_accept_data_product_read_grant",
"engine_rollback_data_product_read_grant",
)
):
die("Engine MCP bounded-change-session contract is incomplete")
if any(
marker not in graph_route
for marker in (
"update_node_no_effect",
"subworkflow_post_write_equality_failed",
"verifiedWrite: true",
"graphDigest: expectedDigest",
)
):
die("Engine MCP graph post-write equality contract is incomplete")
if any(
marker not in installer
for marker in (
"Do not interrupt the user after ordinary recoverable errors",
"Never infer that a provider or managed capability token expired",
"Three identical failures with no new evidence",
"engine_plan_data_product_*_grant",
"durable until an explicit rollback or revoke",
)
):
die("Engine MCP Codex bounded-autonomy policy is incomplete")
combined = "\n".join((gateway, graph_route, installer)).lower()
if "gelios" in combined or "robot2b" in combined:
die("provider identity is forbidden in Engine MCP control-plane source")
archive_path = payload_dir / "nodedc-source/server/assets/nodedc-engine-codex-agent-0.1.4.tgz"
expected_members = {
"package/bin/nodedc-engine-codex-agent.mjs": (
payload_dir
/ "nodedc-source/server/assets/engine-agent-npm/bin/nodedc-engine-codex-agent.mjs"
).read_bytes(),
"package/package.json": (
payload_dir / "nodedc-source/server/assets/engine-agent-npm/package.json"
).read_bytes(),
}
observed_members = {}
try:
with tarfile.open(archive_path, "r:gz") as archive:
for member in archive:
if not (member.isfile() or member.isdir()) or member.name.startswith("/"):
die("Engine MCP Codex installer archive member is unsafe")
if member.isfile():
source = archive.extractfile(member)
if source is None:
die("Engine MCP Codex installer archive member is unreadable")
observed_members[member.name] = source.read(MAX_FILE_BYTES + 1)
except (OSError, tarfile.TarError):
die("Engine MCP Codex installer archive is invalid")
if observed_members != expected_members:
die("Engine MCP Codex installer archive/source equality mismatch")
n8n_route = (payload_dir / "nodedc-source/server/routes/n8n.js").read_text(
encoding="utf-8"
)
if "engineDataProductReadGrantN8nAdapter" not in n8n_route:
die("Engine MCP managed reader native adapter is missing")
if "ndcDataProductReaderApi" not in n8n_route or "ndc_edprb_" not in n8n_route:
die("Engine MCP managed reader credential contract is incomplete")
if "read_grant_must_be_durable" not in n8n_route:
die("Engine MCP managed reader durable lifecycle is missing")
reader_service = (
payload_dir / "nodedc-source/server/dataProductReadGrant/service.js"
).read_text(encoding="utf-8")
if (
"readerGrantLifetime: 'explicit-revoke'" not in reader_service
or "expiresAt: null" not in reader_service
or "ENGINE_READ_GRANT_TTL_DAYS" in reader_service
):
die("Engine MCP managed reader explicit-revoke lifecycle is incomplete")
publish_service = (
payload_dir / "nodedc-source/server/dataProductPublishGrant/service.js"
).read_text(encoding="utf-8")
if (
"writerGrantLifetime: 'explicit-revoke'" not in publish_service
or "migrateCurrentGrantToDurable" not in publish_service
or "expiresAt: null" not in publish_service
or "ENGINE_PUBLISH_GRANT_TTL_DAYS" in publish_service
):
die("Engine MCP managed writer explicit-revoke lifecycle is incomplete")
reader_override = payload_dir / ENGINE_DATA_PRODUCT_READ_GRANT_OVERRIDE_REL
try:
reader_override_text = reader_override.read_text(encoding="utf-8")
except (OSError, UnicodeDecodeError):
die("Engine MCP managed reader runtime override is unreadable")
if reader_override_text != expected_engine_data_product_read_grant_override():
die("Engine MCP managed reader runtime override mismatch")
agent_store = (payload_dir / "nodedc-source/server/engineAgents/store.js").read_text(
encoding="utf-8"
)
for scope in (
"'engine:l2:data-product-read-grant:plan'",
"'engine:l2:data-product-read-grant:write'",
):
if scope not in agent_store:
die(f"Engine MCP managed reader scope is missing: {scope}")
descriptor = read_engine_node_intelligence_descriptor(
payload_dir / ENGINE_MCP_CONTROL_PLANE_DESCRIPTOR_REL,
"Engine MCP control-plane node-intelligence descriptor",
)
if (
descriptor.get("action") != "activate"
or descriptor["source"].get("gatewaySha256")
!= ENGINE_MCP_CONTROL_PLANE_TARGET_SHA256[ENGINE_NODE_INTELLIGENCE_GATEWAY_REL]
or descriptor["source"].get("upstreamProjectionSha256")
!= ENGINE_MCP_CONTROL_PLANE_TARGET_SHA256[
"nodedc-source/server/nodeIntelligence/upstreamProjection.js"
]
):
die("Engine MCP control-plane descriptor target mismatch")
return descriptor
def preflight_engine_mcp_control_plane_predecessor(payload_dir):
candidate = validate_engine_mcp_control_plane_payload(
payload_dir,
ENGINE_MCP_CONTROL_PLANE_ARTIFACT_ENTRIES,
)
installed = current_engine_node_intelligence_descriptor()
validate_installed_engine_node_intelligence_source(installed)
if installed is None or installed.get("action") != "activate":
die("Engine MCP control-plane requires active node intelligence")
expected_candidate = json.loads(json.dumps(installed))
expected_candidate["source"]["gatewaySha256"] = (
ENGINE_MCP_CONTROL_PLANE_TARGET_SHA256[ENGINE_NODE_INTELLIGENCE_GATEWAY_REL]
)
expected_candidate["source"]["upstreamProjectionSha256"] = (
ENGINE_MCP_CONTROL_PLANE_TARGET_SHA256[
"nodedc-source/server/nodeIntelligence/upstreamProjection.js"
]
)
if candidate != expected_candidate:
die("Engine MCP control-plane descriptor crosses node-intelligence source boundary")
root = component_root("engine")
for rel, expected_sha256 in ENGINE_MCP_CONTROL_PLANE_PREDECESSOR_SHA256.items():
path = root / rel
if path.is_symlink() or not path.is_file() or sha256_file(path) != expected_sha256:
die(f"Engine MCP control-plane predecessor drift detected: {rel}")
for rel in ENGINE_MCP_CONTROL_PLANE_NEW_PATHS:
path = root / rel
if path.exists() or path.is_symlink():
die(f"Engine MCP control-plane new path already exists: {rel}")
backend = preflight_engine_credential_backend_runtime()
if backend["mode"] != "verified-derived-retry":
die("Engine MCP control-plane requires the active immutable backend runtime")
return {
"descriptor": candidate,
"predecessor_gateway_sha256": installed["source"]["gatewaySha256"],
"target_gateway_sha256": candidate["source"]["gatewaySha256"],
"backend_mode": backend["mode"],
}
def accept_engine_mcp_control_plane_runtime():
root = component_root("engine")
descriptor = validate_engine_mcp_control_plane_payload(
root,
ENGINE_MCP_CONTROL_PLANE_ARTIFACT_ENTRIES,
)
installed = current_engine_node_intelligence_descriptor()
if installed != descriptor:
die("Engine MCP control-plane installed descriptor equality failed")
validate_installed_engine_node_intelligence_source(installed)
backend = preflight_engine_credential_backend_runtime()
if backend["mode"] != "verified-derived-retry":
die("Engine MCP control-plane backend runtime acceptance failed")
live = run_engine_backend_probe(
(
"node",
"--input-type=module",
"-e",
"""
const module=await import('file:///app/server/routes/engineAgentGateway.js');
const store=await import('file:///app/server/engineAgents/store.js');
const names=new Set(module.engineAgentTools.map((tool)=>tool.name));
const required=['engine_plan_data_product_read_grant','engine_apply_data_product_read_grant','engine_accept_data_product_read_grant','engine_rollback_data_product_read_grant'];
const scopes=['engine:l2:data-product-read-grant:plan','engine:l2:data-product-read-grant:write'];
if(!required.every((name)=>names.has(name))||!scopes.every((scope)=>store.ENGINE_AGENT_SCOPES.includes(scope)))process.exit(2);
process.stdout.write('engine-mcp-reader-grant:0.5.0:'+required.length+':'+scopes.length);
""".strip(),
),
"Engine MCP managed reader capability",
container_id=engine_backend_container_id(),
)
if live != "engine-mcp-reader-grant:0.5.0:4:2":
die("Engine MCP managed reader live capability acceptance mismatch")
return {
"gateway_sha256": descriptor["source"]["gatewaySha256"],
"backend_mode": backend["mode"],
"reader_grant": live,
}
def validate_no_lifecycle_scripts(payload_dir, label): def validate_no_lifecycle_scripts(payload_dir, label):
forbidden = ("preinstall", "install", "postinstall", "prepare", "prepack", "postpack") forbidden = ("preinstall", "install", "postinstall", "prepare", "prepack", "postpack")
for package_path in payload_dir.rglob("package.json"): for package_path in payload_dir.rglob("package.json"):
@ -2800,6 +3143,22 @@ def expected_engine_data_product_publish_grant_override():
)) ))
def expected_engine_data_product_read_grant_override():
return "\n".join((
"services:",
" nodedc-backend:",
" environment:",
f" ENGINE_CONTROL_PLANE_READ_GRANT_ROOT: {ENGINE_READ_GRANT_CONTAINER_PATH}",
" volumes:",
" - type: bind",
f" source: {ENGINE_READ_GRANT_STATE_PATH}",
f" target: {ENGINE_READ_GRANT_CONTAINER_PATH}",
" bind:",
" create_host_path: false",
"",
))
def validate_engine_credential_sink_slice(payload_dir, entries): def validate_engine_credential_sink_slice(payload_dir, entries):
if tuple(entries) != ENGINE_CREDENTIAL_SINK_ARTIFACT_ENTRIES: if tuple(entries) != ENGINE_CREDENTIAL_SINK_ARTIFACT_ENTRIES:
die("Engine credential sink files.txt exact set/order mismatch") die("Engine credential sink files.txt exact set/order mismatch")
@ -3103,20 +3462,30 @@ def load_artifact(artifact, work_dir):
or rel.startswith(ENGINE_NODE_INTELLIGENCE_SERVICE_ROOT_REL + "/") or rel.startswith(ENGINE_NODE_INTELLIGENCE_SERVICE_ROOT_REL + "/")
for rel in entries for rel in entries
) )
if touches_node_intelligence and not is_engine_node_intelligence_transition( if (
manifest["component"], entries touches_node_intelligence
and not is_engine_node_intelligence_transition(manifest["component"], entries)
and not is_engine_mcp_control_plane_slice(manifest["component"], entries)
): ):
die("Engine node-intelligence payload requires the canonical transition") die("Engine node-intelligence payload requires the canonical transition")
if is_engine_node_intelligence_transition(manifest["component"], entries): if is_engine_node_intelligence_transition(manifest["component"], entries):
validate_engine_node_intelligence_transition(payload_dir, entries) validate_engine_node_intelligence_transition(payload_dir, entries)
if is_engine_mcp_control_plane_slice(manifest["component"], entries):
validate_engine_mcp_control_plane_payload(payload_dir, entries)
if touches_engine_credential_sink(manifest["component"], entries): if touches_engine_credential_sink(manifest["component"], entries):
validate_engine_credential_sink_slice(payload_dir, entries) validate_engine_credential_sink_slice(payload_dir, entries)
if ( if (
not is_engine_mcp_control_plane_slice(manifest["component"], entries)
and (
touches_engine_data_product_publish_grant(entries) touches_engine_data_product_publish_grant(entries)
or ENGINE_DATA_PRODUCT_PUBLISH_GRANT_OVERRIDE_REL in entries or ENGINE_DATA_PRODUCT_PUBLISH_GRANT_OVERRIDE_REL in entries
)
): ):
validate_engine_data_product_publish_grant_slice(payload_dir, entries) validate_engine_data_product_publish_grant_slice(payload_dir, entries)
elif ENGINE_AGENT_FULL_GRANT_MIGRATION_STORE_REL in entries: elif (
not is_engine_mcp_control_plane_slice(manifest["component"], entries)
and ENGINE_AGENT_FULL_GRANT_MIGRATION_STORE_REL in entries
):
validate_engine_agent_full_grant_migration_slice(payload_dir, entries) validate_engine_agent_full_grant_migration_slice(payload_dir, entries)
return manifest, entries, payload_dir return manifest, entries, payload_dir
@ -3202,6 +3571,7 @@ def touches_external_data_plane_files(entries):
"platform/packages/external-provider-contract/src/data-plane.mjs", "platform/packages/external-provider-contract/src/data-plane.mjs",
"platform/packages/external-provider-contract/src/data-product.mjs", "platform/packages/external-provider-contract/src/data-product.mjs",
"platform/packages/external-provider-contract/src/intake-batch.mjs", "platform/packages/external-provider-contract/src/intake-batch.mjs",
"platform/packages/external-provider-contract/src/index.mjs",
"platform/packages/external-provider-contract/src/sensitive-field-policy.mjs", "platform/packages/external-provider-contract/src/sensitive-field-policy.mjs",
} }
return any( return any(
@ -3462,6 +3832,12 @@ def is_platform_provider_catalog_only(entries):
def component_services(component, entries=None): def component_services(component, entries=None):
if is_engine_mcp_control_plane_slice(component, entries):
# This slice updates only the existing Engine backend control plane.
# Node intelligence keeps the same immutable sidecar image and n8n/L1
# retain their current generations.
return ("nodedc-backend",)
if is_engine_node_intelligence_transition(component, entries): if is_engine_node_intelligence_transition(component, entries):
if tuple(entries) == ENGINE_NODE_INTELLIGENCE_ARTIFACT_ENTRIES: if tuple(entries) == ENGINE_NODE_INTELLIGENCE_ARTIFACT_ENTRIES:
return (ENGINE_NODE_INTELLIGENCE_SERVICE, "nodedc-backend") return (ENGINE_NODE_INTELLIGENCE_SERVICE, "nodedc-backend")
@ -3653,6 +4029,18 @@ def component_compose_files(component, allow_prepared_engine_backend=False):
# Canonical order is immutable: base, active L2 extension, credential # Canonical order is immutable: base, active L2 extension, credential
# sink, then the additive Publish grant overlay. # sink, then the additive Publish grant overlay.
files.append(publish_grant_override) files.append(publish_grant_override)
read_grant_override = root / ENGINE_DATA_PRODUCT_READ_GRANT_OVERRIDE_REL
if read_grant_override.exists() or read_grant_override.is_symlink():
if read_grant_override.is_symlink() or not read_grant_override.is_file():
die("installed Engine data product read grant override is unsafe")
try:
read_grant_override_text = read_grant_override.read_text(encoding="utf-8")
except (OSError, UnicodeDecodeError):
die("installed Engine data product read grant override cannot be read")
if read_grant_override_text != expected_engine_data_product_read_grant_override():
die("installed Engine data product read grant override drift detected")
# Read authority is additive and ordered after the writer/private-key overlay.
files.append(read_grant_override)
node_intelligence_descriptor = current_engine_node_intelligence_descriptor() node_intelligence_descriptor = current_engine_node_intelligence_descriptor()
if ( if (
node_intelligence_descriptor is not None node_intelligence_descriptor is not None
@ -4164,6 +4552,22 @@ def validate_engine_backend_mounts_for_installed_runtime(
str(ENGINE_EDP_MANAGED_PROVISIONER_PRIVATE_KEY_FILE), str(ENGINE_EDP_MANAGED_PROVISIONER_PRIVATE_KEY_FILE),
), ),
} }
read_override = component_root("engine") / ENGINE_DATA_PRODUCT_READ_GRANT_OVERRIDE_REL
read_installed = read_override.exists() or read_override.is_symlink()
if read_installed:
if read_override.is_symlink() or not read_override.is_file():
die("installed Engine data product read grant override is unsafe")
try:
read_override_text = read_override.read_text(encoding="utf-8")
except (OSError, UnicodeDecodeError):
die("installed Engine data product read grant override cannot be read")
if read_override_text != expected_engine_data_product_read_grant_override():
die("installed Engine data product read grant override drift detected")
expected_publish_mounts[ENGINE_READ_GRANT_CONTAINER_PATH] = (
"bind",
True,
str(ENGINE_READ_GRANT_STATE_PATH),
)
mounts = container.get("Mounts") mounts = container.get("Mounts")
if not isinstance(mounts, list): if not isinstance(mounts, list):
die("Engine backend mount inventory is missing") die("Engine backend mount inventory is missing")
@ -5306,10 +5710,6 @@ def preflight_engine_n8n_transition(descriptor, enforce_expected_current):
"Engine n8n transition stale current state: " "Engine n8n transition stale current state: "
f"expected={descriptor['expectedCurrent']} actual={current_state}" f"expected={descriptor['expectedCurrent']} actual={current_state}"
) )
if current_descriptor and current_state != "verified_inactive":
if current_descriptor.get("packageSha256") != descriptor["packageSha256"]:
if enforce_expected_current:
die("Engine n8n current release package sha256 mismatch")
return { return {
"base_image_id": base_image["id"], "base_image_id": base_image["id"],
"base_image_architecture": base_image["architecture"], "base_image_architecture": base_image["architecture"],
@ -5558,6 +5958,7 @@ def plan_artifact(artifact):
sha = sha256_file(artifact) sha = sha256_file(artifact)
publish_grant_preflight = None publish_grant_preflight = None
node_intelligence_preflight = None node_intelligence_preflight = None
mcp_control_plane_preflight = None
with tempfile.TemporaryDirectory(prefix="plan-", dir=TMP_DIR) as tmp: with tempfile.TemporaryDirectory(prefix="plan-", dir=TMP_DIR) as tmp:
manifest, entries, payload_dir = load_artifact(artifact, Path(tmp)) manifest, entries, payload_dir = load_artifact(artifact, Path(tmp))
transition_descriptor = None transition_descriptor = None
@ -5578,6 +5979,10 @@ def plan_artifact(artifact):
node_intelligence_preflight = preflight_engine_node_intelligence_predecessor( node_intelligence_preflight = preflight_engine_node_intelligence_predecessor(
payload_dir payload_dir
) )
if is_engine_mcp_control_plane_slice(manifest["component"], entries):
mcp_control_plane_preflight = preflight_engine_mcp_control_plane_predecessor(
payload_dir
)
component = manifest["component"] component = manifest["component"]
root = component_root(component) root = component_root(component)
@ -5644,6 +6049,22 @@ def plan_artifact(artifact):
print("node_intelligence_host_ports=none") print("node_intelligence_host_ports=none")
print(f"predecessor_gateway_sha256={node_intelligence_preflight['gateway_sha256']}") print(f"predecessor_gateway_sha256={node_intelligence_preflight['gateway_sha256']}")
print(f"backend_current_barrier={node_intelligence_preflight['backend_mode']}") print(f"backend_current_barrier={node_intelligence_preflight['backend_mode']}")
if mcp_control_plane_preflight:
print("engine_mcp_transition=managed-reader-grant+validator-reconciliation")
print("engine_mcp_version=0.5.0")
print("engine_mcp_installer=0.1.4")
print("engine_mcp_reader_grant=plan+apply+accept+rollback")
print(
"predecessor_gateway_sha256="
f"{mcp_control_plane_preflight['predecessor_gateway_sha256']}"
)
print(
"target_gateway_sha256="
f"{mcp_control_plane_preflight['target_gateway_sha256']}"
)
print(f"backend_current_barrier={mcp_control_plane_preflight['backend_mode']}")
print("node_intelligence_image=preserved")
print("n8n_l1=untouched")
if transition_descriptor: if transition_descriptor:
print(f"n8n_transition={transition_descriptor['action']}") print(f"n8n_transition={transition_descriptor['action']}")
print(f"n8n_version={transition_descriptor['n8nVersion']}") print(f"n8n_version={transition_descriptor['n8nVersion']}")
@ -5665,7 +6086,7 @@ def plan_artifact(artifact):
print("n8n_database=preserved:n8n-postgres") print("n8n_database=preserved:n8n-postgres")
print(f"n8n_expected_node_types={','.join(transition_descriptor['expectedNodeTypes'])}") print(f"n8n_expected_node_types={','.join(transition_descriptor['expectedNodeTypes'])}")
print(f"n8n_expected_credential_types={','.join(transition_descriptor['expectedCredentialTypes'])}") print(f"n8n_expected_credential_types={','.join(transition_descriptor['expectedCredentialTypes'])}")
print("n8n_rollback_baseline=verified_inactive") print(f"n8n_rollback_baseline={transition_descriptor['rollbackBaseline']}")
touches_map_gateway = component == "platform" and any(rel == "platform/services/map-gateway" or rel.startswith("platform/services/map-gateway/") for rel in entries) touches_map_gateway = component == "platform" and any(rel == "platform/services/map-gateway" or rel.startswith("platform/services/map-gateway/") for rel in entries)
touches_external_data_plane = component == "platform" and touches_external_data_plane_files(entries) touches_external_data_plane = component == "platform" and touches_external_data_plane_files(entries)
if component == "module-foundry" or touches_map_gateway: if component == "module-foundry" or touches_map_gateway:
@ -6387,6 +6808,10 @@ def prepare_component_runtime(component, entries=None):
ensure_engine_edp_managed_provisioner_keypair() ensure_engine_edp_managed_provisioner_keypair()
ensure_engine_publish_grant_private_state() ensure_engine_publish_grant_private_state()
if is_engine_mcp_control_plane_slice(component, entries):
ensure_engine_edp_managed_provisioner_keypair()
ensure_engine_data_product_grant_private_state(include_reader=True)
if is_engine_n8n_transition(component, entries): if is_engine_n8n_transition(component, entries):
descriptor = current_engine_n8n_transition_descriptor() descriptor = current_engine_n8n_transition_descriptor()
if descriptor is None: if descriptor is None:
@ -6553,6 +6978,20 @@ def external_data_plane_healthcheck(require_managed=True):
} }
if require_managed: if require_managed:
expected_json["managedWriterBindingProvisioning"] = "enabled" expected_json["managedWriterBindingProvisioning"] = "enabled"
server_source = (
component_root("platform")
/ "platform/services/external-data-plane/src/server.mjs"
)
try:
server_text = server_source.read_text(encoding="utf-8")
except (OSError, UnicodeDecodeError):
server_text = ""
if 'managedReaderBindingProvisioning: config.managedProvisionerApiEnabled' in server_text:
expected_json["managedReaderBindingProvisioning"] = "enabled"
if 'managedWriterBindingLifetime: config.managedProvisionerApiEnabled ? "explicit-revoke" : "disabled"' in server_text:
expected_json["managedWriterBindingLifetime"] = "explicit-revoke"
if 'managedReaderBindingLifetime: config.managedProvisionerApiEnabled ? "explicit-revoke" : "disabled"' in server_text:
expected_json["managedReaderBindingLifetime"] = "explicit-revoke"
return { return {
"url": "http://127.0.0.1:18106/healthz", "url": "http://127.0.0.1:18106/healthz",
"expected_json": expected_json, "expected_json": expected_json,
@ -6576,6 +7015,7 @@ def component_healthchecks(component, entries=None, services=None):
( (
is_engine_data_product_publish_grant_slice(component, entries) is_engine_data_product_publish_grant_slice(component, entries)
or is_engine_agent_full_grant_migration_slice(component, entries) or is_engine_agent_full_grant_migration_slice(component, entries)
or is_engine_mcp_control_plane_slice(component, entries)
) )
and services is not None and services is not None
): ):
@ -6888,10 +7328,12 @@ def run_healthchecks(component, entries=None, services=None):
component, component,
entries, entries,
) )
touches_mcp_control_plane = is_engine_mcp_control_plane_slice(component, entries)
if ( if (
touches_engine_credential_sink(component, entries) touches_engine_credential_sink(component, entries)
or touches_publish_grant or touches_publish_grant
or touches_agent_grant_migration or touches_agent_grant_migration
or touches_mcp_control_plane
): ):
# The HTTP endpoint can become ready before Docker publishes the first # The HTTP endpoint can become ready before Docker publishes the first
# successful health probe. Wait for the Compose health barrier before # successful health probe. Wait for the Compose health barrier before
@ -6903,12 +7345,15 @@ def run_healthchecks(component, entries=None, services=None):
touches_engine_credential_sink(component, entries) touches_engine_credential_sink(component, entries)
or touches_publish_grant or touches_publish_grant
or touches_agent_grant_migration or touches_agent_grant_migration
or touches_mcp_control_plane
): ):
runtime = preflight_engine_credential_backend_runtime() runtime = preflight_engine_credential_backend_runtime()
if runtime["mode"] != "verified-derived-retry": if runtime["mode"] != "verified-derived-retry":
die("Engine backend immutable runtime activation acceptance failed") die("Engine backend immutable runtime activation acceptance failed")
if touches_agent_grant_migration: if touches_agent_grant_migration:
accept_engine_agent_full_grant_migration_runtime() accept_engine_agent_full_grant_migration_runtime()
if touches_mcp_control_plane:
accept_engine_mcp_control_plane_runtime()
container_name = COMPONENTS[component].get("health_container") container_name = COMPONENTS[component].get("health_container")
if container_name: if container_name:
healthcheck_container(container_name) healthcheck_container(container_name)
@ -6983,6 +7428,8 @@ def apply_artifact(artifact):
payload_dir payload_dir
) )
node_intelligence_descriptor = node_intelligence_preflight["descriptor"] node_intelligence_descriptor = node_intelligence_preflight["descriptor"]
if is_engine_mcp_control_plane_slice(component, entries):
preflight_engine_mcp_control_plane_predecessor(payload_dir)
if not artifact_only and not compose_root.is_dir(): if not artifact_only and not compose_root.is_dir():
if bootstrap_root and is_relative_to(compose_root.resolve(strict=False), root.resolve(strict=False)): if bootstrap_root and is_relative_to(compose_root.resolve(strict=False), root.resolve(strict=False)):
compose_root.mkdir(parents=True, exist_ok=True) compose_root.mkdir(parents=True, exist_ok=True)
@ -7159,6 +7606,7 @@ def apply_artifact(artifact):
( (
is_engine_data_product_publish_grant_slice(component, entries) is_engine_data_product_publish_grant_slice(component, entries)
or is_engine_agent_full_grant_migration_slice(component, entries) or is_engine_agent_full_grant_migration_slice(component, entries)
or is_engine_mcp_control_plane_slice(component, entries)
) )
and services is not None and services is not None
): ):

View File

@ -0,0 +1,198 @@
#!/usr/bin/env python3
import hashlib
import importlib.machinery
import importlib.util
import json
import os
import subprocess
import tarfile
import tempfile
import unittest
from unittest import mock
from pathlib import Path
SCRIPT_DIR = Path(__file__).resolve().parent
PLATFORM_ROOT = SCRIPT_DIR.parent.parent
RUNNER_PATH = SCRIPT_DIR / "nodedc-deploy"
BUILDER_PATH = SCRIPT_DIR / "build-engine-mcp-control-plane-artifact.mjs"
TRANSITION_ID = "20260718-003"
STAGE_ARTIFACT = (
PLATFORM_ROOT
/ "infra/deploy-artifacts"
/ "nodedc-engine-mcp-control-plane-20260718-003.tgz"
)
STAGE_ARTIFACT_SHA256 = "249aef9527666c562e9648b15737e66cc5c1dc7c0788b58ea714da270b5eb4ba"
def load_runner():
loader = importlib.machinery.SourceFileLoader("nodedc_engine_mcp_runner", str(RUNNER_PATH))
spec = importlib.util.spec_from_loader(loader.name, loader)
module = importlib.util.module_from_spec(spec)
loader.exec_module(module)
return module
RUNNER = load_runner()
class EngineMcpControlPlaneTest(unittest.TestCase):
@classmethod
def setUpClass(cls):
cls.temporary = tempfile.TemporaryDirectory(prefix="nodedc-engine-mcp-control-plane-")
cls.root = Path(cls.temporary.name)
stage_bytes = STAGE_ARTIFACT.read_bytes()
if hashlib.sha256(stage_bytes).hexdigest() != STAGE_ARTIFACT_SHA256:
raise RuntimeError("engine_mcp_control_plane_stage_artifact_sha256_mismatch")
source_stage = cls.root / "immutable-source"
source_stage.mkdir()
RUNNER.safe_extract(STAGE_ARTIFACT, source_stage)
cls.engine_source_root = source_stage / "payload"
cls.results = []
for index in range(2):
output = cls.root / f"build-{index}"
output.mkdir()
env = os.environ.copy()
env["NODEDC_DEPLOY_ARTIFACT_DIR"] = str(output)
env["NODEDC_ENGINE_SOURCE_ROOT"] = str(cls.engine_source_root)
result = subprocess.run(
["node", str(BUILDER_PATH), TRANSITION_ID],
cwd=PLATFORM_ROOT,
env=env,
check=True,
capture_output=True,
text=True,
)
cls.results.append(json.loads(result.stdout))
@classmethod
def tearDownClass(cls):
cls.temporary.cleanup()
def artifact(self, index=0):
return Path(self.results[index]["artifact"])
def extract(self, artifact, directory):
RUNNER.safe_extract(artifact, directory)
return directory / "payload", RUNNER.parse_files_list(directory / "files.txt")
def test_artifact_is_byte_reproducible_and_canonical(self):
first = self.artifact(0).read_bytes()
second = self.artifact(1).read_bytes()
self.assertEqual(first, second)
self.assertEqual(first, STAGE_ARTIFACT.read_bytes())
self.assertEqual(first[4:8], b"\0\0\0\0")
self.assertEqual(self.results[0]["artifactSha256"], hashlib.sha256(first).hexdigest())
with tarfile.open(self.artifact(0), "r:gz") as archive:
for member in archive:
self.assertTrue(member.isfile() or member.isdir())
self.assertFalse(Path(member.name).name.startswith("._"))
def test_runner_selects_only_existing_backend(self):
with tempfile.TemporaryDirectory() as directory:
manifest, entries, payload = RUNNER.load_artifact(
self.artifact(0),
Path(directory),
)
descriptor = RUNNER.validate_engine_mcp_control_plane_payload(payload, entries)
override_text = (
payload
/ RUNNER.ENGINE_DATA_PRODUCT_READ_GRANT_OVERRIDE_REL
).read_text(encoding="utf-8")
self.assertEqual(manifest["component"], "engine")
self.assertEqual(tuple(entries), RUNNER.ENGINE_MCP_CONTROL_PLANE_ARTIFACT_ENTRIES)
self.assertEqual(RUNNER.component_services("engine", entries), ("nodedc-backend",))
self.assertEqual(descriptor["action"], "activate")
self.assertEqual(
descriptor["source"]["gatewaySha256"],
RUNNER.ENGINE_MCP_CONTROL_PLANE_TARGET_SHA256[
RUNNER.ENGINE_NODE_INTELLIGENCE_GATEWAY_REL
],
)
self.assertEqual(override_text, RUNNER.expected_engine_data_product_read_grant_override())
def test_runtime_preparation_adds_only_private_reader_state(self):
with (
mock.patch.object(
RUNNER,
"ensure_engine_edp_managed_provisioner_keypair",
) as keypair,
mock.patch.object(
RUNNER,
"ensure_engine_data_product_grant_private_state",
) as private_state,
):
RUNNER.prepare_component_runtime(
"engine",
RUNNER.ENGINE_MCP_CONTROL_PLANE_ARTIFACT_ENTRIES,
)
keypair.assert_called_once_with()
private_state.assert_called_once_with(include_reader=True)
def test_candidate_preserves_node_intelligence_identity_except_exact_control_plane_digests(self):
with tempfile.TemporaryDirectory() as directory:
work = Path(directory)
payload, entries = self.extract(self.artifact(0), work)
candidate = RUNNER.validate_engine_mcp_control_plane_payload(payload, entries)
self.assertEqual(candidate["releaseId"], RUNNER.ENGINE_NODE_INTELLIGENCE_RELEASE_ID)
self.assertEqual(candidate["upstream"]["commit"], RUNNER.ENGINE_NODE_INTELLIGENCE_UPSTREAM_COMMIT)
self.assertEqual(candidate["image"]["tag"], RUNNER.ENGINE_NODE_INTELLIGENCE_IMAGE)
self.assertEqual(candidate["expectedCurrent"], "inactive")
self.assertEqual(
candidate["source"]["upstreamProjectionSha256"],
RUNNER.ENGINE_MCP_CONTROL_PLANE_TARGET_SHA256[
"nodedc-source/server/nodeIntelligence/upstreamProjection.js"
],
)
def test_descriptor_boundary_and_source_digest_are_enforced(self):
with tempfile.TemporaryDirectory() as directory:
work = Path(directory)
payload, entries = self.extract(self.artifact(0), work)
descriptor_path = payload / RUNNER.ENGINE_MCP_CONTROL_PLANE_DESCRIPTOR_REL
descriptor = json.loads(descriptor_path.read_text(encoding="utf-8"))
descriptor["source"]["catalogSha256"] = "0" * 64
descriptor_path.write_text(json.dumps(descriptor), encoding="utf-8")
# Payload validation permits only a structurally valid descriptor;
# predecessor equality is the state-aware barrier for untouched
# node-intelligence source identities.
with self.assertRaisesRegex(
RUNNER.DeployError,
"descriptor crosses node-intelligence source boundary",
):
installed = json.loads(json.dumps(descriptor))
installed["source"]["catalogSha256"] = "1" * 64
installed["source"]["gatewaySha256"] = (
RUNNER.ENGINE_MCP_CONTROL_PLANE_PREDECESSOR_SHA256[
RUNNER.ENGINE_NODE_INTELLIGENCE_GATEWAY_REL
]
)
from unittest import mock
with (
mock.patch.object(
RUNNER,
"current_engine_node_intelligence_descriptor",
return_value=installed,
),
mock.patch.object(RUNNER, "validate_installed_engine_node_intelligence_source"),
):
RUNNER.preflight_engine_mcp_control_plane_predecessor(payload)
def test_existing_artifact_is_not_overwritten(self):
env = os.environ.copy()
env["NODEDC_DEPLOY_ARTIFACT_DIR"] = str(self.artifact(0).parent)
env["NODEDC_ENGINE_SOURCE_ROOT"] = str(self.engine_source_root)
result = subprocess.run(
["node", str(BUILDER_PATH), TRANSITION_ID],
cwd=PLATFORM_ROOT,
env=env,
check=False,
capture_output=True,
text=True,
)
self.assertNotEqual(result.returncode, 0)
self.assertIn("artifact_already_exists", result.stderr)
if __name__ == "__main__":
unittest.main(verbosity=2)

View File

@ -21,11 +21,12 @@ BUILDER_PATH = SCRIPT_DIR / "build-engine-n8n-private-extension-artifact.mjs"
STAGE_ARTIFACT = ( STAGE_ARTIFACT = (
PLATFORM_ROOT PLATFORM_ROOT
/ "infra/deploy-artifacts" / "infra/deploy-artifacts"
/ "nodedc-n8n-private-extension-n8n-nodes-ndc-release-20260716-003.tgz" / "nodedc-n8n-private-extension-n8n-nodes-ndc-history-read-20260717-004.tgz"
) )
TRANSITION_ID = "20260717-004" TRANSITION_ID = "20260717-005"
SEALED_RELEASE_ID = "0.1.2-05e4b38b14b4a019" SEALED_RELEASE_ID = "0.1.3-3354149b5245e39a"
SEALED_PACKAGE_SHA256 = "05e4b38b14b4a019ce1f6eee27b9e320094cb3560903bd68074966b3a1267af5" SEALED_PACKAGE_SHA256 = "3354149b5245e39a10f6f03a4b43796602d7e57ea1f91dcc6cb6774ead97501d"
PREDECESSOR_RELEASE_ID = "0.1.2-05e4b38b14b4a019"
BUILDER_ENGINE_PATHS = ( BUILDER_ENGINE_PATHS = (
"nodedc-source/server/assets/n8n/schema/v2.3.2/nodes.catalog.json", "nodedc-source/server/assets/n8n/schema/v2.3.2/nodes.catalog.json",
"nodedc-source/server/assets/n8n/schema/v2.3.2/credentials.catalog.json", "nodedc-source/server/assets/n8n/schema/v2.3.2/credentials.catalog.json",
@ -50,7 +51,7 @@ PRIVATE_EXTENSION_CANON_FUNCTION_SHA256 = {
"engine_n8n_package_loader_probe_script": "2b3b3c96fad6e5e48ebea74426b21bc7eea6b6fe0e786b537da3366d18aedd9b", "engine_n8n_package_loader_probe_script": "2b3b3c96fad6e5e48ebea74426b21bc7eea6b6fe0e786b537da3366d18aedd9b",
"engine_n8n_private_loader_catalog": "a2ae389b4ef215900cf967b863d01056bd2a8eec55554566d0f6005e4e0bcbac", "engine_n8n_private_loader_catalog": "a2ae389b4ef215900cf967b863d01056bd2a8eec55554566d0f6005e4e0bcbac",
"validate_engine_n8n_base_compose_source": "adacc5b20e52684cbc427362ddba5a6d2e13ef2091a89859e2b8f7bbf2e20458", "validate_engine_n8n_base_compose_source": "adacc5b20e52684cbc427362ddba5a6d2e13ef2091a89859e2b8f7bbf2e20458",
"preflight_engine_n8n_transition": "da3cec853e3e3e20df4d1e301aab98b77815f93ec3d68b0b61c84c6e6a3fb335", "preflight_engine_n8n_transition": "3d574acb081fb752e45cc0c3286d3ae4596dcee83a85e2c3fee81ac65ec796e0",
"accept_engine_n8n_runtime": "1a2614a465161e3833e84aca402817682da1a538bdb2aecc4af66b665497338d", "accept_engine_n8n_runtime": "1a2614a465161e3833e84aca402817682da1a538bdb2aecc4af66b665497338d",
} }
@ -138,7 +139,7 @@ class EngineN8nPrivateExtensionTest(unittest.TestCase):
self.assertEqual(first[4:8], b"\0\0\0\0") self.assertEqual(first[4:8], b"\0\0\0\0")
self.assertEqual(first[3] & 0x08, 0) self.assertEqual(first[3] & 0x08, 0)
def test_successful_generation_003_runner_functions_are_frozen(self): def test_state_aware_upgrade_runner_functions_are_frozen(self):
actual = { actual = {
name: hashlib.sha256(inspect.getsource(getattr(RUNNER, name)).encode("utf-8")).hexdigest() name: hashlib.sha256(inspect.getsource(getattr(RUNNER, name)).encode("utf-8")).hexdigest()
for name in PRIVATE_EXTENSION_CANON_FUNCTION_SHA256 for name in PRIVATE_EXTENSION_CANON_FUNCTION_SHA256
@ -164,8 +165,8 @@ class EngineN8nPrivateExtensionTest(unittest.TestCase):
def test_transition_id_rejects_missing_invalid_and_previously_issued_values(self): def test_transition_id_rejects_missing_invalid_and_previously_issued_values(self):
cases = ( cases = (
([], "transition_id_required"), ([], "transition_id_required"),
(["20260716-003"], "transition_id_already_issued"), (["20260717-004"], "transition_id_already_issued"),
(["engine-n8n-private-extension-20260717-004"], "transition_id_invalid"), (["engine-n8n-private-extension-20260717-005"], "transition_id_invalid"),
(["20260230-004"], "transition_id_invalid"), (["20260230-004"], "transition_id_invalid"),
(["20260717-000"], "transition_id_invalid"), (["20260717-000"], "transition_id_invalid"),
([TRANSITION_ID, "unexpected-second-id"], "transition_id_argument_count_invalid"), ([TRANSITION_ID, "unexpected-second-id"], "transition_id_argument_count_invalid"),
@ -202,7 +203,7 @@ class EngineN8nPrivateExtensionTest(unittest.TestCase):
self.assertIn("transition_artifact_already_exists", result.stderr) self.assertIn("transition_artifact_already_exists", result.stderr)
def test_activation_and_rollback_pass_strict_runner_policy(self): def test_activation_and_rollback_pass_strict_runner_policy(self):
expected = {"activation": ("activate", 7), "rollback": ("rollback-inactive", 4)} expected = {"activation": ("activate", 7), "rollback": ("activate", 7)}
for kind, (action, entry_count) in expected.items(): for kind, (action, entry_count) in expected.items():
with self.subTest(kind=kind), tempfile.TemporaryDirectory() as directory: with self.subTest(kind=kind), tempfile.TemporaryDirectory() as directory:
manifest, entries, payload = RUNNER.load_artifact( manifest, entries, payload = RUNNER.load_artifact(
@ -234,7 +235,7 @@ class EngineN8nPrivateExtensionTest(unittest.TestCase):
self.assertTrue(all("usableAsTool" not in item for item in private_nodes)) self.assertTrue(all("usableAsTool" not in item for item in private_nodes))
self.assertEqual((len(nodes), len(credentials)), (437, 388)) self.assertEqual((len(nodes), len(credentials)), (437, 388))
def test_rollback_catalog_restores_verified_inactive_baseline(self): def test_rollback_catalog_restores_verified_predecessor_release(self):
with tempfile.TemporaryDirectory() as directory: with tempfile.TemporaryDirectory() as directory:
_manifest, _entries, payload = RUNNER.load_artifact( _manifest, _entries, payload = RUNNER.load_artifact(
self.artifact(0, "rollback"), self.artifact(0, "rollback"),
@ -242,9 +243,20 @@ class EngineN8nPrivateExtensionTest(unittest.TestCase):
) )
nodes = json.loads((payload / RUNNER.ENGINE_N8N_NODES_CATALOG_REL).read_text()) nodes = json.loads((payload / RUNNER.ENGINE_N8N_NODES_CATALOG_REL).read_text())
credentials = json.loads((payload / RUNNER.ENGINE_N8N_CREDENTIALS_CATALOG_REL).read_text()) credentials = json.loads((payload / RUNNER.ENGINE_N8N_CREDENTIALS_CATALOG_REL).read_text())
self.assertEqual([item for item in nodes if item.get("name", "").startswith("n8n-nodes-ndc.")], []) descriptor = RUNNER.read_engine_n8n_transition_descriptor(
self.assertEqual([item for item in credentials if item.get("name") in EXPECTED_CREDENTIALS], []) payload / RUNNER.ENGINE_N8N_TRANSITION_DESCRIPTOR_REL
self.assertEqual((len(nodes), len(credentials)), (434, 385)) )
self.assertEqual(descriptor["releaseId"], PREDECESSOR_RELEASE_ID)
self.assertEqual(descriptor["expectedCurrent"], SEALED_RELEASE_ID)
self.assertEqual(
[item.get("name") for item in nodes if item.get("name", "").startswith("n8n-nodes-ndc.")],
EXPECTED_NODES,
)
self.assertEqual(
[item.get("name") for item in credentials if item.get("name") in EXPECTED_CREDENTIALS],
EXPECTED_CREDENTIALS,
)
self.assertEqual((len(nodes), len(credentials)), (437, 388))
def test_compose_override_is_runner_derived_and_offline(self): def test_compose_override_is_runner_derived_and_offline(self):
with tempfile.TemporaryDirectory() as directory: with tempfile.TemporaryDirectory() as directory:
@ -259,7 +271,7 @@ class EngineN8nPrivateExtensionTest(unittest.TestCase):
self.assertEqual(override, RUNNER.expected_engine_n8n_compose_override(descriptor)) self.assertEqual(override, RUNNER.expected_engine_n8n_compose_override(descriptor))
self.assertEqual( self.assertEqual(
hashlib.sha256(override.encode("utf-8")).hexdigest(), hashlib.sha256(override.encode("utf-8")).hexdigest(),
"a29e2f92b87dda59da2b4e3ce250bc9705ed9fa551c0fbe4e95464fca5caa3e1", "256142c14ae295bf9be4d74fdc8dcd6e134e4ded8a264f15e66b38e85503b589",
) )
self.assertIn("pull_policy: never", override) self.assertIn("pull_policy: never", override)
self.assertIn("N8N_USER_FOLDER: /home/node", override) self.assertIn("N8N_USER_FOLDER: /home/node", override)
@ -268,7 +280,7 @@ class EngineN8nPrivateExtensionTest(unittest.TestCase):
self.assertNotIn("N8N_CUSTOM_EXTENSIONS", override) self.assertNotIn("N8N_CUSTOM_EXTENSIONS", override)
self.assertNotIn("build:", override) self.assertNotIn("build:", override)
def test_generation_003_override_is_accepted_as_installed_canon(self): def test_upgrade_override_is_accepted_as_installed_canon(self):
original_root = RUNNER.COMPONENTS["engine"]["payload_root"] original_root = RUNNER.COMPONENTS["engine"]["payload_root"]
original_compose = RUNNER.COMPONENTS["engine"]["compose_root"] original_compose = RUNNER.COMPONENTS["engine"]["compose_root"]
try: try:
@ -551,7 +563,7 @@ class EngineN8nPrivateExtensionTest(unittest.TestCase):
def test_sealed_release_exact_set_includes_implicit_directories(self): def test_sealed_release_exact_set_includes_implicit_directories(self):
release_relative = Path( release_relative = Path(
"payload/releases/n8n-nodes-ndc/0.1.2-05e4b38b14b4a019" "payload/releases/n8n-nodes-ndc/0.1.3-3354149b5245e39a"
) )
with tempfile.TemporaryDirectory() as directory: with tempfile.TemporaryDirectory() as directory:
work = Path(directory) work = Path(directory)

View File

@ -19,6 +19,7 @@ EXPECTED_ENTRIES = [
"platform/packages/external-provider-contract/src/data-plane.mjs", "platform/packages/external-provider-contract/src/data-plane.mjs",
"platform/packages/external-provider-contract/src/data-product.mjs", "platform/packages/external-provider-contract/src/data-product.mjs",
"platform/packages/external-provider-contract/src/intake-batch.mjs", "platform/packages/external-provider-contract/src/intake-batch.mjs",
"platform/packages/external-provider-contract/src/index.mjs",
"platform/packages/external-provider-contract/src/sensitive-field-policy.mjs", "platform/packages/external-provider-contract/src/sensitive-field-policy.mjs",
] ]

View File

@ -17,7 +17,7 @@ SCRIPT_DIR = Path(__file__).resolve().parent
PLATFORM_ROOT = SCRIPT_DIR.parent.parent PLATFORM_ROOT = SCRIPT_DIR.parent.parent
RUNNER_PATH = SCRIPT_DIR / "nodedc-deploy" RUNNER_PATH = SCRIPT_DIR / "nodedc-deploy"
BUILDER_PATH = SCRIPT_DIR / "build-n8n-private-extension-artifact.mjs" BUILDER_PATH = SCRIPT_DIR / "build-n8n-private-extension-artifact.mjs"
PATCH_ID = "n8n-nodes-ndc-release-20260716-003" PATCH_ID = "n8n-nodes-ndc-history-read-20260717-004"
EXPECTED_NODES = [ EXPECTED_NODES = [
"dist/nodes/NdcDataProductPublish/NdcDataProductPublish.node.js", "dist/nodes/NdcDataProductPublish/NdcDataProductPublish.node.js",
"dist/nodes/NdcDataProductRead/NdcDataProductRead.node.js", "dist/nodes/NdcDataProductRead/NdcDataProductRead.node.js",
@ -106,7 +106,7 @@ class N8nPrivateExtensionPolicyTest(unittest.TestCase):
manifest, entries, _payload = RUNNER.load_artifact(self.artifacts[0], Path(directory)) manifest, entries, _payload = RUNNER.load_artifact(self.artifacts[0], Path(directory))
self.assertEqual(manifest["component"], "n8n-private-extension") self.assertEqual(manifest["component"], "n8n-private-extension")
self.assertEqual(len(entries), 1) self.assertEqual(len(entries), 1)
self.assertRegex(entries[0], r"^releases/n8n-nodes-ndc/0\.1\.2-[a-f0-9]{16}$") self.assertRegex(entries[0], r"^releases/n8n-nodes-ndc/0\.1\.3-[a-f0-9]{16}$")
with tarfile.open(self.artifacts[0], "r:gz") as archive: with tarfile.open(self.artifacts[0], "r:gz") as archive:
names = archive.getnames() names = archive.getnames()
@ -118,7 +118,7 @@ class N8nPrivateExtensionPolicyTest(unittest.TestCase):
with tarfile.open(fileobj=io.BytesIO(package), mode="r:gz") as archive: with tarfile.open(fileobj=io.BytesIO(package), mode="r:gz") as archive:
package_json_member = archive.getmember("package/package.json") package_json_member = archive.getmember("package/package.json")
package_json = json.loads(archive.extractfile(package_json_member).read()) package_json = json.loads(archive.extractfile(package_json_member).read())
self.assertEqual(package_json["version"], "0.1.2") self.assertEqual(package_json["version"], "0.1.3")
self.assertEqual(package_json["n8n"]["nodes"], EXPECTED_NODES) self.assertEqual(package_json["n8n"]["nodes"], EXPECTED_NODES)
self.assertEqual(package_json["n8n"]["credentials"], EXPECTED_CREDENTIALS) self.assertEqual(package_json["n8n"]["credentials"], EXPECTED_CREDENTIALS)
for node_path in EXPECTED_NODES: for node_path in EXPECTED_NODES:
@ -154,7 +154,7 @@ class N8nPrivateExtensionPolicyTest(unittest.TestCase):
"requiresPreActivationVerification": True, "requiresPreActivationVerification": True,
} }
self.assertEqual(release["schemaVersion"], "nodedc.n8n-private-extension-release/v2") self.assertEqual(release["schemaVersion"], "nodedc.n8n-private-extension-release/v2")
self.assertEqual(release["package"]["version"], "0.1.2") self.assertEqual(release["package"]["version"], "0.1.3")
self.assertEqual(release["activation"]["rollbackBaselinePolicy"], expected_policy) self.assertEqual(release["activation"]["rollbackBaselinePolicy"], expected_policy)
self.assertEqual(rollback["schemaVersion"], "nodedc.n8n-private-extension-rollback/v2") self.assertEqual(rollback["schemaVersion"], "nodedc.n8n-private-extension-rollback/v2")
self.assertEqual(rollback["baselinePolicy"], expected_policy) self.assertEqual(rollback["baselinePolicy"], expected_policy)

View File

@ -91,6 +91,8 @@ Provider-neutral runtime использует отдельные wire schemas:
- `nodedc.data-product.publish/v1` — unscoped publish request от - `nodedc.data-product.publish/v1` — unscoped publish request от
`NDC Data Product Publish`; содержит только batch identity и canonical facts; `NDC Data Product Publish`; содержит только batch identity и canonical facts;
- `nodedc.data-product.snapshot/v1` — согласованный current snapshot с cursor; - `nodedc.data-product.snapshot/v1` — согласованный current snapshot с cursor;
- `nodedc.data-product.history/v1` — bounded Timescale history window с
`from/to`, provider-neutral `sourceIds`, resolution и opaque keyset cursor;
- `nodedc.data-product.patch/v1` — committed upsert operations из durable - `nodedc.data-product.patch/v1` — committed upsert operations из durable
outbox с `previousCursor`/`cursor`. outbox с `previousCursor`/`cursor`.
@ -109,9 +111,11 @@ binding; shared internal bearer и caller-provided scope headers являютс
stream с усечённой базой. `nextPageCursor` зарезервирован для будущего отдельно stream с усечённой базой. `nextPageCursor` зарезервирован для будущего отдельно
версионируемого query contract и текущим bounded runtime не выдаётся. версионируемого query contract и текущим bounded runtime не выдаётся.
Для большей cardinality definition заранее раскладывается по стабильным Для большей current cardinality definition заранее раскладывается по стабильным
partition Data Products с независимыми snapshot/patch cursors либо использует partition Data Products с независимыми snapshot/patch cursors. History является
будущий query contract с единым snapshot barrier и continuation semantics. отдельным immutable-window query: cursor привязан digest-ом к exact
`from/to/resolution/sourceIds`, поэтому его нельзя переиспользовать с другим
запросом; offset pagination запрещена.
Offset/source-ID pagination поверх меняющегося current snapshot запрещена: Offset/source-ID pagination поверх меняющегося current snapshot запрещена:
между страницами она способна потерять или задублировать изменения относительно между страницами она способна потерять или задублировать изменения относительно
patch cursor. patch cursor.

View File

@ -1,9 +1,11 @@
export { EXTERNAL_PROVIDER_CONTRACT_VERSION } from "./contract-version.mjs"; export { EXTERNAL_PROVIDER_CONTRACT_VERSION } from "./contract-version.mjs";
export { validateIntakeBatch } from "./intake-batch.mjs"; export { validateIntakeBatch } from "./intake-batch.mjs";
export { export {
DATA_PRODUCT_HISTORY_SCHEMA_VERSION,
DATA_PRODUCT_PATCH_SCHEMA_VERSION, DATA_PRODUCT_PATCH_SCHEMA_VERSION,
DATA_PRODUCT_PUBLISH_SCHEMA_VERSION, DATA_PRODUCT_PUBLISH_SCHEMA_VERSION,
DATA_PRODUCT_SNAPSHOT_SCHEMA_VERSION, DATA_PRODUCT_SNAPSHOT_SCHEMA_VERSION,
validateDataProductHistory,
validateDataProductPatch, validateDataProductPatch,
validateDataProductPublish, validateDataProductPublish,
validateDataProductSnapshot, validateDataProductSnapshot,

View File

@ -1,6 +1,7 @@
export const DATA_PRODUCT_PUBLISH_SCHEMA_VERSION = "nodedc.data-product.publish/v1"; export const DATA_PRODUCT_PUBLISH_SCHEMA_VERSION = "nodedc.data-product.publish/v1";
export const DATA_PRODUCT_SNAPSHOT_SCHEMA_VERSION = "nodedc.data-product.snapshot/v1"; export const DATA_PRODUCT_SNAPSHOT_SCHEMA_VERSION = "nodedc.data-product.snapshot/v1";
export const DATA_PRODUCT_PATCH_SCHEMA_VERSION = "nodedc.data-product.patch/v1"; export const DATA_PRODUCT_PATCH_SCHEMA_VERSION = "nodedc.data-product.patch/v1";
export const DATA_PRODUCT_HISTORY_SCHEMA_VERSION = "nodedc.data-product.history/v1";
import { SECRET_LIKE_KEY, SECRET_LIKE_VALUE } from "./sensitive-field-policy.mjs"; import { SECRET_LIKE_KEY, SECRET_LIKE_VALUE } from "./sensitive-field-policy.mjs";
@ -71,6 +72,67 @@ export function validateDataProductSnapshot(value) {
return result(errors); return result(errors);
} }
export function validateDataProductHistory(value) {
const errors = envelopeErrors(value, DATA_PRODUCT_HISTORY_SCHEMA_VERSION, "history");
rejectUnknownKeys(value, new Set(["schemaVersion", "dataProduct", "generatedAt", "query", "facts", "nextCursor"]), "history", errors);
requiredIsoTimestamp(value?.generatedAt, "generatedAt", errors);
if (!isPlainObject(value?.query)) {
errors.push("query_must_be_object");
} else {
rejectUnknownKeys(value.query, new Set(["from", "to", "resolutionMs", "sourceIds", "order"]), "query", errors);
requiredIsoTimestamp(value.query.from, "query.from", errors);
requiredIsoTimestamp(value.query.to, "query.to", errors);
if (
!Number.isNaN(Date.parse(value.query.from))
&& !Number.isNaN(Date.parse(value.query.to))
&& Date.parse(value.query.from) >= Date.parse(value.query.to)
) errors.push("query.range_invalid");
if (!Number.isInteger(value.query.resolutionMs) || value.query.resolutionMs < 1000) {
errors.push("query.resolutionMs_invalid");
}
if (!Array.isArray(value.query.sourceIds)) {
errors.push("query.sourceIds_must_be_array");
} else {
value.query.sourceIds.forEach((sourceId, index) => requiredIdentifier(sourceId, `query.sourceIds[${index}]`, errors));
if (JSON.stringify(value.query.sourceIds) !== JSON.stringify([...new Set(value.query.sourceIds)].sort())) {
errors.push("query.sourceIds_must_be_unique_and_sorted");
}
}
if (value.query.order !== "asc") errors.push("query.order_must_be_asc");
}
if (!Array.isArray(value?.facts)) {
errors.push("facts_must_be_array");
} else {
let previousOrderKey = null;
value.facts.forEach((fact, index) => {
validateCanonicalFact(fact, `facts[${index}]`, errors, { allowBucketStart: true });
if (isPlainObject(fact)) {
requiredIsoTimestamp(fact.bucketStart, `facts[${index}].bucketStart`, errors);
const bucketTime = Date.parse(fact.bucketStart);
if (
!Number.isNaN(bucketTime)
&& isPlainObject(value?.query)
&& (
bucketTime < Date.parse(value.query.from)
|| bucketTime >= Date.parse(value.query.to)
)
) errors.push(`facts[${index}].bucketStart_outside_query_range`);
const orderKey = `${fact.bucketStart}\u0000${fact.sourceId}\u0000${fact.semanticType}`;
if (previousOrderKey !== null && orderKey <= previousOrderKey) {
errors.push(`facts[${index}]_not_strictly_ordered`);
}
previousOrderKey = orderKey;
}
});
}
if (value?.nextCursor !== undefined) {
requiredString(value.nextCursor, "nextCursor", errors);
if (!/^[A-Za-z0-9_-]{1,1024}$/.test(String(value.nextCursor || ""))) errors.push("nextCursor_invalid");
}
if (containsSecretLikeMaterial(value)) errors.push("history_must_not_contain_secret_material");
return result(errors);
}
export function validateDataProductPatch(value) { export function validateDataProductPatch(value) {
const errors = envelopeErrors(value, DATA_PRODUCT_PATCH_SCHEMA_VERSION, "patch"); const errors = envelopeErrors(value, DATA_PRODUCT_PATCH_SCHEMA_VERSION, "patch");
rejectUnknownKeys(value, new Set(["schemaVersion", "dataProduct", "cursor", "previousCursor", "emittedAt", "operations"]), "patch", errors); rejectUnknownKeys(value, new Set(["schemaVersion", "dataProduct", "cursor", "previousCursor", "emittedAt", "operations"]), "patch", errors);
@ -108,13 +170,14 @@ function envelopeErrors(value, schemaVersion, label) {
return errors; return errors;
} }
function validateFact(value, path, errors, { maxAttributesBytes, canonical = false }) { function validateFact(value, path, errors, { maxAttributesBytes, canonical = false, allowBucketStart = false }) {
if (!isPlainObject(value)) { if (!isPlainObject(value)) {
errors.push(`${path}_must_be_object`); errors.push(`${path}_must_be_object`);
return; return;
} }
const allowedKeys = new Set(["sourceId", "semanticType", "observedAt", "attributes", "geometry"]); const allowedKeys = new Set(["sourceId", "semanticType", "observedAt", "attributes", "geometry"]);
if (canonical) allowedKeys.add("receivedAt"); if (canonical) allowedKeys.add("receivedAt");
if (allowBucketStart) allowedKeys.add("bucketStart");
rejectUnknownKeys(value, allowedKeys, path, errors); rejectUnknownKeys(value, allowedKeys, path, errors);
requiredIdentifier(value.sourceId, `${path}.sourceId`, errors); requiredIdentifier(value.sourceId, `${path}.sourceId`, errors);
requiredIdentifier(value.semanticType, `${path}.semanticType`, errors); requiredIdentifier(value.semanticType, `${path}.semanticType`, errors);
@ -129,8 +192,8 @@ function validateFact(value, path, errors, { maxAttributesBytes, canonical = fal
if (value.geometry !== undefined) validatePointGeometry(value.geometry, `${path}.geometry`, errors); if (value.geometry !== undefined) validatePointGeometry(value.geometry, `${path}.geometry`, errors);
} }
function validateCanonicalFact(value, path, errors) { function validateCanonicalFact(value, path, errors, { allowBucketStart = false } = {}) {
validateFact(value, path, errors, { maxAttributesBytes: 64 * 1024, canonical: true }); validateFact(value, path, errors, { maxAttributesBytes: 64 * 1024, canonical: true, allowBucketStart });
if (!isPlainObject(value)) return; if (!isPlainObject(value)) return;
requiredIsoTimestamp(value.receivedAt, `${path}.receivedAt`, errors); requiredIsoTimestamp(value.receivedAt, `${path}.receivedAt`, errors);
} }

View File

@ -19,9 +19,11 @@ export {
} from "./provider-package.mjs"; } from "./provider-package.mjs";
export { export {
DATA_PRODUCT_HISTORY_SCHEMA_VERSION,
DATA_PRODUCT_PATCH_SCHEMA_VERSION, DATA_PRODUCT_PATCH_SCHEMA_VERSION,
DATA_PRODUCT_PUBLISH_SCHEMA_VERSION, DATA_PRODUCT_PUBLISH_SCHEMA_VERSION,
DATA_PRODUCT_SNAPSHOT_SCHEMA_VERSION, DATA_PRODUCT_SNAPSHOT_SCHEMA_VERSION,
validateDataProductHistory,
validateDataProductPatch, validateDataProductPatch,
validateDataProductPublish, validateDataProductPublish,
validateDataProductSnapshot, validateDataProductSnapshot,

View File

@ -1,8 +1,10 @@
import assert from "node:assert/strict"; import assert from "node:assert/strict";
import { import {
DATA_PRODUCT_HISTORY_SCHEMA_VERSION,
DATA_PRODUCT_PATCH_SCHEMA_VERSION, DATA_PRODUCT_PATCH_SCHEMA_VERSION,
DATA_PRODUCT_PUBLISH_SCHEMA_VERSION, DATA_PRODUCT_PUBLISH_SCHEMA_VERSION,
DATA_PRODUCT_SNAPSHOT_SCHEMA_VERSION, DATA_PRODUCT_SNAPSHOT_SCHEMA_VERSION,
validateDataProductHistory,
validateDataProductPatch, validateDataProductPatch,
validateDataProductPublish, validateDataProductPublish,
validateDataProductSnapshot, validateDataProductSnapshot,
@ -89,6 +91,45 @@ assert.equal(validateDataProductSnapshot({
...snapshot, ...snapshot,
facts: [{ ...canonicalFact, attributes: { status: "ndc_edprb_forbidden-reader-token" } }], facts: [{ ...canonicalFact, attributes: { status: "ndc_edprb_forbidden-reader-token" } }],
}).errors.includes("snapshot_must_not_contain_secret_material"), true); }).errors.includes("snapshot_must_not_contain_secret_material"), true);
const history = {
schemaVersion: DATA_PRODUCT_HISTORY_SCHEMA_VERSION,
dataProduct: { id: "fleet.positions.current.v1", version: "1.0.0" },
generatedAt: "2026-07-15T10:05:00.000Z",
query: {
from: "2026-07-15T10:00:00.000Z",
to: "2026-07-15T10:05:00.000Z",
resolutionMs: 60_000,
sourceIds: ["unit-42"],
order: "asc",
},
facts: [{ ...canonicalFact, bucketStart: "2026-07-15T10:00:00.000Z" }],
nextCursor: "opaque_cursor",
};
assert.equal(validateDataProductHistory(history).ok, true);
assert.equal(validateDataProductHistory({
...history,
query: { ...history.query, order: "desc" },
}).errors.includes("query.order_must_be_asc"), true);
assert.equal(validateDataProductHistory({
...history,
query: { ...history.query, from: history.query.to, to: history.query.from },
}).errors.includes("query.range_invalid"), true);
assert.equal(validateDataProductHistory({
...history,
query: { ...history.query, sourceIds: ["unit-42", "unit-42"] },
}).errors.includes("query.sourceIds_must_be_unique_and_sorted"), true);
assert.equal(validateDataProductHistory({
...history,
facts: [
{ ...history.facts[0], sourceId: "unit-43" },
history.facts[0],
],
}).errors.includes("facts[1]_not_strictly_ordered"), true);
assert.equal(validateDataProductHistory({
...history,
facts: [{ ...history.facts[0], attributes: { token: "forbidden" } }],
}).errors.includes("history_must_not_contain_secret_material"), true);
assert.equal(validateDataProductSnapshot({ assert.equal(validateDataProductSnapshot({
...snapshot, ...snapshot,
facts: [{ ...canonicalFact, attributes: { status: `ndc_edppr_${"P".repeat(43)}` } }], facts: [{ ...canonicalFact, attributes: { status: `ndc_edppr_${"P".repeat(43)}` } }],

View File

@ -9,8 +9,8 @@ qualified:
- `n8n-nodes-ndc.ndcDataProductPublish` — publishes canonical facts into an - `n8n-nodes-ndc.ndcDataProductPublish` — publishes canonical facts into an
approved Data Product. approved Data Product.
- `n8n-nodes-ndc.ndcDataProductRead` — reads the current snapshot of an - `n8n-nodes-ndc.ndcDataProductRead` — reads the current snapshot or a bounded,
approved Data Product. cursor-paged history window of an approved Data Product.
- `n8n-nodes-ndc.ndcFoundryBinding` — creates or updates a declarative Foundry - `n8n-nodes-ndc.ndcFoundryBinding` — creates or updates a declarative Foundry
page-slot binding. This is a control-plane operation, not a runtime data page-slot binding. This is a control-plane operation, not a runtime data
transport. transport.
@ -42,6 +42,7 @@ never accepts them from a workflow.
- `GET /internal/data-plane/v1/reader/data-products` - `GET /internal/data-plane/v1/reader/data-products`
- `POST /internal/data-plane/v1/data-products/:dataProductId/publish` - `POST /internal/data-plane/v1/data-products/:dataProductId/publish`
- `GET /internal/data-plane/v1/data-products/:dataProductId/snapshot` - `GET /internal/data-plane/v1/data-products/:dataProductId/snapshot`
- `GET /internal/data-plane/v1/data-products/:dataProductId/history`
- `GET /internal/foundry/v1/data-products` - `GET /internal/foundry/v1/data-products`
- `POST /internal/foundry/v1/data-product-bindings` - `POST /internal/foundry/v1/data-product-bindings`
@ -58,14 +59,20 @@ materializes actor, owner and exact application/page/binding/product scope from
that grant; none of those authorization claims are accepted from headers or that grant; none of those authorization claims are accepted from headers or
workflow data. workflow data.
`NDC Data Product Read` supports only the bounded `NDC Data Product Read` supports the bounded
`nodedc.data-product.snapshot/v1` contract. Its `Page Size` parameter is a `nodedc.data-product.snapshot/v1` contract. Its snapshot `Page Size` parameter is a
safety ceiling, not pagination: the complete scoped current projection must fit safety ceiling, not pagination: the complete scoped current projection must fit
within 5000 entity keys. A larger product returns within 5000 entity keys. A larger product returns
`data_product_snapshot_limit_exceeded`; the node must not assemble independent `data_product_snapshot_limit_exceeded`; the node must not assemble independent
pages or start a patch stream from an incomplete snapshot. Large products need pages or start a patch stream from an incomplete snapshot. Large products need
stable partition Data Products or a separately versioned query contract with a stable partition Data Products.
single snapshot barrier.
History mode uses `nodedc.data-product.history/v1` and requires a closed
`from`/`to` interval. It supports provider-neutral `sourceIds`, a resolution
that is a multiple of the Data Product native sampling interval, a limit up to
5000 points and an opaque keyset continuation cursor. The response stays as one
envelope item so L2 logic can preserve `query` and `nextCursor` while iterating.
The workflow never selects a database, provider, tenant or connection.
## Publish input ## Publish input

View File

@ -9,6 +9,7 @@ import { NodeConnectionTypes, NodeOperationError } from 'n8n-workflow';
import { import {
DATA_PRODUCT_BASE_PATH, DATA_PRODUCT_BASE_PATH,
DATA_PRODUCT_HISTORY_SUFFIX,
DATA_PRODUCT_READER_CATALOG_PATH, DATA_PRODUCT_READER_CATALOG_PATH,
DATA_PRODUCT_SNAPSHOT_SUFFIX, DATA_PRODUCT_SNAPSHOT_SUFFIX,
DATA_PLANE_DEFAULT_BASE_URL, DATA_PLANE_DEFAULT_BASE_URL,
@ -16,7 +17,12 @@ import {
NDC_DATA_PRODUCT_READER_CREDENTIAL, NDC_DATA_PRODUCT_READER_CREDENTIAL,
NDC_NODE_ICON, NDC_NODE_ICON,
} from '../shared/constants'; } from '../shared/constants';
import { normalizeSnapshotFacts, snapshotReadParameters } from '../shared/contracts'; import {
historyReadParameters,
normalizeHistoryResponse,
normalizeSnapshotFacts,
snapshotReadParameters,
} from '../shared/contracts';
import { safeHttpError, safeInputError } from '../shared/errors'; import { safeHttpError, safeInputError } from '../shared/errors';
import { loadDataProductOptions, ndcRequest, serviceBaseUrl } from '../shared/http'; import { loadDataProductOptions, ndcRequest, serviceBaseUrl } from '../shared/http';
@ -26,14 +32,25 @@ export class NdcDataProductRead implements INodeType {
name: 'ndcDataProductRead', name: 'ndcDataProductRead',
icon: NDC_NODE_ICON, icon: NDC_NODE_ICON,
group: ['input'], group: ['input'],
version: 1, version: [1, 2],
subtitle: '={{$parameter["dataProductId"]}}', subtitle: '={{$parameter["mode"] || "snapshot"}} · {{$parameter["dataProductId"]}}',
description: 'Read a scoped NDC Data Product snapshot', description: 'Read a scoped NDC Data Product snapshot or bounded history',
defaults: { name: 'NDC Data Product Read' }, defaults: { name: 'NDC Data Product Read' },
inputs: [NodeConnectionTypes.Main], inputs: [NodeConnectionTypes.Main],
outputs: [NodeConnectionTypes.Main], outputs: [NodeConnectionTypes.Main],
credentials: [{ name: NDC_DATA_PRODUCT_READER_CREDENTIAL, required: true }], credentials: [{ name: NDC_DATA_PRODUCT_READER_CREDENTIAL, required: true }],
properties: [ properties: [
{
displayName: 'Mode',
name: 'mode',
type: 'options',
options: [
{ name: 'Current Snapshot', value: 'snapshot' },
{ name: 'History', value: 'history' },
],
default: 'snapshot',
required: true,
},
{ {
displayName: 'Data Product Name or ID', displayName: 'Data Product Name or ID',
name: 'dataProductId', name: 'dataProductId',
@ -50,6 +67,59 @@ export class NdcDataProductRead implements INodeType {
default: 1000, default: 1000,
required: true, required: true,
description: 'Maximum number of snapshot facts to return', description: 'Maximum number of snapshot facts to return',
displayOptions: { show: { mode: ['snapshot'] } },
},
{
displayName: 'From',
name: 'from',
type: 'dateTime',
default: '',
required: true,
displayOptions: { show: { mode: ['history'] } },
},
{
displayName: 'To',
name: 'to',
type: 'dateTime',
default: '',
required: true,
displayOptions: { show: { mode: ['history'] } },
},
{
displayName: 'Resolution (MS)',
name: 'resolutionMs',
type: 'number',
typeOptions: { minValue: 1000, maxValue: 86400000, numberStepSize: 1000 },
default: 60000,
required: true,
description: 'Provider-neutral history bucket size; must be a multiple of the Data Product native history interval',
displayOptions: { show: { mode: ['history'] } },
},
{
displayName: 'Source IDs',
name: 'sourceIds',
type: 'string',
default: '',
description: 'Optional comma-separated entity IDs; empty reads all entities in the scoped Data Product',
displayOptions: { show: { mode: ['history'] } },
},
{
displayName: 'Limit',
name: 'limit',
type: 'number',
typeOptions: { minValue: 1, maxValue: 5000, numberStepSize: 1 },
default: 50,
required: true,
description: 'Max number of results to return',
displayOptions: { show: { mode: ['history'] } },
},
{
displayName: 'Cursor',
name: 'cursor',
type: 'string',
default: '',
description: 'Opaque continuation cursor returned by the previous history page',
displayOptions: { show: { mode: ['history'] } },
}, },
], ],
}; };
@ -69,7 +139,48 @@ export class NdcDataProductRead implements INodeType {
}; };
async execute(this: IExecuteFunctions): Promise<INodeExecutionData[][]> { async execute(this: IExecuteFunctions): Promise<INodeExecutionData[][]> {
const mode = this.getNodeParameter('mode', 0, 'snapshot') as string;
const dataProductId = this.getNodeParameter('dataProductId', 0) as string; const dataProductId = this.getNodeParameter('dataProductId', 0) as string;
if (mode === 'history') {
let parameters;
try {
parameters = historyReadParameters({
dataProductId,
from: this.getNodeParameter('from', 0),
to: this.getNodeParameter('to', 0),
resolutionMs: this.getNodeParameter('resolutionMs', 0, 60000),
sourceIds: this.getNodeParameter('sourceIds', 0, ''),
limit: this.getNodeParameter('limit', 0, 1000),
cursor: this.getNodeParameter('cursor', 0, ''),
});
} catch (error) {
throw new NodeOperationError(this.getNode(), safeInputError(error, 'ndc_data_product_history_input_invalid'));
}
try {
const baseUrl = serviceBaseUrl(DATA_PLANE_BASE_URL_ENV, DATA_PLANE_DEFAULT_BASE_URL);
const response = await ndcRequest(this, NDC_DATA_PRODUCT_READER_CREDENTIAL, {
method: 'GET',
url: `${baseUrl}${DATA_PRODUCT_BASE_PATH}/${parameters.encodedProductId}${DATA_PRODUCT_HISTORY_SUFFIX}`,
qs: {
from: parameters.from,
to: parameters.to,
resolutionMs: parameters.resolutionMs,
...(parameters.sourceIds.length ? { sourceIds: parameters.sourceIds.join(',') } : {}),
limit: parameters.limit,
...(parameters.cursor ? { cursor: parameters.cursor } : {}),
},
});
return [[{ json: normalizeHistoryResponse(response), pairedItem: { item: 0 } }]];
} catch (error) {
const safe = error instanceof Error && error.message === 'data_product_history_invalid'
? safeInputError(error, 'ndc_data_product_history_invalid')
: safeHttpError(error, 'ndc_data_product_history_read_failed');
throw new NodeOperationError(this.getNode(), safe);
}
}
if (mode !== 'snapshot') {
throw new NodeOperationError(this.getNode(), safeInputError(new Error('read_mode_invalid'), 'ndc_data_product_read_input_invalid'));
}
const pageSize = this.getNodeParameter('pageSize', 0, 1000) as number; const pageSize = this.getNodeParameter('pageSize', 0, 1000) as number;
let parameters; let parameters;
try { try {

View File

@ -5,6 +5,7 @@ export const DATA_PRODUCT_WRITER_CATALOG_PATH = '/internal/data-plane/v1/writer/
export const DATA_PRODUCT_READER_CATALOG_PATH = '/internal/data-plane/v1/reader/data-products'; export const DATA_PRODUCT_READER_CATALOG_PATH = '/internal/data-plane/v1/reader/data-products';
export const DATA_PRODUCT_BASE_PATH = '/internal/data-plane/v1/data-products'; export const DATA_PRODUCT_BASE_PATH = '/internal/data-plane/v1/data-products';
export const DATA_PRODUCT_SNAPSHOT_SUFFIX = '/snapshot'; export const DATA_PRODUCT_SNAPSHOT_SUFFIX = '/snapshot';
export const DATA_PRODUCT_HISTORY_SUFFIX = '/history';
export const FOUNDRY_BINDING_PATH = '/internal/foundry/v1/data-product-bindings'; export const FOUNDRY_BINDING_PATH = '/internal/foundry/v1/data-product-bindings';
export const FOUNDRY_CATALOG_PATH = '/internal/foundry/v1/data-products'; export const FOUNDRY_CATALOG_PATH = '/internal/foundry/v1/data-products';

View File

@ -118,6 +118,19 @@ export function normalizeSnapshotFacts(value: unknown): IDataObject[] {
}); });
} }
export function normalizeHistoryResponse(value: unknown): IDataObject {
if (!isObject(value) || value.schemaVersion !== 'nodedc.data-product.history/v1'
|| !isObject(value.dataProduct) || !isObject(value.query) || !Array.isArray(value.facts)) {
throw new Error('data_product_history_invalid');
}
value.facts.forEach((fact, index) => {
if (!isObject(fact) || Number.isNaN(Date.parse(String(fact.bucketStart ?? '')))) {
throw new Error(`history_fact_${index}_invalid`);
}
});
return value;
}
export function snapshotReadParameters( export function snapshotReadParameters(
dataProductId: unknown, dataProductId: unknown,
pageSize: unknown, pageSize: unknown,
@ -132,6 +145,52 @@ export function snapshotReadParameters(
}; };
} }
export function historyReadParameters(value: {
dataProductId: unknown;
from: unknown;
to: unknown;
resolutionMs: unknown;
sourceIds: unknown;
limit: unknown;
cursor: unknown;
}): {
encodedProductId: string;
from: string;
to: string;
resolutionMs: number;
sourceIds: string[];
limit: number;
cursor: string;
} {
const from = requireIsoTimestamp(value.from, 'from');
const to = requireIsoTimestamp(value.to, 'to');
if (Date.parse(from) >= Date.parse(to)) throw new Error('history_range_invalid');
const resolutionMs = Number(value.resolutionMs);
if (!Number.isInteger(resolutionMs) || resolutionMs < 1000 || resolutionMs > 24 * 60 * 60 * 1000) {
throw new Error('history_resolution_invalid');
}
const limit = Number(value.limit);
if (!Number.isInteger(limit) || limit < 1 || limit > 5000) throw new Error('history_limit_invalid');
const sourceIds = [...new Set((Array.isArray(value.sourceIds)
? value.sourceIds
: String(value.sourceIds ?? '').split(','))
.map((item) => String(item ?? '').trim())
.filter(Boolean)
.map((item) => requireIdentifier(item, 'sourceIds')))];
if (sourceIds.length > 1000) throw new Error('history_source_ids_limit_exceeded');
const cursor = String(value.cursor ?? '').trim();
if (cursor && (cursor.length > 1024 || !/^[A-Za-z0-9_-]+$/.test(cursor))) throw new Error('history_cursor_invalid');
return {
encodedProductId: encodeIdentifierPath(value.dataProductId, 'dataProductId'),
from,
to,
resolutionMs,
sourceIds,
limit,
cursor,
};
}
export function dataProductOptions(value: unknown): INodeListSearchItems[] { export function dataProductOptions(value: unknown): INodeListSearchItems[] {
if (!isObject(value)) throw new Error('data_product_catalog_invalid'); if (!isObject(value)) throw new Error('data_product_catalog_invalid');
const rows = Array.isArray(value.dataProducts) const rows = Array.isArray(value.dataProducts)

View File

@ -1,12 +1,12 @@
{ {
"name": "n8n-nodes-ndc", "name": "n8n-nodes-ndc",
"version": "0.1.2", "version": "0.1.3",
"lockfileVersion": 3, "lockfileVersion": 3,
"requires": true, "requires": true,
"packages": { "packages": {
"": { "": {
"name": "n8n-nodes-ndc", "name": "n8n-nodes-ndc",
"version": "0.1.2", "version": "0.1.3",
"license": "UNLICENSED", "license": "UNLICENSED",
"devDependencies": { "devDependencies": {
"@n8n/node-cli": "0.39.3", "@n8n/node-cli": "0.39.3",

View File

@ -1,6 +1,6 @@
{ {
"name": "n8n-nodes-ndc", "name": "n8n-nodes-ndc",
"version": "0.1.2", "version": "0.1.3",
"description": "Private NODE.DC nodes for scoped data products and Foundry bindings.", "description": "Private NODE.DC nodes for scoped data products and Foundry bindings.",
"private": true, "private": true,
"license": "UNLICENSED", "license": "UNLICENSED",

View File

@ -172,8 +172,9 @@ async function main() {
restoreEnvironment('NDC_DATA_PLANE_BASE_URL', previousDefaultProbe); restoreEnvironment('NDC_DATA_PLANE_BASE_URL', previousDefaultProbe);
const readProperties = nodes.get('NdcDataProductRead').description.properties.map((property) => property.name); const readProperties = nodes.get('NdcDataProductRead').description.properties.map((property) => property.name);
assert.equal(readProperties.includes('cursor'), false, 'snapshot endpoint has no cursor input'); assert.equal(readProperties.includes('mode'), true, 'read mode must expose snapshot/history explicitly');
assert.equal(readProperties.includes('history'), false, 'history must stay hidden until its endpoint exists'); assert.equal(readProperties.includes('cursor'), true, 'history endpoint must expose its opaque continuation cursor');
assert.equal(readProperties.includes('sourceIds'), true, 'history filtering remains provider-neutral');
const pageSizeProperty = nodes.get('NdcDataProductRead').description.properties.find((property) => property.name === 'pageSize'); const pageSizeProperty = nodes.get('NdcDataProductRead').description.properties.find((property) => property.name === 'pageSize');
assert.equal(pageSizeProperty.typeOptions.minValue, 1); assert.equal(pageSizeProperty.typeOptions.minValue, 1);
assert.equal(pageSizeProperty.typeOptions.maxValue, 5000, 'bounded snapshot v1 must not expose more than 5000 facts'); assert.equal(pageSizeProperty.typeOptions.maxValue, 5000, 'bounded snapshot v1 must not expose more than 5000 facts');
@ -255,6 +256,51 @@ async function assertNodeHttpContracts(nodes, externalContract, constants) {
assert.deepEqual(readCalls[0].options.qs, { limit: 1000 }); assert.deepEqual(readCalls[0].options.qs, { limit: 1000 });
assert.equal(readResult[0][0].json.sourceId, 'fleet.unit.42'); assert.equal(readResult[0][0].json.sourceId, 'fleet.unit.42');
const historyCalls = [];
const historyResponse = {
schemaVersion: 'nodedc.data-product.history/v1',
dataProduct: { id: 'fleet.positions.current.v1', version: '1.0.0' },
generatedAt: '2026-07-15T12:05:00.000Z',
query: {
from: '2026-07-15T12:00:00.000Z',
to: '2026-07-15T12:05:00.000Z',
resolutionMs: 60000,
sourceIds: ['fleet.unit.42'],
order: 'asc',
},
facts: [{ ...canonicalFact(), receivedAt: '2026-07-15T12:00:00.100Z', bucketStart: '2026-07-15T12:00:00.000Z' }],
nextCursor: 'opaque_cursor',
};
const historyContext = executionContext({
parameters: {
mode: 'history',
dataProductId: 'fleet.positions.current.v1',
from: '2026-07-15T12:00:00.000Z',
to: '2026-07-15T12:05:00.000Z',
resolutionMs: 60000,
sourceIds: 'fleet.unit.42',
limit: 1000,
cursor: '',
},
response: historyResponse,
calls: historyCalls,
});
const historyResult = await nodes.get('NdcDataProductRead').execute.call(historyContext);
assert.equal(historyCalls[0].credentialName, 'ndcDataProductReaderApi');
assert.equal(historyCalls[0].options.method, 'GET');
assert.equal(
historyCalls[0].options.url,
'http://data-plane.test/internal/data-plane/v1/data-products/fleet.positions.current.v1/history',
);
assert.deepEqual(historyCalls[0].options.qs, {
from: '2026-07-15T12:00:00.000Z',
to: '2026-07-15T12:05:00.000Z',
resolutionMs: 60000,
sourceIds: 'fleet.unit.42',
limit: 1000,
});
assert.deepEqual(historyResult[0][0].json, historyResponse);
const foundryCalls = []; const foundryCalls = [];
const foundryContext = executionContext({ const foundryContext = executionContext({
parameters: { parameters: {

View File

@ -1,9 +1,14 @@
import { createHash, randomUUID } from "node:crypto"; import { createHash, randomUUID } from "node:crypto";
import { import {
DATA_PRODUCT_HISTORY_SCHEMA_VERSION,
DATA_PRODUCT_PATCH_SCHEMA_VERSION, DATA_PRODUCT_PATCH_SCHEMA_VERSION,
DATA_PRODUCT_SNAPSHOT_SCHEMA_VERSION, DATA_PRODUCT_SNAPSHOT_SCHEMA_VERSION,
} from "@nodedc/external-provider-contract/data-plane"; } from "@nodedc/external-provider-contract/data-plane";
const IDENTIFIER = /^[a-z][a-z0-9._:-]{2,127}$/;
const MAX_HISTORY_RESOLUTION_MS = 24 * 60 * 60 * 1000;
const MAX_HISTORY_SOURCE_IDS = 1000;
export async function loadDataProductDefinition(db, dataProductId, { activeOnly = true } = {}) { export async function loadDataProductDefinition(db, dataProductId, { activeOnly = true } = {}) {
const result = await db.query( const result = await db.query(
`select id, version, ontology_revision as "ontologyRevision", `select id, version, ontology_revision as "ontologyRevision",
@ -216,6 +221,121 @@ export async function readDataProductSnapshot(pool, binding, definition, { limit
} }
} }
export function normalizeHistoryReadOptions(value, definition) {
const policy = definition?.historyPolicy || {};
if (policy.mode === "none") throw deliveryError("data_product_history_not_supported", 409);
const from = new Date(String(value?.from || ""));
const to = new Date(String(value?.to || ""));
if (Number.isNaN(from.getTime()) || Number.isNaN(to.getTime()) || from >= to) {
throw deliveryError("data_product_history_range_invalid", 400);
}
const retentionDays = Math.max(1, Number(policy.retentionDays) || 1);
if (to.getTime() - from.getTime() > retentionDays * 24 * 60 * 60 * 1000) {
throw deliveryError("data_product_history_range_exceeds_retention", 400);
}
const nativeResolutionMs = policy.mode === "sampled" ? Number(policy.intervalMs) : 1000;
const resolutionMs = value?.resolutionMs === undefined || value?.resolutionMs === ""
? nativeResolutionMs
: Number(value.resolutionMs);
if (
!Number.isInteger(resolutionMs)
|| resolutionMs < nativeResolutionMs
|| resolutionMs > MAX_HISTORY_RESOLUTION_MS
|| resolutionMs % nativeResolutionMs !== 0
) throw deliveryError("data_product_history_resolution_invalid", 400);
const sourceIds = [...new Set((Array.isArray(value?.sourceIds) ? value.sourceIds : [])
.map((item) => String(item || "").trim())
.filter(Boolean))];
if (sourceIds.length > MAX_HISTORY_SOURCE_IDS || sourceIds.some((item) => !IDENTIFIER.test(item))) {
throw deliveryError("data_product_history_source_ids_invalid", 400);
}
const limit = Number(value?.limit ?? 1000);
if (!Number.isInteger(limit) || limit < 1 || limit > 5000) {
throw deliveryError("data_product_history_limit_invalid", 400);
}
const query = {
from: from.toISOString(),
to: to.toISOString(),
resolutionMs,
sourceIds: sourceIds.sort(),
order: "asc",
};
const queryHash = createHash("sha256").update(JSON.stringify(query)).digest("hex");
const cursor = decodeHistoryCursor(value?.cursor, queryHash);
return Object.freeze({ ...query, limit, queryHash, cursor });
}
export async function readDataProductHistory(pool, binding, definition, options) {
const query = normalizeHistoryReadOptions(options, definition);
const cursor = query.cursor || {};
const rows = await pool.query(
`with sampled as (
select
to_timestamp(
floor(extract(epoch from bucket_start) * 1000 / $8::double precision)
* $8::double precision / 1000
) as query_bucket,
source_id, semantic_type, observed_at, received_at, attributes, geometry, fingerprint
from external_data_plane_history
where tenant_id = $1 and connection_id = $2 and provider_id = $3 and data_product_id = $4
and bucket_start >= $5 and bucket_start < $6
and (cardinality($7::text[]) = 0 or source_id = any($7::text[]))
), ranked as (
select *, row_number() over (
partition by query_bucket, source_id, semantic_type
order by observed_at desc, received_at desc, fingerprint desc
) as sample_rank
from sampled
)
select query_bucket as "bucketStart", source_id as "sourceId", semantic_type as "semanticType",
observed_at as "observedAt", received_at as "receivedAt", attributes,
case when geometry is null then null
else jsonb_build_object('type', 'Point', 'coordinates', jsonb_build_array(
ST_X(geometry::geometry), ST_Y(geometry::geometry)
)) end as geometry
from ranked
where sample_rank = 1
and (
$9::boolean = false
or (query_bucket, source_id, semantic_type) > ($10::timestamptz, $11::text, $12::text)
)
order by query_bucket asc, source_id asc, semantic_type asc
limit $13`,
[
binding.tenantId, binding.connectionId, binding.providerId, definition.id,
query.from, query.to, query.sourceIds, query.resolutionMs,
Boolean(query.cursor), cursor.bucketStart || query.from, cursor.sourceId || "", cursor.semanticType || "",
query.limit + 1,
],
);
const page = rows.rows.slice(0, query.limit);
const last = page.at(-1);
return {
schemaVersion: DATA_PRODUCT_HISTORY_SCHEMA_VERSION,
dataProduct: { id: definition.id, version: definition.version },
generatedAt: new Date().toISOString(),
query: {
from: query.from,
to: query.to,
resolutionMs: query.resolutionMs,
sourceIds: query.sourceIds,
order: query.order,
},
facts: page.map((fact) => ({
...publicFact(fact),
bucketStart: new Date(fact.bucketStart).toISOString(),
})),
...(rows.rowCount > query.limit && last ? {
nextCursor: encodeHistoryCursor({
queryHash: query.queryHash,
bucketStart: new Date(last.bucketStart).toISOString(),
sourceId: last.sourceId,
semanticType: last.semanticType,
}),
} : {}),
};
}
export async function readPatchEvents(pool, binding, definition, after, { limit = 100 } = {}) { export async function readPatchEvents(pool, binding, definition, after, { limit = 100 } = {}) {
const client = await pool.connect(); const client = await pool.connect();
try { try {
@ -450,6 +570,36 @@ function publicFact(fact) {
return value; return value;
} }
function encodeHistoryCursor(value) {
return Buffer.from(JSON.stringify(value), "utf8").toString("base64url");
}
function decodeHistoryCursor(value, queryHash) {
const raw = String(value || "").trim();
if (!raw) return null;
if (raw.length > 1024 || !/^[A-Za-z0-9_-]+$/.test(raw)) {
throw deliveryError("data_product_history_cursor_invalid", 400);
}
let cursor;
try {
cursor = JSON.parse(Buffer.from(raw, "base64url").toString("utf8"));
} catch {
throw deliveryError("data_product_history_cursor_invalid", 400);
}
if (
!cursor || typeof cursor !== "object" || Array.isArray(cursor)
|| cursor.queryHash !== queryHash
|| Number.isNaN(Date.parse(cursor.bucketStart))
|| !IDENTIFIER.test(String(cursor.sourceId || ""))
|| !IDENTIFIER.test(String(cursor.semanticType || ""))
) throw deliveryError("data_product_history_cursor_invalid", 400);
return Object.freeze({
bucketStart: new Date(cursor.bucketStart).toISOString(),
sourceId: cursor.sourceId,
semanticType: cursor.semanticType,
});
}
function chunkOperations(values, maxOperations, maxBytes) { function chunkOperations(values, maxOperations, maxBytes) {
const result = []; const result = [];
let current = []; let current = [];

View File

@ -4,7 +4,15 @@ const IDENTIFIER = /^[a-z][a-z0-9._:-]{2,127}$/;
const TOKEN_PREFIX = "ndc_edprb_"; const TOKEN_PREFIX = "ndc_edprb_";
const SECRET_LIKE_KEY = /(token|secret|password|authorization|access[_-]?token|refresh[_-]?token|api[_-]?key)/i; const SECRET_LIKE_KEY = /(token|secret|password|authorization|access[_-]?token|refresh[_-]?token|api[_-]?key)/i;
const REQUEST_KEYS = new Set(["source", "allowedDataProductIds", "expiresAt"]); const REQUEST_KEYS = new Set(["source", "allowedDataProductIds", "expiresAt"]);
const MANAGED_REQUEST_KEYS = new Set([
"source",
"allowedDataProductIds",
"expiresAt",
"generation",
"capabilityDigest",
]);
const SOURCE_KEYS = new Set(["tenantId", "connectionId", "providerId"]); const SOURCE_KEYS = new Set(["tenantId", "connectionId", "providerId"]);
const SHA256_DIGEST = /^[a-f0-9]{64}$/;
export function createReaderToken() { export function createReaderToken() {
return `${TOKEN_PREFIX}${randomBytes(32).toString("base64url")}`; return `${TOKEN_PREFIX}${randomBytes(32).toString("base64url")}`;
@ -35,8 +43,61 @@ export function normalizeReaderBindingRequest(value, { now = new Date(), maxTtlD
return Object.freeze({ tenantId, connectionId, providerId, allowedDataProductIds, expiresAt: expiresAt.toISOString() }); return Object.freeze({ tenantId, connectionId, providerId, allowedDataProductIds, expiresAt: expiresAt.toISOString() });
} }
export function normalizeManagedReaderBindingRequest(value) {
if (!isPlainObject(value) || !isPlainObject(value.source)) {
throw readerError("managed_reader_binding_request_invalid");
}
if (containsSecretLikeKey(value)) {
throw readerError("managed_reader_binding_secret_material_forbidden");
}
if (!hasOnlyKeys(value, MANAGED_REQUEST_KEYS) || !hasOnlyKeys(value.source, SOURCE_KEYS)) {
throw readerError("managed_reader_binding_request_fields_invalid");
}
const tenantId = identifier(value.source.tenantId);
const connectionId = identifier(value.source.connectionId);
const providerId = identifier(value.source.providerId);
const allowedDataProductIds = uniqueIdentifiers(value.allowedDataProductIds);
if (!tenantId || !connectionId || !providerId || !allowedDataProductIds.length) {
throw readerError("managed_reader_binding_scope_invalid");
}
if (value.expiresAt !== null) {
throw readerError("managed_reader_binding_must_be_durable");
}
const generation = Number(value.generation);
if (!Number.isSafeInteger(generation) || generation < 1 || generation > 2_147_483_647) {
throw readerError("managed_reader_binding_generation_invalid");
}
const capabilityDigest = String(value.capabilityDigest || "").toLowerCase();
if (!SHA256_DIGEST.test(capabilityDigest)) {
throw readerError("managed_reader_binding_capability_digest_invalid");
}
return Object.freeze({
tenantId,
connectionId,
providerId,
allowedDataProductIds: Object.freeze([...allowedDataProductIds].sort()),
expiresAt: null,
generation,
capabilityDigest,
});
}
export function readerBindingRequestHash(policy) {
const canonical = JSON.stringify({
tenantId: policy.tenantId,
connectionId: policy.connectionId,
providerId: policy.providerId,
allowedDataProductIds: [...policy.allowedDataProductIds].sort(),
expiresAt: null,
generation: policy.generation,
capabilityDigest: policy.capabilityDigest,
});
return createHash("sha256").update(canonical, "utf8").digest("hex");
}
export function assertReaderProduct(binding, dataProductId, now = new Date()) { export function assertReaderProduct(binding, dataProductId, now = new Date()) {
if (!isPlainObject(binding) || binding.active !== true || new Date(binding.expiresAt) <= now) { const expired = binding?.expiresAt !== null && new Date(binding?.expiresAt) <= now;
if (!isPlainObject(binding) || binding.active !== true || expired) {
throw readerError("reader_binding_inactive", 401); throw readerError("reader_binding_inactive", 401);
} }
const normalized = identifier(dataProductId); const normalized = identifier(dataProductId);
@ -49,12 +110,14 @@ export function assertReaderProduct(binding, dataProductId, now = new Date()) {
export function safeReaderBinding(binding) { export function safeReaderBinding(binding) {
return { return {
id: binding.id, id: binding.id,
...(binding.bindingKey ? { bindingKey: binding.bindingKey } : {}),
...(Number.isInteger(Number(binding.generation)) ? { generation: Number(binding.generation) } : {}),
tenantId: binding.tenantId, tenantId: binding.tenantId,
connectionId: binding.connectionId, connectionId: binding.connectionId,
providerId: binding.providerId, providerId: binding.providerId,
allowedDataProductIds: uniqueIdentifiers(binding.allowedDataProductIds), allowedDataProductIds: uniqueIdentifiers(binding.allowedDataProductIds),
active: binding.active === true, active: binding.active === true,
expiresAt: new Date(binding.expiresAt).toISOString(), expiresAt: binding.expiresAt === null ? null : new Date(binding.expiresAt).toISOString(),
createdAt: binding.createdAt ? new Date(binding.createdAt).toISOString() : undefined, createdAt: binding.createdAt ? new Date(binding.createdAt).toISOString() : undefined,
rotatedAt: binding.rotatedAt ? new Date(binding.rotatedAt).toISOString() : undefined, rotatedAt: binding.rotatedAt ? new Date(binding.rotatedAt).toISOString() : undefined,
revokedAt: binding.revokedAt ? new Date(binding.revokedAt).toISOString() : undefined, revokedAt: binding.revokedAt ? new Date(binding.revokedAt).toISOString() : undefined,

View File

@ -55,7 +55,7 @@ export async function migrate(pool) {
payload_ref text, payload_ref text,
payload_bytes integer, payload_bytes integer,
received_at timestamptz not null, received_at timestamptz not null,
expires_at timestamptz not null, expires_at timestamptz,
check (payload is not null or payload_ref is not null) check (payload is not null or payload_ref is not null)
) )
`); `);
@ -72,7 +72,7 @@ export async function migrate(pool) {
connection_id text not null, connection_id text not null,
provider_id text not null, provider_id text not null,
allowed_data_product_ids jsonb not null, allowed_data_product_ids jsonb not null,
expires_at timestamptz not null, expires_at timestamptz,
active boolean not null default true, active boolean not null default true,
created_at timestamptz not null default now(), created_at timestamptz not null default now(),
rotated_at timestamptz, rotated_at timestamptz,
@ -89,6 +89,7 @@ export async function migrate(pool) {
await pool.query("alter table external_data_plane_writer_bindings add column if not exists binding_key text"); await pool.query("alter table external_data_plane_writer_bindings add column if not exists binding_key text");
await pool.query("alter table external_data_plane_writer_bindings add column if not exists request_hash text"); await pool.query("alter table external_data_plane_writer_bindings add column if not exists request_hash text");
await pool.query("alter table external_data_plane_writer_bindings add column if not exists generation integer not null default 1"); await pool.query("alter table external_data_plane_writer_bindings add column if not exists generation integer not null default 1");
await pool.query("alter table external_data_plane_writer_bindings alter column expires_at drop not null");
await pool.query(` await pool.query(`
do $$ do $$
begin begin
@ -113,6 +114,18 @@ export async function migrate(pool) {
or (binding_key is not null and request_hash ~ '^[a-f0-9]{64}$') or (binding_key is not null and request_hash ~ '^[a-f0-9]{64}$')
); );
end if; end if;
if not exists (
select 1 from pg_constraint
where conrelid = 'external_data_plane_writer_bindings'::regclass
and conname = 'external_data_plane_writer_bindings_lifecycle_ck'
) then
alter table external_data_plane_writer_bindings
add constraint external_data_plane_writer_bindings_lifecycle_ck
check (
(binding_key is null and expires_at is not null)
or (binding_key is not null and expires_at is null)
) not valid;
end if;
end end
$$ $$
`); `);
@ -123,15 +136,19 @@ export async function migrate(pool) {
create table if not exists external_data_plane_reader_bindings ( create table if not exists external_data_plane_reader_bindings (
id uuid primary key, id uuid primary key,
token_hash text not null unique, token_hash text not null unique,
binding_key text,
request_hash text,
generation integer not null default 1,
tenant_id text not null, tenant_id text not null,
connection_id text not null, connection_id text not null,
provider_id text not null, provider_id text not null,
allowed_data_product_ids jsonb not null, allowed_data_product_ids jsonb not null,
expires_at timestamptz not null, expires_at timestamptz,
active boolean not null default true, active boolean not null default true,
created_at timestamptz not null default now(), created_at timestamptz not null default now(),
rotated_at timestamptz, rotated_at timestamptz,
revoked_at timestamptz, revoked_at timestamptz,
check (generation > 0),
check ( check (
case when jsonb_typeof(allowed_data_product_ids) = 'array' case when jsonb_typeof(allowed_data_product_ids) = 'array'
then jsonb_array_length(allowed_data_product_ids) > 0 then jsonb_array_length(allowed_data_product_ids) > 0
@ -140,6 +157,50 @@ export async function migrate(pool) {
) )
) )
`); `);
await pool.query("alter table external_data_plane_reader_bindings add column if not exists binding_key text");
await pool.query("alter table external_data_plane_reader_bindings add column if not exists request_hash text");
await pool.query("alter table external_data_plane_reader_bindings add column if not exists generation integer not null default 1");
await pool.query("alter table external_data_plane_reader_bindings alter column expires_at drop not null");
await pool.query(`
do $$
begin
if not exists (
select 1 from pg_constraint
where conrelid = 'external_data_plane_reader_bindings'::regclass
and conname = 'external_data_plane_reader_bindings_generation_positive_ck'
) then
alter table external_data_plane_reader_bindings
add constraint external_data_plane_reader_bindings_generation_positive_ck
check (generation > 0);
end if;
if not exists (
select 1 from pg_constraint
where conrelid = 'external_data_plane_reader_bindings'::regclass
and conname = 'external_data_plane_reader_bindings_managed_metadata_ck'
) then
alter table external_data_plane_reader_bindings
add constraint external_data_plane_reader_bindings_managed_metadata_ck
check (
(binding_key is null and request_hash is null)
or (binding_key is not null and request_hash ~ '^[a-f0-9]{64}$')
);
end if;
if not exists (
select 1 from pg_constraint
where conrelid = 'external_data_plane_reader_bindings'::regclass
and conname = 'external_data_plane_reader_bindings_lifecycle_ck'
) then
alter table external_data_plane_reader_bindings
add constraint external_data_plane_reader_bindings_lifecycle_ck
check (
(binding_key is null and expires_at is not null)
or (binding_key is not null and expires_at is null)
);
end if;
end
$$
`);
await pool.query("create unique index if not exists external_data_plane_reader_bindings_managed_key_idx on external_data_plane_reader_bindings (binding_key, generation) where binding_key is not null");
await pool.query("create index if not exists external_data_plane_reader_bindings_active_idx on external_data_plane_reader_bindings (active, expires_at)"); await pool.query("create index if not exists external_data_plane_reader_bindings_active_idx on external_data_plane_reader_bindings (active, expires_at)");
await pool.query(` await pool.query(`

View File

@ -12,6 +12,7 @@ import {
pruneBatchReceipts, pruneBatchReceipts,
pruneDataProductHistory, pruneDataProductHistory,
prunePatchOutbox, prunePatchOutbox,
readDataProductHistory,
readDataProductSnapshot, readDataProductSnapshot,
readPatchEvents, readPatchEvents,
} from "./data-product-delivery.mjs"; } from "./data-product-delivery.mjs";
@ -26,12 +27,15 @@ import {
assertReaderProduct, assertReaderProduct,
createReaderToken, createReaderToken,
hashReaderToken, hashReaderToken,
normalizeManagedReaderBindingRequest,
normalizeReaderBindingRequest, normalizeReaderBindingRequest,
readerBindingRequestHash,
safeReaderBinding, safeReaderBinding,
} from "./reader-binding.mjs"; } from "./reader-binding.mjs";
import { migrate } from "./schema.mjs"; import { migrate } from "./schema.mjs";
import { import {
createWriterToken, createWriterToken,
canMigrateManagedWriterBindingToDurable,
hashWriterToken, hashWriterToken,
materializeDataProductPublish, materializeDataProductPublish,
materializeWriterBoundBatch, materializeWriterBoundBatch,
@ -90,12 +94,16 @@ app.get("/healthz", asyncRoute(async (_req, res) => {
providerLogic: "absent", providerLogic: "absent",
commandTransport: "absent", commandTransport: "absent",
writerBindings: "supported", writerBindings: "supported",
managedWriterBindings: "digest+idempotent-generation", managedWriterBindings: "digest+idempotent-generation+explicit-revoke",
readerBindings: "supported", readerBindings: "supported",
dataProductDelivery: "snapshot+durable-patch", managedReaderBindings: "digest+idempotent-generation+explicit-revoke",
dataProductDelivery: "snapshot+history+durable-patch",
legacyIntake: config.legacyIntakeEnabled ? "migration-only" : "disabled", legacyIntake: config.legacyIntakeEnabled ? "migration-only" : "disabled",
legacyCapabilityProvisioning: config.provisionerApiEnabled ? "enabled" : "disabled", legacyCapabilityProvisioning: config.provisionerApiEnabled ? "enabled" : "disabled",
managedWriterBindingProvisioning: config.managedProvisionerApiEnabled ? "enabled" : "disabled", managedWriterBindingProvisioning: config.managedProvisionerApiEnabled ? "enabled" : "disabled",
managedWriterBindingLifetime: config.managedProvisionerApiEnabled ? "explicit-revoke" : "disabled",
managedReaderBindingProvisioning: config.managedProvisionerApiEnabled ? "enabled" : "disabled",
managedReaderBindingLifetime: config.managedProvisionerApiEnabled ? "explicit-revoke" : "disabled",
rawRetentionSweep: { rawRetentionSweep: {
mode: "server-scheduled", mode: "server-scheduled",
lastSweepAt: lastRetentionSweepAt, lastSweepAt: lastRetentionSweepAt,
@ -169,9 +177,7 @@ app.post("/internal/data-plane/v1/intake/writer-bound", requireLegacyIntake, req
app.put("/internal/data-plane/v1/writer-bindings/by-key/:bindingKey", requireManagedProvisionerApi, asyncRoute(async (req, res) => { app.put("/internal/data-plane/v1/writer-bindings/by-key/:bindingKey", requireManagedProvisionerApi, asyncRoute(async (req, res) => {
const bindingKey = requireIdentifier(req.params.bindingKey, "managed_writer_binding_key_invalid"); const bindingKey = requireIdentifier(req.params.bindingKey, "managed_writer_binding_key_invalid");
const policy = normalizeManagedWriterBindingRequest(req.body, { const policy = normalizeManagedWriterBindingRequest(req.body);
maxTtlDays: config.writerBindingMaxTtlDays,
});
await assertRegisteredProductIds(policy.allowedDataProductIds); await assertRegisteredProductIds(policy.allowedDataProductIds);
const requestHash = writerBindingRequestHash(policy); const requestHash = writerBindingRequestHash(policy);
const client = await pool.connect(); const client = await pool.connect();
@ -180,7 +186,7 @@ app.put("/internal/data-plane/v1/writer-bindings/by-key/:bindingKey", requireMan
await client.query("begin"); await client.query("begin");
await client.query("select pg_advisory_xact_lock(hashtextextended($1, 0))", [bindingKey]); await client.query("select pg_advisory_xact_lock(hashtextextended($1, 0))", [bindingKey]);
const existing = await client.query( const existing = await client.query(
`select id, binding_key as "bindingKey", request_hash as "requestHash", generation, `select id, token_hash as "capabilityDigest", binding_key as "bindingKey", request_hash as "requestHash", generation,
tenant_id as "tenantId", connection_id as "connectionId", provider_id as "providerId", tenant_id as "tenantId", connection_id as "connectionId", provider_id as "providerId",
allowed_data_product_ids as "allowedDataProductIds", expires_at as "expiresAt", allowed_data_product_ids as "allowedDataProductIds", expires_at as "expiresAt",
active, created_at as "createdAt", rotated_at as "rotatedAt", revoked_at as "revokedAt" active, created_at as "createdAt", rotated_at as "rotatedAt", revoked_at as "revokedAt"
@ -192,12 +198,27 @@ app.put("/internal/data-plane/v1/writer-bindings/by-key/:bindingKey", requireMan
if (existing.rowCount) { if (existing.rowCount) {
const binding = existing.rows[0]; const binding = existing.rows[0];
if (binding.requestHash !== requestHash) { if (binding.requestHash !== requestHash) {
throw httpError(409, "managed_writer_binding_request_conflict"); const legacyCanMigrate = canMigrateManagedWriterBindingToDurable(binding, policy);
} if (!legacyCanMigrate) throw httpError(409, "managed_writer_binding_request_conflict");
if (binding.active !== true || new Date(binding.expiresAt) <= new Date()) { const migrated = await client.query(
`update external_data_plane_writer_bindings
set request_hash = $3, expires_at = null, rotated_at = now()
where binding_key = $1 and generation = $2 and token_hash = $4
and active = true and expires_at > now()
returning id, binding_key as "bindingKey", generation,
tenant_id as "tenantId", connection_id as "connectionId", provider_id as "providerId",
allowed_data_product_ids as "allowedDataProductIds", expires_at as "expiresAt",
active, created_at as "createdAt", rotated_at as "rotatedAt", revoked_at as "revokedAt"`,
[bindingKey, policy.generation, requestHash, policy.capabilityDigest],
);
if (!migrated.rowCount) throw httpError(409, "managed_writer_binding_generation_inactive");
response = { status: 200, idempotent: false, migrated: true, binding: migrated.rows[0] };
} else {
if (binding.active !== true || binding.expiresAt !== null) {
throw httpError(409, "managed_writer_binding_generation_inactive"); throw httpError(409, "managed_writer_binding_generation_inactive");
} }
response = { status: 200, idempotent: true, binding }; response = { status: 200, idempotent: true, migrated: false, binding };
}
} else { } else {
const inserted = await client.query( const inserted = await client.query(
`insert into external_data_plane_writer_bindings ( `insert into external_data_plane_writer_bindings (
@ -237,6 +258,7 @@ app.put("/internal/data-plane/v1/writer-bindings/by-key/:bindingKey", requireMan
res.status(response.status).json({ res.status(response.status).json({
ok: true, ok: true,
idempotent: response.idempotent, idempotent: response.idempotent,
migrated: response.migrated === true,
writerBinding: safeWriterBinding(response.binding), writerBinding: safeWriterBinding(response.binding),
}); });
})); }));
@ -354,6 +376,126 @@ app.post("/internal/data-plane/v1/writer-bindings/:bindingId/revoke", requirePro
res.json({ ok: true, writerBinding: safeWriterBinding(result.rows[0]) }); res.json({ ok: true, writerBinding: safeWriterBinding(result.rows[0]) });
})); }));
app.put("/internal/data-plane/v1/reader-bindings/by-key/:bindingKey", requireManagedProvisionerApi, asyncRoute(async (req, res) => {
const bindingKey = requireIdentifier(req.params.bindingKey, "managed_reader_binding_key_invalid");
const policy = normalizeManagedReaderBindingRequest(req.body);
await assertRegisteredProductIds(policy.allowedDataProductIds);
const requestHash = readerBindingRequestHash(policy);
const client = await pool.connect();
let response;
try {
await client.query("begin");
await client.query("select pg_advisory_xact_lock(hashtextextended($1, 0))", [bindingKey]);
const existing = await client.query(
`select id, binding_key as "bindingKey", request_hash as "requestHash", generation,
tenant_id as "tenantId", connection_id as "connectionId", provider_id as "providerId",
allowed_data_product_ids as "allowedDataProductIds", expires_at as "expiresAt",
active, created_at as "createdAt", rotated_at as "rotatedAt", revoked_at as "revokedAt"
from external_data_plane_reader_bindings
where binding_key = $1 and generation = $2
for update`,
[bindingKey, policy.generation],
);
if (existing.rowCount) {
const binding = existing.rows[0];
if (binding.requestHash !== requestHash) {
throw httpError(409, "managed_reader_binding_request_conflict");
}
if (binding.active !== true || binding.expiresAt !== null) {
throw httpError(409, "managed_reader_binding_generation_inactive");
}
response = { status: 200, idempotent: true, binding };
} else {
const inserted = await client.query(
`insert into external_data_plane_reader_bindings (
id, token_hash, binding_key, request_hash, generation,
tenant_id, connection_id, provider_id, allowed_data_product_ids, expires_at
) values ($1, $2, $3, $4, $5, $6, $7, $8, $9::jsonb, $10)
returning id, binding_key as "bindingKey", generation,
tenant_id as "tenantId", connection_id as "connectionId", provider_id as "providerId",
allowed_data_product_ids as "allowedDataProductIds", expires_at as "expiresAt",
active, created_at as "createdAt", rotated_at as "rotatedAt", revoked_at as "revokedAt"`,
[
randomUUID(),
policy.capabilityDigest,
bindingKey,
requestHash,
policy.generation,
policy.tenantId,
policy.connectionId,
policy.providerId,
JSON.stringify(policy.allowedDataProductIds),
policy.expiresAt,
],
);
response = { status: 201, idempotent: false, binding: inserted.rows[0] };
}
await client.query("commit");
} catch (error) {
await client.query("rollback");
if (error?.code === "23505") {
throw httpError(409, "managed_reader_binding_capability_digest_conflict");
}
throw error;
} finally {
client.release();
}
res.set("Cache-Control", "no-store, max-age=0");
res.status(response.status).json({
ok: true,
idempotent: response.idempotent,
readerBinding: safeReaderBinding(response.binding),
});
}));
app.post("/internal/data-plane/v1/reader-bindings/by-key/:bindingKey/generations/:generation/revoke", requireManagedProvisionerApi, asyncRoute(async (req, res) => {
const bindingKey = requireIdentifier(req.params.bindingKey, "managed_reader_binding_key_invalid");
const generation = requirePositiveInteger(req.params.generation, "managed_reader_binding_generation_invalid");
const client = await pool.connect();
let binding;
let idempotent;
try {
await client.query("begin");
await client.query("select pg_advisory_xact_lock(hashtextextended($1, 0))", [bindingKey]);
const existing = await client.query(
`select id, binding_key as "bindingKey", generation,
tenant_id as "tenantId", connection_id as "connectionId", provider_id as "providerId",
allowed_data_product_ids as "allowedDataProductIds", expires_at as "expiresAt",
active, created_at as "createdAt", rotated_at as "rotatedAt", revoked_at as "revokedAt"
from external_data_plane_reader_bindings
where binding_key = $1 and generation = $2
for update`,
[bindingKey, generation],
);
if (!existing.rowCount) throw httpError(404, "managed_reader_binding_not_found");
if (existing.rows[0].active === true) {
const revoked = await client.query(
`update external_data_plane_reader_bindings
set active = false, revoked_at = now()
where binding_key = $1 and generation = $2 and active = true
returning id, binding_key as "bindingKey", generation,
tenant_id as "tenantId", connection_id as "connectionId", provider_id as "providerId",
allowed_data_product_ids as "allowedDataProductIds", expires_at as "expiresAt",
active, created_at as "createdAt", rotated_at as "rotatedAt", revoked_at as "revokedAt"`,
[bindingKey, generation],
);
binding = revoked.rows[0];
idempotent = false;
} else {
binding = existing.rows[0];
idempotent = true;
}
await client.query("commit");
} catch (error) {
await client.query("rollback");
throw error;
} finally {
client.release();
}
res.set("Cache-Control", "no-store, max-age=0");
res.json({ ok: true, idempotent, readerBinding: safeReaderBinding(binding) });
}));
app.post("/internal/data-plane/v1/reader-bindings", requireProvisionerApi, asyncRoute(async (req, res) => { app.post("/internal/data-plane/v1/reader-bindings", requireProvisionerApi, asyncRoute(async (req, res) => {
const policy = normalizeReaderBindingRequest(req.body, { const policy = normalizeReaderBindingRequest(req.body, {
maxTtlDays: config.writerBindingMaxTtlDays, maxTtlDays: config.writerBindingMaxTtlDays,
@ -389,7 +531,7 @@ app.post("/internal/data-plane/v1/reader-bindings/:bindingId/rotate", requirePro
const result = await pool.query( const result = await pool.query(
`update external_data_plane_reader_bindings `update external_data_plane_reader_bindings
set token_hash = $2, rotated_at = now() set token_hash = $2, rotated_at = now()
where id = $1 and active = true and expires_at > now() where id = $1 and binding_key is null and active = true and expires_at > now()
returning id, tenant_id as "tenantId", connection_id as "connectionId", returning id, tenant_id as "tenantId", connection_id as "connectionId",
provider_id as "providerId", allowed_data_product_ids as "allowedDataProductIds", provider_id as "providerId", allowed_data_product_ids as "allowedDataProductIds",
expires_at as "expiresAt", active, created_at as "createdAt", expires_at as "expiresAt", active, created_at as "createdAt",
@ -405,7 +547,7 @@ app.post("/internal/data-plane/v1/reader-bindings/:bindingId/revoke", requirePro
const result = await pool.query( const result = await pool.query(
`update external_data_plane_reader_bindings `update external_data_plane_reader_bindings
set active = false, revoked_at = now() set active = false, revoked_at = now()
where id = $1 and active = true where id = $1 and binding_key is null and active = true
returning id, tenant_id as "tenantId", connection_id as "connectionId", returning id, tenant_id as "tenantId", connection_id as "connectionId",
provider_id as "providerId", allowed_data_product_ids as "allowedDataProductIds", provider_id as "providerId", allowed_data_product_ids as "allowedDataProductIds",
expires_at as "expiresAt", active, created_at as "createdAt", expires_at as "expiresAt", active, created_at as "createdAt",
@ -426,6 +568,22 @@ app.get("/internal/data-plane/v1/data-products/:dataProductId/snapshot", require
res.json(snapshot); res.json(snapshot);
})); }));
app.get("/internal/data-plane/v1/data-products/:dataProductId/history", requireReaderBinding, asyncRoute(async (req, res) => {
if (hasScopeHeaders(req)) throw httpError(400, "data_product_read_scope_headers_forbidden");
const dataProductId = assertReaderProduct(req.readerBinding, req.params.dataProductId);
const definition = await loadDataProductDefinition(pool, dataProductId);
if (!definition) throw httpError(404, "data_product_not_found");
const history = await readDataProductHistory(pool, req.readerBinding, definition, {
from: req.query.from,
to: req.query.to,
resolutionMs: req.query.resolutionMs,
sourceIds: parseSourceIds(req.query.sourceIds),
limit: req.query.limit,
cursor: req.query.cursor,
});
res.json(history);
}));
app.get("/internal/data-plane/v1/data-products/:dataProductId/stream", requireReaderBinding, asyncRoute(async (req, res) => { app.get("/internal/data-plane/v1/data-products/:dataProductId/stream", requireReaderBinding, asyncRoute(async (req, res) => {
if (shuttingDown) throw httpError(503, "service_shutting_down"); if (shuttingDown) throw httpError(503, "service_shutting_down");
if (hasScopeHeaders(req)) throw httpError(400, "data_product_read_scope_headers_forbidden"); if (hasScopeHeaders(req)) throw httpError(400, "data_product_read_scope_headers_forbidden");
@ -767,7 +925,8 @@ async function resolveWriterBinding(req) {
expires_at as "expiresAt", active, created_at as "createdAt", expires_at as "expiresAt", active, created_at as "createdAt",
rotated_at as "rotatedAt", revoked_at as "revokedAt" rotated_at as "rotatedAt", revoked_at as "revokedAt"
from external_data_plane_writer_bindings from external_data_plane_writer_bindings
where token_hash = $1 and active = true and expires_at > now()`, where token_hash = $1 and active = true
and (expires_at is null or expires_at > now())`,
[hashWriterToken(token)], [hashWriterToken(token)],
); );
if (!result.rowCount) throw httpError(401, "writer_binding_unauthorized"); if (!result.rowCount) throw httpError(401, "writer_binding_unauthorized");
@ -783,7 +942,8 @@ async function resolveReaderBinding(req) {
expires_at as "expiresAt", active, created_at as "createdAt", expires_at as "expiresAt", active, created_at as "createdAt",
rotated_at as "rotatedAt", revoked_at as "revokedAt" rotated_at as "rotatedAt", revoked_at as "revokedAt"
from external_data_plane_reader_bindings from external_data_plane_reader_bindings
where token_hash = $1 and active = true and expires_at > now()`, where token_hash = $1 and active = true
and (expires_at is null or expires_at > now())`,
[hashReaderToken(token)], [hashReaderToken(token)],
); );
if (!result.rowCount) throw httpError(401, "reader_binding_unauthorized"); if (!result.rowCount) throw httpError(401, "reader_binding_unauthorized");
@ -797,7 +957,8 @@ async function assertReaderStreamAccess(binding, dataProductId) {
join external_data_plane_products as product join external_data_plane_products as product
on product.id = $3 and product.active = true on product.id = $3 and product.active = true
where binding.id = $1 and binding.token_hash = $2 where binding.id = $1 and binding.token_hash = $2
and binding.active = true and binding.expires_at > now() and binding.active = true
and (binding.expires_at is null or binding.expires_at > now())
and binding.allowed_data_product_ids ? $3`, and binding.allowed_data_product_ids ? $3`,
[binding.id, binding.tokenHash, dataProductId], [binding.id, binding.tokenHash, dataProductId],
); );
@ -883,6 +1044,12 @@ function parseCursor(value) {
return cursor; return cursor;
} }
function parseSourceIds(value) {
if (value === undefined || value === null || value === "") return [];
const values = Array.isArray(value) ? value : String(value).split(",");
return values.map((item) => String(item || "").trim()).filter(Boolean);
}
function writePatchEvent(res, event) { function writePatchEvent(res, event) {
return writeSseFrame(res, `id: ${event.cursor}\nevent: nodedc.data-product.patch.v1\ndata: ${JSON.stringify(event)}\n\n`); return writeSseFrame(res, `id: ${event.cursor}\nevent: nodedc.data-product.patch.v1\ndata: ${JSON.stringify(event)}\n\n`);
} }

View File

@ -72,7 +72,7 @@ export function normalizeWriterBindingRequest(value, { now = new Date(), maxTtlD
* generated and stored inside Engine; EDP receives only its SHA-256 digest. * generated and stored inside Engine; EDP receives only its SHA-256 digest.
* `generation` makes a retry address the same immutable binding generation. * `generation` makes a retry address the same immutable binding generation.
*/ */
export function normalizeManagedWriterBindingRequest(value, { now = new Date(), maxTtlDays = 90 } = {}) { export function normalizeManagedWriterBindingRequest(value) {
if (!isPlainObject(value) || !isPlainObject(value.source)) { if (!isPlainObject(value) || !isPlainObject(value.source)) {
throw writerBindingError("managed_writer_binding_request_invalid"); throw writerBindingError("managed_writer_binding_request_invalid");
} }
@ -83,11 +83,16 @@ export function normalizeManagedWriterBindingRequest(value, { now = new Date(),
throw writerBindingError("managed_writer_binding_request_fields_invalid"); throw writerBindingError("managed_writer_binding_request_fields_invalid");
} }
const scope = normalizeWriterBindingRequest({ const tenantId = normalizeIdentifier(value.source.tenantId);
source: value.source, const connectionId = normalizeIdentifier(value.source.connectionId);
allowedDataProductIds: value.allowedDataProductIds, const providerId = normalizeIdentifier(value.source.providerId);
expiresAt: value.expiresAt, const allowedDataProductIds = uniqueIdentifiers(value.allowedDataProductIds);
}, { now, maxTtlDays }); if (!tenantId || !connectionId || !providerId || !allowedDataProductIds.length) {
throw writerBindingError("managed_writer_binding_scope_invalid");
}
if (value.expiresAt !== null) {
throw writerBindingError("managed_writer_binding_must_be_durable");
}
const generation = Number(value.generation); const generation = Number(value.generation);
if (!Number.isSafeInteger(generation) || generation < 1 || generation > 2_147_483_647) { if (!Number.isSafeInteger(generation) || generation < 1 || generation > 2_147_483_647) {
throw writerBindingError("managed_writer_binding_generation_invalid"); throw writerBindingError("managed_writer_binding_generation_invalid");
@ -98,8 +103,11 @@ export function normalizeManagedWriterBindingRequest(value, { now = new Date(),
} }
return Object.freeze({ return Object.freeze({
...scope, tenantId,
allowedDataProductIds: Object.freeze([...scope.allowedDataProductIds].sort()), connectionId,
providerId,
allowedDataProductIds: Object.freeze([...allowedDataProductIds].sort()),
expiresAt: null,
generation, generation,
capabilityDigest, capabilityDigest,
}); });
@ -111,13 +119,26 @@ export function writerBindingRequestHash(policy) {
connectionId: policy.connectionId, connectionId: policy.connectionId,
providerId: policy.providerId, providerId: policy.providerId,
allowedDataProductIds: [...policy.allowedDataProductIds].sort(), allowedDataProductIds: [...policy.allowedDataProductIds].sort(),
expiresAt: new Date(policy.expiresAt).toISOString(), expiresAt: null,
generation: policy.generation, generation: policy.generation,
capabilityDigest: policy.capabilityDigest, capabilityDigest: policy.capabilityDigest,
}); });
return createHash("sha256").update(canonical, "utf8").digest("hex"); return createHash("sha256").update(canonical, "utf8").digest("hex");
} }
export function canMigrateManagedWriterBindingToDurable(binding, policy, now = new Date()) {
if (!isPlainObject(binding) || !isPlainObject(policy) || binding.active !== true) return false;
const expiresAt = new Date(binding.expiresAt);
if (binding.expiresAt === null || Number.isNaN(expiresAt.getTime()) || expiresAt <= now) return false;
return binding.tenantId === policy.tenantId
&& binding.connectionId === policy.connectionId
&& binding.providerId === policy.providerId
&& binding.capabilityDigest === policy.capabilityDigest
&& JSON.stringify(uniqueIdentifiers(binding.allowedDataProductIds).sort())
=== JSON.stringify(uniqueIdentifiers(policy.allowedDataProductIds).sort())
&& policy.expiresAt === null;
}
/** /**
* Converts a caller-provided, deliberately unscoped intake envelope into the * Converts a caller-provided, deliberately unscoped intake envelope into the
* canonical scoped form. Caller scope is rejected, never trusted or merged. * canonical scoped form. Caller scope is rejected, never trusted or merged.
@ -202,7 +223,7 @@ export function safeWriterBinding(binding) {
providerId: binding.providerId, providerId: binding.providerId,
allowedDataProductIds: uniqueIdentifiers(binding.allowedDataProductIds), allowedDataProductIds: uniqueIdentifiers(binding.allowedDataProductIds),
active: binding.active === true, active: binding.active === true,
expiresAt: new Date(binding.expiresAt).toISOString(), expiresAt: binding.expiresAt === null ? null : new Date(binding.expiresAt).toISOString(),
createdAt: binding.createdAt ? new Date(binding.createdAt).toISOString() : undefined, createdAt: binding.createdAt ? new Date(binding.createdAt).toISOString() : undefined,
rotatedAt: binding.rotatedAt ? new Date(binding.rotatedAt).toISOString() : undefined, rotatedAt: binding.rotatedAt ? new Date(binding.rotatedAt).toISOString() : undefined,
revokedAt: binding.revokedAt ? new Date(binding.revokedAt).toISOString() : undefined, revokedAt: binding.revokedAt ? new Date(binding.revokedAt).toISOString() : undefined,
@ -214,6 +235,7 @@ export function writerBindingError(code) {
} }
function bindingIsCurrent(binding, now) { function bindingIsCurrent(binding, now) {
if (binding.expiresAt === null) return true;
const expiresAt = new Date(binding.expiresAt); const expiresAt = new Date(binding.expiresAt);
return !Number.isNaN(expiresAt.getTime()) && expiresAt > now; return !Number.isNaN(expiresAt.getTime()) && expiresAt > now;
} }

View File

@ -5,7 +5,7 @@ import { mkdtemp, rm, writeFile } from "node:fs/promises";
import net from "node:net"; import net from "node:net";
import { tmpdir } from "node:os"; import { tmpdir } from "node:os";
import { join } from "node:path"; import { join } from "node:path";
import { validateDataProductPatch, validateDataProductSnapshot } from "@nodedc/external-provider-contract/data-plane"; import { validateDataProductHistory, validateDataProductPatch, validateDataProductSnapshot } from "@nodedc/external-provider-contract/data-plane";
import { import {
MANAGED_PROVISIONER_HEADERS, MANAGED_PROVISIONER_HEADERS,
managedProvisionerSigningPayload, managedProvisionerSigningPayload,
@ -77,7 +77,7 @@ try {
const managedBody = { const managedBody = {
source: scope, source: scope,
allowedDataProductIds: [productId], allowedDataProductIds: [productId],
expiresAt: expiry, expiresAt: null,
generation: 1, generation: 1,
capabilityDigest: createHash("sha256").update(managedToken, "utf8").digest("hex"), capabilityDigest: createHash("sha256").update(managedToken, "utf8").digest("hex"),
}; };
@ -194,6 +194,48 @@ try {
}); });
assert.equal(rejectedManagedGeneration1.status, 401); assert.equal(rejectedManagedGeneration1.status, 401);
const managedReaderToken = "ndc_edprb_managed_api_test_0123456789abcdefghijklmnopqrstuvwxyz";
const managedReaderBody = {
source: scope,
allowedDataProductIds: [productId],
expiresAt: null,
generation: 1,
capabilityDigest: createHash("sha256").update(managedReaderToken, "utf8").digest("hex"),
};
const managedReaderPath = "/internal/data-plane/v1/reader-bindings/by-key/engine.api-connection.positions-reader";
const managedReaderResults = await Promise.all([
rawJsonRequest(managedReaderPath, { method: "PUT", managedSignature: true, body: managedReaderBody }),
rawJsonRequest(managedReaderPath, { method: "PUT", managedSignature: true, body: managedReaderBody }),
]);
assert.deepEqual(managedReaderResults.map(({ value }) => value.idempotent).sort(), [false, true]);
assert.equal(managedReaderResults[0].value.readerBinding.id, managedReaderResults[1].value.readerBinding.id);
assert.equal("token" in managedReaderResults[0].value, false);
assert.equal(JSON.stringify(managedReaderResults[0].value).includes(managedReaderBody.capabilityDigest), false);
assert.deepEqual(
(await jsonRequest("/internal/data-plane/v1/reader/data-products", { token: managedReaderToken })).dataProducts.map((value) => value.id),
[productId],
);
const legacyManagedReaderRevoke = await fetch(
`${baseUrl}/internal/data-plane/v1/reader-bindings/${managedReaderResults[0].value.readerBinding.id}/revoke`,
{ method: "POST", headers: { Authorization: `Bearer ${provisionerSecret}` } },
);
assert.equal(legacyManagedReaderRevoke.status, 404);
const managedReaderRevoke = await jsonRequest(
`${managedReaderPath}/generations/1/revoke`,
{ method: "POST", managedSignature: true },
);
assert.equal(managedReaderRevoke.idempotent, false);
assert.equal(managedReaderRevoke.readerBinding.active, false);
const managedReaderRevokeRetry = await jsonRequest(
`${managedReaderPath}/generations/1/revoke`,
{ method: "POST", managedSignature: true },
);
assert.equal(managedReaderRevokeRetry.idempotent, true);
const rejectedManagedReader = await fetch(`${baseUrl}/internal/data-plane/v1/reader/data-products`, {
headers: { Authorization: `Bearer ${managedReaderToken}` },
});
assert.equal(rejectedManagedReader.status, 401);
const manualWriterBody = { source: scope, allowedDataProductIds: [productId], expiresAt: expiry }; const manualWriterBody = { source: scope, allowedDataProductIds: [productId], expiresAt: expiry };
const signedOnlyManualProvisioning = await signedFetch("/internal/data-plane/v1/writer-bindings", { const signedOnlyManualProvisioning = await signedFetch("/internal/data-plane/v1/writer-bindings", {
method: "POST", method: "POST",
@ -245,6 +287,13 @@ try {
const snapshot = await jsonRequest(`/internal/data-plane/v1/data-products/${productId}/snapshot`, { token: reader.token }); const snapshot = await jsonRequest(`/internal/data-plane/v1/data-products/${productId}/snapshot`, { token: reader.token });
assert.equal(validateDataProductSnapshot(snapshot).ok, true); assert.equal(validateDataProductSnapshot(snapshot).ok, true);
assert.equal(snapshot.facts.length, 1); assert.equal(snapshot.facts.length, 1);
const history = await jsonRequest(
`/internal/data-plane/v1/data-products/${productId}/history?from=2026-07-15T12%3A00%3A00.000Z&to=2026-07-15T12%3A02%3A00.000Z&resolutionMs=60000&sourceIds=unit-01&limit=100`,
{ token: reader.token },
);
assert.equal(validateDataProductHistory(history).ok, true);
assert.equal(history.facts.length, 1);
assert.equal(history.facts[0].bucketStart, "2026-07-15T12:00:00.000Z");
const controller = new AbortController(); const controller = new AbortController();
const response = await fetch(`${baseUrl}/internal/data-plane/v1/data-products/${productId}/stream?after=${snapshot.cursor}`, { const response = await fetch(`${baseUrl}/internal/data-plane/v1/data-products/${productId}/stream?after=${snapshot.cursor}`, {

View File

@ -1,10 +1,11 @@
import assert from "node:assert/strict"; import assert from "node:assert/strict";
import { Pool } from "pg"; import { Pool } from "pg";
import { validateDataProductPatch, validateDataProductSnapshot } from "@nodedc/external-provider-contract/data-plane"; import { validateDataProductHistory, validateDataProductPatch, validateDataProductSnapshot } from "@nodedc/external-provider-contract/data-plane";
import { import {
loadDataProductDefinition, loadDataProductDefinition,
persistDataProductDefinition, persistDataProductDefinition,
persistDataProductPublish, persistDataProductPublish,
readDataProductHistory,
readDataProductSnapshot, readDataProductSnapshot,
readPatchEvents, readPatchEvents,
} from "../src/data-product-delivery.mjs"; } from "../src/data-product-delivery.mjs";
@ -154,6 +155,47 @@ try {
assert.equal(unitOneHistory.some((row) => new Date(row.observed_at).toISOString() === "2026-07-15T10:01:30.000Z"), true); assert.equal(unitOneHistory.some((row) => new Date(row.observed_at).toISOString() === "2026-07-15T10:01:30.000Z"), true);
assert.equal(unitOneHistory.some((row) => new Date(row.observed_at).toISOString() === "2026-07-15T10:02:00.000Z"), true); assert.equal(unitOneHistory.some((row) => new Date(row.observed_at).toISOString() === "2026-07-15T10:02:00.000Z"), true);
const historyPageOne = await readDataProductHistory(pool, binding, storedDefinition, {
from: "2026-07-15T10:00:00.000Z",
to: "2026-07-15T10:04:00.000Z",
resolutionMs: 60_000,
sourceIds: ["unit-01"],
limit: 2,
});
assert.equal(validateDataProductHistory(historyPageOne).ok, true);
assert.equal(historyPageOne.facts.length, 2);
assert.equal(typeof historyPageOne.nextCursor, "string");
const historyPageTwo = await readDataProductHistory(pool, binding, storedDefinition, {
from: "2026-07-15T10:00:00.000Z",
to: "2026-07-15T10:04:00.000Z",
resolutionMs: 60_000,
sourceIds: ["unit-01"],
limit: 2,
cursor: historyPageOne.nextCursor,
});
assert.equal(validateDataProductHistory(historyPageTwo).ok, true);
assert.equal(historyPageTwo.facts.length, 1);
assert.equal(historyPageTwo.nextCursor, undefined);
assert.deepEqual(
[...historyPageOne.facts, ...historyPageTwo.facts].map((value) => value.bucketStart),
[
"2026-07-15T10:00:00.000Z",
"2026-07-15T10:01:00.000Z",
"2026-07-15T10:02:00.000Z",
],
);
await assert.rejects(
readDataProductHistory(pool, binding, storedDefinition, {
from: "2026-07-15T10:00:00.000Z",
to: "2026-07-15T10:04:00.000Z",
resolutionMs: 60_000,
sourceIds: ["unit-02"],
limit: 2,
cursor: historyPageOne.nextCursor,
}),
(error) => error?.status === 400 && error?.code === "data_product_history_cursor_invalid",
);
console.log("external-data-plane delivery integration: ok"); console.log("external-data-plane delivery integration: ok");
} finally { } finally {
await pool.end(); await pool.end();

View File

@ -3,7 +3,9 @@ import {
assertReaderProduct, assertReaderProduct,
createReaderToken, createReaderToken,
hashReaderToken, hashReaderToken,
normalizeManagedReaderBindingRequest,
normalizeReaderBindingRequest, normalizeReaderBindingRequest,
readerBindingRequestHash,
safeReaderBinding, safeReaderBinding,
} from "../src/reader-binding.mjs"; } from "../src/reader-binding.mjs";
@ -22,4 +24,44 @@ const safe = safeReaderBinding({ id: "reader-01", ...policy, active: true, token
assert.equal("token" in safe, false); assert.equal("token" in safe, false);
assert.equal("tokenHash" in safe, false); assert.equal("tokenHash" in safe, false);
const managed = normalizeManagedReaderBindingRequest({
source: { tenantId: policy.tenantId, connectionId: policy.connectionId, providerId: policy.providerId },
allowedDataProductIds: ["fleet.positions.current.v1"],
expiresAt: null,
generation: 2,
capabilityDigest: hashReaderToken(token),
}, { now });
assert.equal(managed.generation, 2);
assert.match(readerBindingRequestHash(managed), /^[a-f0-9]{64}$/);
const safeManaged = safeReaderBinding({
id: "reader-managed-01",
bindingKey: "dprg-reader-managed",
generation: 2,
...managed,
active: true,
});
assert.equal(safeManaged.bindingKey, "dprg-reader-managed");
assert.equal(safeManaged.generation, 2);
assert.equal(safeManaged.expiresAt, null);
assert.equal(
assertReaderProduct(safeManaged, "fleet.positions.current.v1", new Date("2036-07-15T12:00:00.000Z")),
"fleet.positions.current.v1",
);
assert.equal(JSON.stringify(safeManaged).includes(managed.capabilityDigest), false);
assert.throws(() => normalizeManagedReaderBindingRequest({
source: { tenantId: policy.tenantId, connectionId: policy.connectionId, providerId: policy.providerId },
allowedDataProductIds: managed.allowedDataProductIds,
expiresAt: managed.expiresAt,
generation: managed.generation,
capabilityDigest: managed.capabilityDigest,
token: "forbidden",
}, { now }), /managed_reader_binding_secret_material_forbidden/);
assert.throws(() => normalizeManagedReaderBindingRequest({
source: { tenantId: policy.tenantId, connectionId: policy.connectionId, providerId: policy.providerId },
allowedDataProductIds: managed.allowedDataProductIds,
expiresAt: policy.expiresAt,
generation: managed.generation,
capabilityDigest: managed.capabilityDigest,
}), /managed_reader_binding_must_be_durable/);
console.log("external-data-plane reader bindings: ok"); console.log("external-data-plane reader bindings: ok");

View File

@ -1,6 +1,7 @@
import assert from "node:assert/strict"; import assert from "node:assert/strict";
import { validateIntakeBatch } from "../../../packages/external-provider-contract/src/data-plane.mjs"; import { validateIntakeBatch } from "../../../packages/external-provider-contract/src/data-plane.mjs";
import { import {
canMigrateManagedWriterBindingToDurable,
createWriterToken, createWriterToken,
hashWriterToken, hashWriterToken,
materializeDataProductPublish, materializeDataProductPublish,
@ -38,6 +39,7 @@ assert.notEqual(hashWriterToken(token), hashWriterToken(`${token}x`));
const managedPolicy = normalizeManagedWriterBindingRequest({ const managedPolicy = normalizeManagedWriterBindingRequest({
...request, ...request,
expiresAt: null,
allowedDataProductIds: ["fleet.positions.current.v1", "asset.status.current.v1"], allowedDataProductIds: ["fleet.positions.current.v1", "asset.status.current.v1"],
generation: 1, generation: 1,
capabilityDigest: hashWriterToken(token), capabilityDigest: hashWriterToken(token),
@ -48,26 +50,47 @@ assert.deepEqual(managedPolicy.allowedDataProductIds, [
]); ]);
assert.equal(managedPolicy.generation, 1); assert.equal(managedPolicy.generation, 1);
assert.equal(managedPolicy.capabilityDigest, hashWriterToken(token)); assert.equal(managedPolicy.capabilityDigest, hashWriterToken(token));
assert.equal(managedPolicy.expiresAt, null);
assert.equal(canMigrateManagedWriterBindingToDurable({
...managedPolicy,
expiresAt: "2026-08-01T12:00:00.000Z",
active: true,
capabilityDigest: managedPolicy.capabilityDigest,
}, managedPolicy, now), true);
assert.equal(canMigrateManagedWriterBindingToDurable({
...managedPolicy,
expiresAt: "2026-07-01T12:00:00.000Z",
active: true,
capabilityDigest: managedPolicy.capabilityDigest,
}, managedPolicy, now), false);
assert.equal(writerBindingRequestHash(managedPolicy), writerBindingRequestHash({ assert.equal(writerBindingRequestHash(managedPolicy), writerBindingRequestHash({
...managedPolicy, ...managedPolicy,
allowedDataProductIds: [...managedPolicy.allowedDataProductIds].reverse(), allowedDataProductIds: [...managedPolicy.allowedDataProductIds].reverse(),
})); }));
assert.throws(() => normalizeManagedWriterBindingRequest({ assert.throws(() => normalizeManagedWriterBindingRequest({
...request, ...request,
expiresAt: null,
generation: 0, generation: 0,
capabilityDigest: hashWriterToken(token), capabilityDigest: hashWriterToken(token),
}, { now, maxTtlDays: 90 }), /managed_writer_binding_generation_invalid/); }, { now, maxTtlDays: 90 }), /managed_writer_binding_generation_invalid/);
assert.throws(() => normalizeManagedWriterBindingRequest({ assert.throws(() => normalizeManagedWriterBindingRequest({
...request, ...request,
expiresAt: null,
generation: 1, generation: 1,
capabilityDigest: "not-a-digest", capabilityDigest: "not-a-digest",
}, { now, maxTtlDays: 90 }), /managed_writer_binding_capability_digest_invalid/); }, { now, maxTtlDays: 90 }), /managed_writer_binding_capability_digest_invalid/);
assert.throws(() => normalizeManagedWriterBindingRequest({ assert.throws(() => normalizeManagedWriterBindingRequest({
...request, ...request,
expiresAt: null,
generation: 1, generation: 1,
capabilityDigest: hashWriterToken(token), capabilityDigest: hashWriterToken(token),
token, token,
}, { now, maxTtlDays: 90 }), /managed_writer_binding_secret_material_forbidden/); }, { now, maxTtlDays: 90 }), /managed_writer_binding_secret_material_forbidden/);
assert.throws(() => normalizeManagedWriterBindingRequest({
...request,
generation: 1,
capabilityDigest: hashWriterToken(token),
}), /managed_writer_binding_must_be_durable/);
const unscopedBatch = { const unscopedBatch = {
schemaVersion: "nodedc.external-provider-contract/v1", schemaVersion: "nodedc.external-provider-contract/v1",
@ -148,5 +171,21 @@ const safeBinding = safeWriterBinding({
}); });
assert.equal("token" in safeBinding, false); assert.equal("token" in safeBinding, false);
assert.equal("tokenHash" in safeBinding, false); assert.equal("tokenHash" in safeBinding, false);
const safeManagedBinding = safeWriterBinding({
id: "binding-managed-01",
...managedPolicy,
active: true,
createdAt: now,
});
assert.equal(safeManagedBinding.expiresAt, null);
assert.equal(materializeDataProductPublish({
schemaVersion: "nodedc.data-product.publish/v1",
batch: { runId: "run-durable", sequence: 0, idempotencyKey: "run-durable.batch-0" },
facts: unscopedBatch.facts,
}, safeManagedBinding, {
id: "fleet.positions.current.v1",
version: "1.0.0",
ontologyRevision: "ontology.example-fleet.v1",
}, "fleet.positions.current.v1", { now: new Date("2036-07-15T12:00:00.000Z") }).batch.runId, "run-durable");
console.log("external-data-plane writer bindings: ok"); console.log("external-data-plane writer bindings: ok");