Compare commits

..

4 Commits

71 changed files with 9596 additions and 1690 deletions

View File

@ -49,4 +49,4 @@ Ontology Core живёт в `services/ontology-core` как docs-first сема
Map Gateway живёт в `services/map-gateway` как общий platform boundary для provider credentials, безопасного asset endpoint exchange, tile/3D Tiles proxy и persistent offline TileCache. Runtime cache не является source artifact и хранится в named volume/object storage, а не в Git. Map Gateway живёт в `services/map-gateway` как общий platform boundary для provider credentials, безопасного asset endpoint exchange, tile/3D Tiles proxy и persistent offline TileCache. Runtime cache не является source artifact и хранится в named volume/object storage, а не в Git.
Внешние бизнес-поставщики подключаются по `packages/external-provider-contract`: adapter конкретного API принадлежит изолированному NDC Agent L2 workflow, а Platform даёт один provider-neutral External Data Plane для raw retention, canonical facts, current/history projections и scoped read products. Connection instance несёт tenant scope, credential reference и collection profile, но не secret value. Provider/domain mapping, entity filtering и renderer logic не попадают в Platform Data Plane; writer capability привязана к immutable EDP binding и выдаётся только отдельному Engine provisioner, не shared internal bearer. Подробный канон — `docs/ADR_L2_OWNED_EXTERNAL_CONNECTORS.md`. Внешние бизнес-поставщики подключаются по `packages/external-provider-contract`: adapter конкретного API принадлежит изолированному NDC L2 workflow, а Platform даёт один provider-neutral External Data Plane для raw retention, canonical facts, current/history projections и scoped read products. Connection instance несёт tenant scope, credential reference и collection profile, но не secret value. Provider/domain mapping, entity filtering и renderer logic не попадают в Platform Data Plane. Provider-issued credentials сохраняются в native NDC L2 Credentials; внутренний scoped publish grant генерируется и сохраняется generic control plane внутри native credential boundary, EDP получает только digest, а MCP видит только opaque reference/status. Legacy plaintext provisioning и managed digest-only ensure включаются независимо и по умолчанию закрыты. Подробный канон — `docs/ADR_L2_OWNED_EXTERNAL_CONNECTORS.md`.

View File

@ -1,274 +1,238 @@
# ADR: L2-owned external connectors and provider-neutral Data Plane # ADR: внешние коннекторы принадлежат NDC L2 workflow
Статус: **accepted**. Статус: **accepted**.
Дата: 2026-07-14. Дата: 2026-07-16.
Владелец решения: NODE.DC Platform. Владелец решения: NODE.DC Platform.
## Контекст ## Контекст
NODE.DC — платформа разработки. Подключение внешнего API не должно создавать NODE.DC состоит из двух разных слоёв:
ещё один provider-specific сервис, в котором навсегда зашиты логика клиента,
нормализация и правила отображения. Иначе каждый новый account или новый тип
объекта превращается в изменение Platform runtime.
Нужна одна повторяемая форма: Platform даёт безопасные границы, контракт - платформенный слой даёт NDC L1, NDC L2, Ontology, credentials, Data Products
хранения и выдачи данных; изолированный workflow NDC Agent L2 является и Foundry-boundaries;
адаптером конкретного API и остаётся редактируемым через предоставленный - пользовательские автоматизации собирают из этих возможностей конкретную
Codex/Engine MCP. бизнес-логику.
Это решение заменяет provider-adapter часть Новый account, provider credential, tenant, расписание или интерфейс не должны создавать
`ADR_EXTERNAL_PROVIDER_DATA_PLANE.md` и все provider-specific runtime новый platform service и не должны требовать изменения NODE.DC source. Команда
предписания `ADR_GELIOS_DATA_PLANE.md`. Они сохраняются как исторические NODE.DC подключается только для нового поставщика либо для расширения
черновики, но не являются каноном для новой реализации. подтверждённых capabilities и ontology уже поддержанного поставщика.
Gelios — первый живой источник и acceptance-кейс этого канона, а не отдельная
архитектурная ветка. Документ заменяет provider-runtime решения из
`ADR_EXTERNAL_PROVIDER_DATA_PLANE.md` и `ADR_GELIOS_DATA_PLANE.md`; те
документы остаются только историей решения.
## Решение ## Решение
### 1. Адаптер внешнего API принадлежит L2 ### 1. Пакет поставщика создаётся один раз
Один изолированный L2 workflow представляет одно connection instance и Каждый поддержанный поставщик получает один версионируемый **provider package**.
является единственным местом для: Он является знанием платформы о внешнем API и содержит:
- обращения к API поставщика через ссылку на credential из Engine; - стабильный `providerId` и версии подтверждённого API;
- выбора read-capability, cursor/pagination, rate limit, retry и расписания; - credential contract без значения секрета;
- получения raw envelope, разбиения большого ответа на native batch-операции; - каталог safe-read capabilities, pagination/rate-limit metadata и response
- семантического mapping к версии ontology и формирования canonical facts; shapes;
- idempotency key, watermark и lifecycle источника; - mappings из provider fields в версии Ontology;
- deploy/run/execution/trace через Engine MCP. - совместимые Data Products и scrubbed contract fixtures;
- явно отделённый каталог команд без активного command transport.
L2 не хранит значение provider secret или writer token: он использует только Пакет не является runtime service, scheduler, базой данных или customer
Engine-bound credential references. Plaintext writer token выдаётся trusted configuration. Он расширяется только когда фактически подтверждён новый API,
provisioner ровно один раз при create/rotate binding и затем хранится только в новый тип данных или новая ontology revision.
Engine credential store, привязанном к этому L2; External Data Plane хранит
лишь его hash. Provisioner читает отдельный runner-owned secret file, который
read-only монтируется только в EDP и позднее в dedicated Engine provisioner,
работающий под выделенным UID/GID `11006`; это не shared `.env`, не
`NODEDC_INTERNAL_ACCESS_TOKEN` и не provider credential. До deployment этого
provisioner `EXTERNAL_DATA_PLANE_PROVISIONING_ENABLED=false`, поэтому issuance
routes fail closed. Token не попадает в L2 graph/profile, contract artifact,
execution log/trace, raw envelope, Foundry или UI. Command/write-capabilities
не получают transport route в read workflow.
Для новой учётной записи создаётся новый instance/profile этого workflow с Одна учётная запись поставщика задаётся данными, а не кодом:
другой credential reference и connection configuration. Это не требует
изменения Platform source. Если поставщик раскрывает новый объект или поле,
сначала расширяется ontology/capability contract, затем изменяется именно
соответствующий L2 workflow.
### 2. Platform Data Plane нейтрален к provider и домену `provider package + credential reference + connection profile + NDC L2 workflow instance`
Platform предоставляет один versioned **External Data Plane**. Он принимает Поэтому второй account или другая provider credential pair создаёт ещё один credential/profile и
batch по внутреннему аутентифицированному контракту, хранит raw envelope с workflow instance. Платформенный source, Data Plane и Foundry при этом не
retention/provenance, canonical facts, current/history projections и отдаёт меняются.
scoped data products/realtime updates. В нём запрещены:
- ветвления по названию provider, customer, account, vehicle или renderer; ### 2. NDC L1 проектирует, NDC L2 workflow исполняет
- provider field mapping, unit allowlist, business filtering и visual rules;
- provider token, прямой browser-to-provider доступ и command transport.
Его canonical write-форма, которую Data Plane валидирует и сохраняет после NDC L1 получает пользовательское намерение, проверяет доступные capabilities и
авторизации, содержит только нейтральные элементы: Ontology, проектирует или изменяет разрешённый NDC L2 workflow через NDC MCP и
анализирует execution evidence.
```text Один изолированный NDC L2 workflow представляет один connection instance и
source: providerId + tenantId + connectionId владеет исполняемой механикой:
contract: contractVersion + ontologyRevision + dataProductId
batch: runId + sequence + idempotencyKey + receivedAt
raw: optional restricted ref + contentType + hash + retention policy
facts: stable sourceId + semanticType + observedAt + attributes + geometry?
```
Для нового L2 writer-а это не wire-форма. Он вызывает - safe-read вызовами provider API через credential reference;
`POST /internal/data-plane/v1/data-products/:dataProductId/publish` с bearer - pagination, cursor, retry, rate limit, batch size и collection cadence;
writer token и формой `nodedc.data-product.publish/v1`, в которой вообще нет - проверкой response shape и разбиением ответа на items;
`source`, contract metadata, transport или persistence policy. Любой caller - mapping к точной ontology revision;
scope и scope headers запрещены. - idempotency, watermark и публикацией canonical facts;
- private workflow state, когда он действительно нужен.
External Data Plane разрешает token в собственный immutable writer binding Provider-specific mapping сначала реализуется штатными workflow nodes и малым
`(tenantId, connectionId, providerId, allowedDataProductIds, expiresAt)`, Code-преобразователем. Это позволяет проверить реальный API без преждевременного
проверяет active/non-expired binding, совпадение `providerId` и разрешение создания custom nodes. В custom NDC nodes переносится только повторившаяся и
`dataProductId`, затем сам materializes canonical `source` с tenant/connection доказанная **платформенная boundary-механика**, а не уникальная логика
и только после этого валидирует и сохраняет batch. Присланный caller scope поставщика.
отклоняется, а не merge/override-ится. Изменение tenant, connection, provider,
allowed data products или срока требует нового binding; для существующего
binding допустимы только token rotation и revoke.
Выпуск, rotation и revoke binding принимаются только от dedicated provisioning ### 3. Credentials являются данными connection instance
principal с runner-owned secret file; он не монтируется в shared `.env` или L2.
Shared legacy bearer не может создавать writer capability.
Идентификатор источника стабилен в пределах `(providerId, connectionId, Gelios выдаёт ровно два provider secrets: access token и refresh token. Текущий
sourceId)`. Отсутствие объекта в очередном ответе помечается lifecycle-state HTTP request binding `httpBearerAuth` использует access token. Автоматический
или временем последнего наблюдения; запись и её история не удаляются. refresh в принятом runtime не доказан, поэтому provider package фиксирует его
как `operator_managed`; ни один из token values не попадает в graph, package,
Ontology, Data Plane, Foundry, execution logs, MCP или Ops. Новый account или
provider-issued token pair означает новые native credential records и
connection instance, но не новую Platform-сущность.
### 3. Граница данных и интерфейса Локальное имя credential (например, с суффиксом `read access`) не является
provider scope. В target-контракте Read-классификацию задаёт allowlisted
method/endpoint в capability catalog и Engine workflow policy; тот же access
token нельзя называть отдельным «read token» или «write token» только из-за
label. Сейчас deployed Engine safe-ref policy ограничивает generic HTTP
credential только по host, но ещё не связывает его с package/version и exact
method/path. Поэтому Gelios `GET /api/v1/units` пока является декларативной
capability, а не завершённой runtime-security гарантией; canonical acceptance
требует capability-bound Engine policy и её MCP proof.
```text Writer capability для публикации Data Product — отдельный внутренний native NDC
Provider API (safe read) L2 credential, не третий Gelios token. Generic Engine/Platform control plane
-> Engine provider-credential reference генерирует capability внутри native credential boundary, сохраняет её там же и
-> L2 connector instance (adapter + mapping + batches) передаёт EDP только SHA-256 digest вместе с точным provider, connection и
-> Engine writer-credential reference набором Data Products. Пользователь и MCP получают только opaque
-> External Data Plane Data Product publish (scope materialized server-side) reference/status. Capability не записывается в graph, provider package, env,
-> External Data Plane (generic persistence + projections) file, Ops или trace и не копируется через пользовательский UI.
-> scoped data product / realtime stream
-> Foundry binding / user interface
```
Ontology Core описывает semantic types, связи, capability catalog и mapping Широкий credential sink, resolver/daemon с произвольной записью secret и любой
versions, но не хранит runtime payloads или секреты. Foundry получает только provider-specific credential service запрещены. Нужна одна узкая операция
approved data product и решает presentation: pins, visibility, layers and `ensure data-product publish grant`: scope выводится из granted L1→L2 target,
filters. Оно не получает provider endpoint или credential. зарегистрированного connection profile и разрешённого Data Product; issuance,
rotation и native binding должны быть idempotent, CAS/crash-safe и auditable.
Engine генерирует capability внутри credential boundary и передаёт EDP только
SHA-256 digest через idempotent binding key + generation; EDP не возвращает
plaintext. Новая generation создаётся до переключения node reference, а старая
отзывается только после успешного bind/acceptance.
Старые ручные EDP `POST/rotate` endpoints, которые возвращают capability один
раз, включаются отдельным legacy-флагом и не входят в canonical acceptance.
Digest-only managed endpoint имеет независимый флаг и принимает только
Ed25519-signed Engine service requests: exact audience/method/request target/raw
body hash входят в подпись, timestamp ограничен по skew, nonce защищён bounded
fail-closed replay cache. Legacy provisioner bearer на managed routes не
действует. EDP получает только deployment public key; matching private key
остаётся только внутри Engine server boundary. Endpoint остаётся выключенным,
пока key provisioning, Engine signer и exact native credential binding policy
не пройдут runtime acceptance.
В deployed Engine MCP этой операции пока нет — это текущий platform gap перед
canonical publish proof, а не действие пользователя. Это решение не заявляет
автоматический refresh Gelios: до отдельного runtime proof он остаётся
`operator_managed`.
### 4. Полнота без entity allowlist ### 4. External Data Plane нейтрален к поставщику
Collection profile выбирает разрешённые **read-capabilities**, а не список Platform предоставляет один versioned External Data Plane. Он принимает
конкретных объектов. Если выбранный read endpoint возвращает все доступные canonical facts через scoped writer binding, хранит current/history
источнику сущности, L2 передаёт все валидные элементы. Новые сущности projections и публикует snapshot/patch contracts. В нём запрещены:
добавляются автоматически; скрытие/отображение — задача data product/UI, а не
сбора. Технические лимиты существуют только как размер batch, backpressure,
quota и защита от повреждённого ответа, но не как бизнес-фильтр по ID.
### 5. Масштабирование и native NDC Agent nodes - ветвления по provider, customer, account, entity или renderer;
- provider field mapping и бизнес-фильтрация;
- provider credential, endpoint, schedule или command transport;
- caller-supplied tenant/connection scope.
L2 не запускает отдельный execution на каждую сущность. Он использует native `NDC Data Product Publish` отправляет только
nodes NDC Agent: HTTP Request, Split Out, Split In Batches/Loop Over Items, `nodedc.data-product.publish/v1`. External Data Plane materializes immutable
Aggregate, Postgres (когда нужен private workflow state), Set/If/Merge, scope из writer binding, проверяет разрешённый Data Product и его ontology
Schedule Trigger and Respond to Webhook. Code node допустим только как малый revision, затем сохраняет batch.
преобразователь boundary-shape, когда эквивалентной native node нет; он не
становится скрытым сервисом или provider database.
Shared telemetry/history не записывается L2 напрямую в физические таблицы EDP runtime импортирует только package subpath
Platform Postgres: это создало бы coupling к schema. Direct Postgres допустим `@nodedc/external-provider-contract/data-plane`. Его deploy artifact и image
для private state конкретного workflow или отдельной project-owned базы. содержат только provider-neutral wire validators; `providers/gelios`, mappings,
Общий контур использует только External Data Plane contract и bulk batches. fixtures и tests туда не входят. Изменение или добавление provider package не
пересобирает и не перезапускает EDP: каталог поставщиков разворачивается через
Engine/Ontology/control-plane путь отдельно.
### 5.1. Каталог custom nodes NODE.DC Collection cadence, history cadence и presentation cadence независимы:
Встроенные узлы runtime сохраняют свои штатные названия (`HTTP Request`, - NDC L2 забирает источник с частотой, нужной бизнес-задаче;
`Schedule Trigger`, `Loop Over Items` и т. п.). Любой узел, код которого - current projection принимает каждое валидное изменение;
принадлежит NODE.DC, обязан одновременно выполнять три условия: - declarative history policy может хранить все точки или sampling;
- Foundry читает snapshot+patch и отдельно ограничивает частоту render.
- видимое имя начинается с `NDC `; Для разной частоты БД и интерфейса не создаются второй provider sink, отдельный
- runtime type принадлежит package namespace `n8n-nodes-ndc.*`; gateway или параллельный прямой push в renderer.
- package-level test отклоняет публикацию узла без этого префикса и namespace.
Первый канонический набор состоит из: ### 5. Полнота означает все сущности разрешённой capability
- `NDC Data Product Publish` — runtime write boundary L2 → Data Plane; Connection profile выбирает safe-read capabilities, а не зашитый в Platform
- `NDC Data Product Read` — scoped current snapshot read по reader grant; список entity IDs. Если разрешённый endpoint возвращает все доступные credential
- `NDC Foundry Binding` — deploy/control-plane связь Data Product с сущности, NDC L2 обрабатывает каждый валидный item. Новый объект появляется
`Application → Page → typed slot`, но не транспорт каждого realtime tick. автоматически.
`NDC Foundry Output` не используется как техническое имя: оно ошибочно Пользовательская фильтрация, слои и видимость находятся после сбора — в
подразумевает прямой transport L2 → renderer. Provider-specific adapter, automation/Data Product/Foundry. Технические ограничения допустимы только как
mapping и collection profile остаются логикой конкретного L2 workflow на pagination, quota, batch size, backpressure и защита от повреждённого ответа.
штатных nodes; они не оформляются как custom NODE.DC nodes и не добавляют
provider branch в Data Plane или Foundry.
`NDC Foundry Binding` отправляет только ### 6. Custom NDC nodes ограничены платформенными границами
`nodedc.foundry.binding-upsert/v1`. Её постоянный L2 credential — отдельный
revocable `ndc_fndbg_*` workload grant с allowlist точных
`Application → Page → Binding → Slot → Data Product` targets. Короткоживущая
`fnd1.*` capability интерактивного Foundry MCP, browser session, EDP
writer/reader grant и `NODEDC_INTERNAL_ACCESS_TOKEN` для этой ноды запрещены.
Runtime node не содержит URL, endpoint, provider ID, tenant ID, connection ID, Первый канонический набор:
token, contract version или history cadence. Узел выбирает только разрешённый
Data Product; всё остальное materializes из opaque writer/reader binding и
immutable Data Product definition.
Package `n8n-nodes-ndc` устанавливается как штатный private community package в - `NDC Data Product Publish` — NDC L2 → External Data Plane;
`/home/node/.n8n/nodes/node_modules`, чтобы n8n `PackageDirectoryLoader` - `NDC Data Product Read` — scoped snapshot/patch read;
сохранил namespace `n8n-nodes-ndc.*`. `~/.n8n/custom` и - `NDC Foundry Binding` — control-plane связь Data Product с
`N8N_CUSTOM_EXTENSIONS` запрещены для этого пакета: они создают namespace `Application → Page → typed slot`.
`CUSTOM.*`. Ручное копирование или `npm install` в живом container, patch
Engine core и подмена built-in node запрещены. Production использует
проверенный offline tarball, immutable release, atomic current/previous switch,
read-only mount и restart всех n8n execution processes; acceptance завершается
только когда Engine schema и MCP catalog возвращают точные package-qualified
types.
### 5.2. Realtime delivery contract Эти nodes не содержат provider ID, tenant ID, connection ID, endpoint, token,
mapping или cadence. Runtime package сохраняет технический namespace
`n8n-nodes-ndc.*`, но пользовательские документы и интерфейсы используют
только терминологию NDC.
Новый writer использует: `NDC Foundry Binding` не транспортирует каждый realtime tick. Он создаёт
постоянную связь интерфейса с Data Product; Foundry затем читает его
snapshot+patch contract.
```text ### 7. Реальный Gelios acceptance-кейс
POST /internal/data-plane/v1/data-products/:dataProductId/publish
Authorization: Bearer <opaque writer binding>
```
Body имеет schema `nodedc.data-product.publish/v1` и содержит только batch Первая версия provider package должна доказать путь:
identity и canonical facts. Data Plane одной транзакцией:
1. проверяет idempotency; `Gelios safe read → все доступные units → map.moving_object facts →
2. обновляет current projection только более новым или изменившимся фактом; fleet.positions.current.v1 → Foundry map`
3. применяет declarative history policy (`none`, `all`, `sampled`);
4. добавляет changed facts в durable patch outbox;
5. фиксирует монотонный cursor.
Reader grant получает snapshot `nodedc.data-product.snapshot/v1`, затем Canonical product использует ontology revision
подключается к SSE stream с `after=<snapshot.cursor>` или `Last-Event-ID`. `ontology.map.moving_object.v1` и только объявленные snake_case поля. NDC L2
Каждый outbox event имеет schema `nodedc.data-product.patch/v1`. Если cursor не передаёт caller scope и не собирает собственный batch envelope вокруг
уже удалён retention policy, stream отвечает `409 resync_required`, и consumer `NDC Data Product Publish`.
повторяет snapshot. Browser не получает Data Plane token или scope: Foundry BFF
разрешает persisted binding и читает server-only reader grant.
`snapshot+patch` в v1 является **bounded product contract**. Current projection Acceptance выполняется по порядку:
одного scoped Data Product не может содержать больше 5000 entity keys
`(sourceId, semanticType)`. Snapshot возвращает целую согласованную projection
и один barrier cursor из одной `REPEATABLE READ` transaction. Query-параметр
`limit` — только защитный потолок, а не размер страницы: если полная projection
не помещается, Data Plane отвечает `413 data_product_snapshot_limit_exceeded`
и consumer не начинает patch stream с неполной базой.
Наивная pagination current snapshot по `sourceId`, offset или независимо 1. Platform выполняет generic `ensure data-product publish grant`, выпускает
полученным page cursors запрещена: изменения между страницами могут быть scoped EDP binding и атомарно сохраняет capability в native NDC L2
пропущены или продублированы относительно patch cursor. Product с ожидаемой Credentials;
cardinality выше 5000 обязан **до включения realtime** выбрать один из двух 2. NDC MCP видит только совместимый opaque writer credential reference/status;
отдельно версионируемых контрактов: 3. применить подтверждённый graph patch без legacy intake;
4. validate/preflight;
5. один успешный manual run и проверка execution/trace/Data Product;
6. только затем отдельным изменением добавить Schedule Trigger;
7. после доказанного snapshot+patch подключить Foundry binding.
- stable partitioning в несколько Data Products, где partition key является ### 8. Capacity и topology
частью definition, а каждый partition имеет собственный полный snapshot и
независимый patch cursor;
- новый query delivery contract с явно определёнными snapshot barrier,
continuation cursor, query scope и правилами перехода к patch stream.
Такой query contract не входит в `nodedc.data-product.snapshot/v1` и не может Количество одновременно активных NDC L2 проектов ограничивается фактическими
быть имитирован полем `nextPageCursor`. До его отдельного утверждения runtime и ресурсами железа и профилем нагрузки. Пока capacity проверяется оператором и не
`NDC Data Product Read` работают только с bounded products до 5000 сущностей. автоматизируется. Канон не объявляет отдельные worker/webhook generations или
high-load topology, которых ещё нет в принятом runtime.
### 6. Порядок первой реализации
1. Версионировать нейтральный intake/read contract и поднять External Data
Plane без provider-specific logic.
2. Пересобрать существующий L2 proof в native-node pipeline: safe read →
split/batch → semantic mapping → generic append → safe summary. Никаких
provider ID filters и provider gateway endpoint.
3. Выполнить manual run, проверить execution/trace, idempotency and current
projection; только затем включить Schedule Trigger.
4. Provision immutable writer binding, one-time place its token into an opaque
Engine credential, then switch the L2 to
`/data-products/:dataProductId/publish` and remove caller scope/headers and
the shared legacy credential.
5. Подключить Foundry к data product current positions. История, геозоны и
новые сущности добавляются отдельными L2 collection profiles and ontology
revisions.
## Последствия ## Последствия
- `services/<provider>-gateway` не является шаблоном для новых интеграций. - `services/<provider>-gateway` не является частью канона и не создаётся для
Существующий experimental code не расширяется и не деплоится как часть этого новых provider integrations. Существующий self-hosted Gelios Gateway и его
решения. Timescale volume остаются frozen compatibility contour, воспроизводятся из
- Data Plane может иметь собственную БД/Timescale/PostGIS implementation, но source/deploy и не изменяются новым L2 pilot без отдельного решения.
физическая схема остаётся внутренней деталью neutral service, а не API для - Новый account или provider-issued access/refresh pair — configuration change,
L2 или Foundry. а не platform release.
- Inline raw запрещён в L2 → Data Plane v1: до отдельного raw-vault допускается - Новый provider или неподдержанная capability — versioned provider/Ontology
только restricted ref/hash либо отсутствие raw envelope. Retention reference change с contract tests.
вычисляется по серверному acceptance time, а не по timestamp из batch; Data - Существующий legacy L1/SDK workflow является frozen compatibility boundary:
Plane выполняет bounded retention sweeps после готовности сервиса и по таймеру. новый L2 pilot его не редактирует, не отзывает его credential и не ставит его
- Обычный Codex Desktop получает ровно те L2 grants, которые выданы владельцем вывод условием acceptance.
connection/workflow, и может безопасно развивать adapter только в этой - Legacy `/internal/data-plane/v1/intake` остаётся выключенным migration-only
границе. route и не используется новыми NDC L2 workflows.
- `POST /internal/data-plane/v1/intake` остаётся временным migration-only route: - Команды устройствам остаются отдельным red-domain контуром с явным
он выключен по умолчанию (`EXTERNAL_DATA_PLANE_LEGACY_INTAKE_ENABLED=false`), подтверждением, scope, idempotency и аудитом.
требует `NODEDC_INTERNAL_ACCESS_TOKEN`, canonical scoped batch и совпадающие - Foundry развивается после доказанного Data Product path; provider API и
scope headers. Новый или migrated L2 не получает этот shared token и не credentials в Foundry не попадают.
использует fallback в legacy route. После миграции всех writers он удаляется.

View File

@ -28,7 +28,7 @@ The worker receives only a pairing-bound Hub URL. The Hub verifies that the pair
It intentionally does not expose filesystem paths, evidence ledgers, credentials, raw payloads, live telemetry, databases, command dispatch, workflow mutation or Studio controls. It intentionally does not expose filesystem paths, evidence ledgers, credentials, raw payloads, live telemetry, databases, command dispatch, workflow mutation or Studio controls.
## Boundary for Gelios and future data services ## Boundary for external providers and runtime data
Ontology Core describes the canonical meanings and constraints: Ontology Core describes the canonical meanings and constraints:
@ -36,15 +36,21 @@ Ontology Core describes the canonical meanings and constraints:
gelios.unit -> gelios.telemetry_snapshot -> gelios.position_fix -> map.moving_object gelios.unit -> gelios.telemetry_snapshot -> gelios.position_fix -> map.moving_object
``` ```
It does **not** serve the Gelios database or provider API. The future Gelios Gateway/Data API is a distinct capability with its own scope checks, data contract and transport. That capability may later be supplied to an AI Workspace run as another dynamic MCP server. Ontology then gives the assistant the names, relations, guardrails and allowed data-contract route; the data capability enforces access and returns live data. It does **not** serve provider runtime data or call a provider API. Provider
access and mapping belong to the granted NDC L2 workflow; shared current and
history products belong to the provider-neutral External Data Plane. A scoped
Data Product read capability may later be supplied to an AI Workspace run as a
separate dynamic MCP server. Ontology supplies meanings, relations, guardrails
and the allowed contract route; the data capability enforces access and returns
live data.
This preserves one-way responsibility: This preserves one-way responsibility:
- Ontology Core: semantics, contracts, aliases, guardrails and context advice. - Ontology Core: semantics, contracts, aliases, guardrails and context advice.
- Gelios Gateway/Data API: access-scoped telemetry and history reads. - External Data Plane: access-scoped Data Product snapshots and updates.
- Command Gateway: separate red-domain command route, explicit confirmation and audit. - Command Gateway: separate red-domain command route, explicit confirmation and audit.
- Engine L2: workflow orchestration using granted capabilities. - NDC L2 workflow: provider access, collection and mapping using granted capabilities.
- Studio: later presentation consumer, outside this implementation. - Foundry: presentation consumer, outside this implementation.
## Live catalog updates without AI Workspace rule redeploy ## Live catalog updates without AI Workspace rule redeploy

View File

@ -75,10 +75,20 @@ EXTERNAL_DATA_PLANE_WRITER_BINDING_MAX_TTL_DAYS=90
EXTERNAL_DATA_PLANE_MAX_FUTURE_SKEW_SECONDS=300 EXTERNAL_DATA_PLANE_MAX_FUTURE_SKEW_SECONDS=300
EXTERNAL_DATA_PLANE_RETENTION_SWEEP_MS=3600000 EXTERNAL_DATA_PLANE_RETENTION_SWEEP_MS=3600000
EXTERNAL_DATA_PLANE_LEGACY_INTAKE_ENABLED=false EXTERNAL_DATA_PLANE_LEGACY_INTAKE_ENABLED=false
# The writer-provisioner secret is not an env value. The root-owned deploy # Internal control-plane writer/reader binding issuance; keep false until the
# runner keeps it in /volume1/docker/nodedc-platform/secrets/external-data-plane-provisioner/ # atomic NDC L2 ensure-grant operation is deployed. Legacy path returns a
# and mounts it # plaintext capability and must never be exposed to users.
# only into External Data Plane and the future dedicated Engine provisioner. EXTERNAL_DATA_PLANE_PROVISIONING_ENABLED=false
# Digest-only managed writer-binding ensure. It accepts only signed Engine
# service requests; the legacy provisioner bearer is deliberately invalid here.
# Keep false until the matching Engine private key is provisioned. The trust
# directory is mounted read-only into EDP and contains only `public-key.pem`.
EXTERNAL_DATA_PLANE_MANAGED_PROVISIONING_ENABLED=false
EXTERNAL_DATA_PLANE_MANAGED_PROVISIONER_SERVICE_ID=nodedc-engine
EXTERNAL_DATA_PLANE_MANAGED_PROVISIONER_KEY_ID=engine-edp-managed-provisioner-v1
EXTERNAL_DATA_PLANE_MANAGED_PROVISIONER_AUDIENCE=nodedc-external-data-plane.managed-provisioning.v1
EXTERNAL_DATA_PLANE_MANAGED_PROVISIONER_MAX_SKEW_SECONDS=60
EXTERNAL_DATA_PLANE_MANAGED_PROVISIONER_REPLAY_CACHE_MAX_ENTRIES=10000
# notification core # notification core
NOTIFICATION_PG_DB=nodedc_notifications NOTIFICATION_PG_DB=nodedc_notifications
@ -119,8 +129,10 @@ AI_WORKSPACE_HUB_FALLBACK_URLS=
AI_WORKSPACE_ONTOLOGY_MCP_PUBLIC_URL= AI_WORKSPACE_ONTOLOGY_MCP_PUBLIC_URL=
ONTOLOGY_CORE_HOST_BIND=127.0.0.1:18104 ONTOLOGY_CORE_HOST_BIND=127.0.0.1:18104
# Gelios Gateway — storage/read service. Provider credentials belong to the # Gelios Gateway — frozen legacy storage/read compatibility service. Provider
# protected Engine Collector and are never configured in this service. # credentials belong to the protected Engine Collector and are never configured
# in this service. Keep this contour reproducible; do not use it as a template
# for new providers.
GELIOS_TIMESCALE_IMAGE=timescale/timescaledb-ha:pg16.14-ts2.28.2-all GELIOS_TIMESCALE_IMAGE=timescale/timescaledb-ha:pg16.14-ts2.28.2-all
GELIOS_PG_DB=nodedc_gelios GELIOS_PG_DB=nodedc_gelios
GELIOS_PG_USER=nodedc_gelios GELIOS_PG_USER=nodedc_gelios
@ -128,8 +140,6 @@ GELIOS_PG_PASS=change-me-generate-with-infra-scripts-init-dev-env
# URL-encode reserved characters in GELIOS_PG_PASS when forming this URL. # URL-encode reserved characters in GELIOS_PG_PASS when forming this URL.
GELIOS_DATABASE_URL=postgresql://nodedc_gelios:change-me-generate-with-infra-scripts-init-dev-env@gelios-postgres:5432/nodedc_gelios GELIOS_DATABASE_URL=postgresql://nodedc_gelios:change-me-generate-with-infra-scripts-init-dev-env@gelios-postgres:5432/nodedc_gelios
GELIOS_GATEWAY_HOST_BIND=127.0.0.1:18105 GELIOS_GATEWAY_HOST_BIND=127.0.0.1:18105
# Explicit tenant and connection identifiers are deployment configuration;
# do not encode a customer or pilot name in source defaults.
GELIOS_TENANT_ID=replace-with-tenant-id GELIOS_TENANT_ID=replace-with-tenant-id
GELIOS_CONNECTION_ID=gelios-connection-id GELIOS_CONNECTION_ID=gelios-connection-id
# `allowlist` accepts only GELIOS_ALLOWED_UNIT_IDS. `all` accepts every unit # `allowlist` accepts only GELIOS_ALLOWED_UNIT_IDS. `all` accepts every unit

View File

@ -97,8 +97,15 @@ Engine-owned activator may select it after exact MCP schema acceptance.
The paired Engine activation is built by The paired Engine activation is built by
`build-engine-n8n-private-extension-artifact.mjs`. It deliberately does not `build-engine-n8n-private-extension-artifact.mjs`. It deliberately does not
copy the dirty Engine `docker-compose.yml` and does not build an image. Instead copy from or write generated files into the dirty Engine worktree, and it does
it installs a narrowly scoped Compose override plus a strict transition not build an image. A fresh transition id is mandatory and previously issued
ids are rejected:
```bash
node infra/deploy-runner/build-engine-n8n-private-extension-artifact.mjs 20260717-004
```
The builder emits a narrowly scoped Compose override plus a strict transition
descriptor. On apply, the runner validates the staged release again, verifies descriptor. On apply, the runner validates the staged release again, verifies
that the running n8n container and the NAS-local `2.3.2` tag resolve to the that the running n8n container and the NAS-local `2.3.2` tag resolve to the
same immutable image ID, and extracts the package into the root-owned, same immutable image ID, and extracts the package into the root-owned,
@ -110,10 +117,15 @@ read-only Engine release tree:
The override sets `N8N_USER_FOLDER=/home/node`, which is required because the The override sets `N8N_USER_FOLDER=/home/node`, which is required because the
actual Engine service runs as root while the canonical community package path actual Engine service runs as root while the canonical community package path
is below `/home/node/.n8n`. It enables loading but disables reinstall, mounts is below `/home/node/.n8n`. The live runtime contract remains the successfully
only the exact release read-only, and uses both Compose `pull_policy: never` deployed generation-003 contract and does not set `NODE_PATH`. Only the
and `docker compose up --pull never`. No registry access, lifecycle script, runner's isolated `node -e` package-loader probe temporarily initializes the
database `installed_packages` row or custom-extension loader is involved. dependency tree bundled inside the exact n8n base image; this reproduces n8n's
own loader without redefining the live Compose state. The override enables
loading but disables reinstall, mounts only the exact release read-only, and
uses both Compose `pull_policy: never` and `docker compose up --pull never`.
No registry access, lifecycle script, database `installed_packages` row or
custom-extension loader is involved.
The runner pins the exact Engine service topology observed in source and The runner pins the exact Engine service topology observed in source and
rejects an added worker/webhook generation. Only the single actual `n8n` rejects an added worker/webhook generation. Only the single actual `n8n`
@ -140,20 +152,106 @@ PYTHONDONTWRITEBYTECODE=1 \
``` ```
For `platform` artifacts, the allowlist includes the versioned Ontology Core, For `platform` artifacts, the allowlist includes the versioned Ontology Core,
the legacy Gelios experiment and the provider-neutral External Data Plane the frozen legacy Gelios compatibility service, and the provider-neutral
sources. An External Data Plane artifact builds only its image and starts External Data Plane sources. The Gelios service remains reproducible only to
`external-data-plane-postgres` plus `external-data-plane`; it never contains a protect its existing database/workflow; it is not a template for a provider
provider credential, provider endpoint, collection schedule or command integration. An External Data Plane artifact builds only its image and
transport. Database credentials remain root-owned live `.env.synology` force-recreates only `external-data-plane`; the already healthy
`external-data-plane-postgres` container and its Timescale volume are an
independent deploy prerequisite and are never selected by an EDP application
artifact. The reviewed Compose source pins the Timescale image, named volume
and target, internal database-only network, absence of database host ports,
healthy dependency, localhost-only EDP bind and the two read-only
provisioner/trust mounts in the reviewed Compose source. The runner does not
reinterpret version-dependent `docker compose config` JSON as a second deploy
schema. Its canonical enforcement remains the artifact/path allowlist plus
hard-coded build command, selected service set, runtime-secret preparation and
health acceptance. Post-apply acceptance also requires
`database=ready`; a first-rollout failure removes only the candidate EDP
container without volumes and restores the source overlay. It never contains a
provider credential, provider endpoint,
collection schedule or command
transport. Its contract payload is an exact provider-neutral runtime subset;
`providers/*`, mappings, fixtures and tests are excluded and do not trigger an
EDP rebuild. Database credentials remain root-owned live `.env.synology`
configuration and must not reuse `NODEDC_INTERNAL_ACCESS_TOKEN`. configuration and must not reuse `NODEDC_INTERNAL_ACCESS_TOKEN`.
The External Data Plane writer-provisioner credential is different: on the On the first relevant Platform apply, the root-owned runner creates
first relevant Platform apply, the root-owned runner creates
`/volume1/docker/nodedc-platform/secrets/external-data-plane-provisioner/token` `/volume1/docker/nodedc-platform/secrets/external-data-plane-provisioner/token`
atomically in a dedicated UID/GID `11006` directory (directory `0500`, token atomically in a dedicated UID/GID `11006` directory (directory `0500`, token
`0400`). It is never an `.env` value or an artifact member; Compose mounts it `0400`). It is never an `.env` value or an artifact member and is mounted
read-only only into External Data Plane and, when implemented, its dedicated read-only only into External Data Plane.
Engine provisioner running under the same restricted identity.
Manual one-time binding issuance and digest-only managed writer ensure are
independently disabled by default. The target control-plane operation generates
and stores the capability inside native NDC L2 Credentials and sends only its
digest to EDP; users and MCP consumers receive only an opaque compatible
reference/status. The runner-owned bearer above authenticates only legacy
plaintext issuance and is intentionally rejected by managed ensure/revoke.
Managed requests use a deployment Ed25519 Engine service key; EDP mounts only
the public-key trust directory read-only. On every reviewed Engine apply and on
an EDP runtime apply, this runner creates or validates one matching Ed25519 pair:
the Engine-only private key is `root:root 0400`, while the EDP trust copy is
`root:11006 0440`. Public-only crash state, key mismatch, a non-Ed25519 key,
symlinks and permissive modes fail closed. `plan` discloses both paths without
printing key material. The private key must not be broadened into an L2 graph,
MCP surface, artifact or shared-token boundary.
The reviewed Engine source candidate has dedicated server-derived MCP
plan/apply handling for the exact `ndcDataProductWriterApi` + Data Product
Publish tuple. It is separate from the generic HTTP safe-ref path and accepts no
caller-provided provider/scope/credential identity, capability, generation or
service URL. The production managed flag remains false during staging. In the
explicitly confirmed activation window it is set to true immediately before
the Platform EDP artifact; runner acceptance then requires EDP `/healthz` to
report managed provisioning `enabled`. At that point the old Engine still has
no signer mount, so the endpoint remains usable only after the separately
accepted Engine artifact. Root/UI transfer is emergency
self-hosted diagnostics only, not the user journey or acceptance
path. A provider-specific daemon remains forbidden.
Build the narrow Engine source artifact with:
```bash
node infra/deploy-runner/build-engine-data-product-publish-grant-artifact.mjs \
engine-data-product-publish-grant-YYYYMMDD-NNN
```
The builder includes exactly the pinned provider-security catalog, the
Publish-grant service, Engine Agent scope/gateway wiring, the existing n8n
adapter and the reviewed additive backend runtime overlay. It deliberately
excludes base Compose, frontend/dist, runtime data, tests, native credentials,
the NDC L2 process and the legacy generic credential-sink route/core. The
runner fixes the runtime action to `nodedc-backend` with `--no-deps` and
`--pull never`; n8n, nginx app, database services and volumes are not selected.
Acceptance requires backend health, the active immutable backend identity and
the exact additive mount inventory. Any failure restores the touched source and
recreates only the previous verified backend runtime.
Existing Engine Agents created before Publish-grant support store the original
nine scopes as the complete developer profile. They must not require a new
agent, setup command or device credential when the profile gains a server-owned
capability. Build the compatibility artifact that introduces the durable named
`full-developer` profile with:
```bash
node infra/deploy-runner/build-engine-agent-full-grant-migration-artifact.mjs \
engine-agent-full-grant-migration-YYYYMMDD-NNN
```
This follow-up slice contains exactly
`nodedc-source/server/engineAgents/store.js`. The runner pins its predecessor to
the successfully applied Publish generation, requires the installed Publish
overlay and active immutable backend, recreates only `nodedc-backend`, and
proves the exact candidate SHA, named profile and current expanded runtime
scope view. Store schema v1 is migrated atomically to v2 only when a grant
contains the complete legacy nine-scope developer bundle. From then on the
profile name is the authorization authority and its scope list is derived on
every read, so capabilities deliberately added to `full-developer` immediately
apply to existing full grants without token or store migrations. Partial grants
become `custom` and remain exact; they are never elevated. The artifact does not
touch UI, n8n, credentials, agent tokens, workflow graphs, databases or runtime
payloads.
`module-foundry` is an independent, authenticated application component. Its `module-foundry` is an independent, authenticated application component. Its
artifact contains source and compose infrastructure only; its live artifact contains source and compose infrastructure only; its live
@ -190,12 +288,17 @@ verified; it does not alter NAS routes, VPN, DNS, or Tailscale.
Install or update the root-owned live runner on Synology: Install or update the root-owned live runner on Synology:
```bash ```bash
# First verify that no deploy process is active and state/deploy.lock is absent.
sudo install -o root -g root -m 0755 \ sudo install -o root -g root -m 0755 \
/volume1/docker/nodedc-deploy/runner-install/nodedc-deploy \ /volume1/docker/nodedc-deploy/runner-install/nodedc-deploy \
/usr/local/sbin/nodedc-deploy /usr/local/sbin/nodedc-deploy
sudo /usr/local/sbin/nodedc-deploy verify-install sudo /usr/local/sbin/nodedc-deploy verify-install
``` ```
Runner promotion is a standalone admin step, never an app-overlay artifact and
never part of a running apply. A Python process already executing the old file
keeps the old code in memory; always invoke a fresh verified process afterward.
Normal service deploys must still use explicit artifacts: Normal service deploys must still use explicit artifacts:
```bash ```bash

View File

@ -0,0 +1,121 @@
#!/usr/bin/env node
import { createHash } from 'node:crypto'
import { spawnSync } from 'node:child_process'
import { cp, lstat, mkdir, mkdtemp, readFile, rm, writeFile } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { dirname, join, resolve } from 'node:path'
import { fileURLToPath } from 'node:url'
const here = dirname(fileURLToPath(import.meta.url))
const platformRoot = resolve(here, '../..')
const workspaceRoot = resolve(platformRoot, '..')
const engineRoot = resolve(workspaceRoot, 'NODEDC_ENGINE_INFRA')
const artifactRoot = resolve(process.env.NODEDC_DEPLOY_ARTIFACT_DIR || join(workspaceRoot, 'deploy-artifacts'))
const [patchId = '', ...extra] = process.argv.slice(2)
const storeRelativePath = 'nodedc-source/server/engineAgents/store.js'
const predecessorSha256 = '52daa43499d6d9a97fe7ffa891edb9212b7791e733f91dd3ca686d42739b7e9a'
const targetSha256 = '2e62654c2dc12905efcc83a9dff45a818dd7b47924a10600160835c4416540e9'
const previouslyIssuedPatchIds = new Set([
'engine-agent-full-grant-migration-20260717-001',
])
if (extra.length || !/^engine-agent-full-grant-migration-\d{8}-\d{3}$/.test(patchId)) {
throw new Error('usage: build-engine-agent-full-grant-migration-artifact.mjs <fresh-patch-id>')
}
if (previouslyIssuedPatchIds.has(patchId)) {
throw new Error('engine_agent_full_grant_migration_patch_id_already_issued')
}
const source = join(engineRoot, storeRelativePath)
const sourceInfo = await lstat(source)
if (sourceInfo.isSymbolicLink() || !sourceInfo.isFile()) throw new Error('engine_agent_store_source_unsafe')
const sourceBytes = await readFile(source)
if (digest(sourceBytes) !== targetSha256) throw new Error('engine_agent_store_target_sha256_mismatch')
const sourceText = sourceBytes.toString('utf8')
for (const required of [
'const STORE_VERSION = 2',
"export const ENGINE_AGENT_FULL_DEVELOPER_PROFILE = 'full-developer'",
"export const ENGINE_AGENT_CUSTOM_PROFILE = 'custom'",
'const LEGACY_FULL_DEVELOPER_SCOPES = Object.freeze([',
"'engine:l2:data-product-publish-grant:plan'",
"'engine:l2:data-product-publish-grant:write'",
'const migrateLegacyFullDeveloper = sourceVersion === 1',
'LEGACY_FULL_DEVELOPER_SCOPES.every((scope) => scopes.includes(scope))',
'profile === ENGINE_AGENT_FULL_DEVELOPER_PROFILE ? [...ENGINE_AGENT_SCOPES] : scopes',
"throw new Error('engine_agent_store_version_unsupported')",
]) {
if (!sourceText.includes(required)) throw new Error(`engine_agent_store_contract_missing:${required}`)
}
if (/gelios|robot2b/i.test(sourceText)) throw new Error('engine_agent_store_provider_logic_forbidden')
run('node', ['--check', source])
await mkdir(artifactRoot, { recursive: true })
const artifact = join(artifactRoot, `nodedc-${patchId}.tgz`)
await assertFresh(artifact)
const stage = await mkdtemp(join(tmpdir(), 'nodedc-engine-agent-grant-migration-'))
try {
const destination = join(stage, 'payload', storeRelativePath)
await mkdir(dirname(destination), { recursive: true })
await cp(source, destination, { force: false, verbatimSymlinks: true })
await writeFile(
join(stage, 'manifest.env'),
`id=${patchId}\ncomponent=engine\ntype=app-overlay\n`,
'utf8',
)
await writeFile(join(stage, 'files.txt'), `${storeRelativePath}\n`, 'utf8')
run('python3', ['-c', canonicalTarScript(), artifact, stage])
const artifactBytes = await readFile(artifact)
console.log(JSON.stringify({
ok: true,
patchId,
artifact,
sha256: digest(artifactBytes),
entries: [storeRelativePath],
predecessorSha256,
targetSha256,
services: ['nodedc-backend'],
excluded: ['n8n', 'app', 'databases', 'credentials', 'runtime-data'],
}, null, 2))
} finally {
await rm(stage, { recursive: true, force: true })
}
async function assertFresh(target) {
try {
await lstat(target)
} catch (error) {
if (error?.code === 'ENOENT') return
throw error
}
throw new Error('engine_agent_full_grant_migration_artifact_already_exists')
}
function canonicalTarScript() {
return [
'import gzip, io, pathlib, sys, tarfile',
'root=pathlib.Path(sys.argv[2])',
"with open(sys.argv[1], 'wb') as out:",
" with gzip.GzipFile(filename='', mode='wb', fileobj=out, compresslevel=9, mtime=0) as gz:",
" with tarfile.open(fileobj=gz, mode='w', format=tarfile.PAX_FORMAT) as tar:",
" for top in ('manifest.env','files.txt','payload'):",
" p=root/top; paths=[p] + (sorted(p.rglob('*')) if p.is_dir() else [])",
' for x in paths:',
" info=tar.gettarinfo(str(x), arcname=x.relative_to(root).as_posix())",
" info.uid=info.gid=0; info.uname=info.gname='root'; info.mtime=0; info.mode=0o755 if info.isdir() else 0o644",
" with (open(x,'rb') if info.isfile() else io.BytesIO()) as src: tar.addfile(info, src if info.isfile() else None)",
].join('\n')
}
function run(command, args) {
const result = spawnSync(command, args, {
encoding: 'utf8',
stdio: ['ignore', 'pipe', 'pipe'],
maxBuffer: 16 * 1024 * 1024,
})
if (result.status !== 0) throw new Error(`${command}_failed:${result.stderr || result.stdout}`)
return result
}
function digest(bytes) {
return createHash('sha256').update(bytes).digest('hex')
}

View File

@ -0,0 +1,156 @@
#!/usr/bin/env node
import { createHash } from 'node:crypto'
import { spawnSync } from 'node:child_process'
import { lstat, mkdir, mkdtemp, readFile, rm, writeFile } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { dirname, join, resolve } from 'node:path'
import { fileURLToPath } from 'node:url'
const here = dirname(fileURLToPath(import.meta.url))
const platformRoot = resolve(here, '../..')
const workspaceRoot = resolve(platformRoot, '..')
const artifactRoot = resolve(process.env.NODEDC_DEPLOY_ARTIFACT_DIR || join(workspaceRoot, 'deploy-artifacts'))
const sourceArtifact = resolve(
process.env.NODEDC_ENGINE_CREDENTIAL_SINK_RECOVERY_SOURCE
|| '/Volumes/docker/nodedc-deploy/applied/nodedc-engine-credential-sink-20260716-001.tgz',
)
const [patchId = '', ...extra] = process.argv.slice(2)
if (extra.length || !/^engine-credential-sink-recovery-\d{8}-\d{3}$/.test(patchId)) {
throw new Error('usage: build-engine-credential-sink-recovery-artifact.mjs <fresh-recovery-patch-id>')
}
const sourceArtifactSha256 = '0a96add05fe59db8f490927f66e07a84490474a7afb3ef7de51e1d6fd96f86a2'
const sourceManifest = 'id=engine-credential-sink-20260716-001\ncomponent=engine\ntype=app-overlay\n'
const entries = Object.freeze([
'nodedc-source/server/credentialPolicies/ndcPrivateNode.js',
'nodedc-source/server/credentialSink',
'nodedc-source/server/index.js',
'nodedc-source/server/routes/engineAgentGateway.js',
'nodedc-source/server/routes/engineCredentialSink.js',
'nodedc-source/server/routes/n8n.js',
'nodedc-source/server/routes/ndcAgentMcp.js',
'nodedc-source/services/backend/credential-sink/docker-compose.immutable-runtime.yml',
])
const payloadSha256 = Object.freeze({
'nodedc-source/server/credentialPolicies/ndcPrivateNode.js': '723874a02dc7b8a68b22ff2304431cfe64f28933cedf5f1e6cda79b2e1cf704a',
'nodedc-source/server/credentialSink/core.js': '9f0facc41fd398fcd955cffdd486abb126cdcd756c87ffde667fcbe00e2c41d3',
'nodedc-source/server/credentialSink/requestAuth.js': 'f8c9237c3e6f4219dee0d4f956d6e97f76f5bd7ba8a6fd0b4fb21aceffa38138',
'nodedc-source/server/credentialSink/store.js': 'c78dc285a973b6acd8a2330f0310935ad08720b2905454492f292d235abf12c0',
'nodedc-source/server/credentialSink/vendor/engine-credential-sink.mjs': 'b4800eead9bf94793ff1280d06e34aad6b37ef055a9d7f8d83d7fb5f9bd66d8b',
'nodedc-source/server/index.js': 'b2b790b02839570d967a2ca68b00e2724485a99389ac9b3a589a1b22302a36b8',
'nodedc-source/server/routes/engineAgentGateway.js': 'e3450a4e1d5318dbac37627b67804d62804e27e2032c9e90478a1e739cea6d3d',
'nodedc-source/server/routes/engineCredentialSink.js': '9cbb69dbc8cbe6181cd5b0170fe9c4d717b0a173866ca98cba3e3766c0bab94e',
'nodedc-source/server/routes/n8n.js': '783d822e2457d82e890f43bc00c7e33822077dc2841f0511a89ecb210fd36d48',
'nodedc-source/server/routes/ndcAgentMcp.js': 'fbb3342b1a617b956d5b3a6a40d5111aa107c1176b4ff89c37b104995bada081',
'nodedc-source/services/backend/credential-sink/docker-compose.immutable-runtime.yml': '944fa64b08255eb8207b93fd327aebb98ecd9400d37d25fcfa8e3a040ee44afe',
})
const sourceInfo = await lstat(sourceArtifact)
if (sourceInfo.isSymbolicLink() || !sourceInfo.isFile()) {
throw new Error('engine_credential_sink_recovery_source_unsafe')
}
if (digest(await readFile(sourceArtifact)) !== sourceArtifactSha256) {
throw new Error('engine_credential_sink_recovery_source_sha256_mismatch')
}
await mkdir(artifactRoot, { recursive: true })
const artifact = join(artifactRoot, `nodedc-${patchId}.tgz`)
await assertArtifactTargetFresh(artifact)
const stage = await mkdtemp(join(tmpdir(), 'nodedc-engine-credential-sink-recovery-'))
try {
run('python3', [
'-c', verifiedExtractScript(), sourceArtifact, stage,
sourceArtifactSha256, sourceManifest, JSON.stringify(entries), JSON.stringify(payloadSha256),
])
await writeFile(
join(stage, 'manifest.env'),
`id=${patchId}\ncomponent=engine\ntype=app-overlay\n`,
'utf8',
)
await writeFile(join(stage, 'files.txt'), `${entries.join('\n')}\n`, 'utf8')
run('python3', ['-c', canonicalTarScript(), artifact, stage])
console.log(JSON.stringify({
ok: true,
patchId,
artifact,
sha256: digest(await readFile(artifact)),
sourceArtifact,
sourceArtifactSha256,
entries,
payloadSha256,
changed: ['manifest.env:id'],
}, null, 2))
} finally {
await rm(stage, { recursive: true, force: true })
}
async function assertArtifactTargetFresh(target) {
try {
await lstat(target)
} catch (error) {
if (error?.code === 'ENOENT') return
throw error
}
throw new Error('engine_credential_sink_recovery_artifact_already_exists')
}
function verifiedExtractScript() {
return [
'import hashlib,json,pathlib,sys,tarfile',
'source=pathlib.Path(sys.argv[1]); stage=pathlib.Path(sys.argv[2])',
'expected_archive=sys.argv[3]; expected_manifest=sys.argv[4].encode()',
'entries=json.loads(sys.argv[5]); hashes=json.loads(sys.argv[6])',
"raw=source.read_bytes()",
"assert hashlib.sha256(raw).hexdigest()==expected_archive, 'source-sha256'",
"expected_files={'manifest.env','files.txt'}|{'payload/'+name for name in hashes}",
"expected_dirs={'payload'}",
"for name in hashes:",
" p=pathlib.PurePosixPath('payload/'+name)",
" expected_dirs.update(str(parent) for parent in p.parents if str(parent)!='.')",
"with tarfile.open(source,'r:gz') as tar:",
" members=tar.getmembers(); names={member.name for member in members}",
" assert names==expected_files|expected_dirs, 'source-member-set'",
" for member in members:",
" p=pathlib.PurePosixPath(member.name)",
" assert not p.is_absolute() and '..' not in p.parts and '\\\\' not in member.name, 'unsafe-member'",
" assert member.isdir() if member.name in expected_dirs else member.isfile(), 'unsafe-member-type'",
" assert tar.extractfile('manifest.env').read()==expected_manifest, 'source-manifest'",
" expected_list=('\\n'.join(entries)+'\\n').encode()",
" assert tar.extractfile('files.txt').read()==expected_list, 'source-files-list'",
" for name,wanted in hashes.items():",
" data=tar.extractfile('payload/'+name).read()",
" assert hashlib.sha256(data).hexdigest()==wanted, 'payload-sha256:'+name",
" target=stage/'payload'/name; target.parent.mkdir(parents=True,exist_ok=True); target.write_bytes(data)",
].join('\n')
}
function canonicalTarScript() {
return [
'import gzip,io,pathlib,sys,tarfile',
'root=pathlib.Path(sys.argv[2])',
"with open(sys.argv[1],'wb') as out:",
" with gzip.GzipFile(filename='',mode='wb',fileobj=out,compresslevel=9,mtime=0) as gz:",
" with tarfile.open(fileobj=gz,mode='w',format=tarfile.PAX_FORMAT) as tar:",
" for top in ('manifest.env','files.txt','payload'):",
" p=root/top; paths=[p]+(sorted(p.rglob('*')) if p.is_dir() else [])",
" for x in paths:",
" info=tar.gettarinfo(str(x),arcname=x.relative_to(root).as_posix())",
" info.uid=info.gid=0; info.uname=info.gname='root'; info.mtime=0; info.mode=0o755 if info.isdir() else 0o644",
" with (open(x,'rb') if info.isfile() else io.BytesIO()) as src: tar.addfile(info,src if info.isfile() else None)",
].join('\n')
}
function run(command, args) {
const result = spawnSync(command, args, {
encoding: 'utf8',
stdio: ['ignore', 'pipe', 'pipe'],
maxBuffer: 128 * 1024 * 1024,
})
if (result.status !== 0) throw new Error(`${command}_failed:${result.stderr || result.stdout}`)
}
function digest(bytes) {
return createHash('sha256').update(bytes).digest('hex')
}

View File

@ -0,0 +1,252 @@
#!/usr/bin/env node
import { createHash } from 'node:crypto'
import { spawnSync } from 'node:child_process'
import { cp, lstat, mkdir, mkdtemp, readFile, readdir, rm, writeFile } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { dirname, join, relative, resolve } from 'node:path'
import { fileURLToPath } from 'node:url'
const here = dirname(fileURLToPath(import.meta.url))
const platformRoot = resolve(here, '../..')
const workspaceRoot = resolve(platformRoot, '..')
const engineRoot = resolve(workspaceRoot, 'NODEDC_ENGINE_INFRA')
const artifactRoot = resolve(process.env.NODEDC_DEPLOY_ARTIFACT_DIR || join(workspaceRoot, 'deploy-artifacts'))
const [patchId = '', ...extra] = process.argv.slice(2)
const previouslyIssuedPatchIds = new Set([
'engine-data-product-publish-grant-20260716-001',
'engine-data-product-publish-grant-20260717-001',
'engine-data-product-publish-grant-20260717-002',
])
if (extra.length || !/^engine-data-product-publish-grant-\d{8}-\d{3}$/.test(patchId)) {
throw new Error('usage: build-engine-data-product-publish-grant-artifact.mjs <fresh-patch-id>')
}
if (previouslyIssuedPatchIds.has(patchId)) throw new Error('engine_publish_grant_patch_id_already_issued')
// Deliberately exclude frontend/dist, runtime data, tests, every credential
// value, and the already-installed credential-sink implementation. The sink
// remains mounted and byte-frozen as the predecessor domain; this artifact
// adds the exact Data Product Publish grant flow beside it.
const entries = Object.freeze([
'nodedc-source/server/assets/provider-packages/v1/catalog.json',
'nodedc-source/server/dataProductPublishGrant',
'nodedc-source/server/engineAgents/store.js',
'nodedc-source/server/routes/engineAgentGateway.js',
'nodedc-source/server/routes/n8n.js',
'nodedc-source/services/backend/data-product-publish-grant/docker-compose.immutable-runtime.yml',
])
const ignoredBasenames = new Set(['.DS_Store', '.git', 'node_modules'])
const credentialSinkPredecessorSha256 = Object.freeze({
'docker-compose.yml': '258cebb64ff1943c939655cc55bdce00fc5c4dced67ec291d84d6df066ace50e',
'nodedc-source/server/credentialPolicies/ndcPrivateNode.js': '723874a02dc7b8a68b22ff2304431cfe64f28933cedf5f1e6cda79b2e1cf704a',
'nodedc-source/server/credentialSink/core.js': '9f0facc41fd398fcd955cffdd486abb126cdcd756c87ffde667fcbe00e2c41d3',
'nodedc-source/server/credentialSink/requestAuth.js': 'f8c9237c3e6f4219dee0d4f956d6e97f76f5bd7ba8a6fd0b4fb21aceffa38138',
'nodedc-source/server/credentialSink/store.js': 'c78dc285a973b6acd8a2330f0310935ad08720b2905454492f292d235abf12c0',
'nodedc-source/server/credentialSink/vendor/engine-credential-sink.mjs': 'b4800eead9bf94793ff1280d06e34aad6b37ef055a9d7f8d83d7fb5f9bd66d8b',
'nodedc-source/server/index.js': 'b2b790b02839570d967a2ca68b00e2724485a99389ac9b3a589a1b22302a36b8',
'nodedc-source/server/routes/engineCredentialSink.js': '9cbb69dbc8cbe6181cd5b0170fe9c4d717b0a173866ca98cba3e3766c0bab94e',
'nodedc-source/server/routes/ndcAgentMcp.js': 'fbb3342b1a617b956d5b3a6a40d5111aa107c1176b4ff89c37b104995bada081',
'nodedc-source/services/backend/credential-sink/docker-compose.immutable-runtime.yml': '944fa64b08255eb8207b93fd327aebb98ecd9400d37d25fcfa8e3a040ee44afe',
})
const publishGrantRuntimeOverride = [
'services:',
' nodedc-backend:',
' user: "0:0"',
' environment:',
' ENGINE_DATA_PLANE_BASE_URL: http://external-data-plane:18106',
' ENGINE_EDP_MANAGED_PROVISIONER_PRIVATE_KEY_FILE: /run/nodedc-secrets/engine-edp-managed-provisioner-private.pem',
' ENGINE_CONTROL_PLANE_PUBLISH_GRANT_ROOT: /var/lib/nodedc-control-plane/publish-grants',
' volumes:',
' - type: bind',
' source: /volume2/nodedc-demo/nodedc-control-plane/publish-grants',
' target: /var/lib/nodedc-control-plane/publish-grants',
' bind:',
' create_host_path: false',
' - type: bind',
' source: /volume1/docker/nodedc-platform/secrets/engine-edp-managed-provisioner/private-key.pem',
' target: /run/nodedc-secrets/engine-edp-managed-provisioner-private.pem',
' read_only: true',
' bind:',
' create_host_path: false',
'',
].join('\n')
await assertSourceBoundary()
await mkdir(artifactRoot, { recursive: true })
const artifact = join(artifactRoot, `nodedc-${patchId}.tgz`)
await assertArtifactTargetFresh(artifact)
const stage = await mkdtemp(join(tmpdir(), 'nodedc-engine-publish-grant-artifact-'))
const payload = join(stage, 'payload')
try {
await mkdir(payload, { recursive: true })
for (const entry of entries) {
await copySafe(resolve(engineRoot, entry), join(payload, entry))
}
await writeFile(
join(stage, 'manifest.env'),
`id=${patchId}\ncomponent=engine\ntype=app-overlay\n`,
'utf8',
)
await writeFile(join(stage, 'files.txt'), `${entries.join('\n')}\n`, 'utf8')
run('python3', ['-c', canonicalTarScript(), artifact, stage])
const sha256 = digest(await readFile(artifact))
console.log(JSON.stringify({
ok: true,
patchId,
artifact,
sha256,
entries,
excluded: [
'docker-compose.yml',
'nodedc-source/server/index.js',
'nodedc-source/server/credentialPolicies/ndcPrivateNode.js',
'nodedc-source/server/credentialSink',
'nodedc-source/server/routes/engineCredentialSink.js',
'nodedc-source/server/middleware/demoAccess.js',
'nodedc-source/server/routes/ndcAgentMcp.js',
'nodedc-source/server/data',
'nodedc-source/dist',
'nodedc-source/server/tests',
],
preservedPredecessor: Object.keys(credentialSinkPredecessorSha256),
}, null, 2))
} finally {
await rm(stage, { recursive: true, force: true })
}
async function assertSourceBoundary() {
for (const [relativePath, expectedSha256] of Object.entries(credentialSinkPredecessorSha256)) {
const bytes = await readFile(join(engineRoot, relativePath))
if (digest(bytes) !== expectedSha256) {
throw new Error(`engine_credential_sink_predecessor_drift:${relativePath}`)
}
}
const runtimeOverridePath = join(
engineRoot,
'nodedc-source/services/backend/data-product-publish-grant/docker-compose.immutable-runtime.yml',
)
if (await readFile(runtimeOverridePath, 'utf8') !== publishGrantRuntimeOverride) {
throw new Error('engine_publish_grant_runtime_override_mismatch')
}
const indexSource = await readFile(join(engineRoot, 'nodedc-source/server/index.js'), 'utf8')
if (!indexSource.includes("app.use('/api/engine-agent-mcp', engineAgentMcpRouter)")) {
throw new Error('engine_agent_mcp_mount_missing')
}
if (
!indexSource.includes("import engineCredentialSinkRouter from './routes/engineCredentialSink.js'")
|| !indexSource.includes("app.use('/internal/engine-credential-sink', express.json({")
) throw new Error('engine_credential_sink_predecessor_mount_missing')
const gateway = await readFile(join(engineRoot, 'nodedc-source/server/routes/engineAgentGateway.js'), 'utf8')
for (const tool of [
'engine_plan_data_product_publish_grant',
'engine_apply_data_product_publish_grant',
'engine_accept_data_product_publish_grant',
'engine_rollback_data_product_publish_grant',
]) {
if (!gateway.includes(tool)) throw new Error(`engine_publish_grant_tool_missing:${tool}`)
}
const n8nRoute = await readFile(join(engineRoot, 'nodedc-source/server/routes/n8n.js'), 'utf8')
if (n8nRoute.includes("from '../credentialSink/")) {
throw new Error('engine_publish_grant_depends_on_legacy_sink')
}
if (!n8nRoute.includes('engineDataProductPublishGrantN8nAdapter')) {
throw new Error('engine_publish_grant_native_adapter_missing')
}
if (!n8nRoute.includes('engineCredentialSinkN8nAdapter')) {
throw new Error('engine_credential_sink_native_adapter_missing')
}
const grantDirectory = join(engineRoot, 'nodedc-source/server/dataProductPublishGrant')
const grantFiles = (await readdir(grantDirectory, { withFileTypes: true }))
.filter((entry) => entry.isFile())
.map((entry) => entry.name)
.sort()
const expectedGrantFiles = [
'acceptance.js',
'providerCatalog.js',
'service.js',
'signedDataPlaneClient.js',
'store.js',
]
if (JSON.stringify(grantFiles) !== JSON.stringify(expectedGrantFiles)) {
throw new Error('engine_publish_grant_source_set_mismatch')
}
for (const entry of entries) {
const source = resolve(engineRoot, entry)
const info = await lstat(source)
if (info.isSymbolicLink() || (!info.isFile() && !info.isDirectory())) {
throw new Error(`engine_artifact_source_unsafe:${entry}`)
}
}
for (const source of [
...expectedGrantFiles.map((name) => join(grantDirectory, name)),
join(engineRoot, 'nodedc-source/server/routes/engineAgentGateway.js'),
join(engineRoot, 'nodedc-source/server/routes/n8n.js'),
join(engineRoot, 'nodedc-source/server/routes/ndcAgentMcp.js'),
join(engineRoot, 'nodedc-source/server/engineAgents/store.js'),
]) run('node', ['--check', source])
}
async function assertArtifactTargetFresh(target) {
try {
await lstat(target)
} catch (error) {
if (error?.code === 'ENOENT') return
throw error
}
throw new Error('engine_publish_grant_artifact_already_exists')
}
async function copySafe(source, destination) {
const info = await lstat(source)
if (info.isSymbolicLink()) throw new Error(`source_symlink_rejected:${source}`)
if (info.isFile()) {
await mkdir(dirname(destination), { recursive: true })
await cp(source, destination, { force: false, verbatimSymlinks: true })
return
}
if (!info.isDirectory()) throw new Error(`source_type_rejected:${source}`)
await mkdir(destination, { recursive: true })
for (const entry of await readdir(source, { withFileTypes: true })) {
if (ignoredBasenames.has(entry.name) || entry.name.startsWith('.env')) continue
const childSource = join(source, entry.name)
if (entry.isSymbolicLink()) {
throw new Error(`source_symlink_rejected:${relative(engineRoot, childSource)}`)
}
await copySafe(childSource, join(destination, entry.name))
}
}
function canonicalTarScript() {
return [
'import gzip, io, pathlib, sys, tarfile',
'root=pathlib.Path(sys.argv[2])',
"with open(sys.argv[1], 'wb') as out:",
" with gzip.GzipFile(filename='', mode='wb', fileobj=out, compresslevel=9, mtime=0) as gz:",
" with tarfile.open(fileobj=gz, mode='w', format=tarfile.PAX_FORMAT) as tar:",
" for top in ('manifest.env','files.txt','payload'):",
" p=root/top; paths=[p] + (sorted(p.rglob('*')) if p.is_dir() else [])",
' for x in paths:',
" info=tar.gettarinfo(str(x), arcname=x.relative_to(root).as_posix())",
" info.uid=info.gid=0; info.uname=info.gname='root'; info.mtime=0; info.mode=0o755 if info.isdir() else 0o644",
" with (open(x,'rb') if info.isfile() else io.BytesIO()) as src: tar.addfile(info, src if info.isfile() else None)",
].join('\n')
}
function run(command, args) {
const result = spawnSync(command, args, {
encoding: 'utf8',
stdio: ['ignore', 'pipe', 'pipe'],
maxBuffer: 128 * 1024 * 1024,
})
if (result.status !== 0) throw new Error(`${command}_failed:${result.stderr || result.stdout}`)
return result
}
function digest(bytes) {
return createHash('sha256').update(bytes).digest('hex')
}

View File

@ -1,7 +1,7 @@
#!/usr/bin/env node #!/usr/bin/env node
import { createHash } from "node:crypto"; import { createHash } from "node:crypto";
import { spawnSync } from "node:child_process"; import { spawnSync } from "node:child_process";
import { cp, mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; import { cp, lstat, mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
import { createRequire, Module } from "node:module"; import { createRequire, Module } from "node:module";
import { tmpdir } from "node:os"; import { tmpdir } from "node:os";
import { dirname, join, resolve } from "node:path"; import { dirname, join, resolve } from "node:path";
@ -24,8 +24,10 @@ const n8nVersion = "2.3.2";
const baseImage = "docker.n8n.io/n8nio/n8n:2.3.2"; const baseImage = "docker.n8n.io/n8nio/n8n:2.3.2";
const architecture = "amd64"; const architecture = "amd64";
const generatedAt = "2026-07-15T21:51:41.000Z"; const generatedAt = "2026-07-15T21:51:41.000Z";
const activationId = "engine-n8n-private-extension-20260716-003"; const previouslyIssuedTransitionIds = new Set(["20260715-002", "20260716-003"]);
const rollbackId = "engine-n8n-private-extension-rollback-20260716-003"; const transitionId = readTransitionId(process.argv.slice(2), process.env.NODEDC_N8N_TRANSITION_ID);
const activationId = `engine-n8n-private-extension-${transitionId}`;
const rollbackId = `engine-n8n-private-extension-rollback-${transitionId}`;
const transitionRoot = "nodedc-source/services/n8n/private-extensions"; const transitionRoot = "nodedc-source/services/n8n/private-extensions";
const descriptorRel = `${transitionRoot}/ndc-activation.json`; const descriptorRel = `${transitionRoot}/ndc-activation.json`;
@ -62,6 +64,8 @@ const credentialModules = [
]; ];
await mkdir(artifactRoot, { recursive: true }); await mkdir(artifactRoot, { recursive: true });
await assertArtifactTargetFresh(join(artifactRoot, `nodedc-${activationId}.tgz`));
await assertArtifactTargetFresh(join(artifactRoot, `nodedc-${rollbackId}.tgz`));
assertSha(await readFile(stageArtifact), stageArtifactSha256, "staging artifact"); assertSha(await readFile(stageArtifact), stageArtifactSha256, "staging artifact");
assertEngineBaseline(await readFile(join(engineRoot, "docker-compose.yml"), "utf8")); assertEngineBaseline(await readFile(join(engineRoot, "docker-compose.yml"), "utf8"));
@ -125,13 +129,15 @@ try {
const rollbackDescriptor = descriptor("rollback-inactive", [], [], releaseId); const rollbackDescriptor = descriptor("rollback-inactive", [], [], releaseId);
const override = composeOverride(); const override = composeOverride();
await writeJson(join(engineRoot, nodesCatalogRel), activeNodes); const generatedRoot = join(work, "generated-engine-payload");
await writeJson(join(engineRoot, credentialsCatalogRel), activeCredentials); await writeJson(join(generatedRoot, nodesCatalogRel), activeNodes);
await writeJson(join(engineRoot, metaRel), activeMeta); await writeJson(join(generatedRoot, credentialsCatalogRel), activeCredentials);
await writeJson(join(engineRoot, descriptorRel), activationDescriptor); await writeJson(join(generatedRoot, metaRel), activeMeta);
await writeFile(join(engineRoot, overrideRel), override, "utf8"); await writeJson(join(generatedRoot, descriptorRel), activationDescriptor);
await cp(join(packageRoot, "dist/icons/ndc.svg"), join(engineRoot, iconRel), { force: true }); await writeFile(join(generatedRoot, overrideRel), override, "utf8");
await cp(join(packageRoot, "dist/icons/ndc.dark.svg"), join(engineRoot, darkIconRel), { force: true }); await mkdir(join(generatedRoot, iconRoot), { recursive: true });
await cp(join(packageRoot, "dist/icons/ndc.svg"), join(generatedRoot, iconRel), { force: false });
await cp(join(packageRoot, "dist/icons/ndc.dark.svg"), join(generatedRoot, darkIconRel), { force: false });
const activationEntries = [ const activationEntries = [
descriptorRel, descriptorRel,
@ -144,7 +150,7 @@ try {
]; ];
const activationArtifact = await buildArtifact(work, activationId, activationEntries, async (payload) => { const activationArtifact = await buildArtifact(work, activationId, activationEntries, async (payload) => {
for (const rel of activationEntries) { for (const rel of activationEntries) {
await cp(join(engineRoot, rel), join(payload, rel), { recursive: true, force: false }); await cp(join(generatedRoot, rel), join(payload, rel), { recursive: true, force: false });
} }
}); });
@ -159,6 +165,7 @@ try {
console.log(JSON.stringify({ console.log(JSON.stringify({
ok: true, ok: true,
transitionId,
releaseId, releaseId,
packageSha256, packageSha256,
nodeTypes: expectedNodeTypes, nodeTypes: expectedNodeTypes,
@ -192,6 +199,42 @@ function descriptor(action, nodeTypes, credentialTypes, expectedCurrent) {
}; };
} }
function readTransitionId(args, environmentValue) {
if (args.length > 1) throw new Error("transition_id_argument_count_invalid");
const argumentValue = args[0] || "";
const envValue = String(environmentValue || "").trim();
if (argumentValue && envValue && argumentValue !== envValue) {
throw new Error("transition_id_sources_conflict");
}
const value = argumentValue || envValue;
if (!value) throw new Error("transition_id_required");
const match = /^(\d{4})(\d{2})(\d{2})-([0-9]{3})$/.exec(value);
if (!match || match[4] === "000") throw new Error("transition_id_invalid");
const year = Number(match[1]);
const month = Number(match[2]);
const day = Number(match[3]);
const parsed = new Date(Date.UTC(year, month - 1, day));
if (parsed.getUTCFullYear() !== year
|| parsed.getUTCMonth() !== month - 1
|| parsed.getUTCDate() !== day) {
throw new Error("transition_id_invalid");
}
if (previouslyIssuedTransitionIds.has(value)) {
throw new Error("transition_id_already_issued");
}
return value;
}
async function assertArtifactTargetFresh(path) {
try {
await lstat(path);
} catch (error) {
if (error?.code === "ENOENT") return;
throw error;
}
throw new Error("transition_artifact_already_exists");
}
function composeOverride() { function composeOverride() {
const health = "const http=require('http');const req=http.get('http://127.0.0.1:5678/healthz/readiness',r=>{r.resume();process.exit(r.statusCode===200?0:1)});req.on('error',()=>process.exit(1));req.setTimeout(4000,()=>{req.destroy();process.exit(1)});"; const health = "const http=require('http');const req=http.get('http://127.0.0.1:5678/healthz/readiness',r=>{r.resume();process.exit(r.statusCode===200?0:1)});req.on('error',()=>process.exit(1));req.setTimeout(4000,()=>{req.destroy();process.exit(1)});";
return [ return [

View File

@ -1,14 +1,14 @@
#!/usr/bin/env node #!/usr/bin/env node
import { createHash } from "node:crypto"; import { createHash } from "node:crypto";
import { spawnSync } from "node:child_process"; import { spawnSync } from "node:child_process";
import { cp, lstat, mkdir, mkdtemp, readdir, rm, writeFile } from "node:fs/promises"; import { cp, lstat, mkdir, mkdtemp, readFile, readdir, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os"; import { tmpdir } from "node:os";
import { dirname, join, relative, resolve } from "node:path"; import { dirname, join, relative, resolve } from "node:path";
import { fileURLToPath } from "node:url"; import { fileURLToPath } from "node:url";
const scriptDir = dirname(fileURLToPath(import.meta.url)); const scriptDir = dirname(fileURLToPath(import.meta.url));
const platformRoot = resolve(scriptDir, "../.."); const platformRoot = resolve(scriptDir, "../..");
const artifactDir = resolve(scriptDir, "../deploy-artifacts"); const artifactDir = resolve(process.env.NODEDC_DEPLOY_ARTIFACT_DIR || resolve(scriptDir, "../deploy-artifacts"));
const [patchId = "external-data-plane-20260714-001", ...extra] = process.argv.slice(2); const [patchId = "external-data-plane-20260714-001", ...extra] = process.argv.slice(2);
if (extra.length || !/^[A-Za-z0-9._-]{1,96}$/.test(patchId)) { if (extra.length || !/^[A-Za-z0-9._-]{1,96}$/.test(patchId)) {
@ -18,13 +18,21 @@ if (extra.length || !/^[A-Za-z0-9._-]{1,96}$/.test(patchId)) {
const files = [ const files = [
["infra/synology/docker-compose.external-data-plane.yml", "platform/docker-compose.external-data-plane.yml"], ["infra/synology/docker-compose.external-data-plane.yml", "platform/docker-compose.external-data-plane.yml"],
["services/external-data-plane", "platform/services/external-data-plane"], ["services/external-data-plane", "platform/services/external-data-plane"],
["packages/external-provider-contract", "platform/packages/external-provider-contract"], ["packages/external-provider-contract/package.json", "platform/packages/external-provider-contract/package.json"],
["packages/external-provider-contract/src/contract-version.mjs", "platform/packages/external-provider-contract/src/contract-version.mjs"],
["packages/external-provider-contract/src/data-plane.mjs", "platform/packages/external-provider-contract/src/data-plane.mjs"],
["packages/external-provider-contract/src/data-product.mjs", "platform/packages/external-provider-contract/src/data-product.mjs"],
["packages/external-provider-contract/src/intake-batch.mjs", "platform/packages/external-provider-contract/src/intake-batch.mjs"],
["packages/external-provider-contract/src/sensitive-field-policy.mjs", "platform/packages/external-provider-contract/src/sensitive-field-policy.mjs"],
]; ];
const ignoredBasenames = new Set([".DS_Store", ".git", "node_modules"]); const ignoredBasenames = new Set([".DS_Store", ".git", "node_modules"]);
const ignoredDirectoryNames = new Set(["test"]);
const stage = await mkdtemp(join(tmpdir(), "nodedc-external-data-plane-artifact-")); const stage = await mkdtemp(join(tmpdir(), "nodedc-external-data-plane-artifact-"));
const payload = join(stage, "payload"); const payload = join(stage, "payload");
const target = join(artifactDir, `nodedc-platform-${patchId}.tgz`); const target = join(artifactDir, `nodedc-platform-${patchId}.tgz`);
await assertSourceBoundary();
try { try {
await mkdir(payload, { recursive: true }); await mkdir(payload, { recursive: true });
for (const [sourceRelative, destinationRelative] of files) { for (const [sourceRelative, destinationRelative] of files) {
@ -34,19 +42,66 @@ try {
await writeFile(join(stage, "files.txt"), `${files.map(([, destination]) => destination).join("\n")}\n`, "utf8"); await writeFile(join(stage, "files.txt"), `${files.map(([, destination]) => destination).join("\n")}\n`, "utf8");
await mkdir(artifactDir, { recursive: true }); await mkdir(artifactDir, { recursive: true });
const tar = spawnSync("python3", ["-c", [ const tar = spawnSync("python3", ["-c", canonicalTarScript(), target, stage], {
"import sys, tarfile", encoding: "utf8",
"with tarfile.open(sys.argv[1], 'w:gz', format=tarfile.PAX_FORMAT) as archive:", maxBuffer: 128 * 1024 * 1024,
" [archive.add(name, arcname=name, recursive=True) for name in ('manifest.env', 'files.txt', 'payload')]", });
].join("\n"), target], { cwd: stage, encoding: "utf8" });
if (tar.status !== 0) throw new Error(`tar_failed:${tar.stderr || tar.stdout}`); if (tar.status !== 0) throw new Error(`tar_failed:${tar.stderr || tar.stdout}`);
const digest = createHash("sha256").update(await (await import("node:fs/promises")).readFile(target)).digest("hex"); const digest = createHash("sha256").update(await readFile(target)).digest("hex");
console.log(JSON.stringify({ ok: true, patchId, artifact: target, sha256: digest }, null, 2)); console.log(JSON.stringify({
ok: true,
patchId,
artifact: target,
sha256: digest,
entries: files.map(([, destination]) => destination),
excluded: [
".env*",
"node_modules",
"services/external-data-plane/test",
"private-key.pem",
"secrets",
],
}, null, 2));
} finally { } finally {
await rm(stage, { recursive: true, force: true }); await rm(stage, { recursive: true, force: true });
} }
async function assertSourceBoundary() {
const compose = await readFile(
resolve(platformRoot, "infra/synology/docker-compose.external-data-plane.yml"),
"utf8",
);
for (const fragment of [
"source: /volume1/docker/nodedc-platform/trust/engine-managed-provisioner",
"target: /run/nodedc-trust/engine-managed-provisioner",
"EXTERNAL_DATA_PLANE_MANAGED_PROVISIONER_PUBLIC_KEY_FILE: /run/nodedc-trust/engine-managed-provisioner/public-key.pem",
"create_host_path: false",
]) {
if (!compose.includes(fragment)) throw new Error(`platform_compose_boundary_missing:${fragment}`);
}
if (
compose.includes("private-key.pem")
|| compose.includes("ENGINE_EDP_MANAGED_PROVISIONER_PRIVATE_KEY_FILE")
) throw new Error("platform_artifact_private_key_boundary_violation");
}
function canonicalTarScript() {
return [
"import gzip, io, pathlib, sys, tarfile",
"root=pathlib.Path(sys.argv[2])",
"with open(sys.argv[1], 'wb') as out:",
" with gzip.GzipFile(filename='', mode='wb', fileobj=out, compresslevel=9, mtime=0) as gz:",
" with tarfile.open(fileobj=gz, mode='w', format=tarfile.PAX_FORMAT) as tar:",
" for top in ('manifest.env','files.txt','payload'):",
" p=root/top; paths=[p] + (sorted(p.rglob('*')) if p.is_dir() else [])",
" for x in paths:",
" info=tar.gettarinfo(str(x), arcname=x.relative_to(root).as_posix())",
" info.uid=info.gid=0; info.uname=info.gname='root'; info.mtime=0; info.mode=0o755 if info.isdir() else 0o644",
" with (open(x,'rb') if info.isfile() else io.BytesIO()) as src: tar.addfile(info, src if info.isfile() else None)",
].join("\n");
}
async function copySafe(source, destination) { async function copySafe(source, destination) {
const sourceStat = await lstat(source); const sourceStat = await lstat(source);
if (sourceStat.isSymbolicLink()) throw new Error(`source_symlink_rejected:${source}`); if (sourceStat.isSymbolicLink()) throw new Error(`source_symlink_rejected:${source}`);
@ -60,6 +115,7 @@ async function copySafe(source, destination) {
await mkdir(destination, { recursive: true }); await mkdir(destination, { recursive: true });
for (const entry of await readdir(source, { withFileTypes: true })) { for (const entry of await readdir(source, { withFileTypes: true })) {
if (ignoredBasenames.has(entry.name) || entry.name.startsWith(".env")) continue; if (ignoredBasenames.has(entry.name) || entry.name.startsWith(".env")) continue;
if (entry.isDirectory() && ignoredDirectoryNames.has(entry.name)) continue;
const childSource = join(source, entry.name); const childSource = join(source, entry.name);
const childDestination = join(destination, entry.name); const childDestination = join(destination, entry.name);
if (entry.isSymbolicLink()) throw new Error(`source_symlink_rejected:${relative(platformRoot, childSource)}`); if (entry.isSymbolicLink()) throw new Error(`source_symlink_rejected:${relative(platformRoot, childSource)}`);

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,189 @@
#!/usr/bin/env python3
import hashlib
import importlib.machinery
import importlib.util
import json
import os
import subprocess
import tarfile
import tempfile
import unittest
from pathlib import Path
from unittest import mock
SCRIPT_DIR = Path(__file__).resolve().parent
BUILDER = SCRIPT_DIR / "build-engine-agent-full-grant-migration-artifact.mjs"
RUNNER_PATH = SCRIPT_DIR / "nodedc-deploy"
PATCH_ID = "engine-agent-full-grant-migration-20990101-001"
STORE_REL = "nodedc-source/server/engineAgents/store.js"
PREDECESSOR_SHA256 = "52daa43499d6d9a97fe7ffa891edb9212b7791e733f91dd3ca686d42739b7e9a"
TARGET_SHA256 = "2e62654c2dc12905efcc83a9dff45a818dd7b47924a10600160835c4416540e9"
def load_runner():
loader = importlib.machinery.SourceFileLoader(
"nodedc_agent_grant_migration_runner_under_test",
str(RUNNER_PATH),
)
spec = importlib.util.spec_from_loader(loader.name, loader)
module = importlib.util.module_from_spec(spec)
loader.exec_module(module)
return module
RUNNER = load_runner()
class EngineAgentFullGrantMigrationArtifactTest(unittest.TestCase):
def build(self, artifact_dir, patch_id=PATCH_ID):
environment = os.environ.copy()
environment["NODEDC_DEPLOY_ARTIFACT_DIR"] = str(artifact_dir)
result = subprocess.run(
["node", str(BUILDER), patch_id],
check=True,
capture_output=True,
text=True,
env=environment,
)
return json.loads(result.stdout)
def test_artifact_is_exact_reproducible_and_runner_accepted(self):
with tempfile.TemporaryDirectory(prefix="nodedc-agent-grant-migration-") as directory:
root = Path(directory)
first_dir = root / "first"
second_dir = root / "second"
first_dir.mkdir()
second_dir.mkdir()
first = self.build(first_dir)
second = self.build(second_dir)
first_artifact = Path(first["artifact"])
second_artifact = Path(second["artifact"])
self.assertEqual(first["entries"], [STORE_REL])
self.assertEqual(first["predecessorSha256"], PREDECESSOR_SHA256)
self.assertEqual(first["targetSha256"], TARGET_SHA256)
self.assertEqual(first["sha256"], hashlib.sha256(first_artifact.read_bytes()).hexdigest())
self.assertEqual(first["sha256"], second["sha256"])
self.assertEqual(first_artifact.read_bytes(), second_artifact.read_bytes())
with tarfile.open(first_artifact, "r:gz") as archive:
members = archive.getmembers()
names = {member.name for member in members}
manifest = archive.extractfile("manifest.env").read().decode("utf-8")
files = archive.extractfile("files.txt").read().decode("utf-8")
store = archive.extractfile(f"payload/{STORE_REL}").read()
self.assertEqual(
manifest,
f"id={PATCH_ID}\ncomponent=engine\ntype=app-overlay\n",
)
self.assertEqual(files, f"{STORE_REL}\n")
self.assertEqual(hashlib.sha256(store).hexdigest(), TARGET_SHA256)
self.assertFalse(any(member.issym() or member.islnk() for member in members))
self.assertFalse(any(
name.startswith("payload/nodedc-source/server/data/")
or name.startswith("payload/nodedc-source/dist/")
or name.startswith("payload/nodedc-source/server/credentialSink/")
for name in names
))
with tempfile.TemporaryDirectory(prefix="nodedc-agent-grant-runner-") as work:
manifest_value, entries, _payload = RUNNER.load_artifact(
first_artifact,
Path(work),
)
self.assertEqual(manifest_value["id"], PATCH_ID)
self.assertEqual(tuple(entries), RUNNER.ENGINE_AGENT_FULL_GRANT_MIGRATION_ARTIFACT_ENTRIES)
self.assertTrue(RUNNER.is_engine_agent_full_grant_migration_slice("engine", entries))
self.assertEqual(RUNNER.component_services("engine", entries), ("nodedc-backend",))
self.assertEqual(RUNNER.component_builds("engine", entries), ())
self.assertEqual(
RUNNER.component_healthchecks("engine", entries, ("nodedc-backend",)),
(
"http://127.0.0.1:3001/health",
"http://127.0.0.1:3001/internal/engine-credential-sink/v1/health",
),
)
def test_predecessor_and_runtime_acceptance_are_exact(self):
with tempfile.TemporaryDirectory(prefix="nodedc-agent-grant-predecessor-") as directory:
root = Path(directory)
store = root / STORE_REL
store.parent.mkdir(parents=True)
store.write_text("predecessor", encoding="utf-8")
override = root / RUNNER.ENGINE_DATA_PRODUCT_PUBLISH_GRANT_OVERRIDE_REL
override.parent.mkdir(parents=True)
override.write_text(
RUNNER.expected_engine_data_product_publish_grant_override(),
encoding="utf-8",
)
original_root = RUNNER.COMPONENTS["engine"]["payload_root"]
RUNNER.COMPONENTS["engine"]["payload_root"] = root
try:
with mock.patch.object(RUNNER, "sha256_file", return_value=PREDECESSOR_SHA256):
self.assertEqual(
RUNNER.preflight_engine_agent_full_grant_migration_predecessor(),
PREDECESSOR_SHA256,
)
with mock.patch.object(RUNNER, "sha256_file", return_value="0" * 64):
with self.assertRaisesRegex(RUNNER.DeployError, "predecessor drift"):
RUNNER.preflight_engine_agent_full_grant_migration_predecessor()
finally:
RUNNER.COMPONENTS["engine"]["payload_root"] = original_root
expected = f"store-v2-full-developer:{TARGET_SHA256}"
with mock.patch.object(
RUNNER,
"engine_backend_container_id",
return_value="container-id",
):
with mock.patch.object(
RUNNER,
"run_engine_backend_probe",
return_value=expected,
) as probe:
self.assertEqual(RUNNER.accept_engine_agent_full_grant_migration_runtime(), expected)
arguments, label = probe.call_args.args[:2]
self.assertEqual(label, "agent full grant migration")
self.assertEqual(arguments[0:2], ("node", "--input-type=module"))
self.assertIn(TARGET_SHA256, arguments)
self.assertEqual(probe.call_args.kwargs["container_id"], "container-id")
def test_builder_rejects_invalid_id_and_overwrite(self):
with tempfile.TemporaryDirectory(prefix="nodedc-agent-grant-negative-") as directory:
root = Path(directory)
environment = {**os.environ, "NODEDC_DEPLOY_ARTIFACT_DIR": str(root)}
invalid = subprocess.run(
["node", str(BUILDER), "engine-data-product-publish-grant-20990101-001"],
check=False,
capture_output=True,
text=True,
env=environment,
)
self.assertNotEqual(invalid.returncode, 0)
self.assertIn("fresh-patch-id", invalid.stderr)
issued = subprocess.run(
["node", str(BUILDER), "engine-agent-full-grant-migration-20260717-001"],
check=False,
capture_output=True,
text=True,
env=environment,
)
self.assertNotEqual(issued.returncode, 0)
self.assertIn("patch_id_already_issued", issued.stderr)
self.build(root)
duplicate = subprocess.run(
["node", str(BUILDER), PATCH_ID],
check=False,
capture_output=True,
text=True,
env=environment,
)
self.assertNotEqual(duplicate.returncode, 0)
self.assertIn("artifact_already_exists", duplicate.stderr)
if __name__ == "__main__":
unittest.main(verbosity=2)

View File

@ -0,0 +1,147 @@
#!/usr/bin/env python3
import hashlib
import importlib.machinery
import importlib.util
import json
import os
import shutil
import subprocess
import tarfile
import tempfile
import unittest
from pathlib import Path
SCRIPT_DIR = Path(__file__).resolve().parent
BUILDER = SCRIPT_DIR / "build-engine-credential-sink-recovery-artifact.mjs"
RUNNER_PATH = SCRIPT_DIR / "nodedc-deploy"
SOURCE = Path("/Volumes/docker/nodedc-deploy/applied/nodedc-engine-credential-sink-20260716-001.tgz")
SOURCE_SHA256 = "0a96add05fe59db8f490927f66e07a84490474a7afb3ef7de51e1d6fd96f86a2"
PATCH_ID = "engine-credential-sink-recovery-20990101-001"
def load_runner():
loader = importlib.machinery.SourceFileLoader(
"nodedc_recovery_runner_under_test",
str(RUNNER_PATH),
)
spec = importlib.util.spec_from_loader(loader.name, loader)
module = importlib.util.module_from_spec(spec)
loader.exec_module(module)
return module
RUNNER = load_runner()
class EngineCredentialSinkRecoveryArtifactTest(unittest.TestCase):
def build(self, artifact_dir, patch_id=PATCH_ID, source=SOURCE):
environment = os.environ.copy()
environment["NODEDC_DEPLOY_ARTIFACT_DIR"] = str(artifact_dir)
environment["NODEDC_ENGINE_CREDENTIAL_SINK_RECOVERY_SOURCE"] = str(source)
result = subprocess.run(
["node", str(BUILDER), patch_id],
check=True,
capture_output=True,
text=True,
env=environment,
)
return json.loads(result.stdout)
def test_recovery_changes_only_manifest_id_and_passes_runner_policy(self):
self.assertTrue(SOURCE.is_file())
self.assertEqual(hashlib.sha256(SOURCE.read_bytes()).hexdigest(), SOURCE_SHA256)
with tempfile.TemporaryDirectory(prefix="nodedc-engine-sink-recovery-") as directory:
root = Path(directory)
first_dir = root / "first"
second_dir = root / "second"
first_dir.mkdir()
second_dir.mkdir()
first = self.build(first_dir)
second = self.build(second_dir)
first_artifact = Path(first["artifact"])
second_artifact = Path(second["artifact"])
self.assertEqual(first["sourceArtifactSha256"], SOURCE_SHA256)
self.assertEqual(first["changed"], ["manifest.env:id"])
self.assertEqual(first["sha256"], second["sha256"])
self.assertEqual(first_artifact.read_bytes(), second_artifact.read_bytes())
with tarfile.open(SOURCE, "r:gz") as old, tarfile.open(first_artifact, "r:gz") as new:
old_files = old.extractfile("files.txt").read()
new_files = new.extractfile("files.txt").read()
self.assertEqual(old_files, new_files)
self.assertEqual(
new.extractfile("manifest.env").read().decode("utf-8"),
f"id={PATCH_ID}\ncomponent=engine\ntype=app-overlay\n",
)
for relative_path, expected_sha256 in first["payloadSha256"].items():
old_bytes = old.extractfile(f"payload/{relative_path}").read()
new_bytes = new.extractfile(f"payload/{relative_path}").read()
self.assertEqual(old_bytes, new_bytes)
self.assertEqual(hashlib.sha256(new_bytes).hexdigest(), expected_sha256)
with tempfile.TemporaryDirectory(prefix="nodedc-recovery-runner-") as work:
manifest, entries, _payload = RUNNER.load_artifact(
first_artifact,
Path(work),
)
self.assertEqual(manifest["id"], PATCH_ID)
self.assertEqual(tuple(entries), RUNNER.ENGINE_CREDENTIAL_SINK_ARTIFACT_ENTRIES)
self.assertEqual(RUNNER.component_services("engine", entries), ("nodedc-backend",))
def test_recovery_rejects_tampered_source_invalid_id_and_overwrite(self):
with tempfile.TemporaryDirectory(prefix="nodedc-engine-sink-recovery-negative-") as directory:
root = Path(directory)
tampered = root / "tampered.tgz"
shutil.copyfile(SOURCE, tampered)
with tampered.open("ab") as stream:
stream.write(b"tampered")
environment = os.environ.copy()
environment["NODEDC_DEPLOY_ARTIFACT_DIR"] = str(root / "artifacts")
environment["NODEDC_ENGINE_CREDENTIAL_SINK_RECOVERY_SOURCE"] = str(tampered)
result = subprocess.run(
["node", str(BUILDER), PATCH_ID],
check=False,
capture_output=True,
text=True,
env=environment,
)
self.assertNotEqual(result.returncode, 0)
self.assertIn("source_sha256_mismatch", result.stderr)
invalid = subprocess.run(
["node", str(BUILDER), "engine-credential-sink-20260716-001"],
check=False,
capture_output=True,
text=True,
env={
**os.environ,
"NODEDC_DEPLOY_ARTIFACT_DIR": str(root / "invalid"),
"NODEDC_ENGINE_CREDENTIAL_SINK_RECOVERY_SOURCE": str(SOURCE),
},
)
self.assertNotEqual(invalid.returncode, 0)
self.assertIn("fresh-recovery-patch-id", invalid.stderr)
artifact_dir = root / "duplicate"
artifact_dir.mkdir()
self.build(artifact_dir)
duplicate = subprocess.run(
["node", str(BUILDER), PATCH_ID],
check=False,
capture_output=True,
text=True,
env={
**os.environ,
"NODEDC_DEPLOY_ARTIFACT_DIR": str(artifact_dir),
"NODEDC_ENGINE_CREDENTIAL_SINK_RECOVERY_SOURCE": str(SOURCE),
},
)
self.assertNotEqual(duplicate.returncode, 0)
self.assertIn("artifact_already_exists", duplicate.stderr)
if __name__ == "__main__":
unittest.main(verbosity=2)

View File

@ -1,6 +1,8 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
import importlib.machinery import importlib.machinery
import importlib.util import importlib.util
import hashlib
import inspect
import json import json
import os import os
import subprocess import subprocess
@ -8,6 +10,7 @@ import tarfile
import tempfile import tempfile
import unittest import unittest
from pathlib import Path from pathlib import Path
from unittest import mock
SCRIPT_DIR = Path(__file__).resolve().parent SCRIPT_DIR = Path(__file__).resolve().parent
@ -20,6 +23,18 @@ STAGE_ARTIFACT = (
/ "infra/deploy-artifacts" / "infra/deploy-artifacts"
/ "nodedc-n8n-private-extension-n8n-nodes-ndc-release-20260716-003.tgz" / "nodedc-n8n-private-extension-n8n-nodes-ndc-release-20260716-003.tgz"
) )
TRANSITION_ID = "20260717-004"
SEALED_RELEASE_ID = "0.1.2-05e4b38b14b4a019"
SEALED_PACKAGE_SHA256 = "05e4b38b14b4a019ce1f6eee27b9e320094cb3560903bd68074966b3a1267af5"
BUILDER_ENGINE_PATHS = (
"nodedc-source/server/assets/n8n/schema/v2.3.2/nodes.catalog.json",
"nodedc-source/server/assets/n8n/schema/v2.3.2/credentials.catalog.json",
"nodedc-source/server/assets/n8n/schema/v2.3.2/meta.json",
"nodedc-source/server/assets/n8n/icons/ndc.svg",
"nodedc-source/server/assets/n8n/icons/ndc.dark.svg",
"nodedc-source/services/n8n/private-extensions/ndc-activation.json",
"nodedc-source/services/n8n/private-extensions/docker-compose.ndc-private-extension.yml",
)
EXPECTED_NODES = [ EXPECTED_NODES = [
"n8n-nodes-ndc.ndcDataProductPublish", "n8n-nodes-ndc.ndcDataProductPublish",
"n8n-nodes-ndc.ndcDataProductRead", "n8n-nodes-ndc.ndcDataProductRead",
@ -30,6 +45,14 @@ EXPECTED_CREDENTIALS = [
"ndcDataProductReaderApi", "ndcDataProductReaderApi",
"ndcFoundryBindingApi", "ndcFoundryBindingApi",
] ]
PRIVATE_EXTENSION_CANON_FUNCTION_SHA256 = {
"expected_engine_n8n_compose_override": "8ee93f3e7c407f5557c711119236449a440d73a7fcac1bbfa50a09e6a13c1cff",
"engine_n8n_package_loader_probe_script": "2b3b3c96fad6e5e48ebea74426b21bc7eea6b6fe0e786b537da3366d18aedd9b",
"engine_n8n_private_loader_catalog": "a2ae389b4ef215900cf967b863d01056bd2a8eec55554566d0f6005e4e0bcbac",
"validate_engine_n8n_base_compose_source": "adacc5b20e52684cbc427362ddba5a6d2e13ef2091a89859e2b8f7bbf2e20458",
"preflight_engine_n8n_transition": "da3cec853e3e3e20df4d1e301aab98b77815f93ec3d68b0b61c84c6e6a3fb335",
"accept_engine_n8n_runtime": "1a2614a465161e3833e84aca402817682da1a538bdb2aecc4af66b665497338d",
}
def load_runner(): def load_runner():
@ -40,6 +63,32 @@ def load_runner():
return module return module
def snapshot_engine_builder_paths():
snapshot = {}
for relative in BUILDER_ENGINE_PATHS:
path = ENGINE_ROOT / relative
try:
path_stat = path.lstat()
except FileNotFoundError:
snapshot[relative] = None
continue
if path.is_symlink():
content = ("symlink", os.readlink(path))
elif path.is_file():
content = ("file", path.read_bytes())
else:
content = ("other", None)
snapshot[relative] = (
path_stat.st_mode,
path_stat.st_uid,
path_stat.st_gid,
path_stat.st_ino,
path_stat.st_mtime_ns,
content,
)
return snapshot
RUNNER = load_runner() RUNNER = load_runner()
@ -49,14 +98,21 @@ class EngineN8nPrivateExtensionTest(unittest.TestCase):
cls.temporary = tempfile.TemporaryDirectory(prefix="nodedc-engine-n8n-policy-") cls.temporary = tempfile.TemporaryDirectory(prefix="nodedc-engine-n8n-policy-")
cls.root = Path(cls.temporary.name) cls.root = Path(cls.temporary.name)
cls.results = [] cls.results = []
cls.engine_snapshot_before = snapshot_engine_builder_paths()
for index in range(2): for index in range(2):
output = cls.root / f"build-{index}" output = cls.root / f"build-{index}"
output.mkdir() output.mkdir()
env = os.environ.copy() env = os.environ.copy()
env.pop("NODEDC_N8N_TRANSITION_ID", None)
env["NODEDC_DEPLOY_ARTIFACT_DIR"] = str(output) env["NODEDC_DEPLOY_ARTIFACT_DIR"] = str(output)
env["NODEDC_N8N_EXTENSION_STAGE_ARTIFACT"] = str(STAGE_ARTIFACT) env["NODEDC_N8N_EXTENSION_STAGE_ARTIFACT"] = str(STAGE_ARTIFACT)
command = ["node", str(BUILDER_PATH)]
if index == 0:
command.append(TRANSITION_ID)
else:
env["NODEDC_N8N_TRANSITION_ID"] = TRANSITION_ID
result = subprocess.run( result = subprocess.run(
["node", str(BUILDER_PATH)], command,
cwd=PLATFORM_ROOT, cwd=PLATFORM_ROOT,
env=env, env=env,
check=True, check=True,
@ -64,6 +120,7 @@ class EngineN8nPrivateExtensionTest(unittest.TestCase):
text=True, text=True,
) )
cls.results.append(json.loads(result.stdout)) cls.results.append(json.loads(result.stdout))
cls.engine_snapshot_after = snapshot_engine_builder_paths()
@classmethod @classmethod
def tearDownClass(cls): def tearDownClass(cls):
@ -81,6 +138,69 @@ class EngineN8nPrivateExtensionTest(unittest.TestCase):
self.assertEqual(first[4:8], b"\0\0\0\0") self.assertEqual(first[4:8], b"\0\0\0\0")
self.assertEqual(first[3] & 0x08, 0) self.assertEqual(first[3] & 0x08, 0)
def test_successful_generation_003_runner_functions_are_frozen(self):
actual = {
name: hashlib.sha256(inspect.getsource(getattr(RUNNER, name)).encode("utf-8")).hexdigest()
for name in PRIVATE_EXTENSION_CANON_FUNCTION_SHA256
}
self.assertEqual(actual, PRIVATE_EXTENSION_CANON_FUNCTION_SHA256)
def test_explicit_transition_id_does_not_mutate_sealed_release_identity(self):
expected_ids = {
"activation": f"engine-n8n-private-extension-{TRANSITION_ID}",
"rollback": f"engine-n8n-private-extension-rollback-{TRANSITION_ID}",
}
for result in self.results:
self.assertEqual(result["transitionId"], TRANSITION_ID)
self.assertEqual(result["releaseId"], SEALED_RELEASE_ID)
self.assertEqual(result["packageSha256"], SEALED_PACKAGE_SHA256)
for kind, expected_id in expected_ids.items():
self.assertEqual(result[kind]["id"], expected_id)
self.assertEqual(Path(result[kind]["artifact"]).name, f"nodedc-{expected_id}.tgz")
def test_builder_does_not_mutate_engine_source(self):
self.assertEqual(self.engine_snapshot_after, self.engine_snapshot_before)
def test_transition_id_rejects_missing_invalid_and_previously_issued_values(self):
cases = (
([], "transition_id_required"),
(["20260716-003"], "transition_id_already_issued"),
(["engine-n8n-private-extension-20260717-004"], "transition_id_invalid"),
(["20260230-004"], "transition_id_invalid"),
(["20260717-000"], "transition_id_invalid"),
([TRANSITION_ID, "unexpected-second-id"], "transition_id_argument_count_invalid"),
)
for arguments, expected_error in cases:
with self.subTest(arguments=arguments):
env = os.environ.copy()
env.pop("NODEDC_N8N_TRANSITION_ID", None)
result = subprocess.run(
["node", str(BUILDER_PATH), *arguments],
cwd=PLATFORM_ROOT,
env=env,
check=False,
capture_output=True,
text=True,
)
self.assertNotEqual(result.returncode, 0)
self.assertIn(expected_error, result.stderr)
def test_existing_transition_artifact_is_never_overwritten(self):
env = os.environ.copy()
env.pop("NODEDC_N8N_TRANSITION_ID", None)
env["NODEDC_DEPLOY_ARTIFACT_DIR"] = str(self.artifact(0, "activation").parent)
env["NODEDC_N8N_EXTENSION_STAGE_ARTIFACT"] = str(STAGE_ARTIFACT)
result = subprocess.run(
["node", str(BUILDER_PATH), TRANSITION_ID],
cwd=PLATFORM_ROOT,
env=env,
check=False,
capture_output=True,
text=True,
)
self.assertNotEqual(result.returncode, 0)
self.assertIn("transition_artifact_already_exists", result.stderr)
def test_activation_and_rollback_pass_strict_runner_policy(self): def test_activation_and_rollback_pass_strict_runner_policy(self):
expected = {"activation": ("activate", 7), "rollback": ("rollback-inactive", 4)} expected = {"activation": ("activate", 7), "rollback": ("rollback-inactive", 4)}
for kind, (action, entry_count) in expected.items(): for kind, (action, entry_count) in expected.items():
@ -137,12 +257,213 @@ class EngineN8nPrivateExtensionTest(unittest.TestCase):
) )
override = (payload / RUNNER.ENGINE_N8N_COMPOSE_OVERRIDE_REL).read_text() override = (payload / RUNNER.ENGINE_N8N_COMPOSE_OVERRIDE_REL).read_text()
self.assertEqual(override, RUNNER.expected_engine_n8n_compose_override(descriptor)) self.assertEqual(override, RUNNER.expected_engine_n8n_compose_override(descriptor))
self.assertEqual(
hashlib.sha256(override.encode("utf-8")).hexdigest(),
"a29e2f92b87dda59da2b4e3ce250bc9705ed9fa551c0fbe4e95464fca5caa3e1",
)
self.assertIn("pull_policy: never", override) self.assertIn("pull_policy: never", override)
self.assertIn("N8N_USER_FOLDER: /home/node", override) self.assertIn("N8N_USER_FOLDER: /home/node", override)
self.assertNotIn("NODE_PATH", override)
self.assertIn(":/home/node/.n8n/nodes/node_modules/n8n-nodes-ndc:ro", override) self.assertIn(":/home/node/.n8n/nodes/node_modules/n8n-nodes-ndc:ro", override)
self.assertNotIn("N8N_CUSTOM_EXTENSIONS", override) self.assertNotIn("N8N_CUSTOM_EXTENSIONS", override)
self.assertNotIn("build:", override) self.assertNotIn("build:", override)
def test_generation_003_override_is_accepted_as_installed_canon(self):
original_root = RUNNER.COMPONENTS["engine"]["payload_root"]
original_compose = RUNNER.COMPONENTS["engine"]["compose_root"]
try:
with tempfile.TemporaryDirectory() as directory:
work = Path(directory)
extracted = work / "artifact"
extracted.mkdir()
_manifest, _entries, payload = RUNNER.load_artifact(
self.artifact(0, "activation"),
extracted,
)
engine_root = work / "engine"
engine_root.mkdir()
(engine_root / "docker-compose.yml").write_text("services:\n", encoding="utf-8")
for relative in (
RUNNER.ENGINE_N8N_TRANSITION_DESCRIPTOR_REL,
RUNNER.ENGINE_N8N_COMPOSE_OVERRIDE_REL,
):
target = engine_root / relative
target.parent.mkdir(parents=True, exist_ok=True)
target.write_bytes((payload / relative).read_bytes())
RUNNER.COMPONENTS["engine"]["payload_root"] = engine_root
RUNNER.COMPONENTS["engine"]["compose_root"] = engine_root
compose_files = RUNNER.component_compose_files("engine")
self.assertEqual(compose_files, (
engine_root / "docker-compose.yml",
engine_root / RUNNER.ENGINE_N8N_COMPOSE_OVERRIDE_REL,
))
finally:
RUNNER.COMPONENTS["engine"]["payload_root"] = original_root
RUNNER.COMPONENTS["engine"]["compose_root"] = original_compose
def test_loader_probe_pins_n8n_node_path(self):
container = {
"Mounts": [{
"Destination": RUNNER.ENGINE_N8N_RUNTIME_PACKAGE_PATH,
"RW": False,
}],
}
completed = subprocess.CompletedProcess(
args=[],
returncode=0,
stdout=json.dumps({
"packageName": "n8n-nodes-ndc",
"nodes": ["ndcDataProductPublish", "ndcDataProductRead", "ndcFoundryBinding"],
"credentials": EXPECTED_CREDENTIALS,
}),
stderr="",
)
with (
mock.patch.object(RUNNER, "inspect_container", return_value=container),
mock.patch.object(RUNNER.subprocess, "run", return_value=completed) as run,
):
catalog = RUNNER.engine_n8n_private_loader_catalog("n8n-container")
command = run.call_args.args[0]
self.assertEqual(command[:-1], [
str(RUNNER.DOCKER),
"exec",
"n8n-container",
"node",
"-e",
])
probe_script = command[-1]
self.assertIn(
f"process.env.NODE_PATH='{RUNNER.ENGINE_N8N_NODE_MODULES_PATH}'",
probe_script,
)
self.assertIn("Module._initPaths()", probe_script)
self.assertIn("PackageDirectoryLoader", probe_script)
self.assertEqual(catalog, {
"node_types": EXPECTED_NODES,
"credential_types": EXPECTED_CREDENTIALS,
})
def test_loader_probe_failure_emits_only_allowlisted_diagnostic(self):
container = {
"Mounts": [{
"Destination": RUNNER.ENGINE_N8N_RUNTIME_PACKAGE_PATH,
"RW": False,
}],
}
cases = (
("MODULE_NOT_FOUND", "MODULE_NOT_FOUND"),
("Bearer secret-that-must-not-be-logged", "PROBE_ERROR"),
)
for probe_stderr, expected_category in cases:
with (
self.subTest(stderr=probe_stderr),
mock.patch.object(RUNNER, "inspect_container", return_value=container),
mock.patch.object(
RUNNER.subprocess,
"run",
return_value=subprocess.CompletedProcess(
args=[],
returncode=1,
stdout="",
stderr=probe_stderr,
),
),
):
with self.assertRaises(RUNNER.DeployError) as raised:
RUNNER.engine_n8n_private_loader_catalog("n8n-container")
message = str(raised.exception)
self.assertIn(f"category={expected_category}", message)
self.assertNotIn("secret-that-must-not-be-logged", message)
def test_active_runtime_acceptance_does_not_require_node_path(self):
with tempfile.TemporaryDirectory() as directory:
_manifest, _entries, payload = RUNNER.load_artifact(
self.artifact(0, "activation"),
Path(directory),
)
descriptor = RUNNER.read_engine_n8n_transition_descriptor(
payload / RUNNER.ENGINE_N8N_TRANSITION_DESCRIPTOR_REL
)
container = {
"State": {"Status": "running", "StartedAt": "2026-07-16T00:00:00Z"},
"Image": "sha256:base-image",
"Config": {
"Env": [
"N8N_USER_FOLDER=/home/node",
"N8N_COMMUNITY_PACKAGES_ENABLED=true",
"N8N_COMMUNITY_PACKAGES_PREVENT_LOADING=false",
"N8N_REINSTALL_MISSING_PACKAGES=false",
],
"Labels": {
"nodedc.n8n-private-extension.release": descriptor["releaseId"],
"nodedc.n8n-private-extension.package-sha256": descriptor["packageSha256"],
},
},
"Mounts": [{
"Destination": RUNNER.ENGINE_N8N_RUNTIME_PACKAGE_PATH,
"Source": f"/volume2/nodedc-demo/{descriptor['sealedReleaseRelativePath']}",
"RW": False,
}],
"RestartCount": 0,
}
log_result = subprocess.CompletedProcess(args=[], returncode=0, stdout="", stderr="")
with (
mock.patch.object(
RUNNER,
"inspect_engine_n8n_base_image",
return_value={"id": "sha256:base-image"},
),
mock.patch.object(RUNNER, "engine_n8n_container_ids", return_value=("n8n-container",)),
mock.patch.object(RUNNER, "inspect_container", return_value=container),
mock.patch.object(RUNNER, "wait_engine_n8n_readiness"),
mock.patch.object(RUNNER, "engine_n8n_version", return_value=RUNNER.ENGINE_N8N_VERSION),
mock.patch.object(RUNNER.subprocess, "run", return_value=log_result),
mock.patch.object(
RUNNER,
"engine_n8n_private_loader_catalog",
return_value={
"node_types": EXPECTED_NODES,
"credential_types": EXPECTED_CREDENTIALS,
},
),
mock.patch.object(RUNNER, "validate_engine_n8n_catalog_payload"),
mock.patch.object(RUNNER.time, "sleep"),
):
accepted = RUNNER.accept_engine_n8n_runtime(descriptor)
self.assertEqual(accepted["node_types"], EXPECTED_NODES)
def test_private_extension_transition_does_not_enter_publish_grant_domain(self):
entries = (RUNNER.ENGINE_N8N_TRANSITION_DESCRIPTOR_REL,)
descriptor = {"action": "rollback-inactive"}
self.assertFalse(RUNNER.touches_engine_data_product_publish_grant(entries))
self.assertEqual(RUNNER.component_services("engine", entries), ("n8n",))
with (
mock.patch.object(RUNNER, "current_engine_n8n_transition_descriptor", return_value=descriptor),
mock.patch.object(RUNNER, "ensure_engine_edp_managed_provisioner_keypair") as keypair,
mock.patch.object(RUNNER, "ensure_engine_publish_grant_private_state") as private_state,
):
RUNNER.prepare_component_runtime("engine", entries)
keypair.assert_not_called()
private_state.assert_not_called()
def test_generic_engine_compose_artifact_keeps_legacy_service_selection(self):
self.assertEqual(
RUNNER.component_services("engine", ("docker-compose.yml",)),
RUNNER.COMPONENTS["engine"]["services"],
)
self.assertEqual(
RUNNER.component_services(
"engine",
("docker-compose.yml", "nodedc-source/server/dataProductPublishGrant/store.js"),
),
RUNNER.COMPONENTS["engine"]["services"],
)
self.assertFalse(RUNNER.is_engine_data_product_publish_grant_slice(
"engine",
("docker-compose.yml", "nodedc-source/server/dataProductPublishGrant/store.js"),
))
def test_unknown_descriptor_key_is_rejected(self): def test_unknown_descriptor_key_is_rejected(self):
with tempfile.TemporaryDirectory() as directory: with tempfile.TemporaryDirectory() as directory:
work = Path(directory) work = Path(directory)

View File

@ -0,0 +1,156 @@
#!/usr/bin/env python3
import hashlib
import json
import os
import subprocess
import tarfile
import tempfile
import unittest
from pathlib import Path
SCRIPT_DIR = Path(__file__).resolve().parent
BUILDER = SCRIPT_DIR / "build-engine-data-product-publish-grant-artifact.mjs"
EXPECTED_ENTRIES = [
"nodedc-source/server/assets/provider-packages/v1/catalog.json",
"nodedc-source/server/dataProductPublishGrant",
"nodedc-source/server/engineAgents/store.js",
"nodedc-source/server/routes/engineAgentGateway.js",
"nodedc-source/server/routes/n8n.js",
"nodedc-source/services/backend/data-product-publish-grant/docker-compose.immutable-runtime.yml",
]
PATCH_ID = "engine-data-product-publish-grant-20990101-001"
SUCCESSFUL_CREDENTIAL_SINK_INDEX_SHA256 = (
"b2b790b02839570d967a2ca68b00e2724485a99389ac9b3a589a1b22302a36b8"
)
class EnginePublishGrantArtifactTest(unittest.TestCase):
def build(self, artifact_dir, patch_id):
environment = os.environ.copy()
environment["NODEDC_DEPLOY_ARTIFACT_DIR"] = str(artifact_dir)
result = subprocess.run(
["node", str(BUILDER), patch_id],
check=True,
capture_output=True,
text=True,
env=environment,
)
return json.loads(result.stdout)
def test_artifact_is_narrow_secret_free_and_deterministic(self):
with tempfile.TemporaryDirectory(prefix="nodedc-engine-publish-artifact-") as directory:
root = Path(directory)
first_dir = root / "first"
second_dir = root / "second"
first_dir.mkdir()
second_dir.mkdir()
first = self.build(first_dir, PATCH_ID)
artifact = Path(first["artifact"])
first_bytes = artifact.read_bytes()
second = self.build(second_dir, PATCH_ID)
second_bytes = Path(second["artifact"]).read_bytes()
self.assertEqual(first["entries"], EXPECTED_ENTRIES)
self.assertEqual(first["sha256"], hashlib.sha256(first_bytes).hexdigest())
self.assertEqual(first["sha256"], second["sha256"])
self.assertEqual(first_bytes, second_bytes)
with tarfile.open(artifact, "r:gz") as archive:
names = {member.name for member in archive.getmembers()}
files = archive.extractfile("files.txt").read().decode("utf-8").splitlines()
manifest = archive.extractfile("manifest.env").read().decode("utf-8")
n8n_source = archive.extractfile(
"payload/nodedc-source/server/routes/n8n.js"
).read().decode("utf-8")
gateway_source = archive.extractfile(
"payload/nodedc-source/server/routes/engineAgentGateway.js"
).read().decode("utf-8")
runtime_override = archive.extractfile(
"payload/nodedc-source/services/backend/data-product-publish-grant/"
"docker-compose.immutable-runtime.yml"
).read().decode("utf-8")
self.assertEqual(files, EXPECTED_ENTRIES)
self.assertEqual(
manifest,
f"id={PATCH_ID}\ncomponent=engine\ntype=app-overlay\n",
)
forbidden_roots = (
"payload/docker-compose.yml",
"payload/nodedc-source/server/index.js",
"payload/nodedc-source/server/credentialPolicies/ndcPrivateNode.js",
"payload/nodedc-source/server/credentialSink/",
"payload/nodedc-source/server/routes/engineCredentialSink.js",
"payload/nodedc-source/server/middleware/demoAccess.js",
"payload/nodedc-source/server/routes/ndcAgentMcp.js",
"payload/nodedc-source/server/tests/",
"payload/nodedc-source/dist/",
"payload/nodedc-source/server/data/",
)
self.assertFalse(any(
name == root.rstrip("/") or name.startswith(root)
for name in names
for root in forbidden_roots
))
self.assertNotIn("from '../credentialSink/", n8n_source)
self.assertIn("engineCredentialSinkN8nAdapter", n8n_source)
self.assertIn("engineDataProductPublishGrantN8nAdapter", n8n_source)
for tool in (
"engine_plan_data_product_publish_grant",
"engine_apply_data_product_publish_grant",
"engine_accept_data_product_publish_grant",
"engine_rollback_data_product_publish_grant",
):
self.assertIn(tool, gateway_source)
self.assertIn(" nodedc-backend:", runtime_override)
self.assertNotIn(" n8n:", runtime_override)
self.assertIn(
"source: /volume2/nodedc-demo/nodedc-control-plane/publish-grants",
runtime_override,
)
self.assertNotIn(
"source: /volume2/nodedc-demo/nodedc-control-plane\n",
runtime_override,
)
self.assertIn("read_only: true", runtime_override)
self.assertEqual(runtime_override.count("create_host_path: false"), 2)
def test_builder_requires_a_fresh_never_issued_patch_id(self):
with tempfile.TemporaryDirectory(prefix="nodedc-engine-publish-id-") as directory:
artifact_dir = Path(directory)
environment = os.environ.copy()
environment["NODEDC_DEPLOY_ARTIFACT_DIR"] = str(artifact_dir)
cases = (
([], "fresh-patch-id"),
(["engine-data-product-publish-grant-20260716-001"], "patch_id_already_issued"),
(["engine-data-product-publish-grant-20260717-001"], "patch_id_already_issued"),
(["engine-data-product-publish-grant-20260717-002"], "patch_id_already_issued"),
(["engine-publish-grant-unit-001"], "fresh-patch-id"),
)
for arguments, expected in cases:
with self.subTest(arguments=arguments):
result = subprocess.run(
["node", str(BUILDER), *arguments],
check=False,
capture_output=True,
text=True,
env=environment,
)
self.assertNotEqual(result.returncode, 0)
self.assertIn(expected, result.stderr)
self.build(artifact_dir, PATCH_ID)
duplicate = subprocess.run(
["node", str(BUILDER), PATCH_ID],
check=False,
capture_output=True,
text=True,
env=environment,
)
self.assertNotEqual(duplicate.returncode, 0)
self.assertIn("artifact_already_exists", duplicate.stderr)
if __name__ == "__main__":
unittest.main(verbosity=2)

View File

@ -0,0 +1,88 @@
#!/usr/bin/env python3
import hashlib
import json
import os
import subprocess
import tarfile
import tempfile
import unittest
from pathlib import Path
SCRIPT_DIR = Path(__file__).resolve().parent
BUILDER = SCRIPT_DIR / "build-external-data-plane-artifact.mjs"
EXPECTED_ENTRIES = [
"platform/docker-compose.external-data-plane.yml",
"platform/services/external-data-plane",
"platform/packages/external-provider-contract/package.json",
"platform/packages/external-provider-contract/src/contract-version.mjs",
"platform/packages/external-provider-contract/src/data-plane.mjs",
"platform/packages/external-provider-contract/src/data-product.mjs",
"platform/packages/external-provider-contract/src/intake-batch.mjs",
"platform/packages/external-provider-contract/src/sensitive-field-policy.mjs",
]
class ExternalDataPlaneArtifactTest(unittest.TestCase):
def build(self, artifact_dir, patch_id):
environment = os.environ.copy()
environment["NODEDC_DEPLOY_ARTIFACT_DIR"] = str(artifact_dir)
result = subprocess.run(
["node", str(BUILDER), patch_id],
check=True,
capture_output=True,
text=True,
env=environment,
)
return json.loads(result.stdout)
def test_artifact_is_runtime_only_public_trust_and_deterministic(self):
with tempfile.TemporaryDirectory(prefix="nodedc-edp-artifact-") as directory:
artifact_dir = Path(directory)
first = self.build(artifact_dir, "edp-managed-publish-unit-001")
artifact = Path(first["artifact"])
first_bytes = artifact.read_bytes()
second = self.build(artifact_dir, "edp-managed-publish-unit-001")
second_bytes = Path(second["artifact"]).read_bytes()
self.assertEqual(first["entries"], EXPECTED_ENTRIES)
self.assertEqual(first["sha256"], hashlib.sha256(first_bytes).hexdigest())
self.assertEqual(first["sha256"], second["sha256"])
self.assertEqual(first_bytes, second_bytes)
with tarfile.open(artifact, "r:gz") as archive:
members = archive.getmembers()
names = {member.name for member in members}
files = archive.extractfile("files.txt").read().decode("utf-8").splitlines()
manifest = archive.extractfile("manifest.env").read().decode("utf-8")
compose = archive.extractfile(
"payload/platform/docker-compose.external-data-plane.yml"
).read().decode("utf-8")
regular_payloads = [
archive.extractfile(member).read()
for member in members
if member.isfile()
]
self.assertEqual(files, EXPECTED_ENTRIES)
self.assertEqual(
manifest,
"id=edp-managed-publish-unit-001\ncomponent=platform\ntype=app-overlay\n",
)
self.assertFalse(any(
name.startswith("payload/platform/services/external-data-plane/test/")
or "/node_modules/" in name
or Path(name).name.startswith(".env")
or Path(name).name == "private-key.pem"
for name in names
))
self.assertIn(
"source: /volume1/docker/nodedc-platform/trust/engine-managed-provisioner",
compose,
)
self.assertNotIn("private-key.pem", compose)
self.assertNotIn(b"-----BEGIN PRIVATE KEY-----", b"\n".join(regular_payloads))
if __name__ == "__main__":
unittest.main(verbosity=2)

View File

@ -0,0 +1,926 @@
#!/usr/bin/env python3
import importlib.machinery
import importlib.util
import hashlib
import inspect
import json
import stat
import tempfile
import unittest
from pathlib import Path
from unittest import mock
SCRIPT_DIR = Path(__file__).resolve().parent
RUNNER_PATH = SCRIPT_DIR / "nodedc-deploy"
def load_runner():
loader = importlib.machinery.SourceFileLoader(
"nodedc_platform_deploy_under_test",
str(RUNNER_PATH),
)
spec = importlib.util.spec_from_loader(loader.name, loader)
module = importlib.util.module_from_spec(spec)
loader.exec_module(module)
return module
RUNNER = load_runner()
LEGACY_CREDENTIAL_RUNTIME_CANON_FUNCTION_SHA256 = {
"build_canonical_engine_backend_rootfs": "7f8fca6380632d717ee38a3d58b5bbc58d34b0e6c81e2764a759ddd63bfd07d0",
"engine_backend_container_id": "08dc25b6fee37e8c0d942df91fc575d6766b99c306ad06c2c14afded8de3b375",
"engine_backend_immutable_runtime_is_current": "9c6b9a73ac5004830b1f06f56bfc1f25336d11001a97a5de7a04388e10a20990",
"engine_backend_node_modules_tree_sha256": "01749704e1624d247bdf41242aa71961f5e5bc092ffa906b69cdb911907b70c4",
"engine_backend_rootfs_member_allowed": "62342e5891a1ea885f1ff02169a100f9ca94464eb7dfb62a7cbe67750ebcbb91",
"engine_backend_rootfs_toolchain_link_is_omitted": "83772bc1a8aa654e5c1d0757d43b650ee978dc73aef7ae8fac567dbacc8c7c49",
"engine_backend_tool_versions": "c21d7c11ee8c40ac5e3e5c6c24c441d1ce52c9dd2c22fc1d8f26e121138fa78a",
"ensure_engine_backend_release_image": "f65a5f8573b911980f252dc3beed88cd6ee14e0f0c519c333ea90b78c8910953",
"expected_engine_credential_backend_override": "b7c9db0059a4d262b8fe61dc0745b8ee1a4b75d56d227c39ba08115c7e414486",
"inspect_engine_backend_derived_image": "2336ce8068bd321b6c8ac6e62b430dbcbf991201cb9b4178589294d1baf7e818",
"preflight_engine_credential_sink_ready": "3f084ade6346644c86aa9540fbcf70b3a44e32c7add0876935f45177569aad6d",
"prepare_engine_credential_backend_runtime": "0def464639767dcf85acd7978459b2f5c1ebf963424dd401d4c29aae7bb1aaa5",
"quarantine_engine_backend_partial_runtime": "57e455f874c681815cf42bf529d0b8620ae8317e82551d784cbd6dae913603d2",
"read_engine_backend_runtime_metadata": "7ee0ec2c72eb699db4b6a7d3d8d912f1b0c0c724c2a054eee223fba3ec0a1cb2",
"run_compose": "8425528d4934f82e79ebf3551c4246505e4a3cdf3f450949d7618a4312fee169",
"run_engine_backend_probe": "f0db7210b64cf3c80c1f0a00165f27dcab6962781a69dc2050d84a5373cedaa9",
"touches_engine_credential_sink": "5d0e0d680d1a8508fb1ac21d8c8b3aa92466fe8b219d2ddc1fe5411f94fca912",
"validate_engine_backend_activation_marker": "66c4d5d9b0b608c030c2c9c97e2e46e1f1634b7ba024dc107c15478091765483",
"validate_engine_backend_dependencies": "ca1383dffd5a6397ecfeedcc8f88ab59e90a33d5089590f0249f789b05ed29e2",
"validate_engine_backend_mounts": "5614267d1d0f734dd68ed7565292a53f34777b6dc3640137034f6db093ef7431",
"validate_engine_backend_runtime_override_file": "eae1d32ab75cc59916dbd1d4cd73a072c156abf9014d8ef234fcd2333748139d",
"validate_engine_credential_sink_slice": "87e6571baf47c30af120da32a327a88164415bf65b198e7190c28df7365bcfd5",
}
class CanonicalPlatformRegistryTest(unittest.TestCase):
def test_legacy_credential_runtime_functions_match_last_successful_canon(self):
actual = {
name: hashlib.sha256(
inspect.getsource(getattr(RUNNER, name)).encode("utf-8"),
).hexdigest()
for name in LEGACY_CREDENTIAL_RUNTIME_CANON_FUNCTION_SHA256
}
self.assertEqual(actual, LEGACY_CREDENTIAL_RUNTIME_CANON_FUNCTION_SHA256)
def test_publish_mount_extension_delegates_baseline_inventory_to_legacy_guard(self):
baseline_mounts = [
{
"Type": "bind",
"Source": "/baseline/app",
"Destination": "/app",
"RW": True,
},
]
publish_mounts = [
{
"Type": "bind",
"Source": str(RUNNER.ENGINE_PUBLISH_GRANT_STATE_PATH),
"Destination": RUNNER.ENGINE_PUBLISH_GRANT_CONTAINER_PATH,
"RW": True,
},
{
"Type": "bind",
"Source": str(RUNNER.ENGINE_EDP_MANAGED_PROVISIONER_PRIVATE_KEY_FILE),
"Destination": RUNNER.ENGINE_EDP_PRIVATE_KEY_CONTAINER_PATH,
"RW": False,
},
]
container = {"Mounts": [*baseline_mounts, *publish_mounts]}
with tempfile.TemporaryDirectory(prefix="nodedc-publish-mounts-") as directory:
root = Path(directory)
override = root / RUNNER.ENGINE_DATA_PRODUCT_PUBLISH_GRANT_OVERRIDE_REL
override.parent.mkdir(parents=True)
override.write_text(
RUNNER.expected_engine_data_product_publish_grant_override(),
encoding="utf-8",
)
original_root = RUNNER.COMPONENTS["engine"]["payload_root"]
RUNNER.COMPONENTS["engine"]["payload_root"] = root
try:
with mock.patch.object(
RUNNER,
"validate_engine_backend_mounts",
) as legacy_guard:
RUNNER.validate_engine_backend_mounts_for_installed_runtime(
container,
True,
)
legacy_guard.assert_called_once_with(
{"Mounts": baseline_mounts},
True,
)
wrong_source = {
"Mounts": [
*baseline_mounts,
{**publish_mounts[0], "Source": "/wrong"},
publish_mounts[1],
],
}
with self.assertRaisesRegex(
RUNNER.DeployError,
"Publish mount barrier mismatch",
):
RUNNER.validate_engine_backend_mounts_for_installed_runtime(
wrong_source,
True,
)
finally:
RUNNER.COMPONENTS["engine"]["payload_root"] = original_root
def test_publish_mount_extension_accepts_exact_complete_runtime_inventory(self):
baseline = {
"/app": (True, "/engine/source"),
"/app/deploy/docker-compose.yml": (False, "/engine/docker-compose.yml"),
"/seed-api": (False, "/engine/seed-api"),
"/seed-data": (False, "/engine/seed-data"),
"/app/node_modules": (False, str(RUNNER.ENGINE_CREDENTIAL_BACKEND_NODE_MODULES_DIR)),
"/app/server/data": (True, "/engine/data"),
"/app/server/storage": (True, "/engine/storage"),
"/app/server/logs": (True, "/engine/logs"),
"/var/run/docker.sock": (True, "/var/run/docker.sock"),
}
mounts = [
{
"Type": "bind",
"Source": source,
"Destination": destination,
"RW": writable,
}
for destination, (writable, source) in baseline.items()
]
mounts.extend((
{
"Type": "bind",
"Source": str(RUNNER.ENGINE_PUBLISH_GRANT_STATE_PATH),
"Destination": RUNNER.ENGINE_PUBLISH_GRANT_CONTAINER_PATH,
"RW": True,
},
{
"Type": "bind",
"Source": str(RUNNER.ENGINE_EDP_MANAGED_PROVISIONER_PRIVATE_KEY_FILE),
"Destination": RUNNER.ENGINE_EDP_PRIVATE_KEY_CONTAINER_PATH,
"RW": False,
},
))
with tempfile.TemporaryDirectory(prefix="nodedc-publish-full-mounts-") as directory:
root = Path(directory)
override = root / RUNNER.ENGINE_DATA_PRODUCT_PUBLISH_GRANT_OVERRIDE_REL
override.parent.mkdir(parents=True)
override.write_text(
RUNNER.expected_engine_data_product_publish_grant_override(),
encoding="utf-8",
)
original_root = RUNNER.COMPONENTS["engine"]["payload_root"]
RUNNER.COMPONENTS["engine"]["payload_root"] = root
try:
RUNNER.validate_engine_backend_mounts_for_installed_runtime(
{"Mounts": mounts},
True,
)
finally:
RUNNER.COMPONENTS["engine"]["payload_root"] = original_root
def test_absent_publish_overlay_uses_legacy_mount_guard_unchanged(self):
container = {"Mounts": [{"Destination": "/app"}]}
with tempfile.TemporaryDirectory(prefix="nodedc-no-publish-mounts-") as directory:
original_root = RUNNER.COMPONENTS["engine"]["payload_root"]
RUNNER.COMPONENTS["engine"]["payload_root"] = Path(directory)
try:
with mock.patch.object(
RUNNER,
"validate_engine_backend_mounts",
) as legacy_guard:
RUNNER.validate_engine_backend_mounts_for_installed_runtime(
container,
False,
)
legacy_guard.assert_called_once_with(container, False)
finally:
RUNNER.COMPONENTS["engine"]["payload_root"] = original_root
def test_runner_does_not_reinterpret_compose_schema(self):
self.assertFalse(hasattr(RUNNER, "render_compose_model"))
self.assertFalse(hasattr(RUNNER, "validate_platform_compose_model"))
self.assertFalse(hasattr(RUNNER, "validate_engine_compose_model"))
self.assertFalse(hasattr(RUNNER, "preflight_platform_compose_candidate"))
self.assertFalse(hasattr(RUNNER, "preflight_engine_compose_candidate"))
def test_platform_edp_change_recreates_only_application(self):
entries = (
"platform/docker-compose.external-data-plane.yml",
"platform/services/external-data-plane",
)
services = RUNNER.component_services("platform", entries)
self.assertEqual(services, ("external-data-plane",))
self.assertNotIn("external-data-plane-postgres", services)
builds = RUNNER.component_builds("platform", entries)
self.assertEqual(len(builds), 1)
build_root, build_args = builds[0]
self.assertEqual(
build_root,
Path("/volume1/docker/nodedc-platform/platform"),
)
self.assertEqual(
build_args,
(
"build",
"--no-cache",
"-f",
"services/external-data-plane/Dockerfile",
"-t",
RUNNER.EXTERNAL_DATA_PLANE_IMAGE,
".",
),
)
def test_provider_catalog_is_metadata_only(self):
entries = (
"platform/packages/external-provider-contract/providers/gelios/v1/provider.json",
)
self.assertFalse(RUNNER.touches_external_data_plane_files(entries))
self.assertEqual(RUNNER.component_services("platform", entries), ())
self.assertEqual(RUNNER.component_builds("platform", entries), ())
self.assertEqual(RUNNER.component_healthchecks("platform", entries), ())
with (
mock.patch.object(RUNNER, "run_build") as build,
mock.patch.object(RUNNER, "prepare_component_runtime") as prepare,
mock.patch.object(RUNNER, "run_compose") as compose,
mock.patch.object(RUNNER, "healthcheck_url") as healthcheck,
):
RUNNER.run_component_runtime("platform", entries, ())
RUNNER.run_healthchecks("platform", entries, ())
build.assert_not_called()
prepare.assert_not_called()
compose.assert_not_called()
healthcheck.assert_not_called()
def test_generic_engine_compose_change_keeps_legacy_service_selection(self):
services = RUNNER.component_services(
"engine",
(
"docker-compose.yml",
"nodedc-source/server/index.js",
"nodedc-source/server/dataProductPublishGrant/store.js",
),
)
self.assertEqual(services, ("nodedc-backend", "app"))
self.assertNotIn("n8n-postgres", services)
self.assertNotIn("postgresql", services)
self.assertNotIn("n8n", services)
def test_engine_runtime_prepares_managed_private_state(self):
with (
mock.patch.object(
RUNNER,
"ensure_engine_edp_managed_provisioner_keypair",
) as keypair,
mock.patch.object(
RUNNER,
"ensure_engine_publish_grant_private_state",
) as private_state,
):
RUNNER.prepare_component_runtime(
"engine",
RUNNER.ENGINE_DATA_PRODUCT_PUBLISH_GRANT_ARTIFACT_ENTRIES,
)
keypair.assert_called_once_with()
private_state.assert_called_once_with()
def test_partial_publish_path_has_no_publish_runtime_side_effects(self):
entries = ("nodedc-source/server/dataProductPublishGrant/store.js",)
self.assertTrue(RUNNER.touches_engine_data_product_publish_grant(entries))
self.assertFalse(RUNNER.is_engine_data_product_publish_grant_slice(
"engine",
entries,
))
with (
mock.patch.object(
RUNNER,
"ensure_engine_edp_managed_provisioner_keypair",
) as keypair,
mock.patch.object(
RUNNER,
"ensure_engine_publish_grant_private_state",
) as private_state,
):
RUNNER.prepare_component_runtime("engine", entries)
keypair.assert_not_called()
private_state.assert_not_called()
def test_publish_predecessor_guard_checks_every_pinned_file_before_mutation(self):
with tempfile.TemporaryDirectory(prefix="nodedc-publish-predecessor-") as directory:
root = Path(directory)
files = {
"docker-compose.yml": b"services: {}\n",
"nodedc-source/server/routes/n8n.js": b"export const baseline = true\n",
}
expected = {}
for relative_path, value in files.items():
path = root / relative_path
path.parent.mkdir(parents=True, exist_ok=True)
path.write_bytes(value)
expected[relative_path] = hashlib.sha256(value).hexdigest()
original_root = RUNNER.COMPONENTS["engine"]["payload_root"]
original_hashes = RUNNER.ENGINE_DATA_PRODUCT_PUBLISH_GRANT_PREDECESSOR_SHA256
RUNNER.COMPONENTS["engine"]["payload_root"] = root
RUNNER.ENGINE_DATA_PRODUCT_PUBLISH_GRANT_PREDECESSOR_SHA256 = expected
try:
self.assertEqual(
RUNNER.preflight_engine_data_product_publish_grant_predecessor(),
{
"mode": "credential-sink-initial-transition",
"compose_sha256": expected["docker-compose.yml"],
"changed_paths": (),
},
)
(root / "nodedc-source/server/routes/n8n.js").write_bytes(b"drift\n")
with self.assertRaisesRegex(
RUNNER.DeployError,
"path=nodedc-source/server/routes/n8n.js",
):
RUNNER.preflight_engine_data_product_publish_grant_predecessor()
finally:
RUNNER.ENGINE_DATA_PRODUCT_PUBLISH_GRANT_PREDECESSOR_SHA256 = original_hashes
RUNNER.COMPONENTS["engine"]["payload_root"] = original_root
def test_installed_publish_update_preserves_foundation_and_allows_only_grant_source_changes(self):
with tempfile.TemporaryDirectory(prefix="nodedc-publish-update-") as directory:
base = Path(directory)
root = base / "installed"
payload = base / "payload"
root.mkdir()
payload.mkdir()
compose_bytes = b"services: {}\n"
compose = root / "docker-compose.yml"
compose.write_bytes(compose_bytes)
expected = {
"docker-compose.yml": hashlib.sha256(compose_bytes).hexdigest(),
}
shared_files = {
"nodedc-source/server/assets/provider-packages/v1/catalog.json": "{}\n",
"nodedc-source/server/engineAgents/store.js": "export const profile = 'full-developer'\n",
"nodedc-source/server/routes/engineAgentGateway.js": "\n".join((
"engine_plan_data_product_publish_grant",
"engine_apply_data_product_publish_grant",
"engine_accept_data_product_publish_grant",
"engine_rollback_data_product_publish_grant",
)),
"nodedc-source/server/routes/n8n.js": (
"engineCredentialSinkN8nAdapter\n"
"engineDataProductPublishGrantN8nAdapter\n"
),
RUNNER.ENGINE_DATA_PRODUCT_PUBLISH_GRANT_OVERRIDE_REL: (
RUNNER.expected_engine_data_product_publish_grant_override()
),
}
for name in RUNNER.ENGINE_DATA_PRODUCT_PUBLISH_GRANT_SOURCE_FILES:
shared_files[f"nodedc-source/server/dataProductPublishGrant/{name}"] = (
f"export const source = '{name}'\n"
)
for relative_path, value in shared_files.items():
for destination in (root, payload):
path = destination / relative_path
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(value, encoding="utf-8")
acceptance_rel = "nodedc-source/server/dataProductPublishGrant/acceptance.js"
(payload / acceptance_rel).write_text(
"export const source = 'acceptance-fixed'\n",
encoding="utf-8",
)
original_root = RUNNER.COMPONENTS["engine"]["payload_root"]
original_hashes = RUNNER.ENGINE_DATA_PRODUCT_PUBLISH_GRANT_PREDECESSOR_SHA256
RUNNER.COMPONENTS["engine"]["payload_root"] = root
RUNNER.ENGINE_DATA_PRODUCT_PUBLISH_GRANT_PREDECESSOR_SHA256 = expected
try:
self.assertEqual(
RUNNER.preflight_engine_data_product_publish_grant_predecessor(payload),
{
"mode": "installed-source-update",
"compose_sha256": expected["docker-compose.yml"],
"changed_paths": (acceptance_rel,),
},
)
gateway_rel = "nodedc-source/server/routes/engineAgentGateway.js"
(payload / gateway_rel).write_text("cross-boundary change\n", encoding="utf-8")
with self.assertRaisesRegex(
RUNNER.DeployError,
"crosses its source boundary",
):
RUNNER.preflight_engine_data_product_publish_grant_predecessor(payload)
(payload / gateway_rel).write_text(shared_files[gateway_rel], encoding="utf-8")
compose.write_text("drift\n", encoding="utf-8")
with self.assertRaisesRegex(
RUNNER.DeployError,
"foundation drift",
):
RUNNER.preflight_engine_data_product_publish_grant_predecessor(payload)
finally:
RUNNER.ENGINE_DATA_PRODUCT_PUBLISH_GRANT_PREDECESSOR_SHA256 = original_hashes
RUNNER.COMPONENTS["engine"]["payload_root"] = original_root
def test_edp_healthcheck_requires_database_and_managed_provisioning(self):
checks = RUNNER.component_healthchecks(
"platform",
("platform/services/external-data-plane/src/server.mjs",),
("external-data-plane",),
)
self.assertEqual(
checks,
(RUNNER.external_data_plane_healthcheck(),),
)
def test_edp_runtime_never_probes_unselected_platform_services(self):
entries = ("platform/services/external-data-plane/src/server.mjs",)
services = ("external-data-plane",)
with mock.patch.object(RUNNER, "healthcheck_url") as healthcheck:
RUNNER.run_healthchecks("platform", entries, services)
healthcheck.assert_called_once_with(
RUNNER.external_data_plane_healthcheck(),
)
def test_json_healthcheck_accepts_expected_contract(self):
class Response:
status = 200
def __enter__(self):
return self
def __exit__(self, *_args):
return False
def read(self, _limit):
return json.dumps({
"ok": True,
"database": "ready",
}).encode("utf-8")
with mock.patch.object(
RUNNER.NO_REDIRECT_OPENER,
"open",
return_value=Response(),
):
RUNNER.healthcheck_url({
"url": "http://127.0.0.1:18106/healthz",
"expected_json": {
"ok": True,
"database": "ready",
},
})
def test_json_healthcheck_accepts_restored_edp_baseline_contract(self):
class Response:
status = 200
def __enter__(self):
return self
def __exit__(self, *_args):
return False
def read(self, _limit):
return json.dumps({
"ok": True,
"service": "nodedc-external-data-plane",
"database": "ready",
}).encode("utf-8")
with mock.patch.object(
RUNNER.NO_REDIRECT_OPENER,
"open",
return_value=Response(),
):
RUNNER.healthcheck_url(
RUNNER.external_data_plane_healthcheck(require_managed=False),
)
def test_json_healthcheck_does_not_accept_redirect(self):
class Response:
status = 200
def __enter__(self):
return self
def __exit__(self, *_args):
return False
def read(self, _limit):
return json.dumps({
"ok": True,
"database": "ready",
}).encode("utf-8")
url = "http://127.0.0.1:18106/healthz"
redirect = RUNNER.urllib.error.HTTPError(
url,
302,
"Found",
{},
None,
)
with (
mock.patch.object(
RUNNER.NO_REDIRECT_OPENER,
"open",
side_effect=(redirect, Response()),
) as opener,
mock.patch.object(RUNNER.time, "sleep"),
):
RUNNER.healthcheck_url({
"url": url,
"expected_json": {
"ok": True,
"database": "ready",
},
})
self.assertEqual(opener.call_count, 2)
def test_publish_grant_healthcheck_does_not_enter_n8n_transition(self):
services = ("nodedc-backend",)
with (
mock.patch.object(RUNNER, "accept_engine_n8n_runtime") as accept,
mock.patch.object(RUNNER, "healthcheck_compose_service") as compose_health,
mock.patch.object(RUNNER, "healthcheck_url") as healthcheck,
mock.patch.object(
RUNNER,
"preflight_engine_credential_backend_runtime",
return_value={"mode": "verified-derived-retry"},
) as backend_preflight,
):
RUNNER.run_healthchecks(
"engine",
RUNNER.ENGINE_DATA_PRODUCT_PUBLISH_GRANT_ARTIFACT_ENTRIES,
services,
)
accept.assert_not_called()
compose_health.assert_called_once_with("engine", "nodedc-backend")
healthcheck.assert_called_once_with(
"http://127.0.0.1:3001/health",
)
backend_preflight.assert_called_once_with()
def test_engine_ui_healthcheck_requires_selected_app(self):
grant_entries = RUNNER.ENGINE_DATA_PRODUCT_PUBLISH_GRANT_ARTIFACT_ENTRIES
self.assertEqual(
RUNNER.component_healthchecks(
"engine",
grant_entries,
("nodedc-backend",),
),
("http://127.0.0.1:3001/health",),
)
self.assertEqual(
RUNNER.component_healthchecks(
"engine",
("docker-compose.yml",),
("nodedc-backend", "app"),
),
(
"http://127.0.0.1:8080/",
"http://127.0.0.1:3001/health",
),
)
def test_optional_edp_overlay_is_omitted_until_installed(self):
with tempfile.TemporaryDirectory(
prefix="nodedc-compose-files-",
) as directory:
temporary = Path(directory)
base = temporary / "docker-compose.platform-http.yml"
edp = temporary / "docker-compose.external-data-plane.yml"
base.write_text(
"name: nodedc-platform\n",
encoding="utf-8",
)
original = RUNNER.COMPONENTS["platform"]["compose_files"]
RUNNER.COMPONENTS["platform"]["compose_files"] = (base, edp)
try:
self.assertEqual(
RUNNER.component_compose_files("platform"),
(base,),
)
finally:
RUNNER.COMPONENTS["platform"]["compose_files"] = original
class PlatformOverlayRollbackTest(unittest.TestCase):
def test_engine_runtime_failure_restores_source_and_recreates_exact_services(self):
with tempfile.TemporaryDirectory(prefix="nodedc-engine-runtime-rollback-") as directory:
temporary = Path(directory)
root = temporary / "engine"
backup = temporary / "backup"
runner_tmp = temporary / "tmp"
compose = root / "docker-compose.yml"
backend = root / "nodedc-source/server/index.js"
compose.parent.mkdir(parents=True)
backend.parent.mkdir(parents=True)
backup.mkdir()
runner_tmp.mkdir()
compose.write_text("name: baseline\n", encoding="utf-8")
backend.write_text("export const version = 1;\n", encoding="utf-8")
entries = ["docker-compose.yml", "nodedc-source/server/index.js"]
services = ("n8n", "nodedc-backend", "app")
RUNNER.create_backup(root, backup, entries, include_nginx_html=False)
compose.write_text("name: candidate\n", encoding="utf-8")
backend.write_text("export const version = 2;\n", encoding="utf-8")
def assert_restored(component, rollback_entries, rollback_services):
self.assertEqual(component, "engine")
self.assertEqual(rollback_entries, entries)
self.assertEqual(rollback_services, services)
self.assertEqual(compose.read_text(encoding="utf-8"), "name: baseline\n")
self.assertEqual(
backend.read_text(encoding="utf-8"),
"export const version = 1;\n",
)
with (
mock.patch.object(RUNNER, "TMP_DIR", runner_tmp),
mock.patch.object(
RUNNER,
"run_component_runtime",
side_effect=assert_restored,
) as runtime,
mock.patch.object(RUNNER, "run_healthchecks") as healthchecks,
):
status = RUNNER.rollback_engine_apply(
root,
backup,
entries,
"test-stamp",
runtime_started=True,
applied_services=services,
)
self.assertEqual(status, "source+runtime-restored:2")
runtime.assert_called_once()
healthchecks.assert_called_once_with("engine", entries, services)
def test_candidate_only_service_cleanup_never_requests_volume_removal(self):
with (
mock.patch.object(RUNNER, "component_compose_root", return_value=Path("/live/platform")),
mock.patch.object(RUNNER, "compose_base_cmd", return_value=["docker", "compose"]),
mock.patch.object(RUNNER.subprocess, "run") as run,
):
RUNNER.stop_and_remove_compose_services("platform", ("external-data-plane",))
command = run.call_args.args[0]
self.assertEqual(
command,
["docker", "compose", "rm", "--stop", "--force", "external-data-plane"],
)
self.assertNotIn("--volumes", command)
def test_failed_first_edp_activation_removes_containers_without_deleting_volume(self):
with tempfile.TemporaryDirectory(prefix="nodedc-platform-edp-rollback-") as directory:
temporary = Path(directory)
root = temporary / "root"
backup = temporary / "backup"
runner_tmp = temporary / "tmp"
overlay = root / RUNNER.PLATFORM_EXTERNAL_DATA_PLANE_COMPOSE_REL
overlay.parent.mkdir(parents=True)
backup.mkdir()
runner_tmp.mkdir()
entries = [RUNNER.PLATFORM_EXTERNAL_DATA_PLANE_COMPOSE_REL]
RUNNER.create_backup(root, backup, entries, include_nginx_html=False)
overlay.write_text("name: nodedc-platform\n", encoding="utf-8")
with (
mock.patch.object(RUNNER, "TMP_DIR", runner_tmp),
mock.patch.object(RUNNER, "stop_and_remove_compose_services") as remove,
mock.patch.object(RUNNER, "run_component_runtime") as runtime,
mock.patch.object(RUNNER, "run_healthchecks") as healthchecks,
):
status = RUNNER.rollback_platform_apply(
root,
backup,
entries,
"test-stamp",
runtime_started=True,
applied_services=("external-data-plane",),
)
remove.assert_called_once_with(
"platform",
("external-data-plane",),
)
runtime.assert_not_called()
healthchecks.assert_not_called()
self.assertFalse(overlay.exists())
self.assertEqual(status, "source+runtime-restored:1")
def test_candidate_cleanup_failure_still_restores_source_before_reporting_failure(self):
with tempfile.TemporaryDirectory(prefix="nodedc-platform-edp-rollback-") as directory:
temporary = Path(directory)
root = temporary / "root"
backup = temporary / "backup"
runner_tmp = temporary / "tmp"
overlay = root / RUNNER.PLATFORM_EXTERNAL_DATA_PLANE_COMPOSE_REL
overlay.parent.mkdir(parents=True)
backup.mkdir()
runner_tmp.mkdir()
entries = [RUNNER.PLATFORM_EXTERNAL_DATA_PLANE_COMPOSE_REL]
RUNNER.create_backup(root, backup, entries, include_nginx_html=False)
overlay.write_text("name: rejected\n", encoding="utf-8")
with (
mock.patch.object(RUNNER, "TMP_DIR", runner_tmp),
mock.patch.object(
RUNNER,
"stop_and_remove_compose_services",
side_effect=RuntimeError("simulated cleanup failure"),
),
):
with self.assertRaisesRegex(RUNNER.DeployError, "cleanup failed after source restore"):
RUNNER.rollback_platform_apply(
root,
backup,
entries,
"test-stamp",
runtime_started=True,
applied_services=("external-data-plane",),
)
self.assertFalse(overlay.exists())
def test_runtime_failure_rebuilds_recreates_and_accepts_restored_overlay(self):
with tempfile.TemporaryDirectory(prefix="nodedc-platform-runtime-rollback-") as directory:
temporary = Path(directory)
root = temporary / "root"
backup = temporary / "backup"
runner_tmp = temporary / "tmp"
gateway = root / "platform/gelios-gateway"
gateway.mkdir(parents=True)
backup.mkdir()
runner_tmp.mkdir()
source = gateway / "server.mjs"
source.write_text("export const version = 1;\n", encoding="utf-8")
entries = ["platform/gelios-gateway"]
RUNNER.create_backup(root, backup, entries, include_nginx_html=False)
source.write_text("export const version = 2;\n", encoding="utf-8")
def assert_restored_runtime(component, rollback_entries, services):
self.assertEqual(component, "platform")
self.assertEqual(rollback_entries, entries)
self.assertEqual(services, ("gelios-postgres", "gelios-gateway"))
self.assertEqual(source.read_text(encoding="utf-8"), "export const version = 1;\n")
with (
mock.patch.object(RUNNER, "TMP_DIR", runner_tmp),
mock.patch.object(
RUNNER,
"run_component_runtime",
side_effect=assert_restored_runtime,
) as runtime,
mock.patch.object(RUNNER, "run_healthchecks") as healthchecks,
):
status = RUNNER.rollback_platform_apply(
root,
backup,
entries,
"test-stamp",
runtime_started=True,
applied_services=("gelios-postgres", "gelios-gateway"),
)
self.assertEqual(status, "source+runtime-restored:1")
runtime.assert_called_once()
healthchecks.assert_called_once_with(
"platform",
entries,
("gelios-postgres", "gelios-gateway"),
)
def test_existing_edp_rollback_accepts_only_restored_edp(self):
with tempfile.TemporaryDirectory(prefix="nodedc-platform-edp-runtime-rollback-") as directory:
temporary = Path(directory)
root = temporary / "root"
backup = temporary / "backup"
runner_tmp = temporary / "tmp"
service = root / "platform/services/external-data-plane"
service.mkdir(parents=True)
backup.mkdir()
runner_tmp.mkdir()
source = service / "server.mjs"
source.write_text("export const version = 1;\n", encoding="utf-8")
entries = ["platform/services/external-data-plane"]
RUNNER.create_backup(root, backup, entries, include_nginx_html=False)
source.write_text("export const version = 2;\n", encoding="utf-8")
with (
mock.patch.object(RUNNER, "TMP_DIR", runner_tmp),
mock.patch.object(RUNNER, "run_component_runtime") as runtime,
mock.patch.object(RUNNER, "healthcheck_url") as healthcheck,
):
status = RUNNER.rollback_platform_apply(
root,
backup,
entries,
"test-stamp",
runtime_started=True,
applied_services=("external-data-plane",),
)
self.assertEqual(status, "source+runtime-restored:1")
self.assertEqual(
source.read_text(encoding="utf-8"),
"export const version = 1;\n",
)
runtime.assert_called_once_with(
"platform",
entries,
("external-data-plane",),
)
healthcheck.assert_called_once_with(
RUNNER.external_data_plane_healthcheck(require_managed=False),
)
def test_restores_existing_overlay_and_removes_new_paths(self):
with tempfile.TemporaryDirectory(prefix="nodedc-platform-rollback-test-") as directory:
temporary = Path(directory)
root = temporary / "root"
backup = temporary / "backup"
runner_tmp = temporary / "tmp"
root.mkdir()
backup.mkdir()
runner_tmp.mkdir()
compose = root / "platform/docker-compose.platform-http.yml"
gateway = root / "platform/gelios-gateway"
new_service = root / "platform/services/new-service"
unrelated = root / "platform/unrelated.txt"
compose.parent.mkdir(parents=True)
gateway.mkdir()
compose.write_text("name: nodedc-platform\n", encoding="utf-8")
executable = gateway / "run.sh"
executable.write_text("#!/bin/sh\nexit 0\n", encoding="utf-8")
executable.chmod(0o755)
(gateway / "server.mjs").write_text("export const version = 1;\n", encoding="utf-8")
unrelated.write_text("untouched\n", encoding="utf-8")
entries = [
"platform/docker-compose.platform-http.yml",
"platform/gelios-gateway",
"platform/services/new-service",
]
RUNNER.create_backup(root, backup, entries, include_nginx_html=False)
compose.write_text("name: broken\n", encoding="utf-8")
(gateway / "server.mjs").write_text("export const version = 2;\n", encoding="utf-8")
executable.chmod(0o644)
new_service.mkdir(parents=True)
(new_service / "bad.mjs").write_text("broken\n", encoding="utf-8")
with mock.patch.object(RUNNER, "TMP_DIR", runner_tmp):
restored = RUNNER.restore_platform_overlay(root, backup, entries, "test-stamp")
self.assertEqual(restored, len(entries))
self.assertEqual(compose.read_text(encoding="utf-8"), "name: nodedc-platform\n")
self.assertEqual(
(gateway / "server.mjs").read_text(encoding="utf-8"),
"export const version = 1;\n",
)
self.assertTrue(stat.S_IMODE((gateway / "run.sh").stat().st_mode) & stat.S_IXUSR)
self.assertFalse(new_service.exists())
self.assertEqual(unrelated.read_text(encoding="utf-8"), "untouched\n")
def test_rejects_inconsistent_backup_partition_before_restore(self):
with tempfile.TemporaryDirectory(prefix="nodedc-platform-rollback-test-") as directory:
temporary = Path(directory)
root = temporary / "root"
backup = temporary / "backup"
runner_tmp = temporary / "tmp"
target = root / "platform/docker-compose.platform-http.yml"
target.parent.mkdir(parents=True)
target.write_text("original\n", encoding="utf-8")
backup.mkdir()
runner_tmp.mkdir()
entries = ["platform/docker-compose.platform-http.yml"]
RUNNER.create_backup(root, backup, entries, include_nginx_html=False)
(backup / "missing-files.txt").write_text(
"platform/docker-compose.platform-http.yml\n",
encoding="utf-8",
)
target.write_text("candidate\n", encoding="utf-8")
with mock.patch.object(RUNNER, "TMP_DIR", runner_tmp):
with self.assertRaisesRegex(RUNNER.DeployError, "backup entry set mismatch"):
RUNNER.restore_platform_overlay(root, backup, entries, "test-stamp")
self.assertEqual(target.read_text(encoding="utf-8"), "candidate\n")
if __name__ == "__main__":
unittest.main(verbosity=2)

View File

@ -192,9 +192,8 @@ services:
retries: 3 retries: 3
start_period: 10s start_period: 10s
# External-provider telemetry is isolated from Platform identity state. The # Frozen legacy Gelios compatibility contour. Keep it reproducible and
# database is not published; only the Gateway is exposed on the local host # isolated; new provider integrations use the provider-neutral data plane.
# for operator checks and Engine development.
gelios-postgres: gelios-postgres:
image: ${GELIOS_TIMESCALE_IMAGE:-timescale/timescaledb-ha:pg16.14-ts2.28.2-all} image: ${GELIOS_TIMESCALE_IMAGE:-timescale/timescaledb-ha:pg16.14-ts2.28.2-all}
restart: unless-stopped restart: unless-stopped

View File

@ -97,9 +97,20 @@ EXTERNAL_DATA_PLANE_WRITER_BINDING_MAX_TTL_DAYS=90
EXTERNAL_DATA_PLANE_MAX_FUTURE_SKEW_SECONDS=300 EXTERNAL_DATA_PLANE_MAX_FUTURE_SKEW_SECONDS=300
EXTERNAL_DATA_PLANE_RETENTION_SWEEP_MS=3600000 EXTERNAL_DATA_PLANE_RETENTION_SWEEP_MS=3600000
EXTERNAL_DATA_PLANE_LEGACY_INTAKE_ENABLED=false EXTERNAL_DATA_PLANE_LEGACY_INTAKE_ENABLED=false
# The writer-provisioner secret is a root-owned file under # Internal control-plane writer/reader binding issuance; keep false until the
# /volume1/docker/nodedc-platform/secrets/external-data-plane-provisioner/, # atomic NDC L2 ensure-grant operation is deployed. Legacy path returns a
# never a shared .env value. # plaintext capability and must never be exposed to users.
EXTERNAL_DATA_PLANE_PROVISIONING_ENABLED=false
# Digest-only managed writer-binding ensure. It accepts only signed Engine
# service requests; the legacy provisioner bearer is deliberately invalid here.
# Keep false until the matching Engine private key is provisioned. The trust
# directory is mounted read-only into EDP and contains only `public-key.pem`.
EXTERNAL_DATA_PLANE_MANAGED_PROVISIONING_ENABLED=false
EXTERNAL_DATA_PLANE_MANAGED_PROVISIONER_SERVICE_ID=nodedc-engine
EXTERNAL_DATA_PLANE_MANAGED_PROVISIONER_KEY_ID=engine-edp-managed-provisioner-v1
EXTERNAL_DATA_PLANE_MANAGED_PROVISIONER_AUDIENCE=nodedc-external-data-plane.managed-provisioning.v1
EXTERNAL_DATA_PLANE_MANAGED_PROVISIONER_MAX_SKEW_SECONDS=60
EXTERNAL_DATA_PLANE_MANAGED_PROVISIONER_REPLAY_CACHE_MAX_ENTRIES=10000
NOTIFICATION_PG_DB=nodedc_notifications NOTIFICATION_PG_DB=nodedc_notifications
NOTIFICATION_PG_USER=nodedc_notifications NOTIFICATION_PG_USER=nodedc_notifications
@ -132,8 +143,9 @@ AI_WORKSPACE_ONTOLOGY_MCP_ENABLED=true
AI_WORKSPACE_ONTOLOGY_MCP_PUBLIC_URL= AI_WORKSPACE_ONTOLOGY_MCP_PUBLIC_URL=
ONTOLOGY_CORE_HOST_BIND=127.0.0.1:18104 ONTOLOGY_CORE_HOST_BIND=127.0.0.1:18104
# Gelios Gateway — storage/read service. Provider credentials stay in the # Gelios Gateway — frozen legacy storage/read compatibility service. Provider
# protected Engine Collector. Intake remains disabled until scope is approved. # credentials stay in the protected Engine Collector. Do not extend this
# provider-specific contour for new providers.
GELIOS_TIMESCALE_IMAGE=timescale/timescaledb-ha:pg16.14-ts2.28.2-all GELIOS_TIMESCALE_IMAGE=timescale/timescaledb-ha:pg16.14-ts2.28.2-all
GELIOS_PG_DB=nodedc_gelios GELIOS_PG_DB=nodedc_gelios
GELIOS_PG_USER=nodedc_gelios GELIOS_PG_USER=nodedc_gelios
@ -141,8 +153,6 @@ GELIOS_PG_PASS=replace-with-random-synology-secret
# URL-encode reserved characters in GELIOS_PG_PASS when forming this URL. # URL-encode reserved characters in GELIOS_PG_PASS when forming this URL.
GELIOS_DATABASE_URL=postgresql://nodedc_gelios:replace-with-url-encoded-synology-secret@gelios-postgres:5432/nodedc_gelios GELIOS_DATABASE_URL=postgresql://nodedc_gelios:replace-with-url-encoded-synology-secret@gelios-postgres:5432/nodedc_gelios
GELIOS_GATEWAY_HOST_BIND=127.0.0.1:18105 GELIOS_GATEWAY_HOST_BIND=127.0.0.1:18105
# Explicit tenant and connection identifiers are deployment configuration;
# do not encode a customer or pilot name in source defaults.
GELIOS_TENANT_ID=replace-with-tenant-id GELIOS_TENANT_ID=replace-with-tenant-id
GELIOS_CONNECTION_ID=gelios-connection-id GELIOS_CONNECTION_ID=gelios-connection-id
# `all` accepts every unit returned by this approved tenant + connection. # `all` accepts every unit returned by this approved tenant + connection.

View File

@ -202,7 +202,13 @@ TASKER_SYNC_SOURCE=1 ./infra/synology/deploy-current.sh
Если emergency-fix был сделан прямо на Synology в этих env-файлах, перенести sanitized-значение в `.env.synology.example`/docs, а секрет оставить только в live env. Если emergency-fix был сделан прямо на Synology в этих env-файлах, перенести sanitized-значение в `.env.synology.example`/docs, а секрет оставить только в live env.
### Gelios: сбор всех units доверенного подключения ### Gelios: frozen legacy compatibility contour
`gelios-postgres` и `gelios-gateway` — уже существующий self-hosted контур с
собственным persistent Timescale volume. Он остаётся полностью воспроизводимым
в source/deploy, но не развивается и не используется как шаблон для новых
провайдеров. Его удаление или миграция требуют отдельного подтверждённого плана;
обычный Platform/EDP deploy не удаляет volume и не изменяет старый workflow.
`GELIOS_UNIT_SCOPE=all` означает **все текущие и будущие units только одной уже `GELIOS_UNIT_SCOPE=all` означает **все текущие и будущие units только одной уже
настроенной пары `GELIOS_TENANT_ID` + `GELIOS_CONNECTION_ID`**. Это не wildcard настроенной пары `GELIOS_TENANT_ID` + `GELIOS_CONNECTION_ID`**. Это не wildcard
@ -210,17 +216,71 @@ TASKER_SYNC_SOURCE=1 ./infra/synology/deploy-current.sh
Пропавший из очередного ответа unit не удаляется: его stable provider ID и Пропавший из очередного ответа unit не удаляется: его stable provider ID и
`last_seen_at` сохраняются; видимость на карте — отдельная логика витрины. `last_seen_at` сохраняются; видимость на карте — отдельная логика витрины.
Перед каноническим `nodedc-deploy apply` включить политику на Synology (скрипт Перед узким legacy apply включить политику на Synology (скрипт создаёт backup и
создаёт backup и не выводит секреты): не выводит секреты):
```bash ```bash
sudo bash /volume1/docker/nodedc-deploy/inbox/prepare-gelios-all-units-env.sh sudo bash /volume1/docker/nodedc-deploy/inbox/prepare-gelios-all-units-env.sh
``` ```
Затем применить узкий Platform-артефакт, собранный с `--gateway-only`, через В plan такого Platform-артефакта должны быть только `gelios-postgres` и
`nodedc-deploy`. В его plan должны быть только `gelios-postgres` и `gelios-gateway`; общий compose в узкий архив не входит.
`gelios-gateway`; общий `docker-compose` в такой архив не входит. После apply
Gateway начинает принимать весь состав units этого подключения. ## EDP publish grant: platform-owned credential
`NDC Data Product Writer API` — историческое имя native credential type для
внутреннего scoped EDP publish grant. Это не Gelios access/refresh token и не
право записи в Gelios.
Целевой путь полностью служебный: control plane NDC по разрешённому connection
profile генерирует capability внутри native NDC L2 Credentials, передаёт EDP
только digest для scoped binding и возвращает MCP только opaque compatible
reference/status.
Пользователь не читает, не копирует и не вводит этот внутренний секрет.
EDP уже имеет localhost-only digest-only managed endpoint. Managed endpoint и
legacy plaintext provisioning имеют раздельные флаги и по умолчанию выключены.
Runner-owned provisioner bearer относится только к legacy plaintext routes и
на managed ensure/revoke намеренно не действует. Managed caller contract —
Ed25519-signed Engine service request с exact audience, method, request target,
raw-body SHA-256, timestamp и одноразовым nonce; EDP хранит только public key и
fail-closed replay cache. Ни bearer, ни signing private key нельзя открывать
workflow, MCP или переиспользовать как shared credential. В source-кандидате
Engine добавлен отдельный server-derived plan/apply для exact пары
`ndcDataProductWriterApi` + custom Publish node; он не расширяет generic HTTP
safe-ref policy и не принимает provider/scope/credential identity, capability,
generation или service URL от MCP. До отдельного Engine deploy и runtime proof
текущий deployed MCP этого контракта не имеет. Поэтому отсутствие
совместимого reference — платформенный runtime gap, а не действие пользователя.
Ручной перенос capability из отдельно включаемого legacy endpoint через root/UI допустим только как аварийная
диагностика self-hosted контура; он не является acceptance-путём и не должен
закрепляться в пользовательской автоматизации.
Compose монтирует read-only trust directory
`/volume1/docker/nodedc-platform/trust/engine-managed-provisioner` в EDP; в нём
ожидается один regular, non-symlink файл `public-key.pem` с Ed25519 SPKI public
key. Файл не должен быть group/world-writable и должен читаться runtime UID
`11006`. Private key в этот каталог, Platform artifact, `.env` или EDP
контейнер не попадает. Canonical deploy runner теперь атомарно создаёт или
проверяет matching Ed25519 pair перед relevant Engine/EDP Compose apply: private
key остаётся `root:root 0400`, public trust — `root:11006 0440`; public-only
crash state, mismatch, symlink и неверный тип ключа отклоняются до запуска.
`EXTERNAL_DATA_PLANE_MANAGED_PROVISIONING_ENABLED` остаётся `false` на staging.
В отдельно подтверждённом activation window его переводят в `true`
непосредственно перед Platform EDP artifact; runner принимает deploy только
если `/healthz` явно вернул managed status `enabled`. Старый Engine в этот
момент ещё не имеет signer mount, поэтому usable caller появляется лишь после
следующего отдельно принятого Engine artifact.
EDP application artifact пересобирает и force-recreate только
`external-data-plane`. Уже работающий `external-data-plane-postgres` и его
Timescale volume runner не выбирает и не перезапускает; healthy database —
обязательная deploy prerequisite.
Canonical service/key/audience заданы в `.env.synology.example`; wire headers и
canonical signing payload описаны в `packages/external-provider-contract/README.md`.
До появления opaque compatible reference и успешного manual publish proof
Schedule Trigger не включать. Frozen legacy workflow и его credentials этот
процесс не изменяет.
## AI Hub relay-only deploy ## AI Hub relay-only deploy

View File

@ -38,7 +38,7 @@ echo "== ontology core image build =="
cd "${PLATFORM_DIR}/ontology-core" cd "${PLATFORM_DIR}/ontology-core"
"${DOCKER_BIN}" build --no-cache -t nodedc/ontology-core:local . "${DOCKER_BIN}" build --no-cache -t nodedc/ontology-core:local .
echo "== gelios gateway image build ==" echo "== frozen legacy gelios gateway image build =="
cd "${PLATFORM_DIR}/gelios-gateway" cd "${PLATFORM_DIR}/gelios-gateway"
"${DOCKER_BIN}" build --no-cache -t nodedc/gelios-gateway:local . "${DOCKER_BIN}" build --no-cache -t nodedc/gelios-gateway:local .
@ -53,7 +53,7 @@ echo "== ontology core health check =="
"${DOCKER_BIN}" exec nodedc-platform-ontology-core-1 sh -lc \ "${DOCKER_BIN}" exec nodedc-platform-ontology-core-1 sh -lc \
'node -e '"'"'fetch("http://127.0.0.1:18104/healthz").then(async (response) => { console.log(await response.text()); process.exit(response.ok ? 0 : 1); }).catch((error) => { console.error(error); process.exit(1); })'"'"'' 'node -e '"'"'fetch("http://127.0.0.1:18104/healthz").then(async (response) => { console.log(await response.text()); process.exit(response.ok ? 0 : 1); }).catch((error) => { console.error(error); process.exit(1); })'"'"''
echo "== gelios gateway health check ==" echo "== frozen legacy gelios gateway health check =="
"${DOCKER_BIN}" exec nodedc-platform-gelios-gateway-1 sh -lc \ "${DOCKER_BIN}" exec nodedc-platform-gelios-gateway-1 sh -lc \
'node -e '"'"'fetch("http://127.0.0.1:18105/healthz").then(async (response) => { console.log(await response.text()); process.exit(response.ok ? 0 : 1); }).catch((error) => { console.error(error); process.exit(1); })'"'"'' 'node -e '"'"'fetch("http://127.0.0.1:18105/healthz").then(async (response) => { console.log(await response.text()); process.exit(response.ok ? 0 : 1); }).catch((error) => { console.error(error); process.exit(1); })'"'"''

View File

@ -51,12 +51,23 @@ services:
# Migration-only shared-token routes stay closed in a canonical install. # Migration-only shared-token routes stay closed in a canonical install.
EXTERNAL_DATA_PLANE_LEGACY_INTAKE_ENABLED: ${EXTERNAL_DATA_PLANE_LEGACY_INTAKE_ENABLED:-false} EXTERNAL_DATA_PLANE_LEGACY_INTAKE_ENABLED: ${EXTERNAL_DATA_PLANE_LEGACY_INTAKE_ENABLED:-false}
NODEDC_INTERNAL_ACCESS_TOKEN: ${NODEDC_INTERNAL_ACCESS_TOKEN:-} NODEDC_INTERNAL_ACCESS_TOKEN: ${NODEDC_INTERNAL_ACCESS_TOKEN:-}
# Runner-owned secret file. This credential must never live in the # Legacy manual one-time issuance uses this EDP-only bearer. Managed
# shared .env.synology that is inherited by unrelated services. # Engine ensure/revoke never accepts it and uses the Ed25519 boundary
# below. Neither credential is exposed to a graph, MCP client or provider.
EXTERNAL_DATA_PLANE_PROVISIONER_TOKEN_FILE: /run/nodedc-secrets/external-data-plane-provisioner-token EXTERNAL_DATA_PLANE_PROVISIONER_TOKEN_FILE: /run/nodedc-secrets/external-data-plane-provisioner-token
# This stays false until the dedicated Engine provisioner is deployed and # Disabled until the atomic native NDC L2 ensure-grant control-plane
# receives the same read-only secret mount. # operation is present. This legacy API returns plaintext capabilities;
EXTERNAL_DATA_PLANE_PROVISIONING_ENABLED: "false" # a root/UI transfer is emergency diagnostics only.
EXTERNAL_DATA_PLANE_PROVISIONING_ENABLED: ${EXTERNAL_DATA_PLANE_PROVISIONING_ENABLED:-false}
# Digest-only, idempotent writer-binding ensure path. Enable only after
# Engine owns the matching private key and the public trust file exists.
EXTERNAL_DATA_PLANE_MANAGED_PROVISIONING_ENABLED: ${EXTERNAL_DATA_PLANE_MANAGED_PROVISIONING_ENABLED:-false}
EXTERNAL_DATA_PLANE_MANAGED_PROVISIONER_PUBLIC_KEY_FILE: /run/nodedc-trust/engine-managed-provisioner/public-key.pem
EXTERNAL_DATA_PLANE_MANAGED_PROVISIONER_SERVICE_ID: ${EXTERNAL_DATA_PLANE_MANAGED_PROVISIONER_SERVICE_ID:-nodedc-engine}
EXTERNAL_DATA_PLANE_MANAGED_PROVISIONER_KEY_ID: ${EXTERNAL_DATA_PLANE_MANAGED_PROVISIONER_KEY_ID:-engine-edp-managed-provisioner-v1}
EXTERNAL_DATA_PLANE_MANAGED_PROVISIONER_AUDIENCE: ${EXTERNAL_DATA_PLANE_MANAGED_PROVISIONER_AUDIENCE:-nodedc-external-data-plane.managed-provisioning.v1}
EXTERNAL_DATA_PLANE_MANAGED_PROVISIONER_MAX_SKEW_SECONDS: ${EXTERNAL_DATA_PLANE_MANAGED_PROVISIONER_MAX_SKEW_SECONDS:-60}
EXTERNAL_DATA_PLANE_MANAGED_PROVISIONER_REPLAY_CACHE_MAX_ENTRIES: ${EXTERNAL_DATA_PLANE_MANAGED_PROVISIONER_REPLAY_CACHE_MAX_ENTRIES:-10000}
volumes: volumes:
- type: bind - type: bind
source: /volume1/docker/nodedc-platform/secrets/external-data-plane-provisioner/token source: /volume1/docker/nodedc-platform/secrets/external-data-plane-provisioner/token
@ -64,6 +75,12 @@ services:
read_only: true read_only: true
bind: bind:
create_host_path: false create_host_path: false
- type: bind
source: /volume1/docker/nodedc-platform/trust/engine-managed-provisioner
target: /run/nodedc-trust/engine-managed-provisioner
read_only: true
bind:
create_host_path: false
expose: expose:
- "18106" - "18106"
ports: ports:

View File

@ -184,8 +184,8 @@ services:
networks: networks:
- engine - engine
# The provider database is private. Gateway is the only component that can # Frozen legacy Gelios compatibility contour. Keep it reproducible and
# reach it; the host bind is loopback-only for audited operator diagnostics. # isolated; new provider integrations use the provider-neutral data plane.
gelios-postgres: gelios-postgres:
image: ${GELIOS_TIMESCALE_IMAGE:-timescale/timescaledb-ha:pg16.14-ts2.28.2-all} image: ${GELIOS_TIMESCALE_IMAGE:-timescale/timescaledb-ha:pg16.14-ts2.28.2-all}
restart: unless-stopped restart: unless-stopped

View File

@ -67,7 +67,7 @@ echo "== ontology core health check =="
"${DOCKER_BIN}" exec nodedc-platform-ontology-core-1 sh -lc \ "${DOCKER_BIN}" exec nodedc-platform-ontology-core-1 sh -lc \
'node -e '"'"'fetch("http://127.0.0.1:18104/healthz").then(async (response) => { console.log(await response.text()); process.exit(response.ok ? 0 : 1); }).catch((error) => { console.error(error); process.exit(1); })'"'"'' 'node -e '"'"'fetch("http://127.0.0.1:18104/healthz").then(async (response) => { console.log(await response.text()); process.exit(response.ok ? 0 : 1); }).catch((error) => { console.error(error); process.exit(1); })'"'"''
echo "== gelios gateway health check ==" echo "== frozen legacy gelios gateway health check =="
"${DOCKER_BIN}" exec nodedc-platform-gelios-gateway-1 sh -lc \ "${DOCKER_BIN}" exec nodedc-platform-gelios-gateway-1 sh -lc \
'node -e '"'"'fetch("http://127.0.0.1:18105/healthz").then(async (response) => { console.log(await response.text()); process.exit(response.ok ? 0 : 1); }).catch((error) => { console.error(error); process.exit(1); })'"'"'' 'node -e '"'"'fetch("http://127.0.0.1:18105/healthz").then(async (response) => { console.log(await response.text()); process.exit(response.ok ? 0 : 1); }).catch((error) => { console.error(error); process.exit(1); })'"'"''

View File

@ -5,7 +5,7 @@
исполняемый connector code. `src/index.mjs` даёт dependency-free проверку исполняемый connector code. `src/index.mjs` даёт dependency-free проверку
минимальных v1 contracts; она запускается через `npm run check`. минимальных v1 contracts; она запускается через `npm run check`.
Каждый L2 connector instance должен поставлять совместимые versioned Каждый NDC L2 connector instance должен поставлять совместимые versioned
артефакты: артефакты:
```text ```text
@ -21,16 +21,47 @@ command-catalog (metadata only until a red command gateway is approved)
``` ```
`provider-manifest` — декларативный template-артефакт: provider ID, ontology `provider-manifest` — декларативный template-артефакт: provider ID, ontology
revision, L2 template version, capability catalog и data-product IDs. Он не revision, NDC L2 template version, capability catalog и data-product IDs. Он не
является tenant connection, не содержит endpoint/URL, credential reference, является tenant connection, не содержит endpoint/URL, credential reference,
secret, ручной scope или executable provider code. Concrete non-secret secret, ручной scope или executable provider code. Concrete non-secret
connection profile принадлежит и версионируется вместе с конкретным L2 connection profile принадлежит и версионируется вместе с конкретным NDC L2
workflow; Engine только привязывает к нему opaque credential reference и grant. workflow; caller привязывает только opaque provider credential reference, а
внутренний workload grant разрешает control plane.
## Versioned provider packages
`providers/<provider>/<major>` — устанавливаемая contract/data единица, а не
runtime service и не custom node. Она объединяет manifest, auth-mode metadata,
capability catalog, field policies, collection profiles, Data Product
definitions, semantic mapping contracts, provider-neutral NDC L2 template
descriptor и synthetic fixtures. Package не содержит tenant/account state,
credential values или исполняемый provider daemon.
`instantiateL2Connection` создаёт произвольное число connection instances из
одного immutable package. Каждый instance получает собственные tenant,
connection и opaque provider credential reference, но не требует изменения
Platform source. Внутренние workload bindings объявляются отдельно как
`control_plane_managed` и не являются caller input. Канонический collection
scope — `all_visible_to_credential`: выбранная
read-capability передаёт все entities, которые provider возвращает для
привязанного credential. Unit allowlist и provider-group filter на collection
boundary запрещены; сортировка и видимость принадлежат Data Product consumer и
Foundry.
Первый production-shaped package — `providers/gelios/v1`. Он фиксирует реальный
Gelios REST `GET /api/v1/units` transport contract и точный output
`fleet.positions.current.v1@1.0.0` с revision
`ontology.map.moving_object.v1`. Все Data Product fields используют snake_case.
Его optional `tokenLifecycle` точно описывает два provider-issued artifacts —
`access` и `refresh`: request использует `access`, а refresh пока имеет режим
`operator_managed`. Это metadata без secret values и без заявления о
реализованном автоматическом refresh.
## Проверяемые v1 contracts ## Проверяемые v1 contracts
- `Connection` — provider instance, tenant scope и *ссылка* на credential в - `Connection` — provider instance, tenant scope и *ссылка* на provider
Engine. Любые token/secret/password-like поля запрещены. credential в NDC L2 Credentials. Любые token/secret/password-like поля
запрещены; publisher представлен system-managed declaration/status без ref.
- `Collection Profile` — явная policy сбора. `manual` не может скрыто содержать - `Collection Profile` — явная policy сбора. `manual` не может скрыто содержать
polling interval; `realtime` требует interval не чаще одного раза в секунду. polling interval; `realtime` требует interval не чаще одного раза в секунду.
- `Data Product` — нормализованный versioned output с semantic types, полями и - `Data Product` — нормализованный versioned output с semantic types, полями и
@ -38,18 +69,18 @@ workflow; Engine только привязывает к нему opaque credenti
- `Intake Batch` — canonical **scoped** record, который External Data Plane - `Intake Batch` — canonical **scoped** record, который External Data Plane
валидирует и сохраняет: source, contract revision, idempotency, restricted валидирует и сохраняет: source, contract revision, idempotency, restricted
raw envelope и canonical facts. Его `source` содержит `providerId`, raw envelope и canonical facts. Его `source` содержит `providerId`,
`tenantId` и `connectionId`. Writer-bound L2 request намеренно не является `tenantId` и `connectionId`. Writer-bound NDC L2 request намеренно не является
готовым `Intake Batch`: Data Plane сначала materializes scope и лишь затем готовым `Intake Batch`: Data Plane сначала materializes scope и лишь затем
применяет этот contract. В canonical record нет provider field mapping, применяет этот contract. В canonical record нет provider field mapping,
entity allowlist, token или renderer data. Inline `raw.payload` в v1 entity allowlist, token или renderer data. Inline `raw.payload` в v1
запрещён: если нужна provenance-ссылка, connector передаёт restricted запрещён: если нужна provenance-ссылка, connector передаёт restricted
`raw.ref` вместе с hash. Отдельный raw-vault может быть добавлен только `raw.ref` вместе с hash. Отдельный raw-vault может быть добавлен только
отдельным ADR и не становится частью L2 → Data Plane wire boundary. отдельным ADR и не становится частью NDC L2 → Data Plane wire boundary.
- `NDC Foundry Binding` — адресует data product только в конкретную цепочку - `NDC Foundry Binding` — адресует data product только в конкретную цепочку
`Foundry Application → Page → approved slot`; `templateId` можно сохранить `Foundry Application → Page → approved slot`; `templateId` можно сохранить
как дополнительную типизацию, но он не заменяет `applicationId` и `pageId`. как дополнительную типизацию, но он не заменяет `applicationId` и `pageId`.
В binding запрещены provider transport, endpoint и credential reference. В binding запрещены provider transport, endpoint и credential reference.
- `Provider Manifest` — статическое описание L2 connector template, capability - `Provider Manifest` — статическое описание NDC L2 connector template, capability
catalog и ontology/data-product contracts. Оно не может содержать tenant, catalog и ontology/data-product contracts. Оно не может содержать tenant,
connection, credential, secret или provider transport. connection, credential, secret или provider transport.
@ -94,13 +125,14 @@ PostgreSQL `integer` `0..2147483647`. Эти ограничения нельзя
Manifest, connection/collection profiles, data-product definition и Foundry Manifest, connection/collection profiles, data-product definition и Foundry
binding используют fail-closed allowed-key schemas. Неизвестные поля, а также binding используют fail-closed allowed-key schemas. Неизвестные поля, а также
secret-like имена или значения (включая `ndc_edpwb_`/`ndc_edprb_`) отклоняются secret-like имена или значения (включая `ndc_edpwb_`, `ndc_edprb_` и
`ndc_edppr_`) отклоняются
на общей границе. на общей границе.
Все private custom nodes NODE.DC поставляются package Все private custom nodes NODE.DC поставляются package
`platform/packages/n8n-nodes-ndc`. Их display name обязан начинаться с `NDC `, `platform/packages/n8n-nodes-ndc`. Их display name обязан начинаться с `NDC `,
а runtime type — с `n8n-nodes-ndc.`; package называется строго а runtime type — с `n8n-nodes-ndc.`; package называется строго
`n8n-nodes-ndc`. Provider-specific adapters остаются L2 workflow logic и не `n8n-nodes-ndc`. Provider-specific adapters остаются NDC L2 workflow logic и не
становятся custom nodes или ветками Data Plane. Эти инварианты проверяются становятся custom nodes или ветками Data Plane. Эти инварианты проверяются
package test. package test.
@ -110,76 +142,72 @@ declarative provider artifact и не runtime transport: команда соде
application/page/binding/data-product projection и idempotency key, а право на application/page/binding/data-product projection и idempotency key, а право на
операцию извлекается Foundry из отдельного opaque workload grant. операцию извлекается Foundry из отдельного opaque workload grant.
## Engine opaque credential sink ## Native NDC L2 Credentials
`src/engine-credential-sink.mjs` задаёт dependency-free server-to-server v1 Provider auth и внутренние workload capabilities используют один общий
границу для доставки трёх workload capabilities в Engine Credentials: секретный boundary — native Credentials NDC L2, — но являются разными
credential domains. После сохранения ядро владеет secret, а graph и MCP
используют только opaque reference.
- `external-data-plane.writer` → точная нода Gelios выдаёт ровно access token и refresh token. Текущий credential type
`n8n-nodes-ndc.ndcDataProductPublish` / `ndcDataProductWriterApi`; `httpBearerAuth` использует access token для HTTP request. Автоматический обмен
- `external-data-plane.reader` → точная нода refresh → access в текущем runtime не доказан и поэтому не заявлен: package
`n8n-nodes-ndc.ndcDataProductRead` / `ndcDataProductReaderApi`; фиксирует `refreshMode: operator_managed`. Название credential с текстом вроде
- `foundry.binding` → точная нода `read access` является лишь локальной меткой; read-классификацию задают
`n8n-nodes-ndc.ndcFoundryBinding` / `ndcFoundryBindingApi`. разрешённые endpoint/method в capability catalog и workflow policy, а не scope
самого access token. В deployed Engine exact method/path policy ещё не
подключена: текущая HTTP safe-ref граница проверяет host. Поэтому catalog
classification остаётся декларативной до capability-bound Engine/MCP proof.
Provision request фиксирует `workflowId`, `workflowRevision`, `nodeId`, runtime Для общего Data Product transport используются `ndcDataProductWriterApi`,
node type, credential type, grant ID, expiry и issuer policy hash. Aggregate `ndcDataProductReaderApi` и `ndcFoundryBindingApi`. Это внутренние NDC
`transaction.policyHash` детерминированно считается по всему secret-free capabilities, не Gelios tokens и не часть Gelios token lifecycle. Их значения
descriptor; подмена любой цели или policy ломает валидацию. Единственное поле, не являются частью provider package и не передаются между workflow nodes как
которое переносит plaintext capability, — `bindings[].material.value`; request данные.
нельзя писать в логи, traces, очередь или audit.
Provision и rollback envelopes действуют не более 15 минут: sink отклоняет Connection caller передаёт только provider credential reference. Publisher
истёкшие запросы и допускает максимум 60 секунд положительного clock skew. role остаётся credential binding для `NDC Data Product Publish`, но template
Receipt повторно проверяет freshness относительно собственного `processedAt`, маркирует его `management: control_plane_managed`: ни writer secret, ни writer
чтобы старый запрос нельзя было применить или подтвердить через replay. reference не входят в connection parameters. Instantiation возвращает
декларативный `systemBindings.publisher` с desired state и unresolved status;
дальше его разрешает trusted control plane.
Каждый binding содержит `capabilityDigest = sha256(material.value)`: Engine Canonical managed EDP writer capability генерируется и сохраняется внутри
самостоятельно хеширует полученный plaintext и сравнивает digest. Aggregate Engine credential boundary, а EDP получает только digest. Старые manual
`policyHash` включает этот digest и issuer identity, а provision transaction writer/reader и Foundry issuance paths могут возвращать capability один раз
несёт обязательную Ed25519 attestation. Engine принимает её только по trusted control-plane caller и остаются отдельно закрытыми compatibility
allowlisted `serviceId:keyId`; заменить capability и пересчитать обычный hash границами; MCP и пользователь получают только opaque reference/status.
без приватного issuer key невозможно. Запрещены password input
как пользовательский acceptance-путь, graph parameters, connection/profile
files, env, Ops, logs и traces.
Sink обязан выполнять `rollback-all`: сначала проверить весь request и точное Broad credential sink/resolver/daemon и provider-specific provisioning
состояние graph, затем создать credentials в staging, атомарно привязать весь запрещены. Требуется узкий generic `ensure data-product publish grant` contract
набор и только после commit вернуть opaque `credentialRef`. При любой ошибке с server-derived scope, idempotency, CAS/crash recovery и audit. Текущий
новые credentials удаляются, а прежние bindings остаются без изменений. deployed Engine MCP этой операции ещё не имеет; root/UI transfer допустим
`rollback-failed` означает карантин и ручное восстановление, но никогда не только как emergency self-hosted diagnostics, не как user journey.
возвращает частичные credential refs.
Receipt, rollback request/receipt и audit имеют отдельные strict schemas. Они ## NDC L2 private-extension management
не способны вернуть capability material; audit хранит только hash opaque
credential reference. Explicit rollback адресует committed transaction через
`transactionId`, `policyHash` и hash committed receipt, поэтому не может
случайно откатить другой набор. Реализация sink принадлежит Engine и не даёт
Platform/Codex доступа к Engine core, runtime files или plaintext credential
storage.
## Engine private-extension management
`src/engine-private-extension.mjs` задаёт строгую Platform-side границу для `src/engine-private-extension.mjs` задаёт строгую Platform-side границу для
Engine-owned активации проверенного `n8n-nodes-ndc` release. Контракт не NDC L2-owned активации проверенного `n8n-nodes-ndc` release. Контракт не
устанавливает package и не меняет Engine: он фиксирует async устанавливает package и не меняет NDC L2: он фиксирует async
`plan -> apply receipt -> operation/status` protocol, где `apply` обязан `plan -> apply receipt -> operation/status` protocol, где `apply` обязан
быстро вернуть `queued` и `operationId`, а долгий recreate/acceptance быстро вернуть `queued` и `operationId`, а долгий recreate/acceptance
отслеживается отдельно. отслеживается отдельно.
Активация и rollback требуют отдельной глобальной capability Активация и rollback требуют отдельной глобальной capability
`engine.private-extension.manage`. Обычные L1/L2 grants её не дают. Чтение `engine.private-extension.manage`. Обычные NDC L1/NDC L2 grants её не дают. Чтение
состояния допускает `engine.private-extension.read` или manage-capability. состояния допускает `engine.private-extension.read` или manage-capability.
Запрос выбирает только allowlisted package, digest-bound `releaseId` и Запрос выбирает только allowlisted package, digest-bound `releaseId` и
`packageSha256`; caller не передаёт host path, package bytes, Compose service, `packageSha256`; caller не передаёт host path, package bytes, Compose service,
shell command или credential material. Plan живёт не более 15 минут, является shell command или credential material. Plan живёт не более 15 минут, является
single-use и применяется только с тем же idempotency key и plan hash. single-use и применяется только с тем же idempotency key и plan hash.
Runtime transition для n8n 2.3.2 зафиксирован как community-package loader по Runtime transition использует platform-managed community-package loader для
`/home/node/.n8n/nodes/node_modules/n8n-nodes-ndc`, а не как immutable `n8n-nodes-ndc` release; private runtime paths и environment settings
`N8N_CUSTOM_EXTENSIONS` или `CUSTOM.*` loader: не являются частью публичного provider contract:
- `N8N_COMMUNITY_PACKAGES_ENABLED=true`,
`N8N_COMMUNITY_PACKAGES_PREVENT_LOADING=false`,
`N8N_REINSTALL_MISSING_PACKAGES=false`;
- Deploy/Run quiesced and execution queue drained before the version switch; - Deploy/Run quiesced and execution queue drained before the version switch;
- read-only mount and atomic current/recovery state; - read-only mount and atomic current/recovery state;
- force-recreate main, every worker and every webhook instance as one version - force-recreate main, every worker and every webhook instance as one version
@ -189,24 +217,25 @@ Runtime transition для n8n 2.3.2 зафиксирован как community-pa
Любая ошибка после switch запускает automatic rollback и повторный Любая ошибка после switch запускает automatic rollback и повторный
force-recreate/acceptance. Ошибка самого rollback переводит runtime в force-recreate/acceptance. Ошибка самого rollback переводит runtime в
`quarantined`. Immutable release и существующие Engine Credentials `quarantined`. Immutable release и существующие NDC L2 Credentials
сохраняются. Для первой активации предыдущим проверенным состоянием является сохраняются. Для первой активации предыдущим проверенным состоянием является
`n8n-nodes-ndc.inactive/v1`: rollback в этот baseline удаляет package из `n8n-nodes-ndc.inactive/v1`: rollback в этот baseline удаляет package из
loader surface, но не удаляет credentials. loader surface, но не удаляет credentials.
Public Ops gateway уже умеет прозрачно передавать эти операции через Public Ops gateway уже умеет прозрачно передавать эти операции через
`/engine/mcp`, если Engine реализует соответствующие MCP tools. Так как `/engine/mcp`, если NDC L2 реализует соответствующие MCP tools. Так как
gateway имеет 30-second upstream timeout, side effect остаётся асинхронным; gateway имеет 30-second upstream timeout, side effect остаётся асинхронным;
отдельный public REST proxy для management boundary не требуется. отдельный public REST proxy для management boundary не требуется.
Пример `examples/gelios-positions-current.v1.mjs` — provider-specific fixture `providers/gelios/v1/fixtures` содержит synthetic provider response и ожидаемый
без customer, tenant identity или credential material. publish contract без customer, tenant identity или credential material.
## Ownership ## Ownership
- `platform/packages/external-provider-contract` — общий контракт и schemas. - `platform/packages/external-provider-contract` — общий контракт и schemas.
- NDC Agent L2 — provider API adapter: fetch, pagination, batching, - NDC L2 — provider API adapter: fetch, pagination, batching,
semantic mapping, collection profile и ссылка на credential в Engine. semantic mapping, collection profile и ссылка на credential в NDC L2
Credentials.
- Platform External Data Plane — provider-neutral intake, raw retention, - Platform External Data Plane — provider-neutral intake, raw retention,
canonical facts, current/history projections и scoped read products. Он не canonical facts, current/history projections и scoped read products. Он не
знает provider fields, customer/business filters или renderer rules. знает provider fields, customer/business filters или renderer rules.
@ -222,46 +251,100 @@ gateway имеет 30-second upstream timeout, side effect остаётся ас
## Mandatory connection boundary ## Mandatory connection boundary
`connection` принадлежит одному tenant/client context и содержит только `connection` принадлежит одному tenant/client context и содержит только
ссылку на секрет, утверждённый capability scope, collection profile, field ссылку на provider secret, утверждённый capability scope, collection profile,
policy и retention policy. Во всех runtime records Data Plane сохраняет минимум field policy и retention policy. Во всех runtime records Data Plane сохраняет минимум
`tenant_id`, `connection_id`, `provider_id`, `observed_at`, `received_at` и `tenant_id`, `connection_id`, `provider_id`, `observed_at`, `received_at` и
provenance/version там, где это применимо. provenance/version там, где это применимо.
Writer token — отдельный EDP runtime credential, а не поле `Connection`, Provider credential reference текущего Gelios request адресует access token;
profile, provider manifest или L2 graph. наличие provider-issued refresh token описывается `tokenLifecycle`, но его value
не входит в `Connection`. Writer capability — отдельный внутренний EDP runtime
credential, а не Gelios token и не caller-supplied поле `Connection`, profile,
provider manifest или NDC L2 graph. Connection artifact содержит только
system-managed publisher declaration/status без credential reference.
## Scoped writer binding ## Scoped writer binding
Writer binding — EDP-owned runtime security state, а не versioned artifact Writer binding — EDP-owned runtime security state, а не versioned artifact
provider manifest или connection profile. Он фиксирует `tenantId`, provider manifest или connection profile. Он фиксирует `tenantId`,
`connectionId`, `providerId`, `allowedDataProductIds`, active/revoked state и `connectionId`, `providerId`, `allowedDataProductIds`, active/revoked state и
`expiresAt`. L2 не может редактировать binding или задавать его scope; `expiresAt`. NDC L2 не может редактировать binding или задавать его scope;
изменение любого scope-поля либо TTL создаёт новый binding, а прежний binding изменение любого scope-поля либо TTL создаёт новый binding, а прежний binding
можно только rotate/revoke. можно только rotate/revoke.
Новый L2 вызывает `POST /internal/data-plane/v1/intake/writer-bound` с Binding создаёт, сохраняет и привязывает только trusted control plane. Caller
`Authorization: Bearer <writer-token>` и unscoped envelope. В `source` connection instance не принимает `writerCredentialRef`; publisher role в L2
разрешён только `providerId`; `tenantId`, `connectionId` и template является обязательным `control_plane_managed` binding requirement.
`x-nodedc-*-id` headers запрещены. EDP проверяет token, binding, provider и
`contract.dataProductId`, затем materializes immutable canonical scope и
сохраняет обычный `Intake Batch`. Caller-provided scope отклоняется, а не
доверяется и не объединяется с binding.
Plaintext writer token возвращается только trusted provisioner при создании Новый NDC L2 вызывает
или rotation. Provisioner помещает его непосредственно в opaque Engine `POST /internal/data-plane/v1/data-products/:dataProductId/publish` с opaque
credential, доступный назначенному L2. EDP хранит только hash token и binding writer capability и envelope `nodedc.data-product.publish/v1`. В body есть
metadata; token запрещён в connection/profile/manifest, L2 graph, logs/traces, только batch identity и canonical facts; `source`, provider, tenant,
raw payload, Foundry и UI. connection, ontology revision, product version и persistence policy запрещены.
EDP проверяет binding и path `dataProductId`, materializes immutable scope и
contract из server-owned registries, затем сохраняет canonical batch.
Caller-provided scope отклоняется, а не доверяется и не объединяется с
binding.
Provisioning routes принимают только отдельный secret file, созданный Plaintext writer capability запрещена в provider package, connection/profile,
root-owned deploy runner: NDC L2 graph, files, environment, logs/traces, raw payload, Foundry и MCP.
`/volume1/docker/nodedc-platform/secrets/external-data-plane-provisioner/token`. В canonical пути она генерируется и сохраняется внутри native Engine credential
Он read-only монтируется только в EDP и позднее — в dedicated Engine boundary; EDP получает только SHA-256 digest, а workflow — только opaque
provisioner, работающий под выделенным UID/GID `11006`; это не `.env` value, reference/status. Старые ручные EDP create/rotate endpoints с one-time
не `NODEDC_INTERNAL_ACCESS_TOKEN` и не provider credential. Пока generic Engine plaintext response остаются отдельно выключенным compatibility-механизмом и не
provisioner не развёрнут, `EXTERNAL_DATA_PLANE_PROVISIONING_ENABLED=false`, а являются пользовательским или acceptance-путём.
create/rotate/revoke routes отвечают `503` и не делают fallback к shared
internal bearer. Legacy manual one-time issuance использует отдельный EDP-only runner-managed
bearer. Digest-only managed ensure этот bearer не принимает: Engine подписывает
каждый control-plane запрос отдельным Ed25519 service key, EDP хранит только
публичный ключ. Оба пути имеют независимые флаги и по умолчанию выключены.
Generic `ensure data-product publish grant` генерирует и сохраняет capability в
native Engine credential boundary, передаёт EDP её digest и возвращает только
opaque ref/status. Shared internal bearer и Gelios access/refresh token для
этого запрещены. Deployed Engine MCP умеет читать
opaque refs и bind только HTTP Request credentials; native
`ndcDataProductWriterApi` к custom Publish node этим policy не привязывается.
Поэтому ensure + exact private-node bind остаются реальным platform gap.
Root/UI перенос — только аварийная диагностика.
Control-plane EDP boundary — idempotent
`PUT /internal/data-plane/v1/writer-bindings/by-key/:bindingKey`. Engine создаёт
capability внутри native credential boundary и отправляет только её SHA-256
`capabilityDigest`, immutable scope, `generation` и expiry. Одинаковый
`bindingKey + generation + request hash` возвращает тот же binding без нового
secret; несовпадающий retry получает `409`. Rotation использует новую
generation/credential, поэтому старый binding можно оставить активным до
успешного node bind и затем явно revoke.
Managed revoke выполняется идемпотентно по точным `bindingKey + generation`
через managed-only route; для него не открывается legacy plaintext API.
Managed route включается только через
`EXTERNAL_DATA_PLANE_MANAGED_PROVISIONING_ENABLED`; legacy plaintext routes —
через отдельный `EXTERNAL_DATA_PLANE_PROVISIONING_ENABLED`.
Managed auth wire contract —
`nodedc.external-data-plane.managed-provisioner-request/v1`. Engine отправляет
ровно по одному header:
`x-nodedc-engine-service-id`, `x-nodedc-engine-key-id`,
`x-nodedc-request-audience`, `x-nodedc-request-timestamp`,
`x-nodedc-request-nonce`, `x-nodedc-content-sha256` и
`x-nodedc-request-signature`. Timestamp — canonical UTC ISO с миллисекундами,
nonce и signature — unpadded canonical base64url, body hash — lowercase hex
SHA-256 от точных переданных bytes. Подписываются UTF-8 bytes результата
`JSON.stringify` объекта с полями строго в порядке `schemaVersion`, `audience`,
`serviceId`, `keyId`, `method`, `path`, `timestamp`, `nonce`, `bodySha256`.
`method` и request target `path` (включая query) должны совпадать побайтно.
Canonical identity: service `nodedc-engine`, key
`engine-edp-managed-provisioner-v1`, audience
`nodedc-external-data-plane.managed-provisioning.v1`. EDP проверяет bounded
clock skew и одноразовый nonce в bounded fail-closed replay cache; bearer
fallback для managed routes отсутствует, любой `Authorization` header на них
отклоняется.
EDP импортирует только `@nodedc/external-provider-contract/data-plane`.
Artifact/image allowlist для этого subpath содержит contract version, intake,
publish/snapshot/patch validators и scrub policy; provider packages, mappings,
fixtures и tests физически не входят в EDP runtime и не являются причиной его
restart.
## Legacy intake migration ## Legacy intake migration
@ -271,22 +354,22 @@ existing writers. Он принимает только canonical scoped `Intake
`x-nodedc-tenant-id` и `x-nodedc-connection-id`. Writer token этот route не `x-nodedc-tenant-id` и `x-nodedc-connection-id`. Writer token этот route не
заменяет. заменяет.
Для миграции: создать writer binding, один раз записать выданный token в Engine Для миграции: Platform обеспечивает scoped writer binding и native credential,
credential, переключить L2 на `/intake/writer-bound`, удалить tenant/connection затем через Engine MCP новый workflow branch переключается на Data Product
из body и headers, проверить успешный intake, затем удалить у L2 legacy publish route, удаляет tenant/connection из body и headers и проверяет
credential. Новый или migrated writer не должен fallback-иться на legacy route успешный publish. Существующий legacy workflow/credential остаётся неизменным,
после ошибки writer-bound intake. Когда migrated все writers, legacy route и пока владелец отдельно не решит его вывести; новый writer не должен
его shared-token access удаляются. fallback-иться на legacy route после ошибки publish.
## Intake boundary ## Intake boundary
L2 отправляет в общий Data Plane `Intake Batch`, а не SQL-запрос в общие NDC L2 отправляет в общий Data Plane publish request, а не SQL-запрос в общие
таблицы. Platform проверяет boundary и сохраняет raw/history/current таблицы. Platform проверяет boundary и сохраняет raw/history/current
projections; semantic mapping остаётся в L2. Список конкретных source IDs не projections; semantic mapping остаётся в NDC L2. Список конкретных source IDs не
является частью connection scope: scope выбирает read-capability API, а является частью connection scope: scope выбирает read-capability API, а
visibility решает data-product consumer. visibility решает data-product consumer.
Inline `raw.payload` запрещён: L2 не может записать в Data Plane полный ответ Inline `raw.payload` запрещён: NDC L2 не может записать в Data Plane полный ответ
provider-а или произвольную строку. Пока отдельный raw-vault не утверждён, provider-а или произвольную строку. Пока отдельный raw-vault не утверждён,
batch либо не содержит `raw`, либо содержит только restricted `raw.ref` и hash. batch либо не содержит `raw`, либо содержит только restricted `raw.ref` и hash.
Secret-like keys и распознаваемые bearer/JWT/writer-token values в любом участке Secret-like keys и распознаваемые bearer/JWT/writer-token values в любом участке
@ -299,4 +382,6 @@ service делает sweep expired envelopes при старте и по рас
Capabilities классифицируются как `read`, `metadata`, `write`, `destructive` Capabilities классифицируются как `read`, `metadata`, `write`, `destructive`
или `unknown`. Только `read` и согласованные `metadata` могут попасть в или `unknown`. Только `read` и согласованные `metadata` могут попасть в
collector. `write` и `destructive` остаются каталогизированными, но не имеют collector. `write` и `destructive` остаются каталогизированными, но не имеют
transport route в read adapter. transport route в read adapter. Эта классификация относится к разрешённому
method/endpoint workflow, а не утверждает, что provider access token имеет
read-only scope.

View File

@ -8,15 +8,14 @@ export const geliosPositionsCurrentExample = Object.freeze({
version: "1.0.0", version: "1.0.0",
ontology: { ontology: {
packageId: "gelios", packageId: "gelios",
revision: "gelios.positions.v1", revision: "ontology.gelios.v1",
}, },
l2Template: { l2Template: {
id: "gelios.positions.current", id: "gelios.positions.current.l2.v1",
version: "1.0.0", version: "1.0.0",
}, },
capabilities: [ capabilities: [
{ id: "gelios.units.current.read", classification: "read" }, { id: "gelios.units.current.read", classification: "read" },
{ id: "gelios.units.command.write", classification: "write" },
], ],
dataProductIds: ["fleet.positions.current.v1"], dataProductIds: ["fleet.positions.current.v1"],
}, },
@ -25,10 +24,10 @@ export const geliosPositionsCurrentExample = Object.freeze({
id: "gelios.sample-fleet", id: "gelios.sample-fleet",
providerId: "gelios", providerId: "gelios",
tenantId: "sample-tenant", tenantId: "sample-tenant",
credentialRef: { owner: "engine", reference: "engine-credential-reference" }, credentialRef: { owner: "ndc_l2_credentials", reference: "ndc-credref:provider-example-0001" },
scope: { scope: {
capabilityIds: ["gelios.units.current.read"], capabilityIds: ["gelios.units.current.read"],
fieldPolicyId: "fleet.position.display.v1", fieldPolicyId: "gelios.positions.current.fields.v1",
}, },
}, },
collectionProfile: { collectionProfile: {
@ -36,7 +35,7 @@ export const geliosPositionsCurrentExample = Object.freeze({
id: "gelios.positions.realtime", id: "gelios.positions.realtime",
connectionId: "gelios.sample-fleet", connectionId: "gelios.sample-fleet",
mode: "realtime", mode: "realtime",
schedule: { intervalMs: 15000 }, schedule: { intervalMs: 10000 },
capabilityIds: ["gelios.units.current.read"], capabilityIds: ["gelios.units.current.read"],
dataProductId: "fleet.positions.current.v1", dataProductId: "fleet.positions.current.v1",
}, },

View File

@ -3,8 +3,12 @@
"version": "0.1.0", "version": "0.1.0",
"private": true, "private": true,
"type": "module", "type": "module",
"exports": "./src/index.mjs", "exports": {
".": "./src/index.mjs",
"./data-plane": "./src/data-plane.mjs",
"./providers/*": "./providers/*/index.mjs"
},
"scripts": { "scripts": {
"check": "node test/contract.test.mjs && node test/data-product.test.mjs && node test/engine-credential-sink.test.mjs && node test/engine-private-extension.test.mjs" "check": "node test/contract.test.mjs && node test/data-product.test.mjs && node test/provider-package.test.mjs && node test/engine-private-extension.test.mjs"
} }
} }

View File

@ -0,0 +1,105 @@
# Gelios provider package v1
Это versioned contract/data package для первого живого provider connection в
NODE.DC. Он не является runtime service, custom node, account configuration или
хранилищем telemetry. Provider-specific request и mapping исполняются внутри
изолированного NDC L2 workflow; Platform получает только provider-neutral Data
Product.
Package фиксирует:
- manifest `gelios.provider.v1@1.0.0`;
- точный provider token lifecycle: Gelios выдаёт `access` и `refresh`, а
HTTP request использует `access`;
- safe-read capability `gelios.units.current.read`
(`GET https://api.geliospro.com/api/v1/units` with an opaque bearer
credential);
- dynamic scope `all_visible_to_credential` без unit/group allowlist;
- realtime profile с интервалом 10 секунд и manual profile;
- fail-closed field policy: неизвестные и dynamic fields отбрасываются до
классификации, hardware IDs, phone/address, raw params и sensor payloads не
публикуются;
- mapping в `map.moving_object`;
- стабильный source ID `gelios-unit-<sourceUnitId>` с точным сохранением
строкового provider ID;
- explicit derivation rules для `position_valid`, `operational_status` и
`quality_flags`, включая `no_position` и stale threshold 60 секунд;
- точный Data Product `fleet.positions.current.v1@1.0.0`, revision
`ontology.map.moving_object.v1`, с 13 разрешёнными snake_case fields;
- provider-neutral NDC L2 template descriptor из пяти boundary steps:
trigger → request → extract → map → `NDC Data Product Publish`;
- publish credential binding с `management: control_plane_managed`, без
caller-supplied writer reference;
- synthetic source/publish fixture, включая видимый объект без координат.
## Authentication boundary
Gelios выдаёт ровно два provider secret artifact: access token и refresh
token. Это не отдельные read/write tokens. Текущий NDC L2 HTTP binding типа
`httpBearerAuth` подставляет access token в `Authorization`; автоматический
refresh этим runtime пока не доказан, поэтому package явно фиксирует
`refreshMode: operator_managed`. Ни access, ни refresh value не попадает в
package, graph, fixture, MCP, Ops или trace.
Название native credential, например `Gelios — Robot2B — read access`, является
только операторской меткой. `read` в `gelios.units.current.read`
классификация разрешённого workflow endpoint `GET /api/v1/units`, а не scope
access token. Метка credential не создаёт отдельный «read token» и не сужает
права, которые фактически выдал Gelios.
Unit без last message не исчезает: mapping использует collection receive time
как `observedAt` fallback и публикует subject без geometry с
`position_valid=false`/`operational_status=no_position`.
Префикс source ID для v1 — строго `gelios-unit-`. Сокращённый `unit-` не
является каноническим: он создал бы второй entity key рядом с уже сохранённым
Gelios subject. Значение provider ID после префикса преобразуется только в
строку; ведущие нули и остальные значимые символы не нормализуются.
`fleet.positions.current.v1` ограничен 5000 current entities. Если credential
видит больше, NDC L2 не должен отбрасывать «лишние» units: package/profile
требует остановить Deploy до новой partitioned Data Product revision. Полное
автоматическое enforcement этого правила остаётся gate будущего
package-to-graph compiler/runtime validator.
Один package обслуживает любое число accounts. Для каждого account вызывается
`instantiateL2Connection` с новым `tenantId`, `connectionId`, provider
credential reference. Это единственная credential reference, которую передаёт
caller: она адресует Gelios access binding в native NDC L2 Credentials.
Внутреннюю capability для публикации в External Data Plane выдаёт и сохраняет
control plane; caller не передаёт writer secret или `writerCredentialRef`.
Результат instantiation содержит только provider ref и декларативный
`systemBindings.publisher` со статусом `unresolved`, который Platform должна
разрешить в native credential binding. Эта capability не выдаётся Gelios, не
является третьим Gelios token и не описывает права provider account. Добавление
account не меняет Platform source.
Расширение Gelios происходит новой версией package: сначала подтверждаются
provider capability/fields, затем обновляются ontology, field policy, mapping,
fixtures и tests. Пользовательские workflow, которые уже покрываются
существующим catalog, не требуют участия разработчика Platform.
Технический runtime type publish boundary остаётся
`n8n-nodes-ndc.ndcDataProductPublish`; это идентификатор общего NDC package, а
не Gelios-specific node.
## Текущая готовность
Package, ontology links и synthetic fixtures проверены в source. Production
Engine L2 target использует зарегистрированный Gelios Bearer credential на
точном safe-read endpoint и managed `ndcDataProductWriterApi` generation g3;
оба binding записаны без раскрытия secret values. Canonical graph содержит
provider read, extraction, mapping и native `NDC Data Product Publish`.
Fresh production execution `881722` завершился успешно и опубликовал
`fleet.positions.current.v1`; managed writer g3 принят и имеет state `current`.
Это доказывает manual provider read → canonical Data Product path. Внутренняя
writer capability остаётся NODE.DC capability и не может быть заменена access
или refresh token Gelios.
Не доказаны: автоматический refresh, Schedule Trigger 10 секунд, несколько
последовательных scheduled cycles, retry/backoff, sampled history/retention на
continuous path и Foundry snapshot/patch → moving pins E2E. Provider package
остаётся declarative: MCP package discovery и package/profile → L2 graph
compiler пока отсутствуют, поэтому первый graph собран через общие Engine MCP
graph tools.

View File

@ -0,0 +1,74 @@
import { DATA_PRODUCT_PUBLISH_SCHEMA_VERSION } from "../../../../src/data-product.mjs";
export const geliosUnitsCurrentFixtureV1 = Object.freeze({
sourceResponse: {
units: [{
id: 1001,
name: "Synthetic unit 1001",
hw_id: "restricted-fixture-value",
phone: "restricted-fixture-value",
lastMsg: {
time: 1784203200,
lat: "55.7558",
lon: "37.6173",
speed: 24.5,
course: 91,
height: 164,
gps_valid: true,
sats: 11,
hdop: 0.8,
accuracy: 4.2,
params: "restricted-fixture-value",
},
}, {
id: "001002",
name: "Synthetic unit 1002",
}],
},
collectionContext: {
receivedAt: "2026-07-16T12:00:00.000Z",
},
expectedPublish: {
schemaVersion: DATA_PRODUCT_PUBLISH_SCHEMA_VERSION,
batch: {
runId: "gelios-fixture-run-20260716",
sequence: 0,
idempotencyKey: "gelios-fixture-run-20260716.batch-0",
},
facts: [{
sourceId: "gelios-unit-1001",
semanticType: "map.moving_object",
observedAt: "2026-07-16T12:00:00.000Z",
geometry: {
type: "Point",
coordinates: [37.6173, 55.7558],
},
attributes: {
course_degrees: 91,
display_name: "Synthetic unit 1001",
elevation_meters: 164,
hdop: 0.8,
horizontal_accuracy_meters: 4.2,
object_kind: "tracked_unit",
operational_status: "active",
position_source: "gelios",
position_valid: true,
quality_flags: [],
satellite_count: 11,
speed_kph: 24.5,
},
}, {
sourceId: "gelios-unit-001002",
semanticType: "map.moving_object",
observedAt: "2026-07-16T12:00:00.000Z",
attributes: {
display_name: "Synthetic unit 1002",
object_kind: "tracked_unit",
operational_status: "no_position",
position_source: "gelios",
position_valid: false,
quality_flags: ["missing_geometry"],
},
}],
},
});

View File

@ -0,0 +1,11 @@
export {
GELIOS_POSITIONS_DATA_PRODUCT_ID,
GELIOS_POSITIONS_DATA_PRODUCT_VERSION,
GELIOS_POSITIONS_ONTOLOGY_REVISION,
GELIOS_PROVIDER_PACKAGE_ID,
GELIOS_PROVIDER_PACKAGE_VERSION,
GELIOS_UNIT_SOURCE_ID_PREFIX,
geliosProviderPackageV1,
} from "./package.mjs";
export { geliosUnitsCurrentFixtureV1 } from "./fixtures/units-current.fixture.mjs";

View File

@ -0,0 +1,374 @@
import {
L2_TEMPLATE_SCHEMA_VERSION,
PROVIDER_PACKAGE_SCHEMA_VERSION,
SEMANTIC_MAPPING_SCHEMA_VERSION,
} from "../../../src/provider-package.mjs";
export const GELIOS_PROVIDER_PACKAGE_ID = "gelios.provider.v1";
export const GELIOS_PROVIDER_PACKAGE_VERSION = "1.0.0";
export const GELIOS_POSITIONS_DATA_PRODUCT_ID = "fleet.positions.current.v1";
export const GELIOS_POSITIONS_DATA_PRODUCT_VERSION = "1.0.0";
export const GELIOS_POSITIONS_ONTOLOGY_REVISION = "ontology.map.moving_object.v1";
export const GELIOS_UNIT_SOURCE_ID_PREFIX = "gelios-unit-";
const targetFields = Object.freeze([
"course_degrees",
"display_name",
"elevation_meters",
"geometry",
"hdop",
"horizontal_accuracy_meters",
"object_kind",
"operational_status",
"position_source",
"position_valid",
"quality_flags",
"satellite_count",
"speed_kph",
]);
export const geliosProviderPackageV1 = deepFreeze({
schemaVersion: PROVIDER_PACKAGE_SCHEMA_VERSION,
id: GELIOS_PROVIDER_PACKAGE_ID,
providerId: "gelios",
version: GELIOS_PROVIDER_PACKAGE_VERSION,
manifest: {
id: "gelios.provider.manifest.v1",
providerId: "gelios",
version: GELIOS_PROVIDER_PACKAGE_VERSION,
ontology: {
packageId: "gelios",
revision: "ontology.gelios.v1",
},
authModeIds: ["gelios.rest-bearer.v1"],
capabilityIds: ["gelios.units.current.read"],
fieldPolicyIds: ["gelios.positions.current.fields.v1"],
collectionProfileIds: [
"gelios.positions.current.realtime.v1",
"gelios.positions.current.manual.v1",
],
dataProductIds: [GELIOS_POSITIONS_DATA_PRODUCT_ID],
mappingContractIds: ["gelios.units.to.fleet.positions.current.v1"],
l2TemplateIds: ["gelios.positions.current.l2.v1"],
},
authModes: [{
id: "gelios.rest-bearer.v1",
kind: "api_token",
credentialOwner: "ndc_l2_credentials",
bindingCardinality: "one_per_connection",
transport: {
placement: "header",
name: "Authorization",
},
tokenLifecycle: {
artifacts: ["access", "refresh"],
requestArtifact: "access",
refreshMode: "operator_managed",
},
secretHandling: {
store: "ndc_l2_credentials",
exposeToGraph: false,
persistInPackage: false,
},
}],
capabilities: [{
id: "gelios.units.current.read",
classification: "read",
status: "implemented",
authModeId: "gelios.rest-bearer.v1",
request: {
method: "GET",
baseUrl: "https://api.geliospro.com",
path: "/api/v1/units",
query: {},
response: {
collectionPaths: ["units", "data", "data.units", "result", "$"],
pagination: "single_bounded_response",
},
},
entityScope: {
mode: "all_visible_to_credential",
refresh: "each_collection_run",
businessEntityFilter: "forbidden",
},
}],
fieldPolicies: [{
id: "gelios.positions.current.fields.v1",
version: "1.0.0",
dataProductId: GELIOS_POSITIONS_DATA_PRODUCT_ID,
targetFields: [...targetFields],
unknownSourceFields: "drop",
dynamicSourceFields: "drop_until_classified",
restrictedSourcePaths: [
"decrypt_key",
"hw_id",
"imei",
"info",
"lmsg.address",
"lmsg.params",
"lmsg.sensors",
"lastMsg.address",
"lastMsg.params",
"lastMsg.sensors",
"lastMessage.address",
"lastMessage.params",
"lastMessage.sensors",
"phone",
"phone2",
"sensors",
],
}],
collectionProfiles: [{
id: "gelios.positions.current.realtime.v1",
version: "1.0.0",
mode: "realtime",
schedule: { intervalMs: 10000 },
capabilityIds: ["gelios.units.current.read"],
dataProductId: GELIOS_POSITIONS_DATA_PRODUCT_ID,
mappingContractId: "gelios.units.to.fleet.positions.current.v1",
fieldPolicyId: "gelios.positions.current.fields.v1",
l2TemplateId: "gelios.positions.current.l2.v1",
entityScope: {
mode: "all_visible_to_credential",
refresh: "each_collection_run",
businessEntityFilter: "forbidden",
},
batching: { maxFacts: 5000 },
cardinality: {
maxCurrentEntities: 5000,
onExceed: "require_partitioned_data_product",
},
retry: {
maxAttempts: 4,
backoff: "exponential_with_jitter",
},
}, {
id: "gelios.positions.current.manual.v1",
version: "1.0.0",
mode: "manual",
capabilityIds: ["gelios.units.current.read"],
dataProductId: GELIOS_POSITIONS_DATA_PRODUCT_ID,
mappingContractId: "gelios.units.to.fleet.positions.current.v1",
fieldPolicyId: "gelios.positions.current.fields.v1",
l2TemplateId: "gelios.positions.current.l2.v1",
entityScope: {
mode: "all_visible_to_credential",
refresh: "each_collection_run",
businessEntityFilter: "forbidden",
},
batching: { maxFacts: 5000 },
cardinality: {
maxCurrentEntities: 5000,
onExceed: "require_partitioned_data_product",
},
retry: {
maxAttempts: 4,
backoff: "exponential_with_jitter",
},
}],
dataProducts: [{
id: GELIOS_POSITIONS_DATA_PRODUCT_ID,
version: GELIOS_POSITIONS_DATA_PRODUCT_VERSION,
ontologyRevision: GELIOS_POSITIONS_ONTOLOGY_REVISION,
deliveryMode: "snapshot+patch",
semanticTypes: ["map.moving_object"],
fields: [...targetFields],
history: {
mode: "sampled",
intervalMs: 60000,
strategy: "latest-per-entity-per-bucket",
retentionDays: 90,
},
}],
mappingContracts: [{
schemaVersion: SEMANTIC_MAPPING_SCHEMA_VERSION,
id: "gelios.units.to.fleet.positions.current.v1",
version: "1.0.0",
sourceCapabilityId: "gelios.units.current.read",
fieldPolicyId: "gelios.positions.current.fields.v1",
target: {
dataProductId: GELIOS_POSITIONS_DATA_PRODUCT_ID,
version: GELIOS_POSITIONS_DATA_PRODUCT_VERSION,
ontologyRevision: GELIOS_POSITIONS_ONTOLOGY_REVISION,
semanticType: "map.moving_object",
},
derivations: {
position_valid: {
kind: "boolean_rule",
rules: [
"geometry.coordinates_finite_wgs84",
"provider.gps_valid_not_false",
],
default: false,
},
operational_status: {
kind: "ordered_rules",
rules: [
"position_invalid.no_position",
"observed_age_gt_limit.stale",
"otherwise.active",
],
default: "active",
parameters: { staleAfterMs: 60000 },
},
quality_flags: {
kind: "flag_set",
rules: [
"geometry_missing.missing_geometry",
"provider_gps_false.gps_invalid",
"satellite_count_zero.no_satellites",
"observed_age_gt_limit.stale",
],
default: [],
parameters: { staleAfterMs: 60000 },
},
},
fact: {
sourceId: {
strategy: "first_non_empty",
paths: ["id", "unit_id", "unitId"],
coerce: "string",
prefix: GELIOS_UNIT_SOURCE_ID_PREFIX,
},
semanticType: { constant: "map.moving_object" },
observedAt: {
strategy: "first_non_empty",
paths: [
"lmsg.time",
"lmsg.ts",
"lmsg.timestamp",
"last_msg.time",
"last_msg.timestamp",
"lastMsg.time",
"lastMessage.time",
"last_msg_time",
"lmsg_time",
],
coerce: "unix_or_iso_timestamp",
fallback: "collection_received_at",
},
geometry: {
type: "Point",
longitude: {
strategy: "first_non_empty",
paths: ["lmsg.lon", "lmsg.lng", "lmsg.longitude", "last_msg.lon", "lastMsg.lon", "lastMsg.lng", "lastMsg.longitude", "lastMessage.lon", "lastMessage.lng", "lastMessage.longitude", "lon", "lng", "longitude"],
coerce: "number",
},
latitude: {
strategy: "first_non_empty",
paths: ["lmsg.lat", "lmsg.latitude", "last_msg.lat", "lastMsg.lat", "lastMsg.latitude", "lastMessage.lat", "lastMessage.latitude", "lat", "latitude"],
coerce: "number",
},
omitIfInvalid: true,
},
attributes: {
course_degrees: {
strategy: "first_non_empty",
paths: ["lmsg.course", "lmsg.angle", "last_msg.course", "lastMsg.course", "lastMsg.angle", "lastMessage.course", "lastMessage.angle", "course"],
coerce: "number",
omitIfMissing: true,
},
display_name: {
strategy: "first_non_empty",
paths: ["name", "unit_name", "title", "label"],
coerce: "string",
},
elevation_meters: {
strategy: "first_non_empty",
paths: ["lmsg.height", "lmsg.altitude", "last_msg.height", "lastMsg.height", "lastMsg.altitude", "lastMessage.height", "lastMessage.altitude", "height", "altitude"],
coerce: "number",
omitIfMissing: true,
},
hdop: {
strategy: "first_non_empty",
paths: ["lmsg.hdop", "last_msg.hdop", "lastMsg.hdop", "lastMessage.hdop", "gps_hdop", "hdop"],
coerce: "number",
omitIfMissing: true,
},
horizontal_accuracy_meters: {
strategy: "first_non_empty",
paths: ["lmsg.accuracy", "last_msg.accuracy", "lastMsg.accuracy", "lastMessage.accuracy", "gps_accuracy_m", "accuracy_m", "accuracy"],
coerce: "number",
omitIfMissing: true,
},
object_kind: { constant: "tracked_unit" },
operational_status: { derive: "operational_status" },
position_source: { constant: "gelios" },
position_valid: { derive: "position_valid" },
quality_flags: { derive: "quality_flags" },
satellite_count: {
strategy: "first_non_empty",
paths: ["lmsg.sats", "lmsg.satellites", "last_msg.sats", "lastMsg.sats", "lastMsg.satellites", "lastMessage.sats", "lastMessage.satellites", "gps_sats", "sats", "satellites"],
coerce: "number",
omitIfMissing: true,
},
speed_kph: {
strategy: "first_non_empty",
paths: ["lmsg.speed", "last_msg.speed", "lastMsg.speed", "lastMessage.speed", "speed"],
coerce: "number",
omitIfMissing: true,
},
},
},
}],
l2Templates: [{
schemaVersion: L2_TEMPLATE_SCHEMA_VERSION,
id: "gelios.positions.current.l2.v1",
version: "1.0.0",
runtime: "ndc_l2",
instanceMode: "one_connection_per_workflow",
connectionParameters: [
"tenant_id",
"connection_id",
"collection_profile_id",
"provider_credential_ref",
],
credentialBindings: [{
role: "provider",
owner: "ndc_l2_credentials",
management: "connection_input",
authModeId: "gelios.rest-bearer.v1",
}, {
role: "publisher",
owner: "ndc_l2_credentials",
management: "control_plane_managed",
nodeType: "n8n-nodes-ndc.ndcDataProductPublish",
}],
steps: [{
id: "collection.trigger",
kind: "collection_trigger",
collectionProfileDriven: true,
}, {
id: "provider.fetch",
kind: "provider_request",
capabilityId: "gelios.units.current.read",
}, {
id: "provider.extract-items",
kind: "extract_items",
capabilityId: "gelios.units.current.read",
}, {
id: "ontology.map",
kind: "semantic_mapping",
mappingContractId: "gelios.units.to.fleet.positions.current.v1",
}, {
id: "data-product.publish",
kind: "data_product_publish",
dataProductId: GELIOS_POSITIONS_DATA_PRODUCT_ID,
nodeType: "n8n-nodes-ndc.ndcDataProductPublish",
}],
invariants: {
oneConnectionPerInstance: true,
providerSecretByReferenceOnly: true,
publisherBindingControlPlaneManaged: true,
callerScopeForbidden: true,
providerSpecificServiceForbidden: true,
},
}],
});
function deepFreeze(value) {
if (!value || typeof value !== "object" || Object.isFrozen(value)) return value;
Object.freeze(value);
for (const child of Object.values(value)) deepFreeze(child);
return value;
}

View File

@ -0,0 +1 @@
export const EXTERNAL_PROVIDER_CONTRACT_VERSION = "nodedc.external-provider-contract/v1";

View File

@ -0,0 +1,10 @@
export { EXTERNAL_PROVIDER_CONTRACT_VERSION } from "./contract-version.mjs";
export { validateIntakeBatch } from "./intake-batch.mjs";
export {
DATA_PRODUCT_PATCH_SCHEMA_VERSION,
DATA_PRODUCT_PUBLISH_SCHEMA_VERSION,
DATA_PRODUCT_SNAPSHOT_SCHEMA_VERSION,
validateDataProductPatch,
validateDataProductPublish,
validateDataProductSnapshot,
} from "./data-product.mjs";

View File

@ -2,11 +2,11 @@ export const DATA_PRODUCT_PUBLISH_SCHEMA_VERSION = "nodedc.data-product.publish/
export const DATA_PRODUCT_SNAPSHOT_SCHEMA_VERSION = "nodedc.data-product.snapshot/v1"; export const DATA_PRODUCT_SNAPSHOT_SCHEMA_VERSION = "nodedc.data-product.snapshot/v1";
export const DATA_PRODUCT_PATCH_SCHEMA_VERSION = "nodedc.data-product.patch/v1"; export const DATA_PRODUCT_PATCH_SCHEMA_VERSION = "nodedc.data-product.patch/v1";
import { SECRET_LIKE_KEY, SECRET_LIKE_VALUE } from "./sensitive-field-policy.mjs";
const IDENTIFIER = /^[a-z][a-z0-9._:-]{2,127}$/; const IDENTIFIER = /^[a-z][a-z0-9._:-]{2,127}$/;
const SEMVER = /^\d+\.\d+\.\d+(?:[-+][a-z0-9.-]+)?$/i; const SEMVER = /^\d+\.\d+\.\d+(?:[-+][a-z0-9.-]+)?$/i;
const CURSOR = /^(?:0|[1-9]\d*)$/; const CURSOR = /^(?:0|[1-9]\d*)$/;
const SECRET_LIKE_KEY = /(token|secret|password|authorization|access[_-]?token|refresh[_-]?token|api[_-]?key)/i;
const SECRET_LIKE_VALUE = /(?:ndc_edp(?:wb|rb)_[A-Za-z0-9_-]*|[?&](?:token|secret|password|authorization|access[_-]?token|refresh[_-]?token|api[_-]?key)=|(?:bearer|basic)\s+\S+|eyJ[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+)/i;
const MAX_BATCH_SEQUENCE = 2_147_483_647; const MAX_BATCH_SEQUENCE = 2_147_483_647;
const MAX_FACT_ATTRIBUTES_BYTES = 64 * 1024; const MAX_FACT_ATTRIBUTES_BYTES = 64 * 1024;

View File

@ -1,662 +0,0 @@
import { createHash, verify as verifySignature } from "node:crypto";
export const ENGINE_CREDENTIAL_SINK_PROVISION_SCHEMA_VERSION = "nodedc.engine.credential-sink.provision/v1";
export const ENGINE_CREDENTIAL_SINK_RECEIPT_SCHEMA_VERSION = "nodedc.engine.credential-sink.receipt/v1";
export const ENGINE_CREDENTIAL_SINK_ROLLBACK_SCHEMA_VERSION = "nodedc.engine.credential-sink.rollback/v1";
export const ENGINE_CREDENTIAL_SINK_ROLLBACK_RECEIPT_SCHEMA_VERSION = "nodedc.engine.credential-sink.rollback-receipt/v1";
export const ENGINE_CREDENTIAL_SINK_AUDIT_SCHEMA_VERSION = "nodedc.engine.credential-sink.audit/v1";
const HASH = /^sha256:[a-f0-9]{64}$/;
const IDENTIFIER = /^[a-z][a-z0-9._:-]{2,127}$/;
const OPAQUE_ID = /^[A-Za-z0-9][A-Za-z0-9._:-]{2,159}$/;
const NODE_TYPE = /^[a-z][A-Za-z0-9.-]{2,159}$/;
const CREDENTIAL_TYPE = /^[a-z][A-Za-z0-9]{2,127}$/;
const CREDENTIAL_REFERENCE = /^[A-Za-z0-9][A-Za-z0-9_-]{5,159}$/;
const REASON_CODE = /^[a-z][a-z0-9_.:-]{2,127}$/;
const ED25519_SIGNATURE = /^[A-Za-z0-9_-]{86}$/;
const SECRET_LIKE_KEY = /(token|secret|password|authorization|access[_-]?token|refresh[_-]?token|api[_-]?key|material|value)/i;
const SECRET_LIKE_VALUE = /(?:ndc_(?:edp(?:wb|rb)|fndbg)_[A-Za-z0-9_-]+|(?:bearer|basic)\s+\S+|eyJ[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+)/i;
const MAX_BINDINGS = 32;
const MAX_REQUEST_LIFETIME_MS = 15 * 60 * 1000;
const MAX_REQUEST_CLOCK_SKEW_MS = 60 * 1000;
const CAPABILITY_SPECS = Object.freeze({
"external-data-plane.writer": Object.freeze({
nodeType: "n8n-nodes-ndc.ndcDataProductPublish",
credentialType: "ndcDataProductWriterApi",
materialPattern: /^ndc_edpwb_[A-Za-z0-9_-]{43}$/,
}),
"external-data-plane.reader": Object.freeze({
nodeType: "n8n-nodes-ndc.ndcDataProductRead",
credentialType: "ndcDataProductReaderApi",
materialPattern: /^ndc_edprb_[A-Za-z0-9_-]{43}$/,
}),
"foundry.binding": Object.freeze({
nodeType: "n8n-nodes-ndc.ndcFoundryBinding",
credentialType: "ndcFoundryBindingApi",
materialPattern: /^ndc_fndbg_[A-Za-z0-9_-]{43}$/,
}),
});
export const ENGINE_CREDENTIAL_SINK_CAPABILITY_TYPES = Object.freeze(Object.keys(CAPABILITY_SPECS));
const PROVISION_KEYS = new Set(["schemaVersion", "transaction", "bindings"]);
const TRANSACTION_KEYS = new Set([
"id",
"idempotencyKey",
"requestedAt",
"requestExpiresAt",
"policyHash",
"failureMode",
"issuer",
"attestation",
]);
const ISSUER_KEYS = new Set(["serviceId", "keyId"]);
const ATTESTATION_KEYS = new Set(["algorithm", "signature"]);
const BINDING_KEYS = new Set([
"bindingId",
"capabilityType",
"grantId",
"target",
"expiresAt",
"policyHash",
"capabilityDigest",
"material",
]);
const TARGET_KEYS = new Set([
"workflowId",
"workflowRevision",
"nodeId",
"nodeType",
"credentialType",
]);
const MATERIAL_KEYS = new Set(["format", "value"]);
const RECEIPT_KEYS = new Set([
"schemaVersion",
"transactionId",
"idempotencyKey",
"outcome",
"policyHash",
"processedAt",
"credentials",
"rollback",
"errorCode",
]);
const RECEIPT_CREDENTIAL_KEYS = new Set([
"bindingId",
"capabilityType",
"grantId",
"target",
"credentialRef",
"expiresAt",
"policyHash",
"capabilityDigest",
"disposition",
]);
const RECEIPT_ROLLBACK_KEYS = new Set(["status", "completedAt"]);
const ROLLBACK_REQUEST_KEYS = new Set(["schemaVersion", "rollback"]);
const ROLLBACK_KEYS = new Set([
"id",
"idempotencyKey",
"transactionId",
"requestedAt",
"requestExpiresAt",
"policyHash",
"committedReceiptHash",
"reasonCode",
]);
const ROLLBACK_RECEIPT_KEYS = new Set([
"schemaVersion",
"rollbackId",
"transactionId",
"outcome",
"policyHash",
"committedReceiptHash",
"processedAt",
"errorCode",
]);
const AUDIT_KEYS = new Set([
"schemaVersion",
"eventId",
"transactionId",
"operationId",
"operation",
"outcome",
"occurredAt",
"policyHash",
"principal",
"targets",
"reasonCode",
]);
const PRINCIPAL_KEYS = new Set(["serviceId", "fingerprint"]);
const AUDIT_TARGET_KEYS = new Set([
"bindingId",
"capabilityType",
"grantId",
"target",
"expiresAt",
"policyHash",
"capabilityDigest",
"credentialRefHash",
]);
/**
* Validates the only request allowed to carry plaintext workload capability
* material across the trusted Platform -> Engine server boundary. Callers and
* receivers must never log, trace, persist or return this request body.
*/
export function validateEngineCredentialSinkProvision(value, { now = Date.now(), issuerPublicKeys } = {}) {
const errors = [];
const nowMs = normalizeNow(now, errors);
if (!isPlainObject(value)) return result(["credentialSinkProvision_must_be_object"]);
if (value.schemaVersion !== ENGINE_CREDENTIAL_SINK_PROVISION_SCHEMA_VERSION) errors.push("schemaVersion_mismatch");
rejectUnknownKeys(value, PROVISION_KEYS, "credentialSinkProvision", errors);
if (!isPlainObject(value.transaction)) {
errors.push("transaction_must_be_object");
} else {
rejectUnknownKeys(value.transaction, TRANSACTION_KEYS, "transaction", errors);
requiredOpaqueId(value.transaction.id, "transaction.id", errors);
requiredOpaqueId(value.transaction.idempotencyKey, "transaction.idempotencyKey", errors);
requiredTimestamp(value.transaction.requestedAt, "transaction.requestedAt", errors);
requiredTimestamp(value.transaction.requestExpiresAt, "transaction.requestExpiresAt", errors);
requiredHash(value.transaction.policyHash, "transaction.policyHash", errors);
if (value.transaction.failureMode !== "rollback-all") errors.push("transaction.failureMode_must_be_rollback-all");
validateIssuer(value.transaction.issuer, "transaction.issuer", errors);
validateAttestation(value.transaction.attestation, "transaction.attestation", errors);
validateRequestWindow(value.transaction.requestedAt, value.transaction.requestExpiresAt, nowMs, errors);
}
if (!Array.isArray(value.bindings) || value.bindings.length === 0) {
errors.push("bindings_must_be_nonempty_array");
} else if (value.bindings.length > MAX_BINDINGS) {
errors.push("bindings_limit_exceeded");
} else {
const bindingIds = new Set();
const targets = new Set();
value.bindings.forEach((binding, index) => {
validateProvisionBinding(binding, index, value.transaction?.requestedAt, errors);
if (!isPlainObject(binding)) return;
if (bindingIds.has(binding.bindingId)) errors.push("bindings_bindingId_must_be_unique");
bindingIds.add(binding.bindingId);
const targetKey = targetIdentity(binding.target);
if (targetKey && targets.has(targetKey)) errors.push("bindings_target_credential_must_be_unique");
if (targetKey) targets.add(targetKey);
});
}
if (isPlainObject(value.transaction) && HASH.test(String(value.transaction.policyHash || ""))) {
const expectedHash = computeEngineCredentialSinkPolicyHash(value);
if (value.transaction.policyHash !== expectedHash) errors.push("transaction.policyHash_mismatch");
verifyProvisionAttestation(value.transaction, issuerPublicKeys, errors);
}
return result(errors);
}
/**
* Computes the aggregate policy digest over the complete secret-free request
* descriptor. Plaintext `material` is excluded by construction, while its
* high-entropy capability digest is included; changing a target, grant,
* capability, expiry, individual policy hash or transaction envelope changes
* the aggregate digest and invalidates the issuer attestation.
*/
export function computeEngineCredentialSinkPolicyHash(value) {
const descriptor = {
schemaVersion: value?.schemaVersion,
transaction: isPlainObject(value?.transaction) ? {
id: value.transaction.id,
idempotencyKey: value.transaction.idempotencyKey,
requestedAt: value.transaction.requestedAt,
requestExpiresAt: value.transaction.requestExpiresAt,
failureMode: value.transaction.failureMode,
issuer: value.transaction.issuer,
} : value?.transaction,
bindings: Array.isArray(value?.bindings)
? value.bindings.map(secretFreeBindingDescriptor)
: value?.bindings,
};
return `sha256:${createHash("sha256").update(stableJson(descriptor), "utf8").digest("hex")}`;
}
/**
* Receipt is intentionally incapable of carrying credential material. When a
* request is supplied, the validator also proves the sink committed the exact
* requested workflow/node/type set without target substitution.
*/
export function validateEngineCredentialSinkReceipt(value, { request, issuerPublicKeys } = {}) {
const errors = [];
if (!isPlainObject(value)) return result(["credentialSinkReceipt_must_be_object"]);
if (value.schemaVersion !== ENGINE_CREDENTIAL_SINK_RECEIPT_SCHEMA_VERSION) errors.push("schemaVersion_mismatch");
rejectUnknownKeys(value, RECEIPT_KEYS, "credentialSinkReceipt", errors);
requiredOpaqueId(value.transactionId, "transactionId", errors);
requiredOpaqueId(value.idempotencyKey, "idempotencyKey", errors);
requiredHash(value.policyHash, "policyHash", errors);
requiredTimestamp(value.processedAt, "processedAt", errors);
const outcomes = new Set(["committed", "rolled-back", "rejected", "rollback-failed"]);
if (!outcomes.has(value.outcome)) errors.push("outcome_invalid");
if (!Array.isArray(value.credentials)) {
errors.push("credentials_must_be_array");
} else {
const bindingIds = new Set();
const refs = new Set();
value.credentials.forEach((credential, index) => {
validateReceiptCredential(credential, index, errors);
if (!isPlainObject(credential)) return;
if (bindingIds.has(credential.bindingId)) errors.push("credentials_bindingId_must_be_unique");
bindingIds.add(credential.bindingId);
if (refs.has(credential.credentialRef)) errors.push("credentials_credentialRef_must_be_unique");
refs.add(credential.credentialRef);
});
}
validateReceiptOutcome(value, errors);
if (containsSecretLikeMaterial(value)) errors.push("receipt_must_not_contain_secret_material");
if (request !== undefined) compareReceiptToProvisionRequest(value, request, issuerPublicKeys, errors);
return result(errors);
}
export function computeEngineCredentialSinkReceiptHash(value) {
return `sha256:${createHash("sha256").update(stableJson(value), "utf8").digest("hex")}`;
}
export function validateEngineCredentialSinkRollback(value, { now = Date.now() } = {}) {
const errors = [];
const nowMs = normalizeNow(now, errors);
if (!isPlainObject(value)) return result(["credentialSinkRollback_must_be_object"]);
if (value.schemaVersion !== ENGINE_CREDENTIAL_SINK_ROLLBACK_SCHEMA_VERSION) errors.push("schemaVersion_mismatch");
rejectUnknownKeys(value, ROLLBACK_REQUEST_KEYS, "credentialSinkRollback", errors);
if (!isPlainObject(value.rollback)) {
errors.push("rollback_must_be_object");
return result(errors);
}
rejectUnknownKeys(value.rollback, ROLLBACK_KEYS, "rollback", errors);
requiredOpaqueId(value.rollback.id, "rollback.id", errors);
requiredOpaqueId(value.rollback.idempotencyKey, "rollback.idempotencyKey", errors);
requiredOpaqueId(value.rollback.transactionId, "rollback.transactionId", errors);
requiredTimestamp(value.rollback.requestedAt, "rollback.requestedAt", errors);
requiredTimestamp(value.rollback.requestExpiresAt, "rollback.requestExpiresAt", errors);
requiredHash(value.rollback.policyHash, "rollback.policyHash", errors);
requiredHash(value.rollback.committedReceiptHash, "rollback.committedReceiptHash", errors);
requiredReason(value.rollback.reasonCode, "rollback.reasonCode", errors);
validateRequestWindow(value.rollback.requestedAt, value.rollback.requestExpiresAt, nowMs, errors);
if (containsSecretLikeMaterial(value)) errors.push("rollback_must_not_contain_secret_material");
return result(errors);
}
export function validateEngineCredentialSinkRollbackReceipt(value, { request } = {}) {
const errors = [];
if (!isPlainObject(value)) return result(["credentialSinkRollbackReceipt_must_be_object"]);
if (value.schemaVersion !== ENGINE_CREDENTIAL_SINK_ROLLBACK_RECEIPT_SCHEMA_VERSION) errors.push("schemaVersion_mismatch");
rejectUnknownKeys(value, ROLLBACK_RECEIPT_KEYS, "credentialSinkRollbackReceipt", errors);
requiredOpaqueId(value.rollbackId, "rollbackId", errors);
requiredOpaqueId(value.transactionId, "transactionId", errors);
requiredHash(value.policyHash, "policyHash", errors);
requiredHash(value.committedReceiptHash, "committedReceiptHash", errors);
requiredTimestamp(value.processedAt, "processedAt", errors);
if (!new Set(["rolled-back", "rejected", "rollback-failed"]).has(value.outcome)) errors.push("outcome_invalid");
if (value.outcome === "rolled-back") {
if (value.errorCode !== undefined) errors.push("errorCode_forbidden_for_success");
} else {
requiredReason(value.errorCode, "errorCode", errors);
}
if (containsSecretLikeMaterial(value)) errors.push("rollbackReceipt_must_not_contain_secret_material");
if (request !== undefined && isPlainObject(request?.rollback)) {
if (!validateEngineCredentialSinkRollback(request, { now: value.processedAt }).ok) {
errors.push("request_invalid_for_rollback_receipt_comparison");
}
if (value.rollbackId !== request.rollback.id) errors.push("rollbackId_request_mismatch");
if (value.transactionId !== request.rollback.transactionId) errors.push("transactionId_request_mismatch");
if (value.policyHash !== request.rollback.policyHash) errors.push("policyHash_request_mismatch");
if (value.committedReceiptHash !== request.rollback.committedReceiptHash) {
errors.push("committedReceiptHash_request_mismatch");
}
}
return result(errors);
}
export function validateEngineCredentialSinkAudit(value) {
const errors = [];
if (!isPlainObject(value)) return result(["credentialSinkAudit_must_be_object"]);
if (value.schemaVersion !== ENGINE_CREDENTIAL_SINK_AUDIT_SCHEMA_VERSION) errors.push("schemaVersion_mismatch");
rejectUnknownKeys(value, AUDIT_KEYS, "credentialSinkAudit", errors);
requiredOpaqueId(value.eventId, "eventId", errors);
requiredOpaqueId(value.transactionId, "transactionId", errors);
requiredOpaqueId(value.operationId, "operationId", errors);
if (!new Set(["provision", "rollback"]).has(value.operation)) errors.push("operation_invalid");
if (!new Set(["committed", "rolled-back", "rejected", "rollback-failed"]).has(value.outcome)) errors.push("outcome_invalid");
if (value.operation === "provision" && value.outcome === "rolled-back" && !value.reasonCode) {
errors.push("reasonCode_required");
}
if (value.operation === "rollback" && value.outcome === "committed") errors.push("rollback_outcome_invalid");
requiredTimestamp(value.occurredAt, "occurredAt", errors);
requiredHash(value.policyHash, "policyHash", errors);
if (!isPlainObject(value.principal)) {
errors.push("principal_must_be_object");
} else {
rejectUnknownKeys(value.principal, PRINCIPAL_KEYS, "principal", errors);
requiredIdentifier(value.principal.serviceId, "principal.serviceId", errors);
requiredHash(value.principal.fingerprint, "principal.fingerprint", errors);
}
if (!Array.isArray(value.targets)) {
errors.push("targets_must_be_array");
} else {
value.targets.forEach((target, index) => validateAuditTarget(target, index, errors));
}
if (value.reasonCode !== undefined) requiredReason(value.reasonCode, "reasonCode", errors);
if (new Set(["rejected", "rollback-failed"]).has(value.outcome) && value.reasonCode === undefined) {
errors.push("reasonCode_required");
}
if (containsSecretLikeMaterial(value)) errors.push("audit_must_not_contain_secret_material");
return result(errors);
}
/** Returns audit-safe target descriptors; material and credential refs cannot escape. */
export function engineCredentialSinkAuditTargets(request, receipt) {
const refByBinding = new Map(
Array.isArray(receipt?.credentials)
? receipt.credentials.map((item) => [item.bindingId, item.credentialRef])
: [],
);
return Array.isArray(request?.bindings) ? request.bindings.map((binding) => {
const descriptor = secretFreeBindingDescriptor(binding);
const credentialRef = refByBinding.get(binding.bindingId);
return credentialRef
? { ...descriptor, credentialRefHash: sha256Value(credentialRef) }
: descriptor;
}) : [];
}
function validateProvisionBinding(value, index, requestedAt, errors) {
const path = `bindings[${index}]`;
if (!isPlainObject(value)) {
errors.push(`${path}_must_be_object`);
return;
}
rejectUnknownKeys(value, BINDING_KEYS, path, errors);
requiredIdentifier(value.bindingId, `${path}.bindingId`, errors);
const spec = CAPABILITY_SPECS[value.capabilityType];
if (!spec) errors.push(`${path}.capabilityType_invalid`);
requiredOpaqueId(value.grantId, `${path}.grantId`, errors);
validateTarget(value.target, path, errors);
requiredTimestamp(value.expiresAt, `${path}.expiresAt`, errors);
requiredHash(value.policyHash, `${path}.policyHash`, errors);
requiredHash(value.capabilityDigest, `${path}.capabilityDigest`, errors);
if (isTimestamp(requestedAt) && isTimestamp(value.expiresAt) && Date.parse(value.expiresAt) <= Date.parse(requestedAt)) {
errors.push(`${path}.expiresAt_must_be_after_requestedAt`);
}
if (spec && isPlainObject(value.target)) {
if (value.target.nodeType !== spec.nodeType) errors.push(`${path}.target.nodeType_capability_mismatch`);
if (value.target.credentialType !== spec.credentialType) errors.push(`${path}.target.credentialType_capability_mismatch`);
}
if (!isPlainObject(value.material)) {
errors.push(`${path}.material_must_be_object`);
} else {
rejectUnknownKeys(value.material, MATERIAL_KEYS, `${path}.material`, errors);
if (value.material.format !== "opaque-bearer") errors.push(`${path}.material.format_must_be_opaque-bearer`);
if (!spec || typeof value.material.value !== "string" || !spec.materialPattern.test(value.material.value)) {
errors.push(`${path}.material.value_invalid_for_capability`);
} else if (value.capabilityDigest !== computeEngineCredentialCapabilityDigest(value.material.value)) {
errors.push(`${path}.capabilityDigest_material_mismatch`);
}
}
}
function validateTarget(value, path, errors) {
if (!isPlainObject(value)) {
errors.push(`${path}.target_must_be_object`);
return;
}
rejectUnknownKeys(value, TARGET_KEYS, `${path}.target`, errors);
requiredOpaqueId(value.workflowId, `${path}.target.workflowId`, errors);
requiredOpaqueId(value.workflowRevision, `${path}.target.workflowRevision`, errors);
requiredOpaqueId(value.nodeId, `${path}.target.nodeId`, errors);
if (typeof value.nodeType !== "string" || !NODE_TYPE.test(value.nodeType)) errors.push(`${path}.target.nodeType_invalid`);
if (typeof value.credentialType !== "string" || !CREDENTIAL_TYPE.test(value.credentialType)) {
errors.push(`${path}.target.credentialType_invalid`);
}
}
function validateReceiptCredential(value, index, errors) {
const path = `credentials[${index}]`;
if (!isPlainObject(value)) {
errors.push(`${path}_must_be_object`);
return;
}
rejectUnknownKeys(value, RECEIPT_CREDENTIAL_KEYS, path, errors);
requiredIdentifier(value.bindingId, `${path}.bindingId`, errors);
if (!CAPABILITY_SPECS[value.capabilityType]) errors.push(`${path}.capabilityType_invalid`);
requiredOpaqueId(value.grantId, `${path}.grantId`, errors);
validateTarget(value.target, path, errors);
if (typeof value.credentialRef !== "string" || !CREDENTIAL_REFERENCE.test(value.credentialRef)) {
errors.push(`${path}.credentialRef_invalid`);
}
requiredTimestamp(value.expiresAt, `${path}.expiresAt`, errors);
requiredHash(value.policyHash, `${path}.policyHash`, errors);
requiredHash(value.capabilityDigest, `${path}.capabilityDigest`, errors);
if (!new Set(["created", "reused", "rotated"]).has(value.disposition)) errors.push(`${path}.disposition_invalid`);
}
function validateReceiptOutcome(value, errors) {
if (!isPlainObject(value.rollback)) {
errors.push("rollback_must_be_object");
return;
}
rejectUnknownKeys(value.rollback, RECEIPT_ROLLBACK_KEYS, "rollback", errors);
const expectedRollback = {
committed: "not-required",
"rolled-back": "complete",
rejected: "not-started",
"rollback-failed": "incomplete",
}[value.outcome];
if (expectedRollback && value.rollback.status !== expectedRollback) errors.push("rollback.status_outcome_mismatch");
if (new Set(["complete", "incomplete"]).has(value.rollback.status)) {
requiredTimestamp(value.rollback.completedAt, "rollback.completedAt", errors);
} else if (value.rollback.completedAt !== undefined) {
errors.push("rollback.completedAt_not_allowed");
}
if (value.outcome === "committed") {
if (!Array.isArray(value.credentials) || value.credentials.length === 0) errors.push("committed_credentials_required");
if (value.errorCode !== undefined) errors.push("errorCode_forbidden_for_success");
} else {
if (Array.isArray(value.credentials) && value.credentials.length !== 0) errors.push("noncommitted_credentials_must_be_empty");
requiredReason(value.errorCode, "errorCode", errors);
}
}
function compareReceiptToProvisionRequest(receipt, request, issuerPublicKeys, errors) {
const requestValidation = validateEngineCredentialSinkProvision(request, {
now: receipt.processedAt,
issuerPublicKeys,
});
if (!requestValidation.ok) {
errors.push("request_invalid_for_receipt_comparison");
return;
}
if (receipt.transactionId !== request.transaction.id) errors.push("transactionId_request_mismatch");
if (receipt.idempotencyKey !== request.transaction.idempotencyKey) errors.push("idempotencyKey_request_mismatch");
if (receipt.policyHash !== request.transaction.policyHash) errors.push("policyHash_request_mismatch");
if (receipt.outcome !== "committed") return;
if (receipt.credentials.length !== request.bindings.length) errors.push("credentials_request_count_mismatch");
const requested = new Map(request.bindings.map((binding) => [binding.bindingId, secretFreeBindingDescriptor(binding)]));
for (const credential of receipt.credentials) {
const expected = requested.get(credential.bindingId);
if (!expected || stableJson({
bindingId: credential.bindingId,
capabilityType: credential.capabilityType,
grantId: credential.grantId,
target: credential.target,
expiresAt: credential.expiresAt,
policyHash: credential.policyHash,
capabilityDigest: credential.capabilityDigest,
}) !== stableJson(expected)) {
errors.push("credentials_request_target_mismatch");
}
}
}
function validateAuditTarget(value, index, errors) {
const path = `targets[${index}]`;
if (!isPlainObject(value)) {
errors.push(`${path}_must_be_object`);
return;
}
rejectUnknownKeys(value, AUDIT_TARGET_KEYS, path, errors);
requiredIdentifier(value.bindingId, `${path}.bindingId`, errors);
if (!CAPABILITY_SPECS[value.capabilityType]) errors.push(`${path}.capabilityType_invalid`);
requiredOpaqueId(value.grantId, `${path}.grantId`, errors);
validateTarget(value.target, path, errors);
requiredTimestamp(value.expiresAt, `${path}.expiresAt`, errors);
requiredHash(value.policyHash, `${path}.policyHash`, errors);
requiredHash(value.capabilityDigest, `${path}.capabilityDigest`, errors);
if (value.credentialRefHash !== undefined) requiredHash(value.credentialRefHash, `${path}.credentialRefHash`, errors);
}
function secretFreeBindingDescriptor(value) {
return {
bindingId: value?.bindingId,
capabilityType: value?.capabilityType,
grantId: value?.grantId,
target: value?.target,
expiresAt: value?.expiresAt,
policyHash: value?.policyHash,
capabilityDigest: value?.capabilityDigest,
};
}
export function computeEngineCredentialCapabilityDigest(value) {
return sha256Value(value);
}
function validateIssuer(value, path, errors) {
if (!isPlainObject(value)) {
errors.push(`${path}_must_be_object`);
return;
}
rejectUnknownKeys(value, ISSUER_KEYS, path, errors);
requiredIdentifier(value.serviceId, `${path}.serviceId`, errors);
requiredOpaqueId(value.keyId, `${path}.keyId`, errors);
}
function validateAttestation(value, path, errors) {
if (!isPlainObject(value)) {
errors.push(`${path}_must_be_object`);
return;
}
rejectUnknownKeys(value, ATTESTATION_KEYS, path, errors);
if (value.algorithm !== "Ed25519") errors.push(`${path}.algorithm_must_be_Ed25519`);
if (typeof value.signature !== "string" || !ED25519_SIGNATURE.test(value.signature)) {
errors.push(`${path}.signature_invalid`);
}
}
function verifyProvisionAttestation(transaction, issuerPublicKeys, errors) {
if (!isPlainObject(transaction?.issuer) || !isPlainObject(transaction?.attestation)) return;
if (transaction.attestation.algorithm !== "Ed25519" || !ED25519_SIGNATURE.test(String(transaction.attestation.signature || ""))) return;
const keyIdentity = `${transaction.issuer.serviceId}:${transaction.issuer.keyId}`;
const publicKey = isPlainObject(issuerPublicKeys) && Object.hasOwn(issuerPublicKeys, keyIdentity)
? issuerPublicKeys[keyIdentity]
: undefined;
if (!publicKey) {
errors.push("transaction.issuer_public_key_required");
return;
}
let valid = false;
try {
valid = verifySignature(
null,
Buffer.from(String(transaction.policyHash), "utf8"),
publicKey,
Buffer.from(transaction.attestation.signature, "base64url"),
);
} catch {
valid = false;
}
if (!valid) errors.push("transaction.attestation_invalid");
}
function validateRequestWindow(requestedAt, requestExpiresAt, nowMs, errors) {
if (!isTimestamp(requestedAt) || !isTimestamp(requestExpiresAt)) return;
const requested = Date.parse(requestedAt);
const expires = Date.parse(requestExpiresAt);
if (expires <= requested) errors.push("requestExpiresAt_must_be_after_requestedAt");
if (expires - requested > MAX_REQUEST_LIFETIME_MS) errors.push("request_lifetime_exceeds_15_minutes");
if (Number.isFinite(nowMs)) {
if (requested > nowMs + MAX_REQUEST_CLOCK_SKEW_MS) errors.push("requestedAt_exceeds_clock_skew");
if (expires <= nowMs) errors.push("request_expired");
}
}
function normalizeNow(value, errors) {
const normalized = value instanceof Date ? value.getTime() : typeof value === "string" ? Date.parse(value) : Number(value);
if (!Number.isFinite(normalized)) {
errors.push("validation_now_invalid");
return Number.NaN;
}
return normalized;
}
function targetIdentity(value) {
if (!isPlainObject(value)) return "";
return [value.workflowId, value.workflowRevision, value.nodeId, value.nodeType, value.credentialType].join("\u0000");
}
function requiredIdentifier(value, path, errors) {
if (typeof value !== "string" || !IDENTIFIER.test(value)) errors.push(`${path}_invalid`);
}
function requiredOpaqueId(value, path, errors) {
if (typeof value !== "string" || !OPAQUE_ID.test(value)) errors.push(`${path}_invalid`);
}
function requiredHash(value, path, errors) {
if (typeof value !== "string" || !HASH.test(value)) errors.push(`${path}_invalid`);
}
function requiredReason(value, path, errors) {
if (typeof value !== "string" || !REASON_CODE.test(value)) errors.push(`${path}_invalid`);
}
function requiredTimestamp(value, path, errors) {
if (!isTimestamp(value)) errors.push(`${path}_invalid_timestamp`);
}
function isTimestamp(value) {
return typeof value === "string" && !Number.isNaN(Date.parse(value)) && new Date(value).toISOString() === value;
}
function sha256Value(value) {
return `sha256:${createHash("sha256").update(String(value), "utf8").digest("hex")}`;
}
function stableJson(value) {
return JSON.stringify(sortValue(value));
}
function sortValue(value) {
if (Array.isArray(value)) return value.map(sortValue);
if (!isPlainObject(value)) return value;
return Object.fromEntries(Object.keys(value).sort().map((key) => [key, sortValue(value[key])]));
}
function containsSecretLikeMaterial(value) {
if (typeof value === "string") return SECRET_LIKE_VALUE.test(value);
if (Array.isArray(value)) return value.some(containsSecretLikeMaterial);
if (!isPlainObject(value)) return false;
return Object.entries(value).some(([key, child]) => SECRET_LIKE_KEY.test(key) || containsSecretLikeMaterial(child));
}
function isPlainObject(value) {
return Boolean(value) && typeof value === "object" && !Array.isArray(value);
}
function rejectUnknownKeys(value, allowed, path, errors) {
if (!isPlainObject(value)) return;
for (const key of Object.keys(value)) {
if (!allowed.has(key)) errors.push(`${path}.${key}_not_allowed`);
}
}
function result(errors) {
const uniqueErrors = [...new Set(errors)];
return Object.freeze({ ok: uniqueErrors.length === 0, errors: Object.freeze(uniqueErrors) });
}

View File

@ -1,6 +1,23 @@
export const EXTERNAL_PROVIDER_CONTRACT_VERSION = "nodedc.external-provider-contract/v1"; import { EXTERNAL_PROVIDER_CONTRACT_VERSION } from "./contract-version.mjs";
export { EXTERNAL_PROVIDER_CONTRACT_VERSION } from "./contract-version.mjs";
export { validateIntakeBatch } from "./intake-batch.mjs";
export const FOUNDRY_BINDING_UPSERT_SCHEMA_VERSION = "nodedc.foundry.binding-upsert/v1"; export const FOUNDRY_BINDING_UPSERT_SCHEMA_VERSION = "nodedc.foundry.binding-upsert/v1";
import {
SECRET_LIKE_KEY,
SECRET_LIKE_VALUE,
} from "./sensitive-field-policy.mjs";
export {
L2_CONNECTION_INSTANCE_SCHEMA_VERSION,
L2_TEMPLATE_SCHEMA_VERSION,
PROVIDER_PACKAGE_SCHEMA_VERSION,
SEMANTIC_MAPPING_SCHEMA_VERSION,
instantiateL2Connection,
validateProviderPackage,
} from "./provider-package.mjs";
export { export {
DATA_PRODUCT_PATCH_SCHEMA_VERSION, DATA_PRODUCT_PATCH_SCHEMA_VERSION,
DATA_PRODUCT_PUBLISH_SCHEMA_VERSION, DATA_PRODUCT_PUBLISH_SCHEMA_VERSION,
@ -10,24 +27,6 @@ export {
validateDataProductSnapshot, validateDataProductSnapshot,
} from "./data-product.mjs"; } from "./data-product.mjs";
export {
ENGINE_CREDENTIAL_SINK_AUDIT_SCHEMA_VERSION,
ENGINE_CREDENTIAL_SINK_CAPABILITY_TYPES,
ENGINE_CREDENTIAL_SINK_PROVISION_SCHEMA_VERSION,
ENGINE_CREDENTIAL_SINK_RECEIPT_SCHEMA_VERSION,
ENGINE_CREDENTIAL_SINK_ROLLBACK_RECEIPT_SCHEMA_VERSION,
ENGINE_CREDENTIAL_SINK_ROLLBACK_SCHEMA_VERSION,
computeEngineCredentialCapabilityDigest,
computeEngineCredentialSinkPolicyHash,
computeEngineCredentialSinkReceiptHash,
engineCredentialSinkAuditTargets,
validateEngineCredentialSinkAudit,
validateEngineCredentialSinkProvision,
validateEngineCredentialSinkReceipt,
validateEngineCredentialSinkRollback,
validateEngineCredentialSinkRollbackReceipt,
} from "./engine-credential-sink.mjs";
export { export {
ENGINE_PRIVATE_EXTENSION_APPLY_RECEIPT_SCHEMA_VERSION, ENGINE_PRIVATE_EXTENSION_APPLY_RECEIPT_SCHEMA_VERSION,
ENGINE_PRIVATE_EXTENSION_APPLY_REQUEST_SCHEMA_VERSION, ENGINE_PRIVATE_EXTENSION_APPLY_REQUEST_SCHEMA_VERSION,
@ -55,16 +54,11 @@ export {
const COLLECTION_MODES = new Set(["realtime", "manual", "weekly", "history"]); const COLLECTION_MODES = new Set(["realtime", "manual", "weekly", "history"]);
const DELIVERY_MODES = new Set(["snapshot", "snapshot+patch", "query"]); const DELIVERY_MODES = new Set(["snapshot", "snapshot+patch", "query"]);
const CAPABILITY_CLASSIFICATIONS = new Set(["read", "metadata", "write", "destructive", "unknown"]); const CAPABILITY_CLASSIFICATIONS = new Set(["read", "metadata", "write", "destructive", "unknown"]);
const SECRET_LIKE_KEY = /(token|secret|password|authorization|access[_-]?token|refresh[_-]?token|api[_-]?key)/i;
const SECRET_LIKE_REFERENCE = /(?:[?&](?:token|secret|password|authorization|access[_-]?token|refresh[_-]?token|api[_-]?key)=|(?:bearer|basic)\s+)/i;
const SECRET_LIKE_VALUE = /(?:ndc_edp(?:wb|rb)_[A-Za-z0-9_-]*|[?&](?:token|secret|password|authorization|access[_-]?token|refresh[_-]?token|api[_-]?key)=|(?:bearer|basic)\s+\S+|eyJ[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+)/i;
const IDENTIFIER = /^[a-z][a-z0-9._:-]{2,127}$/; const IDENTIFIER = /^[a-z][a-z0-9._:-]{2,127}$/;
const FOUNDRY_APPLICATION_ID = /^[0-9a-f]{8}-[0-9a-f]{4}-[1-5][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i; const FOUNDRY_APPLICATION_ID = /^[0-9a-f]{8}-[0-9a-f]{4}-[1-5][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i;
const FOUNDRY_PAGE_ID = /^[a-z0-9][a-z0-9-]{0,79}$/; const FOUNDRY_PAGE_ID = /^[a-z0-9][a-z0-9-]{0,79}$/;
const FOUNDRY_SLOT_ID = /^[A-Za-z0-9][A-Za-z0-9-]{0,79}$/; const FOUNDRY_SLOT_ID = /^[A-Za-z0-9][A-Za-z0-9-]{0,79}$/;
const CONTRACT_VERSION = /^\d+\.\d+\.\d+(?:[-+][a-z0-9.-]+)?$/i; const CONTRACT_VERSION = /^\d+\.\d+\.\d+(?:[-+][a-z0-9.-]+)?$/i;
const MAX_BATCH_SEQUENCE = 2_147_483_647;
const MAX_FACT_ATTRIBUTES_BYTES = 64 * 1024;
const PROVIDER_MANIFEST_KEYS = new Set(["schemaVersion", "id", "providerId", "version", "ontology", "l2Template", "capabilities", "dataProductIds"]); const PROVIDER_MANIFEST_KEYS = new Set(["schemaVersion", "id", "providerId", "version", "ontology", "l2Template", "capabilities", "dataProductIds"]);
const CONNECTION_PROFILE_KEYS = new Set(["schemaVersion", "id", "providerId", "tenantId", "credentialRef", "scope"]); const CONNECTION_PROFILE_KEYS = new Set(["schemaVersion", "id", "providerId", "tenantId", "credentialRef", "scope"]);
@ -159,8 +153,8 @@ export function validateConnectionProfile(value) {
requiredString(value?.credentialRef?.owner, "credentialRef.owner", errors); requiredString(value?.credentialRef?.owner, "credentialRef.owner", errors);
requiredString(value?.credentialRef?.reference, "credentialRef.reference", errors); requiredString(value?.credentialRef?.reference, "credentialRef.reference", errors);
if (value?.credentialRef?.owner && value.credentialRef.owner !== "engine") { if (value?.credentialRef?.owner && value.credentialRef.owner !== "ndc_l2_credentials") {
errors.push("credentialRef.owner_must_be_engine"); errors.push("credentialRef.owner_must_be_ndc_l2_credentials");
} }
if (containsSecretLikeMaterial(value)) errors.push("profile_must_not_contain_secret_material"); if (containsSecretLikeMaterial(value)) errors.push("profile_must_not_contain_secret_material");
if (value?.scope !== undefined) { if (value?.scope !== undefined) {
@ -301,45 +295,6 @@ export function validateFoundryBindingUpsert(value) {
return result(errors); return result(errors);
} }
/**
* Provider-neutral batch written by an L2 connector to External Data Plane.
* The payload deliberately describes source facts rather than any provider
* field names, customer entities or renderer representation.
*/
export function validateIntakeBatch(value) {
const errors = baseErrors(value, "intakeBatch");
rejectUnknownKeys(value, new Set(["schemaVersion", "source", "contract", "batch", "raw", "facts"]), "intakeBatch", errors);
rejectUnknownKeys(value?.source, new Set(["providerId", "tenantId", "connectionId"]), "source", errors);
rejectUnknownKeys(value?.contract, new Set(["dataProductId", "ontologyRevision", "version"]), "contract", errors);
rejectUnknownKeys(value?.batch, new Set(["runId", "sequence", "idempotencyKey", "receivedAt"]), "batch", errors);
requiredIdentifier(value?.source?.providerId, "source.providerId", errors);
requiredIdentifier(value?.source?.tenantId, "source.tenantId", errors);
requiredIdentifier(value?.source?.connectionId, "source.connectionId", errors);
requiredIdentifier(value?.contract?.dataProductId, "contract.dataProductId", errors);
requiredIdentifier(value?.contract?.ontologyRevision, "contract.ontologyRevision", errors);
requiredString(value?.contract?.version, "contract.version", errors);
if (value?.contract?.version && !CONTRACT_VERSION.test(value.contract.version)) {
errors.push("contract.version_must_be_semver");
}
requiredIdentifier(value?.batch?.runId, "batch.runId", errors);
requiredIdentifier(value?.batch?.idempotencyKey, "batch.idempotencyKey", errors);
if (!Number.isInteger(value?.batch?.sequence) || value.batch.sequence < 0 || value.batch.sequence > MAX_BATCH_SEQUENCE) {
errors.push("batch.sequence_must_be_integer_0_to_2147483647");
}
requiredIsoTimestamp(value?.batch?.receivedAt, "batch.receivedAt", errors);
if (!Array.isArray(value?.facts) || value.facts.length === 0) {
errors.push("facts_must_be_nonempty_array");
} else {
value.facts.forEach((fact, index) => validateFact(fact, `facts[${index}]`, errors));
}
if (value?.raw !== undefined) validateRawEnvelope(value.raw, errors);
if (containsSecretLikeMaterial(value)) errors.push("intake_must_not_contain_secret_material");
return result(errors);
}
export function assertValid(validator, value) { export function assertValid(validator, value) {
const validation = validator(value); const validation = validator(value);
if (!validation.ok) throw new Error(`external_provider_contract_invalid:${validation.errors.join(",")}`); if (!validation.ok) throw new Error(`external_provider_contract_invalid:${validation.errors.join(",")}`);
@ -394,66 +349,6 @@ function validateUniqueIdentifierArray(value, path, errors) {
if (new Set(value).size !== value.length) errors.push(`${path}_must_not_contain_duplicates`); if (new Set(value).size !== value.length) errors.push(`${path}_must_not_contain_duplicates`);
} }
function requiredIsoTimestamp(value, path, errors) {
if (typeof value !== "string" || Number.isNaN(Date.parse(value))) errors.push(`${path}_invalid_timestamp`);
}
function validateFact(value, path, errors) {
if (!isPlainObject(value)) {
errors.push(`${path}_must_be_object`);
return;
}
rejectUnknownKeys(value, new Set(["sourceId", "semanticType", "observedAt", "attributes", "geometry"]), path, errors);
requiredIdentifier(value.sourceId, `${path}.sourceId`, errors);
requiredIdentifier(value.semanticType, `${path}.semanticType`, errors);
requiredIsoTimestamp(value.observedAt, `${path}.observedAt`, errors);
if (value.attributes !== undefined) {
if (!isPlainObject(value.attributes)) {
errors.push(`${path}.attributes_must_be_object`);
} else if (serializedByteLength(value.attributes) > MAX_FACT_ATTRIBUTES_BYTES) {
errors.push(`${path}.attributes_size_exceeded`);
}
}
if (value.geometry !== undefined) validatePointGeometry(value.geometry, `${path}.geometry`, errors);
}
function validatePointGeometry(value, path, errors) {
if (!isPlainObject(value) || value.type !== "Point" || !Array.isArray(value.coordinates) || value.coordinates.length !== 2) {
errors.push(`${path}_must_be_geojson_point`);
return;
}
rejectUnknownKeys(value, new Set(["type", "coordinates"]), path, errors);
if (!value.coordinates.every((coordinate) => typeof coordinate === "number" && Number.isFinite(coordinate))) {
errors.push(`${path}_coordinates_must_be_finite_numbers`);
return;
}
const [longitude, latitude] = value.coordinates;
if (longitude < -180 || longitude > 180) errors.push(`${path}.longitude_out_of_range`);
if (latitude < -90 || latitude > 90) errors.push(`${path}.latitude_out_of_range`);
}
function validateRawEnvelope(value, errors) {
if (!isPlainObject(value)) {
errors.push("raw_must_be_object");
return;
}
rejectUnknownKeys(value, new Set(["contentType", "payload", "hash", "ref", "retentionDays"]), "raw", errors);
requiredString(value.contentType, "raw.contentType", errors);
if (value.payload === undefined && value.ref === undefined) errors.push("raw_requires_payload_or_ref");
if (value.payload !== undefined) errors.push("raw.inline_payload_not_supported");
if (value.payload === undefined) requiredString(value.hash, "raw.hash", errors);
if (value.hash !== undefined) requiredString(value.hash, "raw.hash", errors);
if (value.ref !== undefined) {
requiredString(value.ref, "raw.ref", errors);
if (typeof value.ref === "string" && SECRET_LIKE_REFERENCE.test(value.ref)) {
errors.push("raw.ref_must_not_contain_secret_material");
}
}
if (value.retentionDays !== undefined && (!Number.isInteger(value.retentionDays) || value.retentionDays < 1)) {
errors.push("raw.retentionDays_must_be_positive_integer");
}
}
function isPlainObject(value) { function isPlainObject(value) {
return Boolean(value) && typeof value === "object" && !Array.isArray(value); return Boolean(value) && typeof value === "object" && !Array.isArray(value);
} }
@ -465,14 +360,6 @@ function rejectUnknownKeys(value, allowedKeys, path, errors) {
} }
} }
function serializedByteLength(value) {
try {
return Buffer.byteLength(JSON.stringify(value));
} catch {
return Number.POSITIVE_INFINITY;
}
}
function containsSecretLikeMaterial(value) { function containsSecretLikeMaterial(value) {
if (typeof value === "string") return SECRET_LIKE_VALUE.test(value); if (typeof value === "string") return SECRET_LIKE_VALUE.test(value);
if (Array.isArray(value)) return value.some(containsSecretLikeMaterial); if (Array.isArray(value)) return value.some(containsSecretLikeMaterial);

View File

@ -0,0 +1,152 @@
import { EXTERNAL_PROVIDER_CONTRACT_VERSION } from "./contract-version.mjs";
import {
SECRET_LIKE_KEY,
SECRET_LIKE_REFERENCE,
SECRET_LIKE_VALUE,
} from "./sensitive-field-policy.mjs";
const IDENTIFIER = /^[a-z][a-z0-9._:-]{2,127}$/;
const CONTRACT_VERSION = /^\d+\.\d+\.\d+(?:[-+][a-z0-9.-]+)?$/i;
const MAX_BATCH_SEQUENCE = 2_147_483_647;
const MAX_FACT_ATTRIBUTES_BYTES = 64 * 1024;
/** Provider-neutral batch written by an L2 connector to External Data Plane. */
export function validateIntakeBatch(value) {
const errors = baseErrors(value, "intakeBatch");
rejectUnknownKeys(value, new Set(["schemaVersion", "source", "contract", "batch", "raw", "facts"]), "intakeBatch", errors);
rejectUnknownKeys(value?.source, new Set(["providerId", "tenantId", "connectionId"]), "source", errors);
rejectUnknownKeys(value?.contract, new Set(["dataProductId", "ontologyRevision", "version"]), "contract", errors);
rejectUnknownKeys(value?.batch, new Set(["runId", "sequence", "idempotencyKey", "receivedAt"]), "batch", errors);
requiredIdentifier(value?.source?.providerId, "source.providerId", errors);
requiredIdentifier(value?.source?.tenantId, "source.tenantId", errors);
requiredIdentifier(value?.source?.connectionId, "source.connectionId", errors);
requiredIdentifier(value?.contract?.dataProductId, "contract.dataProductId", errors);
requiredIdentifier(value?.contract?.ontologyRevision, "contract.ontologyRevision", errors);
requiredString(value?.contract?.version, "contract.version", errors);
if (value?.contract?.version && !CONTRACT_VERSION.test(value.contract.version)) {
errors.push("contract.version_must_be_semver");
}
requiredIdentifier(value?.batch?.runId, "batch.runId", errors);
requiredIdentifier(value?.batch?.idempotencyKey, "batch.idempotencyKey", errors);
if (!Number.isInteger(value?.batch?.sequence) || value.batch.sequence < 0 || value.batch.sequence > MAX_BATCH_SEQUENCE) {
errors.push("batch.sequence_must_be_integer_0_to_2147483647");
}
requiredIsoTimestamp(value?.batch?.receivedAt, "batch.receivedAt", errors);
if (!Array.isArray(value?.facts) || value.facts.length === 0) {
errors.push("facts_must_be_nonempty_array");
} else {
value.facts.forEach((fact, index) => validateFact(fact, `facts[${index}]`, errors));
}
if (value?.raw !== undefined) validateRawEnvelope(value.raw, errors);
if (containsSecretLikeMaterial(value)) errors.push("intake_must_not_contain_secret_material");
return result(errors);
}
function baseErrors(value, label) {
const errors = [];
if (!isPlainObject(value)) return [`${label}_must_be_object`];
if (value.schemaVersion !== EXTERNAL_PROVIDER_CONTRACT_VERSION) errors.push("schemaVersion_mismatch");
return errors;
}
function requiredString(value, path, errors) {
if (typeof value !== "string" || !value.trim()) errors.push(`${path}_required`);
}
function requiredIdentifier(value, path, errors) {
if (typeof value !== "string" || !IDENTIFIER.test(value)) errors.push(`${path}_invalid`);
}
function requiredIsoTimestamp(value, path, errors) {
if (typeof value !== "string" || Number.isNaN(Date.parse(value))) errors.push(`${path}_invalid_timestamp`);
}
function validateFact(value, path, errors) {
if (!isPlainObject(value)) {
errors.push(`${path}_must_be_object`);
return;
}
rejectUnknownKeys(value, new Set(["sourceId", "semanticType", "observedAt", "attributes", "geometry"]), path, errors);
requiredIdentifier(value.sourceId, `${path}.sourceId`, errors);
requiredIdentifier(value.semanticType, `${path}.semanticType`, errors);
requiredIsoTimestamp(value.observedAt, `${path}.observedAt`, errors);
if (value.attributes !== undefined) {
if (!isPlainObject(value.attributes)) {
errors.push(`${path}.attributes_must_be_object`);
} else if (serializedByteLength(value.attributes) > MAX_FACT_ATTRIBUTES_BYTES) {
errors.push(`${path}.attributes_size_exceeded`);
}
}
if (value.geometry !== undefined) validatePointGeometry(value.geometry, `${path}.geometry`, errors);
}
function validatePointGeometry(value, path, errors) {
if (!isPlainObject(value) || value.type !== "Point" || !Array.isArray(value.coordinates) || value.coordinates.length !== 2) {
errors.push(`${path}_must_be_geojson_point`);
return;
}
rejectUnknownKeys(value, new Set(["type", "coordinates"]), path, errors);
if (!value.coordinates.every((coordinate) => typeof coordinate === "number" && Number.isFinite(coordinate))) {
errors.push(`${path}_coordinates_must_be_finite_numbers`);
return;
}
const [longitude, latitude] = value.coordinates;
if (longitude < -180 || longitude > 180) errors.push(`${path}.longitude_out_of_range`);
if (latitude < -90 || latitude > 90) errors.push(`${path}.latitude_out_of_range`);
}
function validateRawEnvelope(value, errors) {
if (!isPlainObject(value)) {
errors.push("raw_must_be_object");
return;
}
rejectUnknownKeys(value, new Set(["contentType", "payload", "hash", "ref", "retentionDays"]), "raw", errors);
requiredString(value.contentType, "raw.contentType", errors);
if (value.payload === undefined && value.ref === undefined) errors.push("raw_requires_payload_or_ref");
if (value.payload !== undefined) errors.push("raw.inline_payload_not_supported");
if (value.payload === undefined) requiredString(value.hash, "raw.hash", errors);
if (value.hash !== undefined) requiredString(value.hash, "raw.hash", errors);
if (value.ref !== undefined) {
requiredString(value.ref, "raw.ref", errors);
if (typeof value.ref === "string" && SECRET_LIKE_REFERENCE.test(value.ref)) {
errors.push("raw.ref_must_not_contain_secret_material");
}
}
if (value.retentionDays !== undefined && (!Number.isInteger(value.retentionDays) || value.retentionDays < 1)) {
errors.push("raw.retentionDays_must_be_positive_integer");
}
}
function isPlainObject(value) {
return Boolean(value) && typeof value === "object" && !Array.isArray(value);
}
function rejectUnknownKeys(value, allowedKeys, path, errors) {
if (!isPlainObject(value)) return;
for (const key of Object.keys(value)) {
if (!allowedKeys.has(key)) errors.push(`${path}.${key}_not_allowed`);
}
}
function serializedByteLength(value) {
try {
return Buffer.byteLength(JSON.stringify(value));
} catch {
return Number.POSITIVE_INFINITY;
}
}
function containsSecretLikeMaterial(value) {
if (typeof value === "string") return SECRET_LIKE_VALUE.test(value);
if (Array.isArray(value)) return value.some(containsSecretLikeMaterial);
if (!isPlainObject(value)) return false;
return Object.entries(value).some(([key, child]) => SECRET_LIKE_KEY.test(key) || containsSecretLikeMaterial(child));
}
function result(errors) {
const uniqueErrors = [...new Set(errors)];
return Object.freeze({ ok: uniqueErrors.length === 0, errors: Object.freeze(uniqueErrors) });
}

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,16 @@
export const NDC_SECRET_CAPABILITY_SOURCE =
String.raw`ndc_(?:edp(?:wb|rb|pr)|fndbg)_[A-Za-z0-9_-]+`;
export const SECRET_LIKE_KEY =
/(token|secret|password|authorization|access[_-]?token|refresh[_-]?token|api[_-]?key)/i;
export const SECRET_LIKE_MATERIAL_KEY =
/(token|secret|password|authorization|access[_-]?token|refresh[_-]?token|api[_-]?key|material|value)/i;
export const SECRET_LIKE_REFERENCE =
/(?:[?&](?:token|secret|password|authorization|access[_-]?token|refresh[_-]?token|api[_-]?key)=|(?:bearer|basic)\s+)/i;
export const SECRET_LIKE_VALUE = new RegExp(
`(?:${NDC_SECRET_CAPABILITY_SOURCE}|[?&](?:token|secret|password|authorization|access[_-]?token|refresh[_-]?token|api[_-]?key)=|(?:bearer|basic)\\s+\\S+|eyJ[A-Za-z0-9_-]+\\.[A-Za-z0-9_-]+\\.[A-Za-z0-9_-]+)`,
"i",
);

View File

@ -15,6 +15,10 @@ import { geliosPositionsCurrentExample } from "../examples/gelios-positions-curr
assert.equal(validateProviderManifest(geliosPositionsCurrentExample.providerManifest).ok, true); assert.equal(validateProviderManifest(geliosPositionsCurrentExample.providerManifest).ok, true);
assert.equal(validateConnectionProfile(geliosPositionsCurrentExample.connection).ok, true); assert.equal(validateConnectionProfile(geliosPositionsCurrentExample.connection).ok, true);
assert.equal(validateConnectionProfile({
...geliosPositionsCurrentExample.connection,
credentialRef: { ...geliosPositionsCurrentExample.connection.credentialRef, owner: "engine" },
}).errors.includes("credentialRef.owner_must_be_ndc_l2_credentials"), true);
assert.equal(validateCollectionProfile(geliosPositionsCurrentExample.collectionProfile).ok, true); assert.equal(validateCollectionProfile(geliosPositionsCurrentExample.collectionProfile).ok, true);
assert.equal(validateDataProduct(geliosPositionsCurrentExample.dataProduct).ok, true); assert.equal(validateDataProduct(geliosPositionsCurrentExample.dataProduct).ok, true);
assert.equal(validateFoundryBinding(geliosPositionsCurrentExample.foundryBinding).ok, true); assert.equal(validateFoundryBinding(geliosPositionsCurrentExample.foundryBinding).ok, true);
@ -108,6 +112,10 @@ assert.equal(validateIntakeBatch({
...neutralIntakeBatch, ...neutralIntakeBatch,
facts: [{ ...neutralIntakeBatch.facts[0], attributes: { metadata: "ndc_edprb_abcdefghijklmnopqrstuvwxyz0123456789ABCDE" } }], facts: [{ ...neutralIntakeBatch.facts[0], attributes: { metadata: "ndc_edprb_abcdefghijklmnopqrstuvwxyz0123456789ABCDE" } }],
}).errors.includes("intake_must_not_contain_secret_material"), true); }).errors.includes("intake_must_not_contain_secret_material"), true);
assert.equal(validateIntakeBatch({
...neutralIntakeBatch,
facts: [{ ...neutralIntakeBatch.facts[0], attributes: { metadata: `ndc_edppr_${"P".repeat(43)}` } }],
}).errors.includes("intake_must_not_contain_secret_material"), true);
assert.equal(validateIntakeBatch({ assert.equal(validateIntakeBatch({
...neutralIntakeBatch, ...neutralIntakeBatch,
raw: { contentType: "application/json", hash: "prefix ndc_edpwb_abcdefghijklmnopqrstuvwxyz0123456789ABCDE suffix", ref: "restricted://raw/fixture" }, raw: { contentType: "application/json", hash: "prefix ndc_edpwb_abcdefghijklmnopqrstuvwxyz0123456789ABCDE suffix", ref: "restricted://raw/fixture" },

View File

@ -34,6 +34,10 @@ assert.equal(validateDataProductPublish({
...publish, ...publish,
facts: [{ ...fact, attributes: { accessToken: "must-never-cross-the-boundary" } }], facts: [{ ...fact, attributes: { accessToken: "must-never-cross-the-boundary" } }],
}).errors.includes("publish_must_not_contain_secret_material"), true); }).errors.includes("publish_must_not_contain_secret_material"), true);
assert.equal(validateDataProductPublish({
...publish,
facts: [{ ...fact, attributes: { status: `ndc_edppr_${"P".repeat(43)}` } }],
}).errors.includes("publish_must_not_contain_secret_material"), true);
assert.equal(validateDataProductPublish({ assert.equal(validateDataProductPublish({
...publish, ...publish,
facts: [fact, { ...fact, observedAt: "2026-07-15T10:00:01.000Z" }], facts: [fact, { ...fact, observedAt: "2026-07-15T10:00:01.000Z" }],
@ -85,6 +89,10 @@ assert.equal(validateDataProductSnapshot({
...snapshot, ...snapshot,
facts: [{ ...canonicalFact, attributes: { status: "ndc_edprb_forbidden-reader-token" } }], facts: [{ ...canonicalFact, attributes: { status: "ndc_edprb_forbidden-reader-token" } }],
}).errors.includes("snapshot_must_not_contain_secret_material"), true); }).errors.includes("snapshot_must_not_contain_secret_material"), true);
assert.equal(validateDataProductSnapshot({
...snapshot,
facts: [{ ...canonicalFact, attributes: { status: `ndc_edppr_${"P".repeat(43)}` } }],
}).errors.includes("snapshot_must_not_contain_secret_material"), true);
const patch = { const patch = {
schemaVersion: DATA_PRODUCT_PATCH_SCHEMA_VERSION, schemaVersion: DATA_PRODUCT_PATCH_SCHEMA_VERSION,

View File

@ -1,242 +0,0 @@
import assert from "node:assert/strict";
import { generateKeyPairSync, sign } from "node:crypto";
import {
ENGINE_CREDENTIAL_SINK_AUDIT_SCHEMA_VERSION,
ENGINE_CREDENTIAL_SINK_PROVISION_SCHEMA_VERSION,
ENGINE_CREDENTIAL_SINK_RECEIPT_SCHEMA_VERSION,
ENGINE_CREDENTIAL_SINK_ROLLBACK_RECEIPT_SCHEMA_VERSION,
ENGINE_CREDENTIAL_SINK_ROLLBACK_SCHEMA_VERSION,
computeEngineCredentialCapabilityDigest,
computeEngineCredentialSinkPolicyHash,
computeEngineCredentialSinkReceiptHash,
engineCredentialSinkAuditTargets,
validateEngineCredentialSinkAudit,
validateEngineCredentialSinkProvision,
validateEngineCredentialSinkReceipt,
validateEngineCredentialSinkRollback,
validateEngineCredentialSinkRollbackReceipt,
} from "../src/index.mjs";
const hash = (character) => `sha256:${character.repeat(64)}`;
const issuer = { serviceId: "platform.external-data-plane", keyId: "edp-issuer-20260715-001" };
const { publicKey: issuerPublicKey, privateKey: issuerPrivateKey } = generateKeyPairSync("ed25519");
const issuerPublicKeys = { [`${issuer.serviceId}:${issuer.keyId}`]: issuerPublicKey };
const target = (nodeId, nodeType, credentialType) => ({
workflowId: "WCb62yGL8v",
workflowRevision: "revision-20260715-001",
nodeId,
nodeType,
credentialType,
});
const request = {
schemaVersion: ENGINE_CREDENTIAL_SINK_PROVISION_SCHEMA_VERSION,
transaction: {
id: "credential-transaction-20260715-001",
idempotencyKey: "credential-transaction-20260715-001",
requestedAt: "2026-07-15T18:00:00.000Z",
requestExpiresAt: "2026-07-15T18:10:00.000Z",
policyHash: hash("0"),
failureMode: "rollback-all",
issuer,
attestation: { algorithm: "Ed25519", signature: "A".repeat(86) },
},
bindings: [
{
bindingId: "positions-writer",
capabilityType: "external-data-plane.writer",
grantId: "writer-grant-001",
target: target(
"publish-node-001",
"n8n-nodes-ndc.ndcDataProductPublish",
"ndcDataProductWriterApi",
),
expiresAt: "2026-08-15T18:00:00.000Z",
policyHash: hash("1"),
material: { format: "opaque-bearer", value: `ndc_edpwb_${"A".repeat(43)}` },
},
{
bindingId: "positions-reader",
capabilityType: "external-data-plane.reader",
grantId: "reader-grant-001",
target: target(
"read-node-001",
"n8n-nodes-ndc.ndcDataProductRead",
"ndcDataProductReaderApi",
),
expiresAt: "2026-08-15T18:00:00.000Z",
policyHash: hash("2"),
material: { format: "opaque-bearer", value: `ndc_edprb_${"B".repeat(43)}` },
},
{
bindingId: "positions-foundry",
capabilityType: "foundry.binding",
grantId: "foundry-grant-001",
target: target(
"foundry-node-001",
"n8n-nodes-ndc.ndcFoundryBinding",
"ndcFoundryBindingApi",
),
expiresAt: "2026-08-15T18:00:00.000Z",
policyHash: hash("3"),
material: { format: "opaque-bearer", value: `ndc_fndbg_${"C".repeat(43)}` },
},
],
};
for (const binding of request.bindings) {
binding.capabilityDigest = computeEngineCredentialCapabilityDigest(binding.material.value);
}
attest(request);
const provisionValidationNow = "2026-07-15T18:00:01.000Z";
assert.equal(validateEngineCredentialSinkProvision(request, { now: provisionValidationNow, issuerPublicKeys }).ok, true);
assert.equal(validateEngineCredentialSinkProvision(request, {
now: provisionValidationNow,
}).errors.includes("transaction.issuer_public_key_required"), true);
assert.equal(validateEngineCredentialSinkProvision({
...request,
transaction: { ...request.transaction, policyHash: hash("f") },
}, { now: provisionValidationNow, issuerPublicKeys }).errors.includes("transaction.policyHash_mismatch"), true);
assert.equal(validateEngineCredentialSinkProvision({
...request,
bindings: [{
...request.bindings[0],
target: { ...request.bindings[0].target, nodeType: "n8n-nodes-ndc.ndcDataProductRead" },
}],
}, { now: provisionValidationNow, issuerPublicKeys }).errors.includes("bindings[0].target.nodeType_capability_mismatch"), true);
assert.equal(validateEngineCredentialSinkProvision({
...request,
bindings: [{
...request.bindings[0],
material: { format: "opaque-bearer", value: request.bindings[1].material.value },
}],
}, { now: provisionValidationNow, issuerPublicKeys }).errors.includes("bindings[0].material.value_invalid_for_capability"), true);
const swappedCapabilityRequest = structuredClone(request);
swappedCapabilityRequest.bindings[0].material.value = `ndc_edpwb_${"D".repeat(43)}`;
assert.equal(validateEngineCredentialSinkProvision(swappedCapabilityRequest, {
now: provisionValidationNow,
issuerPublicKeys,
}).errors.includes("bindings[0].capabilityDigest_material_mismatch"), true);
const resignedByAttackerRequest = structuredClone(swappedCapabilityRequest);
resignedByAttackerRequest.bindings[0].capabilityDigest = computeEngineCredentialCapabilityDigest(
resignedByAttackerRequest.bindings[0].material.value,
);
resignedByAttackerRequest.transaction.policyHash = computeEngineCredentialSinkPolicyHash(resignedByAttackerRequest);
assert.equal(validateEngineCredentialSinkProvision(resignedByAttackerRequest, {
now: provisionValidationNow,
issuerPublicKeys,
}).errors.includes("transaction.attestation_invalid"), true);
const staleRequest = structuredClone(request);
staleRequest.transaction.requestedAt = "2020-01-01T00:00:00.000Z";
staleRequest.transaction.requestExpiresAt = "2020-01-01T00:10:00.000Z";
attest(staleRequest);
assert.equal(validateEngineCredentialSinkProvision(staleRequest, { now: provisionValidationNow, issuerPublicKeys }).errors.includes("request_expired"), true);
const futureRequest = structuredClone(request);
futureRequest.transaction.requestedAt = "2026-07-15T18:02:00.000Z";
futureRequest.transaction.requestExpiresAt = "2026-07-15T18:12:00.000Z";
attest(futureRequest);
assert.equal(validateEngineCredentialSinkProvision(futureRequest, { now: provisionValidationNow, issuerPublicKeys }).errors.includes("requestedAt_exceeds_clock_skew"), true);
const credentials = request.bindings.map((binding, index) => ({
bindingId: binding.bindingId,
capabilityType: binding.capabilityType,
grantId: binding.grantId,
target: binding.target,
credentialRef: `engcred_${index + 1}_opaque_reference`,
expiresAt: binding.expiresAt,
policyHash: binding.policyHash,
capabilityDigest: binding.capabilityDigest,
disposition: "created",
}));
const receipt = {
schemaVersion: ENGINE_CREDENTIAL_SINK_RECEIPT_SCHEMA_VERSION,
transactionId: request.transaction.id,
idempotencyKey: request.transaction.idempotencyKey,
outcome: "committed",
policyHash: request.transaction.policyHash,
processedAt: "2026-07-15T18:00:02.000Z",
credentials,
rollback: { status: "not-required" },
};
assert.equal(validateEngineCredentialSinkReceipt(receipt, { request, issuerPublicKeys }).ok, true);
assert.equal(validateEngineCredentialSinkReceipt({
...receipt,
credentials: [{ ...credentials[0], target: { ...credentials[0].target, nodeId: "wrong-node" } }, ...credentials.slice(1)],
}, { request, issuerPublicKeys }).errors.includes("credentials_request_target_mismatch"), true);
assert.equal(validateEngineCredentialSinkReceipt({
...receipt,
capability: request.bindings[0].material.value,
}).errors.includes("credentialSinkReceipt.capability_not_allowed"), true);
assert.equal(validateEngineCredentialSinkReceipt({
...receipt,
outcome: "rolled-back",
credentials: credentials.slice(0, 1),
rollback: { status: "complete", completedAt: "2026-07-15T18:00:02.000Z" },
errorCode: "engine_binding_failed",
}).errors.includes("noncommitted_credentials_must_be_empty"), true);
const committedReceiptHash = computeEngineCredentialSinkReceiptHash(receipt);
const rollbackRequest = {
schemaVersion: ENGINE_CREDENTIAL_SINK_ROLLBACK_SCHEMA_VERSION,
rollback: {
id: "credential-rollback-20260715-001",
idempotencyKey: "credential-rollback-20260715-001",
transactionId: request.transaction.id,
requestedAt: "2026-07-15T18:05:00.000Z",
requestExpiresAt: "2026-07-15T18:10:00.000Z",
policyHash: request.transaction.policyHash,
committedReceiptHash,
reasonCode: "operator_requested",
},
};
assert.equal(validateEngineCredentialSinkRollback(rollbackRequest, { now: "2026-07-15T18:05:01.000Z" }).ok, true);
const staleRollbackRequest = structuredClone(rollbackRequest);
staleRollbackRequest.rollback.requestedAt = "2020-01-01T00:00:00.000Z";
staleRollbackRequest.rollback.requestExpiresAt = "2020-01-01T00:10:00.000Z";
assert.equal(validateEngineCredentialSinkRollback(staleRollbackRequest, { now: "2026-07-15T18:05:01.000Z" }).errors.includes("request_expired"), true);
const rollbackReceipt = {
schemaVersion: ENGINE_CREDENTIAL_SINK_ROLLBACK_RECEIPT_SCHEMA_VERSION,
rollbackId: rollbackRequest.rollback.id,
transactionId: rollbackRequest.rollback.transactionId,
outcome: "rolled-back",
policyHash: rollbackRequest.rollback.policyHash,
committedReceiptHash,
processedAt: "2026-07-15T18:05:01.000Z",
};
assert.equal(validateEngineCredentialSinkRollbackReceipt(rollbackReceipt, { request: rollbackRequest }).ok, true);
const audit = {
schemaVersion: ENGINE_CREDENTIAL_SINK_AUDIT_SCHEMA_VERSION,
eventId: "credential-audit-20260715-001",
transactionId: request.transaction.id,
operationId: request.transaction.id,
operation: "provision",
outcome: "committed",
occurredAt: receipt.processedAt,
policyHash: request.transaction.policyHash,
principal: { serviceId: "platform.credential-provisioner", fingerprint: hash("a") },
targets: engineCredentialSinkAuditTargets(request, receipt),
};
assert.equal(validateEngineCredentialSinkAudit(audit).ok, true);
assert.equal(JSON.stringify(audit).includes("ndc_edpwb_"), false);
assert.equal(JSON.stringify(audit).includes("engcred_1_opaque_reference"), false);
assert.equal(validateEngineCredentialSinkAudit({
...audit,
authorization: `Bearer ${request.bindings[0].material.value}`,
}).errors.includes("audit_must_not_contain_secret_material"), true);
function attest(value) {
value.transaction.policyHash = computeEngineCredentialSinkPolicyHash(value);
value.transaction.attestation.signature = sign(
null,
Buffer.from(value.transaction.policyHash, "utf8"),
issuerPrivateKey,
).toString("base64url");
}
console.log("external-provider credential sink contract: ok");

View File

@ -132,6 +132,16 @@ assert.equal(
}).errors.includes("transition_policy_mismatch"), }).errors.includes("transition_policy_mismatch"),
true, true,
); );
assert.equal(
validateEnginePrivateExtensionPlan({
...activatePlan,
transition: {
...transition(),
loaderEnvironment: { ...transition().loaderEnvironment, NODE_PATH: "/tmp/untrusted-node-modules" },
},
}).errors.includes("transition.loaderEnvironment.NODE_PATH_not_allowed"),
true,
);
const applyRequest = { const applyRequest = {
schemaVersion: ENGINE_PRIVATE_EXTENSION_APPLY_REQUEST_SCHEMA_VERSION, schemaVersion: ENGINE_PRIVATE_EXTENSION_APPLY_REQUEST_SCHEMA_VERSION,

View File

@ -0,0 +1,464 @@
import assert from "node:assert/strict";
import { readFile } from "node:fs/promises";
import {
L2_CONNECTION_INSTANCE_SCHEMA_VERSION,
instantiateL2Connection,
validateDataProductPublish,
validateProviderPackage,
} from "../src/index.mjs";
import {
GELIOS_POSITIONS_DATA_PRODUCT_ID,
GELIOS_POSITIONS_DATA_PRODUCT_VERSION,
GELIOS_POSITIONS_ONTOLOGY_REVISION,
GELIOS_UNIT_SOURCE_ID_PREFIX,
geliosProviderPackageV1,
geliosUnitsCurrentFixtureV1,
} from "../providers/gelios/v1/index.mjs";
import { normalizeDataProductDefinition } from "../../../services/external-data-plane/src/data-product-policy.mjs";
const expectedFields = [
"course_degrees",
"display_name",
"elevation_meters",
"geometry",
"hdop",
"horizontal_accuracy_meters",
"object_kind",
"operational_status",
"position_source",
"position_valid",
"quality_flags",
"satellite_count",
"speed_kph",
];
assert.deepEqual(validateProviderPackage(geliosProviderPackageV1), { ok: true, errors: [] });
const geliosAuth = geliosProviderPackageV1.authModes[0];
const geliosUnitsRead = geliosProviderPackageV1.capabilities[0];
assert.equal(geliosAuth.id, "gelios.rest-bearer.v1");
assert.deepEqual(geliosAuth.transport, { placement: "header", name: "Authorization" });
assert.deepEqual(geliosAuth.tokenLifecycle, {
artifacts: ["access", "refresh"],
requestArtifact: "access",
refreshMode: "operator_managed",
});
assert.equal(geliosAuth.tokenLifecycle.artifacts.includes("writer"), false);
assert.equal(geliosUnitsRead.request.baseUrl, "https://api.geliospro.com");
assert.equal(geliosUnitsRead.request.path, "/api/v1/units");
assert.deepEqual(geliosUnitsRead.request.query, {});
const product = geliosProviderPackageV1.dataProducts[0];
const registeredProduct = JSON.parse(await readFile(new URL(
"../../../services/external-data-plane/definitions/fleet.positions.current.v1.json",
import.meta.url,
), "utf8"));
assert.equal(product.id, GELIOS_POSITIONS_DATA_PRODUCT_ID);
assert.equal(product.version, GELIOS_POSITIONS_DATA_PRODUCT_VERSION);
assert.equal(product.ontologyRevision, GELIOS_POSITIONS_ONTOLOGY_REVISION);
assert.equal(product.id, "fleet.positions.current.v1");
assert.equal(product.version, "1.0.0");
assert.equal(product.ontologyRevision, "ontology.map.moving_object.v1");
assert.deepEqual(product.fields, expectedFields);
assert.deepEqual(product, registeredProduct);
assert.doesNotThrow(() => normalizeDataProductDefinition(product));
assert.equal(product.fields.every((field) => /^[a-z][a-z0-9_]*$/.test(field)), true);
const mapping = geliosProviderPackageV1.mappingContracts[0];
assert.equal(mapping.target.dataProductId, product.id);
assert.equal(mapping.target.version, product.version);
assert.equal(mapping.target.ontologyRevision, product.ontologyRevision);
assert.deepEqual(
[...Object.keys(mapping.fact.attributes), "geometry"].sort(),
[...product.fields].sort(),
);
assert.equal(mapping.derivations.operational_status.parameters.staleAfterMs, 60000);
assert.deepEqual(mapping.derivations.quality_flags.default, []);
assert.equal(mapping.fact.observedAt.fallback, "collection_received_at");
assert.equal(GELIOS_UNIT_SOURCE_ID_PREFIX, "gelios-unit-");
assert.equal(mapping.fact.sourceId.prefix, GELIOS_UNIT_SOURCE_ID_PREFIX);
assert.equal(mapping.fact.sourceId.coerce, "string");
assert.equal(mapping.fact.geometry.longitude.paths.includes("lastMsg.lon"), true);
assert.equal(mapping.fact.geometry.latitude.paths.includes("lastMsg.lat"), true);
assert.equal(mapping.fact.attributes.speed_kph.paths.includes("lastMsg.speed"), true);
assert.equal(validateDataProductPublish(geliosUnitsCurrentFixtureV1.expectedPublish).ok, true);
assert.equal(
geliosUnitsCurrentFixtureV1.sourceResponse.units.length,
geliosUnitsCurrentFixtureV1.expectedPublish.facts.length,
"every entity returned for the bound credential remains in the publish fixture",
);
assert.deepEqual(
geliosUnitsCurrentFixtureV1.expectedPublish.facts.map((fact) => fact.sourceId),
geliosUnitsCurrentFixtureV1.sourceResponse.units.map(
(unit) => `${GELIOS_UNIT_SOURCE_ID_PREFIX}${String(unit.id)}`,
),
);
assert.equal(
geliosUnitsCurrentFixtureV1.expectedPublish.facts[1].sourceId,
"gelios-unit-001002",
"provider ID strings must remain byte-for-byte stable after the canonical prefix",
);
assert.equal(geliosUnitsCurrentFixtureV1.expectedPublish.facts[1].geometry, undefined);
assert.equal(geliosUnitsCurrentFixtureV1.expectedPublish.facts[1].attributes.operational_status, "no_position");
assert.equal(geliosUnitsCurrentFixtureV1.sourceResponse.units[1].lastMsg, undefined);
assert.equal(
geliosUnitsCurrentFixtureV1.expectedPublish.facts[1].observedAt,
geliosUnitsCurrentFixtureV1.collectionContext.receivedAt,
);
const realtimeProfileId = "gelios.positions.current.realtime.v1";
assert.equal(
geliosProviderPackageV1.collectionProfiles[0].cardinality.onExceed,
"require_partitioned_data_product",
);
const accountA = instantiateL2Connection(geliosProviderPackageV1, {
tenantId: "tenant-alpha",
connectionId: "gelios-account-alpha",
collectionProfileId: realtimeProfileId,
providerCredentialRef: "ndc-credref:provider-alpha-0001",
});
const accountB = instantiateL2Connection(geliosProviderPackageV1, {
tenantId: "tenant-beta",
connectionId: "gelios-account-beta",
collectionProfileId: realtimeProfileId,
providerCredentialRef: "ndc-credref:provider-beta-0001",
});
assert.equal(accountA.schemaVersion, L2_CONNECTION_INSTANCE_SCHEMA_VERSION);
assert.equal(accountA.package.id, accountB.package.id);
assert.equal(accountA.package.version, accountB.package.version);
assert.equal(accountA.l2TemplateId, accountB.l2TemplateId);
assert.notEqual(accountA.connectionId, accountB.connectionId);
assert.notEqual(accountA.credentialRefs.provider.reference, accountB.credentialRefs.provider.reference);
assert.equal(accountA.credentialRefs.provider.owner, "ndc_l2_credentials");
assert.deepEqual(Object.keys(accountA.credentialRefs), ["provider"]);
assert.deepEqual(accountA.systemBindings.publisher, {
role: "publisher",
owner: "ndc_l2_credentials",
management: "control_plane_managed",
nodeType: "n8n-nodes-ndc.ndcDataProductPublish",
desiredState: "bound",
status: "unresolved",
});
assert.equal(accountA.scope.mode, "all_visible_to_credential");
assert.equal(accountA.scope.refresh, "each_collection_run");
assert.equal(Object.isFrozen(accountA), true);
assert.equal(Object.isFrozen(accountA.credentialRefs), true);
assert.equal(Object.isFrozen(accountA.systemBindings), true);
const withEntityAllowlist = structuredClone(geliosProviderPackageV1);
withEntityAllowlist.capabilities[0].entityScope.unitIds = ["gelios-unit-1001"];
assert.equal(
validateProviderPackage(withEntityAllowlist).errors.includes("capabilities[0].entityScope.unitIds_not_allowed"),
true,
);
const withGroupFilter = structuredClone(geliosProviderPackageV1);
withGroupFilter.collectionProfiles[0].entityScope.businessEntityFilter = "provider_group";
assert.equal(
validateProviderPackage(withGroupFilter).errors.includes("collectionProfiles[0].entityScope.businessEntityFilter_must_be_forbidden"),
true,
);
const withWrongRevision = structuredClone(geliosProviderPackageV1);
withWrongRevision.mappingContracts[0].target.ontologyRevision = "gelios.positions.v1";
assert.equal(
validateProviderPackage(withWrongRevision).errors.includes("mappingContract.gelios.units.to.fleet.positions.current.v1.target.ontologyRevision_must_match_data_product"),
true,
);
const withMissingDerivation = structuredClone(geliosProviderPackageV1);
delete withMissingDerivation.mappingContracts[0].derivations.position_valid;
assert.equal(
validateProviderPackage(withMissingDerivation).errors.includes("mappingContracts[0].derive_position_valid_missing_definition"),
true,
);
const withWrongFactSemanticType = structuredClone(geliosProviderPackageV1);
withWrongFactSemanticType.mappingContracts[0].fact.semanticType.constant = "map.other";
assert.equal(
validateProviderPackage(withWrongFactSemanticType).errors.includes(
"mappingContract.gelios.units.to.fleet.positions.current.v1.fact.semanticType_must_equal_target_constant",
),
true,
);
const nonGeospatialPackage = structuredClone(geliosProviderPackageV1);
nonGeospatialPackage.dataProducts[0].fields = ["display_name"];
nonGeospatialPackage.fieldPolicies[0].targetFields = ["display_name"];
delete nonGeospatialPackage.mappingContracts[0].fact.geometry;
nonGeospatialPackage.mappingContracts[0].fact.attributes = {
display_name: { strategy: "first_non_empty", paths: ["name"], coerce: "string" },
};
delete nonGeospatialPackage.mappingContracts[0].derivations;
assert.deepEqual(validateProviderPackage(nonGeospatialPackage), { ok: true, errors: [] });
for (const restrictedPath of ["phone", "phone.number", "lmsg"]) {
const withRestrictedMappingPath = structuredClone(geliosProviderPackageV1);
withRestrictedMappingPath.mappingContracts[0].fact.attributes.display_name.paths = [restrictedPath];
assert.equal(
validateProviderPackage(withRestrictedMappingPath).errors.includes(
"mappingContract.gelios.units.to.fleet.positions.current.v1.fact_source_path_restricted",
),
true,
);
}
const withFieldPolicyProductMismatch = structuredClone(geliosProviderPackageV1);
const secondProduct = structuredClone(withFieldPolicyProductMismatch.dataProducts[0]);
secondProduct.id = "fleet.positions.alternate.v1";
withFieldPolicyProductMismatch.dataProducts.push(secondProduct);
withFieldPolicyProductMismatch.manifest.dataProductIds.push(secondProduct.id);
withFieldPolicyProductMismatch.fieldPolicies[0].dataProductId = secondProduct.id;
assert.equal(
validateProviderPackage(withFieldPolicyProductMismatch).errors.includes(
"mappingContract.gelios.units.to.fleet.positions.current.v1.field_policy_data_product_mismatch",
),
true,
);
const withCamelCaseField = structuredClone(geliosProviderPackageV1);
withCamelCaseField.dataProducts[0].fields[12] = "speedKph";
assert.equal(validateProviderPackage(withCamelCaseField).ok, false);
const withSecretField = structuredClone(geliosProviderPackageV1);
withSecretField.dataProducts[0].fields[12] = "api_token";
withSecretField.fieldPolicies[0].targetFields[12] = "api_token";
withSecretField.mappingContracts[0].fact.attributes.api_token = { constant: "metadata-only" };
delete withSecretField.mappingContracts[0].fact.attributes.speed_kph;
const secretFieldValidation = validateProviderPackage(withSecretField);
assert.equal(secretFieldValidation.errors.includes("dataProducts[0].fields_must_not_contain_secret_fields"), true);
assert.equal(secretFieldValidation.errors.includes("fieldPolicies[0].targetFields_must_not_contain_secret_fields"), true);
assert.equal(secretFieldValidation.errors.includes("mappingContracts[0].fact.attributes.api_token_secret_field_not_allowed"), true);
const withRuntimeAccountState = structuredClone(geliosProviderPackageV1);
withRuntimeAccountState.tenantId = "must-not-live-in-package";
assert.equal(
validateProviderPackage(withRuntimeAccountState).errors.includes("providerPackage_must_not_contain_connection_instance_state"),
true,
);
const withPrimitiveCapability = structuredClone(geliosProviderPackageV1);
withPrimitiveCapability.capabilities.push(null);
assert.doesNotThrow(() => validateProviderPackage(withPrimitiveCapability));
assert.equal(validateProviderPackage(withPrimitiveCapability).ok, false);
const withMalformedNestedArrays = structuredClone(geliosProviderPackageV1);
withMalformedNestedArrays.collectionProfiles[0].capabilityIds = {};
withMalformedNestedArrays.l2Templates[0].credentialBindings = {};
withMalformedNestedArrays.l2Templates[0].steps = {};
assert.doesNotThrow(() => validateProviderPackage(withMalformedNestedArrays));
assert.equal(validateProviderPackage(withMalformedNestedArrays).ok, false);
const withCredentialInStaticQuery = structuredClone(geliosProviderPackageV1);
withCredentialInStaticQuery.authModes[0].transport = { placement: "query", name: "token" };
withCredentialInStaticQuery.capabilities[0].request.query.token = "plain-provider-key";
assert.equal(
validateProviderPackage(withCredentialInStaticQuery).errors.includes("capabilities[0].request.query.token_credential_parameter_not_allowed"),
true,
);
assert.equal(
validateProviderPackage(withCredentialInStaticQuery).errors.includes("capability.gelios.units.current.read.request.query_must_not_embed_credential_parameter"),
true,
);
const withCredentialInBaseUrl = structuredClone(geliosProviderPackageV1);
withCredentialInBaseUrl.capabilities[0].request.baseUrl = "https://user:pass@api.geliospro.com/?token=forbidden";
assert.equal(validateProviderPackage(withCredentialInBaseUrl).ok, false);
for (const unsafePath of ["/ok\nX-Evil: yes", "/back\\slash", "/query?x=1"]) {
const withUnsafeRequestPath = structuredClone(geliosProviderPackageV1);
withUnsafeRequestPath.capabilities[0].request.path = unsafePath;
assert.equal(validateProviderPackage(withUnsafeRequestPath).ok, false);
}
for (const unsafeCollectionPath of ["__proto__.polluted", "data.items[0]", "constructor.prototype"]) {
const withUnsafeCollectionPath = structuredClone(geliosProviderPackageV1);
withUnsafeCollectionPath.capabilities[0].request.response.collectionPaths = [unsafeCollectionPath];
assert.equal(validateProviderPackage(withUnsafeCollectionPath).ok, false);
}
const withUnsafeMappingPath = structuredClone(geliosProviderPackageV1);
withUnsafeMappingPath.mappingContracts[0].fact.attributes.display_name.paths = ["__proto__.polluted"];
assert.equal(validateProviderPackage(withUnsafeMappingPath).ok, false);
const withUnsafeCollectionCapability = structuredClone(geliosProviderPackageV1);
withUnsafeCollectionCapability.capabilities[0].classification = "write";
assert.equal(
validateProviderPackage(withUnsafeCollectionCapability).errors.includes("collectionProfile.gelios.positions.current.realtime.v1.capabilityIds_must_reference_implemented_safe_read"),
true,
);
const withUnexecutedSecondCapability = structuredClone(geliosProviderPackageV1);
const secondCapability = structuredClone(withUnexecutedSecondCapability.capabilities[0]);
secondCapability.id = "gelios.units.metadata.read";
withUnexecutedSecondCapability.capabilities.push(secondCapability);
withUnexecutedSecondCapability.manifest.capabilityIds.push(secondCapability.id);
for (const profile of withUnexecutedSecondCapability.collectionProfiles) {
profile.capabilityIds.push(secondCapability.id);
}
assert.equal(
validateProviderPackage(withUnexecutedSecondCapability).errors.includes(
"collectionProfile.gelios.positions.current.realtime.v1.capabilityIds_must_select_one_v1_read",
),
true,
);
const withReorderedSteps = structuredClone(geliosProviderPackageV1);
[withReorderedSteps.l2Templates[0].steps[0], withReorderedSteps.l2Templates[0].steps[1]] = [
withReorderedSteps.l2Templates[0].steps[1],
withReorderedSteps.l2Templates[0].steps[0],
];
assert.equal(
validateProviderPackage(withReorderedSteps).errors.includes("l2Templates[0].steps_sequence_invalid"),
true,
);
const withUnknownConnectionParameters = structuredClone(geliosProviderPackageV1);
withUnknownConnectionParameters.l2Templates[0].connectionParameters = ["foo"];
assert.equal(
validateProviderPackage(withUnknownConnectionParameters).errors.includes(
"l2Templates[0].connectionParameters_must_match_v1_contract",
),
true,
);
const withCredentialAuthMismatch = structuredClone(geliosProviderPackageV1);
const secondAuthMode = structuredClone(withCredentialAuthMismatch.authModes[0]);
secondAuthMode.id = "gelios.rest-bearer.alternate.v1";
withCredentialAuthMismatch.authModes.push(secondAuthMode);
withCredentialAuthMismatch.manifest.authModeIds.push(secondAuthMode.id);
withCredentialAuthMismatch.l2Templates[0].credentialBindings[0].authModeId = secondAuthMode.id;
assert.equal(
validateProviderPackage(withCredentialAuthMismatch).errors.includes(
"l2Template.gelios.positions.current.l2.v1.provider_credential_auth_mode_mismatch",
),
true,
);
const withCallerManagedPublisher = structuredClone(geliosProviderPackageV1);
withCallerManagedPublisher.l2Templates[0].credentialBindings[1].management = "connection_input";
assert.equal(
validateProviderPackage(withCallerManagedPublisher).errors.includes(
"l2Templates[0].credentialBindings[1].publisher_binding_shape_invalid",
),
true,
);
const withUnknownTokenArtifact = structuredClone(geliosProviderPackageV1);
withUnknownTokenArtifact.authModes[0].tokenLifecycle.artifacts.push("writer");
assert.equal(
validateProviderPackage(withUnknownTokenArtifact).errors.includes(
"authModes[0].tokenLifecycle.artifacts_invalid",
),
true,
);
const withRefreshUsedForRequest = structuredClone(geliosProviderPackageV1);
withRefreshUsedForRequest.authModes[0].tokenLifecycle.requestArtifact = "refresh";
assert.equal(
validateProviderPackage(withRefreshUsedForRequest).errors.includes(
"authModes[0].tokenLifecycle.requestArtifact_must_be_access",
),
true,
);
const withUnmanagedRefreshArtifact = structuredClone(geliosProviderPackageV1);
withUnmanagedRefreshArtifact.authModes[0].tokenLifecycle.refreshMode = "not_applicable";
assert.equal(
validateProviderPackage(withUnmanagedRefreshArtifact).errors.includes(
"authModes[0].tokenLifecycle.refreshMode_required_for_refresh_artifact",
),
true,
);
for (const history of [{
mode: "none",
intervalMs: 60000,
strategy: "latest-per-entity-per-bucket",
retentionDays: 90,
}, {
mode: "sampled",
intervalMs: 24 * 60 * 60 * 1000 + 1,
strategy: "latest-per-entity-per-bucket",
retentionDays: 90,
}, {
mode: "all",
retentionDays: 3651,
}]) {
const withInvalidHistory = structuredClone(geliosProviderPackageV1);
withInvalidHistory.dataProducts[0].history = history;
assert.equal(validateProviderPackage(withInvalidHistory).ok, false);
assert.throws(() => normalizeDataProductDefinition(withInvalidHistory.dataProducts[0]));
}
const withBrokenArtifactChain = structuredClone(geliosProviderPackageV1);
withBrokenArtifactChain.l2Templates[0].steps[4].dataProductId = "unrelated.product.v1";
assert.equal(
validateProviderPackage(withBrokenArtifactChain).errors.includes("collectionProfile.gelios.positions.current.realtime.v1.template_data_product_mismatch"),
true,
);
const withSecretNamedDerivationParameter = structuredClone(geliosProviderPackageV1);
withSecretNamedDerivationParameter.mappingContracts[0].derivations.position_valid.parameters = {
accessToken: "opaque-provider-value",
};
assert.equal(
validateProviderPackage(withSecretNamedDerivationParameter).errors.includes(
"mappingContracts[0].derivations.position_valid.parameters.accessToken_secret_field_not_allowed",
),
true,
);
const withObjectConstant = structuredClone(geliosProviderPackageV1);
withObjectConstant.mappingContracts[0].fact.attributes.object_kind.constant = {
accessToken: "opaque-provider-value",
};
assert.equal(
validateProviderPackage(withObjectConstant).errors.includes(
"mappingContracts[0].fact.attributes.object_kind.constant_must_be_scalar_or_scalar_array",
),
true,
);
assert.throws(() => instantiateL2Connection(geliosProviderPackageV1, {
tenantId: "tenant-alpha",
connectionId: "gelios-account-alpha",
collectionProfileId: realtimeProfileId,
providerCredentialRef: "Bearer plaintext-must-not-cross-boundary",
}), /l2_connection_invalid/);
assert.throws(() => instantiateL2Connection(geliosProviderPackageV1, {
tenantId: "tenant-alpha",
connectionId: "gelios-account-alpha",
collectionProfileId: realtimeProfileId,
providerCredentialRef: "ndc-credref:provider-alpha-0001",
writerCredentialRef: "ndc-credref:caller-writer-forbidden",
}), /options\.writerCredentialRef_not_allowed/);
const serializedPackage = JSON.stringify(geliosProviderPackageV1);
assert.equal(serializedPackage.includes('"prefix":"unit-"'), false);
assert.equal(serializedPackage.includes("unitIds"), false);
assert.equal(serializedPackage.includes("tenant-alpha"), false);
assert.equal(serializedPackage.includes("gelios-gateway"), false);
assert.equal(serializedPackage.includes("writer_credential_ref"), false);
assert.equal(serializedPackage.includes("writerCapabilityByReferenceOnly"), false);
assert.equal(serializedPackage.includes("publisherBindingControlPlaneManaged\":true"), true);
assert.equal(serializedPackage.includes("providerSpecificServiceForbidden\":true"), true);
assert.deepEqual(
geliosProviderPackageV1.l2Templates[0].steps.map((step) => step.kind),
[
"collection_trigger",
"provider_request",
"extract_items",
"semantic_mapping",
"data_product_publish",
],
);
assert.equal(
geliosProviderPackageV1.l2Templates[0].steps.at(-1).nodeType,
"n8n-nodes-ndc.ndcDataProductPublish",
);
console.log("external-provider-contract provider-package: ok");

View File

@ -2,7 +2,12 @@ FROM node:20-alpine AS deps
WORKDIR /workspace WORKDIR /workspace
COPY packages/external-provider-contract ./packages/external-provider-contract COPY packages/external-provider-contract/package.json ./packages/external-provider-contract/package.json
COPY packages/external-provider-contract/src/contract-version.mjs ./packages/external-provider-contract/src/contract-version.mjs
COPY packages/external-provider-contract/src/data-plane.mjs ./packages/external-provider-contract/src/data-plane.mjs
COPY packages/external-provider-contract/src/data-product.mjs ./packages/external-provider-contract/src/data-product.mjs
COPY packages/external-provider-contract/src/intake-batch.mjs ./packages/external-provider-contract/src/intake-batch.mjs
COPY packages/external-provider-contract/src/sensitive-field-policy.mjs ./packages/external-provider-contract/src/sensitive-field-policy.mjs
COPY services/external-data-plane/package.json services/external-data-plane/package-lock.json ./services/external-data-plane/ COPY services/external-data-plane/package.json services/external-data-plane/package-lock.json ./services/external-data-plane/
WORKDIR /workspace/services/external-data-plane WORKDIR /workspace/services/external-data-plane
@ -21,8 +26,8 @@ COPY --from=deps /workspace/packages/external-provider-contract /packages/extern
COPY services/external-data-plane/src ./src COPY services/external-data-plane/src ./src
COPY services/external-data-plane/definitions ./definitions COPY services/external-data-plane/definitions ./definitions
# A dedicated numeric identity can read only the EDP provisioning secret mount; # A dedicated numeric identity owns the legacy EDP bearer boundary and reads the
# it is intentionally not the shared Node/Map Gateway uid/gid (1000). # separate public Engine trust mount; it is not the shared service uid/gid.
RUN addgroup -S -g 11006 nodedc-edp && adduser -S -D -H -u 11006 -G nodedc-edp nodedc-edp RUN addgroup -S -g 11006 nodedc-edp && adduser -S -D -H -u 11006 -G nodedc-edp nodedc-edp
USER 11006:11006 USER 11006:11006

View File

@ -6,9 +6,10 @@
"scripts": { "scripts": {
"start": "node src/server.mjs", "start": "node src/server.mjs",
"dev": "node --watch src/server.mjs", "dev": "node --watch src/server.mjs",
"check": "node --check src/server.mjs && node --check src/schema.mjs && node --check src/config.mjs && node --check src/intake-policy.mjs && node --check src/writer-binding.mjs && node --check src/reader-binding.mjs && node --check src/data-product-policy.mjs && node --check src/data-product-delivery.mjs && node --check src/definitions.mjs", "check": "node --check src/server.mjs && node --check src/schema.mjs && node --check src/config.mjs && node --check src/intake-policy.mjs && node --check src/writer-binding.mjs && node --check src/reader-binding.mjs && node --check src/data-product-policy.mjs && node --check src/data-product-delivery.mjs && node --check src/definitions.mjs && node --check src/managed-provisioner-auth.mjs",
"test": "node test/config.test.mjs && node test/intake-policy.test.mjs && node test/writer-binding.test.mjs && node test/reader-binding.test.mjs && node test/data-product-policy.test.mjs && node test/definitions.test.mjs", "test": "node test/config.test.mjs && node test/managed-provisioner-auth.test.mjs && node test/intake-policy.test.mjs && node test/writer-binding.test.mjs && node test/reader-binding.test.mjs && node test/data-product-policy.test.mjs && node test/definitions.test.mjs",
"test:integration": "node test/data-product-delivery.integration.test.mjs && node test/api.integration.test.mjs" "test:integration": "node test/data-product-delivery.integration.test.mjs && node test/api.integration.test.mjs",
"test:all": "npm test && npm run test:integration"
}, },
"dependencies": { "dependencies": {
"@nodedc/external-provider-contract": "file:../../packages/external-provider-contract", "@nodedc/external-provider-contract": "file:../../packages/external-provider-contract",

View File

@ -1,14 +1,54 @@
import { readFileSync } from "node:fs"; import { readFileSync } from "node:fs";
import {
loadManagedProvisionerPublicKeyFile,
validateManagedProvisionerAudience,
validateManagedProvisionerIdentity,
} from "./managed-provisioner-auth.mjs";
export function readConfig(env = process.env) { export function readConfig(env = process.env) {
const provisionerApiEnabled = boolean(env.EXTERNAL_DATA_PLANE_PROVISIONING_ENABLED, false); const provisionerApiEnabled = boolean(env.EXTERNAL_DATA_PLANE_PROVISIONING_ENABLED, false);
const managedProvisionerApiEnabled = boolean(env.EXTERNAL_DATA_PLANE_MANAGED_PROVISIONING_ENABLED, false);
const managedProvisionerMaxSkewSeconds = integer(
env.EXTERNAL_DATA_PLANE_MANAGED_PROVISIONER_MAX_SKEW_SECONDS,
60,
5,
300,
);
const config = { const config = {
port: integer(env.PORT, 18106, 1, 65535), port: integer(env.PORT, 18106, 1, 65535),
databaseUrl: required(env.EXTERNAL_DATA_PLANE_DATABASE_URL, "EXTERNAL_DATA_PLANE_DATABASE_URL"), databaseUrl: required(env.EXTERNAL_DATA_PLANE_DATABASE_URL, "EXTERNAL_DATA_PLANE_DATABASE_URL"),
databasePoolSize: integer(env.EXTERNAL_DATA_PLANE_DATABASE_POOL_SIZE, 10, 1, 50), databasePoolSize: integer(env.EXTERNAL_DATA_PLANE_DATABASE_POOL_SIZE, 10, 1, 50),
internalAccessToken: optional(env.NODEDC_INTERNAL_ACCESS_TOKEN), internalAccessToken: optional(env.NODEDC_INTERNAL_ACCESS_TOKEN),
provisionerApiEnabled, provisionerApiEnabled,
managedProvisionerApiEnabled,
provisionerAccessToken: provisionerApiEnabled ? secretFile(env.EXTERNAL_DATA_PLANE_PROVISIONER_TOKEN_FILE) : "", provisionerAccessToken: provisionerApiEnabled ? secretFile(env.EXTERNAL_DATA_PLANE_PROVISIONER_TOKEN_FILE) : "",
managedProvisionerPublicKey: managedProvisionerApiEnabled
? loadManagedProvisionerPublicKeyFile(env.EXTERNAL_DATA_PLANE_MANAGED_PROVISIONER_PUBLIC_KEY_FILE)
: null,
managedProvisionerServiceId: managedProvisionerApiEnabled
? validateManagedProvisionerIdentity(
required(env.EXTERNAL_DATA_PLANE_MANAGED_PROVISIONER_SERVICE_ID, "EXTERNAL_DATA_PLANE_MANAGED_PROVISIONER_SERVICE_ID"),
"service_id",
)
: "",
managedProvisionerKeyId: managedProvisionerApiEnabled
? validateManagedProvisionerIdentity(
required(env.EXTERNAL_DATA_PLANE_MANAGED_PROVISIONER_KEY_ID, "EXTERNAL_DATA_PLANE_MANAGED_PROVISIONER_KEY_ID"),
"key_id",
)
: "",
managedProvisionerAudience: managedProvisionerApiEnabled
? validateManagedProvisionerAudience(
required(env.EXTERNAL_DATA_PLANE_MANAGED_PROVISIONER_AUDIENCE, "EXTERNAL_DATA_PLANE_MANAGED_PROVISIONER_AUDIENCE"),
)
: "",
managedProvisionerMaxSkewMs: managedProvisionerMaxSkewSeconds * 1000,
managedProvisionerReplayCacheMaxEntries: integer(
env.EXTERNAL_DATA_PLANE_MANAGED_PROVISIONER_REPLAY_CACHE_MAX_ENTRIES,
10_000,
100,
100_000,
),
rawRetentionDays: integer(env.EXTERNAL_DATA_PLANE_RAW_RETENTION_DAYS, 14, 1, 3650), rawRetentionDays: integer(env.EXTERNAL_DATA_PLANE_RAW_RETENTION_DAYS, 14, 1, 3650),
maxBatchBytes: integer(env.EXTERNAL_DATA_PLANE_MAX_BATCH_BYTES, 5 * 1024 * 1024, 1024, 50 * 1024 * 1024), maxBatchBytes: integer(env.EXTERNAL_DATA_PLANE_MAX_BATCH_BYTES, 5 * 1024 * 1024, 1024, 50 * 1024 * 1024),
maxFactsPerPublish: integer(env.EXTERNAL_DATA_PLANE_MAX_FACTS_PER_PUBLISH, 5000, 1, 100_000), maxFactsPerPublish: integer(env.EXTERNAL_DATA_PLANE_MAX_FACTS_PER_PUBLISH, 5000, 1, 100_000),

View File

@ -2,7 +2,7 @@ import { createHash, randomUUID } from "node:crypto";
import { import {
DATA_PRODUCT_PATCH_SCHEMA_VERSION, DATA_PRODUCT_PATCH_SCHEMA_VERSION,
DATA_PRODUCT_SNAPSHOT_SCHEMA_VERSION, DATA_PRODUCT_SNAPSHOT_SCHEMA_VERSION,
} from "@nodedc/external-provider-contract"; } from "@nodedc/external-provider-contract/data-plane";
export async function loadDataProductDefinition(db, dataProductId, { activeOnly = true } = {}) { export async function loadDataProductDefinition(db, dataProductId, { activeOnly = true } = {}) {
const result = await db.query( const result = await db.query(

View File

@ -0,0 +1,318 @@
import { createHash, createPublicKey, timingSafeEqual, verify as verifySignature } from "node:crypto";
import { closeSync, constants as fsConstants, fstatSync, lstatSync, openSync, readFileSync } from "node:fs";
import { isAbsolute } from "node:path";
export const MANAGED_PROVISIONER_SIGNATURE_SCHEMA = "nodedc.external-data-plane.managed-provisioner-request/v1";
export const MANAGED_PROVISIONER_HEADERS = Object.freeze({
serviceId: "x-nodedc-engine-service-id",
keyId: "x-nodedc-engine-key-id",
audience: "x-nodedc-request-audience",
timestamp: "x-nodedc-request-timestamp",
nonce: "x-nodedc-request-nonce",
bodySha256: "x-nodedc-content-sha256",
signature: "x-nodedc-request-signature",
});
const ISO_TIMESTAMP = /^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}\.\d{3}Z$/;
const BASE64URL = /^[A-Za-z0-9_-]+$/;
const IDENTITY = /^[a-z][a-z0-9._:-]{2,127}$/i;
const AUDIENCE = /^[a-z][a-z0-9._:/-]{2,255}$/i;
export function loadManagedProvisionerPublicKeyFile(pathValue) {
const path = String(pathValue ?? "").trim();
if (!path) throw configError("external_data_plane_managed_provisioner_public_key_file_required");
if (!isAbsolute(path)) throw configError("external_data_plane_managed_provisioner_public_key_file_must_be_absolute");
let stat;
try {
stat = lstatSync(path);
} catch {
throw configError("external_data_plane_managed_provisioner_public_key_file_unreadable");
}
if (!stat.isFile() || stat.isSymbolicLink()) {
throw configError("external_data_plane_managed_provisioner_public_key_file_must_be_regular");
}
if ((stat.mode & 0o022) !== 0) {
throw configError("external_data_plane_managed_provisioner_public_key_file_permissions_invalid");
}
if (stat.size < 1 || stat.size > 8192) {
throw configError("external_data_plane_managed_provisioner_public_key_file_size_invalid");
}
let descriptor;
try {
descriptor = openSync(path, fsConstants.O_RDONLY | fsConstants.O_NOFOLLOW);
} catch {
throw configError("external_data_plane_managed_provisioner_public_key_file_unreadable");
}
let pem;
try {
const openedStat = fstatSync(descriptor);
if (!openedStat.isFile()) {
throw configError("external_data_plane_managed_provisioner_public_key_file_must_be_regular");
}
if ((openedStat.mode & 0o022) !== 0) {
throw configError("external_data_plane_managed_provisioner_public_key_file_permissions_invalid");
}
if (openedStat.size < 1 || openedStat.size > 8192) {
throw configError("external_data_plane_managed_provisioner_public_key_file_size_invalid");
}
pem = readFileSync(descriptor, "utf8");
} catch (error) {
if (String(error?.code || "").startsWith("external_data_plane_managed_provisioner_")) throw error;
throw configError("external_data_plane_managed_provisioner_public_key_file_unreadable");
} finally {
closeSync(descriptor);
}
const normalizedPem = pem.replace(/\r\n/g, "\n").trim();
if (!/^-----BEGIN PUBLIC KEY-----\n[A-Za-z0-9+/=\n]+\n-----END PUBLIC KEY-----$/.test(normalizedPem)) {
throw configError("external_data_plane_managed_provisioner_public_key_file_format_invalid");
}
let publicKey;
try {
publicKey = createPublicKey(normalizedPem);
} catch {
throw configError("external_data_plane_managed_provisioner_public_key_file_format_invalid");
}
if (publicKey.type !== "public" || publicKey.asymmetricKeyType !== "ed25519") {
throw configError("external_data_plane_managed_provisioner_public_key_algorithm_invalid");
}
return publicKey;
}
export function validateManagedProvisionerIdentity(value, name) {
const normalized = String(value ?? "").trim();
if (!IDENTITY.test(normalized)) throw configError(`external_data_plane_managed_provisioner_${name}_invalid`);
return normalized;
}
export function validateManagedProvisionerAudience(value) {
const normalized = String(value ?? "").trim();
if (!AUDIENCE.test(normalized)) throw configError("external_data_plane_managed_provisioner_audience_invalid");
return normalized;
}
export function managedProvisionerSigningPayload({
audience,
serviceId,
keyId,
method,
path,
timestamp,
nonce,
bodySha256,
}) {
return JSON.stringify({
schemaVersion: MANAGED_PROVISIONER_SIGNATURE_SCHEMA,
audience,
serviceId,
keyId,
method,
path,
timestamp,
nonce,
bodySha256,
});
}
export function sha256RawBody(rawBody) {
if (!Buffer.isBuffer(rawBody)) throw new TypeError("raw_body_buffer_required");
return createHash("sha256").update(rawBody).digest("hex");
}
export class ManagedProvisionerReplayCache {
constructor({ maxEntries }) {
if (!Number.isInteger(maxEntries) || maxEntries < 1) throw new TypeError("replay_cache_max_entries_invalid");
this.maxEntries = maxEntries;
this.entries = new Map();
}
consume(key, expiresAt, now = Date.now()) {
this.prune(now);
const existing = this.entries.get(key);
if (existing !== undefined && existing > now) {
throw authError(409, "managed_provisioner_request_replayed");
}
if (this.entries.size >= this.maxEntries) {
throw authError(503, "managed_provisioner_replay_cache_exhausted");
}
this.entries.set(key, expiresAt);
}
prune(now = Date.now()) {
for (const [key, expiresAt] of this.entries) {
if (expiresAt <= now) this.entries.delete(key);
}
}
}
export function createManagedProvisionerRequestVerifier({
publicKey,
serviceId,
keyId,
audience,
maxSkewMs,
replayCache,
now = Date.now,
}) {
if (publicKey?.type !== "public" || publicKey?.asymmetricKeyType !== "ed25519") {
throw configError("external_data_plane_managed_provisioner_public_key_algorithm_invalid");
}
if (!IDENTITY.test(serviceId)) throw configError("external_data_plane_managed_provisioner_service_id_invalid");
if (!IDENTITY.test(keyId)) throw configError("external_data_plane_managed_provisioner_key_id_invalid");
if (!AUDIENCE.test(audience)) throw configError("external_data_plane_managed_provisioner_audience_invalid");
if (!Number.isInteger(maxSkewMs) || maxSkewMs < 1) {
throw configError("external_data_plane_managed_provisioner_max_skew_invalid");
}
if (!(replayCache instanceof ManagedProvisionerReplayCache)) {
throw configError("external_data_plane_managed_provisioner_replay_cache_invalid");
}
return function verifyManagedProvisionerRequest(req) {
const requestServiceId = exactHeader(req, MANAGED_PROVISIONER_HEADERS.serviceId);
const requestKeyId = exactHeader(req, MANAGED_PROVISIONER_HEADERS.keyId);
const requestAudience = exactHeader(req, MANAGED_PROVISIONER_HEADERS.audience);
const timestamp = exactHeader(req, MANAGED_PROVISIONER_HEADERS.timestamp);
const nonce = exactHeader(req, MANAGED_PROVISIONER_HEADERS.nonce);
const declaredBodySha256 = exactHeader(req, MANAGED_PROVISIONER_HEADERS.bodySha256);
const encodedSignature = exactHeader(req, MANAGED_PROVISIONER_HEADERS.signature);
if (hasHeader(req, "authorization")) {
throw authError(401, "managed_provisioner_authorization_header_forbidden");
}
if (!safeEqualText(requestServiceId, serviceId)
|| !safeEqualText(requestKeyId, keyId)
|| !safeEqualText(requestAudience, audience)) {
throw authError(401, "managed_provisioner_identity_mismatch");
}
const timestampMs = parseTimestamp(timestamp);
const verificationTime = Number(now());
if (!Number.isFinite(verificationTime) || Math.abs(verificationTime - timestampMs) >= maxSkewMs) {
throw authError(401, "managed_provisioner_timestamp_outside_window");
}
const nonceBytes = decodeCanonicalBase64Url(nonce, null, "managed_provisioner_nonce_invalid");
if (nonceBytes.length < 16 || nonceBytes.length > 96) throw authError(401, "managed_provisioner_nonce_invalid");
if (!/^[a-f0-9]{64}$/.test(declaredBodySha256)) {
throw authError(401, "managed_provisioner_body_hash_invalid");
}
const method = exactMethod(req);
const path = exactPath(req);
const rawBody = exactRawBody(req);
const actualBodySha256 = sha256RawBody(rawBody);
if (!safeEqualText(declaredBodySha256, actualBodySha256)) {
throw authError(401, "managed_provisioner_body_hash_mismatch");
}
const signature = decodeCanonicalBase64Url(encodedSignature, 64, "managed_provisioner_signature_invalid");
const payload = managedProvisionerSigningPayload({
audience: requestAudience,
serviceId: requestServiceId,
keyId: requestKeyId,
method,
path,
timestamp,
nonce,
bodySha256: declaredBodySha256,
});
if (!verifySignature(null, Buffer.from(payload, "utf8"), publicKey, signature)) {
throw authError(401, "managed_provisioner_signature_invalid");
}
const replayKey = `${requestServiceId}\0${requestKeyId}\0${nonce}`;
replayCache.consume(replayKey, timestampMs + maxSkewMs, verificationTime);
return Object.freeze({ serviceId: requestServiceId, keyId: requestKeyId });
};
}
function exactHeader(req, name) {
const rawHeaders = req?.rawHeaders;
if (Array.isArray(rawHeaders)) {
const values = [];
for (let index = 0; index < rawHeaders.length; index += 2) {
if (String(rawHeaders[index]).toLowerCase() === name) values.push(rawHeaders[index + 1]);
}
if (values.length !== 1 || typeof values[0] !== "string" || !values[0]) {
throw authError(401, "managed_provisioner_header_invalid");
}
return values[0];
}
const value = req?.headers?.[name];
if (typeof value !== "string" || !value) throw authError(401, "managed_provisioner_header_invalid");
return value;
}
function hasHeader(req, name) {
if (Array.isArray(req?.rawHeaders)) {
for (let index = 0; index < req.rawHeaders.length; index += 2) {
if (String(req.rawHeaders[index]).toLowerCase() === name) return true;
}
return false;
}
return Object.hasOwn(req?.headers || {}, name);
}
function exactMethod(req) {
const method = String(req?.method ?? "");
if (!/^[A-Z]{3,10}$/.test(method)) throw authError(401, "managed_provisioner_method_invalid");
return method;
}
function exactPath(req) {
const path = String(req?.originalUrl ?? "");
if (!path.startsWith("/") || path.length > 2048 || /[\u0000-\u0020#]/.test(path)) {
throw authError(401, "managed_provisioner_path_invalid");
}
return path;
}
function exactRawBody(req) {
if (req?.rawBodyCaptured === true && Buffer.isBuffer(req.rawBody)) return req.rawBody;
const contentLength = req?.headers?.["content-length"];
const transferEncoding = req?.headers?.["transfer-encoding"];
if ((contentLength === undefined || contentLength === "0") && transferEncoding === undefined) {
return Buffer.alloc(0);
}
throw authError(401, "managed_provisioner_raw_body_unavailable");
}
function parseTimestamp(value) {
if (!ISO_TIMESTAMP.test(value)) throw authError(401, "managed_provisioner_timestamp_invalid");
const timestampMs = Date.parse(value);
if (!Number.isFinite(timestampMs) || new Date(timestampMs).toISOString() !== value) {
throw authError(401, "managed_provisioner_timestamp_invalid");
}
return timestampMs;
}
function decodeCanonicalBase64Url(value, expectedBytes, errorCode) {
if (!BASE64URL.test(value)) throw authError(401, errorCode);
let decoded;
try {
decoded = Buffer.from(value, "base64url");
} catch {
throw authError(401, errorCode);
}
if ((expectedBytes !== null && decoded.length !== expectedBytes) || decoded.toString("base64url") !== value) {
throw authError(401, errorCode);
}
return decoded;
}
function safeEqualText(left, right) {
const leftBuffer = Buffer.from(String(left), "utf8");
const rightBuffer = Buffer.from(String(right), "utf8");
return leftBuffer.length === rightBuffer.length && timingSafeEqual(leftBuffer, rightBuffer);
}
function authError(status, code) {
return Object.assign(new Error(code), { status, code });
}
function configError(code) {
return Object.assign(new Error(code), { code });
}

View File

@ -65,6 +65,9 @@ export async function migrate(pool) {
create table if not exists external_data_plane_writer_bindings ( create table if not exists external_data_plane_writer_bindings (
id uuid primary key, id uuid primary key,
token_hash text not null unique, token_hash text not null unique,
binding_key text,
request_hash text,
generation integer not null default 1,
tenant_id text not null, tenant_id text not null,
connection_id text not null, connection_id text not null,
provider_id text not null, provider_id text not null,
@ -74,6 +77,7 @@ export async function migrate(pool) {
created_at timestamptz not null default now(), created_at timestamptz not null default now(),
rotated_at timestamptz, rotated_at timestamptz,
revoked_at timestamptz, revoked_at timestamptz,
check (generation > 0),
check ( check (
case when jsonb_typeof(allowed_data_product_ids) = 'array' case when jsonb_typeof(allowed_data_product_ids) = 'array'
then jsonb_array_length(allowed_data_product_ids) > 0 then jsonb_array_length(allowed_data_product_ids) > 0
@ -82,6 +86,37 @@ export async function migrate(pool) {
) )
) )
`); `);
await pool.query("alter table external_data_plane_writer_bindings add column if not exists binding_key text");
await pool.query("alter table external_data_plane_writer_bindings add column if not exists request_hash text");
await pool.query("alter table external_data_plane_writer_bindings add column if not exists generation integer not null default 1");
await pool.query(`
do $$
begin
if not exists (
select 1 from pg_constraint
where conrelid = 'external_data_plane_writer_bindings'::regclass
and conname = 'external_data_plane_writer_bindings_generation_positive_ck'
) then
alter table external_data_plane_writer_bindings
add constraint external_data_plane_writer_bindings_generation_positive_ck
check (generation > 0);
end if;
if not exists (
select 1 from pg_constraint
where conrelid = 'external_data_plane_writer_bindings'::regclass
and conname = 'external_data_plane_writer_bindings_managed_metadata_ck'
) then
alter table external_data_plane_writer_bindings
add constraint external_data_plane_writer_bindings_managed_metadata_ck
check (
(binding_key is null and request_hash is null)
or (binding_key is not null and request_hash ~ '^[a-f0-9]{64}$')
);
end if;
end
$$
`);
await pool.query("create unique index if not exists external_data_plane_writer_bindings_managed_key_idx on external_data_plane_writer_bindings (binding_key, generation) where binding_key is not null");
await pool.query("create index if not exists external_data_plane_writer_bindings_active_idx on external_data_plane_writer_bindings (active, expires_at)"); await pool.query("create index if not exists external_data_plane_writer_bindings_active_idx on external_data_plane_writer_bindings (active, expires_at)");
await pool.query(` await pool.query(`

View File

@ -2,7 +2,7 @@ import express from "express";
import { createHash, randomUUID, timingSafeEqual } from "node:crypto"; import { createHash, randomUUID, timingSafeEqual } from "node:crypto";
import { createServer } from "node:http"; import { createServer } from "node:http";
import { Pool } from "pg"; import { Pool } from "pg";
import { validateDataProductPublish, validateIntakeBatch } from "@nodedc/external-provider-contract"; import { validateDataProductPublish, validateIntakeBatch } from "@nodedc/external-provider-contract/data-plane";
import { readConfig } from "./config.mjs"; import { readConfig } from "./config.mjs";
import { import {
loadDataProductDefinition, loadDataProductDefinition,
@ -18,6 +18,10 @@ import {
import { normalizeDataProductDefinition, safeDataProductDefinition } from "./data-product-policy.mjs"; import { normalizeDataProductDefinition, safeDataProductDefinition } from "./data-product-policy.mjs";
import { reconcileDataProductDefinitions } from "./definitions.mjs"; import { reconcileDataProductDefinitions } from "./definitions.mjs";
import { assertBatchTimeBounds, rawRetentionExpiry } from "./intake-policy.mjs"; import { assertBatchTimeBounds, rawRetentionExpiry } from "./intake-policy.mjs";
import {
createManagedProvisionerRequestVerifier,
ManagedProvisionerReplayCache,
} from "./managed-provisioner-auth.mjs";
import { import {
assertReaderProduct, assertReaderProduct,
createReaderToken, createReaderToken,
@ -31,11 +35,26 @@ import {
hashWriterToken, hashWriterToken,
materializeDataProductPublish, materializeDataProductPublish,
materializeWriterBoundBatch, materializeWriterBoundBatch,
normalizeManagedWriterBindingRequest,
normalizeWriterBindingRequest, normalizeWriterBindingRequest,
safeWriterBinding, safeWriterBinding,
writerBindingRequestHash,
} from "./writer-binding.mjs"; } from "./writer-binding.mjs";
const config = readConfig(); const config = readConfig();
const managedProvisionerReplayCache = config.managedProvisionerApiEnabled
? new ManagedProvisionerReplayCache({ maxEntries: config.managedProvisionerReplayCacheMaxEntries })
: null;
const verifyManagedProvisionerRequest = config.managedProvisionerApiEnabled
? createManagedProvisionerRequestVerifier({
publicKey: config.managedProvisionerPublicKey,
serviceId: config.managedProvisionerServiceId,
keyId: config.managedProvisionerKeyId,
audience: config.managedProvisionerAudience,
maxSkewMs: config.managedProvisionerMaxSkewMs,
replayCache: managedProvisionerReplayCache,
})
: null;
const pool = new Pool({ connectionString: config.databaseUrl, max: config.databasePoolSize }); const pool = new Pool({ connectionString: config.databaseUrl, max: config.databasePoolSize });
const app = express(); const app = express();
const httpServer = createServer(app); const httpServer = createServer(app);
@ -47,7 +66,19 @@ let shuttingDown = false;
let shutdownPromise = null; let shutdownPromise = null;
app.disable("x-powered-by"); app.disable("x-powered-by");
app.use(express.json({ limit: config.maxBatchBytes })); app.use((req, _res, next) => {
req.rawBody = Buffer.alloc(0);
req.rawBodyCaptured = false;
next();
});
app.use(express.json({
limit: config.maxBatchBytes,
inflate: false,
verify(req, _res, buffer) {
req.rawBody = Buffer.from(buffer);
req.rawBodyCaptured = true;
},
}));
app.get("/healthz", asyncRoute(async (_req, res) => { app.get("/healthz", asyncRoute(async (_req, res) => {
await pool.query("select 1"); await pool.query("select 1");
@ -59,10 +90,12 @@ app.get("/healthz", asyncRoute(async (_req, res) => {
providerLogic: "absent", providerLogic: "absent",
commandTransport: "absent", commandTransport: "absent",
writerBindings: "supported", writerBindings: "supported",
managedWriterBindings: "digest+idempotent-generation",
readerBindings: "supported", readerBindings: "supported",
dataProductDelivery: "snapshot+durable-patch", dataProductDelivery: "snapshot+durable-patch",
legacyIntake: config.legacyIntakeEnabled ? "migration-only" : "disabled", legacyIntake: config.legacyIntakeEnabled ? "migration-only" : "disabled",
writerBindingProvisioning: config.provisionerApiEnabled ? "enabled" : "disabled", legacyCapabilityProvisioning: config.provisionerApiEnabled ? "enabled" : "disabled",
managedWriterBindingProvisioning: config.managedProvisionerApiEnabled ? "enabled" : "disabled",
rawRetentionSweep: { rawRetentionSweep: {
mode: "server-scheduled", mode: "server-scheduled",
lastSweepAt: lastRetentionSweepAt, lastSweepAt: lastRetentionSweepAt,
@ -134,6 +167,128 @@ app.post("/internal/data-plane/v1/intake/writer-bound", requireLegacyIntake, req
res.status(result.idempotent ? 200 : 201).json({ ok: true, ...result }); res.status(result.idempotent ? 200 : 201).json({ ok: true, ...result });
})); }));
app.put("/internal/data-plane/v1/writer-bindings/by-key/:bindingKey", requireManagedProvisionerApi, asyncRoute(async (req, res) => {
const bindingKey = requireIdentifier(req.params.bindingKey, "managed_writer_binding_key_invalid");
const policy = normalizeManagedWriterBindingRequest(req.body, {
maxTtlDays: config.writerBindingMaxTtlDays,
});
await assertRegisteredProductIds(policy.allowedDataProductIds);
const requestHash = writerBindingRequestHash(policy);
const client = await pool.connect();
let response;
try {
await client.query("begin");
await client.query("select pg_advisory_xact_lock(hashtextextended($1, 0))", [bindingKey]);
const existing = await client.query(
`select id, binding_key as "bindingKey", request_hash as "requestHash", generation,
tenant_id as "tenantId", connection_id as "connectionId", provider_id as "providerId",
allowed_data_product_ids as "allowedDataProductIds", expires_at as "expiresAt",
active, created_at as "createdAt", rotated_at as "rotatedAt", revoked_at as "revokedAt"
from external_data_plane_writer_bindings
where binding_key = $1 and generation = $2
for update`,
[bindingKey, policy.generation],
);
if (existing.rowCount) {
const binding = existing.rows[0];
if (binding.requestHash !== requestHash) {
throw httpError(409, "managed_writer_binding_request_conflict");
}
if (binding.active !== true || new Date(binding.expiresAt) <= new Date()) {
throw httpError(409, "managed_writer_binding_generation_inactive");
}
response = { status: 200, idempotent: true, binding };
} else {
const inserted = await client.query(
`insert into external_data_plane_writer_bindings (
id, token_hash, binding_key, request_hash, generation,
tenant_id, connection_id, provider_id, allowed_data_product_ids, expires_at
) values ($1, $2, $3, $4, $5, $6, $7, $8, $9::jsonb, $10)
returning id, binding_key as "bindingKey", generation,
tenant_id as "tenantId", connection_id as "connectionId", provider_id as "providerId",
allowed_data_product_ids as "allowedDataProductIds", expires_at as "expiresAt",
active, created_at as "createdAt", rotated_at as "rotatedAt", revoked_at as "revokedAt"`,
[
randomUUID(),
policy.capabilityDigest,
bindingKey,
requestHash,
policy.generation,
policy.tenantId,
policy.connectionId,
policy.providerId,
JSON.stringify(policy.allowedDataProductIds),
policy.expiresAt,
],
);
response = { status: 201, idempotent: false, binding: inserted.rows[0] };
}
await client.query("commit");
} catch (error) {
await client.query("rollback");
if (error?.code === "23505") {
throw httpError(409, "managed_writer_binding_capability_digest_conflict");
}
throw error;
} finally {
client.release();
}
res.set("Cache-Control", "no-store, max-age=0");
res.status(response.status).json({
ok: true,
idempotent: response.idempotent,
writerBinding: safeWriterBinding(response.binding),
});
}));
app.post("/internal/data-plane/v1/writer-bindings/by-key/:bindingKey/generations/:generation/revoke", requireManagedProvisionerApi, asyncRoute(async (req, res) => {
const bindingKey = requireIdentifier(req.params.bindingKey, "managed_writer_binding_key_invalid");
const generation = requirePositiveInteger(req.params.generation, "managed_writer_binding_generation_invalid");
const client = await pool.connect();
let binding;
let idempotent;
try {
await client.query("begin");
await client.query("select pg_advisory_xact_lock(hashtextextended($1, 0))", [bindingKey]);
const existing = await client.query(
`select id, binding_key as "bindingKey", generation,
tenant_id as "tenantId", connection_id as "connectionId", provider_id as "providerId",
allowed_data_product_ids as "allowedDataProductIds", expires_at as "expiresAt",
active, created_at as "createdAt", rotated_at as "rotatedAt", revoked_at as "revokedAt"
from external_data_plane_writer_bindings
where binding_key = $1 and generation = $2
for update`,
[bindingKey, generation],
);
if (!existing.rowCount) throw httpError(404, "managed_writer_binding_not_found");
if (existing.rows[0].active === true) {
const revoked = await client.query(
`update external_data_plane_writer_bindings
set active = false, revoked_at = now()
where binding_key = $1 and generation = $2 and active = true
returning id, binding_key as "bindingKey", generation,
tenant_id as "tenantId", connection_id as "connectionId", provider_id as "providerId",
allowed_data_product_ids as "allowedDataProductIds", expires_at as "expiresAt",
active, created_at as "createdAt", rotated_at as "rotatedAt", revoked_at as "revokedAt"`,
[bindingKey, generation],
);
binding = revoked.rows[0];
idempotent = false;
} else {
binding = existing.rows[0];
idempotent = true;
}
await client.query("commit");
} catch (error) {
await client.query("rollback");
throw error;
} finally {
client.release();
}
res.set("Cache-Control", "no-store, max-age=0");
res.json({ ok: true, idempotent, writerBinding: safeWriterBinding(binding) });
}));
app.post("/internal/data-plane/v1/writer-bindings", requireProvisionerApi, asyncRoute(async (req, res) => { app.post("/internal/data-plane/v1/writer-bindings", requireProvisionerApi, asyncRoute(async (req, res) => {
const policy = normalizeWriterBindingRequest(req.body, { const policy = normalizeWriterBindingRequest(req.body, {
maxTtlDays: config.writerBindingMaxTtlDays, maxTtlDays: config.writerBindingMaxTtlDays,
@ -172,7 +327,7 @@ app.post("/internal/data-plane/v1/writer-bindings/:bindingId/rotate", requirePro
const result = await pool.query( const result = await pool.query(
`update external_data_plane_writer_bindings `update external_data_plane_writer_bindings
set token_hash = $2, rotated_at = now() set token_hash = $2, rotated_at = now()
where id = $1 and active = true and expires_at > now() where id = $1 and binding_key is null and active = true and expires_at > now()
returning id, tenant_id as "tenantId", connection_id as "connectionId", returning id, tenant_id as "tenantId", connection_id as "connectionId",
provider_id as "providerId", allowed_data_product_ids as "allowedDataProductIds", provider_id as "providerId", allowed_data_product_ids as "allowedDataProductIds",
expires_at as "expiresAt", active, created_at as "createdAt", expires_at as "expiresAt", active, created_at as "createdAt",
@ -188,7 +343,7 @@ app.post("/internal/data-plane/v1/writer-bindings/:bindingId/revoke", requirePro
const result = await pool.query( const result = await pool.query(
`update external_data_plane_writer_bindings `update external_data_plane_writer_bindings
set active = false, revoked_at = now() set active = false, revoked_at = now()
where id = $1 and active = true where id = $1 and binding_key is null and active = true
returning id, tenant_id as "tenantId", connection_id as "connectionId", returning id, tenant_id as "tenantId", connection_id as "connectionId",
provider_id as "providerId", allowed_data_product_ids as "allowedDataProductIds", provider_id as "providerId", allowed_data_product_ids as "allowedDataProductIds",
expires_at as "expiresAt", active, created_at as "createdAt", expires_at as "expiresAt", active, created_at as "createdAt",
@ -564,6 +719,21 @@ function requireLegacyIntake(_req, _res, next) {
function requireProvisionerApi(req, _res, next) { function requireProvisionerApi(req, _res, next) {
if (!config.provisionerApiEnabled) return next(httpError(503, "provisioner_api_disabled")); if (!config.provisionerApiEnabled) return next(httpError(503, "provisioner_api_disabled"));
return requireProvisionerCredential(req, next);
}
function requireManagedProvisionerApi(req, _res, next) {
if (!config.managedProvisionerApiEnabled) return next(httpError(503, "managed_provisioner_api_disabled"));
if (!verifyManagedProvisionerRequest) return next(httpError(503, "managed_provisioner_api_not_configured"));
try {
req.managedProvisionerIdentity = verifyManagedProvisionerRequest(req);
return next();
} catch (error) {
return next(error);
}
}
function requireProvisionerCredential(req, next) {
if (!config.provisionerAccessToken) return next(httpError(503, "provisioner_api_not_configured")); if (!config.provisionerAccessToken) return next(httpError(503, "provisioner_api_not_configured"));
const value = bearerToken(req); const value = bearerToken(req);
if (!value || !safeEqual(value, config.provisionerAccessToken)) return next(httpError(401, "provisioner_unauthorized")); if (!value || !safeEqual(value, config.provisionerAccessToken)) return next(httpError(401, "provisioner_unauthorized"));
@ -697,6 +867,14 @@ function requireIdentifier(value, code) {
return normalized; return normalized;
} }
function requirePositiveInteger(value, code) {
const normalized = String(value ?? "");
if (!/^[1-9]\d*$/.test(normalized)) throw httpError(400, code);
const number = Number(normalized);
if (!Number.isSafeInteger(number) || number > 2_147_483_647) throw httpError(400, code);
return number;
}
function parseCursor(value) { function parseCursor(value) {
const normalized = String(value ?? ""); const normalized = String(value ?? "");
if (!/^(?:0|[1-9]\d*)$/.test(normalized)) throw httpError(400, "data_product_cursor_invalid"); if (!/^(?:0|[1-9]\d*)$/.test(normalized)) throw httpError(400, "data_product_cursor_invalid");

View File

@ -4,7 +4,15 @@ const IDENTIFIER = /^[a-z][a-z0-9._:-]{2,127}$/;
const TOKEN_PREFIX = "ndc_edpwb_"; const TOKEN_PREFIX = "ndc_edpwb_";
const SECRET_LIKE_KEY = /(token|secret|password|access[_-]?token|refresh[_-]?token|api[_-]?key)/i; const SECRET_LIKE_KEY = /(token|secret|password|access[_-]?token|refresh[_-]?token|api[_-]?key)/i;
const BINDING_REQUEST_KEYS = new Set(["source", "allowedDataProductIds", "expiresAt"]); const BINDING_REQUEST_KEYS = new Set(["source", "allowedDataProductIds", "expiresAt"]);
const MANAGED_BINDING_REQUEST_KEYS = new Set([
"source",
"allowedDataProductIds",
"expiresAt",
"generation",
"capabilityDigest",
]);
const BINDING_SOURCE_KEYS = new Set(["tenantId", "connectionId", "providerId"]); const BINDING_SOURCE_KEYS = new Set(["tenantId", "connectionId", "providerId"]);
const SHA256_DIGEST = /^[a-f0-9]{64}$/;
/** /**
* Produces an opaque, high-entropy capability. The plaintext is returned only * Produces an opaque, high-entropy capability. The plaintext is returned only
@ -59,6 +67,57 @@ export function normalizeWriterBindingRequest(value, { now = new Date(), maxTtlD
}); });
} }
/**
* Validates the zero-touch control-plane form. The native credential secret is
* generated and stored inside Engine; EDP receives only its SHA-256 digest.
* `generation` makes a retry address the same immutable binding generation.
*/
export function normalizeManagedWriterBindingRequest(value, { now = new Date(), maxTtlDays = 90 } = {}) {
if (!isPlainObject(value) || !isPlainObject(value.source)) {
throw writerBindingError("managed_writer_binding_request_invalid");
}
if (containsSecretLikeKey(value)) {
throw writerBindingError("managed_writer_binding_secret_material_forbidden");
}
if (!hasOnlyKeys(value, MANAGED_BINDING_REQUEST_KEYS) || !hasOnlyKeys(value.source, BINDING_SOURCE_KEYS)) {
throw writerBindingError("managed_writer_binding_request_fields_invalid");
}
const scope = normalizeWriterBindingRequest({
source: value.source,
allowedDataProductIds: value.allowedDataProductIds,
expiresAt: value.expiresAt,
}, { now, maxTtlDays });
const generation = Number(value.generation);
if (!Number.isSafeInteger(generation) || generation < 1 || generation > 2_147_483_647) {
throw writerBindingError("managed_writer_binding_generation_invalid");
}
const capabilityDigest = String(value.capabilityDigest || "").toLowerCase();
if (!SHA256_DIGEST.test(capabilityDigest)) {
throw writerBindingError("managed_writer_binding_capability_digest_invalid");
}
return Object.freeze({
...scope,
allowedDataProductIds: Object.freeze([...scope.allowedDataProductIds].sort()),
generation,
capabilityDigest,
});
}
export function writerBindingRequestHash(policy) {
const canonical = JSON.stringify({
tenantId: policy.tenantId,
connectionId: policy.connectionId,
providerId: policy.providerId,
allowedDataProductIds: [...policy.allowedDataProductIds].sort(),
expiresAt: new Date(policy.expiresAt).toISOString(),
generation: policy.generation,
capabilityDigest: policy.capabilityDigest,
});
return createHash("sha256").update(canonical, "utf8").digest("hex");
}
/** /**
* Converts a caller-provided, deliberately unscoped intake envelope into the * Converts a caller-provided, deliberately unscoped intake envelope into the
* canonical scoped form. Caller scope is rejected, never trusted or merged. * canonical scoped form. Caller scope is rejected, never trusted or merged.
@ -136,6 +195,8 @@ export function materializeDataProductPublish(value, binding, definition, dataPr
export function safeWriterBinding(binding) { export function safeWriterBinding(binding) {
return { return {
id: binding.id, id: binding.id,
...(binding.bindingKey ? { bindingKey: binding.bindingKey } : {}),
...(Number.isInteger(Number(binding.generation)) ? { generation: Number(binding.generation) } : {}),
tenantId: binding.tenantId, tenantId: binding.tenantId,
connectionId: binding.connectionId, connectionId: binding.connectionId,
providerId: binding.providerId, providerId: binding.providerId,

View File

@ -1,10 +1,16 @@
import assert from "node:assert/strict"; import assert from "node:assert/strict";
import { spawn } from "node:child_process"; import { spawn } from "node:child_process";
import { createHash, generateKeyPairSync, randomBytes, sign } from "node:crypto";
import { mkdtemp, rm, writeFile } from "node:fs/promises"; import { mkdtemp, rm, writeFile } from "node:fs/promises";
import net from "node:net"; import net from "node:net";
import { tmpdir } from "node:os"; import { tmpdir } from "node:os";
import { join } from "node:path"; import { join } from "node:path";
import { validateDataProductPatch, validateDataProductSnapshot } from "@nodedc/external-provider-contract"; import { validateDataProductPatch, validateDataProductSnapshot } from "@nodedc/external-provider-contract/data-plane";
import {
MANAGED_PROVISIONER_HEADERS,
managedProvisionerSigningPayload,
sha256RawBody,
} from "../src/managed-provisioner-auth.mjs";
const databaseUrl = process.env.EXTERNAL_DATA_PLANE_TEST_DATABASE_URL; const databaseUrl = process.env.EXTERNAL_DATA_PLANE_TEST_DATABASE_URL;
if (!databaseUrl) throw new Error("EXTERNAL_DATA_PLANE_TEST_DATABASE_URL_required"); if (!databaseUrl) throw new Error("EXTERNAL_DATA_PLANE_TEST_DATABASE_URL_required");
@ -13,7 +19,17 @@ const baseUrl = `http://127.0.0.1:${port}`;
const directory = await mkdtemp(join(tmpdir(), "nodedc-edp-api-test-")); const directory = await mkdtemp(join(tmpdir(), "nodedc-edp-api-test-"));
const provisionerSecret = "nodedc_edp_integration_provisioner_secret_0123456789abcdef"; const provisionerSecret = "nodedc_edp_integration_provisioner_secret_0123456789abcdef";
const secretPath = join(directory, "provisioner-token"); const secretPath = join(directory, "provisioner-token");
const managedProvisionerServiceId = "nodedc-engine";
const managedProvisionerKeyId = "engine-edp-managed-provisioner-v1";
const managedProvisionerAudience = "nodedc-external-data-plane.managed-provisioning.v1";
const managedProvisionerPublicKeyPath = join(directory, "engine-public-key.pem");
const { privateKey: managedProvisionerPrivateKey, publicKey: managedProvisionerPublicKey } = generateKeyPairSync("ed25519");
await writeFile(secretPath, `${provisionerSecret}\n`, { mode: 0o600 }); await writeFile(secretPath, `${provisionerSecret}\n`, { mode: 0o600 });
await writeFile(
managedProvisionerPublicKeyPath,
managedProvisionerPublicKey.export({ type: "spki", format: "pem" }),
{ mode: 0o600 },
);
const child = spawn(process.execPath, ["src/server.mjs"], { const child = spawn(process.execPath, ["src/server.mjs"], {
cwd: new URL("..", import.meta.url), cwd: new URL("..", import.meta.url),
@ -22,7 +38,12 @@ const child = spawn(process.execPath, ["src/server.mjs"], {
PORT: String(port), PORT: String(port),
EXTERNAL_DATA_PLANE_DATABASE_URL: databaseUrl, EXTERNAL_DATA_PLANE_DATABASE_URL: databaseUrl,
EXTERNAL_DATA_PLANE_PROVISIONING_ENABLED: "true", EXTERNAL_DATA_PLANE_PROVISIONING_ENABLED: "true",
EXTERNAL_DATA_PLANE_MANAGED_PROVISIONING_ENABLED: "true",
EXTERNAL_DATA_PLANE_PROVISIONER_TOKEN_FILE: secretPath, EXTERNAL_DATA_PLANE_PROVISIONER_TOKEN_FILE: secretPath,
EXTERNAL_DATA_PLANE_MANAGED_PROVISIONER_PUBLIC_KEY_FILE: managedProvisionerPublicKeyPath,
EXTERNAL_DATA_PLANE_MANAGED_PROVISIONER_SERVICE_ID: managedProvisionerServiceId,
EXTERNAL_DATA_PLANE_MANAGED_PROVISIONER_KEY_ID: managedProvisionerKeyId,
EXTERNAL_DATA_PLANE_MANAGED_PROVISIONER_AUDIENCE: managedProvisionerAudience,
EXTERNAL_DATA_PLANE_RETENTION_SWEEP_MS: "60000", EXTERNAL_DATA_PLANE_RETENTION_SWEEP_MS: "60000",
EXTERNAL_DATA_PLANE_STREAM_POLL_MS: "250", EXTERNAL_DATA_PLANE_STREAM_POLL_MS: "250",
EXTERNAL_DATA_PLANE_STREAM_HEARTBEAT_MS: "5000", EXTERNAL_DATA_PLANE_STREAM_HEARTBEAT_MS: "5000",
@ -52,10 +73,139 @@ try {
const expiry = new Date(Date.now() + 24 * 60 * 60 * 1000).toISOString(); const expiry = new Date(Date.now() + 24 * 60 * 60 * 1000).toISOString();
const scope = { tenantId: "api-tenant", connectionId: "api-connection", providerId: "api-provider" }; const scope = { tenantId: "api-tenant", connectionId: "api-connection", providerId: "api-provider" };
const managedToken = "ndc_edpwb_managed_api_test_0123456789abcdefghijklmnopqrstuvwxyz";
const managedBody = {
source: scope,
allowedDataProductIds: [productId],
expiresAt: expiry,
generation: 1,
capabilityDigest: createHash("sha256").update(managedToken, "utf8").digest("hex"),
};
const managedPath = "/internal/data-plane/v1/writer-bindings/by-key/engine.api-connection.positions";
const bearerOnlyManaged = await fetch(`${baseUrl}${managedPath}`, {
method: "PUT",
headers: { Authorization: `Bearer ${provisionerSecret}`, "Content-Type": "application/json" },
body: JSON.stringify(managedBody),
});
assert.equal(bearerOnlyManaged.status, 401);
assert.equal((await bearerOnlyManaged.json()).error, "managed_provisioner_header_invalid");
const managedResults = await Promise.all([
rawJsonRequest(managedPath, {
method: "PUT",
managedSignature: true,
body: managedBody,
}),
rawJsonRequest(managedPath, {
method: "PUT",
managedSignature: true,
body: managedBody,
}),
]);
assert.deepEqual(managedResults.map(({ value }) => value.idempotent).sort(), [false, true]);
assert.equal(managedResults[0].value.writerBinding.id, managedResults[1].value.writerBinding.id);
assert.equal("token" in managedResults[0].value, false);
assert.equal(JSON.stringify(managedResults[0].value).includes(managedBody.capabilityDigest), false);
const managedCatalog = await jsonRequest("/internal/data-plane/v1/writer/data-products", { token: managedToken });
assert.deepEqual(managedCatalog.dataProducts.map((value) => value.id), [productId]);
const replayBody = JSON.stringify(managedBody);
const replayHeaders = signedManagedHeaders(managedPath, "PUT", Buffer.from(replayBody, "utf8"));
const firstReplayAttempt = await fetch(`${baseUrl}${managedPath}`, {
method: "PUT", headers: { ...replayHeaders, "Content-Type": "application/json" }, body: replayBody,
});
assert.equal(firstReplayAttempt.status, 200);
const rejectedReplay = await fetch(`${baseUrl}${managedPath}`, {
method: "PUT", headers: { ...replayHeaders, "Content-Type": "application/json" }, body: replayBody,
});
assert.equal(rejectedReplay.status, 409);
assert.equal((await rejectedReplay.json()).error, "managed_provisioner_request_replayed");
const tamperHeaders = signedManagedHeaders(managedPath, "PUT", Buffer.from(replayBody, "utf8"));
const tamperedBody = await fetch(`${baseUrl}${managedPath}`, {
method: "PUT",
headers: { ...tamperHeaders, "Content-Type": "application/json" },
body: `${replayBody} `,
});
assert.equal(tamperedBody.status, 401);
assert.equal((await tamperedBody.json()).error, "managed_provisioner_body_hash_mismatch");
const pathHeaders = signedManagedHeaders(managedPath, "PUT", Buffer.from(replayBody, "utf8"));
const tamperedPath = await fetch(`${baseUrl}${managedPath}?unexpected=true`, {
method: "PUT", headers: { ...pathHeaders, "Content-Type": "application/json" }, body: replayBody,
});
assert.equal(tamperedPath.status, 401);
assert.equal((await tamperedPath.json()).error, "managed_provisioner_signature_invalid");
const audienceHeaders = signedManagedHeaders(managedPath, "PUT", Buffer.from(replayBody, "utf8"));
audienceHeaders[MANAGED_PROVISIONER_HEADERS.audience] = "other-audience";
const wrongAudience = await fetch(`${baseUrl}${managedPath}`, {
method: "PUT", headers: { ...audienceHeaders, "Content-Type": "application/json" }, body: replayBody,
});
assert.equal(wrongAudience.status, 401);
assert.equal((await wrongAudience.json()).error, "managed_provisioner_identity_mismatch");
const managedConflict = await signedFetch(managedPath, {
method: "PUT",
body: {
...managedBody,
capabilityDigest: createHash("sha256").update(`${managedToken}-different`, "utf8").digest("hex"),
},
});
assert.equal(managedConflict.status, 409);
assert.equal((await managedConflict.json()).error, "managed_writer_binding_request_conflict");
const legacyManagedRevoke = await fetch(
`${baseUrl}/internal/data-plane/v1/writer-bindings/${managedResults[0].value.writerBinding.id}/revoke`,
{ method: "POST", headers: { Authorization: `Bearer ${provisionerSecret}` } },
);
assert.equal(legacyManagedRevoke.status, 404);
const managedTokenGeneration2 = `${managedToken}_generation_2`;
const managedGeneration2 = await jsonRequest(
"/internal/data-plane/v1/writer-bindings/by-key/engine.api-connection.positions",
{
method: "PUT",
managedSignature: true,
body: {
...managedBody,
generation: 2,
capabilityDigest: createHash("sha256").update(managedTokenGeneration2, "utf8").digest("hex"),
},
},
);
assert.equal(managedGeneration2.writerBinding.generation, 2);
assert.deepEqual(
(await jsonRequest("/internal/data-plane/v1/writer/data-products", { token: managedTokenGeneration2 })).dataProducts.map((value) => value.id),
[productId],
);
const managedRevoke = await jsonRequest(
"/internal/data-plane/v1/writer-bindings/by-key/engine.api-connection.positions/generations/1/revoke",
{ method: "POST", managedSignature: true },
);
assert.equal(managedRevoke.idempotent, false);
assert.equal(managedRevoke.writerBinding.active, false);
const managedRevokeRetry = await jsonRequest(
"/internal/data-plane/v1/writer-bindings/by-key/engine.api-connection.positions/generations/1/revoke",
{ method: "POST", managedSignature: true },
);
assert.equal(managedRevokeRetry.idempotent, true);
const rejectedManagedGeneration1 = await fetch(`${baseUrl}/internal/data-plane/v1/writer/data-products`, {
headers: { Authorization: `Bearer ${managedToken}` },
});
assert.equal(rejectedManagedGeneration1.status, 401);
const manualWriterBody = { source: scope, allowedDataProductIds: [productId], expiresAt: expiry };
const signedOnlyManualProvisioning = await signedFetch("/internal/data-plane/v1/writer-bindings", {
method: "POST",
body: manualWriterBody,
});
assert.equal(signedOnlyManualProvisioning.status, 401);
assert.equal((await signedOnlyManualProvisioning.json()).error, "provisioner_unauthorized");
const writerIssuance = await rawJsonRequest("/internal/data-plane/v1/writer-bindings", { const writerIssuance = await rawJsonRequest("/internal/data-plane/v1/writer-bindings", {
method: "POST", method: "POST",
token: provisionerSecret, token: provisionerSecret,
body: { source: scope, allowedDataProductIds: [productId], expiresAt: expiry }, body: manualWriterBody,
}); });
const readerIssuance = await rawJsonRequest("/internal/data-plane/v1/reader-bindings", { const readerIssuance = await rawJsonRequest("/internal/data-plane/v1/reader-bindings", {
method: "POST", method: "POST",
@ -176,26 +326,66 @@ function publish(runId, observedAt, longitude) {
}; };
} }
async function jsonRequest(path, { method = "GET", token, body } = {}) { async function jsonRequest(path, { method = "GET", token, managedSignature = false, body } = {}) {
const { response, value } = await rawJsonRequest(path, { method, token, body }); const { response, value } = await rawJsonRequest(path, { method, token, managedSignature, body });
if (!response.ok) throw new Error(`request_failed:${response.status}:${value.error}`); if (!response.ok) throw new Error(`request_failed:${response.status}:${value.error}`);
return value; return value;
} }
async function rawJsonRequest(path, { method = "GET", token, body } = {}) { async function rawJsonRequest(path, { method = "GET", token, managedSignature = false, body } = {}) {
const rawBody = body === undefined ? Buffer.alloc(0) : Buffer.from(JSON.stringify(body), "utf8");
const response = await fetch(`${baseUrl}${path}`, { const response = await fetch(`${baseUrl}${path}`, {
method, method,
headers: { headers: {
...(token ? { Authorization: `Bearer ${token}` } : {}), ...(token ? { Authorization: `Bearer ${token}` } : {}),
...(body ? { "Content-Type": "application/json" } : {}), ...(managedSignature ? signedManagedHeaders(path, method, rawBody) : {}),
...(body === undefined ? {} : { "Content-Type": "application/json" }),
}, },
body: body ? JSON.stringify(body) : undefined, body: body === undefined ? undefined : rawBody,
}); });
const value = await response.json(); const value = await response.json();
if (!response.ok) throw new Error(`request_failed:${response.status}:${value.error}`); if (!response.ok) throw new Error(`request_failed:${response.status}:${value.error}`);
return { response, value }; return { response, value };
} }
async function signedFetch(path, { method, body } = {}) {
const rawBody = body === undefined ? Buffer.alloc(0) : Buffer.from(JSON.stringify(body), "utf8");
return fetch(`${baseUrl}${path}`, {
method,
headers: {
...signedManagedHeaders(path, method, rawBody),
...(body === undefined ? {} : { "Content-Type": "application/json" }),
},
body: body === undefined ? undefined : rawBody,
});
}
function signedManagedHeaders(path, method, rawBody) {
const timestamp = new Date().toISOString();
const nonce = randomBytes(24).toString("base64url");
const bodySha256 = sha256RawBody(rawBody);
const payload = managedProvisionerSigningPayload({
audience: managedProvisionerAudience,
serviceId: managedProvisionerServiceId,
keyId: managedProvisionerKeyId,
method,
path,
timestamp,
nonce,
bodySha256,
});
const signature = sign(null, Buffer.from(payload, "utf8"), managedProvisionerPrivateKey).toString("base64url");
return {
[MANAGED_PROVISIONER_HEADERS.serviceId]: managedProvisionerServiceId,
[MANAGED_PROVISIONER_HEADERS.keyId]: managedProvisionerKeyId,
[MANAGED_PROVISIONER_HEADERS.audience]: managedProvisionerAudience,
[MANAGED_PROVISIONER_HEADERS.timestamp]: timestamp,
[MANAGED_PROVISIONER_HEADERS.nonce]: nonce,
[MANAGED_PROVISIONER_HEADERS.bodySha256]: bodySha256,
[MANAGED_PROVISIONER_HEADERS.signature]: signature,
};
}
function assertOneTimeCapabilityResponse(response) { function assertOneTimeCapabilityResponse(response) {
assert.equal(response.headers.get("cache-control"), "no-store, max-age=0"); assert.equal(response.headers.get("cache-control"), "no-store, max-age=0");
assert.equal(response.headers.get("pragma"), "no-cache"); assert.equal(response.headers.get("pragma"), "no-cache");

View File

@ -1,4 +1,5 @@
import assert from "node:assert/strict"; import assert from "node:assert/strict";
import { generateKeyPairSync } from "node:crypto";
import { mkdtemp, rm, writeFile } from "node:fs/promises"; import { mkdtemp, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os"; import { tmpdir } from "node:os";
import { join } from "node:path"; import { join } from "node:path";
@ -11,9 +12,12 @@ const base = {
const directory = await mkdtemp(join(tmpdir(), "nodedc-edp-config-")); const directory = await mkdtemp(join(tmpdir(), "nodedc-edp-config-"));
const secretPath = join(directory, "provisioner-token"); const secretPath = join(directory, "provisioner-token");
const publicKeyPath = join(directory, "engine-public-key.pem");
const secret = "provisioner_secret_is_separate_from_legacy_internal_token_123456"; const secret = "provisioner_secret_is_separate_from_legacy_internal_token_123456";
try { try {
await writeFile(secretPath, `${secret}\n`, { encoding: "utf8", mode: 0o600 }); await writeFile(secretPath, `${secret}\n`, { encoding: "utf8", mode: 0o600 });
const { publicKey } = generateKeyPairSync("ed25519");
await writeFile(publicKeyPath, publicKey.export({ type: "spki", format: "pem" }), { mode: 0o600 });
const config = readConfig({ const config = readConfig({
...base, ...base,
EXTERNAL_DATA_PLANE_PROVISIONING_ENABLED: "true", EXTERNAL_DATA_PLANE_PROVISIONING_ENABLED: "true",
@ -22,6 +26,8 @@ try {
EXTERNAL_DATA_PLANE_RETENTION_SWEEP_MS: "60000", EXTERNAL_DATA_PLANE_RETENTION_SWEEP_MS: "60000",
}); });
assert.equal(config.provisionerAccessToken, secret); assert.equal(config.provisionerAccessToken, secret);
assert.equal(config.provisionerApiEnabled, true);
assert.equal(config.managedProvisionerApiEnabled, false);
assert.equal(config.maxFutureSkewSeconds, 120); assert.equal(config.maxFutureSkewSeconds, 120);
assert.equal(config.retentionSweepMs, 60000); assert.equal(config.retentionSweepMs, 60000);
assert.equal(config.maxPatchBytes, 262144); assert.equal(config.maxPatchBytes, 262144);
@ -39,10 +45,49 @@ try {
EXTERNAL_DATA_PLANE_PROVISIONING_ENABLED: "true", EXTERNAL_DATA_PLANE_PROVISIONING_ENABLED: "true",
EXTERNAL_DATA_PLANE_PROVISIONER_TOKEN_FILE: join(directory, "missing"), EXTERNAL_DATA_PLANE_PROVISIONER_TOKEN_FILE: join(directory, "missing"),
}), /external_data_plane_provisioner_token_file_unreadable/); }), /external_data_plane_provisioner_token_file_unreadable/);
const managedConfig = readConfig({
...base,
EXTERNAL_DATA_PLANE_PROVISIONING_ENABLED: "false",
EXTERNAL_DATA_PLANE_MANAGED_PROVISIONING_ENABLED: "true",
EXTERNAL_DATA_PLANE_PROVISIONER_TOKEN_FILE: join(directory, "missing-token-is-ignored"),
EXTERNAL_DATA_PLANE_MANAGED_PROVISIONER_PUBLIC_KEY_FILE: publicKeyPath,
EXTERNAL_DATA_PLANE_MANAGED_PROVISIONER_SERVICE_ID: "nodedc-engine",
EXTERNAL_DATA_PLANE_MANAGED_PROVISIONER_KEY_ID: "engine-edp-managed-provisioner-v1",
EXTERNAL_DATA_PLANE_MANAGED_PROVISIONER_AUDIENCE: "nodedc-external-data-plane.managed-provisioning.v1",
EXTERNAL_DATA_PLANE_MANAGED_PROVISIONER_MAX_SKEW_SECONDS: "45",
EXTERNAL_DATA_PLANE_MANAGED_PROVISIONER_REPLAY_CACHE_MAX_ENTRIES: "500",
});
assert.equal(managedConfig.provisionerApiEnabled, false);
assert.equal(managedConfig.managedProvisionerApiEnabled, true);
assert.equal(managedConfig.provisionerAccessToken, "");
assert.equal(managedConfig.managedProvisionerPublicKey.asymmetricKeyType, "ed25519");
assert.equal(managedConfig.managedProvisionerServiceId, "nodedc-engine");
assert.equal(managedConfig.managedProvisionerKeyId, "engine-edp-managed-provisioner-v1");
assert.equal(managedConfig.managedProvisionerAudience, "nodedc-external-data-plane.managed-provisioning.v1");
assert.equal(managedConfig.managedProvisionerMaxSkewMs, 45_000);
assert.equal(managedConfig.managedProvisionerReplayCacheMaxEntries, 500);
assert.throws(() => readConfig({
...base,
EXTERNAL_DATA_PLANE_MANAGED_PROVISIONING_ENABLED: "true",
EXTERNAL_DATA_PLANE_MANAGED_PROVISIONER_PUBLIC_KEY_FILE: join(directory, "missing"),
EXTERNAL_DATA_PLANE_MANAGED_PROVISIONER_SERVICE_ID: "nodedc-engine",
EXTERNAL_DATA_PLANE_MANAGED_PROVISIONER_KEY_ID: "engine-edp-managed-provisioner-v1",
EXTERNAL_DATA_PLANE_MANAGED_PROVISIONER_AUDIENCE: "nodedc-external-data-plane.managed-provisioning.v1",
}), /external_data_plane_managed_provisioner_public_key_file_unreadable/);
assert.throws(() => readConfig({
...base,
EXTERNAL_DATA_PLANE_MANAGED_PROVISIONING_ENABLED: "true",
EXTERNAL_DATA_PLANE_MANAGED_PROVISIONER_PUBLIC_KEY_FILE: publicKeyPath,
EXTERNAL_DATA_PLANE_MANAGED_PROVISIONER_SERVICE_ID: "invalid service id",
EXTERNAL_DATA_PLANE_MANAGED_PROVISIONER_KEY_ID: "engine-edp-managed-provisioner-v1",
EXTERNAL_DATA_PLANE_MANAGED_PROVISIONER_AUDIENCE: "nodedc-external-data-plane.managed-provisioning.v1",
}), /external_data_plane_managed_provisioner_service_id_invalid/);
assert.equal(readConfig({ assert.equal(readConfig({
...base, ...base,
EXTERNAL_DATA_PLANE_PROVISIONING_ENABLED: "false", EXTERNAL_DATA_PLANE_PROVISIONING_ENABLED: "false",
EXTERNAL_DATA_PLANE_MANAGED_PROVISIONING_ENABLED: "false",
EXTERNAL_DATA_PLANE_PROVISIONER_TOKEN_FILE: join(directory, "missing"), EXTERNAL_DATA_PLANE_PROVISIONER_TOKEN_FILE: join(directory, "missing"),
EXTERNAL_DATA_PLANE_MANAGED_PROVISIONER_PUBLIC_KEY_FILE: join(directory, "missing"),
}).provisionerAccessToken, ""); }).provisionerAccessToken, "");
} finally { } finally {
await rm(directory, { recursive: true, force: true }); await rm(directory, { recursive: true, force: true });

View File

@ -1,6 +1,6 @@
import assert from "node:assert/strict"; import assert from "node:assert/strict";
import { Pool } from "pg"; import { Pool } from "pg";
import { validateDataProductPatch, validateDataProductSnapshot } from "@nodedc/external-provider-contract"; import { validateDataProductPatch, validateDataProductSnapshot } from "@nodedc/external-provider-contract/data-plane";
import { import {
loadDataProductDefinition, loadDataProductDefinition,
persistDataProductDefinition, persistDataProductDefinition,

View File

@ -0,0 +1,217 @@
import assert from "node:assert/strict";
import {
generateKeyPairSync,
randomBytes,
sign,
} from "node:crypto";
import { chmod, mkdtemp, rm, symlink, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import {
createManagedProvisionerRequestVerifier,
loadManagedProvisionerPublicKeyFile,
MANAGED_PROVISIONER_HEADERS,
managedProvisionerSigningPayload,
ManagedProvisionerReplayCache,
sha256RawBody,
} from "../src/managed-provisioner-auth.mjs";
const serviceId = "nodedc-engine";
const keyId = "engine-edp-managed-provisioner-v1";
const audience = "nodedc-external-data-plane.managed-provisioning.v1";
const nowMs = Date.parse("2026-07-16T12:00:00.000Z");
const { privateKey, publicKey } = generateKeyPairSync("ed25519");
let nonceSequence = 0;
const replayCache = new ManagedProvisionerReplayCache({ maxEntries: 100 });
const verify = createManagedProvisionerRequestVerifier({
publicKey,
serviceId,
keyId,
audience,
maxSkewMs: 60_000,
replayCache,
now: () => nowMs,
});
const valid = signedRequest();
assert.deepEqual(verify(valid), { serviceId, keyId });
assert.throws(() => verify(valid), hasCode("managed_provisioner_request_replayed", 409));
const whitespaceTamper = signedRequest();
whitespaceTamper.rawBody = Buffer.from(`${whitespaceTamper.rawBody.toString("utf8")} `, "utf8");
assert.throws(() => verify(whitespaceTamper), hasCode("managed_provisioner_body_hash_mismatch", 401));
const methodTamper = signedRequest();
methodTamper.method = "POST";
assert.throws(() => verify(methodTamper), hasCode("managed_provisioner_signature_invalid", 401));
const pathTamper = signedRequest();
pathTamper.originalUrl = `${pathTamper.originalUrl}?unexpected=true`;
assert.throws(() => verify(pathTamper), hasCode("managed_provisioner_signature_invalid", 401));
const audienceTamper = signedRequest();
setHeader(audienceTamper, MANAGED_PROVISIONER_HEADERS.audience, "other-audience");
assert.throws(() => verify(audienceTamper), hasCode("managed_provisioner_identity_mismatch", 401));
const serviceTamper = signedRequest();
setHeader(serviceTamper, MANAGED_PROVISIONER_HEADERS.serviceId, "other-engine");
assert.throws(() => verify(serviceTamper), hasCode("managed_provisioner_identity_mismatch", 401));
const keyTamper = signedRequest();
setHeader(keyTamper, MANAGED_PROVISIONER_HEADERS.keyId, "other-key-v1");
assert.throws(() => verify(keyTamper), hasCode("managed_provisioner_identity_mismatch", 401));
const forbiddenBearer = signedRequest();
forbiddenBearer.headers.authorization = "Bearer value-that-must-not-appear-in-errors";
forbiddenBearer.rawHeaders.push("Authorization", forbiddenBearer.headers.authorization);
assert.throws(
() => verify(forbiddenBearer),
(error) => hasCode("managed_provisioner_authorization_header_forbidden", 401)(error)
&& !error.message.includes("value-that-must-not-appear-in-errors"),
);
const stale = signedRequest({ timestamp: new Date(nowMs - 60_000).toISOString() });
assert.throws(() => verify(stale), hasCode("managed_provisioner_timestamp_outside_window", 401));
const future = signedRequest({ timestamp: new Date(nowMs + 60_000).toISOString() });
assert.throws(() => verify(future), hasCode("managed_provisioner_timestamp_outside_window", 401));
const invalidTimestamp = signedRequest();
setHeader(invalidTimestamp, MANAGED_PROVISIONER_HEADERS.timestamp, "2026-07-16T12:00:00Z");
assert.throws(() => verify(invalidTimestamp), hasCode("managed_provisioner_timestamp_invalid", 401));
const duplicateHeader = signedRequest();
duplicateHeader.rawHeaders.push(MANAGED_PROVISIONER_HEADERS.nonce, header(duplicateHeader, MANAGED_PROVISIONER_HEADERS.nonce));
assert.throws(() => verify(duplicateHeader), hasCode("managed_provisioner_header_invalid", 401));
const uncaptured = signedRequest();
uncaptured.rawBodyCaptured = false;
uncaptured.headers["content-length"] = String(uncaptured.rawBody.length);
assert.throws(() => verify(uncaptured), hasCode("managed_provisioner_raw_body_unavailable", 401));
const badSignature = signedRequest();
setHeader(badSignature, MANAGED_PROVISIONER_HEADERS.signature, randomBytes(64).toString("base64url"));
assert.throws(() => verify(badSignature), hasCode("managed_provisioner_signature_invalid", 401));
const paddedSignature = signedRequest();
setHeader(
paddedSignature,
MANAGED_PROVISIONER_HEADERS.signature,
`${header(paddedSignature, MANAGED_PROVISIONER_HEADERS.signature)}=`,
);
assert.throws(() => verify(paddedSignature), hasCode("managed_provisioner_signature_invalid", 401));
const uppercaseHash = signedRequest();
setHeader(
uppercaseHash,
MANAGED_PROVISIONER_HEADERS.bodySha256,
header(uppercaseHash, MANAGED_PROVISIONER_HEADERS.bodySha256).toUpperCase(),
);
assert.throws(() => verify(uppercaseHash), hasCode("managed_provisioner_body_hash_invalid", 401));
const cache = new ManagedProvisionerReplayCache({ maxEntries: 1 });
cache.consume("first", nowMs + 1000, nowMs);
assert.throws(() => cache.consume("second", nowMs + 1000, nowMs), hasCode("managed_provisioner_replay_cache_exhausted", 503));
assert.throws(() => cache.consume("first", nowMs + 1000, nowMs), hasCode("managed_provisioner_request_replayed", 409));
cache.consume("second", nowMs + 2000, nowMs + 1000);
const directory = await mkdtemp(join(tmpdir(), "nodedc-edp-managed-auth-"));
try {
const publicKeyPath = join(directory, "engine-public-key.pem");
const rsaPublicKeyPath = join(directory, "rsa-public-key.pem");
const privateKeyPath = join(directory, "engine-private-key.pem");
const symlinkPath = join(directory, "engine-public-key-link.pem");
await writeFile(publicKeyPath, publicKey.export({ type: "spki", format: "pem" }), { mode: 0o600 });
assert.equal(loadManagedProvisionerPublicKeyFile(publicKeyPath).asymmetricKeyType, "ed25519");
assert.throws(
() => loadManagedProvisionerPublicKeyFile("relative-public-key.pem"),
/external_data_plane_managed_provisioner_public_key_file_must_be_absolute/,
);
assert.throws(
() => loadManagedProvisionerPublicKeyFile(join(directory, "missing.pem")),
/external_data_plane_managed_provisioner_public_key_file_unreadable/,
);
await symlink(publicKeyPath, symlinkPath);
assert.throws(
() => loadManagedProvisionerPublicKeyFile(symlinkPath),
/external_data_plane_managed_provisioner_public_key_file_must_be_regular/,
);
await chmod(publicKeyPath, 0o622);
assert.throws(
() => loadManagedProvisionerPublicKeyFile(publicKeyPath),
/external_data_plane_managed_provisioner_public_key_file_permissions_invalid/,
);
await chmod(publicKeyPath, 0o600);
const { publicKey: rsaPublicKey } = generateKeyPairSync("rsa", { modulusLength: 2048 });
await writeFile(rsaPublicKeyPath, rsaPublicKey.export({ type: "spki", format: "pem" }), { mode: 0o600 });
assert.throws(
() => loadManagedProvisionerPublicKeyFile(rsaPublicKeyPath),
/external_data_plane_managed_provisioner_public_key_algorithm_invalid/,
);
await writeFile(privateKeyPath, privateKey.export({ type: "pkcs8", format: "pem" }), { mode: 0o600 });
assert.throws(
() => loadManagedProvisionerPublicKeyFile(privateKeyPath),
/external_data_plane_managed_provisioner_public_key_file_format_invalid/,
);
} finally {
await rm(directory, { recursive: true, force: true });
}
console.log("external-data-plane managed provisioner auth: ok");
function signedRequest({
method = "PUT",
path = "/internal/data-plane/v1/writer-bindings/by-key/engine.test.positions",
rawBody = Buffer.from('{"generation":1}', "utf8"),
timestamp = new Date(nowMs).toISOString(),
nonce = nextNonce(),
} = {}) {
const bodySha256 = sha256RawBody(rawBody);
const signature = sign(null, Buffer.from(managedProvisionerSigningPayload({
audience,
serviceId,
keyId,
method,
path,
timestamp,
nonce,
bodySha256,
}), "utf8"), privateKey).toString("base64url");
const headers = {
[MANAGED_PROVISIONER_HEADERS.serviceId]: serviceId,
[MANAGED_PROVISIONER_HEADERS.keyId]: keyId,
[MANAGED_PROVISIONER_HEADERS.audience]: audience,
[MANAGED_PROVISIONER_HEADERS.timestamp]: timestamp,
[MANAGED_PROVISIONER_HEADERS.nonce]: nonce,
[MANAGED_PROVISIONER_HEADERS.bodySha256]: bodySha256,
[MANAGED_PROVISIONER_HEADERS.signature]: signature,
"content-length": String(rawBody.length),
};
return {
method,
originalUrl: path,
headers,
rawHeaders: Object.entries(headers).flat(),
rawBody,
rawBodyCaptured: true,
};
}
function nextNonce() {
nonceSequence += 1;
return Buffer.from(`managed-auth-test-${String(nonceSequence).padStart(6, "0")}`).toString("base64url");
}
function header(req, name) {
return req.headers[name];
}
function setHeader(req, name, value) {
req.headers[name] = value;
const index = req.rawHeaders.findIndex((candidate) => String(candidate).toLowerCase() === name);
req.rawHeaders[index + 1] = value;
}
function hasCode(code, status) {
return (error) => error?.code === code && error?.status === status;
}

View File

@ -1,12 +1,14 @@
import assert from "node:assert/strict"; import assert from "node:assert/strict";
import { validateIntakeBatch } from "../../../packages/external-provider-contract/src/index.mjs"; import { validateIntakeBatch } from "../../../packages/external-provider-contract/src/data-plane.mjs";
import { import {
createWriterToken, createWriterToken,
hashWriterToken, hashWriterToken,
materializeDataProductPublish, materializeDataProductPublish,
materializeWriterBoundBatch, materializeWriterBoundBatch,
normalizeManagedWriterBindingRequest,
normalizeWriterBindingRequest, normalizeWriterBindingRequest,
safeWriterBinding, safeWriterBinding,
writerBindingRequestHash,
} from "../src/writer-binding.mjs"; } from "../src/writer-binding.mjs";
const now = new Date("2026-07-15T12:00:00.000Z"); const now = new Date("2026-07-15T12:00:00.000Z");
@ -34,6 +36,39 @@ assert.match(token, /^ndc_edpwb_[A-Za-z0-9_-]{40,}$/);
assert.equal(hashWriterToken(token), hashWriterToken(token)); assert.equal(hashWriterToken(token), hashWriterToken(token));
assert.notEqual(hashWriterToken(token), hashWriterToken(`${token}x`)); assert.notEqual(hashWriterToken(token), hashWriterToken(`${token}x`));
const managedPolicy = normalizeManagedWriterBindingRequest({
...request,
allowedDataProductIds: ["fleet.positions.current.v1", "asset.status.current.v1"],
generation: 1,
capabilityDigest: hashWriterToken(token),
}, { now, maxTtlDays: 90 });
assert.deepEqual(managedPolicy.allowedDataProductIds, [
"asset.status.current.v1",
"fleet.positions.current.v1",
]);
assert.equal(managedPolicy.generation, 1);
assert.equal(managedPolicy.capabilityDigest, hashWriterToken(token));
assert.equal(writerBindingRequestHash(managedPolicy), writerBindingRequestHash({
...managedPolicy,
allowedDataProductIds: [...managedPolicy.allowedDataProductIds].reverse(),
}));
assert.throws(() => normalizeManagedWriterBindingRequest({
...request,
generation: 0,
capabilityDigest: hashWriterToken(token),
}, { now, maxTtlDays: 90 }), /managed_writer_binding_generation_invalid/);
assert.throws(() => normalizeManagedWriterBindingRequest({
...request,
generation: 1,
capabilityDigest: "not-a-digest",
}, { now, maxTtlDays: 90 }), /managed_writer_binding_capability_digest_invalid/);
assert.throws(() => normalizeManagedWriterBindingRequest({
...request,
generation: 1,
capabilityDigest: hashWriterToken(token),
token,
}, { now, maxTtlDays: 90 }), /managed_writer_binding_secret_material_forbidden/);
const unscopedBatch = { const unscopedBatch = {
schemaVersion: "nodedc.external-provider-contract/v1", schemaVersion: "nodedc.external-provider-contract/v1",
source: { providerId: "example-provider" }, source: { providerId: "example-provider" },

View File

@ -1,19 +1,19 @@
{ {
"version": "0.1.0", "version": "1.0.0",
"updatedAt": "2026-07-13", "updatedAt": "2026-07-16",
"aliases": [ "aliases": [
{ "alias": "gelios", "canonicalId": "gelios.integration" }, { "alias": "gelios", "canonicalId": "gelios.integration" },
{ "alias": "gelius", "canonicalId": "gelios.integration" }, { "alias": "gelius", "canonicalId": "gelios.integration" },
{ "alias": "helius", "canonicalId": "gelios.integration" }, { "alias": "helius", "canonicalId": "gelios.integration" },
{ "alias": "hel i os", "canonicalId": "gelios.integration" }, { "alias": "hel i os", "canonicalId": "gelios.integration" },
{ "alias": "гелиос", "canonicalId": "gelios.integration" }, { "alias": "гелиос", "canonicalId": "gelios.integration" },
{ "alias": "трайк robot2b", "canonicalId": "gelios.unit" }, { "alias": "отслеживаемый объект гелиос", "canonicalId": "gelios.unit" },
{ "alias": "юнит гелиос", "canonicalId": "gelios.unit" }, { "alias": "юнит гелиос", "canonicalId": "gelios.unit" },
{ "alias": "группа юнитов гелиос", "canonicalId": "gelios.unit_group" }, { "alias": "группа юнитов гелиос", "canonicalId": "gelios.unit_group" },
{ "alias": "последняя телеметрия гелиос", "canonicalId": "gelios.telemetry_snapshot" }, { "alias": "последняя телеметрия гелиос", "canonicalId": "gelios.telemetry_snapshot" },
{ "alias": "сырая телеметрия гелиос", "canonicalId": "gelios.raw_telemetry_message" }, { "alias": "сырая телеметрия гелиос", "canonicalId": "gelios.raw_telemetry_message" },
{ "alias": "позиция трайка", "canonicalId": "gelios.position_fix" }, { "alias": "позиция объекта гелиос", "canonicalId": "gelios.position_fix" },
{ "alias": "статус трайка", "canonicalId": "gelios.operational_status" }, { "alias": "статус объекта гелиос", "canonicalId": "gelios.operational_status" },
{ "alias": "датчик гелиос", "canonicalId": "gelios.sensor_definition" }, { "alias": "датчик гелиос", "canonicalId": "gelios.sensor_definition" },
{ "alias": "показание датчика", "canonicalId": "gelios.sensor_reading" }, { "alias": "показание датчика", "canonicalId": "gelios.sensor_reading" },
{ "alias": "калибровка датчика", "canonicalId": "gelios.sensor_conversion" }, { "alias": "калибровка датчика", "canonicalId": "gelios.sensor_conversion" },
@ -23,7 +23,7 @@
{ "alias": "шаблон команды гелиос", "canonicalId": "gelios.command_template" }, { "alias": "шаблон команды гелиос", "canonicalId": "gelios.command_template" },
{ "alias": "диспетчеризация команды гелиос", "canonicalId": "gelios.command_dispatch" }, { "alias": "диспетчеризация команды гелиос", "canonicalId": "gelios.command_dispatch" },
{ "alias": "аудит команды гелиос", "canonicalId": "gelios.command_audit" }, { "alias": "аудит команды гелиос", "canonicalId": "gelios.command_audit" },
{ "alias": "контур robot2b", "canonicalId": "gelios.access_scope" }, { "alias": "область доступа credential гелиос", "canonicalId": "gelios.access_scope" },
{ "alias": "курсор загрузки гелиос", "canonicalId": "gelios.ingestion_cursor" } { "alias": "курсор загрузки гелиос", "canonicalId": "gelios.ingestion_cursor" }
] ]
} }

View File

@ -1,19 +1,19 @@
{ {
"version": "0.1.0", "version": "1.0.0",
"updatedAt": "2026-07-13", "updatedAt": "2026-07-16",
"entities": [ "entities": [
{ "id": "gelios.integration", "name": "Gelios Integration", "surface": "platform", "status": ["product-required", "source-evidenced"], "authority": "Gelios Gateway", "summary": "Server-side provider integration boundary for Gelios REST OAuth and legacy compatibility. It owns no UI, map renderer or workflow secret." }, { "id": "gelios.integration", "name": "Gelios Integration", "surface": "platform", "status": ["product-required", "source-evidenced"], "authority": "NDC L2 connection", "summary": "One tenant-scoped Gelios provider connection backed by one opaque NDC L2 credential reference. It owns no UI, renderer, secret value or provider-specific Platform service." },
{ "id": "gelios.access_scope", "name": "Gelios Access Scope", "surface": "platform", "status": ["product-required", "source-evidenced"], "authority": "Robot2B owner + Gelios Gateway", "summary": "Approved collection boundary expressed as an allowlist or owner rule for units, groups and allowed capabilities; distinct from broad provider-account visibility." }, { "id": "gelios.access_scope", "name": "Gelios Access Scope", "surface": "platform", "status": ["product-required", "source-evidenced"], "authority": "Gelios credential visibility + NDC L2 connection policy", "summary": "Dynamic collection boundary: selected safe-read capabilities include every entity visible to the bound credential on each run. Entity allowlists and provider-group business filters are not part of collection scope." },
{ "id": "gelios.unit", "name": "Gelios Unit", "surface": "domain", "status": ["source-evidenced"], "authority": "Gelios Pro", "summary": "Tracked operational object, including a Robot2B trike when its stable source unit ID is inside the approved scope." }, { "id": "gelios.unit", "name": "Gelios Unit", "surface": "domain", "status": ["source-evidenced"], "authority": "Gelios Pro", "summary": "Tracked operational object with a stable provider source ID. Every unit returned for the bound credential is eligible for canonical mapping." },
{ "id": "gelios.unit_group", "name": "Gelios Unit Group", "surface": "domain", "status": ["source-evidenced"], "authority": "Gelios Pro", "summary": "Provider-managed grouping of units. It is evidence for navigation and access, not an automatic definition of the Robot2B business scope." }, { "id": "gelios.unit_group", "name": "Gelios Unit Group", "surface": "domain", "status": ["source-evidenced"], "authority": "Gelios Pro", "summary": "Provider-managed grouping of units used as domain data and optional consumer metadata, never as an implicit collection filter." },
{ "id": "gelios.tracker_device", "name": "Tracker Device", "surface": "domain", "status": ["source-evidenced"], "authority": "Gelios Pro", "summary": "Hardware tracker associated with a unit, carrying device type/manufacturer and restricted hardware identity fields." }, { "id": "gelios.tracker_device", "name": "Tracker Device", "surface": "domain", "status": ["source-evidenced"], "authority": "Gelios Pro", "summary": "Hardware tracker associated with a unit, carrying device type/manufacturer and restricted hardware identity fields." },
{ "id": "gelios.telemetry_snapshot", "name": "Telemetry Snapshot", "surface": "telemetry", "status": ["source-evidenced", "product-required"], "authority": "Gelios Gateway", "summary": "Normalized current telemetry state for one unit at observed and received time; derived from the Gelios pure last message contract." }, { "id": "gelios.telemetry_snapshot", "name": "Telemetry Snapshot", "surface": "telemetry", "status": ["source-evidenced", "product-required"], "authority": "NDC L2 semantic mapping", "summary": "Normalized current telemetry state for one unit at observed and received time; derived from the versioned Gelios last-message mapping contract." },
{ "id": "gelios.raw_telemetry_message", "name": "Raw Telemetry Message", "surface": "telemetry", "status": ["source-evidenced"], "authority": "Gelios Pro", "summary": "Provider hardware message preserved only behind restricted raw-data policy; it is not the default Studio or analytics contract." }, { "id": "gelios.raw_telemetry_message", "name": "Raw Telemetry Message", "surface": "telemetry", "status": ["source-evidenced"], "authority": "Gelios Pro", "summary": "Provider hardware message preserved only behind restricted raw-data policy; it is not the default Studio or analytics contract." },
{ "id": "gelios.position_fix", "name": "Position Fix", "surface": "telemetry", "status": ["source-evidenced", "product-required"], "authority": "Gelios Gateway", "summary": "Time-qualified geographic position with latitude, longitude, height, course, speed, satellite and quality attributes for one unit." }, { "id": "gelios.position_fix", "name": "Position Fix", "surface": "telemetry", "status": ["source-evidenced", "product-required"], "authority": "NDC L2 semantic mapping", "summary": "Time-qualified geographic position with latitude, longitude, height, course, speed, satellite and quality attributes for one unit." },
{ "id": "gelios.telemetry_parameter", "name": "Telemetry Parameter", "surface": "telemetry", "status": ["source-evidenced"], "authority": "Gelios Pro", "summary": "Name/value parameter received with a message. Parameters are dynamic and require a namespaced registry and whitelist before product exposure." }, { "id": "gelios.telemetry_parameter", "name": "Telemetry Parameter", "surface": "telemetry", "status": ["source-evidenced"], "authority": "Gelios Pro", "summary": "Name/value parameter received with a message. Parameters are dynamic and require a namespaced registry and whitelist before product exposure." },
{ "id": "gelios.operational_status", "name": "Operational Status", "surface": "telemetry", "status": ["source-evidenced", "product-required"], "authority": "Gelios Gateway", "summary": "Derived unit state such as online, offline, moving, parked, no-position or low-GPS; distinct from raw provider message fields." }, { "id": "gelios.operational_status", "name": "Operational Status", "surface": "telemetry", "status": ["source-evidenced", "product-required"], "authority": "NDC L2 semantic mapping", "summary": "Derived unit state such as active, stale, moving, parked, no-position or low-GPS; distinct from raw provider message fields." },
{ "id": "gelios.sensor_definition", "name": "Sensor Definition", "surface": "domain", "status": ["source-evidenced"], "authority": "Gelios Pro", "summary": "Declared sensor on a unit with message parameter, type, unit of measure, visibility and optional fuel semantics." }, { "id": "gelios.sensor_definition", "name": "Sensor Definition", "surface": "domain", "status": ["source-evidenced"], "authority": "Gelios Pro", "summary": "Declared sensor on a unit with message parameter, type, unit of measure, visibility and optional fuel semantics." },
{ "id": "gelios.sensor_reading", "name": "Sensor Reading", "surface": "telemetry", "status": ["source-evidenced", "product-required"], "authority": "Gelios Gateway", "summary": "Time-qualified normalized reading of a defined sensor, retaining both numeric value and human-readable representation when permitted." }, { "id": "gelios.sensor_reading", "name": "Sensor Reading", "surface": "telemetry", "status": ["source-evidenced", "product-required"], "authority": "NDC L2 semantic mapping", "summary": "Time-qualified normalized reading of a defined sensor, retaining both numeric value and human-readable representation only when permitted by a versioned field policy." },
{ "id": "gelios.sensor_conversion", "name": "Sensor Conversion", "surface": "domain", "status": ["source-evidenced"], "authority": "Gelios Pro", "summary": "Calibration/conversion rule or row for translating raw sensor values into operational measures." }, { "id": "gelios.sensor_conversion", "name": "Sensor Conversion", "surface": "domain", "status": ["source-evidenced"], "authority": "Gelios Pro", "summary": "Calibration/conversion rule or row for translating raw sensor values into operational measures." },
{ "id": "gelios.fuel_profile", "name": "Fuel Profile", "surface": "domain", "status": ["source-evidenced"], "authority": "Gelios Pro", "summary": "Fuel and consumption configuration associated with a unit or fuel sensor; use only after sensor semantics are approved." }, { "id": "gelios.fuel_profile", "name": "Fuel Profile", "surface": "domain", "status": ["source-evidenced"], "authority": "Gelios Pro", "summary": "Fuel and consumption configuration associated with a unit or fuel sensor; use only after sensor semantics are approved." },
{ "id": "gelios.maintenance_plan", "name": "Maintenance Plan", "surface": "domain", "status": ["source-evidenced"], "authority": "Gelios Pro", "summary": "Planned maintenance configuration for a unit, separate from historical maintenance events." }, { "id": "gelios.maintenance_plan", "name": "Maintenance Plan", "surface": "domain", "status": ["source-evidenced"], "authority": "Gelios Pro", "summary": "Planned maintenance configuration for a unit, separate from historical maintenance events." },
@ -22,13 +22,13 @@
{ "id": "gelios.geozone_group", "name": "Gelios Geozone Group", "surface": "spatial", "status": ["source-evidenced"], "authority": "Gelios Pro", "summary": "Provider grouping of geozones." }, { "id": "gelios.geozone_group", "name": "Gelios Geozone Group", "surface": "spatial", "status": ["source-evidenced"], "authority": "Gelios Pro", "summary": "Provider grouping of geozones." },
{ "id": "gelios.geopoint", "name": "Gelios Geopoint", "surface": "spatial", "status": ["source-evidenced"], "authority": "Gelios Pro", "summary": "Named provider spatial point; it must remain distinct from a live unit position." }, { "id": "gelios.geopoint", "name": "Gelios Geopoint", "surface": "spatial", "status": ["source-evidenced"], "authority": "Gelios Pro", "summary": "Named provider spatial point; it must remain distinct from a live unit position." },
{ "id": "gelios.report_template", "name": "Gelios Report Template", "surface": "analytics", "status": ["source-evidenced"], "authority": "Gelios Pro", "summary": "Reusable provider report definition available to the account or scope." }, { "id": "gelios.report_template", "name": "Gelios Report Template", "surface": "analytics", "status": ["source-evidenced"], "authority": "Gelios Pro", "summary": "Reusable provider report definition available to the account or scope." },
{ "id": "gelios.report_result", "name": "Gelios Report Result", "surface": "analytics", "status": ["source-evidenced"], "authority": "Gelios Pro + Gelios Gateway", "summary": "Bounded result, table, graphic or map output produced from an approved report request; not a default realtime stream." }, { "id": "gelios.report_result", "name": "Gelios Report Result", "surface": "analytics", "status": ["source-evidenced"], "authority": "Gelios Pro + NDC L2 semantic mapping", "summary": "Bounded result, table, graphic or map output produced from an approved report request; not a default realtime stream." },
{ "id": "gelios.collection_run", "name": "Gelios Collection Run", "surface": "platform", "status": ["product-required"], "authority": "Gelios Gateway", "summary": "Auditable read-only collection attempt with scope, endpoint family, volume controls and outcome; it contains no token or raw payload." }, { "id": "gelios.collection_run", "name": "Gelios Collection Run", "surface": "platform", "status": ["product-required"], "authority": "NDC L2 workflow", "summary": "Auditable safe-read collection attempt with credential-visible entity scope, endpoint family, volume controls and outcome; it contains no credential value or raw payload." },
{ "id": "gelios.ingestion_cursor", "name": "Gelios Ingestion Cursor", "surface": "platform", "status": ["product-required"], "authority": "Gelios Gateway + telemetry storage", "summary": "Checkpoint and watermark controlling resumable historical or incremental ingestion for an approved unit and data family." }, { "id": "gelios.ingestion_cursor", "name": "Gelios Ingestion Cursor", "surface": "platform", "status": ["product-required"], "authority": "NDC L2 workflow state + External Data Plane", "summary": "Checkpoint and watermark controlling resumable historical or incremental ingestion for a connection and data family." },
{ "id": "gelios.command_template", "name": "Gelios Command Template", "surface": "command", "status": ["source-evidenced", "product-required"], "authority": "Gelios Pro", "summary": "Read-only catalogued command template assigned to a unit. Template content is red-domain configuration, not a telemetry field." }, { "id": "gelios.command_template", "name": "Gelios Command Template", "surface": "command", "status": ["source-evidenced", "product-required"], "authority": "Gelios Pro", "summary": "Read-only catalogued command template assigned to a unit. Template content is red-domain configuration, not a telemetry field." },
{ "id": "gelios.command_group", "name": "Gelios Command Group", "surface": "command", "status": ["source-evidenced", "product-required"], "authority": "Gelios Pro", "summary": "Provider grouping of command templates, managed separately from operational unit groups." }, { "id": "gelios.command_group", "name": "Gelios Command Group", "surface": "command", "status": ["source-evidenced", "product-required"], "authority": "Gelios Pro", "summary": "Provider grouping of command templates, managed separately from operational unit groups." },
{ "id": "gelios.command_dispatch", "name": "Gelios Command Dispatch", "surface": "command", "status": ["product-required"], "authority": "Future Command Gateway", "summary": "Explicit, user-confirmed request to send a command to one approved unit or approved group. It must never be created by background ingestion or map rendering." }, { "id": "gelios.command_dispatch", "name": "Gelios Command Dispatch", "surface": "command", "status": ["future-concept"], "authority": "Unimplemented NDC red-domain command boundary", "summary": "Future explicit, user-confirmed request to send a command to one approved unit or approved group. It must never be created by background ingestion or map rendering." },
{ "id": "gelios.command_delivery", "name": "Gelios Command Delivery", "surface": "command", "status": ["product-required", "source-evidenced"], "authority": "Gelios Pro + Future Command Gateway", "summary": "Provider delivery/task status associated with a command dispatch." }, { "id": "gelios.command_delivery", "name": "Gelios Command Delivery", "surface": "command", "status": ["future-concept", "source-evidenced"], "authority": "Gelios Pro; future NDC red-domain reconciliation", "summary": "Provider delivery/task status that may be reconciled with a future governed command dispatch; no command transport exists in the current package." },
{ "id": "gelios.command_audit", "name": "Gelios Command Audit", "surface": "command", "status": ["product-required"], "authority": "Future Command Gateway", "summary": "Immutable governance record of an explicit command decision, confirmation, target, outcome and actor; separate from provider command history." } { "id": "gelios.command_audit", "name": "Gelios Command Audit", "surface": "command", "status": ["future-concept"], "authority": "Unimplemented NDC red-domain command boundary", "summary": "Future immutable governance record of an explicit command decision, confirmation, target, outcome and actor; separate from provider command history." }
] ]
} }

View File

@ -1,6 +1,6 @@
{ {
"version": "0.1.0", "version": "1.0.0",
"updatedAt": "2026-07-13", "updatedAt": "2026-07-16",
"sourceRoots": [ "sourceRoots": [
{ {
"surface": "gelios-rest", "surface": "gelios-rest",
@ -8,9 +8,9 @@
"mode": "OpenAPI inspection and safe read-only runtime audit; no write routes executed" "mode": "OpenAPI inspection and safe read-only runtime audit; no write routes executed"
}, },
{ {
"surface": "engine", "surface": "ndc-l2-legacy-evidence",
"path": "/Users/dcconstructions/Downloads/mnt/NODEDC/NODEDC_ENGINE_INFRA/nodedc-source", "path": "/Users/dcconstructions/Downloads/mnt/NODEDC/NODEDC_ENGINE_INFRA/nodedc-source",
"mode": "Read-only MMAP workflow, Gelios node and historical snapshot donor inspection" "mode": "Read-only historical NDC L2 donor, Gelios mapping and synthetic-safe field-shape inspection"
}, },
{ {
"surface": "module-studio", "surface": "module-studio",
@ -41,9 +41,9 @@
"baselineDocs": ["docs/GELIOS_DOMAIN_ONTOLOGY.md"], "baselineDocs": ["docs/GELIOS_DOMAIN_ONTOLOGY.md"],
"restrictions": [ "restrictions": [
"Do not copy access/refresh tokens, passwords, hardware decrypt keys or runtime snapshots into Ontology Core.", "Do not copy access/refresh tokens, passwords, hardware decrypt keys or runtime snapshots into Ontology Core.",
"Do not make the current Engine Gelios node, Cesium entities or provider group names canonical domain identities.", "Do not make a current NDC L2 workflow node, renderer entities or provider group names canonical domain identities.",
"Do not invoke command send, create, update, delete or purge routes as ontology evidence.", "Do not invoke command send, create, update, delete or purge routes as ontology evidence.",
"Do not treat the 107 currently visible provider-account units or the 95 legacy snapshot units as the final Robot2B scope without owner approval.", "Robot2B pilot counts (107 credential-visible units and 95 legacy snapshot units) are historical evidence only; canonical collection scope is dynamically all entities visible to the bound credential.",
"Do not bulk-download history, media or full geozone geometry before collection policy and storage architecture are approved." "Do not bulk-download history, media or full geozone geometry before collection policy and storage architecture are approved."
] ]
} }

View File

@ -1,6 +1,6 @@
{ {
"version": "0.1.0", "version": "1.0.0",
"updatedAt": "2026-07-13", "updatedAt": "2026-07-16",
"rules": [ "rules": [
{ {
"id": "guardrail.gelios.ontology_not_runtime_store", "id": "guardrail.gelios.ontology_not_runtime_store",
@ -8,10 +8,16 @@
"summary": "Ontology Core describes Gelios meanings and contracts only. It must not contain access/refresh tokens, raw messages, live positions, provider command text, hardware decrypt keys or bulk geometry.", "summary": "Ontology Core describes Gelios meanings and contracts only. It must not contain access/refresh tokens, raw messages, live positions, provider command text, hardware decrypt keys or bulk geometry.",
"entityIds": ["gelios.integration", "gelios.telemetry_snapshot", "gelios.raw_telemetry_message", "gelios.position_fix", "gelios.command_template", "gelios.geozone"] "entityIds": ["gelios.integration", "gelios.telemetry_snapshot", "gelios.raw_telemetry_message", "gelios.position_fix", "gelios.command_template", "gelios.geozone"]
}, },
{
"id": "guardrail.gelios.provider_auth_pair_not_capability_scope",
"severity": "error",
"summary": "Gelios provider authentication consists of access and refresh tokens only. Credential labels and safe-read classifications are workflow policy, not provider token scope; internal External Data Plane capabilities are not Gelios tokens.",
"entityIds": ["gelios.integration", "gelios.access_scope", "gelios.collection_run"]
},
{ {
"id": "guardrail.gelios.scope_precedes_collection", "id": "guardrail.gelios.scope_precedes_collection",
"severity": "error", "severity": "error",
"summary": "A Gelios collection run must resolve an approved Robot2B access scope before addressing units. Provider-account visibility and provider group membership are not sufficient product scope on their own.", "summary": "A Gelios safe-read collection run must process every valid entity returned for the bound credential and selected capability. Static unit allowlists and provider-group business filters are forbidden at collection boundary; consumer visibility is handled downstream.",
"entityIds": ["gelios.integration", "gelios.access_scope", "gelios.collection_run", "gelios.unit", "gelios.unit_group"] "entityIds": ["gelios.integration", "gelios.access_scope", "gelios.collection_run", "gelios.unit", "gelios.unit_group"]
}, },
{ {
@ -41,7 +47,7 @@
{ {
"id": "guardrail.gelios.history_requires_cursor", "id": "guardrail.gelios.history_requires_cursor",
"severity": "warning", "severity": "warning",
"summary": "Any future history ingestion must be bounded by unit scope, time window, field whitelist and durable cursor. It must not turn a historical API family into an uncontrolled account dump.", "summary": "Any future history ingestion must be bounded by connection, credential-visible entity scope, time window, field policy and durable cursor. It must not turn a historical API family into an uncontrolled account dump.",
"entityIds": ["gelios.collection_run", "gelios.ingestion_cursor", "gelios.raw_telemetry_message", "gelios.report_result"] "entityIds": ["gelios.collection_run", "gelios.ingestion_cursor", "gelios.raw_telemetry_message", "gelios.report_result"]
} }
], ],

View File

@ -1,7 +1,7 @@
{ {
"id": "gelios", "id": "gelios",
"version": "0.1.0", "version": "1.0.0",
"updatedAt": "2026-07-13", "updatedAt": "2026-07-16",
"status": "product-required/source-evidenced", "status": "product-required/source-evidenced",
"summary": "Gelios Pro telemetry, fleet, spatial, reporting and command-domain ontology for the Robot2B client context. The package describes canonical meanings and contracts; it does not store credentials or runtime telemetry." "summary": "Provider-neutral Gelios Pro telemetry, fleet, spatial, reporting and command-domain ontology. It describes canonical meanings and contracts; it does not store credentials, account scope or runtime telemetry."
} }

View File

@ -1,23 +1,23 @@
{ {
"version": "0.1.0", "version": "1.0.0",
"updatedAt": "2026-07-13", "updatedAt": "2026-07-16",
"relations": [ "relations": [
{ "id": "gelios.integration.is_provider_connection", "from": ["gelios.integration"], "to": ["integration.connection"], "status": "product-required", "summary": "A Gelios integration is a tenant-scoped instance of the common external provider connection contract." }, { "id": "gelios.integration.is_provider_connection", "from": ["gelios.integration"], "to": ["integration.connection"], "status": "product-required", "summary": "A Gelios integration is a tenant-scoped instance of the common external provider connection contract." },
{ "id": "gelios.access_scope.specializes_integration_scope", "from": ["gelios.access_scope"], "to": ["integration.access_scope"], "status": "product-required", "summary": "Robot2B/Gelios object approval specializes the common provider access-scope contract." }, { "id": "gelios.access_scope.specializes_integration_scope", "from": ["gelios.access_scope"], "to": ["integration.access_scope"], "status": "product-required", "summary": "Gelios credential-visible capability scope specializes the common provider access-scope contract." },
{ "id": "gelios.collection_run.is_integration_collection_run", "from": ["gelios.collection_run"], "to": ["integration.collection_run"], "status": "product-required", "summary": "Gelios collection runs inherit the bounded collection and audit lifecycle of external integrations." }, { "id": "gelios.collection_run.is_integration_collection_run", "from": ["gelios.collection_run"], "to": ["integration.collection_run"], "status": "product-required", "summary": "Gelios collection runs inherit the bounded collection and audit lifecycle of external integrations." },
{ "id": "gelios.raw_message.is_integration_raw_envelope", "from": ["gelios.raw_telemetry_message"], "to": ["integration.raw_envelope"], "status": "product-required", "summary": "A Gelios raw message is a restricted provider envelope, not a UI read model." }, { "id": "gelios.raw_message.is_integration_raw_envelope", "from": ["gelios.raw_telemetry_message"], "to": ["integration.raw_envelope"], "status": "product-required", "summary": "A Gelios raw message is a restricted provider envelope, not a UI read model." },
{ "id": "gelios.unit.is_integration_canonical_subject", "from": ["gelios.unit"], "to": ["integration.canonical_subject"], "status": "product-required", "summary": "A Gelios unit is a stable canonical subject produced by the provider adapter." }, { "id": "gelios.unit.is_integration_canonical_subject", "from": ["gelios.unit"], "to": ["integration.canonical_subject"], "status": "product-required", "summary": "A Gelios unit is a stable canonical subject produced by the provider adapter." },
{ "id": "gelios.command_template.is_integration_command_capability", "from": ["gelios.command_template"], "to": ["integration.command_capability"], "status": "product-required", "summary": "Gelios command templates are catalogued provider command capabilities only." }, { "id": "gelios.command_template.is_integration_command_capability", "from": ["gelios.command_template"], "to": ["integration.command_capability"], "status": "product-required", "summary": "Gelios command templates are catalogued provider command capabilities only." },
{ "id": "gelios.command_dispatch.is_integration_command_intent", "from": ["gelios.command_dispatch"], "to": ["integration.command_intent"], "status": "future-concept", "summary": "Any future Gelios command dispatch must become an explicitly governed generic command intent." }, { "id": "gelios.command_dispatch.is_integration_command_intent", "from": ["gelios.command_dispatch"], "to": ["integration.command_intent"], "status": "future-concept", "summary": "Any future Gelios command dispatch must become an explicitly governed generic command intent." },
{ "id": "gelios.command_audit.is_integration_command_audit", "from": ["gelios.command_audit"], "to": ["integration.command_audit"], "status": "future-concept", "summary": "Gelios command audit specializes the generic red-domain audit contract." }, { "id": "gelios.command_audit.is_integration_command_audit", "from": ["gelios.command_audit"], "to": ["integration.command_audit"], "status": "future-concept", "summary": "Gelios command audit specializes the generic red-domain audit contract." },
{ "id": "gelios.integration.enforces_scope", "from": ["gelios.integration"], "to": ["gelios.access_scope"], "status": "product-required", "summary": "The Gateway applies an approved Robot2B scope before reading or publishing data." }, { "id": "gelios.integration.enforces_scope", "from": ["gelios.integration"], "to": ["gelios.access_scope"], "status": "product-required", "summary": "NDC L2 resolves selected safe-read capabilities and current provider visibility from the bound credential on every collection run." },
{ "id": "gelios.access_scope.includes_unit", "from": ["gelios.access_scope"], "to": ["gelios.unit", "gelios.unit_group"], "status": "product-required", "summary": "An approved scope can include explicit units and approved group criteria, but group membership alone is not authoritative." }, { "id": "gelios.access_scope.includes_unit", "from": ["gelios.access_scope"], "to": ["gelios.unit", "gelios.unit_group"], "status": "product-required", "summary": "Scope dynamically includes every unit and group record returned by the selected capability; it contains no static entity allowlist." },
{ "id": "gelios.unit.belongs_to_group", "from": ["gelios.unit"], "to": ["gelios.unit_group"], "status": "source-evidenced", "summary": "A provider unit can belong to one or more provider unit groups." }, { "id": "gelios.unit.belongs_to_group", "from": ["gelios.unit"], "to": ["gelios.unit_group"], "status": "source-evidenced", "summary": "A provider unit can belong to one or more provider unit groups." },
{ "id": "gelios.unit.uses_tracker_device", "from": ["gelios.unit"], "to": ["gelios.tracker_device"], "status": "source-evidenced", "summary": "A tracked unit is associated with a hardware tracker device." }, { "id": "gelios.unit.uses_tracker_device", "from": ["gelios.unit"], "to": ["gelios.tracker_device"], "status": "source-evidenced", "summary": "A tracked unit is associated with a hardware tracker device." },
{ "id": "gelios.unit.has_telemetry_snapshot", "from": ["gelios.unit"], "to": ["gelios.telemetry_snapshot"], "status": "product-required", "summary": "The Gateway maintains normalized current telemetry for an approved unit." }, { "id": "gelios.unit.has_telemetry_snapshot", "from": ["gelios.unit"], "to": ["gelios.telemetry_snapshot"], "status": "product-required", "summary": "The versioned NDC L2 mapping produces normalized current telemetry for every valid returned unit." },
{ "id": "gelios.raw_message.normalizes_to_snapshot", "from": ["gelios.raw_telemetry_message"], "to": ["gelios.telemetry_snapshot"], "status": "product-required", "summary": "Restricted raw provider telemetry can be normalized into the stable snapshot contract." }, { "id": "gelios.raw_message.normalizes_to_snapshot", "from": ["gelios.raw_telemetry_message"], "to": ["gelios.telemetry_snapshot"], "status": "product-required", "summary": "Restricted raw provider telemetry can be normalized into the stable snapshot contract." },
{ "id": "gelios.telemetry_snapshot.has_position_fix", "from": ["gelios.telemetry_snapshot"], "to": ["gelios.position_fix"], "status": "product-required", "summary": "A current telemetry snapshot can carry one time-qualified position fix." }, { "id": "gelios.telemetry_snapshot.has_position_fix", "from": ["gelios.telemetry_snapshot"], "to": ["gelios.position_fix"], "status": "product-required", "summary": "A current telemetry snapshot can carry one time-qualified position fix." },
{ "id": "gelios.telemetry_snapshot.has_operational_status", "from": ["gelios.telemetry_snapshot"], "to": ["gelios.operational_status"], "status": "product-required", "summary": "Gateway derives normalized operational state from telemetry and data-quality policy." }, { "id": "gelios.telemetry_snapshot.has_operational_status", "from": ["gelios.telemetry_snapshot"], "to": ["gelios.operational_status"], "status": "product-required", "summary": "The semantic mapping derives normalized operational state from telemetry and data-quality policy." },
{ "id": "gelios.telemetry_snapshot.has_parameter", "from": ["gelios.telemetry_snapshot"], "to": ["gelios.telemetry_parameter"], "status": "source-evidenced", "summary": "A provider snapshot can contain dynamic telemetry parameters." }, { "id": "gelios.telemetry_snapshot.has_parameter", "from": ["gelios.telemetry_snapshot"], "to": ["gelios.telemetry_parameter"], "status": "source-evidenced", "summary": "A provider snapshot can contain dynamic telemetry parameters." },
{ "id": "gelios.unit.has_sensor", "from": ["gelios.unit"], "to": ["gelios.sensor_definition"], "status": "source-evidenced", "summary": "A unit exposes zero or more sensor definitions." }, { "id": "gelios.unit.has_sensor", "from": ["gelios.unit"], "to": ["gelios.sensor_definition"], "status": "source-evidenced", "summary": "A unit exposes zero or more sensor definitions." },
{ "id": "gelios.sensor_definition.has_conversion", "from": ["gelios.sensor_definition"], "to": ["gelios.sensor_conversion"], "status": "source-evidenced", "summary": "A sensor can define conversion/calibration rows." }, { "id": "gelios.sensor_definition.has_conversion", "from": ["gelios.sensor_definition"], "to": ["gelios.sensor_conversion"], "status": "source-evidenced", "summary": "A sensor can define conversion/calibration rows." },
@ -29,14 +29,14 @@
{ "id": "gelios.geozone.belongs_to_group", "from": ["gelios.geozone"], "to": ["gelios.geozone_group"], "status": "source-evidenced", "summary": "A provider geozone can belong to a provider geozone group." }, { "id": "gelios.geozone.belongs_to_group", "from": ["gelios.geozone"], "to": ["gelios.geozone_group"], "status": "source-evidenced", "summary": "A provider geozone can belong to a provider geozone group." },
{ "id": "gelios.report_result.uses_template", "from": ["gelios.report_result"], "to": ["gelios.report_template"], "status": "source-evidenced", "summary": "A report result is produced from an approved report template and bounded request." }, { "id": "gelios.report_result.uses_template", "from": ["gelios.report_result"], "to": ["gelios.report_template"], "status": "source-evidenced", "summary": "A report result is produced from an approved report template and bounded request." },
{ "id": "gelios.collection_run.checkpoints_with_cursor", "from": ["gelios.collection_run"], "to": ["gelios.ingestion_cursor"], "status": "product-required", "summary": "Collection runs advance resumable ingestion only through a durable cursor." }, { "id": "gelios.collection_run.checkpoints_with_cursor", "from": ["gelios.collection_run"], "to": ["gelios.ingestion_cursor"], "status": "product-required", "summary": "Collection runs advance resumable ingestion only through a durable cursor." },
{ "id": "gelios.collection_run.collects_unit", "from": ["gelios.collection_run"], "to": ["gelios.unit"], "status": "product-required", "summary": "A read-only collection run records the approved unit scope it addressed." }, { "id": "gelios.collection_run.collects_unit", "from": ["gelios.collection_run"], "to": ["gelios.unit"], "status": "product-required", "summary": "A safe-read collection run records every valid credential-visible unit returned by the selected capability." },
{ "id": "gelios.unit.has_command_template", "from": ["gelios.unit"], "to": ["gelios.command_template"], "status": "source-evidenced", "summary": "Command templates are catalogued per unit through read-only routes." }, { "id": "gelios.unit.has_command_template", "from": ["gelios.unit"], "to": ["gelios.command_template"], "status": "source-evidenced", "summary": "Command templates are catalogued per unit through read-only routes." },
{ "id": "gelios.command_template.belongs_to_group", "from": ["gelios.command_template"], "to": ["gelios.command_group"], "status": "source-evidenced", "summary": "A command template can belong to a provider command group." }, { "id": "gelios.command_template.belongs_to_group", "from": ["gelios.command_template"], "to": ["gelios.command_group"], "status": "source-evidenced", "summary": "A command template can belong to a provider command group." },
{ "id": "gelios.command_dispatch.uses_template", "from": ["gelios.command_dispatch"], "to": ["gelios.command_template"], "status": "product-required", "summary": "A dispatch may reference a reviewed command template." }, { "id": "gelios.command_dispatch.uses_template", "from": ["gelios.command_dispatch"], "to": ["gelios.command_template"], "status": "future-concept", "summary": "A future dispatch may reference a reviewed command template." },
{ "id": "gelios.command_dispatch.targets_scope", "from": ["gelios.command_dispatch"], "to": ["gelios.unit", "gelios.unit_group", "gelios.access_scope"], "status": "product-required", "summary": "A dispatch target must resolve to an approved unit or explicitly approved group in scope." }, { "id": "gelios.command_dispatch.targets_scope", "from": ["gelios.command_dispatch"], "to": ["gelios.unit", "gelios.unit_group", "gelios.access_scope"], "status": "future-concept", "summary": "A future dispatch target must resolve to an approved unit or explicitly approved group in scope." },
{ "id": "gelios.command_dispatch.has_delivery", "from": ["gelios.command_dispatch"], "to": ["gelios.command_delivery"], "status": "product-required", "summary": "A dispatch is associated with delivery/task state rather than assumed successful on request creation." }, { "id": "gelios.command_dispatch.has_delivery", "from": ["gelios.command_dispatch"], "to": ["gelios.command_delivery"], "status": "future-concept", "summary": "A future dispatch is associated with delivery/task state rather than assumed successful on request creation." },
{ "id": "gelios.command_dispatch.has_audit", "from": ["gelios.command_dispatch"], "to": ["gelios.command_audit"], "status": "product-required", "summary": "Every dispatch requires immutable user-confirmation and outcome audit." }, { "id": "gelios.command_dispatch.has_audit", "from": ["gelios.command_dispatch"], "to": ["gelios.command_audit"], "status": "future-concept", "summary": "Every future dispatch requires immutable user-confirmation and outcome audit." },
{ "id": "gelios.unit.is_map_moving_object", "from": ["gelios.unit"], "to": ["map.moving_object"], "status": "product-required", "summary": "An approved Gelios unit is rendered through the provider-neutral moving-object contract, never as a Cesium entity identity." }, { "id": "gelios.unit.is_map_moving_object", "from": ["gelios.unit"], "to": ["map.moving_object"], "status": "product-required", "summary": "A valid credential-visible Gelios unit is exposed through the provider-neutral moving-object contract, never as a renderer entity identity." },
{ "id": "gelios.position_fix.positions_map_subject", "from": ["gelios.position_fix"], "to": ["map.moving_object"], "status": "product-required", "summary": "A normalized position fix updates the stable map subject bound to the Gelios unit." }, { "id": "gelios.position_fix.positions_map_subject", "from": ["gelios.position_fix"], "to": ["map.moving_object"], "status": "product-required", "summary": "A normalized position fix updates the stable map subject bound to the Gelios unit." },
{ "id": "gelios.geozone.is_map_zone", "from": ["gelios.geozone"], "to": ["map.zone"], "status": "product-required", "summary": "An approved Gelios geozone is exposed to the map through the generic zone contract." }, { "id": "gelios.geozone.is_map_zone", "from": ["gelios.geozone"], "to": ["map.zone"], "status": "product-required", "summary": "An approved Gelios geozone is exposed to the map through the generic zone contract." },
{ "id": "gelios.geopoint.is_map_place_target", "from": ["gelios.geopoint"], "to": ["map.place_target"], "status": "product-required", "summary": "An approved Gelios geopoint is exposed through a provider-neutral place target." } { "id": "gelios.geopoint.is_map_place_target", "from": ["gelios.geopoint"], "to": ["map.place_target"], "status": "product-required", "summary": "An approved Gelios geopoint is exposed through a provider-neutral place target." }

View File

@ -1,6 +1,6 @@
{ {
"version": "0.1.0", "version": "1.0.0",
"updatedAt": "2026-07-13", "updatedAt": "2026-07-16",
"aliases": [ "aliases": [
{"alias":"коннектор","canonicalId":"integration.connection","language":"ru","status":"product-required"}, {"alias":"коннектор","canonicalId":"integration.connection","language":"ru","status":"product-required"},
{"alias":"connector","canonicalId":"integration.connection","language":"en","status":"product-required"}, {"alias":"connector","canonicalId":"integration.connection","language":"en","status":"product-required"},

View File

@ -1,21 +1,21 @@
{ {
"version": "0.1.0", "version": "1.0.0",
"updatedAt": "2026-07-13", "updatedAt": "2026-07-16",
"entities": [ "entities": [
{"id":"integration.provider","name":"External Provider","surface":"integration","status":["product-required"],"authority":"Platform External Provider Data Plane","summary":"A third-party product or API family integrated through an app-owned adapter, not a tenant or a renderer."}, {"id":"integration.provider","name":"External Provider","surface":"integration","status":["product-required"],"authority":"Versioned provider package + Ontology Core","summary":"A third-party product or API family integrated through NDC L2, not a tenant, runtime service or renderer."},
{"id":"integration.connection","name":"Provider Connection","surface":"integration","status":["product-required"],"authority":"Provider Gateway","summary":"A tenant-scoped configured connection to one provider account or endpoint; it contains policy and secret references, never secret values."}, {"id":"integration.connection","name":"Provider Connection","surface":"integration","status":["product-required"],"authority":"NDC L2 connection","summary":"A tenant-scoped instance of one versioned provider package for one provider account; it contains policy and opaque credential references, never secret values."},
{"id":"integration.credential_reference","name":"Provider Credential Reference","surface":"integration","status":["product-required"],"authority":"Provider Gateway / secret store","summary":"Opaque reference to a server-side secret or token lifecycle, not a token, login or password."}, {"id":"integration.credential_reference","name":"Provider Credential Reference","surface":"integration","status":["product-required"],"authority":"NDC L2 Credentials","summary":"Opaque reference to an NDC L2-managed provider credential lifecycle, not a credential value, login or password."},
{"id":"integration.access_scope","name":"Integration Access Scope","surface":"integration","status":["product-required"],"authority":"Client owner + Provider Gateway","summary":"Approved product boundary for source accounts, objects, fields and permitted operations; provider visibility alone is insufficient."}, {"id":"integration.access_scope","name":"Integration Access Scope","surface":"integration","status":["product-required"],"authority":"Provider credential visibility + NDC L2 connection profile","summary":"Selected safe-read capabilities and fields applied to every valid entity visible to the bound credential. Static entity allowlists and provider-group business filters are not collection scope."},
{"id":"integration.capability","name":"Provider Capability","surface":"integration","status":["product-required"],"authority":"Provider Adapter","summary":"Versioned provider operation or feature classified as read, metadata, write, destructive or unknown."}, {"id":"integration.capability","name":"Provider Capability","surface":"integration","status":["product-required"],"authority":"Versioned provider package","summary":"Versioned provider operation or feature classified as read, metadata, write, destructive or unknown."},
{"id":"integration.field_definition","name":"Provider Field Definition","surface":"integration","status":["product-required"],"authority":"Provider Adapter + Ontology Core","summary":"A documented source field, its semantics, sensitivity, units and normalisation mapping; it is not a live value."}, {"id":"integration.field_definition","name":"Provider Field Definition","surface":"integration","status":["product-required"],"authority":"Versioned provider package + Ontology Core","summary":"A documented source field, its semantics, sensitivity, units and normalisation mapping; it is not a live value."},
{"id":"integration.collection_profile","name":"Collection Profile","surface":"integration","status":["product-required"],"authority":"Provider Gateway","summary":"Approved rate, paging, cursor, capability and field selection for a connection; prevents uncontrolled account-wide polling."}, {"id":"integration.collection_profile","name":"Collection Profile","surface":"integration","status":["product-required"],"authority":"NDC L2 connection profile","summary":"Rate, paging, cursor, safe capability, field policy and batching selection for one connection; every valid entity returned for the credential is processed."},
{"id":"integration.retention_policy","name":"Integration Retention Policy","surface":"integration","status":["product-required"],"authority":"Client owner + Provider Gateway","summary":"Lifecycle rule for current state, raw envelope, normalised history, aggregates and backups."}, {"id":"integration.retention_policy","name":"Integration Retention Policy","surface":"integration","status":["product-required"],"authority":"External Data Plane Data Product definition","summary":"Lifecycle rule for current state, normalised history, aggregates and any separately approved raw reference."},
{"id":"integration.collection_run","name":"Integration Collection Run","surface":"integration","status":["product-required"],"authority":"Provider Gateway","summary":"Auditable bounded execution of a safe-read collection profile with cursor, response metrics and outcome."}, {"id":"integration.collection_run","name":"Integration Collection Run","surface":"integration","status":["product-required"],"authority":"NDC L2 workflow","summary":"Auditable bounded execution of a safe-read collection profile with credential-visible entity scope, cursor, response metrics and outcome."},
{"id":"integration.raw_envelope","name":"Raw Provider Envelope","surface":"integration","status":["product-required"],"authority":"Provider Gateway","summary":"Restricted provenance-preserving representation of a safe-read provider response, stored in a bounded cold layer rather than exposed to UI."}, {"id":"integration.raw_envelope","name":"Raw Provider Envelope","surface":"integration","status":["product-required"],"authority":"Restricted raw-data policy","summary":"Optional provenance reference to a separately approved bounded raw layer; inline provider payload is not a Data Product publish field."},
{"id":"integration.canonical_subject","name":"Canonical Integrated Subject","surface":"integration","status":["product-required"],"authority":"Provider Domain Adapter","summary":"Stable domain subject normalised from a provider object and suitable for relations, analytics and interfaces."}, {"id":"integration.canonical_subject","name":"Canonical Integrated Subject","surface":"integration","status":["product-required"],"authority":"NDC L2 semantic mapping + Ontology Core","summary":"Stable domain subject normalised from a provider object and suitable for relations, analytics and interfaces."},
{"id":"integration.read_model","name":"Integration Read Model","surface":"integration","status":["product-required"],"authority":"Provider Gateway","summary":"Scoped, versioned current, history or aggregate projection delivered to approved consumers without raw provider bypass."}, {"id":"integration.read_model","name":"Integration Read Model","surface":"integration","status":["product-required"],"authority":"External Data Plane","summary":"Scoped, versioned Data Product current/history projection delivered to approved consumers without provider transport or raw bypass."},
{"id":"integration.realtime_channel","name":"Integration Realtime Channel","surface":"integration","status":["product-required"],"authority":"Provider Gateway","summary":"Internal bounded event or subscription contract for projection changes; its emission rate is independent from provider collection rate."}, {"id":"integration.realtime_channel","name":"Integration Realtime Channel","surface":"integration","status":["product-required"],"authority":"External Data Plane","summary":"Bounded snapshot-and-patch contract for Data Product changes; its delivery and history cadence are independent from provider collection rate."},
{"id":"integration.command_capability","name":"Provider Command Capability","surface":"integration","status":["product-required"],"authority":"Provider Adapter","summary":"Catalogued command metadata, parameters and risk classification. It is not permission or a send route."}, {"id":"integration.command_capability","name":"Provider Command Capability","surface":"integration","status":["product-required"],"authority":"Versioned provider package","summary":"Catalogued command metadata, parameters and risk classification. It is not permission or a send route."},
{"id":"integration.command_intent","name":"Provider Command Intent","surface":"integration","status":["future-concept"],"authority":"Future Provider Command Gateway","summary":"A future explicitly confirmed, scoped and idempotent request to execute a catalogued command; no read collector may create it."}, {"id":"integration.command_intent","name":"Provider Command Intent","surface":"integration","status":["future-concept"],"authority":"Future Provider Command Gateway","summary":"A future explicitly confirmed, scoped and idempotent request to execute a catalogued command; no read collector may create it."},
{"id":"integration.command_audit","name":"Provider Command Audit","surface":"integration","status":["future-concept"],"authority":"Future Provider Command Gateway","summary":"Immutable approval, execution and delivery audit for a red-domain command lifecycle."} {"id":"integration.command_audit","name":"Provider Command Audit","surface":"integration","status":["future-concept"],"authority":"Future Provider Command Gateway","summary":"Immutable approval, execution and delivery audit for a red-domain command lifecycle."}
] ]

View File

@ -2,11 +2,11 @@
"sourceRoots": ["platform/docs", "platform/packages", "platform/services"], "sourceRoots": ["platform/docs", "platform/packages", "platform/services"],
"ledgers": [ "ledgers": [
{ {
"id": "external_provider_data_plane_adr", "id": "l2_owned_external_connectors_adr",
"path": "../../docs/ADR_EXTERNAL_PROVIDER_DATA_PLANE.md", "path": "../../docs/ADR_L2_OWNED_EXTERNAL_CONNECTORS.md",
"entityIds": ["integration.provider", "integration.connection", "integration.access_scope", "integration.capability", "integration.collection_profile", "integration.read_model", "integration.command_capability"] "entityIds": ["integration.provider", "integration.connection", "integration.access_scope", "integration.capability", "integration.collection_profile", "integration.read_model", "integration.command_capability"]
} }
], ],
"baselineDocs": ["../../docs/ADR_EXTERNAL_PROVIDER_DATA_PLANE.md", "../../packages/external-provider-contract/README.md"], "baselineDocs": ["../../docs/ADR_L2_OWNED_EXTERNAL_CONNECTORS.md", "../../packages/external-provider-contract/README.md"],
"restrictions": ["The package is a semantic contract. Provider secrets, runtime data and command transport remain outside Ontology Core."] "restrictions": ["The package is a semantic contract. Provider secrets, runtime data and command transport remain outside Ontology Core."]
} }

View File

@ -1,11 +1,11 @@
{ {
"version": "0.1.0", "version": "1.0.0",
"updatedAt": "2026-07-13", "updatedAt": "2026-07-16",
"rules": [ "rules": [
{"id":"guardrail.integration.ontology_is_not_data_plane","severity":"error","summary":"Ontology Core may describe providers and contracts but must not store secret values, live payloads, tenant telemetry, raw envelopes or provider execution state.","entityIds":["integration.provider","integration.connection","integration.credential_reference","integration.raw_envelope","integration.read_model"]}, {"id":"guardrail.integration.ontology_is_not_data_plane","severity":"error","summary":"Ontology Core may describe providers and contracts but must not store secret values, live payloads, tenant telemetry, raw envelopes or provider execution state.","entityIds":["integration.provider","integration.connection","integration.credential_reference","integration.raw_envelope","integration.read_model"]},
{"id":"guardrail.integration.scope_precedes_collection","severity":"error","summary":"A collection run must resolve tenant scope, field policy and bounded collection profile before it addresses a provider. Account visibility does not define product scope.","entityIds":["integration.connection","integration.access_scope","integration.collection_profile","integration.collection_run"]}, {"id":"guardrail.integration.scope_precedes_collection","severity":"error","summary":"A collection run must resolve tenant/connection, safe capabilities, field policy and bounded profile before calling a provider, then process every valid entity returned for the bound credential. Static entity allowlists and provider-group business filters are forbidden at collection boundary.","entityIds":["integration.connection","integration.access_scope","integration.collection_profile","integration.collection_run"]},
{"id":"guardrail.integration.read_model_not_raw_bypass","severity":"error","summary":"L2 workflows, interfaces and renderer adapters consume scoped read models or realtime channels, never provider credentials, raw envelopes or direct adapter database access.","entityIds":["integration.credential_reference","integration.raw_envelope","integration.read_model","integration.realtime_channel"]}, {"id":"guardrail.integration.read_model_not_raw_bypass","severity":"error","summary":"NDC L2 workflows, interfaces and renderer adapters consume scoped Data Products or realtime channels, never provider credentials, raw envelopes or direct adapter database access.","entityIds":["integration.credential_reference","integration.raw_envelope","integration.read_model","integration.realtime_channel"]},
{"id":"guardrail.integration.commands_are_separate_red_domain","severity":"error","summary":"Cataloguing a command does not activate it. Read collectors, L2 workflows, maps and AI Workspace may not create command intent or delivery. A later command gateway requires explicit human confirmation, scope, idempotency and audit.","entityIds":["integration.command_capability","integration.command_intent","integration.command_audit","integration.collection_run"]}, {"id":"guardrail.integration.commands_are_separate_red_domain","severity":"error","summary":"Cataloguing a command does not activate it. Read collectors, NDC L2 workflows, maps and AI Workspace may not create command intent or delivery. A later command boundary requires explicit human confirmation, scope, idempotency and audit.","entityIds":["integration.command_capability","integration.command_intent","integration.command_audit","integration.collection_run"]},
{"id":"guardrail.integration.full_catalog_not_full_rate","severity":"warning","summary":"Every provider field may be catalogued, but enabling continuous collection requires an explicit collection, retention and rate policy. Do not turn a capability catalog into unbounded account polling.","entityIds":["integration.capability","integration.field_definition","integration.collection_profile","integration.retention_policy"]} {"id":"guardrail.integration.full_catalog_not_full_rate","severity":"warning","summary":"Every provider field may be catalogued, but enabling continuous collection requires an explicit collection, retention and rate policy. Do not turn a capability catalog into unbounded account polling.","entityIds":["integration.capability","integration.field_definition","integration.collection_profile","integration.retention_policy"]}
], ],
"blockedConflations": [ "blockedConflations": [

View File

@ -1,7 +1,7 @@
{ {
"id": "integration", "id": "integration",
"version": "0.1.0", "version": "1.0.0",
"updatedAt": "2026-07-13", "updatedAt": "2026-07-16",
"status": "product-required/source-evidenced", "status": "product-required/source-evidenced",
"summary": "Provider-neutral external integration ontology: tenant-scoped connections, capabilities, collection, controlled storage projections and a separated red command domain." "summary": "Provider-neutral external integration ontology: tenant-scoped connections, capabilities, collection, controlled storage projections and a separated red command domain."
} }

View File

@ -1,13 +1,13 @@
{ {
"version": "0.1.0", "version": "1.0.0",
"updatedAt": "2026-07-13", "updatedAt": "2026-07-16",
"relations": [ "relations": [
{"id":"integration.provider.offers_connection","from":["integration.provider"],"to":["integration.connection"],"status":"product-required","summary":"A provider is connected through one or more tenant-scoped connection instances."}, {"id":"integration.provider.offers_connection","from":["integration.provider"],"to":["integration.connection"],"status":"product-required","summary":"A provider is connected through one or more tenant-scoped connection instances."},
{"id":"integration.connection.references_credential","from":["integration.connection"],"to":["integration.credential_reference"],"status":"product-required","summary":"A connection uses an opaque server-side credential reference rather than exposing a secret."}, {"id":"integration.connection.references_credential","from":["integration.connection"],"to":["integration.credential_reference"],"status":"product-required","summary":"A connection uses an opaque server-side credential reference rather than exposing a secret."},
{"id":"integration.connection.enforces_scope","from":["integration.connection"],"to":["integration.access_scope"],"status":"product-required","summary":"Every source request is bounded by an approved product scope."}, {"id":"integration.connection.enforces_scope","from":["integration.connection"],"to":["integration.access_scope"],"status":"product-required","summary":"Every source request is bounded by selected safe capabilities and field policy while entity membership is dynamically all valid objects visible to the bound credential."},
{"id":"integration.provider.declares_capability","from":["integration.provider"],"to":["integration.capability","integration.command_capability"],"status":"product-required","summary":"Adapter manifests catalogue source read and command capabilities independently from their activation."}, {"id":"integration.provider.declares_capability","from":["integration.provider"],"to":["integration.capability","integration.command_capability"],"status":"product-required","summary":"Adapter manifests catalogue source read and command capabilities independently from their activation."},
{"id":"integration.capability.exposes_field","from":["integration.capability"],"to":["integration.field_definition"],"status":"product-required","summary":"A safe-read capability documents the fields it may return before a collection profile enables them."}, {"id":"integration.capability.exposes_field","from":["integration.capability"],"to":["integration.field_definition"],"status":"product-required","summary":"A safe-read capability documents the fields it may return before a collection profile enables them."},
{"id":"integration.collection_profile.selects_capability","from":["integration.collection_profile"],"to":["integration.capability","integration.field_definition"],"status":"product-required","summary":"A profile selects bounded capabilities and fields rather than polling every API feature continuously."}, {"id":"integration.collection_profile.selects_capability","from":["integration.collection_profile"],"to":["integration.capability","integration.field_definition"],"status":"product-required","summary":"A profile selects bounded safe capabilities and fields rather than polling every API feature; it does not select business entity IDs."},
{"id":"integration.collection_profile.applies_retention","from":["integration.collection_profile"],"to":["integration.retention_policy"],"status":"product-required","summary":"Collection and storage lifecycle are explicit per connection profile."}, {"id":"integration.collection_profile.applies_retention","from":["integration.collection_profile"],"to":["integration.retention_policy"],"status":"product-required","summary":"Collection and storage lifecycle are explicit per connection profile."},
{"id":"integration.collection_run.uses_connection","from":["integration.collection_run"],"to":["integration.connection","integration.collection_profile"],"status":"product-required","summary":"A collection run records which connection and bounded policy it used."}, {"id":"integration.collection_run.uses_connection","from":["integration.collection_run"],"to":["integration.connection","integration.collection_profile"],"status":"product-required","summary":"A collection run records which connection and bounded policy it used."},
{"id":"integration.collection_run.records_raw_envelope","from":["integration.collection_run"],"to":["integration.raw_envelope"],"status":"product-required","summary":"Safe-read response provenance is retained only according to the active policy."}, {"id":"integration.collection_run.records_raw_envelope","from":["integration.collection_run"],"to":["integration.raw_envelope"],"status":"product-required","summary":"Safe-read response provenance is retained only according to the active policy."},

View File

@ -2,108 +2,202 @@
Package: `catalog/domain-packages/gelios` Package: `catalog/domain-packages/gelios`
Status: `v0.1.0` source-evidenced and product-required slice for the Robot2B client context and the Gelios provider. Status: `v1.0.0`, source-evidenced and product-required.
## Purpose and boundary ## Purpose and boundary
The package describes the meaning of Gelios fleet, telemetry, sensor, spatial, report and command concepts. It is not a Gelios client, token store, telemetry database, Cesium implementation or Engine workflow. The package describes Gelios fleet, telemetry, sensor, spatial, report and
command meanings. It is not a provider client, credential store, runtime
database, renderer implementation, NDC L1 agent or NDC L2 workflow.
The practical audit established a wide REST read surface: 107 currently visible units, 105 with `lastMsg`, a visible administrative account, unit groups, telemetry variants, sensor catalogues, command-template catalogues, geozone groups and report templates. The REST OpenAPI documents 295 operations. This ontology therefore separates three things that must not be conflated: Gelios is the first live provider used to prove the common provider-package
model. Runtime transport, collection cadence and semantic mapping belong to an
isolated NDC L2 connection instance. External Data Plane persists and delivers
provider-neutral Data Products. Ontology Core describes meaning only.
1. documented provider capability; The matching contract/data artifact is
2. actual runtime read evidence for the current account; `platform/packages/external-provider-contract/providers/gelios/v1`. Adding a
3. approved Robot2B product scope. second account creates another connection instance with different opaque
credential references; it does not create another ontology package, Platform
service or custom node.
The 95-unit legacy Engine snapshot is a temporary compatibility reference only. It is not the final business scope. The provider's current groups also do not resolve that scope automatically. A Robot2B owner must approve an allowlist or equivalent rule represented by `gelios.access_scope`. ## Provider authentication boundary
Gelios issues exactly two provider secret artifacts: an access token and a
refresh token. The current NDC L2 `httpBearerAuth` request binding uses the
access token. Automatic refresh has not been proven in the deployed runtime, so
the matching provider package records refresh as `operator_managed`; neither
token value belongs in Ontology, a workflow graph, MCP, Ops or a trace.
A credential label such as `read access` is operator metadata, not a Gelios
token scope. `gelios.units.current.read` is classified as read because the
approved workflow transport is `GET /api/v1/units`. The label does not create a
separate read token or constrain other rights that Gelios may have granted to
the same access token. The scoped Data Product writer credential later in the
runtime path is an internal NDC/External Data Plane capability, not a third
Gelios token.
## Dynamic connection scope
`gelios.access_scope` is derived from two inputs on every collection run:
1. safe-read capabilities selected in the connection profile;
2. entities currently visible to the bound provider credential.
For `gelios.units.current.read`, NDC L2 processes every valid unit returned by
`GET /api/v1/units`. Static unit IDs, owner allowlists and provider-group business
filters are forbidden at collection boundary. New units visible to the same
credential appear automatically. Foundry and other Data Product consumers own
layer selection, sorting, filtering and visibility.
Technical limits such as a 5000-fact batch ceiling, retry policy and response
validation protect runtime health; they are not business entity filters.
## Canonical value contracts ## Canonical value contracts
### Unit and device ### Unit and device
`gelios.unit` is the stable business subject. The durable key is a namespaced provider identity such as `gelios.unit:<sourceUnitId>`, never a renderer entity ID or a display name. A unit can use a `gelios.tracker_device`, belong to multiple `gelios.unit_group` records and expose sensor, fuel, maintenance and custom-field configurations. `gelios.unit` is a stable provider subject. The published v1 source ID is
`gelios-unit-<sourceUnitId>` inside immutable `(providerId, connectionId)`
scope; a renderer entity ID or display name is never identity. The provider ID
suffix is preserved as an exact string (including leading zeros) and is never
parsed or renumbered. A unit can use a
`gelios.tracker_device`, belong to multiple `gelios.unit_group` records and
expose sensor, fuel, maintenance and custom-field configurations.
Hardware IDs, IMEI, phones and decrypt-related fields are restricted or secret-class data. They are not part of the default Studio contract. Hardware IDs, IMEI, phones, address, decrypt-related fields, raw `params` and
unclassified sensor payloads are excluded by the v1 field policy. New provider
fields remain dropped until they are evidenced, classified and introduced by a
new package/ontology revision.
### Current telemetry ### Current telemetry and position
`gelios.telemetry_snapshot` is the normalized current state produced by the Gateway, with at least: `gelios.telemetry_snapshot` is normalized by the versioned NDC L2 mapping.
`gelios.position_fix` is its time-qualified spatial portion; it is not a pin or
other renderer object.
The first approved output is exactly:
```text ```text
unitSubjectId Data Product: fleet.positions.current.v1
observedAt version: 1.0.0
receivedAt ontology revision: ontology.map.moving_object.v1
position: { latitude, longitude, height?, course?, speed?, satellites? } semantic type: map.moving_object
operationalStatus delivery: snapshot+patch
sensorReadings[] history: sampled, latest entity per 60-second bucket, 90 days
counterValues[]
qualityFlags[]
source: gelios-rest
``` ```
The source supports a raw and a pure last-message variant. Pure data may include sensors, counters and accumulated fields. Raw messages and dynamic `params` remain behind restricted policy and may be retained separately for diagnostics only. `gelios.position_fix` is the time-qualified position portion used by spatial consumers; it is not itself a pin. Allowed Data Product fields are:
```text
course_degrees
display_name
elevation_meters
geometry
hdop
horizontal_accuracy_meters
object_kind
operational_status
position_source
position_valid
quality_flags
satellite_count
speed_kph
```
All attribute names are snake_case. GeoJSON `geometry` is omitted when valid
coordinates are unavailable, but the unit remains in the product with
`position_valid=false`, `operational_status=no_position` and appropriate
`quality_flags`. If the provider has no last-message timestamp, the collection
receive time becomes the explicit `observedAt` fallback so the credential-visible
unit is not silently dropped.
### Sensors and operational semantics ### Sensors and operational semantics
`gelios.sensor_definition`, `gelios.sensor_conversion` and `gelios.sensor_reading` separate stable device configuration, calibration and time-series readings. Fuel profile, maintenance plan and custom field are independent configuration concepts. No assumed sensor meaning should be added to analytics until its message parameter, measure and conversion semantics are approved. `gelios.sensor_definition`, `gelios.sensor_conversion` and
`gelios.sensor_reading` separate device configuration, calibration and
time-series values. Fuel profile, maintenance plan and custom field are
independent concepts. A sensor meaning cannot enter a Data Product until its
message parameter, unit, conversion, classification and field policy are
approved.
### Spatial binding and Cesium pins ### Spatial binding
Gelios is a source domain; Cesium is a renderer adapter. The binding is deliberately provider-neutral: Gelios is a source domain. The provider-neutral relation is:
```text ```text
gelios.unit gelios.unit + gelios.position_fix
-> map.moving_object (stable spatial subject) -> map.moving_object
gelios.position_fix -> fleet.positions.current.v1
-> map.moving_object (current position update) -> Foundry Data Product binding
map.moving_object -> map layers, selection and telemetry panel
-> map.pin + map.label + map.visibility_rule
-> map.renderer_adapter (Cesium today, replaceable later)
``` ```
The future map payload selects a stable `subjectId`, position, normalized operational status, observed time and approved label fields. It must not contain Gelios credentials, raw payloads or Cesium transient entity references. This is the replacement boundary for the current M-map node-to-Cesium coupling. The publish boundary contains stable facts only. It cannot contain provider
credentials, tenant/connection claims supplied by the workflow, raw payloads,
The Map Page resolves the selected Studio profile to the shared map.pin contract. The initial accepted presentation is an elevated spike — ground anchor, stem, outlined head and label — but that visual configuration remains on the Studio side. Gelios publishes data for a stable subject; it never publishes Cesium primitive settings, pin pixels or a renderer entity ID. pixel values, styles or transient renderer identities. The synthetic contract
fixture is
The proposed renderer-neutral payload is in [`examples/gelios-map-moving-object.fixture.json`](../examples/gelios-map-moving-object.fixture.json). It uses synthetic coordinates only: it specifies the interface between Gateway and Map View, not a live trike position and not a Cesium entity. [`examples/gelios-map-moving-object.fixture.json`](../examples/gelios-map-moving-object.fixture.json).
### Commands ### Commands
`gelios.command_template` and `gelios.command_group` are read-only catalogue concepts. `gelios.command_dispatch`, `gelios.command_delivery` and `gelios.command_audit` are a separate red domain. `gelios.command_template` and `gelios.command_group` are read-only catalogue
concepts. `gelios.command_dispatch`, `gelios.command_delivery` and
`gelios.command_audit` are a separate red domain.
No collection run, workflow, map click or autonomous agent may create a dispatch. A future dispatch requires an explicit human action, approved scope, selected unit/group and template/parameters, confirmation, an audit record and delivery-state reconciliation. This ontology package does not grant or test a write permission. No collection run, workflow, map click or autonomous agent may create a
dispatch. A future command path requires explicit human intent, confirmation,
separate authorization, immutable audit and delivery reconciliation. This
ontology and the Gelios v1 provider package expose no command transport.
## Future data plane ## Canonical runtime flow
The intended flow is:
```text ```text
Gelios REST OAuth Gelios safe-read API
-> server-side Gelios Gateway -> opaque Gelios access credential in NDC L2 Credentials
-> scope filter + field policy + normalizer -> isolated NDC L2 connection workflow
-> durable telemetry/configuration storage -> versioned semantic mapping
-> NDC level-2 workflow / internal data contract -> NDC Data Product Publish through a native scoped writer credential
-> Map View binding / Cesium renderer adapter -> External Data Plane
-> scoped snapshot + patch stream
-> Foundry Data Product binding
``` ```
`gelios.collection_run` and `gelios.ingestion_cursor` define collection governance independently of the transport implementation. They support bounded incremental history ingestion later: one approved unit scope, limited time window, field whitelist, volume limit and durable checkpoint. They do not imply that all history should be collected now. There is no Gelios-specific Platform service or database in this model.
Provider transport never enters External Data Plane or Foundry. `gelios.collection_run`
Database technology is intentionally not chosen by Ontology Core. The proposed platform data-plane boundary and its validation gates are documented in [`platform/docs/ADR_GELIOS_DATA_PLANE.md`](../../../docs/ADR_GELIOS_DATA_PLANE.md). The ontology supplies the record boundaries needed by that ADR. and `gelios.ingestion_cursor` describe audit and resumability independently of
physical storage.
## Guardrails ## Guardrails
- Ontology Core never stores secrets, runtime snapshots, full raw telemetry or large geometry datasets. - Ontology Core never stores credentials, account state, runtime snapshots,
- Current provider-account visibility is not the approved Robot2B scope; collection must enforce an owner-approved scope before every run. full raw telemetry or bulk geometry.
- GET alone does not make a route safe: login-as, temporary tokens, configuration and WLN download are excluded from telemetry collection. - Gelios authentication has exactly access and refresh artifacts; credential
- Geozones require paging, filtering, volume controls and map LOD. labels and workflow classifications must not be reinterpreted as additional
- Command send and mutation routes remain red even if the account has documented access. provider token types or scopes.
- Map selection and pins target stable domain subjects, not provider payload IDs or renderer objects. - Every safe-read collection includes all valid entities returned for the
bound credential; unit allowlists and group filters are not collection
scope.
- A read method alone does not make a capability safe: impersonation,
temporary credential issuance, configuration mutation and downloads remain
excluded until separately classified.
- Dynamic provider fields are dropped until evidenced and classified.
- Geozones and history require bounded loading, paging/cursors, retention and
volume controls.
- Command send and mutation capabilities remain red even when provider access
exists.
- Foundry targets stable domain subjects and Data Product fields, not provider
payloads or renderer identities.
## Evidence and next implementation decisions ## Evidence and evolution
Evidence is limited to official REST OpenAPI, safe GET results, historical Engine donor inspection and the MAP package. The next decisions outside this ontology package are: Evidence is limited to official Gelios API documentation, safe-read results,
historical NDC L2 donor inspection and provider-neutral map contracts. The
Robot2B pilot counts (107 credential-visible units and a 95-unit legacy
snapshot) are retained only as historical evidence of response shape and
cardinality. They do not define canonical IDs, collection allowlists or product
scope.
1. owner-approved Robot2B allowlist/rule; Extension order is mandatory: establish safe provider evidence, extend
2. field policy for Studio, analytics and raw retention; capability/field classification, update ontology and mapping revision, add
3. data-plane ADR for realtime telemetry and history; synthetic fixtures and contract tests, then deploy an updated NDC L2 workflow.
4. Gelios Gateway implementation boundary and deployment location;
5. fixture and renderer-adapter contract for Map View pins and labels.

View File

@ -1,25 +1,33 @@
{ {
"contractVersion": "0.1.0", "schemaVersion": "nodedc.data-product.publish/v1",
"kind": "map.moving_object.current_position", "batch": {
"source": "gelios", "runId": "gelios-ontology-fixture-20260716",
"subjectId": "gelios.unit:example-unit-id", "sequence": 0,
"observedAt": "2026-07-13T00:00:00.000Z", "idempotencyKey": "gelios-ontology-fixture-20260716.batch-0"
"receivedAt": "2026-07-13T00:00:03.000Z",
"position": {
"latitude": 55.000001,
"longitude": 37.000001,
"heightMeters": 0,
"courseDegrees": 90,
"speedKph": 0,
"satellites": 0
}, },
"operationalStatus": "unknown", "facts": [
"display": { {
"label": "Robot2B unit", "sourceId": "gelios-unit-1001",
"visibilityProfile": "fleet-default" "semanticType": "map.moving_object",
"observedAt": "2026-07-16T12:00:00.000Z",
"geometry": {
"type": "Point",
"coordinates": [37.6173, 55.7558]
}, },
"policy": { "attributes": {
"scopeApproved": true, "course_degrees": 90,
"fieldSet": "studio-current-position-v1" "display_name": "Synthetic Gelios unit",
"elevation_meters": 0,
"hdop": 0.8,
"horizontal_accuracy_meters": 4.2,
"object_kind": "tracked_unit",
"operational_status": "active",
"position_source": "gelios",
"position_valid": true,
"quality_flags": [],
"satellite_count": 11,
"speed_kph": 0
} }
}
]
} }