feat(observatory): add portable calculation profiles

This commit is contained in:
DCCONSTRUCTIONS
2026-08-31 15:42:56 +03:00
parent d1b75efcea
commit 9beb534108
75 changed files with 24419 additions and 348 deletions
+37 -22
View File
@@ -278,11 +278,13 @@ an integrity or product need justifies a targeted migration.
## Observatory durable recorded queue and Worker dispatch
Selecting a source and a laboratory setup in Observatory is not itself a run.
The public submission contains only `source_session_id`, `setup_id` and an
idempotency key. It cannot supply commands, executable text, filesystem paths,
container images, model identities, resource limits or priority. The server
resolves the allowlisted pair and seals all executable identity into one durable
record:
The exact legacy submission contains only `source_session_id`, `setup_id` and an
idempotency key. A portable submission additionally returns the server-owned
`definition_sha256` and one content-bound `check_sha256` obtained from preflight;
both must be echoed unchanged during submit. Neither request can supply commands,
executable text, filesystem paths, container images, model identities, resource
limits or priority. The server resolves the allowlisted identities and seals all
executable identity into one durable record:
- the current source-catalog snapshot captured at admission, plus immutable source
bundle and source-capability-manifest SHA-256 identities;
@@ -307,22 +309,33 @@ not an unknown model dependency. The binding pins the exact source pack rather
than a volatile whole-catalog digest; the server captures and seals the current
catalog snapshot into each admitted job.
`LAB V1 · EoMT Cityscapes Large 1024 + DDRNet-39` now has a source-independent
portable RunDefinition and lightweight recorded-source capability probe. The
definition seals the exact model/component/resource identities, observation-only
authority and generic `missioncore.recorded-eomt-ddrnet-review/v2` result
contract; it contains no source Session id or label. Compatibility is derived
from the selected Session's real K1 capabilities rather than its name. A
compatible source may therefore report capability `pass` independently from
executor readiness.
Two source-independent definitions are projected by the portable catalog:
That portable foundation is not an executable product path yet. Its executor is
`not-installed`, and there is no accepted server-side definition-SHA/check-SHA
fenced check/submit API, generic v2 result assembler/publisher or deployed Worker
executor. Preflight consequently remains blocked and the UI must not promise or
expose enqueue. The old `missioncore.lab-v1-vegetation-shadow/v1` result is not an
exact/existing result of the generic portable definition, even for its original
source; it remains available only in the immutable legacy LAB catalog.
- `LAB V1 · EoMT Cityscapes Large 1024 + DDRNet-39` seals the exact
model/component/resource identities and the
`missioncore.recorded-eomt-ddrnet-review/v2` result contract;
- `M4.9T5 · TRAVEL TGS · CPU-only, без ML` v2 seals an explicitly empty model
manifest, dynamic source-derived frame counts, causal TGS invariants and the
`missioncore.recorded-tgs-costmap-review/v2` result contract.
Neither portable definition contains a source Session id, label, fixed frame
count or filesystem path. Compatibility is derived independently for every
setup from the selected Session's real K1 capabilities. A compatible source may
therefore report capability `pass` while that setup's executor remains blocked.
Blocked definitions stay projectable and do not prevent an unrelated ready
definition from entering the durable queue allowlist.
The server now owns a definition-SHA/check-SHA fenced portable check/submit
boundary. Submission still fails closed unless that exact definition has a
sealed `ready` executor release and image and is present in the durable queue.
Both repository portable definitions currently declare `not-installed`, so
their preflight remains blocked; no release or image hash is fabricated. The
generic v2 assemblers, verified publisher and Worker transport exist as a
dormant fail-closed implementation, but exact executor installation, central
storage, authentication, tunnel acceptance and end-to-end canaries remain
pending. The old `missioncore.lab-v1-vegetation-shadow/v1` result and the exact
RAVNOVES00 M4.9 result remain only in their existing immutable catalogs;
neither is reclassified as a result of a portable definition.
Recorded work has priority rank `100`. A future live K1 lease has rank `0` and
closes new recorded claims while it is pending or active. Cooperative executors
@@ -342,8 +355,10 @@ foundation. The production app hard-disables the Worker router even when a valid
credential is present until an expiring claim lease and verified result publisher
are accepted. Installation of the exact executors, Worker deployment and wiring
from the real K1 lifecycle to live-lease triggers remain pending. Therefore a
submitted M4.9T5 job may honestly wait in `queued` without implying that Worker
006 can execute it yet; portable LAB V1 cannot currently be submitted at all.
submitted exact legacy M4.9T5 job may honestly wait in `queued` without implying
that Worker 006 can execute it yet. Portable LAB V1 and portable M4.9T5 cannot
currently be submitted because their executor states are `not-installed` and
the authenticated Worker dispatch gate is closed.
Worker telemetry remains secondary observation evidence. It does not replace the
authoritative queue ledger, result validation or common laboratory receipt. K1
@@ -0,0 +1,121 @@
# ADR 0047: Verified portable Observatory result publication
Date: 2026-08-31
Status: accepted as a backend foundation; production validators, transport and
executor wiring remain blocked
## Context
The durable Observatory queue deliberately treats a Worker `succeed` call as a
transport acknowledgement. Its `result_id` and SHA-256 do not prove that an
artifact exists, that it was computed from the admitted source, that it obeys
the selected RunDefinition, or that it has observation-only authority. Publishing
that acknowledgement directly as a LAB session would let incomplete, corrupt or
mislabelled output enter the same catalog as immutable evidence.
The current canonical LAB V1 result is a separate preserved legacy result. It
must not be reinterpreted as a portable v2 result, rewritten with new provenance,
or used as evidence that the portable executor exists.
## Decision
Portable results cross a new backend-only verification boundary implemented in
`k1link.observatory.portable_result_contract` and
`k1link.observatory.portable_result_publisher`. The boundary does not change the
queue, the source-admission service, Session API, setup projection, UI, K1,
Simulation, or the legacy canonical publisher.
A Worker-side assembler must produce one directory whose basename is the
SHA-256 of its canonical `manifest.json`. The manifest schema is
`missioncore.observatory-portable-result-package/v1`; JSON bytes are canonical
UTF-8 with sorted keys and no insignificant whitespace. Its separately hashed
identity binds:
1. queue `job_id`, request identity, execution identity, submission receipt and
claim generation;
2. source Session, catalog snapshot, source bundle, capability manifest and
source-adapter identities;
3. the complete portable RunDefinition identity, including source requirements,
components, models, resource profile, result contract, executor and authority;
4. result id, result schema, result kind and result-contract SHA-256;
5. a canonical, role-sorted artifact list with confined relative paths, media
types, byte lengths and SHA-256 identities;
6. observation-only authority.
Exactly one non-empty `result-document` JSON artifact is required. Package roots,
the manifest, every path component and every artifact are checked without
following symlinks outside the package. The queue's terminal result SHA must equal
the canonical manifest SHA and the package directory name.
Before catalog publication, the server also:
- resolves the exact `(setup_id, definition_sha256)` in the portable registry and
proves every recorded queue field equals the resulting RunDefinition;
- rejects not-installed executors and the reserved legacy
`lab-v1-vegetation-shadow-<sha256>` namespace;
- rechecks the current SessionStore catalog snapshot;
- reads both source-admission documents from
`observatory-portable-source-contracts/<sha256>.json`, verifies their bytes,
schemas, cross-links, adapter, source and authority;
- invokes a validator registered by exact result-contract SHA-256;
- copies the package manifest and all output artifacts into the central immutable
content-addressed artifact store and records its manifest id;
- publishes one idempotent `LabSessionBinding` through `SessionStore`.
There is intentionally no generic “JSON looks plausible” validator. An unknown
result contract fails before artifact-store or SessionStore mutation. A validator
must understand the exact result schema and determine that the result document
and supporting artifacts are accepted evidence.
## Calculation-profile provenance
The publisher does not read a browser selection or infer a profile from a result
name. It requires a server-owned policy bound to the exact definition id, version
and SHA-256. The resulting immutable provenance contains
`missioncore.observatory-calculation-profile/v1` with:
- `setup_id`;
- full display name;
- origin `archived-definition`;
- definition id, version and SHA-256.
It also stores a SHA-256 of that calculation-profile document. A later Session API
projection can therefore read `calculation_profile` from the result's provenance
instead of reporting whichever setup happens to be selected now.
No replay capability is invented. The portable result package is preserved in
the central artifact store, while a result-schema-specific viewer/replay adapter
must be accepted separately before the catalog binding can claim visual replay.
## Current fail-closed blockers
The publisher and focused contract tests are implemented, but the end-to-end
production loop remains unavailable for concrete reasons:
1. neither `recorded-eomt-ddrnet-review-v2` nor
`m49-tgs-portable-review-v2` has an installed exact result-contract validator;
2. the current Worker protocol returns only `result_id` and manifest SHA-256; it
has no accepted package upload/CAS handoff that gives Mission Core the matching
content-addressed directory;
3. portable executor releases that emit this package contract are not installed
and physically accepted on Worker 006;
4. the application has not registered definition-bound calculation-profile
publication policies or wired the publisher into the terminal Worker flow;
5. no result-schema-specific replay-capability adapter has been accepted.
These are explicit blockers. The backend must not fabricate a package, reuse a
legacy result, trust a Worker success receipt, guess model/profile provenance, or
expose an enqueue/result promise to bypass them.
## Consequences
- Queue success and Observatory publication remain distinct evidence states.
- Exact retry is safe: content-addressed artifact publication and the immutable
SessionStore binding are idempotent; a conflicting result id fails closed.
- Source, definition, model, resource-profile and calculation-profile identities
remain available in one portable provenance document.
- Artifact-store writes may leave harmless immutable unreferenced objects if the
final SessionStore transaction detects a conflict; they cannot overwrite an
existing identity.
- The legacy canonical LAB V1 result and its admission logic remain byte-for-byte
outside this publisher.
@@ -0,0 +1,196 @@
# Portable Observatory Worker 006 operational boundary
## Scope and current state
This runbook covers only recorded, observation-only Observatory jobs. It does
not change K1 acquisition/control, Simulation/Gaussian, legacy LAB execution,
navigation or safety authority.
The server queue, renewable claim lease, verified source/result transport and
result publisher exist, but the production Worker API remains deliberately
disabled. All three application gates stay `False` until exact executors are
installed and a complete transport smoke has passed:
- `OBSERVATORY_WORKER_CLAIM_LEASE_READY`;
- `OBSERVATORY_WORKER_VERIFIED_RESULT_PUBLISHER_READY`;
- `OBSERVATORY_WORKER_PRODUCTION_API_ENABLED`.
The canonical central artifact store is already declared in the installed
Mission Core LaunchAgent as:
```text
MISSIONCORE_ARTIFACT_STORE_ROOT=/Volumes/docker/nodedc-mission-core/artifact-store
```
On 2026-08-31 `/Volumes/docker` was not mounted. This is a fail-closed
preflight failure, not permission to create a checkout-local substitute. The
portable Worker server composition now requires `central_status=ready` before
it can be constructed.
The installed `com.nodedc.mission-core.local` LaunchAgent was running from the
M5 Observatory feature worktree while retaining the established Mission Core
data directory in the main checkout. It declares the central artifact-store
root and cache limits, but not the two portable storage roots below. Do not
silently edit or restart this hybrid local service; carry the environment and
working-directory change through one separately reviewed, hash-gated service
transition after the code release is sealed.
Large portable inputs and in-flight results have no Mission Core data-directory
fallback. The server requires both roots through environment-only
configuration:
```text
MISSIONCORE_OBSERVATORY_WORKER_SOURCE_CAS_ROOT=/Volumes/docker/nodedc-mission-core/observatory-worker/source-cas
MISSIONCORE_OBSERVATORY_WORKER_RESULT_STAGING_ROOT=/Volumes/docker/nodedc-mission-core/observatory-worker/result-staging
```
Both directories must already exist, be canonical non-symlink directories,
remain inside `/Volumes/docker/nodedc-mission-core`, and be disjoint from each
other and from `artifact-store`. Mission Core never creates these configured
roots. If `/Volumes/docker` is absent or is only a local directory rather than
a mounted volume, composition fails closed before queue or Worker API exposure.
Provision the directories only through the reviewed server deployment after
the SMB mount preflight succeeds.
Read-only Worker evidence on the same date:
- strict-pinned `ssh -o BatchMode=yes mission-gpu` succeeds;
- Worker host identity remains `DESKTOP-OPJ8J04`;
- Windows OpenSSH `sshd` is running;
- `AllowTcpForwarding` and `GatewayPorts` use OpenSSH defaults: forwarding is
allowed and remote listeners are not exposed beyond loopback;
- Worker loopback port `18080` had no listener.
## Network shape
Mission Core remains the only backend on Mac loopback port `8000`. The Mac
owns one reverse SSH tunnel through the existing strict-pinned `mission-gpu`
alias:
```text
Worker 006 process
-> http://127.0.0.1:18080
-> encrypted SSH reverse forwarding
-> Mac http://127.0.0.1:8000
```
The exact forwarding declaration is:
```text
-R 127.0.0.1:18080:127.0.0.1:8000
```
It neither opens a LAN listener nor sends a credential in process arguments.
`ObservatoryWorkerHttpGateway` independently rejects plaintext HTTP to any
non-loopback host. `worker_tunnel_launchd.py` builds a pure, hashable launchd
plan; `scripts/plan_observatory_worker_tunnel.py` prints that plan and performs
no installation.
## Bearer credential
The server reads one credential from the fixed private data path:
```text
<MISSIONCORE_DATA_DIR>/worker-auth/observatory-worker.token
```
The installed Worker release receives the same secret through its own private,
runner-managed file and passes only its path as
`MISSIONCORE_OBSERVATORY_WORKER_TOKEN_FILE`. The token is ASCII, 32512 bytes,
has no newline, is never stored in Git, an artifact, a plist, an environment
value, a command line or Ops plaintext, and is read with no-follow semantics.
The admitted service runtime is POSIX and requires mode `0600` or narrower;
native Windows ACL handling is intentionally not guessed.
The Worker service accepts only these non-executable settings:
```text
MISSIONCORE_OBSERVATORY_WORKER_BASE_URL=http://127.0.0.1:18080
MISSIONCORE_OBSERVATORY_WORKER_TOKEN_FILE=<absolute-private-path>
MISSIONCORE_OBSERVATORY_WORKER_WORK_ROOT=<absolute-private-D-backed-path>
MISSIONCORE_OBSERVATORY_WORKER_IDLE_POLL_SECONDS=1
MISSIONCORE_OBSERVATORY_WORKER_TRANSPORT_BACKOFF_SECONDS=5
MISSIONCORE_OBSERVATORY_WORKER_MAX_TRANSPORT_FAILURES=12
```
There is no configurable module, command, image or executable entrypoint.
## Install-time executor seam
`compose_installed_observatory_worker_service` is called only by a reviewed
Worker release. That release injects a constructed
`ObservatoryWorkerExecutorRegistry` directly in memory. Before opening the
HTTP gateway, service composition resolves the four-digest identity of every
portable RunDefinition whose executor state is `ready`:
- executor release SHA-256;
- executor image SHA-256;
- model manifest SHA-256;
- resource profile SHA-256.
No ready definitions, an empty registry, or one missing identity stops the
service before its first claim. Blocked candidates cannot be selected through
configuration and are not silently registered.
## Staged deployment and smoke sequence
No step below was applied by this implementation increment.
1. Seal each executor release and installation receipt. Change a portable
RunDefinition to `ready` only when its exact release/image identities and
local adapter admission agree.
2. Mount the existing canonical SMB artifact store. Require Mission Core
artifact status `central_status=ready`; do not initialize a local surrogate.
Through the reviewed server deployment, provision the two disjoint portable
roots above, set both environment values, and verify they resolve inside the
same mounted `/Volumes/docker/nodedc-mission-core` boundary.
3. Provision one bearer credential through the deployment-owned secret path
on both Mac and the admitted POSIX Worker service runtime. Verify file type,
no-link handling and private permissions without printing the value.
4. Generate the reverse-tunnel launchd plan, review its SHA-256 and exact
arguments, then install it through a separate hash-gated local-service
change. Accept only Worker-side `127.0.0.1:18080/api/health` reaching the
canonical Mac service; no second backend is started.
5. Install the exact Worker release. Its fixed entrypoint loads the sealed
RunDefinition registry and its in-memory executor registry, then calls
`compose_installed_observatory_worker_service`. The process must refuse a
missing token, non-loopback plaintext URL, unsafe work root, absent ready
definition or executor coverage gap.
6. Restart the canonical Mission Core process once with CAS and server token
available. Keep the Worker route disabled and verify that K1,
Simulation/Gaussian and legacy LAB surfaces are unchanged.
7. In a separate reviewed source gate, flip the claim-lease, verified-publisher
and production-API flags together. Restart only the canonical port `8000`
service. An authenticated empty claim must return `204`; missing/wrong bearer
and wrong contour identity must remain `401/403`.
8. Submit one short recorded K1 canary for each profile. Require exact source
materialization, lease heartbeat, result-package digest validation, central
artifact publication, immutable calculation-profile provenance and a
reopenable Observatory result.
9. During a bounded recorded canary, start the existing live K1 priority
transition. Require the recorded job to pause/defer and resume only after
live K1 releases the single Worker resource. This test never grants control
or navigation authority.
## Remaining blockers
- canonical SMB artifact store is currently unmounted on the Mac;
- portable source CAS and result-staging directories/environment values are not
provisioned;
- the installed Mission Core LaunchAgent has not received a sealed
working-directory/environment transition for this release;
- the shared bearer credential has not been provisioned;
- the reverse tunnel plan has not been installed or smoked;
- the Worker polling entrypoint has not been packaged into an admitted POSIX
Worker release;
- both exact executor releases/install receipts still need their own seals;
- production flags and the server route remain off by design;
- no end-to-end result has yet crossed Worker 006 -> central store ->
Observatory under this new path.
The Synology root-owned `nodedc-deploy` registry has no
`mission-core-worker` component. Older Mission Core Worker shadow artifacts
explicitly declare that they are outside that registry. Do not route a Windows
Worker install through an unrelated NAS component or weaken the deploy canon;
the durable Worker release needs its own exact reviewed installation transition
and rollback evidence.