diff --git a/deploy/telemetry-plane/README.md b/deploy/telemetry-plane/README.md index c598aa5..20551f5 100644 --- a/deploy/telemetry-plane/README.md +++ b/deploy/telemetry-plane/README.md @@ -65,6 +65,24 @@ in process arguments or committed files. `--initialize` generates `.env` with mo `0600` and refuses to replace existing credentials. The generated `.env` and `runtime/` directory are ignored by Git. +After initial provisioning, the selected compute contour owns the operational network +profile in `Система → Вычислительные модули → Настройки контура`. The profile keeps +these concerns separate: + +- the stable broker hostname resolved by the worker, preferably the operator Mac's + `.local` name; +- the exact private LAN address on which Docker publishes the authenticated listener; +- the stable worker hostname used by the trusted SSH bootstrap profile. + +Mesh access points inside one LAN are not registered as separate broker endpoints. +Roaming between them preserves the same hostname-based profile. Moving the kit to a +different LAN requires selecting the Mac's new private bind address, applying it to the +broker, and then applying the saved endpoint to the worker. Both actions are explicit. +The broker update changes only `MISSIONCORE_MQTT_BIND_ADDRESS` in the private `.env`; +the worker update changes only the Telegraf service endpoint and interval, preserves its +scoped credential, verifies DNS plus TCP, and rolls back on failure. Wildcard +`0.0.0.0` and public MQTT endpoints are rejected. + The product architecture and topic contract are defined in `docs/adr/0031-local-compute-contour-telemetry-plane.md`. @@ -101,13 +119,30 @@ The same host service reads the existing perception worker's loopback `/health` contract with `Get-NdcMissionCorePipelineTelemetry.ps1` and publishes nine stage-keyed snapshots to the contour's `pipeline` topic. The perception container does not receive broker credentials and no second agent container is introduced. -These snapshots expose current durable-worker state and cumulative stage timing; -native per-run lifecycle events remain a separate compute contract. +These snapshots expose current durable-worker state and cumulative stage timing. + +The runner also appends canonical run lifecycle documents plus one bounded +start/aggregate-terminal pair per stage to `pipeline-telemetry.jsonl` in the +already-mounted persistent publish directory. Stage records retain the first/last +frame boundary and aggregate activation count without producing one MQTT row per +frame activation. +Telegraf's `inputs.tail` owns the saved file offset, keeps at most 1000 undelivered +lines in flight and publishes the records through the same QoS 1 pipeline output. +The normalizer verifies the topic-bound record and restores the original native +document before storage. A broker outage therefore stays inside the existing +Telegraf buffer; the perception container receives neither MQTT credentials nor a +second network client. Telemetry write failure is reported in `/health` but does not +change inference control flow. When the mounted perception runner itself changes, use `Update-NdcMissionCorePerceptionRunner.ps1` with exact predecessor and candidate -digests. It backs up the mounted runner, restarts the same container, accepts only a -ready health document with stage metrics, and restores the predecessor on failure. +digests for both the runner and its standalone telemetry module. It backs up the +mounted files, restarts the same container, accepts only a ready health document with +stage metrics and a ready native journal transport, and restores the predecessor on +failure. Update the token-stdin launcher separately with +`Update-NdcMissionCorePersistentLauncher.ps1`; this keeps the operational +contour/agent/node identity explicit and reviewable without changing the K1 command +sequence. The stack and agent are intentionally not started by repository tests. Provisioning a machine is a separate, explicit operation. diff --git a/docs/adr/0031-local-compute-contour-telemetry-plane.md b/docs/adr/0031-local-compute-contour-telemetry-plane.md index b175e97..debfca3 100644 --- a/docs/adr/0031-local-compute-contour-telemetry-plane.md +++ b/docs/adr/0031-local-compute-contour-telemetry-plane.md @@ -16,6 +16,13 @@ The first local contour is active: - Mission Core on canonical port `8000` uses this adapter for live System telemetry and reports the source as `agent-mqtt`; - SSH is retained only for explicit profile diagnostics and bootstrap checks. +- the selected contour owns a managed network profile: Worker 006 uses + `desktop-opj8j04.local`, resolves the broker through the operator Mac's stable + `.local` name, and keeps the current private Docker bind address as a separate + reviewed value; +- the existing contour settings can probe DNS/TCP from both Mac and Worker, apply the + listener address to the local broker, and apply the endpoint to the Telegraf Windows + service without exposing or replacing its scoped credential. The existing Worker 006 containers were renamed in place to `ndc-mission-core-triton` and `ndc-mission-core-perception-worker`. Their @@ -60,6 +67,13 @@ would hide part of the host it is meant to observe. The direct SSH/PowerShell probe remains a bounded bootstrap and diagnostic fallback for the existing Worker 006 profile. It is not the target live telemetry transport. +Mesh access points belonging to one LAN are not modeled as separate broker addresses. +Roaming keeps the stable hostname contract. A move to another LAN changes the reviewed +private bind address, not the agent identity or MQTT topic identity. Applying a saved +profile is explicit, verifies DNS and TCP reachability, and rolls back the affected +configuration when verification fails. MQTT without TLS cannot be configured for a +public endpoint or wildcard listener. + ## Why these components - **Telegraf** is the universal host agent. It already supports Windows and Linux host @@ -196,7 +210,12 @@ subscribe to `mission-core/v1/contours/+/agents/+/+`. - Mission Core code stays responsible for product semantics and stable API contracts, not OS-specific metric collection. - Host, container, network and inference telemetry is stored in the normalized path. -- Per-stage LAB processing telemetry has a native producer contract and an admitted - already-connected MQTT sink. It remains unavailable for a durable worker run until - that worker actually injects the sink and publishes the `pipeline` topic; consumers - must not infer stages from aggregate hardware load. +- Per-stage processing telemetry derives from the actual durable-worker spans. It + emits explicit run start/terminal outcomes and one bounded start/aggregate-terminal + pair per stage and run, with exact run, source, package, method, activation count and + first/last frame identity. The inference container appends canonical documents to + its existing D:-backed publish mount. The host Telegraf service tails that journal + with a saved offset and publishes it through its existing authenticated QoS 1 output. +- Broker credentials remain outside the perception container. Journal write failures + are observable but fail open for inference; consumers must not infer stages from + aggregate hardware load or from the periodic snapshot alone.