docs(telemetry): record managed contour operations
This commit is contained in:
@@ -65,6 +65,24 @@ in process arguments or committed files. `--initialize` generates `.env` with mo
|
|||||||
`0600` and refuses to replace existing credentials. The generated `.env` and `runtime/`
|
`0600` and refuses to replace existing credentials. The generated `.env` and `runtime/`
|
||||||
directory are ignored by Git.
|
directory are ignored by Git.
|
||||||
|
|
||||||
|
After initial provisioning, the selected compute contour owns the operational network
|
||||||
|
profile in `Система → Вычислительные модули → Настройки контура`. The profile keeps
|
||||||
|
these concerns separate:
|
||||||
|
|
||||||
|
- the stable broker hostname resolved by the worker, preferably the operator Mac's
|
||||||
|
`.local` name;
|
||||||
|
- the exact private LAN address on which Docker publishes the authenticated listener;
|
||||||
|
- the stable worker hostname used by the trusted SSH bootstrap profile.
|
||||||
|
|
||||||
|
Mesh access points inside one LAN are not registered as separate broker endpoints.
|
||||||
|
Roaming between them preserves the same hostname-based profile. Moving the kit to a
|
||||||
|
different LAN requires selecting the Mac's new private bind address, applying it to the
|
||||||
|
broker, and then applying the saved endpoint to the worker. Both actions are explicit.
|
||||||
|
The broker update changes only `MISSIONCORE_MQTT_BIND_ADDRESS` in the private `.env`;
|
||||||
|
the worker update changes only the Telegraf service endpoint and interval, preserves its
|
||||||
|
scoped credential, verifies DNS plus TCP, and rolls back on failure. Wildcard
|
||||||
|
`0.0.0.0` and public MQTT endpoints are rejected.
|
||||||
|
|
||||||
The product architecture and topic contract are defined in
|
The product architecture and topic contract are defined in
|
||||||
`docs/adr/0031-local-compute-contour-telemetry-plane.md`.
|
`docs/adr/0031-local-compute-contour-telemetry-plane.md`.
|
||||||
|
|
||||||
@@ -101,13 +119,30 @@ The same host service reads the existing perception worker's loopback `/health`
|
|||||||
contract with `Get-NdcMissionCorePipelineTelemetry.ps1` and publishes nine
|
contract with `Get-NdcMissionCorePipelineTelemetry.ps1` and publishes nine
|
||||||
stage-keyed snapshots to the contour's `pipeline` topic. The perception container
|
stage-keyed snapshots to the contour's `pipeline` topic. The perception container
|
||||||
does not receive broker credentials and no second agent container is introduced.
|
does not receive broker credentials and no second agent container is introduced.
|
||||||
These snapshots expose current durable-worker state and cumulative stage timing;
|
These snapshots expose current durable-worker state and cumulative stage timing.
|
||||||
native per-run lifecycle events remain a separate compute contract.
|
|
||||||
|
The runner also appends canonical run lifecycle documents plus one bounded
|
||||||
|
start/aggregate-terminal pair per stage to `pipeline-telemetry.jsonl` in the
|
||||||
|
already-mounted persistent publish directory. Stage records retain the first/last
|
||||||
|
frame boundary and aggregate activation count without producing one MQTT row per
|
||||||
|
frame activation.
|
||||||
|
Telegraf's `inputs.tail` owns the saved file offset, keeps at most 1000 undelivered
|
||||||
|
lines in flight and publishes the records through the same QoS 1 pipeline output.
|
||||||
|
The normalizer verifies the topic-bound record and restores the original native
|
||||||
|
document before storage. A broker outage therefore stays inside the existing
|
||||||
|
Telegraf buffer; the perception container receives neither MQTT credentials nor a
|
||||||
|
second network client. Telemetry write failure is reported in `/health` but does not
|
||||||
|
change inference control flow.
|
||||||
|
|
||||||
When the mounted perception runner itself changes, use
|
When the mounted perception runner itself changes, use
|
||||||
`Update-NdcMissionCorePerceptionRunner.ps1` with exact predecessor and candidate
|
`Update-NdcMissionCorePerceptionRunner.ps1` with exact predecessor and candidate
|
||||||
digests. It backs up the mounted runner, restarts the same container, accepts only a
|
digests for both the runner and its standalone telemetry module. It backs up the
|
||||||
ready health document with stage metrics, and restores the predecessor on failure.
|
mounted files, restarts the same container, accepts only a ready health document with
|
||||||
|
stage metrics and a ready native journal transport, and restores the predecessor on
|
||||||
|
failure. Update the token-stdin launcher separately with
|
||||||
|
`Update-NdcMissionCorePersistentLauncher.ps1`; this keeps the operational
|
||||||
|
contour/agent/node identity explicit and reviewable without changing the K1 command
|
||||||
|
sequence.
|
||||||
|
|
||||||
The stack and agent are intentionally not started by repository tests. Provisioning a
|
The stack and agent are intentionally not started by repository tests. Provisioning a
|
||||||
machine is a separate, explicit operation.
|
machine is a separate, explicit operation.
|
||||||
|
|||||||
@@ -16,6 +16,13 @@ The first local contour is active:
|
|||||||
- Mission Core on canonical port `8000` uses this adapter for live System
|
- Mission Core on canonical port `8000` uses this adapter for live System
|
||||||
telemetry and reports the source as `agent-mqtt`;
|
telemetry and reports the source as `agent-mqtt`;
|
||||||
- SSH is retained only for explicit profile diagnostics and bootstrap checks.
|
- SSH is retained only for explicit profile diagnostics and bootstrap checks.
|
||||||
|
- the selected contour owns a managed network profile: Worker 006 uses
|
||||||
|
`desktop-opj8j04.local`, resolves the broker through the operator Mac's stable
|
||||||
|
`.local` name, and keeps the current private Docker bind address as a separate
|
||||||
|
reviewed value;
|
||||||
|
- the existing contour settings can probe DNS/TCP from both Mac and Worker, apply the
|
||||||
|
listener address to the local broker, and apply the endpoint to the Telegraf Windows
|
||||||
|
service without exposing or replacing its scoped credential.
|
||||||
|
|
||||||
The existing Worker 006 containers were renamed in place to
|
The existing Worker 006 containers were renamed in place to
|
||||||
`ndc-mission-core-triton` and `ndc-mission-core-perception-worker`. Their
|
`ndc-mission-core-triton` and `ndc-mission-core-perception-worker`. Their
|
||||||
@@ -60,6 +67,13 @@ would hide part of the host it is meant to observe.
|
|||||||
The direct SSH/PowerShell probe remains a bounded bootstrap and diagnostic fallback for
|
The direct SSH/PowerShell probe remains a bounded bootstrap and diagnostic fallback for
|
||||||
the existing Worker 006 profile. It is not the target live telemetry transport.
|
the existing Worker 006 profile. It is not the target live telemetry transport.
|
||||||
|
|
||||||
|
Mesh access points belonging to one LAN are not modeled as separate broker addresses.
|
||||||
|
Roaming keeps the stable hostname contract. A move to another LAN changes the reviewed
|
||||||
|
private bind address, not the agent identity or MQTT topic identity. Applying a saved
|
||||||
|
profile is explicit, verifies DNS and TCP reachability, and rolls back the affected
|
||||||
|
configuration when verification fails. MQTT without TLS cannot be configured for a
|
||||||
|
public endpoint or wildcard listener.
|
||||||
|
|
||||||
## Why these components
|
## Why these components
|
||||||
|
|
||||||
- **Telegraf** is the universal host agent. It already supports Windows and Linux host
|
- **Telegraf** is the universal host agent. It already supports Windows and Linux host
|
||||||
@@ -196,7 +210,12 @@ subscribe to `mission-core/v1/contours/+/agents/+/+`.
|
|||||||
- Mission Core code stays responsible for product semantics and stable API contracts,
|
- Mission Core code stays responsible for product semantics and stable API contracts,
|
||||||
not OS-specific metric collection.
|
not OS-specific metric collection.
|
||||||
- Host, container, network and inference telemetry is stored in the normalized path.
|
- Host, container, network and inference telemetry is stored in the normalized path.
|
||||||
- Per-stage LAB processing telemetry has a native producer contract and an admitted
|
- Per-stage processing telemetry derives from the actual durable-worker spans. It
|
||||||
already-connected MQTT sink. It remains unavailable for a durable worker run until
|
emits explicit run start/terminal outcomes and one bounded start/aggregate-terminal
|
||||||
that worker actually injects the sink and publishes the `pipeline` topic; consumers
|
pair per stage and run, with exact run, source, package, method, activation count and
|
||||||
must not infer stages from aggregate hardware load.
|
first/last frame identity. The inference container appends canonical documents to
|
||||||
|
its existing D:-backed publish mount. The host Telegraf service tails that journal
|
||||||
|
with a saved offset and publishes it through its existing authenticated QoS 1 output.
|
||||||
|
- Broker credentials remain outside the perception container. Journal write failures
|
||||||
|
are observable but fail open for inference; consumers must not infer stages from
|
||||||
|
aggregate hardware load or from the periodic snapshot alone.
|
||||||
|
|||||||
Reference in New Issue
Block a user