# NDC Mission Core local telemetry plane Portable local-only telemetry infrastructure for compute contours. One Docker Compose project, `ndc-mission-core-telemetry`, owns three long-running containers and one bounded bootstrap job: - `ndc-mission-core-mqtt-broker` — Eclipse Mosquitto `2.1.2-alpine`; - `ndc-mission-core-telemetry-timescaledb` — TimescaleDB HA OSS `pg16.14-ts2.28.2-all-oss`; - `ndc-mission-core-telemetry-bootstrap` — one-shot database role/bootstrap job; - `ndc-mission-core-telemetry-normalizer` — Mission Core telemetry normalizer. The Telegraf configuration templates cover Windows and Linux, but the agent runs as a host service rather than inside this Compose project. This preserves access to native Windows performance counters, Docker Desktop and NVIDIA telemetry. The normalizer exposes a read-only normalized telemetry adapter on `127.0.0.1:18030`. Mission Core reads this adapter; the browser never receives MQTT or Timescale credentials, and Timescale is not published on a host port. The normalizer uses a separate `missioncore_normalizer` database role with only `SELECT`, `INSERT`, `UPDATE`, and bounded retention `DELETE` on the telemetry hypertable. Raw telemetry older than 30 days is removed in batches of at most 1000 rows per minute, on a separate connection/thread (2s statement / 250ms lock deadline). Retention never runs inside the MQTT callback. This stays compatible with the Apache-licensed Timescale image without depending on the Timescale License retention scheduler. The 1 GiB database container explicitly sets 128 MiB shared buffers and 64 MiB maintenance memory. The HA image's automatic tuning sees the Docker VM instead of this container limit; do not inherit its multi-GiB defaults. For a memory-constrained local repair, `compose.source.yaml` mounts the reviewed normalizer source read-only into the **existing** image. Set `MISSIONCORE_NORMALIZER_SOURCE` to the absolute `normalizer.py` path, pass both Compose files and the existing private `--env-file` / `--project-directory`, and use `up -d --no-deps --no-build normalizer`. This is an explicit local source override, not a rebuilt portable image. Normal restart/reboot retains the mount. The stack is one deployment contour, not one multi-process container. Keeping broker, normalizer and database in separate containers preserves independent health checks, least-privilege boundaries and rollback while Compose provides one operator lifecycle: ```bash docker compose ps docker compose up -d docker compose down ``` All NODE.DC-owned Docker objects use the lowercase `ndc-` namespace. Docker names are case-sensitive identifiers, so the product prefix is normalized to lowercase while the product name remains NODE.DC in operator-facing copy. This directory does not contain credentials. Copy `.env.example` to `.env`, generate unique passwords and build the ACL/password files before starting the stack. ```bash uv run python prepare.py --initialize --mqtt-bind-address uv run python prepare.py --enroll-agent worker-006 --contour-id worker-006 uv run python prepare.py docker compose up -d --build ``` `--enroll-agent` is generic: repeat it with a new globally unique `agent-id` and the contour it owns. Agent credentials live only in the private `runtime/agents.json` registry with mode `0600`. A normal `prepare.py` run rebuilds the Mosquitto password file from that exact registry and generates one non-wildcard writer ACL per agent, so removed or renamed identities cannot survive in the broker password file by accident. Existing installations can use `--migrate` to import the old Worker 006 variables once without replacing their credential. Build the reviewed Windows installer bundle and export the selected agent's private stdin payload as separate files: ```bash uv run python prepare.py --build-agent-bundle windows \ --output runtime/missioncore-telemetry-agent-windows.zip uv run python prepare.py --export-agent-payload worker-006 \ --node-id DESKTOP-OPJ8J04 \ --output runtime/worker-006.private.json ``` The ZIP is deterministic, content-addressed and contains only the pinned installer, updater, collector and configuration template. It contains no credential. Transfer the ZIP and private payload separately; on the Worker, unpack the ZIP and pass the payload through stdin: ```powershell Get-Content .\worker-006.private.json -Raw | .\Install-NdcMissionCoreTelegraf.ps1 ``` Delete the transferred payload after the service has been accepted. The installer persists the scoped credential only in the ACL-restricted Windows service environment. Expected Docker object names: ```text project: ndc-mission-core-telemetry network: ndc-mission-core-telemetry containers: ndc-mission-core-mqtt-broker ndc-mission-core-telemetry-normalizer ndc-mission-core-telemetry-timescaledb volumes: ndc-mission-core-mqtt-data ndc-mission-core-telemetry-timescale-data ``` `prepare.py` passes passwords to `mosquitto_passwd` through stdin. Secrets are not placed in process arguments or committed files. `--initialize` generates `.env` with mode `0600` and refuses to replace existing credentials. The generated `.env` and `runtime/` directory are ignored by Git. After initial provisioning, the selected compute contour owns the operational network profile in `Система → Вычислительные модули → Настройки контура`. The profile keeps these concerns separate: - the stable broker hostname resolved by the worker, preferably the operator Mac's `.local` name; - the exact private LAN address on which Docker publishes the authenticated listener; - the stable worker hostname used by the trusted SSH bootstrap profile. Mesh access points inside one LAN are not registered as separate broker endpoints. Roaming between them preserves the same hostname-based profile. Moving the kit to a different LAN requires selecting the Mac's new private bind address, applying it to the broker, and then applying the saved endpoint to the worker. Both actions are explicit. The broker update changes only `MISSIONCORE_MQTT_BIND_ADDRESS` in the private `.env`; the worker update changes only the Telegraf service endpoint and interval, preserves its scoped credential, verifies DNS plus TCP, and rolls back on failure. Wildcard `0.0.0.0` and public MQTT endpoints are rejected. The product architecture and topic contract are defined in `docs/adr/0031-local-compute-contour-telemetry-plane.md`. ## Security boundary The current MQTT listener is authenticated and contour-scoped, but intentionally uses plaintext MQTT inside one owner-controlled laboratory LAN. Do not expose port 1883 through a router, cellular WAN, public Wi-Fi, or an Internet-facing host. A remote or shared-network deployment requires a reviewed TLS listener, a private CA distributed to every agent, and credential rotation. Wi-Fi link encryption is not a replacement for MQTT TLS. Each agent credential is bound to one exact `contours//agents//+` prefix. Adding another contour requires `--enroll-agent`; the wildcard contour writer is not permitted. An `agent-id` is globally unique because Mosquitto ACL ownership is username-based. ## Worker agent MQTT outputs use `startup_error_behavior="retry"`, with a 2000-metric buffer and the configured flush interval. A missing broker at agent startup must not end the service. See [Telegraf's startup policy](https://docs.influxdata.com/telegraf/v1/configuration/plugin-options/). If Docker lost a published listener while the saved LAN address is unchanged, the explicit broker Apply action reconciles that listener; a telemetry GET never restarts infrastructure. Node connectivity does not prove a profile is ready. The streaming profile can export one bounded (8 KiB) current observation using `--telemetry-snapshot /telemetry/current.json`, with `/telemetry` mounted to `C:\ProgramData\NDC\MissionCore\telemetry-agent\perception` on Worker. The existing collector reads it with a 5s freshness limit and forwards it over the same MQTT pipeline topic. It overrides the legacy service only when explicitly present; an expired/broken file reports unavailable rather than relabeling a legacy model. The publisher runs independently from heartbeat/inference and never renews a lease, grants GPU ownership, or qualifies real-time. Queue memory includes active input and decoder scratch. Stage timings are unavailable until actually measured. No camera, point cloud, model result, MQTT secret, or motor command goes here. Worker 006 uses the official Windows Telegraf distribution as the host service `NDC Mission Core Telemetry Agent`. Install or update it with: ```powershell .\telegraf\Install-NdcMissionCoreTelegraf.ps1 .\telegraf\Update-NdcMissionCoreTelegraf.ps1 ``` The update path validates the candidate configuration, backs up the active configuration and rolls back if the service does not return to `Running`. MQTT credentials are scoped to the service environment and must not be passed on a command line or stored in the repository. Both install and update configure Windows Service Control Manager recovery for Telegraf: restart after 5 seconds, then 30 seconds, then 60 seconds, with the failure counter reset after one day. The compute-contour worker network transition applies the same policy to already-provisioned agents and fails closed unless the worker confirms it. The same host service reads the existing perception worker's loopback `/health` contract with `Get-NdcMissionCorePipelineTelemetry.ps1` and publishes nine stage-keyed snapshots to the contour's `pipeline` topic. The perception container does not receive broker credentials and no second agent container is introduced. These snapshots expose current durable-worker state and cumulative stage timing. The runner also appends canonical run lifecycle documents plus one bounded start/aggregate-terminal pair per stage to `pipeline-telemetry.jsonl` in the already-mounted persistent publish directory. Stage records retain the first/last frame boundary and aggregate activation count without producing one MQTT row per frame activation. Telegraf's `inputs.tail` owns the saved file offset, keeps at most 1000 undelivered lines in flight and publishes the records through the same QoS 1 pipeline output. The source outbox rotates at 64 MiB into content-addressed `pipeline-telemetry..jsonl` segments. Telegraf tails both the active file and segments. At eight retained segments the writer fails telemetry publication observably instead of deleting evidence which may not yet have been acknowledged. Segment reclamation therefore remains an explicit operator action after normalized storage is verified; inference control flow remains fail-open. The normalizer verifies the topic-bound record and restores the original native document before storage. A broker outage therefore stays inside the existing Telegraf buffer; the perception container receives neither MQTT credentials nor a second network client. Telemetry write failure is reported in `/health` but does not change inference control flow. When the mounted perception runner itself changes, use `Update-NdcMissionCorePerceptionRunner.ps1` with exact predecessor and candidate digests for the runner, its `e15_shadow_runtime.py` companion and its standalone telemetry module. The three files form one deployment unit: updating the runner without the matching runtime is rejected by the digest-gated transaction. It backs up the mounted files, restarts the same container, accepts only a ready health document with stage metrics and a ready native journal transport, and restores the predecessor set on failure. Update the token-stdin launcher separately with `Update-NdcMissionCorePersistentLauncher.ps1`. The launcher update always installs and digest-checks `MissionCoreDockerNames.ps1` in the same transaction because the launcher dot-sources that companion contract before any worker request. Both files are restored (or a previously absent companion is removed) if validation fails. This keeps the operational contour/agent/node identity explicit and reviewable without changing the K1 command sequence. The stack and agent are intentionally not started by repository tests. Provisioning a machine is a separate, explicit operation.