diff --git a/README.md b/README.md index 17d069c..1940656 100644 --- a/README.md +++ b/README.md @@ -103,6 +103,21 @@ uv run k1link doctor uv run pytest ``` +## External perception worker + +Mission Core can now package one sealed camera epoch as an immutable, +content-addressed `missioncore.compute-job/v1` without changing the source +evidence. The first recorded TEST007 job was digest-verified and decoded on a +separate Windows RTX 4090 worker, executed through pinned Triton/YOLOX-S and +returned a content-addressed `missioncore.compute-result/v1` containing 56 +session-timestamped generic COCO detections. The worker never connects to K1 and +does not own the raw archive. + +This is a recorded execution acceptance, not a navigation stack: Rerun result +projection, large-epoch performance, direct worker transport and bounded live +fan-out remain open. See [ADR 0014](docs/adr/0014-bounded-external-perception-worker.md) +and the [external worker contract](docs/10_EXTERNAL_PERCEPTION_WORKER.md). + ## Mission Core Control Station and visualization adapters The browser application is the universal Mission Core Control Station rather than diff --git a/docs/01_IMPLEMENTATION_PLAN.md b/docs/01_IMPLEMENTATION_PLAN.md index c21d9ef..615ab3d 100644 --- a/docs/01_IMPLEMENTATION_PLAN.md +++ b/docs/01_IMPLEMENTATION_PLAN.md @@ -298,6 +298,37 @@ confirms it. MVP GO: K1-to-Mac scan-correlated data can be captured reliably and at least one useful stream is decoded or structurally identified. +## Stage 8 — replaceable external perception worker + +Stages 6–7 established the raw-first live bridge and durable combined session +archive described above. Stage 8 introduces a replaceable GPU executor without +giving it K1 command authority or the authoritative archive. + +Accepted recorded gate, 2026-07-19: + +- the Mac validates one sealed camera epoch through the same digest and + ISO-BMFF timing inspector used by saved-session replay; +- `k1link compute prepare-camera-job` publishes an immutable + `missioncore.compute-job/v1` below ignored runtime storage; +- the first TEST007 job contains only one bounded 56-frame camera epoch, not the + MQTT archive or another K1 control path; +- Windows revalidates all 59 job files, reconstructs the exact canonical stream + digest and decodes all 56 frames; +- pinned Apache-2.0 YOLOX-S executes through pinned Triton 2.70.0 on the RTX + 4090 and atomically publishes one content-addressed + `missioncore.compute-result/v1`; +- repeating the exact result identity returns the existing result without + increasing Triton inference count. + +The accepted result is perception evidence only. Rerun projection, the larger +TEST007 epoch, direct/routed worker transport and bounded live fan-out remain +separate gates. SSH/SCP is allowed only as the recorded laboratory bootstrap. +The worker never connects to K1. + +See [ADR 0014](adr/0014-bounded-external-perception-worker.md), the +[worker contract](10_EXTERNAL_PERCEPTION_WORKER.md) and +[Lab 005](lab/005_FIRST_RECORDED_PERCEPTION_20260719.redacted.md). + ## Deferred work - ROS2/MCAP bridge; diff --git a/docs/10_EXTERNAL_PERCEPTION_WORKER.md b/docs/10_EXTERNAL_PERCEPTION_WORKER.md new file mode 100644 index 0000000..ab8891c --- /dev/null +++ b/docs/10_EXTERNAL_PERCEPTION_WORKER.md @@ -0,0 +1,85 @@ +# External perception worker contract + +## Boundary + +```text +K1 + -> XGRIDS device plugin / Mission Core Edge + -> authoritative raw session on Mac + -> bounded missioncore.compute-job/v1 + -> replaceable GPU worker / Triton + -> content-addressed missioncore.compute-result/v1 + -> future optional Rerun derived layer +``` + +The GPU worker cannot discover, provision, start or stop a K1. It receives an +observation package and returns derived observations. The Mac archive is never +rewritten when a job is prepared or a result is received. + +## Recorded camera job v1 + +`k1link compute prepare-camera-job` accepts a sealed cataloged session, one +camera source and one physical codec epoch. It performs the canonical camera +checks before and after copying into a private staging directory: + +- summary schema/source/epoch and per-segment durability contract; +- exact ordered JSONL index and its summary SHA-256; +- init SHA-256, every segment length/SHA-256 and complete stream SHA-256; +- continuous ISO-BMFF decode timeline and finite media duration; +- source session-time mapping through the captured host clock origin. + +Publication atomically renames this layout below ignored storage: + +```text +// +├── job.json +└── input/camera//epoch-N/ + ├── summary.json + ├── index.jsonl + ├── init.mp4 + └── segments/*.m4s +``` + +`input_sha256` is SHA-256 over canonical JSON for the complete path-free input +descriptor. `job_id` includes its first 96 bits and validators require the full +digest, so a prefix collision fails closed. + +## Recorded result v1 + +The accepted worker profile verifies every transferred file, reconstructs the +stream outside the job, decodes frames and preserves strictly increasing +best-effort timestamps in session seconds. Its result identity is canonical JSON +covering: + +- job and full input generation; +- pipeline ID/version; +- model ID/version/weight SHA-256; +- score/NMS thresholds, tensor shape, color order and letterbox policy. + +The identity's full SHA-256 names an immutable `result-` directory. +The result manifest binds the detection artifact's length and SHA-256. A repeat +with the same identity validates the existing directory and returns it without +calling Triton. + +## Accepted model profile + +- YOLOX-S official ONNX from upstream release `0.1.1rc0`; +- Apache-2.0; +- SHA-256 + `c5c2d13e59ae883e6af3b45daea64af4833a4951c92d116ec270d9ddbe998063`; +- FP32 input `[1,3,640,640]`, BGR, bilinear top-left letterbox, pad 114; +- FP32 output `[1,8400,85]`, official grid/stride decode, COCO-80; +- Triton 2.70.0 / ONNX Runtime GPU backend in the pinned 26.06 image. + +This generic model is useful for proving the contract and timing path. It is not +accepted for obstacle avoidance, free-space estimation or safety decisions. + +## Open gates + +1. Import the accepted result as an optional session-timestamped Rerun layer. +2. Qualify TEST007's 206-second epoch and record throughput/resource telemetry. +3. Replace SSH/SCP with a reviewed authenticated worker transport. +4. Add bounded live queues, sampling/drop policy and acquisition-isolation + tests. +5. Introduce tracking, segmentation/free-space, calibration and point-cloud + models as separate versioned pipelines. diff --git a/docs/adr/0014-bounded-external-perception-worker.md b/docs/adr/0014-bounded-external-perception-worker.md new file mode 100644 index 0000000..2da5f19 --- /dev/null +++ b/docs/adr/0014-bounded-external-perception-worker.md @@ -0,0 +1,73 @@ +# ADR 0014: Bounded external perception worker + +## Status + +Accepted for recorded laboratory execution on 2026-07-19. Live transport and +behavior integration are not accepted. + +## Context + +Mission Core has a canonical K1 control/archive path and a second workstation +with an RTX 4090. Sending the entire K1 archive to an ad hoc model environment, +letting the GPU host connect to K1 or installing another monolithic Mission Core +would erase the product boundary already established around the device plugin. + +The first useful hypothesis is narrower: can one sealed observation be sent to +a replaceable executor, processed by a standard model server and returned as a +versioned derived observation with enough identity to reproduce and reject +stale results? + +## Decision + +1. Mission Core Edge on the Mac remains the sole K1 command owner and + authoritative raw archive owner. +2. The edge publishes a bounded `missioncore.compute-job/v1`. The first profile + contains one canonical camera codec epoch, its archive index/summary and a + path-free manifest; it never contains the full MQTT capture. +3. The input generation SHA-256 covers session/source identity, media contract, + timeline, ordered file identities and byte counts. The job directory name is + derived from that generation and publication is atomic. +4. Camera packaging uses the saved-session `RecordedMediaInspector` timing and + digest contract. It additionally verifies summary-level index and complete + stream digests before handoff. +5. The Windows worker is an executor, not a second Mission Core. It revalidates + every staged file, decodes a derived copy and cannot talk to K1. +6. NVIDIA Triton is the first replaceable inference runtime. YOLOX-S is the + first generic detector because its official repository/deployment path is + Apache-2.0 and an official ONNX artifact can be pinned by SHA-256. Model + weights stay outside Git. +7. A result identity covers the full input generation, pipeline, model digest + and parameters. `missioncore.compute-result/v1` plus its detection artifact + is atomically published under `result-`. Exact repeats + validate and return the existing generation without inference. +8. Detections are observations only. They cannot emit scanner, motion or mission + commands. +9. SSH/SCP is permitted for the recorded lab bootstrap only. Direct Triton or a + later routed transport must receive its own address, authentication, bounded + queue and failure-policy review before live use. + +## Accepted evidence + +TEST007 left epoch 1 produced job +`recorded-camera-fae25b92ecf645c2c9765dd4`. All 59 transferred files and +4,567,569 package bytes were verified. The worker reconstructed the canonical +4,549,458-byte H.264 stream with SHA-256 +`bceca577f762fdb79c4e8901e5c2a2330fd07d24dc091b1b703c5af4010d8a56` +and decoded 56 frames over 5.497 seconds. + +Pinned YOLOX-S produced the accepted result +`result-5484a72e81192b19f4e1da2a2dcfd10c876c277ff3e96c6d60cbc9d917b9f604`. +It contains 56 generic `person` detections over 54 frames. Mean client-observed +Triton request latency was 15.611 ms and p95 was 23.728 ms. The result is not a +quality acceptance for navigation. + +## Consequences + +- Model/runtime changes can be compared against the same immutable input. +- The worker can later move from the lab LAN to a routed host without changing + K1 ownership. +- Storage temporarily duplicates the bounded encoded stream and preprocessed + frames on the worker; retention remains unimplemented. +- Host-arrival camera timing does not become sensor calibration. +- The next accepted vertical must project this result into the saved Rerun + timeline before live fan-out is introduced. diff --git a/docs/lab/005_FIRST_RECORDED_PERCEPTION_20260719.redacted.md b/docs/lab/005_FIRST_RECORDED_PERCEPTION_20260719.redacted.md new file mode 100644 index 0000000..b0a05dd --- /dev/null +++ b/docs/lab/005_FIRST_RECORDED_PERCEPTION_20260719.redacted.md @@ -0,0 +1,50 @@ +# Lab 005: First recorded external perception acceptance + +Date: 2026-07-19 (Europe/Moscow). + +## Scope + +One sealed TEST007 left-camera epoch was packaged by Mission Core Edge, copied +to the Windows RTX 4090 worker, decoded and submitted to pinned YOLOX-S through +Triton. No K1 command was sent and no raw source file was modified. + +## Evidence identities + +- session: `20260718T201659Z_viewer_live`; +- job: `recorded-camera-fae25b92ecf645c2c9765dd4`; +- input SHA-256: + `fae25b92ecf645c2c9765dd4b4963f23cf8c101f22fd5a2e14a104f0b447140d`; +- 59 transferred files, 4,567,569 bytes; +- canonical stream: 4,549,458 bytes, SHA-256 + `bceca577f762fdb79c4e8901e5c2a2330fd07d24dc091b1b703c5af4010d8a56`; +- decode: H.264 High, 800x600, 56 frames, 5.497 seconds; +- model SHA-256: + `c5c2d13e59ae883e6af3b45daea64af4833a4951c92d116ec270d9ddbe998063`; +- result: + `result-5484a72e81192b19f4e1da2a2dcfd10c876c277ff3e96c6d60cbc9d917b9f604`; +- detections artifact: 23,802 bytes, SHA-256 + `71b291616e87bbdd2b7a295b1e00dac2ff3e60c627e63b72a27ba802c32a3ae0`. + +## Observed result + +All 56 frames were processed. The generic detector returned 56 `person` +detections across 54 frames, with confidence 0.250729–0.530741. Client-observed +Triton request latency was mean 15.611 ms, p50 15.001 ms, p95 23.728 ms and max +31.099 ms. + +Fragment boundaries contain repeated DTS values, but all 56 frames decode and +the persisted best-effort timestamps are strictly increasing. Synchronization +remains host-arrival best effort. + +The first runner attempt completed all inference requests but failed a +post-inference EOF guard before publication. No partial result directory was +published. After correction, the content-addressed result was published. An +identical repeat left Triton's inference count unchanged at 168, proving reuse +of the existing result. + +## Interpretation + +This accepts the external recorded compute boundary, not detector quality or a +vehicle decision loop. The result has not yet been projected into Rerun. SSH/SCP +is retained only as laboratory bootstrap evidence and is not the product data +plane.