From ada2a55ee6616ffbe322f1ab3538b42ea8cb3c4a Mon Sep 17 00:00:00 2001 From: DCCONSTRUCTIONS Date: Sun, 19 Jul 2026 17:39:28 +0300 Subject: [PATCH] docs(perception): record Rerun projection acceptance --- README.md | 9 +++-- docs/01_IMPLEMENTATION_PLAN.md | 13 ++++--- docs/09_OBSERVATION_SESSIONS.md | 26 +++++++++----- docs/10_EXTERNAL_PERCEPTION_WORKER.md | 34 +++++++++++++++---- ...0014-bounded-external-perception-worker.md | 7 ++-- ...T_RECORDED_PERCEPTION_20260719.redacted.md | 22 ++++++++++-- 6 files changed, 84 insertions(+), 27 deletions(-) diff --git a/README.md b/README.md index 1940656..d05db64 100644 --- a/README.md +++ b/README.md @@ -113,9 +113,12 @@ returned a content-addressed `missioncore.compute-result/v1` containing 56 session-timestamped generic COCO detections. The worker never connects to K1 and does not own the raw archive. -This is a recorded execution acceptance, not a navigation stack: Rerun result -projection, large-epoch performance, direct worker transport and bounded live -fan-out remain open. See [ADR 0014](docs/adr/0014-bounded-external-perception-worker.md) +Mission Core now validates and projects that optional result into the same saved +Rerun recording. Open TEST007 and use **Распознавание** to see the recorded +left-camera frames and boxes; **Облако точек** returns to 3D. This remains a +recorded camera-only acceptance, not a navigation stack: large-epoch +performance, direct worker transport and bounded live fan-out remain open. See +[ADR 0014](docs/adr/0014-bounded-external-perception-worker.md) and the [external worker contract](docs/10_EXTERNAL_PERCEPTION_WORKER.md). ## Mission Core Control Station and visualization adapters diff --git a/docs/01_IMPLEMENTATION_PLAN.md b/docs/01_IMPLEMENTATION_PLAN.md index 615ab3d..2f70b36 100644 --- a/docs/01_IMPLEMENTATION_PLAN.md +++ b/docs/01_IMPLEMENTATION_PLAN.md @@ -318,12 +318,15 @@ Accepted recorded gate, 2026-07-19: 4090 and atomically publishes one content-addressed `missioncore.compute-result/v1`; - repeating the exact result identity returns the existing result without - increasing Triton inference count. + increasing Triton inference count; +- Mission Core revalidates the returned generation, projects the exact camera + frames and boxes onto `session_time`, and exposes the optional native Rerun + view only after its complete RRF2 layer is accepted. -The accepted result is perception evidence only. Rerun projection, the larger -TEST007 epoch, direct/routed worker transport and bounded live fan-out remain -separate gates. SSH/SCP is allowed only as the recorded laboratory bootstrap. -The worker never connects to K1. +The accepted result is perception evidence only. The larger TEST007 epoch, +direct/routed worker transport and bounded live fan-out remain separate gates. +SSH/SCP is allowed only as the recorded laboratory bootstrap. The worker never +connects to K1. See [ADR 0014](adr/0014-bounded-external-perception-worker.md), the [worker contract](10_EXTERNAL_PERCEPTION_WORKER.md) and diff --git a/docs/09_OBSERVATION_SESSIONS.md b/docs/09_OBSERVATION_SESSIONS.md index d604a92..ca90d23 100644 --- a/docs/09_OBSERVATION_SESSIONS.md +++ b/docs/09_OBSERVATION_SESSIONS.md @@ -4,10 +4,11 @@ Status: implemented for native K1 point/pose evidence and host-side camera archival. Valid K1 `ModelingReport` scan time, distance and speed are also materialized as recorded Rerun time series. Recorded spatial playback is exposed in the Mission Core observation workspace. The camera archive/player contract -is implemented and covered by tests, but all retained real K1 sessions predate -canonical camera archival. Physical shared-timeline playback of one selected -camera plus point cloud therefore remains an open acceptance gate; historical -sessions contain no recoverable video. +is implemented and covered by tests. TEST007 physically accepted shared-timeline +playback of the saved cameras and point cloud. It also carries the first optional +external-perception result projected into the same Rerun recording; older +sessions without canonical camera evidence or an admitted compute result remain +fully usable with only their available modalities. ## Operator path @@ -19,8 +20,8 @@ sessions contain no recoverable video. 3. Stop acquisition normally, or allow the local service to recover an unexpected interruption on its next start. Session recording is automatic; the disk action is not required. -4. Open **Сохранённые сессии** in the observation header. The menu shows the - three most recent indexed runs, their date, duration, state, modalities and +4. Open **Сохранённые сессии** in the observation header. The menu shows up to + 100 indexed runs, their date, duration, state, modalities and background preparation state. 5. Choose a replayable run. Opening a replay never performs conversion in the request. A ready recording opens immediately; otherwise the client receives @@ -35,7 +36,11 @@ sessions contain no recoverable video. 7. Use **Воспроизвести / Пауза**, the scrubber, **К началу** and **К концу** on the right half of the bottom timeline. The left half changes point-cloud accumulation. The recorded timeline is zero-based `session_time`. -8. Use the first disk button to save the current workspace layout. The next +8. If a validated derived camera result exists, **Распознавание** appears in the + scene toolbar. It switches the active native Rerun view to camera frames and + boxes; **Облако точек** returns to the 3D view. No button is shown for an + absent or rejected result, and the base recording remains usable. +9. Use the first disk button to save the current workspace layout. The next opening restores display settings and dynamic sensor-source window positions. The transient **Движок / Слои / Отображение** tool windows are scoped to the mounted spatial workspace and are never persisted. This action @@ -137,11 +142,13 @@ checkout: │ │ ├── mqtt.timeline.session-.json # sealed generation │ │ └── mqtt.summary.json # atomic active-envelope pointer │ └── media/ # canonical camera archives -└── recordings/ +├── recordings/ ├── .export.lock # cross-process conversion/eviction lock └── / ├── scene.rrd └── scene.rrd.cache.json +└── perception-overlays/ + └── //.rrd ``` `MISSIONCORE_DATA_DIR` relocates the catalog and derived cache. Unless it is @@ -177,6 +184,9 @@ camera epoch there by preserving its valid segment prefix and writing an Never add `.runtime/`, `sessions/`, raw captures, RRD files or camera media to Git. They can contain mapped interiors, trajectories and identifiable images. +The recorded perception importer reads repo-local ignored +`.runtime/compute-jobs` and `.runtime/compute-results`; those derived handoff +artifacts are likewise private and never become browser paths or Git inputs. ## Session source of record diff --git a/docs/10_EXTERNAL_PERCEPTION_WORKER.md b/docs/10_EXTERNAL_PERCEPTION_WORKER.md index ab8891c..8e7cc7b 100644 --- a/docs/10_EXTERNAL_PERCEPTION_WORKER.md +++ b/docs/10_EXTERNAL_PERCEPTION_WORKER.md @@ -9,7 +9,7 @@ K1 -> bounded missioncore.compute-job/v1 -> replaceable GPU worker / Triton -> content-addressed missioncore.compute-result/v1 - -> future optional Rerun derived layer + -> optional validated Rerun derived layer ``` The GPU worker cannot discover, provision, start or stop a K1. It receives an @@ -61,6 +61,29 @@ The result manifest binds the detection artifact's length and SHA-256. A repeat with the same identity validates the existing directory and returns it without calling Triton. +## Recorded Rerun projection + +Mission Core discovers only results whose validated job names the opened +session. Before projection it revalidates the complete job/result binding, +artifact digests, ordered timestamps and result metrics. It then reconstructs +the exact fMP4 epoch, verifies decoded dimensions/frame count with `ffprobe`, +decodes exactly the admitted number of RGB frames and rejects boxes outside the +image. + +The optional `perception.rrd` endpoint returns either HTTP 204 or one complete, +bounded RRF2 stream for the opened Rerun recording ID. The browser never splits +that stream and never replaces the base recording when the optional layer is +absent or rejected. Accepted frames are logged under `/perception/camera` on +the canonical zero-based `session_time` timeline; `Image` and `Boxes2D` rows +therefore seek with the point-cloud archive. Generated overlays are a private, +content-addressed cache and remain rebuildable from the immutable job/result. + +In **Наблюдение → Пространственная сцена**, a **Распознавание** button appears +only after the layer has been admitted. It switches the active native Rerun view +between the camera/detections and the point cloud. TEST007's accepted fragment +occupies approximately `01:07.351–01:12.848`; Rerun's latest-at query retains +the last admitted frame after that short fragment. + ## Accepted model profile - YOLOX-S official ONNX from upstream release `0.1.1rc0`; @@ -76,10 +99,9 @@ accepted for obstacle avoidance, free-space estimation or safety decisions. ## Open gates -1. Import the accepted result as an optional session-timestamped Rerun layer. -2. Qualify TEST007's 206-second epoch and record throughput/resource telemetry. -3. Replace SSH/SCP with a reviewed authenticated worker transport. -4. Add bounded live queues, sampling/drop policy and acquisition-isolation +1. Qualify TEST007's 206-second epoch and record throughput/resource telemetry. +2. Replace SSH/SCP with a reviewed authenticated worker transport. +3. Add bounded live queues, sampling/drop policy and acquisition-isolation tests. -5. Introduce tracking, segmentation/free-space, calibration and point-cloud +4. Introduce tracking, segmentation/free-space, calibration and point-cloud models as separate versioned pipelines. diff --git a/docs/adr/0014-bounded-external-perception-worker.md b/docs/adr/0014-bounded-external-perception-worker.md index 2da5f19..478b783 100644 --- a/docs/adr/0014-bounded-external-perception-worker.md +++ b/docs/adr/0014-bounded-external-perception-worker.md @@ -45,6 +45,9 @@ stale results? 9. SSH/SCP is permitted for the recorded lab bootstrap only. Direct Triton or a later routed transport must receive its own address, authentication, bounded queue and failure-policy review before live use. +10. Mission Core may project an exact validated result into the matching saved + recording as an optional camera/boxes Rerun layer. The base recording stays + authoritative and usable when the layer is absent or rejected. ## Accepted evidence @@ -69,5 +72,5 @@ quality acceptance for navigation. - Storage temporarily duplicates the bounded encoded stream and preprocessed frames on the worker; retention remains unimplemented. - Host-arrival camera timing does not become sensor calibration. -- The next accepted vertical must project this result into the saved Rerun - timeline before live fan-out is introduced. +- The recorded Rerun projection is accepted; large-epoch qualification and a + bounded authenticated live fan-out remain separate gates. diff --git a/docs/lab/005_FIRST_RECORDED_PERCEPTION_20260719.redacted.md b/docs/lab/005_FIRST_RECORDED_PERCEPTION_20260719.redacted.md index b0a05dd..840aebf 100644 --- a/docs/lab/005_FIRST_RECORDED_PERCEPTION_20260719.redacted.md +++ b/docs/lab/005_FIRST_RECORDED_PERCEPTION_20260719.redacted.md @@ -45,6 +45,22 @@ of the existing result. ## Interpretation This accepts the external recorded compute boundary, not detector quality or a -vehicle decision loop. The result has not yet been projected into Rerun. SSH/SCP -is retained only as laboratory bootstrap evidence and is not the product data -plane. +vehicle decision loop. SSH/SCP is retained only as laboratory bootstrap evidence +and is not the product data plane. + +## Rerun projection acceptance + +Mission Core subsequently revalidated the exact job/result generation, +reconstructed and decoded the 56-frame camera epoch, and serialized one complete +RRF2 overlay (approximately 62.55 MB) for the active TEST007 recording ID. A +second render of the same recording/result identity reused the private cache. +The HTTP endpoint returned the exact declared length, RRF2 magic and +`application/vnd.rerun.rrd` without exposing storage paths. + +The Control Station accepted the overlay on an isolated Rerun channel. The +operator-facing **Распознавание** switch appeared only after acceptance and +selected the native `/perception/camera` Rerun view. Browser QA at the shared +session timeline displayed the recorded left-camera image with a green +`person · 34%` box; **Облако точек** returned to the spatial view. The tested +result is camera-only: no point-cloud model, image/point fusion, tracking, +free-space estimate or live inference was executed.