# LAB E16 — Persistent AI worker and unified live perception publication

Date: 2026-07-22

Acceptance time: 2026-07-22 21:23 MSK / 2026-07-22T18:23:49Z

## Purpose

LAB E16 validates the infrastructure between the previously accepted E15 perception
pipeline and one unified Mission Core Rerun scene. This is an infrastructure
acceptance, not a physical K1 perception acceptance. No device, acquisition or camera
session was active during this lab.

The target operator contract is one original-video timeline with independently
switchable derived layers:

- tracked 2D detections with confidence and metric distance when fusion provides it;
- semantic segmentation on the original camera image;
- accepted, calibrated 3D cuboids in the LiDAR/world scene.

## Safety and authority

- The production Mission Core process at `127.0.0.1:8000` was not stopped or restarted.
- UI and backend acceptance used an isolated shadow process at `127.0.0.1:8001`.
- The AI worker ran only on the Windows worker's `D:` drive.
- VPN, Hidemyname, Wi-Fi and host network configuration were not modified.
- Worker authority remained `commands_enabled=false` and
  `navigation_or_safety_accepted=false`.
- The result direction is diagnostic-only and contains no command endpoint.

## Immutable worker inputs

- Runner SHA-256:
  `f2f2889e07c609e3ba53b4504c3f2417f151fbb9920828f4680ea375a5d3010e`.
- Persistent-service orchestrator SHA-256:
  `ed2dc35de423d0d9f8ac0501db283a10081e78b6535ebeb8d521a58ca6988b11`.
- Persistent-run orchestrator SHA-256:
  `1c6af15736396ee0124d4aabe309b7bc24bb1425d88e24295c5675c686fc27bc`.
- Worker package:
  `e15-worker-package-34fb22ce506aab6e3b6a0fd8058c868e24076a8cfc739b4299b2474f2cb92b99`.
- Runner root on the worker:
  `D:\NDC_MISSIONCORE\runtime\derived\e16-runner-f2f2889e07c6-ed2dc35de423-1c6af1573639-34fb22ce506a`.
- Worker package root:
  `D:\NDC_MISSIONCORE\runtime\inputs\e16\worker-packages\e15-worker-package-34fb22ce506aab6e3b6a0fd8058c868e24076a8cfc739b4299b2474f2cb92b99`.
- Container: `mission-core-perception-worker`.
- Pinned base image: `nvcr.io/nvidia/tritonserver:26.06-py3` with the E15 digest.

## Persistent model-loaded worker result

- GPU: NVIDIA GeForce RTX 4090.
- Model load and warm-up: 38.296654 s.
- Health: `ready`.
- Models loaded: true.
- Completed physical runs: 0.
- Failed physical runs: 0.
- D: free after startup: 418,065,694,720 bytes, approximately 389.354 GiB.
- Guarded D: floor: 360 GiB plus a 512 MiB startup reserve.

The service binds its control HTTP endpoint only to `127.0.0.1` inside the worker
container's shared Triton network namespace. It has no exposed LAN control port. A run
request is single-flight, authenticated by the existing shadow token, limited to one
direct child of the guarded D:-backed publication root and reuses already loaded models.

## Result transport contract

The authenticated K1 shadow WebSocket remains the sensor ingress and now has one
bounded worker-to-Mission-Core diagnostic result direction.

- Result schema: `missioncore.live-perception-result-wire/v1`.
- Result queue: latest-wins, capacity 2.
- Maximum result payload: 2 MiB.
- Maximum JSON header: 256 KiB.
- Maximum original JPEG: 1 MiB.
- Segmentation: fixed 800x600 uint8 mask compressed with zlib.
- Maximum objects per frame: 128.
- Payload SHA-256, lengths, geometry, timestamps and authority are validated before
  the frame is admitted to the visualization runtime.
- Partial cuboids, non-finite geometry, invalid JPEG/mask payloads and any authority
  claim are rejected.

## Unified Rerun publication

Accepted live results use the same timeline and stable entities:

- `/perception/camera/image` — original worker-decoded camera frame;
- `/perception/camera/detections` — 2D boxes, track id, class, confidence and distance;
- `/perception/camera/segmentation` — semantic mask;
- `/world/perception/boxes3d` — calibrated accepted cuboids.

The live blueprint switches between the normal 3D-only scene and a single horizontal
original-video plus 3D-world composition. The three AI controls are independent; they
are not mutually exclusive views. When dynamic perception is active, Rerun uses
latest-at presentation so historical cuboids do not stack into a cloud of boxes.

## Validation

- Complete Python test suite: passed.
- Focused live result, WebSocket return, visualization runtime and Rerun bridge tests:
  passed.
- Ruff on the changed Python surface: passed.
- Frontend TypeScript build: passed.
- Frontend unit suite: 144/144 passed.
- Production frontend build: passed.
- Windows PowerShell parser accepted both service and run orchestrators.
- In-container encode/decode probe returned a 600x800 segmentation mask successfully.
- Shadow backend health at `127.0.0.1:8001`: healthy.
- Production backend health at `127.0.0.1:8000`: healthy and uninterrupted.
- Browser QA opened RAVNOVES00, reached Rerun ready state, enabled 2D,
  segmentation and 3D cuboids simultaneously, and reported zero browser errors.

## Disk handling

The earlier 418 MiB temporary recorded-perception RRD used for payload inspection was
removed after validation. It was a regenerable temporary duplicate, not source evidence.
No source recording or user data was deleted.

## Accepted conclusions

1. The models can remain loaded between runs; the previous double-load startup cost is
   removed from subsequent run requests.
2. Mission Core has a bounded, authenticated, non-authoritative return path for live AI
   presentation.
3. Original video, 2D detections, segmentation and 3D cuboids now share one Rerun
   timeline and one operator composition.
4. Layer visibility is independently controlled and no longer modeled as separate
   perception streams.

## Explicitly not accepted yet

- Physical K1 shadow inference: no device session was active.
- End-to-end detector, segmentation and fusion FPS through the new persistent service:
  requires the next physical 20-second K1 gate.
- The new backend code is intentionally not active on production port 8000 yet because
  that process was not restarted. Port 8001 is the accepted shadow build.
- 3D semantic point coloring is not yet returned by the live result frame; E16 covers
  camera segmentation and 3D cuboids.
- The recorded full-session perception RRD remains a large monolith and is not the
  target real-time transport.
- None of the outputs are ground truth or safety/navigation accepted.

## Next gate

1. Connect K1 and establish a normal Mission Core device/acquisition/camera session.
2. Run one 20-second persistent physical shadow request without command authority.
3. Record detector FPS, semantic FPS, fusion fraction, result age, result payload rate,
   GPU telemetry, queue drops and D: free space.
4. Verify the same four entities visually in the live Rerun scene.
5. Add bounded semantic LiDAR point coloring, then repeat as LAB E17.
