# LAB E10 — Integrated source-paced perception and world state

Date: 2026-07-22 (Europe/Moscow)
Run timestamp: 2026-07-21T22:30:03.785Z
State: completed; 60-second qualification accepted
Ground truth: no
Navigation/safety acceptance: no

## Purpose and answer

LAB E10 is the first Mission Core experiment that executes the current
perception branches as one source-paced pipeline rather than composing separate
offline outputs:

```text
recorded K1 camera at original timestamps
  -> valid fisheye FOV
  -> YOLOX/Triton detections at ~10 Hz
  -> ByteTrack-style temporal IDs
  + concurrent EoMT semantics at ~2 Hz
  + factory KB4 camera/LiDAR projection
  + strict recorded LiDAR synchronization gate
  -> point-supported distance and oriented 3D cuboids
  -> clearance sectors and transient world state
  -> Mission Core/Rerun recorded-session overlay
```

The answer is **yes for the accepted recorded 60-second interval on the RTX
4090 worker**:

- detector/tracker: 601/601 frames, 0 drops, 9.996120 FPS;
- semantics: 121/121 scheduled frames, 0 drops, 2.012530 FPS;
- fresh semantic coverage: 597/601 states, 99.3344%;
- LiDAR-fused states: 525;
- accepted point-supported cuboids: 760 observations;
- world-state age p95: 76.514918 ms;
- measured replay wall time: 60.123327 seconds for 59.969 source seconds;
- every frozen acceptance check passed.

This closes the integration gap left by E9. Semantic masks used by LiDAR
association were the actual newest in-memory EoMT results from the concurrent
worker. E6 fusion and E7-style world-state calculation were executed inside the
same critical replay loop. Per-frame masks were not synchronously encoded as
PNG during the measurement.

This result does **not** prove direct live K1 transport, long-duration stability,
forest-domain accuracy, target onboard compute performance or safety fitness.

## Immutable input identity

- Compute job: `recorded-camera-602ac89026ed12978619801d`
- Input SHA-256:
  `602ac89026ed12978619801d4edea0cae24b5cc3afabd9f7af2858de6505a20e`
- Session title: `RAVNOVES00`
- Session ID: `20260720T065719Z_viewer_live`
- Camera source: `sensor.camera.right`
- Resolution: 800x600
- Factory calibration slot: `camera_1`
- Calibration content SHA-256:
  `05f3ad9b38b3a4fc95388a8ec83da83c745e217709e51787b3d5aad0969f6fa9`
- Source frame selection: 1,000–1,600 inclusive, 601 frames
- Session timeline: 135.365857292–195.334857292 seconds
- Timeline SHA-256:
  `06e3514228b488137b92c303cbca171154ede00368783e32107ed2c39c41baa6`
- Source span: 59.969 seconds

Valid-FOV identity:

- generation:
  `valid-fov-mask-b4dd8ddf2b87c1d520ee8a0868c4fea062d7c14d1bae73ccabd3abe1f3acbac2`;
- mask SHA-256:
  `a40cee06b7c6f69b6a09a11563dcfd237f3de833b1ccd31459e66692e528ba63`;
- valid pixels: 270,606 of 480,000.

The FOV mask is generated once and content-addressed. Pixels outside the usable
fisheye circle are excluded before model processing; no repeated per-frame mask
discovery occurs.

## LiDAR replay pack

The 60-second LiDAR input is a private immutable replay pack derived from the
accepted E6 synchronization evidence:

- pack ID:
  `e10-lidar-pack-5da0396d32a27f9d1ca537cc2e8a371d386078d6f0dc71737b78620992af9625`;
- 601 camera timeline rows;
- 526 LiDAR rows passing the accepted strict synchronization gate;
- 1,182,292 total points;
- source coordinate system: K1 map frame;
- target camera: `sensor.camera.right` / factory `camera_1`;
- projection model: KB4, 800x600;
- temporal contract: accepted E6 nearest-host-arrival best effort;
- `lidar-pack.npz`: 9,629,449 bytes;
- pack SHA-256:
  `72aa73340b20fcfaa21b330ef5b93b975c14a70e2cd16a57752d9952ff05ad9a`;
- manifest SHA-256:
  `189a56660b69b05d39308b8dfa6c5ad8ed8812e870a218188181787cc6ea00ba`.

The pack is a replay optimization only. It does not change camera/LiDAR
calibration, points, timestamps or the E6 synchronization decision.

## Models and worker runtime

Detector/tracker branch:

- official Megvii YOLOX-S ONNX release `0.1.1rc0`;
- COCO-80 classes;
- model SHA-256:
  `c5c2d13e59ae883e6af3b45daea64af4833a4951c92d116ec270d9ddbe998063`;
- ONNX Runtime GPU through the existing Triton container;
- ByteTrack-style two-stage IoU tracking inherited from E5/E8/E9.

Semantic branch:

- `tue-mps/cityscapes_semantic_eomt_large_1024`;
- revision: `8d6b6d1a3f7b50d441afd7d247c2ed10db186e8f`;
- weights: 1,276,175,488 bytes;
- weights SHA-256:
  `c265da9a74f58f5c3f4826d23ca4ca78beac0b106cca5842beca61580de5b782`;
- batch size one, FP16 autocast;
- Cityscapes output mapped to the existing Mission Core 0–15 taxonomy.

Worker runtime:

- GPU: NVIDIA GeForce RTX 4090;
- container: `nvcr.io/nvidia/tritonserver:26.06-py3`;
- Python 3.12.3;
- PyTorch 2.13.0+cu130;
- Transformers 4.57.6;
- NumPy 1.26.4;
- SciPy 1.16.3.

## Frozen scheduling and fusion configuration

Detector path:

- every source frame, nominal 10 Hz;
- bounded latest-wins queue capacity: 2;
- observed maximum queue depth: 1;
- overflow drops: 0.

Semantic path:

- every fifth source frame, nominal 2 Hz;
- bounded latest-wins queue capacity: 1;
- TTL: 750 ms;
- overflow drops: 0;
- states before the first completed mask are explicitly `unavailable`.

LiDAR association:

- nearest-depth KB4 projection buffer;
- horizontal box inset 5%; top inset 4%; bottom inset 2%;
- semantic support required for person, bicycle, motorcycle and vehicle groups;
- class-dependent spatial clustering radius: 0.9 m for person/two-wheelers,
  1.5 m for vehicles;
- minimum support points: 3 for person/two-wheelers, 5 for vehicles;
- depth-cluster split: max(0.5 m, 6% of depth);
- same-group NMS IoU: 0.55;
- distance history: 5 frames;
- distance innovation gate: max(1.5 m, 25%);
- oriented cuboid dimensions use robust p05–p95 surface extents and explicit
  class-dependent plausibility limits.

World state:

- velocity history: 1.0 second;
- clearance: 72 azimuth sectors;
- considered range: 0.5–30 m;
- obstacle height above estimated ground: 0.2–3.0 m;
- forward clearance half-angle: 15 degrees.

## Frozen acceptance contract

| Check | Threshold | Result |
|---|---:|---:|
| Detector effective rate | >= 9.5 FPS | 9.996120 FPS |
| Detector drop fraction | <= 0.5% | 0% |
| Semantic effective rate | >= 1.8 FPS | 2.012530 FPS |
| Semantic drop fraction | <= 5% | 0% |
| Semantic completion age p95 | <= 400 ms | 179.682194 ms |
| Fresh semantic coverage | >= 90% | 99.3344% |
| World-state age p95 | <= 175 ms | 76.514918 ms |
| LiDAR-fused frames | >= 450 | 525 |
| Accepted cuboid observations | >= 100 | 760 |
| Failures | zero | zero |

All checks were frozen before the qualification output and all passed. The
result remains explicitly `navigation_or_safety_accepted=false`.

## Timing result

### Critical detector/fusion/world-state path

| Stage | Mean | p50 | p95 | Max |
|---|---:|---:|---:|---:|
| World-state age from source deadline | 56.074 ms | 52.017 ms | 76.515 ms | 118.755 ms |
| Camera decode | 28.826 ms | 28.332 ms | 33.181 ms | 51.831 ms |
| Detector inference path | 23.442 ms | 20.298 ms | 36.127 ms | 48.794 ms |
| Detector queue wait | 0.087 ms | 0.066 ms | 0.155 ms | 5.694 ms |
| LiDAR projection | 0.411 ms | 0.358 ms | 0.723 ms | 4.258 ms |
| Semantic/LiDAR association | 1.649 ms | 1.391 ms | 3.858 ms | 12.513 ms |
| Clearance | 0.187 ms | 0.175 ms | 0.394 ms | 0.667 ms |
| World-state serialization/calculation | 0.088 ms | 0.076 ms | 0.226 ms | 0.586 ms |

The critical path stayed below its 175 ms laboratory ceiling without frame
drops. Decode plus detection dominates the 10 Hz path; LiDAR projection,
association, cuboid generation, clearance and world-state computation consume
only a few milliseconds at p95 on this input.

### Semantic branch

| Stage | Mean | p50 | p95 | Max |
|---|---:|---:|---:|---:|
| Completion age from source deadline | 155.067 ms | 150.439 ms | 179.682 ms | 384.761 ms |
| Processing after dequeue | 124.748 ms | 122.595 ms | 138.430 ms | 353.011 ms |
| EoMT forward | 98.849 ms | 96.101 ms | 112.343 ms | 309.115 ms |
| Processor | 9.647 ms | 9.347 ms | 11.958 ms | 31.297 ms |
| Host to device | 5.680 ms | 6.063 ms | 8.184 ms | 11.627 ms |
| Model postprocess | 7.451 ms | 6.703 ms | 11.166 ms | 12.413 ms |
| Valid-FOV fill | 1.927 ms | 1.882 ms | 2.249 ms | 3.075 ms |
| Queue wait | 0.145 ms | 0.155 ms | 0.194 ms | 0.308 ms |

The semantic branch is deliberately asynchronous. A semantic outlier therefore
does not block detector admission or raw recording. The newest completed mask
is bound only to current/future detector states and only while its TTL remains
valid.

## Fusion and world-state output

Fusion states across 601 detector rows:

- `fused`: 525;
- `depth-unavailable-sync-gate`: 75;
- `semantic-unavailable`: 1.

The LiDAR pack exposes 526 strict-sync frames. One of those occurred before the
first semantic result completed, explaining 525 actual fused states without
inventing a mask.

Cuboid association counts:

- accepted point-supported cuboid observations: 760;
- rejected, fewer than five clustered points: 996;
- rejected, no semantic LiDAR support: 574;
- rejected, distance innovation: 172;
- rejected, fewer than three clustered points: 45;
- rejected, implausible cuboid: 14.

The 760 value is a count of accepted per-frame cuboid observations, not 760
unique physical objects. Cuboids bound the visible LiDAR-supported surface. They
must not be interpreted as complete object volume or ground truth.

## GPU, memory and disk

Across 61 one-second telemetry samples:

- GPU utilization: 45.57% mean, 61% p95, 65% max;
- board memory used: 13,119.30 MiB mean, 13,128 MiB max;
- board memory utilization: 19.51% mean, 26% max;
- power: 154.70 W mean, 166.75 W p95, 178.45 W max;
- temperature: 43.87 C mean, 46 C max;
- EoMT process CUDA peak allocated: 2,107.925 MiB;
- EoMT process CUDA peak reserved: 2,840.0 MiB;
- process peak RSS: 2,022.535 MiB.

This shows usable compute headroom on the laboratory RTX 4090 for this frozen
configuration. It says nothing yet about the final onboard GPU/accelerator.
That hardware must receive its own experiment and budgets.

Task-controlled worker storage remained under `D:\NDC_MISSIONCORE`:

| Disk point | Free bytes | Approx. free GiB |
|---|---:|---:|
| Before measured replay | 404,946,915,328 | 377.136 |
| After measured replay | 404,933,578,752 | 377.124 |
| Enforced floor | 386,547,056,640 | 360.000 |

Temporary decoded frames were cleaned from the exact run-owned D path. The
YOLOX model state was restored after the run. Triton, Frigate and Ollama were
left running. Windows C was not used or modified.

## Result artifacts and integrity

Published result:

`e10-integrated-perception-c6e0263b9e18be7e4f447597065b5ff93b8511463e8f453800f98a61f25c37df`

| Artifact | Bytes | SHA-256 |
|---|---:|---|
| `semantic-frames.jsonl` | 55,025 | `16cb53c7df8491b6620d0483f5004953d0aa54e83198bb101ae357c06e34be89` |
| `fusion-frames.jsonl` | 1,418,427 | `1b28baf9dfc6c8a793f60233f5ffd03a3e40f63a4dfac7a97bff2813f5bd967d` |
| `world-state.jsonl` | 1,524,066 | `9bdab9c1749fca98797ab428bd94f3e1b5a80b7977f013f323283ce9559040ee` |
| `transient-perception.npz` | 1,106,484 | `7e0d0761584f4700e55caf31965d53de1c13b4f8a17c7cfbd2279af4c5f4cabd` |
| `gpu-telemetry.jsonl` | 13,704 | `e036cdd35addc96152c3c534e75e249ac25b18b588cecabb30a8729d36d80f3c` |
| `run-report.json` | 13,563 | `a053d629ea1b48e4a039d7277e0753108926e6efa0000971bc89008eac66ec26` |
| `result.json` | immutable manifest | `48ac8865ff629b00e56ca90cb4a3a68b161f06275465a86ae7973808f8472c2b` |

Accepted producer identities:

- runner SHA-256:
  `3f667e291f18bcccd58d0ced391e23e0cca05ba3cdeb0da50b0f2fb814c4ff70`;
- fusion runtime SHA-256:
  `4d4d9d7fcf8f3b86ac84228902df1948aafc2f48643c53f77250e77bf9854d5d`;
- profile SHA-256:
  `42e2a49945e65daf86e7891794e30ef2c5182b71d073c76ae764c03ea2501589`;
- accepted run orchestrator SHA-256:
  `8bc939e9a5e94707cd30bf7d948fed893087c03d3345e31c20c853b237ecde76`.

After acceptance, only the orchestrator's human-readable free-space line was
changed from pipeline output to host output so future variable assignment does
not capture the label. Current orchestrator SHA-256 is
`ac0ee42495839773a2de60090ebcb49eee979dc7ee3b8587662a3c59e19a4d03`.
This post-run correction does not alter or relabel the accepted E10 result.

The local E10 Python sources were subsequently normalized by the repository's
Ruff formatter and passed the same 47-test regression suite. The accepted
worker staging directory remains the immutable source of the producer hashes
above. Current local source hashes for the next run are:

- fusion runtime:
  `042136fffbc6e06ef62c6d223d5272ce48e0136651f430ab1d59729b32c4bd90`;
- LiDAR-pack builder:
  `e75875342f11a0573eb5eef106f10ff9545debf6c012e717c1df281d479b746b`;
- worker runner:
  `9e434a82d370d1ecf079d76cec5c7a4cf955ca631651862eacb8572b94e33c8b`;
- Mission Core validator/renderer:
  `260a03c623bcab15c6dafd63caa36514e9e35a40e0e6bd4d6551eb3db832c058`.

Durable local evidence root:

```text
.runtime/compute-experiments/e10/worker-results/
```

Worker result root:

```text
D:\NDC_MISSIONCORE\runtime\derived\
  e10-integrated-perception-c6e0263b9e18be7e4f447597065b5ff93b8511463e8f453800f98a61f25c37df
```

## Independent validation and regression checks

The independent E10 validator confirmed:

- content-addressed result identity and artifact hashes;
- exact job/session/source/calibration/model/profile/runner binding;
- detector and semantic queue accounting;
- ordered source timestamps and semantic stride;
- semantic binding never comes from a future source frame;
- unavailable states contain no invented perception payload;
- LiDAR pack identity, size and hash;
- every frozen acceptance check.

The standalone E10 fusion runtime was also run against the accepted E6
precomputed E4/E5 inputs. It reproduced E6 exactly: 526 fused frames and 918
accepted cuboid observations. This is the control that the E10 integration did
not silently change the established E6 geometry algorithm.

Targeted automated tests passed: 47 tests across E10, compute-result discovery,
session API, E9, and E6. Direct Rerun rendering produced a valid 58,342,680-byte
`RRF2` stream. The Mission Core perception endpoint returned HTTP 200 with a
58,342,936-byte `RRF2` response.

## Mission Core / Rerun publication

Mission Core now discovers the newest accepted E10 result for the active
recording before falling back to E6 or legacy 2D results. The generated Rerun
stream contains:

- camera frames;
- semantic masks;
- 2D tracked detections;
- raw map-frame point cloud;
- LiDAR support points;
- oriented point-supported 3D cuboids;
- range and world-state evidence.

Visual acceptance was completed in the local Control Station:

1. `Наблюдение` -> `Пространственная сцена`;
2. saved session `RAVNOVES00`;
3. three active sources loaded;
4. original camera and `Сегментация · камера right` played together on the
   common session timeline;
5. real point cloud and trajectory remained visible behind the camera panels;
6. playback time advanced normally, proving the result is not a single image.

The E10 perception overlay covers only its qualified 60-second source interval.
The underlying saved session remains the full 8:55.718 recording.

## Accepted scope

- actual concurrent YOLOX/Triton and EoMT/PyTorch execution;
- actual current in-memory semantic mask consumed by LiDAR association;
- source-paced detector/tracker near 10 Hz and semantics near 2 Hz;
- bounded latest-wins queues with zero measured drops;
- factory KB4 projection using XGRIDS calibration;
- point-supported distance and oriented 3D cuboids;
- clearance and transient world-state publication in the critical loop;
- 60-second latency, GPU, memory and disk evidence;
- playback in Mission Core/Rerun for the recorded session;
- D-only worker storage and restored service/model state.

## Not accepted

- direct K1 MQTT/camera transport into the perception scheduler;
- the full 8:55.718 session or multi-hour continuous operation;
- hardware-clock camera/LiDAR synchronization proof;
- failure behavior after semantic-worker loss;
- forest/off-road model quality or class coverage;
- final onboard compute rate, thermals or power;
- complete-object 3D geometry;
- navigation or safety use.

## Next gates

The next work is aimed at the unmanned-system operating contour, not cosmetic
offline quality:

1. **Semantic-loss negative control.** Kill or stall the semantic branch during
   source-paced replay and prove detector/raw recording continue, freshness
   becomes stale/unavailable, and no false fresh mask is published.
2. **Full RAVNOVES00 stability run.** Build a full-session immutable LiDAR pack,
   run the same frozen pipeline for approximately nine minutes, verify bounded
   memory/queues, zero recording interference and D free-space floor.
3. **Direct K1 transport.** Replace the recorded camera/LiDAR source adapter
   with the existing live K1 ingestion while preserving the accepted scheduler,
   freshness and world-state contracts.
4. **Target onboard worker qualification.** Repeat the same gates on the actual
   deployable compute hardware; optimize models only against explicit rate,
   latency, thermal and power budgets.
5. **Domain quality branch.** In parallel with runtime work, create forest and
   off-road evaluation data for classes, traversability and geometry. This must
   not block proving the live transport and scheduling contour.

Item 1 was subsequently completed and accepted as
`LAB_E10_N1_REPORT_2026-07-22.md`: detector continuity remained 10.010952 FPS
with zero drops, 498 states became explicitly stale, and fusion recorded zero
freshness violations after the controlled semantic stop. The immediate gate is
therefore the full-session run. No model upgrade or UI polish is justified
before long-duration boundedness is measured.
