# LAB E9 — Concurrent multirate detector/tracker and semantics

Date: 2026-07-21
State: completed; full qualification accepted
Ground truth: no
Navigation/safety acceptance: no

## Purpose and answer

LAB E9 tests the next autonomous-perception requirement after E8:

> Can the RTX worker preserve the approximately 10 Hz detector/tracker path
> while running the existing EoMT semantic model concurrently at a lower rate,
> with independent bounded queues and explicit semantic freshness?

The answer is **yes for the accepted recorded 60-second interval**:

- detector/tracker: 601/601 frames, 0 drops, 9.984702 FPS;
- detector result-age p95: 92.583758 ms;
- semantics: 121/121 scheduled frames, 0 drops, 2.010231 FPS;
- semantic completion-age p95: 250.404840 ms;
- fresh semantic binding: 597/601 detector states (99.3344%);
- detector queue maximum: 1/2;
- semantic queue maximum: 1/1.

This is a concurrent worker measurement, not an offline composition of
precomputed E4 and E5 artifacts. Both YOLOX/Triton and EoMT/PyTorch executed on
the RTX 4090 during the source-paced interval.

## Input identity

- Compute job: `recorded-camera-602ac89026ed12978619801d`
- Session: `RAVNOVES00`
- Source: `sensor.camera.right`
- Calibration slot: `camera_1`
- Calibration content SHA-256:
  `05f3ad9b38b3a4fc95388a8ec83da83c745e217709e51787b3d5aad0969f6fa9`
- Resolution: 800x600
- Full qualification selection: source frames 1,000–1,600 inclusive
- Detector frames scheduled: 601
- Semantic stride: every fifth source frame
- Semantic frames scheduled: 121
- Session timeline: 135.365857292–195.334857292 seconds
- Source span: 59.969 seconds

The immutable valid-FOV mask is
`valid-fov-mask-b4dd8ddf2b87c1d520ee8a0868c4fea062d7c14d1bae73ccabd3abe1f3acbac2`.
Mask SHA-256:
`a40cee06b7c6f69b6a09a11563dcfd237f3de833b1ccd31459e66692e528ba63`.

## Models and runtime

Detector/tracker branch:

- official Megvii YOLOX-S ONNX release `0.1.1rc0`;
- model SHA-256:
  `c5c2d13e59ae883e6af3b45daea64af4833a4951c92d116ec270d9ddbe998063`;
- ONNX Runtime GPU through the existing Triton container;
- accepted E8 detector profile SHA-256:
  `819bdfb0da521d187ec9ce3d3039a0ff75daa0ab473fb3d8e5fa92db9c1ddca1`;
- ByteTrack-style two-stage IoU tracking, unchanged from E5/E8.

Semantic branch:

- `tue-mps/cityscapes_semantic_eomt_large_1024`;
- revision: `8d6b6d1a3f7b50d441afd7d247c2ed10db186e8f`;
- weights: 1,276,175,488 bytes;
- model SHA-256:
  `c265da9a74f58f5c3f4826d23ca4ca78beac0b106cca5842beca61580de5b782`;
- precision: FP16 autocast, batch size one;
- accepted E4 semantic profile SHA-256:
  `ea583966bc3409f5cf563cbf4fad05e366907e67187082eb692aff53d9f5d875`;
- Cityscapes output mapped into the existing Mission Core 0–15 taxonomy.

Worker runtime:

- NVIDIA GeForce RTX 4090;
- Python 3.12.3;
- PyTorch 2.13.0+cu130;
- Transformers 4.57.6;
- NumPy 1.26.4;
- SciPy 1.16.3;
- container: `nvcr.io/nvidia/tritonserver:26.06-py3`.

Accepted producer identities:

- runner SHA-256:
  `89e3e6de6ed3d6c70687927d6b55335399ac5af83f60e41b09b8b1936def390f`;
- orchestrator SHA-256:
  `ea4351d8761fa03c53d434b35ee64d56c12baec6a18d5cdf2ae7c0b7b461c655`;
- accepted multirate profile SHA-256:
  `4c6a68e5be69b2140cd9efbcfdbc76a97d382a1de3822eb10ffcf47f94a2108e`;
- independent validator SHA-256:
  `ba796085856c91918dcb5d3e3b48ccf4989354da40ccdbbe6d936319e0cfedf2`.

## Scheduling contract

```text
recorded source at exact session timestamps
  ├─ detector queue: capacity 2, every frame (~10 Hz)
  │    -> valid-FOV -> YOLOX/Triton -> tracking
  │
  └─ semantic queue: capacity 1, every fifth frame (~2 Hz)
       -> valid-FOV -> EoMT/PyTorch -> semantic mask identity/classes

detector result
  + newest completed semantic result whose source timestamp is not future
  -> semantic status: fresh / stale / unavailable
```

Both queues use latest-wins overflow behavior. A slow semantic inference cannot
block detector admission, raw recording or source timing. Semantic TTL is
750 ms. Before the first semantic result completes, merged detector states are
explicitly `unavailable`; no mask is invented. Afterward, a mask is `fresh`
only while its source age is within TTL.

The paced interval begins only after both models are loaded and warmed. The
orchestrator intentionally performs full model/hash preflight and then launches
an isolated run container, so the laboratory wall time includes two EoMT loads.
A persistent production worker would load once. Startup is reported separately
from frame latency and is not hidden inside the 60-second replay metrics.

## Pilot history and profile freeze

### Pilot 1 — rejected strict budget

Result:
`e9-multirate-perception-4dba9d71ee14e8be6d66f4ae0a18d31c0be7d442039d400a3d8fd5b4396a3af3`.

The first 64-frame pilot required detector result-age p95 at or below 100 ms.
It produced:

- detector: 64/64, 0 drops, 10.041497 FPS;
- detector result-age p95: 121.054151 ms, max 126.765087 ms;
- semantics: 13/13, 0 drops, 2.039679 FPS;
- semantic completion-age p95: 244.136232 ms;
- fresh coverage: 92.1875%;
- queue maxima: detector 1/2, semantic 1/1.

Only the detector p95 check failed. GPU contention was measured directly:
Triton request p95 rose from E8's 18.589 ms to 32.569 ms while the concurrent
EoMT forward pass occupied the GPU. No queue accumulation, drop, stale detector
state or >150 ms result occurred.

The rejected result remains immutable evidence. It was not retroactively
accepted.

### Pilot 2 — accepted multirate budget

The second profile defines a multirate detector p95 ceiling of 150 ms, matching
the already established E8 stale boundary. This is not a safety claim; it is an
explicit near-real-time laboratory budget for the combined workload.

Result:
`e9-multirate-perception-51b279ffa70c9ce98e9bd41498821aa527022effe3bcf5fa0b0fac9c5f31cfae`.

- detector: 64/64, 0 drops, 10.074834 FPS;
- detector result-age p95: 104.053520 ms;
- semantics: 13/13, 0 drops, 2.046451 FPS;
- semantic completion-age p95: 244.872902 ms;
- fresh coverage: 93.75%;
- accepted.

The v2 profile was frozen before the full run. No threshold was changed after
seeing full-run output.

## Full qualification

Published result:
`e9-multirate-perception-161e0601d2a344e0a48c2498b68cefcac340e0afb77e4a45fecfa05bb44e4f2a`.

| Measurement | Detector/tracker | Semantics |
|---|---:|---:|
| Scheduled | 601 | 121 |
| Processed | 601 | 121 |
| Dropped | 0 | 0 |
| Effective rate | 9.984702 FPS | 2.010231 FPS |
| Queue capacity / max | 2 / 1 | 1 / 1 |
| Result/completion age p95 | 92.583758 ms | 250.404840 ms |
| Result/completion age max | 139.250452 ms | 443.920248 ms |

Merged semantic binding:

- fresh: 597 detector states;
- unavailable during startup: 4 detector states;
- stale: 0;
- fresh coverage: 99.3344%.

Detector output remained consistent with E5/E8:

- detections: 4,049;
- unique confirmed track IDs: 167;
- confirmed track observations: 3,320;
- same-class duplicate pairs at IoU >= 0.8: 0.

### Detector critical-path latency

| Stage | Mean | p50 | p95 | Max |
|---|---:|---:|---:|---:|
| Result age from source deadline | 64.529 ms | 61.538 ms | 92.584 ms | 139.250 ms |
| Decode | 31.736 ms | 29.785 ms | 44.604 ms | 62.679 ms |
| Queue wait | 0.175 ms | 0.073 ms | 0.183 ms | 16.178 ms |
| Processing after dequeue | 30.839 ms | 28.704 ms | 46.099 ms | 76.031 ms |
| Triton request | 16.898 ms | 14.462 ms | 30.324 ms | 39.523 ms |
| Tracking | 0.379 ms | 0.286 ms | 0.909 ms | 4.888 ms |

### Semantic latency

| Stage | Mean | p50 | p95 | Max |
|---|---:|---:|---:|---:|
| Completion age from source deadline | 216.777 ms | 212.143 ms | 250.405 ms | 443.920 ms |
| Processing after dequeue | 183.485 ms | 180.182 ms | 199.038 ms | 413.802 ms |
| EoMT forward | 150.739 ms | 149.165 ms | 162.677 ms | 376.747 ms |
| Processor | 12.344 ms | 11.261 ms | 20.817 ms | 47.351 ms |
| Host to device | 7.952 ms | 7.730 ms | 11.735 ms | 15.397 ms |
| Model postprocess | 9.324 ms | 9.145 ms | 13.969 ms | 20.305 ms |
| Queue wait | 0.192 ms | 0.170 ms | 0.277 ms | 1.786 ms |

## GPU and memory

Across 61 one-second samples during the measured interval:

- GPU utilization: 80.21% mean, 93% p95, 94% max;
- board memory used: 13,558.15 MiB mean, 13,568 MiB max;
- power: 200.66 W mean, 210.16 W p95, 225.70 W max;
- temperature: 47.11 C mean, 52 C max;
- EoMT process CUDA peak allocated: 2,107.925 MiB;
- EoMT process CUDA peak reserved: 2,840.0 MiB;
- process peak RSS: 2,007.344 MiB.

Board memory includes Triton and neighboring services. The high 93% p95 GPU
utilization is important: the accepted configuration has limited compute
headroom. Increasing semantic frequency or adding another large model cannot be
assumed safe without a new experiment.

## Disk and service safety

All task-controlled paths were confined to `D:\NDC_MISSIONCORE`. The full run
reserved 4.273 GiB and enforced the 360 GiB free-space floor.

| Disk point | Free bytes | Free GiB |
|---|---:|---:|
| Before full run | 406,969,856,000 | 379.020 |
| After temporary frame extraction | 406,493,011,968 | 378.576 |
| After replay/publication, before temp cleanup | 406,374,842,368 | 378.466 |
| After all E9 cleanup/evidence transfer | 406,836,731,904 | 378.896 |
| Enforced floor | 386,547,056,640 | 360.000 |

The temporary stream and PNG frame set were removed by exact run-owned path.
No unrelated data was removed. YOLOX was restored to its pre-run unloaded
state. Triton and Frigate remained healthy; Ollama remained running.

## Artifacts and integrity

| Artifact | Bytes | SHA-256 |
|---|---:|---|
| `detector-frames.jsonl` | 1,115,551 | `c17a82bc71abe5deb2542f9df257cbe52fa2ae88b1c4ae1986587e207b2243e0` |
| `semantic-frames.jsonl` | 87,225 | `6da2beca271922ebd9491ccef35e833c6d607387bcde816045a298cedb9a9f45` |
| `merged-frames.jsonl` | 689,114 | `c9a6c60df8c646067f988e17fe2a7b1b943200e88c5caaf4dca400c24cd5d5b3` |
| `gpu-telemetry.jsonl` | 13,710 | `7390a32aa924cbe87a5b0beab261de9429712db4794a25d77dfa6b171b33b6da` |
| `run-report.json` | 11,605 | `4be352a59665d1fe71630894a8a9324d5f27dde934650f65cef4305ac610a4e6` |

The semantic mask itself is held in memory in this scheduling gate. Each
semantic row records the exact source frame, completion age, taxonomy counts
and SHA-256 of the 800x600 target mask. Mask images are not synchronously written
and therefore cannot distort the measured critical path.

Independent validation confirmed for the rejected pilot, accepted pilot and
accepted full result:

- content-addressed identity and all artifact hashes;
- exact job/session/source/calibration/model/profile/runner binding;
- detector and semantic queue accounting;
- ordered detector and semantic source timestamps;
- semantic frames occur only on the declared stride;
- merged semantics never come from a future source frame;
- unavailable bindings contain no invented mask data;
- all three results pass the dedicated E9 validator.

Durable local evidence root:

```text
.runtime/compute-experiments/e9/worker-results/
```

Worker full-result root:

```text
D:\NDC_MISSIONCORE\runtime\derived\
  e9-multirate-perception-161e0601d2a344e0a48c2498b68cefcac340e0afb77e4a45fecfa05bb44e4f2a
```

## Accepted scope

- actual concurrent YOLOX/Triton and EoMT/PyTorch execution on the RTX worker;
- detector/tracker near 10 Hz with no drops;
- semantics near 2 Hz with no drops;
- independent bounded latest-wins queues;
- exact semantic source binding, TTL and explicit unavailable state;
- 60-second pacing, latency, GPU, memory and disk evidence;
- D-only task-controlled storage and restored model/service state.

## Not accepted

- live K1 transport or arbitrarily long continuous operation;
- semantic mask delivery into Mission Core/Rerun;
- live LiDAR fusion, metric distance or 3D cuboids in this concurrent run;
- domain accuracy for forest/off-road navigation;
- optimized semantic model, TensorRT semantic deployment or extra GPU headroom;
- navigation or safety use.

## Next gate

The next step is integration rather than another isolated AI benchmark:

1. carry the latest in-memory semantic mask and exact freshness into the E7
   world-state publisher;
2. run existing E6 LiDAR association/cuboids against the live 10 Hz tracking
   branch;
3. show transient 2D segmentation plus 3D objects/range in Rerun without writing
   per-frame PNGs;
4. inject semantic-worker loss and verify tracking/LiDAR continue with explicit
   degraded state;
5. only then replace recorded pacing with direct K1/worker transport.

Ops publication remains pending because this Codex session exposes no direct
`nodedc-ops-agent` tasker tools. No legacy Ops writer was used.
