# LAB E5 — Instance detection and temporal tracking qualification

Date: 2026-07-21
Completed at: 2026-07-21T08:45:46.307Z
State: completed qualification baseline, model output, not ground truth
Ops card: MISSIONCOR-19

## Objective

Establish the first measured temporal object-identity baseline on recorded K1
camera data before attempting LiDAR association and 3D cuboids. The acceptance
target was deliberately bounded:

1. run an upstream, reproducibly pinned detector through the existing Triton
   service on the AI worker;
2. reuse the immutable calibration-bound valid-FOV mask instead of processing
   the black fisheye border;
3. attach persistent IDs with a ByteTrack-style two-stage association loop;
4. produce a watchable 2D tracking video and exact timestamped frame metadata;
5. measure detector, tracking, artifact, GPU, memory and disk behavior;
6. retain all task-controlled worker data under `D:\NDC_MISSIONCORE`, enforce
   the 360 GiB free-space floor and restore the prior Triton model state;
7. record visible failure modes honestly before using these identities for 3D.

LAB E5 is a qualification clip, not a full-session or saved-session production
source. It does not claim tracking ground truth, ID-switch accuracy, instance
masks, 3D cuboids, LiDAR distance, point-cloud labels, real-time operation or
safety fitness.

## Immutable input and clip selection

- UI session alias: `RAVNOVES00`
- Session ID: `20260720T065719Z_viewer_live`
- Compute job: `recorded-camera-602ac89026ed12978619801d`
- Source: `sensor.camera.right`
- Factory calibration slot: `camera_1`
- Input SHA-256:
  `602ac89026ed12978619801d4edea0cae24b5cc3afabd9f7af2858de6505a20e`
- Factory calibration content SHA-256:
  `05f3ad9b38b3a4fc95388a8ec83da83c745e217709e51787b3d5aad0969f6fa9`
- Codec epoch: 1
- Resolution: 800x600
- Full source: 4,489 frames, 35.421857292–484.144857292 session seconds
- Qualification selection: source frames 1,000–1,600 inclusive
- Frames admitted: 601/601
- Qualification timeline: 135.365857292–195.334857292 session seconds
- Qualification media duration: 60.068948 seconds
- Qualification timeline SHA-256:
  `06e3514228b488137b92c303cbca171154ede00368783e32107ed2c39c41baa6`

Frames 1,000–1,600 were selected before the full run. The contiguous interval
contains stationary and changing views, multiple parked vehicles, distant
pedestrians, and a woman with a child/stroller passing close to the camera.
That close pass deliberately stresses scale change, fisheye distortion,
occlusion and identity continuity. It was not selected from the final output to
make the result look favorable.

The valid image circle is the immutable LAB E1 artifact
`valid-fov-mask-b4dd8ddf2b87c1d520ee8a0868c4fea062d7c14d1bae73ccabd3abe1f3acbac2`.
Its mask SHA-256 is
`a40cee06b7c6f69b6a09a11563dcfd237f3de833b1ccd31459e66692e528ba63`.
It admits 270,606 of 480,000 pixels. The mask is loaded once and reused; it is
not recomputed per frame.

## Accepted software configuration

Detector:

- official Megvii YOLOX-S ONNX release `0.1.1rc0`;
- license: Apache-2.0;
- classes: COCO-80, filtered to person, bicycle, car, motorcycle, bus and
  truck;
- model SHA-256:
  `c5c2d13e59ae883e6af3b45daea64af4833a4951c92d116ec270d9ddbe998063`;
- Triton config SHA-256:
  `5795c737a7935a655961b069e8404d336d891f9762fb6dffb93956a076479604`;
- input: FP32 `[1,3,640,640]`, BGR, bilinear top-left letterbox, fill 114;
- output: FP32 `[1,8400,85]`, official grid/stride decode;
- container: `nvcr.io/nvidia/tritonserver:26.06-py3`;
- Triton backend: ONNX Runtime GPU.

Detection and FOV filtering:

- detector minimum score: 0.10;
- high/new-track threshold: 0.25;
- same-class NMS IoU: 0.45;
- same-class containment suppression: 0.80;
- minimum box area: 64 pixels;
- maximum box area: 50% of image;
- minimum valid-FOV box fraction: 0.50;
- box center must lie inside the valid FOV.

Tracking:

- algorithm: `bytetrack-style-two-stage-iou/v1`;
- assignment: SciPy linear-sum assignment;
- primary high-score match IoU: 0.20;
- secondary low-score match IoU: 0.10;
- lost-track buffer: 15 frames;
- confirmation: 2 hits;
- association is class-consistent;
- no appearance ReID and no camera-motion compensation.

Producer identities:

- pipeline: `recorded-yolox-bytetrack-style-qualification/v1`;
- runner SHA-256:
  `adc7c924cb8d01278ccf63c9eeb3c9f810c09d2d1d7fe169e136102ee32c247a`;
- orchestrator SHA-256:
  `8fff9c87e6213883ebde257a4226765d96b16cd4b8a2a5bb492aef728e3e2242`;
- profile SHA-256:
  `1de8a83ef0fe630ca317b70bbe3a8ec497ae7cb5267fabda2ea234ee2cf2e562`;
- Python 3.12.3, NumPy 2.5.1 and SciPy 1.16.3 inside the bounded run
  container.

The profile was frozen after the second visual pilot. No thresholds were tuned
against the 601-frame result.

## Execution topology

The pipeline was sequential at frame level:

```text
recorded fMP4 epoch
  -> exact RGB frame reconstruction
  -> fixed calibration-bound valid-FOV fill
  -> YOLOX-S Triton request
  -> YOLOX decode and class/FOV filtering
  -> same-class NMS plus nested-box suppression
  -> high-score association
  -> low-score association to unmatched tracks
  -> confirm/age/retire tracks
  -> write ID boxes and trails
  -> append exact timestamped detections/tracks
  -> encode overlays as H.264
  -> hash, validate and publish qualification result
```

Batch size was one. Frames were not parallelized. Triton/CUDA may execute
internal work asynchronously, but each source frame completed decode,
inference, association and overlay writing before the next frame was admitted.
NVENC encoded the completed overlay sequence after inference.

The model was absent from Triton's ready set before the run. The orchestrator
loaded it only for LAB E5 and restored the original unloaded state in `finally`.
The pre-existing `mission-core-triton`, `sentinel-frigate` and
`sentinel-ollama` services remained running.

## Pilot history and configuration freeze

The first launch failed closed during preflight because the base Triton image
did not contain Pillow. It performed no inference, loaded no model and
published no result. The correction mounted the already existing D-only
perception environment read-only at `/opt/env`; no new environment was placed
on C:.

The first successful 32-frame pilot used frames 1,230–1,261 and published the
immutable diagnostic result
`e5-tracking-005090ae34dc2c09d75f0dcfe68b4f9eed9e011d480e0c4b0d650be9369ed841`.
It exposed two implementation effects:

1. nested same-class body/torso boxes survived conventional IoU NMS;
2. the first SciPy assignment import contaminated the first tracking latency
   with a 422 ms one-time cost.

The accepted runner added 80% containment suppression and warmed the assignment
solver before timed frames. A second 32-frame pilot then published
`e5-tracking-8035c0013e36a7cfb68aa3652b05ab79fe36baf136462d5fbf91b348ee85ec08`.
It processed 32/32 frames in 4.843450 seconds at 6.606861 FPS, reduced
detections from 235 to 227, retained zero same-class pairs at IoU >= 0.8 and
reduced tracking p95 to 0.401 ms. Visual inspection confirmed that the remaining
nearby person boxes represented the woman and child, not nested duplicates.

Both successful pilots remain immutable evidence. Only the second profile was
admitted to the main qualification run.

## Qualification result

Published result:
`e5-tracking-88aace13ef9963f8dc07f85228e530f9d28c2b49aca9192409f7975512b058f6`.

| Measurement | Result |
|---|---:|
| Frames expected / processed | 601 / 601 |
| Frames failed / skipped | 0 / 0 |
| Frames with confirmed tracks | 600 |
| Inference/tracking/artifact loop | 88.092835 s |
| Loop throughput | 6.822348 FPS |
| Stream extraction | 10.243353 s |
| H.264 encoding | 0.686277 s |
| Full worker wall | 166.335664 s |
| End-to-end throughput | 3.613176 FPS |
| Process peak RSS | 126.199 MiB |

The source rate is approximately 10.004 FPS. The measured frame loop is about
1.47 times slower than source real time, and the complete worker run is about
2.77 times slower. This exact artifact-heavy path is therefore offline. It is
not admitted as a live pipeline.

### Per-frame latency

| Stage | Mean | p50 | p95 | Max |
|---|---:|---:|---:|---:|
| End to end | 146.279 ms | 144.622 ms | 163.340 ms | 215.759 ms |
| Overlay write | 87.363 ms | 86.227 ms | 99.219 ms | 139.534 ms |
| Image decode | 30.132 ms | 28.455 ms | 40.723 ms | 63.155 ms |
| Triton request | 12.359 ms | 12.148 ms | 17.948 ms | 31.662 ms |
| Detector postprocess | 7.307 ms | 7.016 ms | 10.734 ms | 21.003 ms |
| Preprocess | 6.202 ms | 5.803 ms | 7.991 ms | 18.645 ms |
| Tracking | 0.247 ms | 0.234 ms | 0.390 ms | 0.894 ms |

The tracker is not the speed problem. The Triton request itself is also below
the approximately 100 ms/frame source budget on average. Synchronous PNG
overlay writing and image decoding dominate this qualification loop. A live
design must decouple visualization/artifact production from the inference
critical path and define bounded queues and a drop policy; it must not simply
run this script against an unbounded stream.

## Detection and identity observations

| Metric | Result |
|---|---:|
| Detections admitted | 4,049 |
| Confirmed track observations | 3,320 |
| Confirmed track IDs | 167 |
| Tracker tracks created / retired | 177 / 166 |
| Confirmed length mean / p50 / p95 / max | 19.88 / 6 / 94 / 212 frames |
| Same-class duplicate pairs at IoU >= 0.8 | 0 |
| Rejected oversized boxes | 7 |

Admitted detections by detector label:

| Label | Detections | Confirmed observations | Confirmed IDs |
|---|---:|---:|---:|
| car | 2,972 | 2,556 | 123 |
| truck | 646 | 483 | 15 |
| person | 364 | 249 | 24 |
| bicycle | 45 | 29 | 3 |
| motorcycle | 16 | 2 | 1 |
| bus | 6 | 1 | 1 |

Track persistence without ground-truth interpretation:

- 150/167 IDs had at least 2 confirmed observations;
- 97/167 had at least 5 observations;
- 63/167 had at least 10 observations;
- 30/167 had at least 30 observations;
- 15/167 had at least 60 observations;
- 7/167 had at least 100 observations.

These counts prove that the temporal state machine is operating; they do not
measure accuracy. The high number of short vehicle IDs is a visible baseline
limitation, driven by changing viewpoint, intermittent low-confidence
detections and class changes between `car` and `truck` while association is
class-consistent.

### Visual findings

1. Boxes remain inside the calibrated valid image circle; the black lens border
   no longer produces visual clutter or tracking candidates.
2. Parked vehicles and normally moving people retain IDs over useful clear-view
   intervals. Several vehicle tracks persist for 100–212 frames.
3. The close woman/child sequence demonstrates both success and failure. IDs
   54 and 60 persist through the unobstructed approach. When the woman/stroller
   fills the lower-left fisheye region and becomes heavily occluded, ID 60 is
   lost and a new person ID 76 begins. This is a real identity fragmentation,
   not hidden by the report.
4. YOLOX-S sometimes classifies the stroller as `motorcycle`, and the same
   parked vehicle can alternate between `car` and `truck`. COCO has no stroller
   class, and the generic road detector is not calibrated for the K1 fisheye.
5. Conventional NMS plus containment suppression removed the visible nested
   body/torso duplicates from the first pilot without suppressing the distinct
   woman and child.
6. There is no human-reviewed identity ground truth. IDF1, HOTA, MOTA and true
   ID-switch count therefore remain unknown and must not be inferred from the
   track-length statistics.

## GPU and neighboring services

Worker GPU: NVIDIA GeForce RTX 4090.

- one-second GPU samples: 88;
- GPU utilization: 51.65% mean, 52% p50, 55% p95, 56% max;
- board memory used, including pre-existing services: 10,239.0 MiB mean,
  10,249 MiB max;
- memory-controller utilization: 6.11% mean, 7% max;
- power: 144.78 W mean, 146.71 W p95, 148.40 W max;
- temperature: 42.60 C mean, 43 C max;
- process peak RSS: 126.199 MiB.

The board-memory value is total device use observed by `nvidia-smi`, not an E5
process allocation. This run did not expose a trustworthy per-process CUDA
peak, so none is claimed.

## Disk policy and measured usage

The orchestrator admitted only explicit worker paths rooted at
`D:\NDC_MISSIONCORE`. It reserved a conservative 4,726,173,047-byte working set,
checked D: at each major phase and every 100 frames, and failed closed if the
360 GiB floor would be crossed.

| Disk point | Free bytes | Free GiB |
|---|---:|---:|
| Before run | 409,176,440,832 | 381.075 |
| After frame extraction | 408,682,668,032 | 380.615 |
| After inference | 408,303,550,464 | 380.262 |
| After final artifacts | 408,300,429,312 | 380.259 |
| After exact temporary cleanup | 408,660,688,896 | 380.595 |
| Enforced floor | 386,547,056,640 | 360.000 |

The floor was never approached closer than 20.259 GiB. Only exact E5 temporary
stream, frame and overlay roots were removed. The immutable pilots, accepted
result, telemetry, runner and profiles remain on D:. No task-controlled C: path
was created, mounted or used. This does not claim visibility into incidental
internal writes by Windows or the pre-existing Docker Desktop engine.

## Artifacts and independent validation

| Artifact | Bytes | SHA-256 |
|---|---:|---|
| `tracking.mp4` | 12,270,533 | `d911efc46c5c3df38ca961700f82c407c44222ba465af752d2a7de75911740ba` |
| `frames.jsonl` | 1,116,700 | `e5f19e5dd8bf76040cb7859ffada3f1bb68a0f5cd93fac243947276899cce014` |
| `gpu-telemetry.jsonl` | 19,694 | `84cbaa4d5a7d503129f2fa329ab2b231cff54a6972cd31ba95fe91f48f01adad` |
| `contact-sheet.png` | 926,575 | `d9a23cc76e1a9a4aba1156609f85b781bc2502cd80f3b17ac83ca4228f2121a8` |
| `run-report.json` | 9,183 | `96d1cb059b35a0aa3fcc7c3458a0f47f57a4bd52a64c34f3c7607302e49206d9` |

Independent local validation confirmed:

- the result directory name equals its canonical identity SHA-256;
- all five artifact size/digest pairs match `result.json`;
- the result is bound to the exact job, input, session, source, clip bounds and
  timestamp basis;
- `frames.jsonl` contains exactly 601 ordered source indices and strictly
  increasing session timestamps;
- every reported detection and track box is finite and inside 800x600;
- no frame failed or was skipped;
- FFprobe reports H.264, yuv420p, 800x600, 60.068948 seconds, and exactly
  601 declared/read frames;
- the custom result validator accepts the immutable result;
- the E5 source and test files pass Ruff, and the focused E5 test suite covers
  profile pinning, FOV math, YOLOX decode, containment suppression, two-stage
  low-score identity retention, class isolation and nonzero source timelines.

Durable local result root:

```text
.runtime/compute-experiments/e5/results/
  e5-tracking-88aace13ef9963f8dc07f85228e530f9d28c2b49aca9192409f7975512b058f6/
```

Worker evidence root:

```text
D:\NDC_MISSIONCORE\runtime\derived\
  e5-tracking-88aace13ef9963f8dc07f85228e530f9d28c2b49aca9192409f7975512b058f6
```

## Acceptance status

Accepted:

- reproducible official YOLOX-S inference through the existing Triton worker;
- fixed calibration-bound valid-FOV preprocessing;
- ByteTrack-style high/low-score temporal association;
- 601/601 exact recorded frames with no failures or skips;
- watchable 2D ID overlay, trails and complete frame metadata;
- honest timing, GPU, process and disk telemetry;
- content-addressed immutable qualification artifacts;
- D-only task-controlled worker storage and restoration of Triton model state.

Not accepted:

- production-quality object classes or identity continuity;
- IDF1, HOTA, MOTA or measured ID-switch rate;
- handling of heavy close occlusion, re-entry or long disappearance;
- appearance ReID or camera-motion compensation;
- instance masks, 3D cuboids, LiDAR association or metric distance;
- a live unbounded stream or safety use;
- promotion of the partial clip into the saved-session production source list.

## Decision and next gate

LAB E5 closes the first temporal identity baseline. The result is useful enough
to become input to a bounded 3D fusion experiment, but not strong enough to be
called production tracking.

The next gate should be LAB E6 on the same frozen 601-frame interval:

1. use factory `camera_1` calibration to project synchronized LiDAR points into
   each accepted 2D observation;
2. reject points outside the valid FOV and use robust depth/cluster gates rather
   than a single nearest pixel;
3. compute per-object range and coarse point-supported 3D cuboids with explicit
   support counts and confidence;
4. smooth distance over the existing track ID where continuity is available;
5. treat COCO `car`/`truck` as one vehicle association group in the fusion
   layer so detector class flicker does not automatically break geometry;
6. publish a bounded Rerun qualification recording only after visual and
   numeric checks pass.

In parallel with, but not as a prerequisite for, this first 3D proof, the 2D
branch should later compare camera-motion compensation and appearance ReID on a
small human-reviewed identity set. Full-session and live runs should wait until
the artifact-writing path is removed from the inference critical section and a
bounded queue/drop policy has been tested.
