# LAB E6 — Factory-calibrated tracked LiDAR fusion

Date: 2026-07-21
Completed at: 2026-07-21T09:18:15.532Z
State: completed qualification baseline, diagnostic geometry, not ground truth
Ops card: pending direct `nodedc-ops-agent` availability

## Objective

Connect the already qualified E5 temporal identities to metric observations from
the K1 LiDAR without inventing depth from image boxes. The bounded acceptance
target was:

1. reuse the exact 601-frame E5 interval and E4 semantic masks;
2. bind the right camera to factory calibration slot `camera_1`;
3. project raw K1 points with the factory KB4 intrinsics and camera/LiDAR
   extrinsics;
4. reject camera/point/pose combinations outside a strict 100 ms host-arrival
   gate;
5. associate only semantically compatible, nearest-depth, spatially connected
   LiDAR support with each tracked image object;
6. estimate a robust metric range and a point-supported oriented 3D envelope;
7. publish a watchable overlay and a standalone Rerun recording containing the
   camera view, map-frame point cloud, support points and translucent `Boxes3D`;
8. measure the geometry path and preserve every pilot and accepted artifact.

LAB E6 proves that the existing XGRIDS factory calibration can drive useful
recorded camera/LiDAR fusion. It does not claim amodal vehicle dimensions,
object-complete 3D boxes, ground truth, live-rate admission or safety fitness.

## Immutable inputs

- UI session alias: `RAVNOVES00`
- Session ID: `20260720T065719Z_viewer_live`
- Camera job: `recorded-camera-602ac89026ed12978619801d`
- Camera source: `sensor.camera.right`
- Factory calibration slot: `camera_1`
- Camera input SHA-256:
  `602ac89026ed12978619801d4edea0cae24b5cc3afabd9f7af2858de6505a20e`
- Factory calibration content SHA-256:
  `05f3ad9b38b3a4fc95388a8ec83da83c745e217709e51787b3d5aad0969f6fa9`
- Raw MQTT capture SHA-256:
  `70da0edad5cbf0e89b6e2355c0aac294f8c36a67249cd33b8918353fd88c83af`
- E4 semantic result:
  `result-793785170472c519486ccd666be102fb04d169d92383acda3fcc29eecf045d30`
- E5 tracking result:
  `e5-tracking-88aace13ef9963f8dc07f85228e530f9d28c2b49aca9192409f7975512b058f6`
- Source frames: 1,000–1,600 inclusive
- Frames admitted: 601/601
- Session interval: 135.365857292–195.334857292 seconds
- Media duration: 60.068948 seconds
- Resolution: 800x600

The 127 MiB MQTT capture contains the raw point and pose observations used by
the run. The calibration snapshot files are preserved separately from Git:

| File | SHA-256 |
|---|---|
| `camera.yaml` | `d123ebd2207e18a800cf5d07d055bf38231d28390bf3ba8c4e718e3cb3a7775e` |
| `extrinsic_camera_lidar.yaml` | `d09b2eda51e1deb7e3a31ff8441cba6006e8da60afc3e33de44a857437dfcfb8` |
| `manifest.json` | `75e1d582f661dabafb48fc6e8c7aabe0f217e0d95ed934b7d9dac18bdfc1216c` |

## Accepted configuration

Producer identities:

- pipeline: `factory-kb4-tracked-box-semantic-lidar/v1`;
- profile schema: `missioncore.e6-tracking-lidar-profile/v1`;
- profile SHA-256:
  `b8d3a6f043fde54ffd32ea05bfe580e2be7aace1a23cdbf7e266eb3325d147b2`;
- runner SHA-256:
  `9765be5e0338b9978db7a58a739313b228bc7ceee1352a7a76af0ff2a528a953`.

Temporal binding:

- nearest host-arrival observation, best effort;
- maximum LiDAR/camera delta: 100 ms;
- maximum pose/point delta: 100 ms;
- frames outside the gate fail closed for depth; image tracking remains visible.

Projection and occlusion:

- factory KB4 projection for `camera_1`;
- source points transformed from the K1 map frame through the recorded pose;
- only camera-front, in-frame points are retained;
- one nearest-depth point is retained per rounded image pixel;
- no fabricated depth and no 2D-box extrusion.

Association:

- image boxes are inset 5% horizontally, 4% at the top and 2% at the bottom;
- `car`, `truck` and `bus` share the `vehicle` association group;
- same-frame vehicle overlap suppression uses IoU 0.55;
- E4 semantic IDs gate LiDAR support: person 1, bicycle 2, motorcycle 3,
  vehicle 4/5;
- depth splitting uses `max(0.5 m, 6% of range)`;
- the selected depth component is split again by 3D connected components;
- spatial radii are 0.9 m for person/bicycle/motorcycle and 1.5 m for vehicle;
- minimum support is 3 points for person/bicycle/motorcycle and 5 for vehicle;
- distance uses cluster median, with p10 retained for diagnostics;
- a five-frame track history smooths distance;
- an observation is rejected when its distance innovation exceeds
  `max(1.5 m, 25% of track history)`.

Cuboid construction:

- yaw comes from PCA over supported map-frame XY points;
- dimensions are the oriented p05–p95 support extent;
- every axis has a 0.15 m diagnostic minimum for visibility;
- maximum accepted extents are 1.5x1.5x2.8 m for person,
  3.5x2.0x2.5 m for bicycle/motorcycle and 6.5x3.5x4.0 m for vehicle;
- rejected or unsupported detections never produce a 3D cuboid.

## Execution topology and resource boundary

The E6 geometry stage ran as a single local CPU process on the Mission Core
host, an Apple M3 Pro with 18 GiB RAM. It used Python 3.12.13, NumPy 2.5.1,
Pillow 12.3.0, Rerun 0.34.1 and FFmpeg 7.1.1. The external AI worker and its
Triton model state were not changed. No path on worker disk C: or D: was read,
written or cleaned by this stage.

```text
exact E5 frame + track IDs
  + exact E4 semantic mask
  + nearest gated K1 point/pose sample
  -> factory KB4 camera projection
  -> per-pixel nearest-depth buffer
  -> box inset + semantic support gate
  -> 1D depth split + 3D connected component
  -> range innovation gate
  -> robust range + PCA-yaw p05/p95 envelope
  -> overlay / NPZ / JSONL / Rerun recording
  -> content hash, qualification validation and publish
```

Frames were processed sequentially. Artifact writing was performed inside the
same measured process. This is a recorded qualification path, not the live
queue/worker architecture.

## Pilot history and profile freeze

Three immutable 32-frame pilots were retained:

1. `e6-fusion-a72468...` exposed 7–8 m person envelopes when a depth-only
   cluster admitted background structure. It was rejected.
2. `e6-fusion-693246...` added 3D connectivity and plausible size gates. Box
   dimensions became plausible, but range could still jump between foreground
   and background support on the same track. It was rejected.
3. `e6-fusion-70fa670ed9005957266ae7a46ac2bfd2d526f1ea170cd052f4494a2397fff76f`
   added the distance-innovation gate and was accepted. It fused 32/32 frames,
   produced 47 cuboids across five track IDs, rejected two range jumps, ran in
   3.681 seconds and peaked at 159.828 MiB RSS.

The accepted configuration was frozen after the third pilot. The 601-frame
result was not used to tune its thresholds.

## Qualification result

Published result:
`e6-fusion-b4e4226674a66f6196c033785eb307c255a7bdd8493ef9809dfb1d6e5bd68eaa`.

| Measurement | Result |
|---|---:|
| Frames expected / processed | 601 / 601 |
| Frames fused | 526 |
| Frames outside strict sync gate | 75 |
| Frames with at least one accepted cuboid | 438 |
| Accepted cuboids | 918 |
| Unique track IDs with depth | 56 |
| Source cloud points consumed | 1,182,292 |
| Accepted support points | 14,025 |
| Geometry/artifact wall time | 14.774254 s |
| Effective throughput | 40.678 FPS |
| User / system CPU | 13.571966 / 0.772126 s |
| Process peak RSS | 240.469 MiB |

This recorded geometry/artifact stage is about 4.07 times faster than the
approximately 10.004 FPS camera source. That does not establish a live
end-to-end rate: E4 segmentation and E5 detection/tracking are precomputed
inputs and were not included in the 14.774-second measurement.

### Synchronization

| Delta | Mean | p50 | p95 | Max |
|---|---:|---:|---:|---:|
| LiDAR to camera | 29.389 ms | 23.905 ms | 70.794 ms | 96.107 ms |
| Pose to point cloud | 7.940 ms | 6.370 ms | 19.113 ms | 46.139 ms |

All fused frames satisfy the configured 100 ms gates. The 75 nonconforming
camera frames retain their 2D presentation but carry no depth or 3D claim.
Host-arrival time is still only a best-effort clock; hardware timestamps have
not been ground-truthed.

### Accepted geometry distribution

| Group | Cuboids | Unique track IDs |
|---|---:|---:|
| Vehicle | 809 | 43 |
| Person | 96 | 12 |
| Bicycle | 13 | 1 |
| Total | 918 | 56 |

- accepted range: 1.239–37.749 m, p50 10.189 m, p95 24.126 m;
- support per cuboid: minimum 3, p50 10, p95 45, maximum 89 points;
- p95 oriented extent: 5.383x1.197x2.222 m;
- maximum admitted extent: 6.490x2.948x2.783 m.

The extent is an envelope of the visible supported surface. It is intentionally
not an amodal estimate of the object's complete physical volume. A parked car
seen from one side can therefore receive a thin cuboid that does not wrap the
whole vehicle silhouette.

### Fail-closed decisions

| Status | Observations |
|---|---:|
| Accepted point-supported cuboid | 918 |
| Fewer than 5 vehicle support points | 1,023 |
| No semantically compatible LiDAR support | 365 |
| Rejected distance innovation | 209 |
| Fewer than 3 non-vehicle support points | 28 |
| Implausible cuboid extent | 24 |

These rejection counts are part of the result, not discarded errors. In
particular, the high low-support count prevents distant image detections from
being turned into false metric objects.

## Rerun and visual QA

The Rerun recording logs seven entities across the `session_time` timeline:

- encoded camera fusion overlay;
- raw map-frame LiDAR points;
- selected object support points;
- translucent oriented `Boxes3D` with track/class/range labels;
- a visible qualification contract stating the timing and safety limits.

The web viewer was loaded from the local gRPC proxy and inspected at multiple
timestamps. Camera overlay, point cloud, support and `Boxes3D` layers load
together. Selecting a cuboid exposes its real center, half-sizes, quaternion,
fill mode and label in Rerun. Browser logs contained no load/render errors;
only Rerun web-backend deprecation/default warnings were present.

## Artifacts

| Artifact | Bytes | SHA-256 |
|---|---:|---|
| `fusion.npz` | 9,745,489 | `3e0e63797c56505eb9755d253f262459d46ec841e80cc5b785a6b495ed2007f9` |
| `fusion-frames.jsonl` | 1,607,124 | `ea1621bb337c6c268f91f6e2c7cef997d78672da4025975328c9e2583a0718e0` |
| `box-labels.json` | 30,328 | `0c423024a086344fcdbac8665635a45e88fef81270be73d9e1af3e6bc4eb860c` |
| `fusion-overlay.mp4` | 24,156,459 | `a863d1910eb973cef1f4cca28432cb16b26030149e197ff7f9ad3d168b69954c` |
| `fusion.rrd` | 63,092,299 | `77b0b91171176dd7ea6d2ef726cc946ab1fc24367750101d3f10617360cb4d63` |
| `contact-sheet.png` | 3,066,509 | `adcad4db4fd3089175ed947b312129fc86f04abc065314175eceb62368adac49` |
| `run-report.json` | 5,405 | `ca01cb5fa0325d80504199b623f2972141f45b680b4b16ad6a5271f9dd1deb2c` |

FFprobe independently confirmed H.264/yuv420p, 800x600, 601 declared/read
frames and 60.068948 seconds. The custom E6 validator rehashed every artifact
and verified the exact session, job, camera, calibration, clip, upstream result
and configuration identities. The complete E6 workspace including all pilots
uses 124 MiB on the Mission Core host.

## What is proven and what remains open

Proven:

- the right-camera/`camera_1` factory calibration is operational for recorded
  K1 camera/LiDAR projection;
- E5 track identities can receive robust metric LiDAR ranges;
- unsupported 2D boxes do not become fake 3D geometry;
- Rerun can display the camera, map cloud, support and true oriented 3D
  primitives on one timeline;
- the geometry stage itself is comfortably faster than source rate on this
  host.

Not proven:

- object-complete vehicle/person volume or metric box accuracy;
- calibration error in pixels or centimetres against a measured target;
- hardware-clock synchronization accuracy;
- tracking accuracy, instance-mask accuracy or semantic ground truth;
- live end-to-end throughput, bounded queues, drop policy or recovery;
- safety or navigation fitness.

## Next gate — LAB E7

Use this exact E6 result as the control and improve 3D geometry without hiding
rejections. The bounded E7 experiment should add ground removal, object-aware
support completion and a reviewed upstream 3D detection baseline; compare box
coverage, range stability, support/rejection rates and runtime on the same
601-frame interval. Only after that comparison should the winning geometry
path be connected to a bounded live queue. Multi-camera fusion and product UI
work remain outside this gate.
