refactor(lab): restore canonical RAV004 replay
This commit is contained in:
DCCONSTRUCTIONS
Binary file not shown.
|
After Width: | Height: | Size: 85 KiB |
@@ -0,0 +1,167 @@
|
||||
# RAVNOVES004TREE canonical LAB replay audit
|
||||
|
||||
Date: 2026-08-30
|
||||
|
||||
Scope: Mission Core recorded LAB replay, RAVNOVES004TREE, OPS perception state
|
||||
|
||||
Excluded: Gaussian/simulation workers and their artifacts
|
||||
|
||||
## Outcome
|
||||
|
||||
RAVNOVES004TREE no longer owns a custom LAB viewer. It supplies recording and
|
||||
model configuration to the same `M4ReplayThreatVisual` and
|
||||
`CanonicalRecordedLabReplay` implementation used by the accepted recorded LAB.
|
||||
No new window or status type was added. The stable interaction contract remains:
|
||||
|
||||
- media: `SEMANTICS`, model selector, `VIDEO` / `CAMERA`;
|
||||
- spatial: `SOURCE POINTS`, `LOCAL SLAM`, `TGS COSTMAP`, `SEMANTICS`, `3D` / `PLAN`;
|
||||
- one timeline, one resizable split and one media-owned playback clock.
|
||||
|
||||
Models, result IDs, endpoints, labels and replay transport are configuration.
|
||||
Window structure, switching, seek, buffering and spatial scene code are shared.
|
||||
|
||||
## Why the previous LAB failed
|
||||
|
||||
### Video and spatial state had different clocks
|
||||
|
||||
The removed RAV004 viewer advanced an animation/host clock even when the browser
|
||||
decoder stopped. The point cloud therefore continued while the camera frame and
|
||||
timeline could remain frozen. The shared viewer now uses the decoded media time
|
||||
as the external clock, and image masks/boxes are rendered only when their time is
|
||||
within 250 ms of the actually presented video time.
|
||||
|
||||
The RAV004 MP4 itself is not clean. An independent `ffmpeg` decode around the
|
||||
reproducible stop at 11.422 s reported non-monotonic DTS values and corrupt H.264
|
||||
macroblocks. The RAV004 profile therefore uses the shared segmented MSE transport
|
||||
and an opt-in timestamp recovery rule. Recovery is allowed only when all of these
|
||||
conditions are true:
|
||||
|
||||
- playback is requested and the media element is not paused, ended or seeking;
|
||||
- decoded media time has not advanced by 20 ms for at least 1.25 s;
|
||||
- the browser reports decoded media buffered ahead of the frozen timestamp.
|
||||
|
||||
Only then is the broken timestamp interval skipped by 180 ms. The media clock
|
||||
immediately remains authoritative; the host does not free-run. A stale callback
|
||||
from the old MSE window is also prevented from undoing an operator seek.
|
||||
|
||||
### LiDAR orientation inherited the wrong axes
|
||||
|
||||
The RRD declares `/world` as RFU (`Right`, `Forward`, `Up`) and logs
|
||||
`/world/points` in map space. The earlier adapter treated raw LiDAR quaternion
|
||||
columns as rover forward/left/up and inherited sensor roll/pitch. That is why the
|
||||
grid, rover and facade could visibly disagree.
|
||||
|
||||
The v3 adapter now uses:
|
||||
|
||||
- map `+Z` as gravity/up;
|
||||
- the smoothed pose-trajectory tangent projected onto the ground as forward;
|
||||
- `left = up × forward`;
|
||||
- projected sensor `+Y` only as a fallback when the tangent is unavailable.
|
||||
|
||||
This is a deterministic coordinate contract, not a visual angle correction.
|
||||
|
||||
### Sensor height was treated as a constant
|
||||
|
||||
RAV004 does not have a stable 0.4 m mounting height throughout the recording.
|
||||
The adapter now estimates the local ground plane from a causal one-second
|
||||
near-field point window and uses the sealed session estimate only as fallback.
|
||||
Observed local heights include approximately 0.17 m, 1.24 m, 1.05 m and 0.22 m
|
||||
at different route positions; a single hand-entered value is therefore invalid.
|
||||
|
||||
### Sparse LiDAR frames were held incorrectly
|
||||
|
||||
Camera is approximately 9.51 Hz while source points arrive at approximately
|
||||
2 Hz. A camera frame without a new LiDAR increment used to retain whichever
|
||||
spatial frame happened to finish loading last; under fast playback this could be
|
||||
dozens of seconds old. The buffer now loads the active chunk first, the preceding
|
||||
chunk second and the next chunk as prefetch. The scene selects the latest proven
|
||||
source increment whose sequence is not later than the active camera frame.
|
||||
|
||||
At the final UI check, camera frame 189 causally held spatial frame 184. Before
|
||||
the fix the same point could hold frame 16.
|
||||
|
||||
## Capability ledger
|
||||
|
||||
| Layer | RAV004 full route | UI behavior | Authority |
|
||||
|---|---:|---|---|
|
||||
| Recorded RIGHT camera | 6830/6830 | `VIDEO` / `CAMERA`, segmented playback | recorded evidence |
|
||||
| DDRNet semantic mask | 6830/6830 | selectable, opaque enough for review | diagnostic prediction |
|
||||
| EoMT semantic mask | 6830/6830 | selectable | diagnostic prediction |
|
||||
| Diagnostic object boxes | derived from connected EoMT mask components | media-time gated | not an independent detector |
|
||||
| Source points | 1444 increments | `SOURCE POINTS` | recorded geometry |
|
||||
| Bounded Local SLAM | causal 5 s / 27k-point limit | `LOCAL SLAM` | visual-derived |
|
||||
| Full-route TGS | **absent** | canonical `TGS COSTMAP` control is visible but disabled | unavailable, fail closed |
|
||||
| Point-aligned 3D semantics | **absent** | canonical `SEMANTICS` control is visible but disabled | unavailable |
|
||||
| Independent person/vehicle detector | **absent** | no STOP claim | unavailable |
|
||||
|
||||
Ten old TGS review anchors exist, but they are not a continuous route artifact.
|
||||
They are not repeated or held as if they were full TGS. The accepted RAVNOVES00
|
||||
full-TGS result is also not reused because it has a different source identity and
|
||||
4489-frame timeline.
|
||||
|
||||
## Performance evidence
|
||||
|
||||
Measured on the canonical local service and current immutable artifacts:
|
||||
|
||||
- replay launch POST: 3.55 s on first opening;
|
||||
- timeline metadata: 0.02 s warm;
|
||||
- active spatial chunk, eight camera frames: 35.17 s first process-local RRD
|
||||
index build, 0.67 s warm, approximately 3.81 MB;
|
||||
- UI replay: passed the previously deterministic 11.422 s decoder stop, then
|
||||
continued to 59 s with media and timeline advancing together;
|
||||
- operator reset seek: 16.4 s to 0 s, one mounted media worker, successful;
|
||||
- browser console after the acceptance run: no warnings or errors.
|
||||
|
||||
The first RRD index is still process-local rather than a persistent disk cache.
|
||||
That is an explicit remaining performance gap; warm playback is the admitted
|
||||
profile, cold restart latency is not yet accepted.
|
||||
|
||||
## Nature perception: current OPS stopping point
|
||||
|
||||
OPS card `MISSIONCOR-65` defines the intended independent layers as EoMT,
|
||||
DDRNet, frozen YOLOX and TGS. The current immutable RAV004 artifact proves full
|
||||
EoMT and DDRNet inference only. It does not prove full TGS, negative-obstacle
|
||||
handling, an independent person/vehicle STOP layer or combined real-time load.
|
||||
|
||||
Isolated full-route measurements:
|
||||
|
||||
- DDRNet-39: p95 27.44 ms, 52.67 inference FPS, validation mean IoU 29.715%,
|
||||
vegetation mean IoU 0.3701;
|
||||
- EoMT: p95 361.62 ms, approximately 3.01 inference FPS;
|
||||
- prior accepted RAVNOVES00 TGS: p95 1.694 ms CPU-only, but this is algorithm
|
||||
performance on another source, not RAV004 proof.
|
||||
|
||||
The DDRNet isolated throughput is sufficient for a 10 FPS budget. DDRNet is not
|
||||
accepted for driving policy because temporal stability and nature quality are
|
||||
not sufficient: the OPS temporal sample recorded adjacent-frame IoU near 0.195
|
||||
for high grass and 0.400 for woody vegetation. EoMT does not meet 10 FPS in its
|
||||
current form. The next evidentiary milestone is therefore not another UI model
|
||||
toggle; it is synchronized truth for grass/tree/ditch/drop-off, full TGS and
|
||||
negative-obstacle evidence, frozen independent detector output and a combined
|
||||
load test at at least 10 FPS.
|
||||
|
||||
Worker 006 was audited read-only. Triton and the Gaussian containers were left
|
||||
untouched. The separate Mission Core perception worker is currently in a restart
|
||||
loop (404 during model inference startup); this audit did not stop, recreate or
|
||||
deploy it.
|
||||
|
||||
## Acceptance performed
|
||||
|
||||
- 44 focused backend tests passed;
|
||||
- 37 frontend replay, buffering and LAB contract tests passed;
|
||||
- TypeScript project typecheck passed;
|
||||
- production Vite build passed (only existing large-chunk warnings);
|
||||
- `git diff --check` passed;
|
||||
- live browser run verified the shared controls, disabled unsealed TGS/3D
|
||||
semantics, continuous media recovery, causal spatial hold and clean console.
|
||||
|
||||
Visual QA: `docs/handoff/2026-08-30_RAV004_CANONICAL_LAB_QA.jpg`.
|
||||
|
||||
## External coordinate and media references
|
||||
|
||||
- Rerun ViewCoordinates: <https://rerun.io/docs/reference/types/datatypes/view_coordinates>
|
||||
- Rerun transform relation: <https://rerun.io/docs/reference/types/components/transform_relation>
|
||||
- Rerun transforms: <https://rerun.io/docs/concepts/logging-and-ingestion/transforms>
|
||||
- Rerun Transform3D: <https://rerun.io/docs/reference/types/archetypes/transform3d>
|
||||
- WHATWG media element model: <https://html.spec.whatwg.org/multipage/media.html>
|
||||
- W3C Media Source Extensions: <https://www.w3.org/TR/media-source-2/>
|
||||
Reference in New Issue
Block a user