docs(lidar): record GOOSE qualification evidence
This commit is contained in:
parent
d6decbe05c
commit
cea63f9a1e
|
|
@ -313,8 +313,9 @@ harness work.
|
|||
claiming full vehicle TF, deskew or a general normalized scan.
|
||||
- [x] Run current ground and Patchwork++ against the same GOOSE native scan and
|
||||
ground truth; keep the result one-frame diagnostic.
|
||||
- [ ] Expand Current/Patchwork++ qualification to the validation split.
|
||||
- [ ] Add named range/FOV/density/noise/dropout degradation profiles without
|
||||
- [x] Expand Current/Patchwork++ qualification to all 961 validation frames
|
||||
through an immutable R1 Polygon replay-shadow run.
|
||||
- [x] Add named range/FOV/density/noise/dropout degradation profiles without
|
||||
overwriting the native frame.
|
||||
|
||||
The architectural contract and run sequence are fixed in
|
||||
|
|
@ -390,8 +391,10 @@ The near-term value is not a prettier point cloud:
|
|||
- hardware selection becomes evidence-driven: a future vehicle LiDAR is
|
||||
accepted by its timing/fields/profile, not by vendor marketing.
|
||||
|
||||
The highest-value immediate work is now the validation-split
|
||||
Current/Patchwork++ gate plus named range/FOV/density/noise/dropout degradation
|
||||
profiles. LiDAR-native detection follows on the same gateway. Nvblox and
|
||||
alternative SLAM remain later because their timing, pose and scan-geometry
|
||||
gates are not yet satisfied.
|
||||
The GOOSE validation-split and deterministic degradation gates are complete:
|
||||
pinned Patchwork++ improves micro Ground IoU from `47.42%` to `66.40%`, raises
|
||||
natural-ground recall from `51.76%` to `76.22%` and stays below `23.41 ms` p95
|
||||
across 961 frames. The decision remains shadow-only. The highest-value
|
||||
immediate work is now a second-source cross-check plus LiDAR-native detection
|
||||
on the same gateway. Nvblox and alternative SLAM remain later because their
|
||||
timing, pose and scan-geometry gates are not yet satisfied.
|
||||
|
|
|
|||
|
|
@ -163,9 +163,11 @@ time; TTL/dynamic filtering prevents stale ghosts.
|
|||
numeric vehicle TF or deskewed `normalized-scan`.
|
||||
7. [One-frame A/B done] Run the current ground heuristic and pinned
|
||||
Patchwork++ against the same native scan and independent labels.
|
||||
8. Add sensor-degradation profiles for range, FOV, density, noise and dropout.
|
||||
9. Expand the A/B to the validation split; only after that qualification gate,
|
||||
add a rolling-map sequence with localization evidence.
|
||||
8. [Done] Add deterministic sensor-degradation profiles for range, FOV,
|
||||
density, noise and dropout without modifying native source frames.
|
||||
9. [Done] Expand the A/B to all 961 validation frames and seal the result as a
|
||||
Polygon replay-shadow run. Add a rolling-map sequence with localization
|
||||
evidence only after this qualification boundary.
|
||||
|
||||
## Acceptance checklist
|
||||
|
||||
|
|
@ -184,8 +186,8 @@ time; TTL/dynamic filtering prevents stale ghosts.
|
|||
- [ ] Complete vehicle transform and general `normalized-scan` admitted.
|
||||
- [x] Current local-percentile baseline measured against ground truth.
|
||||
- [x] Patchwork++ one-frame accuracy measured against ground truth.
|
||||
- [ ] Current/Patchwork++ validation-split gate qualified.
|
||||
- [ ] Sensor-degradation matrix qualified.
|
||||
- [x] Current/Patchwork++ validation-split gate qualified.
|
||||
- [x] Sensor-degradation matrix qualified.
|
||||
- [ ] Rolling local map with pose/TTL/dynamic policy qualified.
|
||||
|
||||
## First GOOSE admission evidence
|
||||
|
|
@ -285,6 +287,50 @@ run. The result is deliberately `one-frame-diagnostic`: it makes Patchwork++
|
|||
the candidate for validation-split and degradation qualification, not an
|
||||
accepted navigation or safety provider.
|
||||
|
||||
## Validation-split qualification
|
||||
|
||||
The immutable R1 replay-shadow run
|
||||
`goose-ground-035de3ae6155844b636d` processed all `961` aligned validation
|
||||
frames directly from the D-only archive. It is bound to Mission Core commit
|
||||
`d6decbe05ccecd32d6a745767ee7a795bd1c88b1`, qualification identity
|
||||
`035de3ae6155844b636d89d8f23b36e561edab105720fa850094a5c6f680c0ef`
|
||||
and the same pinned Patchwork++ source/binary as the one-frame diagnostic.
|
||||
|
||||
| Validation micro metric | Current | Patchwork++ |
|
||||
| --- | ---: | ---: |
|
||||
| Ground IoU | `47.4199%` | `66.3974%` |
|
||||
| Natural-ground recall | `51.7650%` | `76.2215%` |
|
||||
| Obstacle non-ground recall | `93.5831%` | `94.6666%` |
|
||||
| Assigned fraction | `100%` | `100%` |
|
||||
| Latency p50 | `3527.02 ms` | `16.04 ms` |
|
||||
| Latency p95 | `6115.18 ms` | `23.40 ms` |
|
||||
| Maximum latency | `11162.62 ms` | `34.12 ms` |
|
||||
|
||||
The predeclared nominal gates passed: Ground IoU gain is `18.98` percentage
|
||||
points (required `>= 7`), natural-ground recall gain is `24.46` points
|
||||
(required `>= 10`), Patchwork++ obstacle non-ground recall is `94.67%`
|
||||
(required `>= 90%`), assigned fraction is `100%` (required `>= 99.9%`) and
|
||||
p95 latency is `23.40 ms` (required `<= 50 ms`). No frame exceeded the
|
||||
catastrophic regression boundary of `-20` Ground-IoU points; the worst observed
|
||||
delta was `-13.51` points and remains visible as a bounded failure preview.
|
||||
|
||||
| Deterministic profile | Source coverage | Ground IoU | Natural recall | p95 latency |
|
||||
| --- | ---: | ---: | ---: | ---: |
|
||||
| `range-20m` | `52.70%` | `63.1304%` | `72.1331%` | `16.29 ms` |
|
||||
| `range-40m` | `80.03%` | `68.5958%` | `80.0635%` | `20.74 ms` |
|
||||
| `density-50` | `50.00%` | `66.3575%` | `76.8925%` | `11.78 ms` |
|
||||
| `density-25` | `25.00%` | `65.5975%` | `76.8987%` | `5.99 ms` |
|
||||
| `noise-05m` | `100%` | `63.4378%` | `70.7234%` | `23.28 ms` |
|
||||
| `dropout-30` | `70.00%` | `66.4658%` | `76.6721%` | `16.24 ms` |
|
||||
| `front-180` | `51.48%` | `67.6924%` | `77.8012%` | `11.82 ms` |
|
||||
|
||||
Every degradation profile stayed inside the predeclared `15`-point Ground-IoU
|
||||
loss, `20`-point natural-recall loss and `50 ms` p95 limits. The sealed
|
||||
decision is therefore `shadow-candidate`, explicitly
|
||||
`promoted_to_navigation_or_safety = false`. The compact `12 MB` run journal,
|
||||
report and five worst-frame previews are available in `Polygon -> Runs`; the
|
||||
source archive and crash-resume frame cache remain only on worker D.
|
||||
|
||||
Primary source evidence:
|
||||
|
||||
- [GOOSE MuCAR-3 sensor setup](https://goose-dataset.de/docs/mucar3/);
|
||||
|
|
|
|||
Loading…
Reference in New Issue