feat(lidar): qualify Patchwork++ on GOOSE

2026-07-25 15:22:28 +03:00
parent 951b40c870
commit 60ba64004b
15 changed files with 1082 additions and 256 deletions
@@ -2,7 +2,7 @@

 Date: 2026-07-25
 Status: accepted architecture plan; L0/L1 implemented; L2 diagnostic A/B complete;
-Dataset Gateway S0 implemented, first real GOOSE import pending worker D storage
+Dataset Gateway first-frame Current/Patchwork++ A/B complete
 Scope: real scanner records, replay and future live shadow processing
 Explicitly out of scope: Unreal U0/U1, Gaussian assets and simulator rendering

@@ -293,7 +293,7 @@ path. K1 manual review or a new real vehicle dataset is reserved for later
 domain adaptation after a public baseline proves that the pipeline and metric
 harness work.

-### L2.5 — Dataset Gateway — S0 complete, real import pending
+### L2.5 — Dataset Gateway — first-frame A/B complete

 - [x] Define separate `native-scan`, `normalized-scan` and
  `rolling-local-map` representations.
@@ -304,13 +304,16 @@ harness work.
  per-point time and line/ring fields.
 - [x] Require operator-admitted storage under
  `D:\NDC_MISSIONCORE\datasets`.
- [ ] Configure the worker dataset root and record disk/resource baseline.
- [ ] Download only the 3.3 GB GOOSE validation archive first and record its
+- [x] Configure the worker dataset root and record disk/resource baseline.
+- [x] Download only the 3.3 GB GOOSE validation archive first and record its
  hash/license/provenance.
- [ ] Show one real labeled revolution in React with native remission and
+- [x] Show one real labeled revolution in React with native remission and
  ground-truth superclass coloring.
- [ ] Admit an explicit frame/mounting profile before normalization.
- [ ] Run current ground and Patchwork++ against GOOSE ground truth.
+- [x] Admit the published Patchwork-specific axes/height profile without
+  claiming full vehicle TF, deskew or a general normalized scan.
+- [x] Run current ground and Patchwork++ against the same GOOSE native scan and
+  ground truth; keep the result one-frame diagnostic.
+- [ ] Expand Current/Patchwork++ qualification to the validation split.
 - [ ] Add named range/FOV/density/noise/dropout degradation profiles without
  overwriting the native frame.

@@ -387,8 +390,8 @@ The near-term value is not a prettier point cloud:
 - hardware selection becomes evidence-driven: a future vehicle LiDAR is
  accepted by its timing/fields/profile, not by vendor marketing.

-The highest-value immediate work is the worker-side GOOSE validation import,
-the first labeled native scan in React and an accuracy-bearing ground A/B.
-LiDAR-native detection follows on the same gateway. Nvblox and alternative
-SLAM remain later because their timing, pose and scan-geometry gates are not yet
-satisfied.
+The highest-value immediate work is now the validation-split
+Current/Patchwork++ gate plus named range/FOV/density/noise/dropout degradation
+profiles. LiDAR-native detection follows on the same gateway. Nvblox and
+alternative SLAM remain later because their timing, pose and scan-geometry
+gates are not yet satisfied.
@@ -39,6 +39,12 @@ Implemented now:
  `Открыть` action;
 - one admitted GOOSE validation frame with source colors, normalized remission
  and an independent ground-truth view;
+- a published, dimensioned MuCAR-3/VLS-128 Patchwork++ input profile:
+  sensor frame `x-forward / y-left / z-up`, physical height `2.24 m`, native
+  one-revolution scan and explicit absence of deskew/full vehicle TF claims;
+- a reproducible Current/Patchwork++ A/B artifact with point-aligned masks,
+  disagreement views, independent-label metrics, latency and exact provider
+  identities;
 - a separate **Парк → Диагностика LiDAR** surface containing only real sensor
  recordings and their operational evidence.

@@ -152,11 +158,13 @@ time; TTL/dynamic filtering prevents stale ghosts.
 4. [Done] Import one labeled frame and expose it in React as `native-scan`.
 5. [Done] Show native remission, the original semantic palette and
   ground-truth superclass coloring.
-6. Add a declared GOOSE frame/mounting profile and produce `normalized-scan`.
-7. [Current baseline done; Patchwork++ blocked on mounting evidence] Run the
-   current ground heuristic and Patchwork++ against independent labels.
+6. [Patchwork-specific profile done; general normalization still pending] Admit
+   the published VLS-128 axes and physical height without claiming a complete
+   numeric vehicle TF or deskewed `normalized-scan`.
+7. [One-frame A/B done] Run the current ground heuristic and pinned
+   Patchwork++ against the same native scan and independent labels.
 8. Add sensor-degradation profiles for range, FOV, density, noise and dropout.
-9. Only after the one-frame contract passes, expand to the validation split and
+9. Expand the A/B to the validation split; only after that qualification gate,
   add a rolling-map sequence with localization evidence.

 ## Acceptance checklist
@@ -172,9 +180,11 @@ time; TTL/dynamic filtering prevents stale ghosts.
 - [x] Worker D root configured.
 - [x] GOOSE validation archive hash recorded.
 - [x] First real labeled frame visible in React.
- [ ] Coordinate and mounting profile admitted.
+- [x] Patchwork-specific axes and physical-height profile admitted.
+- [ ] Complete vehicle transform and general `normalized-scan` admitted.
 - [x] Current local-percentile baseline measured against ground truth.
- [ ] Patchwork++ accuracy measured against ground truth.
+- [x] Patchwork++ one-frame accuracy measured against ground truth.
+- [ ] Current/Patchwork++ validation-split gate qualified.
 - [ ] Sensor-degradation matrix qualified.
 - [ ] Rolling local map with pose/TTL/dynamic policy qualified.

@@ -229,10 +239,54 @@ previous exact-radius result. This removes the previous all-points scan for
 every occupied cell, but the measured latency still classifies it as a
 diagnostic baseline rather than an onboard candidate.

-Patchwork++ is intentionally not scored yet. The validation ZIP contains XYZI,
-labels, mapping, LICENSE and CHANGELOG but no numeric TF/mounting calibration.
-GOOSE documents the VLS-128 as a roof LiDAR and publishes a separate MuCAR-3 TF
-tree, but the graph image alone is not physical-height evidence. The next gate
-is to admit the numeric transform from `base_link_ground` to
-`sensor/lidar/vls128_roof`, declare the source axis convention, and only then
-run Patchwork++.
+## First Patchwork++ A/B result
+
+GOOSE's dimensioned MuCAR-3 schematic provides the two vertical dimensions
+needed by this algorithm-specific gate: `base_link` is `0.64 m` above ground
+and the VLS-128 optical center is `1.60 m` above `base_link`. The admitted
+Patchwork++ height is therefore `2.24 m`. The same published schematic declares
+the sensor axes as `x` forward, `y` left and `z` up. An independent fit to
+GOOSE-labeled near-field ground observed a `-2.18 .. -2.14 m` intercept; this
+was a non-calibrating cross-check, not the source of the height.
+
+This narrowly admits the input required by Patchwork++ on the native
+sensor-centric scan. It does **not** claim a complete numeric vehicle transform,
+per-point timing, deskew or a general `normalized-scan`.
+
+Official Patchwork++ `v1.4.1`, source commit
+`3e6903a1d5537a4cc2ace897b0bbb98a92d6014c`, was run against the same first
+frame and independent labels as Current:
+
+| Metric | Current | Patchwork++ |
+| --- | ---: | ---: |
+| Ground IoU | `49.6165%` | `60.3702%` |
+| Precision | `73.5892%` | `72.6612%` |
+| Recall | `60.3659%` | `78.1130%` |
+| F1 | `66.3249%` | `75.2886%` |
+| Artificial-ground recall | `95.2364%` | `98.9688%` |
+| Natural-ground recall | `49.4516%` | `71.5853%` |
+| Obstacle non-ground recall | `79.6145%` | `92.1729%` |
+| Worker latency | `3966.53 ms` | `15.89 ms` |
+
+Reproducibility pins:
+
+- benchmark identity:
+  `2a6d05f54a9e2ac727d9c850c1133f2fb539bfeddbd97c07cfd198ccb239162c`;
+- full point-aligned prediction SHA-256:
+  `190f455c6e47911921b7f6454913e1bb2d0b8b5814ee2e34d7b63295afddc7ef`;
+- bounded browser preview SHA-256:
+  `e23ca573103faf931523415e6c263248737eb6600f482b63de685b725bbc28c3`;
+- Patchwork++ binary SHA-256:
+  `be8038b2098c83fe53841aa8ae19e362910e9056fe0ee7b9304c1ee5c5941094`.
+
+Patchwork++ wins this frame by `10.75` percentage points of Ground IoU, raises
+natural-ground recall by `22.13` points and is roughly `250x` faster in this
+run. The result is deliberately `one-frame-diagnostic`: it makes Patchwork++
+the candidate for validation-split and degradation qualification, not an
+accepted navigation or safety provider.
+
+Primary source evidence:
+
+- [GOOSE MuCAR-3 sensor setup](https://goose-dataset.de/docs/mucar3/);
+- [GOOSE paper, Figure 3](https://arxiv.org/pdf/2310.16788);
+- [Patchwork++ v1.4.1](https://github.com/url-kaist/patchwork-plusplus/tree/v1.4.1).
@@ -87,24 +87,34 @@ unavailable rather than inventing timestamps.
 - Sensor adaptation may change range, FOV, point density, noise and dropout for
  robustness experiments, but it cannot recreate lost timestamps, occlusions
  or material response.
- Patchwork++ becomes eligible for a real quality gate only on a sensor-centric
-  scan with declared scan geometry and physical mounting height plus
-  independent labels.
+- Patchwork++ becomes eligible for an algorithm-specific quality gate on a
+  sensor-centric scan with declared axes, physical mounting height and
+  independent labels. That narrow admission does not itself create a general
+  `normalized-scan`.
 - Stable operator visualization is owned by rolling-map policy, not by the
  ground classifier.
 - Public labels may qualify an algorithm independently of the production
  sensor. The first GOOSE frame exposed a `49.62%` Ground IoU and only `49.45%`
  natural-ground recall for the current local-percentile baseline; these are
  diagnostic results, not production promotion.
- A dataset label contract does not imply a mounting contract. Patchwork++
-  remains blocked until the numeric GOOSE roof-LiDAR transform and physical
-  height are admitted from source evidence.
+- A dataset label contract does not imply a mounting contract. For MuCAR-3,
+  GOOSE's published dimensioned schematic admits the Patchwork-specific
+  `2.24 m` sensor height (`0.64 + 1.60 m`) and `x-forward / y-left / z-up`
+  axes. The complete numeric vehicle transform, deskew and general
+  `normalized-scan` remain unavailable.
+- On the first independently labeled frame, pinned Patchwork++ `v1.4.1`
+  achieved `60.37%` Ground IoU, `71.59%` natural-ground recall and `15.89 ms`
+  worker latency versus Current's `49.62%`, `49.45%` and `3966.53 ms`.
+  This is a one-frame diagnostic candidate decision, not production promotion.

 ## Primary references

 - [GOOSE dataset structure](https://goose-dataset.de/docs/dataset-structure/)
 - [GOOSE setup and archive sizes](https://goose-dataset.de/docs/setup/)
 - [GOOSE 3D challenge ontology](https://goose-dataset.de/docs/3d-semantic-segmentation-challenge/)
+- [GOOSE MuCAR-3 sensor setup](https://goose-dataset.de/docs/mucar3/)
+- [GOOSE paper and dimensioned sensor schematic](https://arxiv.org/pdf/2310.16788)
+- [Patchwork++ v1.4.1](https://github.com/url-kaist/patchwork-plusplus/tree/v1.4.1)
 - [Livox ROS Driver 2 point formats](https://github.com/Livox-SDK/livox_ros_driver2)
 - [Livox LIO motion-distortion handling](https://github.com/Livox-SDK/LIO-Livox)
 - [ROS FilterDeskew timestamp requirement](https://docs.ros.org/en/noetic/api/mp2p_icp/html/classmp2p__icp__filters_1_1FilterDeskew.html)