feat(lidar): add ground segmentation diagnostic benchmark

This commit is contained in:
DCCONSTRUCTIONS
2026-07-25 02:09:27 +03:00
parent e4fdd9fa20
commit 75a3e669d9
14 changed files with 2235 additions and 15 deletions
@@ -0,0 +1,69 @@
# ADR 0020: do not promote Patchwork++ on vendor-mapped LiDAR increments
Date: 2026-07-25
Status: accepted and implemented as a diagnostic gate
## Context
The L2 roadmap selected Patchwork++ as the first non-neural ground segmentation
baseline. The official implementation is designed around a sensor-centric
LiDAR scan, radial zones and a physical sensor height. Current K1 `lio_pcl`
evidence instead contains a vendor-mapped point increment in `map`, an
independent best-effort pose and no admitted raw sweep or scan geometry. The
pose describes vendor odometry; it does not prove the physical height of the
LiDAR above terrain.
Treating a successful Patchwork++ call as a valid baseline would conflate API
compatibility with input-domain compatibility. Choosing a synthetic Z
translation until the output looks plausible would use the candidate itself to
define the normalization.
## Decision
1. Pin the official Patchwork++ v1.4.1 source at commit
`3e6903a1d5537a4cc2ace897b0bbb98a92d6014c`.
2. Run it only behind the provider-neutral `lidar-ground/v1` diagnostic
boundary.
3. Bind the exact source commit and compiled binary SHA-256 into every result.
4. Preserve point-aligned ground and assigned masks separately from raw replay.
5. Compare it with the current local-percentile proposal using
algorithm-to-algorithm IoU, disagreement and latency.
6. Mark input-domain acceptance, labeled accuracy and production promotion
false for current K1 vendor-map evidence.
7. Never present algorithm IoU as ground IoU.
8. Create an all-ignore, content-bound annotation template; only a separate
human-reviewed generation may unlock ground IoU, curb/low-obstacle recall
and reflection-noise rejection.
9. Keep command, navigation and safety authority false.
## Consequences
- Patchwork++ remains a useful candidate for a future raw vehicle LiDAR feed.
- Its sub-millisecond host latency on the current slice does not compensate for
the unaccepted input domain.
- Current K1 work should prioritize independent labels and source evidence, not
threshold tuning around a mis-specified sensor model.
- The same immutable benchmark/API/React surface can compare a future raw scan,
replay or simulation provider without moving C++ processing into React.
## Real diagnostic evidence
Benchmark
`ground-benchmark-68cfd7a8f1dd4c0006183bb4f63a23f9ff1dd7459317ff0886e995f6c320d984`
processed 66 frames and 226,963 points:
| Measurement | Current local percentile | Patchwork++ |
| --- | ---: | ---: |
| Ground fraction p50 | 18.31% | 0.53% |
| Host latency p95 | 9.74 ms | 0.26 ms |
Algorithm ground IoU was 2.90% p50 and point disagreement was 17.92% p50.
Neither is an accuracy metric. Independent labels are still missing.
## References
- `src/k1link/compute/lidar_ground.py`
- `experiments/perception/run_lidar_ground_benchmark.py`
- `src/k1link/web/lidar_api.py`
- `apps/control-station/src/workspaces/LidarQualityWorkspace.tsx`
- `docs/13_LIDAR_WORKER_PRODUCT_AND_ROADMAP.md`