feat(lidar): add ground segmentation diagnostic benchmark
This commit is contained in:
@@ -0,0 +1,69 @@
|
||||
# ADR 0020: do not promote Patchwork++ on vendor-mapped LiDAR increments
|
||||
|
||||
Date: 2026-07-25
|
||||
Status: accepted and implemented as a diagnostic gate
|
||||
|
||||
## Context
|
||||
|
||||
The L2 roadmap selected Patchwork++ as the first non-neural ground segmentation
|
||||
baseline. The official implementation is designed around a sensor-centric
|
||||
LiDAR scan, radial zones and a physical sensor height. Current K1 `lio_pcl`
|
||||
evidence instead contains a vendor-mapped point increment in `map`, an
|
||||
independent best-effort pose and no admitted raw sweep or scan geometry. The
|
||||
pose describes vendor odometry; it does not prove the physical height of the
|
||||
LiDAR above terrain.
|
||||
|
||||
Treating a successful Patchwork++ call as a valid baseline would conflate API
|
||||
compatibility with input-domain compatibility. Choosing a synthetic Z
|
||||
translation until the output looks plausible would use the candidate itself to
|
||||
define the normalization.
|
||||
|
||||
## Decision
|
||||
|
||||
1. Pin the official Patchwork++ v1.4.1 source at commit
|
||||
`3e6903a1d5537a4cc2ace897b0bbb98a92d6014c`.
|
||||
2. Run it only behind the provider-neutral `lidar-ground/v1` diagnostic
|
||||
boundary.
|
||||
3. Bind the exact source commit and compiled binary SHA-256 into every result.
|
||||
4. Preserve point-aligned ground and assigned masks separately from raw replay.
|
||||
5. Compare it with the current local-percentile proposal using
|
||||
algorithm-to-algorithm IoU, disagreement and latency.
|
||||
6. Mark input-domain acceptance, labeled accuracy and production promotion
|
||||
false for current K1 vendor-map evidence.
|
||||
7. Never present algorithm IoU as ground IoU.
|
||||
8. Create an all-ignore, content-bound annotation template; only a separate
|
||||
human-reviewed generation may unlock ground IoU, curb/low-obstacle recall
|
||||
and reflection-noise rejection.
|
||||
9. Keep command, navigation and safety authority false.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Patchwork++ remains a useful candidate for a future raw vehicle LiDAR feed.
|
||||
- Its sub-millisecond host latency on the current slice does not compensate for
|
||||
the unaccepted input domain.
|
||||
- Current K1 work should prioritize independent labels and source evidence, not
|
||||
threshold tuning around a mis-specified sensor model.
|
||||
- The same immutable benchmark/API/React surface can compare a future raw scan,
|
||||
replay or simulation provider without moving C++ processing into React.
|
||||
|
||||
## Real diagnostic evidence
|
||||
|
||||
Benchmark
|
||||
`ground-benchmark-68cfd7a8f1dd4c0006183bb4f63a23f9ff1dd7459317ff0886e995f6c320d984`
|
||||
processed 66 frames and 226,963 points:
|
||||
|
||||
| Measurement | Current local percentile | Patchwork++ |
|
||||
| --- | ---: | ---: |
|
||||
| Ground fraction p50 | 18.31% | 0.53% |
|
||||
| Host latency p95 | 9.74 ms | 0.26 ms |
|
||||
|
||||
Algorithm ground IoU was 2.90% p50 and point disagreement was 17.92% p50.
|
||||
Neither is an accuracy metric. Independent labels are still missing.
|
||||
|
||||
## References
|
||||
|
||||
- `src/k1link/compute/lidar_ground.py`
|
||||
- `experiments/perception/run_lidar_ground_benchmark.py`
|
||||
- `src/k1link/web/lidar_api.py`
|
||||
- `apps/control-station/src/workspaces/LidarQualityWorkspace.tsx`
|
||||
- `docs/13_LIDAR_WORKER_PRODUCT_AND_ROADMAP.md`
|
||||
Reference in New Issue
Block a user