docs(perception): record PointPillars transfer verdict
This commit is contained in:
DCCONSTRUCTIONS
@@ -0,0 +1,251 @@
|
||||
# L3 PointPillars transfer report — 2026-07-31
|
||||
|
||||
## Decision
|
||||
|
||||
The LiDAR-native detector seam is operational on Worker 006, but the frozen
|
||||
NVIDIA deployable checkpoint is **rejected as a K1 product detector
|
||||
candidate**.
|
||||
|
||||
The result is not a runtime failure. The exact ONNX model was built on the
|
||||
target GPU, loaded into the existing canonical Triton service and evaluated
|
||||
over all `3,769` frozen KITTI validation frames. Runtime latency is acceptable,
|
||||
but the public cross-domain accuracy is not.
|
||||
|
||||
Immutable admission:
|
||||
|
||||
`l3-pointpillars-admission-857a8c6a958850f82db7a58e894e13e2a54add09379ca58fb8e47db1e6aba65c`
|
||||
|
||||
Immutable transfer probe:
|
||||
|
||||
`l3-pointpillars-kitti-1a6b499e194a363644854dc324bd1b565c100b809f145c1324a25328e7ae0910`
|
||||
|
||||
Worker package:
|
||||
|
||||
`l3-pointpillars-worker-package-c219d3b35be63b03d68be318a0709af2422aa67ae12561cec6c1de6252abad12`
|
||||
|
||||
The next gate is a separately admitted training/adaptation candidate. The
|
||||
current deployable checkpoint must not be promoted into a detector claim,
|
||||
navigation, safety or camera-replacement path.
|
||||
|
||||
## Corrected model contract
|
||||
|
||||
The generic KITTI example configuration is not the contract of the downloaded
|
||||
NGC checkpoint. The source of truth is the exact ONNX graph with SHA-256:
|
||||
|
||||
`2dcabddc3a365e9608a112d7bbbb7db769a6dddeeaa59aa03611a83113326da1`.
|
||||
|
||||
Read-only graph inspection produced the sealed evidence file
|
||||
`l3_pointpillars_onnx_contract_2026-07-31.json`, SHA-256:
|
||||
|
||||
`2fd29cd054ab058c2cfec3dfba305c71e123ef3f04b457d0c64de0c8dac2e1be`.
|
||||
|
||||
The embedded contract is:
|
||||
|
||||
- input: `points [batch, 204800, 4] FP32`;
|
||||
- count: `num_points [batch] INT32`;
|
||||
- fields: LiDAR-frame `x, y, z, intensity`;
|
||||
- voxel range:
|
||||
`[-51.2000008, -51.2000008, -1.39999998]` to
|
||||
`[51.2000008, 51.2000008, 4.4000001]`;
|
||||
- voxel size: `[0.2, 0.2, 5.8]`;
|
||||
- maximum voxels: `10,000`;
|
||||
- maximum points per voxel: `32`;
|
||||
- embedded score threshold: `0.1`;
|
||||
- native labels: `Vehicle`, `Pedestrian`, `Cyclist`;
|
||||
- decoded output:
|
||||
`output_boxes [batch, 393216, 9] FP32` plus
|
||||
`num_boxes [batch] INT32`;
|
||||
- decoded row:
|
||||
`x, y, z, length, width, height, yaw, class_id, score`.
|
||||
|
||||
External postprocessing is pinned to NVIDIA
|
||||
`tao_toolkit_recipes@a540badc47812a17a94e924b537d49ad3969b5a8`:
|
||||
stable descending score order, `4,096` pre-NMS candidates and class-agnostic
|
||||
oriented-BEV NMS at IoU `0.01`.
|
||||
|
||||
The NGC model card states that this checkpoint was trained and evaluated on a
|
||||
proprietary solid-state LiDAR dataset. Its native accuracy cannot be
|
||||
independently reproduced from the public model package. KITTI is therefore a
|
||||
public **cross-domain transfer probe**, not a native model benchmark.
|
||||
|
||||
## Runtime evidence
|
||||
|
||||
No second container or serving stack was created.
|
||||
|
||||
- canonical service: `ndc-mission-core-triton`;
|
||||
- image:
|
||||
`nvcr.io/nvidia/tritonserver:26.06-py3@sha256:58df7489c3f2276f9591d500a012dee03e23d35543ce3c390b4c001e6bf90794`;
|
||||
- model control: explicit;
|
||||
- strict readiness: enabled;
|
||||
- model repository mount: read-only;
|
||||
- GPU: NVIDIA GeForce RTX 4090, compute capability `8.9`;
|
||||
- TensorRT: `11.0.0`;
|
||||
- precision policy: strongly typed;
|
||||
- engine SHA-256:
|
||||
`12005d972a4632d56342a5da44442b632c1dcc5144fa3c70b162dec334532481`;
|
||||
- engine size: `8,785,436` bytes;
|
||||
- target build time: `9.13932 s`;
|
||||
- observed peak builder allocation: approximately `192 MiB`.
|
||||
|
||||
The model was added to the existing repository and loaded through Triton's
|
||||
explicit load endpoint. The container was not restarted.
|
||||
|
||||
One GOOSE native XYZI frame with `169,883` points passed a representation
|
||||
smoke at `55.2069 ms` GPU compute time. One KITTI frame with `120,268` points
|
||||
also passed the live schema smoke. These smokes prove input/output
|
||||
compatibility only.
|
||||
|
||||
## Dataset admission
|
||||
|
||||
KITTI 3D Object Detection 2017 was admitted archive-only on Worker 006:
|
||||
|
||||
- release identity:
|
||||
`2c9615bedca56b492b204b614d4419db6431626e2a6867e207a95999beefcf47`;
|
||||
- labeled training frames: `7,481`;
|
||||
- official test frames: `7,518`;
|
||||
- split source: OpenPCDet commit
|
||||
`233f849829b6ac19afb8af8837a0246890908755`;
|
||||
- train split: `3,712`, SHA-256
|
||||
`b6417a1d9b18c8fdb085128e633d28ff321b7674a6d1b3841b8f43d865b281cb`;
|
||||
- validation split: `3,769`, SHA-256
|
||||
`657ac4bcc1e156e5b106a4ca18e1f88e012787ea1d2b5d0adeea97fee903fa86`.
|
||||
|
||||
The archives remain on Worker 006 and were not copied to the Mac. Admission
|
||||
verifies official archive byte lengths and digests, safe ZIP structure, XYZI
|
||||
packing, label/calibration alignment, finite positive target cuboids and the
|
||||
complete disjoint split.
|
||||
|
||||
GOOSE and RELLIS remain valid for representation, semantic and instance
|
||||
experiments. Their point-wise labels are not silently converted into oriented
|
||||
3D-box truth.
|
||||
|
||||
## Evaluation boundary
|
||||
|
||||
KITTI's common detector volume and the embedded NVIDIA checkpoint volume are
|
||||
different. The measured intersection is:
|
||||
|
||||
`[0, -39.68, -1.39999998]` to `[51.2000008, 39.68, 1]`.
|
||||
|
||||
Predictions outside this shared volume are excluded rather than counted as
|
||||
false positives against a region the probe does not jointly cover.
|
||||
|
||||
The metric is deliberately not the official KITTI server metric:
|
||||
|
||||
- 40-point interpolated BEV and 3D AP;
|
||||
- no KITTI easy/moderate/hard filtering;
|
||||
- IoU `0.7` for `Car`;
|
||||
- IoU `0.5` for `Pedestrian` and `Cyclist`;
|
||||
- fixed mapping `Vehicle → Car`;
|
||||
- no validation retuning;
|
||||
- one sequential worker process.
|
||||
|
||||
## Complete transfer result
|
||||
|
||||
All `3,769/3,769` validation frames completed.
|
||||
|
||||
| Metric | Result |
|
||||
| --- | ---: |
|
||||
| BEV mAP40 | `0.0246322` (`2.46322%`) |
|
||||
| 3D mAP40 | `0.00000216745` (`0.000216745%`) |
|
||||
| False occupied rate | `0.999981` (`99.9981%`) |
|
||||
| Mean inference latency | `31.1366 ms` |
|
||||
| P50 inference latency | `29.0994 ms` |
|
||||
| P95 inference latency | `45.9418 ms` |
|
||||
| Maximum inference latency | `68.8731 ms` |
|
||||
|
||||
Per class:
|
||||
|
||||
| Class | BEV AP40 | 3D AP40 | TP | FP | GT |
|
||||
| --- | ---: | ---: | ---: | ---: | ---: |
|
||||
| Car | `0.0734788` | `0.0000034125` | `2` | `71,888` | `12,912` |
|
||||
| Pedestrian | `0.0000400679` | `0.00000308985` | `1` | `12,612` | `2,239` |
|
||||
| Cyclist | `0.000377644` | `0` | `0` | `74,760` | `801` |
|
||||
|
||||
Prediction accounting:
|
||||
|
||||
- post-NMS model boxes: `315,589`;
|
||||
- boxes inside the shared evaluation volume: `159,263`;
|
||||
- boxes outside the shared volume and explicitly ignored: `156,326`.
|
||||
|
||||
The frame-result set identity is:
|
||||
|
||||
`30b1933d508a09025a7d3c3c460fc2d06128e4bbe96a753bec7ba8545fda3e9c`.
|
||||
|
||||
The compact report SHA-256 is:
|
||||
|
||||
`f53d8d03a7c1e092d87b732872b26fea16447f373cf567bfe57737cb3ccbebba`.
|
||||
|
||||
## Interpretation
|
||||
|
||||
The worker seam, TensorRT engine, Triton serving path, bounded postprocess and
|
||||
full-dataset runner are usable. The pretrained checkpoint is not.
|
||||
|
||||
The result demonstrates severe domain mismatch; it does not demonstrate that
|
||||
PointPillars as an architecture is unsuitable for K1. It demonstrates that
|
||||
this exact proprietary-domain checkpoint cannot be used as the product
|
||||
detector without an admitted adaptation/training step.
|
||||
|
||||
No threshold was tuned to improve the result. Changing the score threshold on
|
||||
the frozen validation output would be validation leakage and would not repair
|
||||
the sensor-domain mismatch.
|
||||
|
||||
## Next gate
|
||||
|
||||
1. Freeze a new training/adaptation admission. It must name the trainable
|
||||
checkpoint, training container/toolchain, dataset identities, resource
|
||||
limits and output model identity before execution.
|
||||
2. Train only on the frozen `3,712` KITTI train frames.
|
||||
3. Keep all `3,769` validation frames untouched until the candidate is sealed.
|
||||
4. Export a new ONNX, build its TensorRT engine on Worker 006 and evaluate
|
||||
through the already proven canonical Triton seam.
|
||||
5. Require a useful public box-truth result before K1 accuracy language.
|
||||
6. Run `RAVNOVES01` as a K1 representation/stability transfer after the public
|
||||
candidate is useful. Because that session has no camera and no independent
|
||||
3D cuboids, it can prove deterministic replay, output stability, latency,
|
||||
queue/drop behavior and visual plausibility, but not detector accuracy.
|
||||
|
||||
The adaptation task is a new admission. This report does not authorize a
|
||||
training container, a second serving stack, K1 quality claims or navigation
|
||||
authority.
|
||||
|
||||
## Claim boundary
|
||||
|
||||
Not proved:
|
||||
|
||||
- native accuracy of the proprietary NVIDIA checkpoint;
|
||||
- K1 detector precision, recall, mAP or range/yaw accuracy;
|
||||
- that absence of a detection means free space;
|
||||
- superiority over the camera-first object candidate;
|
||||
- navigation or safety fitness.
|
||||
|
||||
Explicitly false:
|
||||
|
||||
- command authority;
|
||||
- navigation authority;
|
||||
- safety acceptance;
|
||||
- camera-first replacement;
|
||||
- LAB product publication.
|
||||
|
||||
## Reproduction
|
||||
|
||||
Admission:
|
||||
|
||||
```bash
|
||||
uv run python experiments/perception/run_l3_pointpillars_admission.py \
|
||||
--profile experiments/perception/l3_pointpillars_benchmark_profile.json \
|
||||
--dataset-inventory experiments/perception/l3_dataset_inventory_2026-07-30.json \
|
||||
--worker-inventory experiments/perception/l3_worker_inventory_2026-07-30.json
|
||||
```
|
||||
|
||||
The Worker package is built by:
|
||||
|
||||
```bash
|
||||
uv run python experiments/perception/prepare_l3_pointpillars_worker_package.py \
|
||||
--repository-root . \
|
||||
--output-root <worker-package-output> \
|
||||
--admission-result <accepted-admission-result>
|
||||
```
|
||||
|
||||
The complete probe runs only from the exact package on Worker 006 with
|
||||
`PYTHONPATH=<package>/runtime`, Python `-B`, the admitted KITTI root, the
|
||||
canonical model provenance and the existing local Triton endpoint.
|
||||
Reference in New Issue
Block a user