feat(perception): add full camera-first shadow
This commit is contained in:
DCCONSTRUCTIONS
@@ -415,6 +415,13 @@ Dataset expansion is no longer the next gate.
|
||||
- [x] Add a separate diagnostic motion-observation and near-occupied proximity
|
||||
derivative with temporal freshness, source provenance and no persistent-map
|
||||
mutation.
|
||||
- [x] Run the candidate camera detector through the canonical E29 association
|
||||
over all 4,489 RAVNOVES00 frames. Keep camera semantic ownership, publish
|
||||
LiDAR range only from qualified occupied support and retain the detector as
|
||||
a diagnostic shadow pending independent quality review.
|
||||
- [x] Version the vehicle-body/LiDAR-mount contract and bind RAVNOVES00
|
||||
fail-closed as a portable, physically unbound K1 source without invented
|
||||
dimensions or transforms.
|
||||
- [ ] Admit a `recent-collision` state only after vehicle-body and LiDAR-mount
|
||||
geometry are bound; proximity is not collision truth.
|
||||
- [x] Reuse the accepted camera-to-LiDAR projection as an optional semantic
|
||||
|
||||
@@ -176,6 +176,13 @@ real authority.
|
||||
- [x] Verify strict round-trip plus adversarial duplicate ownership, unknown
|
||||
owner, camera-only/conflict range, geometry-only semantics, held/persistent
|
||||
ownership, unavailable source and authority mutation cases.
|
||||
- [x] Reuse the canonical E29 frame association in E53 over all 4,489
|
||||
RAVNOVES00 frames with a second camera candidate. Preserve camera semantic
|
||||
ownership, keep unavailable LiDAR range explicit and reject runtime
|
||||
promotion without independent truth.
|
||||
- [x] Add `missioncore.rig-geometry/v1` with strict unbound/measured/qualified
|
||||
states. Bind the portable RAVNOVES00 K1 as `unbound`; do not infer a vehicle
|
||||
body or LiDAR-to-body mount from the recording.
|
||||
|
||||
The frozen A3 engineering and human generations cover all 486 items and
|
||||
retains the exact A2 materialization/review-pack and 42-sheet evidence
|
||||
|
||||
@@ -0,0 +1,57 @@
|
||||
# ADR 0035: Camera-first shadow and physical rig-geometry boundary
|
||||
|
||||
Date: 2026-07-30
|
||||
Status: accepted and implemented
|
||||
|
||||
## Decision
|
||||
|
||||
Mission Core preserves one perception ownership rule:
|
||||
|
||||
```text
|
||||
camera owns semantic class and object proposal
|
||||
LiDAR owns metric support and range
|
||||
```
|
||||
|
||||
The canonical E29 association is exposed as one reusable frame evaluator and
|
||||
is reused by E29 and full-source candidate shadows. A camera proposal remains
|
||||
an object when LiDAR support is unavailable; its range remains unavailable.
|
||||
Unassociated LiDAR geometry remains geometry without a semantic class.
|
||||
|
||||
E53 runs a candidate camera detector over every RAVNOVES00 frame through that
|
||||
unchanged association. It is a diagnostic shadow and cannot mutate E29,
|
||||
persistent reconstruction or runtime detector selection.
|
||||
|
||||
## Rig geometry
|
||||
|
||||
Vehicle-body and LiDAR mount/extrinsic use
|
||||
`missioncore.rig-geometry/v1`. The contract has three states:
|
||||
|
||||
- `unbound`: no physical body or mount values may be present;
|
||||
- `measured`: body and rigid transform require immutable measurement evidence
|
||||
and uncertainty, but collision remains unavailable;
|
||||
- `qualified`: geometry passed its named qualification method, while
|
||||
collision still requires a separate algorithm and safety gate.
|
||||
|
||||
RAVNOVES00 binds `ravnoves00-portable-k1/unbound-v1`. It is portable K1
|
||||
evidence and supplies no vehicle body or measured LiDAR→body transform.
|
||||
Invented dimensions, sensor height or identity transforms are rejected.
|
||||
|
||||
## Resource correction
|
||||
|
||||
The first E53 full pass exposed repeated decompression of
|
||||
`cloud_points_map` inside the frame loop. That immutable result failed its
|
||||
50 ms p95 gate at 210.7093 ms.
|
||||
|
||||
The accepted implementation materializes compressed source members once
|
||||
before sequential frame processing. The corrected run processed all 4 489
|
||||
frames at 1.7993 ms p95 and 36.8594 MiB peak-RSS growth. No Docker, worker,
|
||||
network service or second Mission Core backend participates in the path.
|
||||
|
||||
## Authority
|
||||
|
||||
E53 proves technical full-source execution, not detector accuracy:
|
||||
|
||||
- E29 overlap is comparison, not ground truth;
|
||||
- adjacent image-space IoU is continuity diagnostics, not tracking truth;
|
||||
- the candidate detector is not operationally promoted;
|
||||
- collision, navigation, safety and commands remain false.
|
||||
@@ -0,0 +1,116 @@
|
||||
# LAB E53 · полный camera-first shadow
|
||||
|
||||
Дата: 2026-07-30
|
||||
|
||||
Статус: принят технический диагностический shadow; camera detector не
|
||||
повышен до operational runtime
|
||||
|
||||
## Цель
|
||||
|
||||
Проверить на всех 4 489 кадрах RAVNOVES00 восстановленный архитектурный
|
||||
принцип:
|
||||
|
||||
```text
|
||||
camera detection and semantic class
|
||||
→ canonical E29 camera↔LiDAR association
|
||||
→ LiDAR metric support and range
|
||||
```
|
||||
|
||||
LiDAR не создаёт семантический класс. Camera-объект без квалифицированной
|
||||
LiDAR-поддержки остаётся camera-only объектом без выдуманной дальности.
|
||||
|
||||
## Неизменяемые входы
|
||||
|
||||
- E10 LiDAR pack:
|
||||
`e10-lidar-pack-576c994a6c814e2592dd6240ace3902a5db94843312c759a73ba0c9166157d2b`;
|
||||
- local surface:
|
||||
`k1-local-surface-23762244c8bdb97de26fb721ac957d7a00bc9a63571ac4cfa4be19c4effc7d55`;
|
||||
- camera candidate:
|
||||
`result-f4cebdea8a82698a5b8a65d2c3fbdb0428b88b9dc49fe45f8cb37d740ed83d02`;
|
||||
- accepted E29:
|
||||
`e29-camera-geometry-421a9d930638bef12cd5eb10979a477917fa4a389e655ed95f73ba4bd62e13dc`;
|
||||
- rig geometry:
|
||||
`ravnoves00-portable-k1/unbound-v1`.
|
||||
|
||||
Rig profile намеренно не содержит корпуса, трансформации LiDAR→body или
|
||||
физических размеров: RAVNOVES00 записан переносным K1 и не доказывает
|
||||
установку на транспортное средство.
|
||||
|
||||
## Метод
|
||||
|
||||
E53 потоково читает detector и E29 metadata, один раз материализует compressed
|
||||
source arrays и для каждого source-aligned кадра:
|
||||
|
||||
1. оставляет camera-классы `person`, `bicycle`, `car`, `motorcycle`, `bus`,
|
||||
`truck` с score не ниже 0.5;
|
||||
2. отбрасывает bbox больше 25% изображения;
|
||||
3. применяет без изменения профиль
|
||||
`camera-first-local-surface-validation/v1`;
|
||||
4. сохраняет camera semantic ownership;
|
||||
5. публикует LiDAR range только при связанном occupied support;
|
||||
6. измеряет adjacent-frame family+IoU continuity и source-bound overlap с E29;
|
||||
7. оставляет collision unavailable.
|
||||
|
||||
Сравнение с E29 и adjacent IoU не являются ground truth или persistent
|
||||
tracking truth.
|
||||
|
||||
## Отвергнутый первый проход
|
||||
|
||||
Первый immutable результат:
|
||||
|
||||
```text
|
||||
e53-camera-first-shadow-94eb57cce8dbfb0ea679486ed55aec32c2f38c93ae67e1c606787587cd3a0da5
|
||||
```
|
||||
|
||||
Он обработал все 4 489 кадров, но был отвергнут: p95 составил 210.7093 ms
|
||||
против заранее заданных 50 ms. Причина — обращение к compressed NPZ member
|
||||
`cloud_points_map` внутри frame loop, из-за чего массив 9.2 млн точек
|
||||
распаковывался повторно.
|
||||
|
||||
Исправление не меняло входы, пороги или ассоциацию: source arrays
|
||||
материализуются один раз до цикла.
|
||||
|
||||
## Принятый результат
|
||||
|
||||
```text
|
||||
e53-camera-first-shadow-e6f03cf8bfb15db86100239b060e13f914532618b7b99e811866e4e6a555186c
|
||||
```
|
||||
|
||||
| Измерение | Результат |
|
||||
| --- | ---: |
|
||||
| Кадры | 4 489 / 4 489 |
|
||||
| Кадры с LiDAR | 3 928 |
|
||||
| Кадры с camera candidates | 4 471 |
|
||||
| Принятые camera candidates | 48 944 |
|
||||
| Pathological large boxes отклонено | 946 |
|
||||
| Camera + LiDAR agree | 7 686 |
|
||||
| Camera-only | 34 940 |
|
||||
| Source unavailable | 6 318 |
|
||||
| Объекты с LiDAR range | 10 183 |
|
||||
| Доля объектов с range | 20.8054% |
|
||||
| E29 current family+IoU overlap | 18 199 / 19 625 = 92.7338% |
|
||||
| Adjacent family+IoU continuity | 31 476 / 48 933 = 64.3247% |
|
||||
| Frame latency p50 / p95 / max | 0.9881 / 1.7993 / 9.4521 ms |
|
||||
| Полный elapsed | 4 962.34 ms |
|
||||
| Peak-RSS growth | 36.8594 MiB |
|
||||
|
||||
Распределение camera candidates: 44 498 `car`, 2 341 `person`, 1 876
|
||||
`truck`, 142 `bicycle`, 66 `motorcycle`, 21 `bus`.
|
||||
|
||||
## Решение
|
||||
|
||||
Полный source-bound camera-first shadow технически принят:
|
||||
|
||||
- accounting и binding закрыты;
|
||||
- E29 association переиспользована;
|
||||
- latency и memory gate пройдены;
|
||||
- camera semantic ownership сохранён;
|
||||
- LiDAR-native class, false free space и collision не публикуются.
|
||||
|
||||
Mask R-CNN не принят как operational detector. Большое число camera candidates,
|
||||
64.32% adjacent continuity и 92.73% overlap с E29 требуют независимой
|
||||
source-bound quality review. E29 также не является ground truth.
|
||||
|
||||
Physical vehicle-body и LiDAR mount/extrinsic остаются отдельным измерительным
|
||||
gate. До их квалификации `recent-collision`, navigation, safety и commands
|
||||
остаются недоступны.
|
||||
@@ -0,0 +1,56 @@
|
||||
{
|
||||
"schema_version": "missioncore.e53-camera-first-shadow-profile/v1",
|
||||
"profile_id": "e53-ravnoves00-camera-first-full-shadow/v1",
|
||||
"expected": {
|
||||
"source_pack_id": "e10-lidar-pack-576c994a6c814e2592dd6240ace3902a5db94843312c759a73ba0c9166157d2b",
|
||||
"local_surface_model_id": "k1-local-surface-23762244c8bdb97de26fb721ac957d7a00bc9a63571ac4cfa4be19c4effc7d55",
|
||||
"detector_result_id": "result-f4cebdea8a82698a5b8a65d2c3fbdb0428b88b9dc49fe45f8cb37d740ed83d02",
|
||||
"baseline_e29_result_id": "e29-camera-geometry-421a9d930638bef12cd5eb10979a477917fa4a389e655ed95f73ba4bd62e13dc"
|
||||
},
|
||||
"camera_candidate": {
|
||||
"target_labels": [
|
||||
"person",
|
||||
"bicycle",
|
||||
"car",
|
||||
"motorcycle",
|
||||
"bus",
|
||||
"truck"
|
||||
],
|
||||
"label_groups": {
|
||||
"person": [
|
||||
"person"
|
||||
],
|
||||
"two-wheel": [
|
||||
"bicycle",
|
||||
"motorcycle"
|
||||
],
|
||||
"vehicle": [
|
||||
"car",
|
||||
"bus",
|
||||
"truck"
|
||||
]
|
||||
},
|
||||
"minimum_score": 0.5,
|
||||
"maximum_bbox_image_fraction": 0.25
|
||||
},
|
||||
"temporal": {
|
||||
"minimum_adjacent_family_iou": 0.25,
|
||||
"minimum_baseline_family_iou": 0.25,
|
||||
"interpretation": "adjacent-image-space-continuity-diagnostic-not-track-truth"
|
||||
},
|
||||
"acceptance": {
|
||||
"maximum_frame_latency_p95_ms": 50.0,
|
||||
"maximum_rss_growth_mib": 512.0,
|
||||
"require_complete_frame_accounting": true,
|
||||
"require_canonical_e29_association": true,
|
||||
"require_camera_semantic_ownership": true,
|
||||
"require_no_lidar_native_semantics": true,
|
||||
"require_no_false_free_space": true,
|
||||
"require_no_collision_authority": true,
|
||||
"require_upstream_immutability": true
|
||||
},
|
||||
"authority": {
|
||||
"commands_enabled": false,
|
||||
"navigation_or_safety_accepted": false
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,20 @@
|
||||
{
|
||||
"schema_version": "missioncore.rig-geometry/v1",
|
||||
"profile_id": "ravnoves00-portable-k1/unbound-v1",
|
||||
"rig_kind": "portable",
|
||||
"qualification": {
|
||||
"state": "unbound",
|
||||
"reason": "portable-k1-source-has-no-vehicle-body-or-measured-lidar-to-body-mount",
|
||||
"evidence_sha256": []
|
||||
},
|
||||
"coordinate_frames": {
|
||||
"lidar_frame": "k1-lidar",
|
||||
"body_frame": null,
|
||||
"body_from_lidar": null
|
||||
},
|
||||
"vehicle_body": null,
|
||||
"authority": {
|
||||
"commands_enabled": false,
|
||||
"navigation_or_safety_accepted": false
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,51 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Build the immutable full-source E53 camera-first shadow."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
from k1link.compute.e53_camera_first_shadow import (
|
||||
build_e53_camera_first_shadow,
|
||||
)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--source-pack", type=Path, required=True)
|
||||
parser.add_argument("--local-surface", type=Path, required=True)
|
||||
parser.add_argument("--detector-result", type=Path, required=True)
|
||||
parser.add_argument("--baseline-e29", type=Path, required=True)
|
||||
parser.add_argument("--rig-geometry", type=Path, required=True)
|
||||
parser.add_argument("--profile", type=Path, required=True)
|
||||
parser.add_argument("--output-root", type=Path, required=True)
|
||||
args = parser.parse_args()
|
||||
result = build_e53_camera_first_shadow(
|
||||
source_pack_root=args.source_pack,
|
||||
local_surface_root=args.local_surface,
|
||||
detector_result_root=args.detector_result,
|
||||
baseline_e29_root=args.baseline_e29,
|
||||
rig_geometry_path=args.rig_geometry,
|
||||
profile_path=args.profile,
|
||||
output_root=args.output_root,
|
||||
)
|
||||
print(
|
||||
json.dumps(
|
||||
{
|
||||
"result_id": result.result_id,
|
||||
"result_root": str(result.result_root),
|
||||
"accepted": result.report["acceptance"]["accepted"],
|
||||
"metrics": result.report["metrics"],
|
||||
"decision": result.report["decision"],
|
||||
},
|
||||
ensure_ascii=False,
|
||||
sort_keys=True,
|
||||
)
|
||||
)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,237 @@
|
||||
"""Versioned, fail-closed vehicle-body and sensor-mount geometry."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import math
|
||||
import re
|
||||
from collections.abc import Mapping
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Any, Final, Literal
|
||||
|
||||
import numpy as np
|
||||
|
||||
RIG_GEOMETRY_SCHEMA: Final = "missioncore.rig-geometry/v1"
|
||||
_SHA256 = re.compile(r"^[a-f0-9]{64}$")
|
||||
_SAFE_ID = re.compile(r"^[a-z0-9][a-z0-9._/-]{0,159}$")
|
||||
_AUTHORITY: Final = {
|
||||
"commands_enabled": False,
|
||||
"navigation_or_safety_accepted": False,
|
||||
}
|
||||
|
||||
|
||||
class RigGeometryError(ValueError):
|
||||
"""A rig geometry document is incomplete, ambiguous, or inconsistent."""
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class RigGeometry:
|
||||
"""Validated rig geometry plus its content identity."""
|
||||
|
||||
document: Mapping[str, Any]
|
||||
identity_sha256: str
|
||||
qualification_state: Literal["unbound", "measured", "qualified"]
|
||||
|
||||
@property
|
||||
def metric_body_geometry_available(self) -> bool:
|
||||
return self.qualification_state in {"measured", "qualified"}
|
||||
|
||||
@property
|
||||
def collision_geometry_qualified(self) -> bool:
|
||||
return self.qualification_state == "qualified"
|
||||
|
||||
def collision_contract(self) -> dict[str, object]:
|
||||
if self.collision_geometry_qualified:
|
||||
return {
|
||||
"state": "geometry-qualified",
|
||||
"geometry_profile_sha256": self.identity_sha256,
|
||||
"recent_collision_publishable": False,
|
||||
"reason": "collision-algorithm-and-independent-safety-gate-not-qualified",
|
||||
}
|
||||
return {
|
||||
"state": "unavailable",
|
||||
"geometry_profile_sha256": self.identity_sha256,
|
||||
"recent_collision_publishable": False,
|
||||
"reason": (
|
||||
"vehicle-body-and-lidar-mount-geometry-not-bound"
|
||||
if self.qualification_state == "unbound"
|
||||
else "vehicle-body-and-lidar-mount-geometry-not-qualified"
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
def load_rig_geometry(path: Path) -> RigGeometry:
|
||||
"""Read and validate one regular JSON geometry profile."""
|
||||
|
||||
source = path.expanduser().absolute()
|
||||
if source.is_symlink():
|
||||
raise RigGeometryError("rig geometry profile cannot be a symlink")
|
||||
resolved = source.resolve(strict=True)
|
||||
if not resolved.is_file():
|
||||
raise RigGeometryError("rig geometry profile must be a regular file")
|
||||
try:
|
||||
value = json.loads(resolved.read_text(encoding="utf-8"))
|
||||
except (OSError, json.JSONDecodeError) as exc:
|
||||
raise RigGeometryError("rig geometry profile is not valid JSON") from exc
|
||||
if not isinstance(value, dict):
|
||||
raise RigGeometryError("rig geometry profile must be an object")
|
||||
return parse_rig_geometry(value)
|
||||
|
||||
|
||||
def parse_rig_geometry(value: Mapping[str, Any]) -> RigGeometry:
|
||||
"""Validate an in-memory v1 profile without inferring physical values."""
|
||||
|
||||
document = dict(value)
|
||||
profile_id = document.get("profile_id")
|
||||
rig_kind = document.get("rig_kind")
|
||||
qualification = _mapping(document.get("qualification"), "qualification")
|
||||
frames = _mapping(document.get("coordinate_frames"), "coordinate_frames")
|
||||
state = qualification.get("state")
|
||||
reason = qualification.get("reason")
|
||||
evidence_ids = qualification.get("evidence_sha256")
|
||||
if (
|
||||
document.get("schema_version") != RIG_GEOMETRY_SCHEMA
|
||||
or not isinstance(profile_id, str)
|
||||
or _SAFE_ID.fullmatch(profile_id) is None
|
||||
or rig_kind not in {"portable", "vehicle-mounted"}
|
||||
or state not in {"unbound", "measured", "qualified"}
|
||||
or not isinstance(reason, str)
|
||||
or not reason.strip()
|
||||
or len(reason) > 240
|
||||
or not isinstance(evidence_ids, list)
|
||||
or any(
|
||||
not isinstance(item, str) or _SHA256.fullmatch(item) is None
|
||||
for item in evidence_ids
|
||||
)
|
||||
or len(set(evidence_ids)) != len(evidence_ids)
|
||||
or document.get("authority") != _AUTHORITY
|
||||
):
|
||||
raise RigGeometryError("rig geometry profile identity is invalid")
|
||||
|
||||
lidar_frame = frames.get("lidar_frame")
|
||||
body_frame = frames.get("body_frame")
|
||||
body_from_lidar = frames.get("body_from_lidar")
|
||||
body = document.get("vehicle_body")
|
||||
if not isinstance(lidar_frame, str) or _SAFE_ID.fullmatch(lidar_frame) is None:
|
||||
raise RigGeometryError("rig geometry LiDAR frame is invalid")
|
||||
|
||||
if state == "unbound":
|
||||
if (
|
||||
rig_kind != "portable"
|
||||
or body_frame is not None
|
||||
or body_from_lidar is not None
|
||||
or body is not None
|
||||
or evidence_ids
|
||||
):
|
||||
raise RigGeometryError("unbound rig geometry must not contain physical values")
|
||||
else:
|
||||
if (
|
||||
rig_kind != "vehicle-mounted"
|
||||
or not isinstance(body_frame, str)
|
||||
or _SAFE_ID.fullmatch(body_frame) is None
|
||||
):
|
||||
raise RigGeometryError("mounted rig body frame is invalid")
|
||||
_validate_transform(
|
||||
_mapping(body_from_lidar, "body_from_lidar"),
|
||||
body_frame=body_frame,
|
||||
lidar_frame=lidar_frame,
|
||||
)
|
||||
_validate_body(_mapping(body, "vehicle_body"), body_frame=body_frame)
|
||||
if not evidence_ids:
|
||||
raise RigGeometryError("measured rig geometry requires measurement evidence")
|
||||
uncertainty = _mapping(
|
||||
qualification.get("uncertainty"),
|
||||
"qualification uncertainty",
|
||||
)
|
||||
_positive_number(
|
||||
uncertainty.get("translation_1sigma_m"),
|
||||
"translation uncertainty",
|
||||
)
|
||||
_positive_number(
|
||||
uncertainty.get("rotation_1sigma_deg"),
|
||||
"rotation uncertainty",
|
||||
)
|
||||
_positive_number(
|
||||
uncertainty.get("body_dimension_1sigma_m"),
|
||||
"body-dimension uncertainty",
|
||||
)
|
||||
if state == "qualified":
|
||||
qualification_method = qualification.get("qualification_method")
|
||||
if (
|
||||
not isinstance(qualification_method, str)
|
||||
or not qualification_method.strip()
|
||||
or len(qualification_method) > 240
|
||||
):
|
||||
raise RigGeometryError("qualified rig geometry needs a method")
|
||||
|
||||
canonical = _canonical_json(document)
|
||||
return RigGeometry(
|
||||
document=document,
|
||||
identity_sha256=hashlib.sha256(canonical).hexdigest(),
|
||||
qualification_state=state,
|
||||
)
|
||||
|
||||
|
||||
def _validate_transform(
|
||||
value: Mapping[str, Any],
|
||||
*,
|
||||
body_frame: str,
|
||||
lidar_frame: str,
|
||||
) -> None:
|
||||
translation = _vector(value.get("translation_m"), 3, "mount translation")
|
||||
quaternion = _vector(value.get("quaternion_xyzw"), 4, "mount quaternion")
|
||||
if (
|
||||
value.get("from_frame") != lidar_frame
|
||||
or value.get("to_frame") != body_frame
|
||||
or not math.isclose(float(np.linalg.norm(quaternion)), 1.0, abs_tol=1e-6)
|
||||
or np.linalg.norm(translation) > 20.0
|
||||
):
|
||||
raise RigGeometryError("rig mount transform is invalid")
|
||||
|
||||
|
||||
def _validate_body(value: Mapping[str, Any], *, body_frame: str) -> None:
|
||||
minimum = _vector(value.get("minimum_xyz_m"), 3, "body minimum")
|
||||
maximum = _vector(value.get("maximum_xyz_m"), 3, "body maximum")
|
||||
if (
|
||||
value.get("frame") != body_frame
|
||||
or value.get("shape") != "axis-aligned-box"
|
||||
or np.any(maximum <= minimum)
|
||||
or np.any(maximum - minimum > 30.0)
|
||||
):
|
||||
raise RigGeometryError("vehicle body envelope is invalid")
|
||||
|
||||
|
||||
def _mapping(value: object, label: str) -> Mapping[str, Any]:
|
||||
if not isinstance(value, Mapping):
|
||||
raise RigGeometryError(f"{label} must be an object")
|
||||
return value
|
||||
|
||||
|
||||
def _vector(value: object, length: int, label: str) -> np.ndarray:
|
||||
if not isinstance(value, list) or len(value) != length:
|
||||
raise RigGeometryError(f"{label} is invalid")
|
||||
array = np.asarray(value, dtype=np.float64)
|
||||
if not np.isfinite(array).all():
|
||||
raise RigGeometryError(f"{label} is invalid")
|
||||
return array
|
||||
|
||||
|
||||
def _positive_number(value: object, label: str) -> float:
|
||||
if not isinstance(value, (int, float)) or isinstance(value, bool):
|
||||
raise RigGeometryError(f"{label} is invalid")
|
||||
number = float(value)
|
||||
if not math.isfinite(number) or number <= 0.0:
|
||||
raise RigGeometryError(f"{label} is invalid")
|
||||
return number
|
||||
|
||||
|
||||
def _canonical_json(value: object) -> bytes:
|
||||
return json.dumps(
|
||||
value,
|
||||
ensure_ascii=False,
|
||||
sort_keys=True,
|
||||
separators=(",", ":"),
|
||||
allow_nan=False,
|
||||
).encode()
|
||||
@@ -138,6 +138,65 @@ class _GeometryClusterSupport:
|
||||
occupied_source_indices: IntArray
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class CameraGeometryFrameEvaluation:
|
||||
"""One camera-owned semantic frame with LiDAR-owned metric association."""
|
||||
|
||||
semantic_observations: tuple[dict[str, object], ...]
|
||||
geometry_only_occupied: tuple[dict[str, object], ...]
|
||||
claimed_source_indices: frozenset[int]
|
||||
|
||||
|
||||
def evaluate_camera_geometry_frame(
|
||||
*,
|
||||
objects: Iterable[Mapping[str, Any]],
|
||||
projected: ProjectedPointCloud | None,
|
||||
frame_points_map: FloatArray,
|
||||
point_class: npt.NDArray[np.uint8],
|
||||
point_height_m: npt.NDArray[np.float32],
|
||||
sensor_position_map: FloatArray,
|
||||
source_available: bool,
|
||||
surface_valid: bool,
|
||||
profile: CameraGeometryFusionProfile = DEFAULT_CAMERA_GEOMETRY_FUSION_PROFILE,
|
||||
include_geometry_only: bool = True,
|
||||
) -> CameraGeometryFrameEvaluation:
|
||||
"""Apply the canonical E29 association without changing semantic ownership."""
|
||||
|
||||
supports = tuple(
|
||||
_semantic_support(
|
||||
item,
|
||||
projected=projected,
|
||||
frame_points_map=frame_points_map,
|
||||
point_class=point_class,
|
||||
point_height_m=point_height_m,
|
||||
source_available=source_available,
|
||||
surface_valid=surface_valid,
|
||||
profile=profile,
|
||||
)
|
||||
for item in objects
|
||||
)
|
||||
claimed = frozenset(_claimed_indices(supports))
|
||||
geometry = (
|
||||
tuple(
|
||||
_geometry_clusters(
|
||||
points_map=frame_points_map,
|
||||
point_class=point_class,
|
||||
point_height_m=point_height_m,
|
||||
sensor_position_map=sensor_position_map,
|
||||
claimed_source_indices=set(claimed),
|
||||
profile=profile,
|
||||
)
|
||||
)
|
||||
if include_geometry_only
|
||||
else ()
|
||||
)
|
||||
return CameraGeometryFrameEvaluation(
|
||||
semantic_observations=tuple(support.document for support in supports),
|
||||
geometry_only_occupied=geometry,
|
||||
claimed_source_indices=claimed,
|
||||
)
|
||||
|
||||
|
||||
def build_camera_geometry_fusion(
|
||||
*,
|
||||
fusion_frames_path: Path,
|
||||
@@ -229,7 +288,6 @@ def build_camera_geometry_fusion(
|
||||
frame_heights = point_height[start:end]
|
||||
source_available = bool(arrays["sample_available"][expected_frame_index])
|
||||
surface_valid = bool(surface.arrays["frame_valid"][expected_frame_index])
|
||||
semantic_supports: list[_SemanticSupport] = []
|
||||
if source_available and surface_valid:
|
||||
position = arrays["pose_positions_map"][expected_frame_index]
|
||||
orientation = arrays["pose_quaternions_map_from_lidar"][expected_frame_index]
|
||||
@@ -251,40 +309,32 @@ def build_camera_geometry_fusion(
|
||||
else:
|
||||
projected = None
|
||||
|
||||
for item in frame["objects"]:
|
||||
support = _semantic_support(
|
||||
item,
|
||||
projected=projected,
|
||||
frame_points_map=frame_points,
|
||||
point_class=frame_classes,
|
||||
point_height_m=frame_heights,
|
||||
source_available=source_available,
|
||||
surface_valid=surface_valid,
|
||||
profile=profile,
|
||||
)
|
||||
semantic_supports.append(support)
|
||||
semantic_observations += 1
|
||||
geometry_status = str(support.document["geometry_status"])
|
||||
group = str(support.document["association_group"])
|
||||
motion_status = str(support.document["motion_status"])
|
||||
status_counts[geometry_status] += 1
|
||||
group_status_counts[(group, geometry_status)] += 1
|
||||
motion_status_counts[(motion_status, geometry_status)] += 1
|
||||
if support.document["semantic_current"] is True:
|
||||
semantic_current_observations += 1
|
||||
|
||||
claimed = _claimed_indices(semantic_supports)
|
||||
geometry_clusters = _geometry_clusters(
|
||||
points_map=frame_points,
|
||||
evaluation = evaluate_camera_geometry_frame(
|
||||
objects=frame["objects"],
|
||||
projected=projected,
|
||||
frame_points_map=frame_points,
|
||||
point_class=frame_classes,
|
||||
point_height_m=frame_heights,
|
||||
sensor_position_map=np.asarray(
|
||||
arrays["pose_positions_map"][expected_frame_index],
|
||||
dtype=np.float64,
|
||||
),
|
||||
claimed_source_indices=claimed,
|
||||
source_available=source_available,
|
||||
surface_valid=surface_valid,
|
||||
profile=profile,
|
||||
)
|
||||
for support_document in evaluation.semantic_observations:
|
||||
semantic_observations += 1
|
||||
geometry_status = str(support_document["geometry_status"])
|
||||
group = str(support_document["association_group"])
|
||||
motion_status = str(support_document["motion_status"])
|
||||
status_counts[geometry_status] += 1
|
||||
group_status_counts[(group, geometry_status)] += 1
|
||||
motion_status_counts[(motion_status, geometry_status)] += 1
|
||||
if support_document["semantic_current"] is True:
|
||||
semantic_current_observations += 1
|
||||
|
||||
geometry_clusters = list(evaluation.geometry_only_occupied)
|
||||
if geometry_clusters:
|
||||
frames_with_geometry_only += 1
|
||||
geometry_only_clusters_per_frame.append(float(len(geometry_clusters)))
|
||||
@@ -308,7 +358,7 @@ def build_camera_geometry_fusion(
|
||||
"session_seconds": frame["session_seconds"],
|
||||
"source_available": source_available,
|
||||
"local_surface_valid": surface_valid,
|
||||
"semantic_observations": [support.document for support in semantic_supports],
|
||||
"semantic_observations": list(evaluation.semantic_observations),
|
||||
"geometry_only_occupied": geometry_clusters,
|
||||
"policy": {
|
||||
"camera_owns_semantics": True,
|
||||
@@ -741,7 +791,9 @@ def _spatial_cluster(
|
||||
return rows[np.asarray(selected, dtype=np.int64)]
|
||||
|
||||
|
||||
def _projection_profile(source: E10LidarFieldSource) -> Kb4ProjectionProfile:
|
||||
def projection_profile_from_source(source: E10LidarFieldSource) -> Kb4ProjectionProfile:
|
||||
"""Build the source-bound factory projection used by E29 association."""
|
||||
|
||||
identity_projection = source.identity.get("projection")
|
||||
if not isinstance(identity_projection, Mapping):
|
||||
raise SemanticGeometryFusionError("source projection identity is missing")
|
||||
@@ -771,6 +823,10 @@ def _projection_profile(source: E10LidarFieldSource) -> Kb4ProjectionProfile:
|
||||
)
|
||||
|
||||
|
||||
def _projection_profile(source: E10LidarFieldSource) -> Kb4ProjectionProfile:
|
||||
return projection_profile_from_source(source)
|
||||
|
||||
|
||||
def _fusion_frame(
|
||||
line: str,
|
||||
*,
|
||||
|
||||
@@ -0,0 +1,96 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
from k1link.compute.e53_camera_first_shadow import (
|
||||
_Profile,
|
||||
adjacent_family_matches,
|
||||
select_candidate_detections,
|
||||
)
|
||||
|
||||
|
||||
def _profile() -> _Profile:
|
||||
return _Profile(
|
||||
raw={},
|
||||
expected_source_pack_id="source",
|
||||
expected_local_surface_id="surface",
|
||||
expected_detector_result_id="detector",
|
||||
expected_baseline_e29_id="baseline",
|
||||
target_labels=frozenset({"person", "car", "truck"}),
|
||||
label_groups={
|
||||
"person": ("person",),
|
||||
"vehicle": ("car", "truck"),
|
||||
},
|
||||
minimum_detector_score=0.5,
|
||||
maximum_bbox_image_fraction=0.25,
|
||||
adjacent_family_iou=0.25,
|
||||
baseline_family_iou=0.25,
|
||||
maximum_latency_p95_ms=50.0,
|
||||
maximum_rss_growth_mib=512.0,
|
||||
)
|
||||
|
||||
|
||||
def test_candidate_filter_keeps_camera_semantics_and_rejects_large_box() -> None:
|
||||
selected, counts = select_candidate_detections(
|
||||
[
|
||||
{
|
||||
"instance_id": 1,
|
||||
"label": "person",
|
||||
"score": 0.99,
|
||||
"box_xyxy": [100.0, 100.0, 180.0, 300.0],
|
||||
"mask_pixels": 4000,
|
||||
},
|
||||
{
|
||||
"instance_id": 2,
|
||||
"label": "car",
|
||||
"score": 0.95,
|
||||
"box_xyxy": [0.0, 0.0, 800.0, 600.0],
|
||||
"mask_pixels": 480000,
|
||||
},
|
||||
{
|
||||
"instance_id": 3,
|
||||
"label": "laptop",
|
||||
"score": 0.98,
|
||||
"box_xyxy": [10.0, 10.0, 100.0, 100.0],
|
||||
},
|
||||
],
|
||||
width=800,
|
||||
height=600,
|
||||
profile=_profile(),
|
||||
)
|
||||
|
||||
assert len(selected) == 1
|
||||
assert selected[0]["label"] == "person"
|
||||
assert selected[0]["association_group"] == "person"
|
||||
assert selected[0]["cuboid_status"] == "camera-candidate-no-cuboid"
|
||||
assert counts["pathological_large_box"] == 1
|
||||
assert counts["non_target"] == 1
|
||||
|
||||
|
||||
def test_adjacent_matching_is_family_aware_and_one_to_one() -> None:
|
||||
previous: list[dict[str, Any]] = [
|
||||
{
|
||||
"association_group": "vehicle",
|
||||
"bbox_xyxy": [10.0, 10.0, 110.0, 110.0],
|
||||
},
|
||||
{
|
||||
"association_group": "person",
|
||||
"bbox_xyxy": [200.0, 20.0, 240.0, 120.0],
|
||||
},
|
||||
]
|
||||
current: list[dict[str, Any]] = [
|
||||
{
|
||||
"association_group": "vehicle",
|
||||
"bbox_xyxy": [12.0, 12.0, 112.0, 112.0],
|
||||
},
|
||||
{
|
||||
"association_group": "vehicle",
|
||||
"bbox_xyxy": [14.0, 14.0, 114.0, 114.0],
|
||||
},
|
||||
{
|
||||
"association_group": "person",
|
||||
"bbox_xyxy": [202.0, 22.0, 242.0, 122.0],
|
||||
},
|
||||
]
|
||||
|
||||
assert adjacent_family_matches(previous, current, minimum_iou=0.25) == 2
|
||||
@@ -0,0 +1,95 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from k1link.compute.rig_geometry import (
|
||||
RIG_GEOMETRY_SCHEMA,
|
||||
RigGeometryError,
|
||||
parse_rig_geometry,
|
||||
)
|
||||
|
||||
|
||||
def _unbound() -> dict[str, object]:
|
||||
return {
|
||||
"schema_version": RIG_GEOMETRY_SCHEMA,
|
||||
"profile_id": "portable-k1/unbound-v1",
|
||||
"rig_kind": "portable",
|
||||
"qualification": {
|
||||
"state": "unbound",
|
||||
"reason": "no measured mount",
|
||||
"evidence_sha256": [],
|
||||
},
|
||||
"coordinate_frames": {
|
||||
"lidar_frame": "k1-lidar",
|
||||
"body_frame": None,
|
||||
"body_from_lidar": None,
|
||||
},
|
||||
"vehicle_body": None,
|
||||
"authority": {
|
||||
"commands_enabled": False,
|
||||
"navigation_or_safety_accepted": False,
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def test_unbound_portable_rig_fails_closed_without_inventing_geometry() -> None:
|
||||
geometry = parse_rig_geometry(_unbound())
|
||||
|
||||
assert geometry.metric_body_geometry_available is False
|
||||
assert geometry.collision_geometry_qualified is False
|
||||
assert geometry.collision_contract()["state"] == "unavailable"
|
||||
assert geometry.collision_contract()["recent_collision_publishable"] is False
|
||||
|
||||
|
||||
def test_unbound_rig_rejects_hidden_physical_values() -> None:
|
||||
value = _unbound()
|
||||
value["coordinate_frames"] = {
|
||||
"lidar_frame": "k1-lidar",
|
||||
"body_frame": "vehicle-body",
|
||||
"body_from_lidar": None,
|
||||
}
|
||||
|
||||
with pytest.raises(RigGeometryError, match="must not contain physical values"):
|
||||
parse_rig_geometry(value)
|
||||
|
||||
|
||||
def test_measured_mount_does_not_grant_collision_authority() -> None:
|
||||
value = _unbound()
|
||||
value.update(
|
||||
{
|
||||
"profile_id": "measured-rig/v1",
|
||||
"rig_kind": "vehicle-mounted",
|
||||
"qualification": {
|
||||
"state": "measured",
|
||||
"reason": "bench measurement only",
|
||||
"evidence_sha256": ["a" * 64],
|
||||
"uncertainty": {
|
||||
"translation_1sigma_m": 0.01,
|
||||
"rotation_1sigma_deg": 0.2,
|
||||
"body_dimension_1sigma_m": 0.01,
|
||||
},
|
||||
},
|
||||
"coordinate_frames": {
|
||||
"lidar_frame": "k1-lidar",
|
||||
"body_frame": "vehicle-body",
|
||||
"body_from_lidar": {
|
||||
"from_frame": "k1-lidar",
|
||||
"to_frame": "vehicle-body",
|
||||
"translation_m": [0.0, 0.0, 1.0],
|
||||
"quaternion_xyzw": [0.0, 0.0, 0.0, 1.0],
|
||||
},
|
||||
},
|
||||
"vehicle_body": {
|
||||
"frame": "vehicle-body",
|
||||
"shape": "axis-aligned-box",
|
||||
"minimum_xyz_m": [-1.0, -0.5, -0.2],
|
||||
"maximum_xyz_m": [1.0, 0.5, 1.2],
|
||||
},
|
||||
}
|
||||
)
|
||||
|
||||
geometry = parse_rig_geometry(value)
|
||||
|
||||
assert geometry.metric_body_geometry_available is True
|
||||
assert geometry.collision_geometry_qualified is False
|
||||
assert geometry.collision_contract()["state"] == "unavailable"
|
||||
Reference in New Issue
Block a user