feat(perception): complete E32 full replay
This commit is contained in:
@@ -645,19 +645,26 @@ with a complete `missioncore.laboratory-method/v1` manifest.
|
||||
The current E30 AI-assisted engineering generation is
|
||||
`e30-engineering-generation-62a4fea10dea9b77f69ceac1af5bf0e4928d9c7716083c22258a03670fe5bd4f`.
|
||||
It covers all `486` selected items: `403` confirmed, `81` corrected and `2`
|
||||
retained as `insufficient-evidence` human exceptions. The earlier five-case
|
||||
generation and its human draft remain immutable history. Frames 1213, 162 and
|
||||
1823 are now automatic engineering outcomes; only geometry frames 2622 and
|
||||
4147 require a human decision. It found no systematic camera↔LiDAR
|
||||
registration failure in the review set. The dominant actionable signals are
|
||||
detector errors (`110`, including barrier false positives and 21 missed
|
||||
class-bearing objects), sparse occupied support (`98`), held-track time
|
||||
freshness (`92`) and self points (`10`). This is diagnostic engineering
|
||||
evidence, not human ground truth, detection accuracy or safety acceptance. The
|
||||
exception budget is 1% only for diagnostic residual work and never overrides a
|
||||
repeated-cause or high-impact blocker. The current queue is `2 / 486` (`0.41%`);
|
||||
E31 remains blocked until those two decisions freeze the E30 minimum correction
|
||||
set.
|
||||
initially routed human exceptions. The immutable human generation
|
||||
`e30-review-generation-7982a882558d0be690b4c7092e328c080bfcbf52478a220452be7e887a588250`
|
||||
closes both: geometry `2622:4` is background/noise and `4147:3` is a real
|
||||
occupied object. The earlier five-case generation and drafts remain immutable
|
||||
history. E30 found no systematic camera↔LiDAR registration failure in the
|
||||
review set. The dominant actionable signals remain detector errors (`110`,
|
||||
including barrier false positives and 21 missed class-bearing objects), sparse
|
||||
occupied support (`98`), held-track time freshness (`92`) and self points
|
||||
(`10`). This is diagnostic engineering evidence, not human ground truth,
|
||||
detection accuracy or safety acceptance.
|
||||
|
||||
E31 accepted immutable source-scoped profile
|
||||
`e31-source-qualification-b2460a5eb143688c7eea6821b2277e13aea79868abe81d83f7e78548c119159a`.
|
||||
E32 then produced
|
||||
`e32-track-geometry-a14ca0e7fb3850ca0dfa3c41634e1b490a2d58ab74d101afc6d6921fbdb0e6fd`:
|
||||
all 4,489 E29 frames reproduce exactly, all source/object/point claims close,
|
||||
the self and exact correction set is applied without a generic geometry mask,
|
||||
and exclusive PointSlab ownership is enforced. Conflict count remains `38`;
|
||||
the result is accepted only as the source-scoped diagnostic/shadow input for
|
||||
E33 and is not promoted as a detector-accuracy or staleness improvement.
|
||||
|
||||
Execution is strictly sequential through E33: E30 determines what E31 is
|
||||
allowed to change; E31 determines the E32 profile; E32 determines the E33
|
||||
|
||||
@@ -171,9 +171,31 @@ RAVNOVES00 diagnostic binding and does not claim that host arrival is hardware
|
||||
firing time. The factory KB4 identity is exact, but a measured
|
||||
calibration-target residual and physical body/mount dimensions are
|
||||
unavailable. The accepted profile is therefore source-session scoped and
|
||||
cannot transfer to another mount. A6/E32 full replay is the next critical-path
|
||||
implementation and must bind every published frame to both `TrackGeometry v1`
|
||||
and the accepted E31 profile.
|
||||
cannot transfer to another mount.
|
||||
|
||||
A6/E32 is complete in immutable result
|
||||
`e32-track-geometry-a14ca0e7fb3850ca0dfa3c41634e1b490a2d58ab74d101afc6d6921fbdb0e6fd`.
|
||||
It reproduces all 4,489 E29 frames exactly before correction and binds every
|
||||
published frame to `TrackGeometry v1` plus the accepted E31 profile. All
|
||||
20,513 semantic observations, 21,321 geometry clusters and 2,125,813 qualified
|
||||
point claims close without hidden loss. The replay removes 385 source-scoped
|
||||
self observations and three evidenced geometry clusters, withholds 1,558
|
||||
unqualified ranges and resolves 4,461 overlapping point claims. Of 709
|
||||
arbitrated semantic observations, 563 retain `agree` and 146 conservatively
|
||||
become `unknown`. The 38 E29 conflicts remain 38; A6 is accepted as a
|
||||
diagnostic/shadow contract, not as detector-accuracy improvement. A7/E33
|
||||
recorded-source-paced worker execution is now the critical path.
|
||||
|
||||
- [x] Reproduce all 4,489 immutable E29 frames with the exact frozen profile
|
||||
before applying E31/E30 changes.
|
||||
- [x] Apply only the admitted semantic self-mask, two exact geometry
|
||||
corrections and the complete A3 human exception set.
|
||||
- [x] Enforce exclusive PointSlab ownership and journal every arbitration,
|
||||
excluded claim and conservative state transition.
|
||||
- [x] Compare E29/E32 by status, class, range, 60-second scene interval and
|
||||
cause while preserving source availability and unknown/free-space policy.
|
||||
- [x] Add a digest-bound compact binary PointSlab encoding and strict
|
||||
TrackGeometryFrame reconstruction/validation.
|
||||
|
||||
### A3 residual and human-exception policy
|
||||
|
||||
|
||||
@@ -0,0 +1,92 @@
|
||||
# ADR 0025 — E32 full replay and exclusive point ownership
|
||||
|
||||
Date: 2026-07-27
|
||||
|
||||
Status: accepted for source-scoped diagnostic/shadow use
|
||||
|
||||
## Context
|
||||
|
||||
A5 defined `TrackGeometry v1` and `PointSlab`, but E29 did not retain the exact
|
||||
frame-local point indices that produced each semantic and geometry result.
|
||||
E32 therefore had to recover those indices without changing the frozen E29
|
||||
thresholds, apply only the accepted E31/E30 corrections and prove that the
|
||||
translated result had no hidden source loss.
|
||||
|
||||
The accepted E31 profile is limited to the immutable RAVNOVES00 source session.
|
||||
It admits one normalized person self-mask, rejects a generic geometry mask and
|
||||
binds two exact geometry self-corrections. The A3 human generation additionally
|
||||
rejects geometry cluster `2622:4` as background/noise and retains `4147:3` as a
|
||||
real occupied object.
|
||||
|
||||
## Decision
|
||||
|
||||
1. E32 recomputes E29 support indices using the exact E29 profile and immutable
|
||||
camera, LiDAR, pose and local-surface inputs.
|
||||
2. Every recomputed frame must equal the stored E29 frame before any E31/E30
|
||||
correction is applied. A difference in any observation, cluster, metric,
|
||||
policy or frame binding fails the replay closed.
|
||||
3. E32 applies only:
|
||||
- the admitted E31 person self-mask by normalized bbox centre;
|
||||
- the two exact E31 geometry correction item locators;
|
||||
- the complete immutable A3 human geometry dispositions.
|
||||
4. The rejected generic E31 geometry mask is never applied. E29 thresholds are
|
||||
never retuned and excluded points are never reclassified as free space.
|
||||
5. `PointSlab` grants one owner to each published source point. When two E29
|
||||
camera observations claim the same point, E32 uses a deterministic
|
||||
non-threshold policy: smallest bbox, then higher detector score, lower track
|
||||
id and lower observation ordinal. A track that retains points stays `agree`;
|
||||
a track that loses all exclusive points becomes explicit `unknown`.
|
||||
6. Camera-only and conflict observations cannot publish an E29 range derived
|
||||
from support that failed the E29 qualification threshold. Their semantic
|
||||
observation remains, but `metric_basis=unavailable` and `range_m=null`.
|
||||
7. A held observation is published only when an earlier current observation
|
||||
for the same track exists in the replay. A held observation without such
|
||||
provenance is explicitly journalled and omitted instead of inventing
|
||||
`held_from_frame_index`.
|
||||
8. Compact storage uses deterministic `.npy` arrays for frame offsets,
|
||||
frame-local source indices, map-frame float32 points and local owner
|
||||
indices. The JSONL frame record retains the complete TrackGeometry table and
|
||||
PointSlab reference. The public reader reconstructs and validates the exact
|
||||
`TrackGeometryFrame`.
|
||||
9. Every excluded or altered E29 product is written to
|
||||
`e29-e32-changes.jsonl` with its source locator, before/after state, reason,
|
||||
decision identity when applicable and affected claim count.
|
||||
10. E32 remains diagnostic. It grants no command, navigation, traversability,
|
||||
free-space or safety authority and does not modify the persistent map.
|
||||
|
||||
## Accepted result
|
||||
|
||||
The immutable result is
|
||||
`e32-track-geometry-a14ca0e7fb3850ca0dfa3c41634e1b490a2d58ab74d101afc6d6921fbdb0e6fd`.
|
||||
|
||||
- 4,489 / 4,489 E29 frames reproduce exactly.
|
||||
- Source availability remains 3,928 available and 561 unavailable frames.
|
||||
- 20,513 semantic observations close as 20,119 published, 385 source-scoped
|
||||
self-mask exclusions and 9 held observations without prior-current
|
||||
provenance.
|
||||
- 21,321 geometry clusters close as 21,318 published, two exact E31
|
||||
self-corrections and one A3 human background/noise exclusion. The A3
|
||||
object-present cluster remains published.
|
||||
- 2,125,813 E29 qualified point claims close as 2,119,302 published rows,
|
||||
2,050 explicitly excluded claims and 4,461 overlapping claims removed by
|
||||
ownership arbitration.
|
||||
- 709 semantic observations required point-ownership arbitration: 563 retained
|
||||
`agree`; 146 became `unknown`.
|
||||
- 1,558 ranges backed only by unqualified semantic support are withheld.
|
||||
- E29/E32 conflict count remains 38 and no conflict observation changes. E32
|
||||
is not presented as a detector-accuracy improvement.
|
||||
- `agree` changes from 6,341 to 6,195, `single-source-camera` from 13,246 to
|
||||
12,861, `unknown` from 888 to 1,025 and geometry-only from 21,321 to 21,318.
|
||||
|
||||
## Consequences
|
||||
|
||||
- A6 is complete as a reproducible, source-scoped TrackGeometry input for the
|
||||
A7/E33 recorded-source-paced worker shadow.
|
||||
- E33 receives a closed ownership and source-accounting contract instead of
|
||||
ambiguous E29 point claims.
|
||||
- The unchanged 38 conflicts and retained held observations remain perception
|
||||
limitations. Runtime qualification must not describe them as corrected.
|
||||
- The 146 conservative `agree → unknown` transitions are an intentional
|
||||
consequence of exclusive ownership, not threshold degradation.
|
||||
- A second source or changed mount must pass a new qualification; this E32
|
||||
result cannot be transferred by assumption.
|
||||
@@ -17,6 +17,16 @@ from .e31_source_qualification import (
|
||||
E31SourceQualificationProfile,
|
||||
build_e31_source_qualification,
|
||||
)
|
||||
from .e32_track_geometry_replay import (
|
||||
E32_TRACK_GEOMETRY_RECORD_SCHEMA,
|
||||
E32_TRACK_GEOMETRY_REPLAY_SCHEMA,
|
||||
E32_TRACK_GEOMETRY_REPORT_SCHEMA,
|
||||
E32TrackGeometryReplay,
|
||||
E32TrackGeometryReplayError,
|
||||
build_e32_track_geometry_replay,
|
||||
e32_track_geometry_frame,
|
||||
read_e32_track_geometry_replay,
|
||||
)
|
||||
from .evaluation_pack import (
|
||||
ANNOTATION_CONTRACT_SCHEMA,
|
||||
EVALUATION_PACK_SCHEMA,
|
||||
@@ -354,6 +364,14 @@ __all__ = [
|
||||
"assess_lidar_profile",
|
||||
"build_lidar_replay_pack_v2",
|
||||
"build_e31_source_qualification",
|
||||
"build_e32_track_geometry_replay",
|
||||
"read_e32_track_geometry_replay",
|
||||
"e32_track_geometry_frame",
|
||||
"E32_TRACK_GEOMETRY_REPLAY_SCHEMA",
|
||||
"E32_TRACK_GEOMETRY_REPORT_SCHEMA",
|
||||
"E32_TRACK_GEOMETRY_RECORD_SCHEMA",
|
||||
"E32TrackGeometryReplay",
|
||||
"E32TrackGeometryReplayError",
|
||||
"build_lidar_ground_annotation_template",
|
||||
"build_lidar_ground_benchmark",
|
||||
"build_k1_local_surface",
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,347 @@
|
||||
"""Compact, strict storage adapter for E32 TrackGeometry frames."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import math
|
||||
from collections.abc import Mapping
|
||||
from pathlib import Path
|
||||
from typing import Any, Final, cast
|
||||
|
||||
import numpy as np
|
||||
import numpy.typing as npt
|
||||
|
||||
from .track_geometry import (
|
||||
POINT_SLAB_SCHEMA,
|
||||
TRACK_GEOMETRY_FRAME_SCHEMA,
|
||||
PointSlab,
|
||||
TrackGeometry,
|
||||
TrackGeometryFrame,
|
||||
TrackGeometrySourceBinding,
|
||||
)
|
||||
|
||||
E32_TRACK_GEOMETRY_RECORD_SCHEMA: Final = "missioncore.e32-track-geometry-record/v1"
|
||||
E32_POINT_SLAB_REFERENCE_SCHEMA: Final = "missioncore.e32-point-slab-reference/v1"
|
||||
|
||||
E32_FRAMES_NAME: Final = "track-geometry-frames.jsonl"
|
||||
E32_FRAME_OFFSETS_NAME: Final = "frame-point-offsets.npy"
|
||||
E32_SOURCE_INDICES_NAME: Final = "point-source-indices.npy"
|
||||
E32_POINTS_NAME: Final = "point-coordinates-map-f32.npy"
|
||||
E32_OWNER_INDICES_NAME: Final = "point-owner-indices.npy"
|
||||
|
||||
Int64Array = npt.NDArray[np.int64]
|
||||
UInt32Array = npt.NDArray[np.uint32]
|
||||
|
||||
|
||||
class E32TrackGeometryStorageError(ValueError):
|
||||
"""Compact E32 storage no longer satisfies the TrackGeometry contract."""
|
||||
|
||||
|
||||
def write_point_storage(
|
||||
*,
|
||||
staging: Path,
|
||||
frame_offsets: Int64Array,
|
||||
source_indices: Int64Array,
|
||||
points: npt.NDArray[np.float32],
|
||||
owner_indices: UInt32Array,
|
||||
) -> None:
|
||||
"""Write deterministic non-pickle arrays for the compact frame stream."""
|
||||
|
||||
_save_npy(staging / E32_FRAME_OFFSETS_NAME, frame_offsets)
|
||||
_save_npy(staging / E32_SOURCE_INDICES_NAME, source_indices)
|
||||
_save_npy(staging / E32_POINTS_NAME, points)
|
||||
_save_npy(staging / E32_OWNER_INDICES_NAME, owner_indices)
|
||||
|
||||
|
||||
def validate_storage(
|
||||
*,
|
||||
artifacts: Mapping[str, Path],
|
||||
identity: Mapping[str, object],
|
||||
) -> None:
|
||||
"""Reconstruct and validate every persisted TrackGeometry frame."""
|
||||
|
||||
frame_count = _positive_int(identity.get("frame_count"), "E32 frame count")
|
||||
frame_offsets, source_indices, points, owner_indices = load_point_storage(
|
||||
artifacts,
|
||||
frame_count=frame_count,
|
||||
)
|
||||
binding = TrackGeometrySourceBinding.from_dict(
|
||||
identity.get("track_geometry_binding")
|
||||
)
|
||||
observed_frames = 0
|
||||
with artifacts["track-geometry-frames"].open("r", encoding="utf-8") as stream:
|
||||
for expected_frame_index, line in enumerate(stream):
|
||||
if expected_frame_index >= frame_count:
|
||||
raise E32TrackGeometryStorageError(
|
||||
"E32 frame stream has extra rows"
|
||||
)
|
||||
value = record(line, expected_frame_index=expected_frame_index)
|
||||
frame_from_record(
|
||||
record_value=value,
|
||||
binding=binding,
|
||||
frame_offsets=frame_offsets,
|
||||
source_indices=source_indices,
|
||||
points=points,
|
||||
owner_indices=owner_indices,
|
||||
)
|
||||
observed_frames += 1
|
||||
if observed_frames != frame_count:
|
||||
raise E32TrackGeometryStorageError("E32 frame stream is incomplete")
|
||||
|
||||
|
||||
def load_point_storage(
|
||||
artifacts: Mapping[str, Path],
|
||||
*,
|
||||
frame_count: int,
|
||||
) -> tuple[
|
||||
Int64Array,
|
||||
Int64Array,
|
||||
npt.NDArray[np.float32],
|
||||
UInt32Array,
|
||||
]:
|
||||
"""Open the four digest-verified E32 arrays as read-only memory maps."""
|
||||
|
||||
try:
|
||||
frame_offsets = np.load(
|
||||
artifacts["frame-point-offsets"],
|
||||
allow_pickle=False,
|
||||
mmap_mode="r",
|
||||
)
|
||||
source_indices = np.load(
|
||||
artifacts["point-source-indices"],
|
||||
allow_pickle=False,
|
||||
mmap_mode="r",
|
||||
)
|
||||
points = np.load(
|
||||
artifacts["point-coordinates-map-f32"],
|
||||
allow_pickle=False,
|
||||
mmap_mode="r",
|
||||
)
|
||||
owner_indices = np.load(
|
||||
artifacts["point-owner-indices"],
|
||||
allow_pickle=False,
|
||||
mmap_mode="r",
|
||||
)
|
||||
except (KeyError, OSError, ValueError) as exc:
|
||||
raise E32TrackGeometryStorageError(
|
||||
"E32 point storage is unreadable"
|
||||
) from exc
|
||||
if (
|
||||
frame_offsets.dtype != np.dtype("<i8")
|
||||
or frame_offsets.shape != (frame_count + 1,)
|
||||
or source_indices.dtype != np.dtype("<i8")
|
||||
or source_indices.ndim != 1
|
||||
or points.dtype != np.dtype("<f4")
|
||||
or points.shape != (source_indices.size, 3)
|
||||
or owner_indices.dtype != np.dtype("<u4")
|
||||
or owner_indices.shape != (source_indices.size,)
|
||||
or frame_offsets[0] != 0
|
||||
or frame_offsets[-1] != source_indices.size
|
||||
or np.any(np.diff(frame_offsets) < 0)
|
||||
or not np.isfinite(points).all()
|
||||
):
|
||||
raise E32TrackGeometryStorageError(
|
||||
"E32 point storage contract changed"
|
||||
)
|
||||
return (
|
||||
cast(Int64Array, frame_offsets),
|
||||
cast(Int64Array, source_indices),
|
||||
cast(npt.NDArray[np.float32], points),
|
||||
cast(UInt32Array, owner_indices),
|
||||
)
|
||||
|
||||
|
||||
def frame_from_record(
|
||||
*,
|
||||
record_value: Mapping[str, object],
|
||||
binding: TrackGeometrySourceBinding,
|
||||
frame_offsets: Int64Array,
|
||||
source_indices: Int64Array,
|
||||
points: npt.NDArray[np.float32],
|
||||
owner_indices: UInt32Array,
|
||||
) -> TrackGeometryFrame:
|
||||
"""Reconstruct one TrackGeometryFrame from its JSON row and slab slices."""
|
||||
|
||||
expected_keys = {
|
||||
"schema_version",
|
||||
"track_geometry_frame_schema",
|
||||
"frame_index",
|
||||
"source_frame_index",
|
||||
"session_seconds",
|
||||
"source_available",
|
||||
"point_slab",
|
||||
"geometries",
|
||||
"policy",
|
||||
"authority",
|
||||
}
|
||||
if (
|
||||
set(record_value) != expected_keys
|
||||
or record_value.get("schema_version") != E32_TRACK_GEOMETRY_RECORD_SCHEMA
|
||||
or record_value.get("track_geometry_frame_schema")
|
||||
!= TRACK_GEOMETRY_FRAME_SCHEMA
|
||||
or record_value.get("authority") != _authority()
|
||||
or record_value.get("policy")
|
||||
!= {
|
||||
"camera_owns_semantics": True,
|
||||
"one_owner_per_source_point": True,
|
||||
"current_held_persistent_are_separate": True,
|
||||
"absence_of_points_means_free": False,
|
||||
"unknown_remains_unknown": True,
|
||||
}
|
||||
):
|
||||
raise E32TrackGeometryStorageError(
|
||||
"E32 frame record contract changed"
|
||||
)
|
||||
frame_index = _nonnegative_int(record_value.get("frame_index"), "frame index")
|
||||
if frame_index + 1 >= frame_offsets.size:
|
||||
raise E32TrackGeometryStorageError(
|
||||
"E32 frame point offset is missing"
|
||||
)
|
||||
row_start = int(frame_offsets[frame_index])
|
||||
row_end = int(frame_offsets[frame_index + 1])
|
||||
slab_reference = _object(
|
||||
record_value.get("point_slab"),
|
||||
"E32 PointSlab reference",
|
||||
)
|
||||
if (
|
||||
set(slab_reference)
|
||||
!= {
|
||||
"schema_version",
|
||||
"contract_schema",
|
||||
"source_point_count",
|
||||
"coordinate_frame",
|
||||
"owner_keys",
|
||||
"row_count",
|
||||
}
|
||||
or slab_reference.get("schema_version")
|
||||
!= E32_POINT_SLAB_REFERENCE_SCHEMA
|
||||
or slab_reference.get("contract_schema") != POINT_SLAB_SCHEMA
|
||||
or slab_reference.get("row_count") != row_end - row_start
|
||||
):
|
||||
raise E32TrackGeometryStorageError(
|
||||
"E32 PointSlab reference changed"
|
||||
)
|
||||
owner_key_values = slab_reference.get("owner_keys")
|
||||
geometry_values = record_value.get("geometries")
|
||||
if not isinstance(owner_key_values, list) or not isinstance(
|
||||
geometry_values,
|
||||
list,
|
||||
):
|
||||
raise E32TrackGeometryStorageError(
|
||||
"E32 frame owner or geometry table changed"
|
||||
)
|
||||
slab = PointSlab(
|
||||
frame_index=frame_index,
|
||||
source_frame_index=_nonnegative_int(
|
||||
record_value.get("source_frame_index"),
|
||||
"source frame index",
|
||||
),
|
||||
source_point_count=_nonnegative_int(
|
||||
slab_reference.get("source_point_count"),
|
||||
"source point count",
|
||||
),
|
||||
coordinate_frame=_string(
|
||||
slab_reference.get("coordinate_frame"),
|
||||
"point coordinate frame",
|
||||
),
|
||||
owner_keys=tuple(
|
||||
_string(value, "point owner key") for value in owner_key_values
|
||||
),
|
||||
source_indices=np.asarray(source_indices[row_start:row_end], dtype="<i8"),
|
||||
points_xyz_m=np.asarray(points[row_start:row_end], dtype="<f4"),
|
||||
owner_indices=np.asarray(owner_indices[row_start:row_end], dtype="<u4"),
|
||||
)
|
||||
return TrackGeometryFrame(
|
||||
binding=binding,
|
||||
frame_index=frame_index,
|
||||
source_frame_index=slab.source_frame_index,
|
||||
session_seconds=_nonnegative_float(
|
||||
record_value.get("session_seconds"),
|
||||
"session time",
|
||||
),
|
||||
source_available=_boolean(
|
||||
record_value.get("source_available"),
|
||||
"source availability",
|
||||
),
|
||||
point_slab=slab,
|
||||
geometries=tuple(
|
||||
TrackGeometry.from_dict(value) for value in geometry_values
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def record(line: str, *, expected_frame_index: int) -> dict[str, Any]:
|
||||
"""Parse one ordered compact frame record."""
|
||||
|
||||
try:
|
||||
value = json.loads(line)
|
||||
except json.JSONDecodeError as exc:
|
||||
raise E32TrackGeometryStorageError(
|
||||
"E32 frame record JSON is invalid"
|
||||
) from exc
|
||||
result = _object(value, "E32 frame record")
|
||||
if result.get("frame_index") != expected_frame_index:
|
||||
raise E32TrackGeometryStorageError(
|
||||
"E32 frame record order changed"
|
||||
)
|
||||
return result
|
||||
|
||||
|
||||
def _save_npy(path: Path, value: npt.NDArray[Any]) -> None:
|
||||
if path.exists():
|
||||
raise E32TrackGeometryStorageError(
|
||||
"E32 point artifact already exists"
|
||||
)
|
||||
with path.open("xb") as stream:
|
||||
np.save(stream, value, allow_pickle=False)
|
||||
|
||||
|
||||
def _nonnegative_float(value: object, label: str) -> float:
|
||||
if (
|
||||
not isinstance(value, (int, float))
|
||||
or isinstance(value, bool)
|
||||
or not math.isfinite(float(value))
|
||||
or float(value) < 0.0
|
||||
):
|
||||
raise E32TrackGeometryStorageError(f"{label} is invalid")
|
||||
return float(value)
|
||||
|
||||
|
||||
def _nonnegative_int(value: object, label: str) -> int:
|
||||
if not isinstance(value, int) or isinstance(value, bool) or value < 0:
|
||||
raise E32TrackGeometryStorageError(f"{label} is invalid")
|
||||
return value
|
||||
|
||||
|
||||
def _positive_int(value: object, label: str) -> int:
|
||||
result = _nonnegative_int(value, label)
|
||||
if result == 0:
|
||||
raise E32TrackGeometryStorageError(f"{label} is invalid")
|
||||
return result
|
||||
|
||||
|
||||
def _boolean(value: object, label: str) -> bool:
|
||||
if not isinstance(value, bool):
|
||||
raise E32TrackGeometryStorageError(f"{label} is invalid")
|
||||
return value
|
||||
|
||||
|
||||
def _string(value: object, label: str) -> str:
|
||||
if not isinstance(value, str) or not value or len(value) > 256:
|
||||
raise E32TrackGeometryStorageError(f"{label} is invalid")
|
||||
return value
|
||||
|
||||
|
||||
def _object(value: object, label: str) -> dict[str, Any]:
|
||||
if not isinstance(value, dict) or any(
|
||||
not isinstance(key, str) for key in value
|
||||
):
|
||||
raise E32TrackGeometryStorageError(f"{label} must be an object")
|
||||
return value
|
||||
|
||||
|
||||
def _authority() -> dict[str, bool]:
|
||||
return {
|
||||
"commands_enabled": False,
|
||||
"navigation_or_safety_accepted": False,
|
||||
}
|
||||
@@ -0,0 +1,432 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from collections import Counter
|
||||
|
||||
import numpy as np
|
||||
import pytest
|
||||
|
||||
from k1link.compute.e32_track_geometry_replay import (
|
||||
E32TrackGeometryReplayError,
|
||||
_comparison_document,
|
||||
_CorrectionPlan,
|
||||
_require_exact_e29_reproduction,
|
||||
_translate_frame,
|
||||
)
|
||||
from k1link.compute.e32_track_geometry_storage import frame_from_record
|
||||
from k1link.compute.semantic_geometry_fusion import (
|
||||
CAMERA_GEOMETRY_FRAME_SCHEMA,
|
||||
_GeometryClusterSupport,
|
||||
_SemanticSupport,
|
||||
)
|
||||
from k1link.compute.sensor_representation import K1_LIO_PCL_CAPABILITIES
|
||||
from k1link.compute.track_geometry import (
|
||||
TrackGeometryCurrentness,
|
||||
TrackGeometryEvidenceState,
|
||||
TrackGeometryMetricBasis,
|
||||
TrackGeometrySourceBinding,
|
||||
)
|
||||
|
||||
|
||||
def _binding() -> TrackGeometrySourceBinding:
|
||||
return TrackGeometrySourceBinding(
|
||||
source_pack_id="e10-lidar-pack-" + "a" * 64,
|
||||
source_session_id="source-session",
|
||||
representation_profile_id=K1_LIO_PCL_CAPABILITIES.profile_id,
|
||||
e31_qualification_id="e31-source-qualification-" + "b" * 64,
|
||||
calibration_sha256="c" * 64,
|
||||
coordinate_frame="map",
|
||||
time_basis="nearest-host-arrival-best-effort",
|
||||
selected_offset_ms=0,
|
||||
)
|
||||
|
||||
|
||||
def _semantic(
|
||||
*,
|
||||
track_id: int,
|
||||
label: str,
|
||||
status: str,
|
||||
indices: list[int],
|
||||
bbox: list[float],
|
||||
current: bool = True,
|
||||
) -> _SemanticSupport:
|
||||
return _SemanticSupport(
|
||||
document={
|
||||
"source_track_id": track_id,
|
||||
"track_id": track_id,
|
||||
"label": label,
|
||||
"association_group": label,
|
||||
"score": 0.9,
|
||||
"bbox_xyxy": bbox,
|
||||
"semantic_current": current,
|
||||
"camera_motion_state": "unknown",
|
||||
"camera_motion_confidence": None,
|
||||
"motion_state": "unknown",
|
||||
"motion_status": "unknown",
|
||||
"unknown_is_occupied": True,
|
||||
"navigation_or_safety_accepted": False,
|
||||
"geometry_status": status,
|
||||
"geometry_reason": (
|
||||
"camera-semantic-with-connected-occupied-lidar-support"
|
||||
if status == "agree"
|
||||
else (
|
||||
"semantic-observation-not-current"
|
||||
if not current
|
||||
else "camera-semantic-without-qualified-occupied-lidar-support"
|
||||
)
|
||||
),
|
||||
"range_m": 4.0 if status == "agree" else None,
|
||||
"occupied_centroid_map_xyz_m": None,
|
||||
"occupied_height_range_m": None,
|
||||
"support": {
|
||||
"projected_points_in_bbox": len(indices),
|
||||
"classified_points_in_bbox": len(indices),
|
||||
"surface_points_in_bbox": 0,
|
||||
"occupied_points_in_bbox": len(indices),
|
||||
"below_surface_points_in_bbox": 0,
|
||||
"connected_occupied_points": len(indices),
|
||||
"connected_occupied_voxels": int(bool(indices)),
|
||||
},
|
||||
},
|
||||
occupied_source_indices=np.asarray(indices, dtype=np.int64),
|
||||
)
|
||||
|
||||
|
||||
def _geometry(indices: list[int], range_m: float) -> _GeometryClusterSupport:
|
||||
return _GeometryClusterSupport(
|
||||
document={
|
||||
"geometry_status": "single-source-geometry",
|
||||
"semantic_class": None,
|
||||
"point_count": len(indices),
|
||||
"voxel_count": 1,
|
||||
"centroid_map_xyz_m": [0.0, 0.0, 0.0],
|
||||
"bounds_map_xyz_m": [[0.0, 0.0, 0.0], [1.0, 1.0, 1.0]],
|
||||
"height_range_m": [0.2, 1.0],
|
||||
"nearest_range_m": range_m,
|
||||
"unknown_is_occupied": True,
|
||||
"navigation_or_safety_accepted": False,
|
||||
},
|
||||
occupied_source_indices=np.asarray(indices, dtype=np.int64),
|
||||
)
|
||||
|
||||
|
||||
def test_e32_translation_applies_only_bound_corrections_and_closes_point_ownership() -> None:
|
||||
points = np.asarray(
|
||||
[[float(index), 0.0, 1.0] for index in range(8)],
|
||||
dtype=np.float64,
|
||||
)
|
||||
semantic_supports = (
|
||||
_semantic(
|
||||
track_id=7,
|
||||
label="car",
|
||||
status="agree",
|
||||
indices=[0, 1],
|
||||
bbox=[10.0, 10.0, 100.0, 100.0],
|
||||
),
|
||||
_semantic(
|
||||
track_id=8,
|
||||
label="person",
|
||||
status="single-source-camera",
|
||||
indices=[2],
|
||||
bbox=[300.0, 500.0, 340.0, 580.0],
|
||||
),
|
||||
_semantic(
|
||||
track_id=9,
|
||||
label="car",
|
||||
status="unknown",
|
||||
indices=[],
|
||||
bbox=[200.0, 100.0, 250.0, 150.0],
|
||||
current=False,
|
||||
),
|
||||
)
|
||||
geometry_supports = (
|
||||
_geometry([3, 4], 3.0),
|
||||
_geometry([5], 4.0),
|
||||
_geometry([6, 7], 5.0),
|
||||
)
|
||||
corrections = _CorrectionPlan(
|
||||
semantic_rectangle_normalized_xyxy=(0.30, 0.75, 0.50, 1.0),
|
||||
semantic_class_allowlist=frozenset({"person"}),
|
||||
image_width=800,
|
||||
image_height=600,
|
||||
exact_geometry_corrections={(12, 0): "e30-review-item-" + "d" * 64},
|
||||
human_geometry_dispositions={
|
||||
(12, 1): ("background-or-noise", "e30-review-item-" + "e" * 64),
|
||||
(12, 2): ("object-present", "e30-review-item-" + "f" * 64),
|
||||
},
|
||||
)
|
||||
|
||||
translated = _translate_frame(
|
||||
frame_index=12,
|
||||
source_frame_index=120,
|
||||
session_seconds=42.0,
|
||||
source_available=True,
|
||||
frame_points=points,
|
||||
semantic_supports=semantic_supports,
|
||||
geometry_supports=geometry_supports,
|
||||
binding=_binding(),
|
||||
corrections=corrections,
|
||||
last_current_frame={},
|
||||
)
|
||||
|
||||
frame = translated.frame
|
||||
assert [geometry.owner_key for geometry in frame.geometries] == [
|
||||
"track:7",
|
||||
"geometry:2",
|
||||
]
|
||||
assert frame.point_slab.owner_keys == ("track:7", "geometry:2")
|
||||
assert frame.point_slab.source_indices.tolist() == [0, 1, 6, 7]
|
||||
assert frame.point_slab.owner_indices.tolist() == [0, 0, 1, 1]
|
||||
assert frame.geometries[1].reason_codes == (
|
||||
"e29-unassociated-occupied-component",
|
||||
"a3-human-object-present",
|
||||
)
|
||||
assert translated.semantic_published == 1
|
||||
assert translated.semantic_masked == 1
|
||||
assert translated.semantic_unpublishable_held == 1
|
||||
assert translated.geometry_published == 1
|
||||
assert translated.geometry_exact_excluded == 1
|
||||
assert translated.geometry_human_excluded == 1
|
||||
assert translated.baseline_qualified_points == 7
|
||||
assert translated.published_qualified_points == 4
|
||||
assert translated.excluded_qualified_points == 3
|
||||
assert translated.ownership_overlap_claims == 0
|
||||
assert translated.unqualified_semantic_support_points == 1
|
||||
assert [change["reason"] for change in translated.changes] == [
|
||||
"e31-semantic-self-mask",
|
||||
"held-without-prior-current-provenance",
|
||||
"e31-exact-geometry-correction",
|
||||
"a3-human-background-or-noise",
|
||||
]
|
||||
restored = frame_from_record(
|
||||
record_value=translated.record,
|
||||
binding=_binding(),
|
||||
frame_offsets=np.asarray([0] * 13 + [4], dtype="<i8"),
|
||||
source_indices=translated.point_source_indices,
|
||||
points=translated.point_coordinates,
|
||||
owner_indices=translated.point_owner_indices,
|
||||
)
|
||||
assert restored.to_dict() == translated.frame.to_dict()
|
||||
|
||||
|
||||
def test_e32_held_track_keeps_prior_current_provenance_without_current_points() -> None:
|
||||
held = _semantic(
|
||||
track_id=11,
|
||||
label="truck",
|
||||
status="unknown",
|
||||
indices=[],
|
||||
bbox=[20.0, 20.0, 60.0, 60.0],
|
||||
current=False,
|
||||
)
|
||||
translated = _translate_frame(
|
||||
frame_index=15,
|
||||
source_frame_index=150,
|
||||
session_seconds=45.0,
|
||||
source_available=False,
|
||||
frame_points=np.empty((0, 3), dtype=np.float64),
|
||||
semantic_supports=(held,),
|
||||
geometry_supports=(),
|
||||
binding=_binding(),
|
||||
corrections=_CorrectionPlan(
|
||||
semantic_rectangle_normalized_xyxy=(0.30, 0.75, 0.50, 1.0),
|
||||
semantic_class_allowlist=frozenset({"person"}),
|
||||
image_width=800,
|
||||
image_height=600,
|
||||
exact_geometry_corrections={},
|
||||
human_geometry_dispositions={},
|
||||
),
|
||||
last_current_frame={11: 13},
|
||||
)
|
||||
|
||||
geometry = translated.frame.geometries[0]
|
||||
assert geometry.currentness is TrackGeometryCurrentness.HELD
|
||||
assert geometry.evidence_state is TrackGeometryEvidenceState.UNKNOWN
|
||||
assert geometry.metric_basis is TrackGeometryMetricBasis.UNAVAILABLE
|
||||
assert geometry.held_from_frame_index == 13
|
||||
assert translated.frame.point_slab.row_count == 0
|
||||
|
||||
|
||||
def test_e32_arbitrates_overlapping_camera_claims_without_duplicate_points() -> None:
|
||||
larger = _semantic(
|
||||
track_id=20,
|
||||
label="car",
|
||||
status="agree",
|
||||
indices=[0, 1],
|
||||
bbox=[10.0, 10.0, 100.0, 100.0],
|
||||
)
|
||||
smaller = _semantic(
|
||||
track_id=21,
|
||||
label="person",
|
||||
status="agree",
|
||||
indices=[0, 1],
|
||||
bbox=[20.0, 20.0, 40.0, 70.0],
|
||||
)
|
||||
translated = _translate_frame(
|
||||
frame_index=20,
|
||||
source_frame_index=200,
|
||||
session_seconds=50.0,
|
||||
source_available=True,
|
||||
frame_points=np.asarray(
|
||||
[[1.0, 0.0, 1.0], [2.0, 0.0, 1.0]],
|
||||
dtype=np.float64,
|
||||
),
|
||||
semantic_supports=(larger, smaller),
|
||||
geometry_supports=(),
|
||||
binding=_binding(),
|
||||
corrections=_CorrectionPlan(
|
||||
semantic_rectangle_normalized_xyxy=(0.30, 0.75, 0.50, 1.0),
|
||||
semantic_class_allowlist=frozenset({"person"}),
|
||||
image_width=800,
|
||||
image_height=600,
|
||||
exact_geometry_corrections={},
|
||||
human_geometry_dispositions={},
|
||||
),
|
||||
last_current_frame={},
|
||||
)
|
||||
|
||||
assert translated.frame.point_slab.source_indices.tolist() == [0, 1]
|
||||
assert translated.frame.point_slab.owner_keys == ("track:21",)
|
||||
assert translated.frame.geometries[0].evidence_state is TrackGeometryEvidenceState.UNKNOWN
|
||||
assert translated.frame.geometries[0].reason_codes[-1] == (
|
||||
"e32-point-ownership-collision"
|
||||
)
|
||||
assert translated.frame.geometries[1].evidence_state is TrackGeometryEvidenceState.AGREE
|
||||
assert translated.baseline_qualified_points == 4
|
||||
assert translated.published_qualified_points == 2
|
||||
assert translated.ownership_overlap_claims == 2
|
||||
assert translated.excluded_qualified_points == 0
|
||||
assert translated.changes[0]["reason"] == "point-ownership-arbitration"
|
||||
|
||||
|
||||
def test_e32_withholds_unqualified_e29_range_and_retains_camera_state() -> None:
|
||||
camera_only = _semantic(
|
||||
track_id=30,
|
||||
label="car",
|
||||
status="single-source-camera",
|
||||
indices=[0],
|
||||
bbox=[10.0, 10.0, 100.0, 100.0],
|
||||
)
|
||||
camera_only.document["range_m"] = 6.0
|
||||
translated = _translate_frame(
|
||||
frame_index=30,
|
||||
source_frame_index=300,
|
||||
session_seconds=60.0,
|
||||
source_available=True,
|
||||
frame_points=np.asarray([[1.0, 0.0, 1.0]], dtype=np.float64),
|
||||
semantic_supports=(camera_only,),
|
||||
geometry_supports=(),
|
||||
binding=_binding(),
|
||||
corrections=_CorrectionPlan(
|
||||
semantic_rectangle_normalized_xyxy=(0.30, 0.75, 0.50, 1.0),
|
||||
semantic_class_allowlist=frozenset({"person"}),
|
||||
image_width=800,
|
||||
image_height=600,
|
||||
exact_geometry_corrections={},
|
||||
human_geometry_dispositions={},
|
||||
),
|
||||
last_current_frame={},
|
||||
)
|
||||
|
||||
geometry = translated.frame.geometries[0]
|
||||
assert geometry.evidence_state is TrackGeometryEvidenceState.CAMERA_ONLY
|
||||
assert geometry.metric_basis is TrackGeometryMetricBasis.UNAVAILABLE
|
||||
assert geometry.range_m is None
|
||||
assert geometry.reason_codes[-1] == "e32-unqualified-range-withheld"
|
||||
assert translated.unqualified_ranges_withheld == 1
|
||||
assert translated.changes[0]["reason"] == "unqualified-range-withheld"
|
||||
|
||||
|
||||
def test_e32_replay_rejects_any_e29_reproduction_drift() -> None:
|
||||
semantic = _semantic(
|
||||
track_id=1,
|
||||
label="car",
|
||||
status="single-source-camera",
|
||||
indices=[],
|
||||
bbox=[10.0, 10.0, 20.0, 20.0],
|
||||
)
|
||||
fusion_frame = {
|
||||
"source_frame_index": 10,
|
||||
"session_seconds": 3.0,
|
||||
}
|
||||
baseline = {
|
||||
"schema_version": CAMERA_GEOMETRY_FRAME_SCHEMA,
|
||||
"frame_index": 0,
|
||||
"source_frame_index": 10,
|
||||
"session_seconds": 3.0,
|
||||
"source_available": False,
|
||||
"local_surface_valid": False,
|
||||
"semantic_observations": [semantic.document],
|
||||
"geometry_only_occupied": [],
|
||||
"policy": {
|
||||
"camera_owns_semantics": True,
|
||||
"lidar_owns_metric_geometry": True,
|
||||
"absence_of_points_means_free": False,
|
||||
"unknown_is_occupied": True,
|
||||
},
|
||||
"authority": {
|
||||
"commands_enabled": False,
|
||||
"navigation_or_safety_accepted": False,
|
||||
},
|
||||
}
|
||||
_require_exact_e29_reproduction(
|
||||
e29_frame=baseline,
|
||||
frame_index=0,
|
||||
fusion_frame=fusion_frame,
|
||||
source_available=False,
|
||||
surface_valid=False,
|
||||
semantic_supports=(semantic,),
|
||||
geometry_supports=(),
|
||||
)
|
||||
baseline["semantic_observations"] = []
|
||||
with pytest.raises(E32TrackGeometryReplayError, match="exactly reproduce"):
|
||||
_require_exact_e29_reproduction(
|
||||
e29_frame=baseline,
|
||||
frame_index=0,
|
||||
fusion_frame=fusion_frame,
|
||||
source_available=False,
|
||||
surface_valid=False,
|
||||
semantic_supports=(semantic,),
|
||||
geometry_supports=(),
|
||||
)
|
||||
|
||||
|
||||
def test_e32_comparison_exposes_status_class_range_scene_and_cause_deltas() -> None:
|
||||
baseline = {
|
||||
(
|
||||
"agree",
|
||||
"car",
|
||||
"middle",
|
||||
"000-060s",
|
||||
"connected-support",
|
||||
): 2,
|
||||
(
|
||||
"single-source-geometry",
|
||||
"__geometry__",
|
||||
"near",
|
||||
"000-060s",
|
||||
"unassociated",
|
||||
): 1,
|
||||
}
|
||||
current = {
|
||||
(
|
||||
"agree",
|
||||
"car",
|
||||
"middle",
|
||||
"000-060s",
|
||||
"connected-support",
|
||||
): 1,
|
||||
}
|
||||
|
||||
comparison = _comparison_document(
|
||||
Counter(baseline),
|
||||
Counter(current),
|
||||
)
|
||||
|
||||
assert comparison["by_status"]["agree"] == {
|
||||
"e29": 2,
|
||||
"e32": 1,
|
||||
"delta": -1,
|
||||
}
|
||||
assert comparison["by_class"]["__geometry__"]["delta"] == -1
|
||||
assert comparison["by_range"]["near"]["delta"] == -1
|
||||
assert comparison["by_scene"]["000-060s"]["delta"] == -2
|
||||
assert comparison["by_cause"]["unassociated"]["delta"] == -1
|
||||
Reference in New Issue
Block a user