feat(perception): qualify world-frame motion tracking
This commit is contained in:
parent
23181c867b
commit
e6dd8c2cf1
|
|
@ -0,0 +1,195 @@
|
||||||
|
# LAB E24 — world-frame tracking and motion-state qualification
|
||||||
|
|
||||||
|
Date: 2026-07-24
|
||||||
|
Status: diagnostic artifact accepted; motion benchmark not accepted
|
||||||
|
Best immutable replay:
|
||||||
|
`LAB E24.5 · Final reproducible world motion · full RAVNOVES00`
|
||||||
|
|
||||||
|
## Objective
|
||||||
|
|
||||||
|
Test whether the already calculated camera/LiDAR perception result can support a
|
||||||
|
bounded, no-lookahead tracker that keeps object identity in `k1-map` and emits a
|
||||||
|
conservative `static`, `dynamic`, or `unknown` motion state. The goal is not to
|
||||||
|
make prettier boxes. The target is a streaming-equivalent world-state component
|
||||||
|
that can later feed collision and path-planning logic.
|
||||||
|
|
||||||
|
## Source data and immutable inputs
|
||||||
|
|
||||||
|
- Source session: `20260720T065719Z_viewer_live` (`RAVNOVES00`).
|
||||||
|
- Camera source: `sensor.camera.right`, calibration slot `camera_1`.
|
||||||
|
- Calibration SHA-256:
|
||||||
|
`05f3ad9b38b3a4fc95388a8ec83da83c745e217709e51787b3d5aad0969f6fa9`.
|
||||||
|
- Source integrated result:
|
||||||
|
`e10-integrated-perception-34ade557b5636717aa497fc00355b84df7063e483a175f2f5d3f04c03df1c898`.
|
||||||
|
- Source result SHA-256:
|
||||||
|
`418cd598277d50f4ee81c042d98ece9be10b1b8dc49bf3e2dcc2f2a9ca1c6dc3`.
|
||||||
|
- Frames: 4,489.
|
||||||
|
- Source timeline: 35.421857292–484.044857292 session seconds.
|
||||||
|
- Source detector: YOLOX-S, COCO-80, 640×640.
|
||||||
|
- Source semantic model:
|
||||||
|
`tue-mps/cityscapes_semantic_eomt_large_1024`, FP16 autocast.
|
||||||
|
- Source fusion: 3,915 LiDAR-fused frames and 3,349 accepted cuboids.
|
||||||
|
- Raw point cloud, ground returns, trajectory, camera media, calibration,
|
||||||
|
detector output and semantic masks were not changed.
|
||||||
|
|
||||||
|
The E24 runs reuse the accepted E19 result. They do not repeat GPU inference.
|
||||||
|
E24 is a deterministic CPU post-fusion stage executed locally with Python
|
||||||
|
3.12/NumPy 2.5.1.
|
||||||
|
|
||||||
|
## Operator benchmark
|
||||||
|
|
||||||
|
The committed benchmark uses the operator's approximate RAVNOVES00 timeline:
|
||||||
|
|
||||||
|
- 58 s: woman in a pink shirt with a dog, left;
|
||||||
|
- 65 s: person loading a car, right;
|
||||||
|
- 115 s: adult and child, left;
|
||||||
|
- 127 s: pedestrian on the road, right;
|
||||||
|
- 155 s: woman with stroller and child, left;
|
||||||
|
- 170 s: oncoming vehicle, right;
|
||||||
|
- 173 s: vulnerable-road-user group, left;
|
||||||
|
- 183 s: vehicle moving in the rig direction, right;
|
||||||
|
- three static-vehicle control windows before the moving-vehicle events.
|
||||||
|
|
||||||
|
The benchmark is not labeled ground truth. It is a reproducible operator
|
||||||
|
annotation contract with explicit time windows, class groups, minimum hits and
|
||||||
|
minimum observed spans.
|
||||||
|
|
||||||
|
## Implemented pipeline
|
||||||
|
|
||||||
|
E24 implements tracking-by-detection in the map frame:
|
||||||
|
|
||||||
|
1. accepted 3D cuboids are measurements in `k1-map`;
|
||||||
|
2. visible LiDAR support rejected only by amodal-completion coverage may enter as
|
||||||
|
a high-uncertainty provisional measurement;
|
||||||
|
3. a bounded constant-velocity Kalman state estimates position, velocity and
|
||||||
|
covariance;
|
||||||
|
4. global greedy association uses class compatibility, predicted map position,
|
||||||
|
shape distance, detector-ID aliases and strict ID-switch gates;
|
||||||
|
5. provisional tracks require repeated support before a cuboid is published;
|
||||||
|
6. short measurement gaps are explicit predicted holds, limited to 450 ms;
|
||||||
|
7. motion state requires a bounded history, minimum displacement, robust speed,
|
||||||
|
direction consistency and hysteresis;
|
||||||
|
8. every world object exposes observation age, track age, position/velocity
|
||||||
|
covariance, motion confidence and source track ID.
|
||||||
|
|
||||||
|
State is bounded to 192 tracks, has no lookahead, and has no command,
|
||||||
|
navigation, or safety authority.
|
||||||
|
|
||||||
|
## Run series
|
||||||
|
|
||||||
|
### E24.1 — accepted cuboids only
|
||||||
|
|
||||||
|
- Benchmark: 2/11.
|
||||||
|
- Processing p95: 0.600 ms/frame.
|
||||||
|
- Finding: static vehicle tracks can be held, but people and moving vehicles
|
||||||
|
often lack enough accepted 3D measurements to leave `unknown`.
|
||||||
|
|
||||||
|
### E24.2 — provisional visible-support measurements
|
||||||
|
|
||||||
|
- Benchmark: 6/11.
|
||||||
|
- Processing p95: 0.581 ms/frame.
|
||||||
|
- Finding: recovered motion evidence for the 58 s, 115 s and 155 s pedestrian
|
||||||
|
windows and the 183 s vehicle; broad reassociation also merged nearby vehicle
|
||||||
|
tracks and created false motion.
|
||||||
|
|
||||||
|
### E24.3 — conservative ID-switch gates
|
||||||
|
|
||||||
|
- Benchmark: 7/11.
|
||||||
|
- Processing p95: 0.638 ms/frame.
|
||||||
|
- Finding: strict ID-switch gates reduced false merges, but result identity did
|
||||||
|
not yet contain the postprocessor implementation SHA-256.
|
||||||
|
|
||||||
|
### E24.4 — reproducible conservative ID-switch gates
|
||||||
|
|
||||||
|
- Benchmark: 7/11.
|
||||||
|
- Processing p95: 0.611 ms/frame.
|
||||||
|
- Finding: implementation SHA-256 was added before the formatter produced the
|
||||||
|
final source bytes, so a final identity-exact replay was required.
|
||||||
|
|
||||||
|
### E24.5 — final identity-exact replay
|
||||||
|
|
||||||
|
- Result:
|
||||||
|
`e10-integrated-perception-e56fa878cba3f9f9e5943831cad0292643e49571f212a4d341bca28626515d80`.
|
||||||
|
- Profile SHA-256:
|
||||||
|
`dff304161ace7846dd7886f43877c6cae35737d4e7384b76dae6a430aeec1bc9`.
|
||||||
|
- Benchmark SHA-256:
|
||||||
|
`f1fa0f2b17b7148a1f5371339faf47a9ef25bfe9003fce4559f73be4f408818b`.
|
||||||
|
- Implementation SHA-256:
|
||||||
|
`3ca4c464eabaab3a89e7e684704d6d0288bb1dcd3dd455423456bed91d9d2ecc`.
|
||||||
|
- Benchmark: 7/11.
|
||||||
|
- Processing mean/p50/p95/max:
|
||||||
|
0.328/0.287/0.713/53.743 ms per frame.
|
||||||
|
- Peak live track states: 10 of 192.
|
||||||
|
- Created tracks: 251.
|
||||||
|
- Cross-source-ID reassociations: 24.
|
||||||
|
- Explicit held cuboids: 1,859.
|
||||||
|
- Published motion states: 3,139 unknown, 1,494 static, 875 dynamic.
|
||||||
|
|
||||||
|
Passed events:
|
||||||
|
|
||||||
|
- dynamic person at 58 s;
|
||||||
|
- dynamic person at 65 s;
|
||||||
|
- static vehicle evidence at 70–90 s;
|
||||||
|
- dynamic person at 115 s;
|
||||||
|
- dynamic person at 155 s;
|
||||||
|
- oncoming dynamic vehicle at 170 s;
|
||||||
|
- same-direction dynamic vehicle at 183 s.
|
||||||
|
|
||||||
|
Failed events:
|
||||||
|
|
||||||
|
- 127 s pedestrian: only one publishable 3D observation after confirmation;
|
||||||
|
- 132–148 s static vehicles: visible-surface/amodal-center drift still creates
|
||||||
|
false dynamic hypotheses;
|
||||||
|
- 161–166 s static vehicles: insufficient persistent 3D support;
|
||||||
|
- 173 s vulnerable-road-user group: measurements are too sparse after strict
|
||||||
|
identity separation.
|
||||||
|
|
||||||
|
## Main conclusion
|
||||||
|
|
||||||
|
E24 proves that the tracker itself fits an eventual near-real-time budget by a
|
||||||
|
large margin. It also proves that temporal smoothing and threshold tuning alone
|
||||||
|
cannot make the current cuboid center a reliable motion measurement.
|
||||||
|
|
||||||
|
For parked vehicles, the fitted/completed cuboid center can drift systematically
|
||||||
|
as the rig passes and a different visible LiDAR surface becomes dominant. A
|
||||||
|
constant-velocity tracker correctly interprets that input as motion because the
|
||||||
|
measurement itself is moving. Relaxed association hides gaps but merges nearby
|
||||||
|
objects; strict association avoids most merges but exposes the missing
|
||||||
|
measurement continuity.
|
||||||
|
|
||||||
|
Therefore E24.5 is an accepted diagnostic artifact, not an accepted motion
|
||||||
|
classifier. It must not feed navigation or safety decisions.
|
||||||
|
|
||||||
|
## Next experimental gate
|
||||||
|
|
||||||
|
E25 must improve the measurement model rather than retune E24:
|
||||||
|
|
||||||
|
- maintain object-level LiDAR support/occupancy evidence across scans in the map
|
||||||
|
frame instead of tracking only a completed box center;
|
||||||
|
- estimate a static-landmark hypothesis from persistent support and compare it
|
||||||
|
against a moving-object hypothesis;
|
||||||
|
- keep raw ground and the full cloud visible; exclude ground only from the
|
||||||
|
object-proposal branch;
|
||||||
|
- split coverage, association, motion-state and false-dynamic metrics so a
|
||||||
|
missing observation cannot be mistaken for a correct static classification;
|
||||||
|
- use the same operator anchors plus explicit per-object spatial annotations for
|
||||||
|
the failed 127 s, 132–148 s, 161–166 s and 173 s windows;
|
||||||
|
- accept E25 only after false dynamic vehicle tracks are reduced without losing
|
||||||
|
the two known moving vehicles.
|
||||||
|
|
||||||
|
The planner-facing representation should be occupancy/footprint, velocity,
|
||||||
|
uncertainty and short-horizon prediction. 3D boxes remain an operator-facing
|
||||||
|
representation and one measurement carrier, not the sole collision model.
|
||||||
|
|
||||||
|
## Validation
|
||||||
|
|
||||||
|
- Unit tests cover bounded state, ID reassociation, static/dynamic
|
||||||
|
classification, provisional confirmation, short holds and benchmark parsing.
|
||||||
|
- E24.5 result passes the existing integrated-perception validator.
|
||||||
|
- Saved session `lab-e24-5-world-motion` is `ready` and replayable on
|
||||||
|
`http://127.0.0.1:8000/`.
|
||||||
|
- A cold full-session perception request completed with HTTP 200 in 51.958 s
|
||||||
|
and produced a 69,303,555-byte Rerun stream with `RRF2` magic. This cold
|
||||||
|
materialization time is a viewer-publication limitation, not E24 tracking
|
||||||
|
latency; the generated overlay is now cached.
|
||||||
|
- The existing localhost server was not stopped or restarted.
|
||||||
|
|
@ -0,0 +1,97 @@
|
||||||
|
{
|
||||||
|
"schema_version": "missioncore.e24-motion-benchmark/v1",
|
||||||
|
"benchmark_id": "ravnoves00-operator-anchors-v1",
|
||||||
|
"source_session_id": "20260720T065719Z_viewer_live",
|
||||||
|
"timeline": "session_seconds",
|
||||||
|
"annotation_status": "operator-approximate",
|
||||||
|
"events": [
|
||||||
|
{
|
||||||
|
"id": "dynamic-person-woman-dog-left-58s",
|
||||||
|
"window_seconds": [54.0, 62.0],
|
||||||
|
"class_group": "person",
|
||||||
|
"expected_motion": "dynamic",
|
||||||
|
"minimum_hits": 3,
|
||||||
|
"minimum_span_seconds": 0.2
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "dynamic-person-loading-car-right-65s",
|
||||||
|
"window_seconds": [62.0, 69.0],
|
||||||
|
"class_group": "person",
|
||||||
|
"expected_motion": "dynamic",
|
||||||
|
"minimum_hits": 3,
|
||||||
|
"minimum_span_seconds": 0.2
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "static-vehicles-before-moving-70-90s",
|
||||||
|
"window_seconds": [70.0, 90.0],
|
||||||
|
"class_group": "vehicle",
|
||||||
|
"expected_motion": "static",
|
||||||
|
"minimum_hits": 8,
|
||||||
|
"minimum_span_seconds": 0.7
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "dynamic-person-adult-child-left-115s",
|
||||||
|
"window_seconds": [111.0, 120.0],
|
||||||
|
"class_group": "person",
|
||||||
|
"expected_motion": "dynamic",
|
||||||
|
"minimum_hits": 3,
|
||||||
|
"minimum_span_seconds": 0.2
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "dynamic-person-road-right-127s",
|
||||||
|
"window_seconds": [124.0, 132.0],
|
||||||
|
"class_group": "person",
|
||||||
|
"expected_motion": "dynamic",
|
||||||
|
"minimum_hits": 3,
|
||||||
|
"minimum_span_seconds": 0.2
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "static-vehicles-mid-run-132-148s",
|
||||||
|
"window_seconds": [132.0, 148.0],
|
||||||
|
"class_group": "vehicle",
|
||||||
|
"expected_motion": "static",
|
||||||
|
"minimum_hits": 8,
|
||||||
|
"minimum_span_seconds": 0.7
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "dynamic-person-stroller-group-left-155s",
|
||||||
|
"window_seconds": [151.0, 161.0],
|
||||||
|
"class_group": "person",
|
||||||
|
"expected_motion": "dynamic",
|
||||||
|
"minimum_hits": 3,
|
||||||
|
"minimum_span_seconds": 0.2
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "static-vehicles-before-oncoming-161-166s",
|
||||||
|
"window_seconds": [161.0, 166.0],
|
||||||
|
"class_group": "vehicle",
|
||||||
|
"expected_motion": "static",
|
||||||
|
"minimum_hits": 8,
|
||||||
|
"minimum_span_seconds": 0.7
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "dynamic-vehicle-oncoming-right-170s",
|
||||||
|
"window_seconds": [167.0, 173.0],
|
||||||
|
"class_group": "vehicle",
|
||||||
|
"expected_motion": "dynamic",
|
||||||
|
"minimum_hits": 3,
|
||||||
|
"minimum_span_seconds": 0.15
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "dynamic-person-bike-group-left-173s",
|
||||||
|
"window_seconds": [170.0, 179.0],
|
||||||
|
"class_group": "vulnerable_road_user",
|
||||||
|
"expected_motion": "dynamic",
|
||||||
|
"minimum_hits": 3,
|
||||||
|
"minimum_span_seconds": 0.2
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "dynamic-vehicle-same-direction-right-183s",
|
||||||
|
"window_seconds": [180.0, 188.0],
|
||||||
|
"class_group": "vehicle",
|
||||||
|
"expected_motion": "dynamic",
|
||||||
|
"minimum_hits": 3,
|
||||||
|
"minimum_span_seconds": 0.2
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
|
@ -0,0 +1,100 @@
|
||||||
|
{
|
||||||
|
"schema_version": "missioncore.e24-world-motion-profile/v1",
|
||||||
|
"profile_id": "lab-e24-world-frame-motion-v1",
|
||||||
|
"mode": "recorded-streaming-qualification",
|
||||||
|
"source": {
|
||||||
|
"source_id": "sensor.camera.right",
|
||||||
|
"coordinate_frame": "k1-map",
|
||||||
|
"calibration_slot": "camera_1",
|
||||||
|
"calibration_sha256": "05f3ad9b38b3a4fc95388a8ec83da83c745e217709e51787b3d5aad0969f6fa9"
|
||||||
|
},
|
||||||
|
"association": {
|
||||||
|
"maximum_center_distance_m": 2.2,
|
||||||
|
"maximum_speed_mps": 16.0,
|
||||||
|
"maximum_gap_allowance_m": 2.5,
|
||||||
|
"maximum_reassociation_gap_seconds": 0.55,
|
||||||
|
"maximum_reassociation_distance_m": {
|
||||||
|
"person": 1.2,
|
||||||
|
"bicycle": 1.2,
|
||||||
|
"motorcycle": 1.2,
|
||||||
|
"vehicle": 0.75,
|
||||||
|
"default": 0.75
|
||||||
|
},
|
||||||
|
"source_track_alias_bonus": 0.8,
|
||||||
|
"size_distance_weight": 0.35
|
||||||
|
},
|
||||||
|
"filter": {
|
||||||
|
"initial_position_sigma_m": 0.8,
|
||||||
|
"initial_velocity_sigma_mps": 2.0,
|
||||||
|
"measurement_sigma_m": 1.6,
|
||||||
|
"provisional_measurement_sigma_m": 2.8,
|
||||||
|
"minimum_measurement_sigma_m": 0.18,
|
||||||
|
"support_points_saturation": 64,
|
||||||
|
"acceleration_sigma_mps2": 2.0,
|
||||||
|
"shape_alpha": 0.18
|
||||||
|
},
|
||||||
|
"provisional": {
|
||||||
|
"allowed_groups": [
|
||||||
|
"person",
|
||||||
|
"bicycle",
|
||||||
|
"motorcycle",
|
||||||
|
"vehicle"
|
||||||
|
],
|
||||||
|
"nominal_half_size_m": {
|
||||||
|
"person": [0.35, 0.35, 0.9],
|
||||||
|
"bicycle": [0.9, 0.325, 0.75],
|
||||||
|
"motorcycle": [0.9, 0.35, 0.75],
|
||||||
|
"vehicle": [2.25, 0.925, 0.775],
|
||||||
|
"default": [0.5, 0.5, 0.5]
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"tracking": {
|
||||||
|
"minimum_confirmation_hits": 4,
|
||||||
|
"provisional_confirmation_hits": 3,
|
||||||
|
"hold_seconds": 0.45
|
||||||
|
},
|
||||||
|
"motion": {
|
||||||
|
"history_seconds": 1.8,
|
||||||
|
"minimum_history_seconds": 0.8,
|
||||||
|
"dynamic_enter_speed_mps": {
|
||||||
|
"person": 0.45,
|
||||||
|
"bicycle": 0.55,
|
||||||
|
"motorcycle": 0.65,
|
||||||
|
"vehicle": 0.75,
|
||||||
|
"default": 0.75
|
||||||
|
},
|
||||||
|
"dynamic_exit_speed_mps": 0.4,
|
||||||
|
"dynamic_minimum_displacement_m": {
|
||||||
|
"person": 0.45,
|
||||||
|
"bicycle": 0.5,
|
||||||
|
"motorcycle": 0.55,
|
||||||
|
"vehicle": 0.65,
|
||||||
|
"default": 0.65
|
||||||
|
},
|
||||||
|
"minimum_direction_consistency": 0.62,
|
||||||
|
"dynamic_confirmation_frames": {
|
||||||
|
"person": 1,
|
||||||
|
"bicycle": 1,
|
||||||
|
"motorcycle": 1,
|
||||||
|
"vehicle": 2,
|
||||||
|
"default": 2
|
||||||
|
},
|
||||||
|
"static_enter_speed_mps": 0.28,
|
||||||
|
"static_exit_speed_mps": 0.55,
|
||||||
|
"static_maximum_displacement_m": 0.42,
|
||||||
|
"static_confirmation_frames": 7
|
||||||
|
},
|
||||||
|
"bounds": {
|
||||||
|
"track_id_start": 240001,
|
||||||
|
"maximum_tracks": 192,
|
||||||
|
"maximum_track_idle_seconds": 1.5
|
||||||
|
},
|
||||||
|
"acceptance": {
|
||||||
|
"maximum_processing_p95_ms": 8.0,
|
||||||
|
"maximum_tracks_observed": 192
|
||||||
|
},
|
||||||
|
"authority": {
|
||||||
|
"commands_enabled": false,
|
||||||
|
"navigation_or_safety_accepted": false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -37,9 +37,11 @@ from .jobs import (
|
||||||
from .lab_instances import (
|
from .lab_instances import (
|
||||||
PublishedIntegratedLabInstance,
|
PublishedIntegratedLabInstance,
|
||||||
PublishedTemporalLabInstance,
|
PublishedTemporalLabInstance,
|
||||||
|
PublishedWorldMotionLabInstance,
|
||||||
publish_e21_lab_instance,
|
publish_e21_lab_instance,
|
||||||
publish_e22_lab_instance,
|
publish_e22_lab_instance,
|
||||||
publish_e23_lab_instance,
|
publish_e23_lab_instance,
|
||||||
|
publish_e24_lab_instance,
|
||||||
publish_integrated_lab_instance,
|
publish_integrated_lab_instance,
|
||||||
)
|
)
|
||||||
from .live_perception import (
|
from .live_perception import (
|
||||||
|
|
@ -119,6 +121,7 @@ __all__ = [
|
||||||
"IntegratedPerceptionOverlayStore",
|
"IntegratedPerceptionOverlayStore",
|
||||||
"PublishedIntegratedLabInstance",
|
"PublishedIntegratedLabInstance",
|
||||||
"PublishedTemporalLabInstance",
|
"PublishedTemporalLabInstance",
|
||||||
|
"PublishedWorldMotionLabInstance",
|
||||||
"IntegratedPerceptionResult",
|
"IntegratedPerceptionResult",
|
||||||
"LiveReplayQualificationResult",
|
"LiveReplayQualificationResult",
|
||||||
"MultiratePerceptionArtifact",
|
"MultiratePerceptionArtifact",
|
||||||
|
|
@ -150,6 +153,7 @@ __all__ = [
|
||||||
"publish_e21_lab_instance",
|
"publish_e21_lab_instance",
|
||||||
"publish_e22_lab_instance",
|
"publish_e22_lab_instance",
|
||||||
"publish_e23_lab_instance",
|
"publish_e23_lab_instance",
|
||||||
|
"publish_e24_lab_instance",
|
||||||
"publish_integrated_lab_instance",
|
"publish_integrated_lab_instance",
|
||||||
"validate_multirate_perception_qualification_result",
|
"validate_multirate_perception_qualification_result",
|
||||||
"prepare_recorded_qualification_slice",
|
"prepare_recorded_qualification_slice",
|
||||||
|
|
|
||||||
|
|
@ -33,6 +33,7 @@ from .temporal_stability import (
|
||||||
_quality_metrics,
|
_quality_metrics,
|
||||||
build_temporal_stability_result,
|
build_temporal_stability_result,
|
||||||
)
|
)
|
||||||
|
from .world_motion import WorldMotionBuild, build_world_motion_result
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True, slots=True)
|
@dataclass(frozen=True, slots=True)
|
||||||
|
|
@ -50,6 +51,14 @@ class PublishedTemporalLabInstance:
|
||||||
build: TemporalStabilityBuild
|
build: TemporalStabilityBuild
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True, slots=True)
|
||||||
|
class PublishedWorldMotionLabInstance:
|
||||||
|
binding: LabSessionBinding
|
||||||
|
job: CameraComputeJob
|
||||||
|
result: IntegratedPerceptionResult
|
||||||
|
build: WorldMotionBuild
|
||||||
|
|
||||||
|
|
||||||
def publish_integrated_lab_instance(
|
def publish_integrated_lab_instance(
|
||||||
*,
|
*,
|
||||||
repository_root: Path,
|
repository_root: Path,
|
||||||
|
|
@ -477,6 +486,110 @@ def publish_e23_lab_instance(
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def publish_e24_lab_instance(
|
||||||
|
*,
|
||||||
|
repository_root: Path,
|
||||||
|
source_result_root: Path,
|
||||||
|
profile_path: Path,
|
||||||
|
benchmark_path: Path,
|
||||||
|
lab_session_id: str,
|
||||||
|
lab_id: str,
|
||||||
|
display_name: str,
|
||||||
|
) -> PublishedWorldMotionLabInstance:
|
||||||
|
"""Derive and publish a bounded world-frame motion-tracking LAB run."""
|
||||||
|
|
||||||
|
root = repository_root.expanduser().resolve(strict=True)
|
||||||
|
jobs_root = root / ".runtime" / "compute-jobs"
|
||||||
|
results_root = root / ".runtime" / "compute-experiments" / "e10" / "worker-results"
|
||||||
|
packs_root = root / ".runtime" / "compute-experiments" / "e10" / "lidar-packs"
|
||||||
|
source_path = source_result_root.expanduser().resolve(strict=True)
|
||||||
|
source_document = _read_object(source_path / "result.json", source_path)
|
||||||
|
identity = source_document.get("identity")
|
||||||
|
if not isinstance(identity, dict) or not isinstance(identity.get("job_id"), str):
|
||||||
|
raise SessionIntegrityError("E24 source has no job identity")
|
||||||
|
source = validate_integrated_perception_result(
|
||||||
|
jobs_root / identity["job_id"],
|
||||||
|
source_path,
|
||||||
|
packs_root,
|
||||||
|
)
|
||||||
|
if not source.accepted:
|
||||||
|
raise SessionIntegrityError("E24 source result is not accepted")
|
||||||
|
|
||||||
|
lab_job = _publish_lab_job(source.job, jobs_root, lab_session_id)
|
||||||
|
lab_pack = _publish_lab_pack(source, lab_job, packs_root, lab_session_id)
|
||||||
|
build = build_world_motion_result(
|
||||||
|
source=source,
|
||||||
|
lab_job=lab_job,
|
||||||
|
lab_pack=lab_pack,
|
||||||
|
results_root=results_root,
|
||||||
|
profile_path=profile_path,
|
||||||
|
benchmark_path=benchmark_path,
|
||||||
|
)
|
||||||
|
validated = validate_integrated_perception_result(
|
||||||
|
lab_job.job_root,
|
||||||
|
build.result_root,
|
||||||
|
packs_root,
|
||||||
|
)
|
||||||
|
if not validated.accepted:
|
||||||
|
failed = [
|
||||||
|
name for name, accepted in build.report["acceptance"]["checks"].items() if not accepted
|
||||||
|
]
|
||||||
|
raise SessionIntegrityError(
|
||||||
|
f"E24 world-motion artifact acceptance failed: {', '.join(failed)}"
|
||||||
|
)
|
||||||
|
|
||||||
|
store = SessionStore(root)
|
||||||
|
source_lab = store.get_lab_instance(source.job.session_id)
|
||||||
|
source_session_id = (
|
||||||
|
source.job.session_id if source_lab is None else source_lab.source_session_id
|
||||||
|
)
|
||||||
|
publish_lab_replay_cache(
|
||||||
|
store.data_dir,
|
||||||
|
source_session_id=source_session_id,
|
||||||
|
lab_session_id=lab_session_id,
|
||||||
|
timeline_start_ns=round(validated.timeline_start_seconds * 1_000_000_000),
|
||||||
|
timeline_end_ns=round(validated.timeline_end_seconds * 1_000_000_000),
|
||||||
|
)
|
||||||
|
metrics = build.report["metrics"]
|
||||||
|
binding = store.publish_lab_instance(
|
||||||
|
session_id=lab_session_id,
|
||||||
|
source_session_id=source_session_id,
|
||||||
|
display_name=display_name,
|
||||||
|
lab_id=lab_id,
|
||||||
|
result_kind="e24-world-motion",
|
||||||
|
result_id=validated.result_id,
|
||||||
|
source_result_id=source.result_id,
|
||||||
|
config_sha256=build.profile_sha256,
|
||||||
|
run_created_at_utc=validated.created_at_utc,
|
||||||
|
duration_seconds=(validated.timeline_end_seconds - validated.timeline_start_seconds),
|
||||||
|
include_recorded_media=False,
|
||||||
|
provenance={
|
||||||
|
"schema_version": "missioncore.e24-lab-publication/v1",
|
||||||
|
"storage_mode": "bounded-world-frame-tracking-and-immutable-source-replay",
|
||||||
|
"source_result_id": source.result_id,
|
||||||
|
"source_lab_session_id": (None if source_lab is None else source_lab.session_id),
|
||||||
|
"source_payloads_mutated": False,
|
||||||
|
"coordinate_frame": "k1-map",
|
||||||
|
"lookahead_frames": 0,
|
||||||
|
"benchmark_sha256": build.benchmark_sha256,
|
||||||
|
"benchmark_passed": metrics["benchmark"]["passed"],
|
||||||
|
"benchmark_passed_events": metrics["benchmark"]["passed_events"],
|
||||||
|
"benchmark_total_events": metrics["benchmark"]["total_events"],
|
||||||
|
"world_motion_processing_p95_ms": metrics["runtime"][
|
||||||
|
"world_motion_frame_processing_ms"
|
||||||
|
]["p95"],
|
||||||
|
"peak_tracks": metrics["runtime"]["peak_tracks"],
|
||||||
|
"navigation_or_safety_accepted": False,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
return PublishedWorldMotionLabInstance(
|
||||||
|
binding=binding,
|
||||||
|
job=lab_job,
|
||||||
|
result=validated,
|
||||||
|
build=build,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def _validate_e23_inputs(
|
def _validate_e23_inputs(
|
||||||
worker_root: Path,
|
worker_root: Path,
|
||||||
source_report_path: Path,
|
source_report_path: Path,
|
||||||
|
|
|
||||||
File diff suppressed because it is too large
Load Diff
|
|
@ -22,6 +22,7 @@ from k1link.compute import (
|
||||||
publish_e21_lab_instance,
|
publish_e21_lab_instance,
|
||||||
publish_e22_lab_instance,
|
publish_e22_lab_instance,
|
||||||
publish_e23_lab_instance,
|
publish_e23_lab_instance,
|
||||||
|
publish_e24_lab_instance,
|
||||||
publish_integrated_lab_instance,
|
publish_integrated_lab_instance,
|
||||||
)
|
)
|
||||||
from k1link.device_plugins.xgrids_k1.analyze import (
|
from k1link.device_plugins.xgrids_k1.analyze import (
|
||||||
|
|
@ -581,6 +582,80 @@ def publish_e23_lab(
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@lab_app.command("publish-e24")
|
||||||
|
def publish_e24_lab(
|
||||||
|
result: Annotated[
|
||||||
|
Path,
|
||||||
|
typer.Option(
|
||||||
|
exists=True,
|
||||||
|
file_okay=False,
|
||||||
|
readable=True,
|
||||||
|
resolve_path=True,
|
||||||
|
help="Accepted full-session integrated result used as the E24 source.",
|
||||||
|
),
|
||||||
|
],
|
||||||
|
profile: Annotated[
|
||||||
|
Path,
|
||||||
|
typer.Option(
|
||||||
|
exists=True,
|
||||||
|
dir_okay=False,
|
||||||
|
readable=True,
|
||||||
|
resolve_path=True,
|
||||||
|
help="Bounded E24 world-motion profile.",
|
||||||
|
),
|
||||||
|
],
|
||||||
|
benchmark: Annotated[
|
||||||
|
Path,
|
||||||
|
typer.Option(
|
||||||
|
exists=True,
|
||||||
|
dir_okay=False,
|
||||||
|
readable=True,
|
||||||
|
resolve_path=True,
|
||||||
|
help="Operator-anchored E24 motion benchmark.",
|
||||||
|
),
|
||||||
|
],
|
||||||
|
session_id: Annotated[
|
||||||
|
str,
|
||||||
|
typer.Option("--session-id", help="New immutable LAB session id."),
|
||||||
|
],
|
||||||
|
lab_id: Annotated[
|
||||||
|
str,
|
||||||
|
typer.Option("--lab-id", help="LAB marker, for example 'LAB E24'."),
|
||||||
|
],
|
||||||
|
display_name: Annotated[
|
||||||
|
str,
|
||||||
|
typer.Option("--display-name", help="Operator-facing saved-session title."),
|
||||||
|
],
|
||||||
|
) -> None:
|
||||||
|
"""Build and publish a bounded world-frame motion-tracking run."""
|
||||||
|
|
||||||
|
repository_root = Path(__file__).resolve().parents[4]
|
||||||
|
try:
|
||||||
|
published = publish_e24_lab_instance(
|
||||||
|
repository_root=repository_root,
|
||||||
|
source_result_root=result,
|
||||||
|
profile_path=profile,
|
||||||
|
benchmark_path=benchmark,
|
||||||
|
lab_session_id=session_id,
|
||||||
|
lab_id=lab_id,
|
||||||
|
display_name=display_name,
|
||||||
|
)
|
||||||
|
except (OSError, SessionIntegrityError, RuntimeError, ValueError) as exc:
|
||||||
|
console.print(f"[red]E24 LAB publication failed:[/red] {exc}")
|
||||||
|
raise typer.Exit(code=2) from exc
|
||||||
|
metrics = published.build.report["metrics"]
|
||||||
|
console.print(
|
||||||
|
"[green]E24 LAB instance published.[/green] "
|
||||||
|
f"session={published.binding.session_id}; "
|
||||||
|
f"source={published.binding.source_session_id}; "
|
||||||
|
f"result={published.binding.result_id}; "
|
||||||
|
f"benchmark={metrics['benchmark']['passed_events']}/"
|
||||||
|
f"{metrics['benchmark']['total_events']}; "
|
||||||
|
f"p95={metrics['runtime']['world_motion_frame_processing_ms']['p95']:.3f}ms; "
|
||||||
|
"source_payloads_mutated=false"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
@app.command("serve")
|
@app.command("serve")
|
||||||
def serve_console(
|
def serve_console(
|
||||||
port: Annotated[
|
port: Annotated[
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,176 @@
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from k1link.compute.world_motion import (
|
||||||
|
WorldMotionTracker,
|
||||||
|
evaluate_motion_benchmark,
|
||||||
|
read_motion_benchmark,
|
||||||
|
read_world_motion_profile,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _profile() -> dict[str, object]:
|
||||||
|
root = Path(__file__).resolve().parents[1]
|
||||||
|
profile, digest = read_world_motion_profile(
|
||||||
|
root / "experiments" / "perception" / "e24_world_motion_profile.json"
|
||||||
|
)
|
||||||
|
assert len(digest) == 64
|
||||||
|
return profile
|
||||||
|
|
||||||
|
|
||||||
|
def _object(track_id: int, x: float, *, group: str = "vehicle") -> dict[str, object]:
|
||||||
|
label = "person" if group == "person" else "car"
|
||||||
|
half_size = [0.35, 0.35, 0.9] if group == "person" else [2.25, 0.925, 0.775]
|
||||||
|
return {
|
||||||
|
"association_group": group,
|
||||||
|
"bbox_xyxy": [100.0, 100.0, 200.0, 200.0],
|
||||||
|
"clustered_points": 32,
|
||||||
|
"cuboid_center_map": [x, 2.0, 0.8],
|
||||||
|
"cuboid_half_size": half_size,
|
||||||
|
"cuboid_quaternion_xyzw": [0.0, 0.0, 0.0, 1.0],
|
||||||
|
"cuboid_status": "accepted-class-prior-amodal-v1",
|
||||||
|
"distance_smoothed_m": 10.0,
|
||||||
|
"geometry": "class-prior-completed-from-visible-lidar-support",
|
||||||
|
"label": label,
|
||||||
|
"score": 0.9,
|
||||||
|
"track_id": track_id,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _provisional_person(track_id: int, x: float) -> dict[str, object]:
|
||||||
|
value = _object(track_id, x, group="person")
|
||||||
|
value.update(
|
||||||
|
{
|
||||||
|
"cuboid_center_map": None,
|
||||||
|
"cuboid_half_size": None,
|
||||||
|
"cuboid_quaternion_xyzw": None,
|
||||||
|
"cuboid_status": "rejected-amodal-completion-support-coverage-below-threshold",
|
||||||
|
"observed_cuboid_center_map": [x, 2.0, 0.8],
|
||||||
|
"observed_cuboid_quaternion_xyzw": [0.0, 0.0, 0.0, 1.0],
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def test_e24_profile_is_bounded_and_has_no_control_authority() -> None:
|
||||||
|
profile = _profile()
|
||||||
|
|
||||||
|
assert profile["source"]["coordinate_frame"] == "k1-map"
|
||||||
|
assert profile["bounds"]["maximum_tracks"] == 192
|
||||||
|
assert profile["authority"] == {
|
||||||
|
"commands_enabled": False,
|
||||||
|
"navigation_or_safety_accepted": False,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def test_e24_reassociates_changed_source_id_and_classifies_static() -> None:
|
||||||
|
tracker = WorldMotionTracker(_profile())
|
||||||
|
last = []
|
||||||
|
for frame in range(30):
|
||||||
|
source_id = 10 if frame < 12 else 77
|
||||||
|
jitter = 0.025 if frame % 2 else -0.025
|
||||||
|
last, world = tracker.update(
|
||||||
|
frame_index=frame,
|
||||||
|
session_seconds=frame * 0.1,
|
||||||
|
objects=[_object(source_id, 10.0 + jitter)],
|
||||||
|
)
|
||||||
|
|
||||||
|
assert last[0]["track_id"] == 240001
|
||||||
|
assert last[0]["source_track_id"] == 77
|
||||||
|
assert world[0]["motion_state"] == "static"
|
||||||
|
assert tracker.snapshot()["reassociated_source_ids"] == 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_e24_classifies_sustained_world_motion_dynamic() -> None:
|
||||||
|
tracker = WorldMotionTracker(_profile())
|
||||||
|
world = []
|
||||||
|
for frame in range(35):
|
||||||
|
_, world = tracker.update(
|
||||||
|
frame_index=frame,
|
||||||
|
session_seconds=frame * 0.1,
|
||||||
|
objects=[_object(20, 5.0 + frame * 0.12, group="person")],
|
||||||
|
)
|
||||||
|
|
||||||
|
assert world[0]["motion_state"] == "dynamic"
|
||||||
|
assert world[0]["speed_mps"] is not None
|
||||||
|
assert world[0]["speed_mps"] > 0.75
|
||||||
|
|
||||||
|
|
||||||
|
def test_e24_confirms_sparse_visible_support_without_promoting_first_hit() -> None:
|
||||||
|
tracker = WorldMotionTracker(_profile())
|
||||||
|
first, first_world = tracker.update(
|
||||||
|
frame_index=0,
|
||||||
|
session_seconds=0.0,
|
||||||
|
objects=[_provisional_person(21, 5.0)],
|
||||||
|
)
|
||||||
|
tracker.update(
|
||||||
|
frame_index=1,
|
||||||
|
session_seconds=0.5,
|
||||||
|
objects=[_provisional_person(21, 5.5)],
|
||||||
|
)
|
||||||
|
third, third_world = tracker.update(
|
||||||
|
frame_index=2,
|
||||||
|
session_seconds=1.0,
|
||||||
|
objects=[_provisional_person(21, 6.0)],
|
||||||
|
)
|
||||||
|
|
||||||
|
assert first[0]["cuboid_status"].startswith("rejected-")
|
||||||
|
assert first_world == []
|
||||||
|
assert third[0]["cuboid_status"] == "accepted-world-track-provisional-e24-v1"
|
||||||
|
assert third_world[0]["track_hits"] == 3
|
||||||
|
|
||||||
|
|
||||||
|
def test_e24_holds_confirmed_track_briefly_then_expires() -> None:
|
||||||
|
tracker = WorldMotionTracker(_profile())
|
||||||
|
for frame in range(12):
|
||||||
|
tracker.update(
|
||||||
|
frame_index=frame,
|
||||||
|
session_seconds=frame * 0.1,
|
||||||
|
objects=[_object(30, 3.0)],
|
||||||
|
)
|
||||||
|
held, held_world = tracker.update(
|
||||||
|
frame_index=12,
|
||||||
|
session_seconds=1.2,
|
||||||
|
objects=[],
|
||||||
|
)
|
||||||
|
expired, expired_world = tracker.update(
|
||||||
|
frame_index=20,
|
||||||
|
session_seconds=2.0,
|
||||||
|
objects=[],
|
||||||
|
)
|
||||||
|
|
||||||
|
assert held[0]["cuboid_status"] == "accepted-world-track-hold-e24-v1"
|
||||||
|
assert held_world[0]["observation_age_ms"] == pytest.approx(100.0)
|
||||||
|
assert expired == []
|
||||||
|
assert expired_world == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_e24_benchmark_contract_and_evaluator() -> None:
|
||||||
|
root = Path(__file__).resolve().parents[1]
|
||||||
|
benchmark, digest = read_motion_benchmark(
|
||||||
|
root / "experiments" / "perception" / "e24_motion_benchmark.json"
|
||||||
|
)
|
||||||
|
event = benchmark["events"][0]
|
||||||
|
rows = [
|
||||||
|
{
|
||||||
|
"session_seconds": 55.0 + index * 0.2,
|
||||||
|
"objects": [
|
||||||
|
{
|
||||||
|
"track_id": 1,
|
||||||
|
"class": event["class_group"],
|
||||||
|
"motion_state": event["expected_motion"],
|
||||||
|
"observation_age_ms": 0.0,
|
||||||
|
}
|
||||||
|
],
|
||||||
|
}
|
||||||
|
for index in range(5)
|
||||||
|
]
|
||||||
|
one_event = {**benchmark, "events": [event]}
|
||||||
|
result = evaluate_motion_benchmark(rows, one_event)
|
||||||
|
|
||||||
|
assert len(digest) == 64
|
||||||
|
assert result["passed"] is True
|
||||||
|
assert result["events"][0]["best_hits"] == 5
|
||||||
Loading…
Reference in New Issue