feat(perception): qualify world-frame motion tracking

2026-07-24 11:17:10 +03:00
parent 23181c867b
commit e6dd8c2cf1
8 changed files with 1828 additions and 0 deletions
@@ -0,0 +1,195 @@
+# LAB E24 — world-frame tracking and motion-state qualification
+
+Date: 2026-07-24
+Status: diagnostic artifact accepted; motion benchmark not accepted
+Best immutable replay:
+`LAB E24.5 · Final reproducible world motion · full RAVNOVES00`
+
+## Objective
+
+Test whether the already calculated camera/LiDAR perception result can support a
+bounded, no-lookahead tracker that keeps object identity in `k1-map` and emits a
+conservative `static`, `dynamic`, or `unknown` motion state. The goal is not to
+make prettier boxes. The target is a streaming-equivalent world-state component
+that can later feed collision and path-planning logic.
+
+## Source data and immutable inputs
+
+- Source session: `20260720T065719Z_viewer_live` (`RAVNOVES00`).
+- Camera source: `sensor.camera.right`, calibration slot `camera_1`.
+- Calibration SHA-256:
+  `05f3ad9b38b3a4fc95388a8ec83da83c745e217709e51787b3d5aad0969f6fa9`.
+- Source integrated result:
+  `e10-integrated-perception-34ade557b5636717aa497fc00355b84df7063e483a175f2f5d3f04c03df1c898`.
+- Source result SHA-256:
+  `418cd598277d50f4ee81c042d98ece9be10b1b8dc49bf3e2dcc2f2a9ca1c6dc3`.
+- Frames: 4,489.
+- Source timeline: 35.421857292–484.044857292 session seconds.
+- Source detector: YOLOX-S, COCO-80, 640×640.
+- Source semantic model:
+  `tue-mps/cityscapes_semantic_eomt_large_1024`, FP16 autocast.
+- Source fusion: 3,915 LiDAR-fused frames and 3,349 accepted cuboids.
+- Raw point cloud, ground returns, trajectory, camera media, calibration,
+  detector output and semantic masks were not changed.
+
+The E24 runs reuse the accepted E19 result. They do not repeat GPU inference.
+E24 is a deterministic CPU post-fusion stage executed locally with Python
+3.12/NumPy 2.5.1.
+
+## Operator benchmark
+
+The committed benchmark uses the operator's approximate RAVNOVES00 timeline:
+
+- 58 s: woman in a pink shirt with a dog, left;
+- 65 s: person loading a car, right;
+- 115 s: adult and child, left;
+- 127 s: pedestrian on the road, right;
+- 155 s: woman with stroller and child, left;
+- 170 s: oncoming vehicle, right;
+- 173 s: vulnerable-road-user group, left;
+- 183 s: vehicle moving in the rig direction, right;
+- three static-vehicle control windows before the moving-vehicle events.
+
+The benchmark is not labeled ground truth. It is a reproducible operator
+annotation contract with explicit time windows, class groups, minimum hits and
+minimum observed spans.
+
+## Implemented pipeline
+
+E24 implements tracking-by-detection in the map frame:
+
+1. accepted 3D cuboids are measurements in `k1-map`;
+2. visible LiDAR support rejected only by amodal-completion coverage may enter as
+   a high-uncertainty provisional measurement;
+3. a bounded constant-velocity Kalman state estimates position, velocity and
+   covariance;
+4. global greedy association uses class compatibility, predicted map position,
+   shape distance, detector-ID aliases and strict ID-switch gates;
+5. provisional tracks require repeated support before a cuboid is published;
+6. short measurement gaps are explicit predicted holds, limited to 450 ms;
+7. motion state requires a bounded history, minimum displacement, robust speed,
+   direction consistency and hysteresis;
+8. every world object exposes observation age, track age, position/velocity
+   covariance, motion confidence and source track ID.
+
+State is bounded to 192 tracks, has no lookahead, and has no command,
+navigation, or safety authority.
+
+## Run series
+
+### E24.1 — accepted cuboids only
+
+- Benchmark: 2/11.
+- Processing p95: 0.600 ms/frame.
+- Finding: static vehicle tracks can be held, but people and moving vehicles
+  often lack enough accepted 3D measurements to leave `unknown`.
+
+### E24.2 — provisional visible-support measurements
+
+- Benchmark: 6/11.
+- Processing p95: 0.581 ms/frame.
+- Finding: recovered motion evidence for the 58 s, 115 s and 155 s pedestrian
+  windows and the 183 s vehicle; broad reassociation also merged nearby vehicle
+  tracks and created false motion.
+
+### E24.3 — conservative ID-switch gates
+
+- Benchmark: 7/11.
+- Processing p95: 0.638 ms/frame.
+- Finding: strict ID-switch gates reduced false merges, but result identity did
+  not yet contain the postprocessor implementation SHA-256.
+
+### E24.4 — reproducible conservative ID-switch gates
+
+- Benchmark: 7/11.
+- Processing p95: 0.611 ms/frame.
+- Finding: implementation SHA-256 was added before the formatter produced the
+  final source bytes, so a final identity-exact replay was required.
+
+### E24.5 — final identity-exact replay
+
+- Result:
+  `e10-integrated-perception-e56fa878cba3f9f9e5943831cad0292643e49571f212a4d341bca28626515d80`.
+- Profile SHA-256:
+  `dff304161ace7846dd7886f43877c6cae35737d4e7384b76dae6a430aeec1bc9`.
+- Benchmark SHA-256:
+  `f1fa0f2b17b7148a1f5371339faf47a9ef25bfe9003fce4559f73be4f408818b`.
+- Implementation SHA-256:
+  `3ca4c464eabaab3a89e7e684704d6d0288bb1dcd3dd455423456bed91d9d2ecc`.
+- Benchmark: 7/11.
+- Processing mean/p50/p95/max:
+  0.328/0.287/0.713/53.743 ms per frame.
+- Peak live track states: 10 of 192.
+- Created tracks: 251.
+- Cross-source-ID reassociations: 24.
+- Explicit held cuboids: 1,859.
+- Published motion states: 3,139 unknown, 1,494 static, 875 dynamic.
+
+Passed events:
+
+- dynamic person at 58 s;
+- dynamic person at 65 s;
+- static vehicle evidence at 70–90 s;
+- dynamic person at 115 s;
+- dynamic person at 155 s;
+- oncoming dynamic vehicle at 170 s;
+- same-direction dynamic vehicle at 183 s.
+
+Failed events:
+
+- 127 s pedestrian: only one publishable 3D observation after confirmation;
+- 132–148 s static vehicles: visible-surface/amodal-center drift still creates
+  false dynamic hypotheses;
+- 161–166 s static vehicles: insufficient persistent 3D support;
+- 173 s vulnerable-road-user group: measurements are too sparse after strict
+  identity separation.
+
+## Main conclusion
+
+E24 proves that the tracker itself fits an eventual near-real-time budget by a
+large margin. It also proves that temporal smoothing and threshold tuning alone
+cannot make the current cuboid center a reliable motion measurement.
+
+For parked vehicles, the fitted/completed cuboid center can drift systematically
+as the rig passes and a different visible LiDAR surface becomes dominant. A
+constant-velocity tracker correctly interprets that input as motion because the
+measurement itself is moving. Relaxed association hides gaps but merges nearby
+objects; strict association avoids most merges but exposes the missing
+measurement continuity.
+
+Therefore E24.5 is an accepted diagnostic artifact, not an accepted motion
+classifier. It must not feed navigation or safety decisions.
+
+## Next experimental gate
+
+E25 must improve the measurement model rather than retune E24:
+
+- maintain object-level LiDAR support/occupancy evidence across scans in the map
+  frame instead of tracking only a completed box center;
+- estimate a static-landmark hypothesis from persistent support and compare it
+  against a moving-object hypothesis;
+- keep raw ground and the full cloud visible; exclude ground only from the
+  object-proposal branch;
+- split coverage, association, motion-state and false-dynamic metrics so a
+  missing observation cannot be mistaken for a correct static classification;
+- use the same operator anchors plus explicit per-object spatial annotations for
+  the failed 127 s, 132–148 s, 161–166 s and 173 s windows;
+- accept E25 only after false dynamic vehicle tracks are reduced without losing
+  the two known moving vehicles.
+
+The planner-facing representation should be occupancy/footprint, velocity,
+uncertainty and short-horizon prediction. 3D boxes remain an operator-facing
+representation and one measurement carrier, not the sole collision model.
+
+## Validation
+
+- Unit tests cover bounded state, ID reassociation, static/dynamic
+  classification, provisional confirmation, short holds and benchmark parsing.
+- E24.5 result passes the existing integrated-perception validator.
+- Saved session `lab-e24-5-world-motion` is `ready` and replayable on
+  `http://127.0.0.1:8000/`.
+- A cold full-session perception request completed with HTTP 200 in 51.958 s
+  and produced a 69,303,555-byte Rerun stream with `RRF2` magic. This cold
+  materialization time is a viewer-publication limitation, not E24 tracking
+  latency; the generated overlay is now cached.
+- The existing localhost server was not stopped or restarted.
@@ -0,0 +1,97 @@
+{
+  "schema_version": "missioncore.e24-motion-benchmark/v1",
+  "benchmark_id": "ravnoves00-operator-anchors-v1",
+  "source_session_id": "20260720T065719Z_viewer_live",
+  "timeline": "session_seconds",
+  "annotation_status": "operator-approximate",
+  "events": [
+    {
+      "id": "dynamic-person-woman-dog-left-58s",
+      "window_seconds": [54.0, 62.0],
+      "class_group": "person",
+      "expected_motion": "dynamic",
+      "minimum_hits": 3,
+      "minimum_span_seconds": 0.2
+    },
+    {
+      "id": "dynamic-person-loading-car-right-65s",
+      "window_seconds": [62.0, 69.0],
+      "class_group": "person",
+      "expected_motion": "dynamic",
+      "minimum_hits": 3,
+      "minimum_span_seconds": 0.2
+    },
+    {
+      "id": "static-vehicles-before-moving-70-90s",
+      "window_seconds": [70.0, 90.0],
+      "class_group": "vehicle",
+      "expected_motion": "static",
+      "minimum_hits": 8,
+      "minimum_span_seconds": 0.7
+    },
+    {
+      "id": "dynamic-person-adult-child-left-115s",
+      "window_seconds": [111.0, 120.0],
+      "class_group": "person",
+      "expected_motion": "dynamic",
+      "minimum_hits": 3,
+      "minimum_span_seconds": 0.2
+    },
+    {
+      "id": "dynamic-person-road-right-127s",
+      "window_seconds": [124.0, 132.0],
+      "class_group": "person",
+      "expected_motion": "dynamic",
+      "minimum_hits": 3,
+      "minimum_span_seconds": 0.2
+    },
+    {
+      "id": "static-vehicles-mid-run-132-148s",
+      "window_seconds": [132.0, 148.0],
+      "class_group": "vehicle",
+      "expected_motion": "static",
+      "minimum_hits": 8,
+      "minimum_span_seconds": 0.7
+    },
+    {
+      "id": "dynamic-person-stroller-group-left-155s",
+      "window_seconds": [151.0, 161.0],
+      "class_group": "person",
+      "expected_motion": "dynamic",
+      "minimum_hits": 3,
+      "minimum_span_seconds": 0.2
+    },
+    {
+      "id": "static-vehicles-before-oncoming-161-166s",
+      "window_seconds": [161.0, 166.0],
+      "class_group": "vehicle",
+      "expected_motion": "static",
+      "minimum_hits": 8,
+      "minimum_span_seconds": 0.7
+    },
+    {
+      "id": "dynamic-vehicle-oncoming-right-170s",
+      "window_seconds": [167.0, 173.0],
+      "class_group": "vehicle",
+      "expected_motion": "dynamic",
+      "minimum_hits": 3,
+      "minimum_span_seconds": 0.15
+    },
+    {
+      "id": "dynamic-person-bike-group-left-173s",
+      "window_seconds": [170.0, 179.0],
+      "class_group": "vulnerable_road_user",
+      "expected_motion": "dynamic",
+      "minimum_hits": 3,
+      "minimum_span_seconds": 0.2
+    },
+    {
+      "id": "dynamic-vehicle-same-direction-right-183s",
+      "window_seconds": [180.0, 188.0],
+      "class_group": "vehicle",
+      "expected_motion": "dynamic",
+      "minimum_hits": 3,
+      "minimum_span_seconds": 0.2
+    }
+  ]
+}
@@ -0,0 +1,100 @@
+{
+  "schema_version": "missioncore.e24-world-motion-profile/v1",
+  "profile_id": "lab-e24-world-frame-motion-v1",
+  "mode": "recorded-streaming-qualification",
+  "source": {
+    "source_id": "sensor.camera.right",
+    "coordinate_frame": "k1-map",
+    "calibration_slot": "camera_1",
+    "calibration_sha256": "05f3ad9b38b3a4fc95388a8ec83da83c745e217709e51787b3d5aad0969f6fa9"
+  },
+  "association": {
+    "maximum_center_distance_m": 2.2,
+    "maximum_speed_mps": 16.0,
+    "maximum_gap_allowance_m": 2.5,
+    "maximum_reassociation_gap_seconds": 0.55,
+    "maximum_reassociation_distance_m": {
+      "person": 1.2,
+      "bicycle": 1.2,
+      "motorcycle": 1.2,
+      "vehicle": 0.75,
+      "default": 0.75
+    },
+    "source_track_alias_bonus": 0.8,
+    "size_distance_weight": 0.35
+  },
+  "filter": {
+    "initial_position_sigma_m": 0.8,
+    "initial_velocity_sigma_mps": 2.0,
+    "measurement_sigma_m": 1.6,
+    "provisional_measurement_sigma_m": 2.8,
+    "minimum_measurement_sigma_m": 0.18,
+    "support_points_saturation": 64,
+    "acceleration_sigma_mps2": 2.0,
+    "shape_alpha": 0.18
+  },
+  "provisional": {
+    "allowed_groups": [
+      "person",
+      "bicycle",
+      "motorcycle",
+      "vehicle"
+    ],
+    "nominal_half_size_m": {
+      "person": [0.35, 0.35, 0.9],
+      "bicycle": [0.9, 0.325, 0.75],
+      "motorcycle": [0.9, 0.35, 0.75],
+      "vehicle": [2.25, 0.925, 0.775],
+      "default": [0.5, 0.5, 0.5]
+    }
+  },
+  "tracking": {
+    "minimum_confirmation_hits": 4,
+    "provisional_confirmation_hits": 3,
+    "hold_seconds": 0.45
+  },
+  "motion": {
+    "history_seconds": 1.8,
+    "minimum_history_seconds": 0.8,
+    "dynamic_enter_speed_mps": {
+      "person": 0.45,
+      "bicycle": 0.55,
+      "motorcycle": 0.65,
+      "vehicle": 0.75,
+      "default": 0.75
+    },
+    "dynamic_exit_speed_mps": 0.4,
+    "dynamic_minimum_displacement_m": {
+      "person": 0.45,
+      "bicycle": 0.5,
+      "motorcycle": 0.55,
+      "vehicle": 0.65,
+      "default": 0.65
+    },
+    "minimum_direction_consistency": 0.62,
+    "dynamic_confirmation_frames": {
+      "person": 1,
+      "bicycle": 1,
+      "motorcycle": 1,
+      "vehicle": 2,
+      "default": 2
+    },
+    "static_enter_speed_mps": 0.28,
+    "static_exit_speed_mps": 0.55,
+    "static_maximum_displacement_m": 0.42,
+    "static_confirmation_frames": 7
+  },
+  "bounds": {
+    "track_id_start": 240001,
+    "maximum_tracks": 192,
+    "maximum_track_idle_seconds": 1.5
+  },
+  "acceptance": {
+    "maximum_processing_p95_ms": 8.0,
+    "maximum_tracks_observed": 192
+  },
+  "authority": {
+    "commands_enabled": false,
+    "navigation_or_safety_accepted": false
+  }
+}