feat(perception): define stream-first profile and qualification contracts

docs/adr/0049-stream-first-perception-profiles.md

@@ -0,0 +1,236 @@
+# ADR 0049 — Stream-first full perception profiles
+
+Date: 2026-09-01. Status: stage-1 contract candidate; not a runtime cutover.
+Execution order remains the four stages in
+[the ExecPlan](../OBSERVATORY_REALTIME_PROFILES_EXECPLAN.md).
+
+## Decision and owner intent
+
+Observatory LABs test the real-time behavior of a complete perception profile,
+not just the quality of an offline export. A recording replaces the physical
+source, preserving original release timing at 1×. It must not enable a different
+batch implementation. The first profile contains DDRNet-39 GOOSE, RF-DETR native,
+online LiDAR geometry/ranges, temporal/motion, CPU TRAVEL TGS/costmap and advisory
+policy. One dedicated RTX 4090 owns one active full profile.
+
+The first rural prototype accepts existing coarse `hard_surface`, including
+asphalt, sidewalk, bikeway and cobble. Existing `static_obstacle`/`static.unknown`
+is sufficient. No new bollard classifier, extra segmenter, road/sidewalk taxonomy
+or annotation campaign precedes this prototype. Other material rules remain an
+explicit effective mission configuration; rural context alone is not permission
+for all vegetation or soil. Geometry/unknown/freshness overrides material allowance.
+
+EoMT is preserved as a separate future profile. It is not loaded with DDRNet.
+RF-DETR and DDRNet are two required components of the same first profile: one
+supervisor and serialized GPU scheduling, not competing profile jobs.
+
+## CURRENT → TARGET / ownership
+
+| Boundary | Current evidence | Decision / stage |
+| --- | --- | --- |
+| Portable definitions | `portable_run_definitions.py` owns pins, resources and admission | KEEP identity/sealing; EXTEND stream capabilities and qualification in stage 3 |
+| Source requirements | Recorded-only requirements include seekability, one media epoch and init digest | REPLACE as live admission criteria with channel/format/calibration/clock requirements; keep init digest as per-stream integrity |
+| Job queue | `recorded_jobs.py` has durable claims but global single-active-job scope | KEEP durable queue; EXTEND worker-scoped fencing, not unbounded GPU concurrency |
+| Worker agent | Blocking `execute(job)` and final publication | EXTEND incremental lifecycle and cancellation; no per-LAB agent |
+| Runtime | `portable_worker_runtime.py:735` materializes before execution | NEW stream runtime in stage 2; never call this materializer in the real-time path |
+| Source transport | Whole camera archive download/hash/extraction before use | REPLACE startup barrier with bounded metadata, chunks and incremental integrity |
+| Installed LAB V1 | prepare → complete EoMT → complete DDRNet → assemble | Legacy evidence only; not the new profile and not real-time qualified |
+| Camera/source replay | `recorded_source.py` has useful pacing and bounded prefetch, but source-specific timeline and paths | REUSE mechanisms; generic unknown-duration source adapter, no route-sized prefetch |
+| Existing live ingress | `LivePerceptionIngress`, `LiveIngressEvent.wire_bytes`, `LiveSensorSynchronizer` already provide raw-first modality queues, hashes, epochs and bounded binding | REUSE, not a second acquisition system; extend worker/profile fencing and aggregate byte limits; map existing envelopes to the selected transport |
+| Local surface | `RecordedGeometryStore` reads prepared NPZ | REUSE `K1LocalSurfaceShadowEstimator.process` and `lidar_local_surface_geometry`; feed current points/pose |
+| Geometry/ranges | Current point association, exclusive ownership and geometry-only clusters exist | REUSE algorithms; remove recorded-store dependency and expose estimator/frame identity |
+| Temporal/motion | `BoundedSpatialTemporalProvider`, `ClassIndependentMotionEstimator` | REUSE with current-run inputs and reset on epoch/lease change |
+| TGS | Core algorithm reusable; old runners read a full schedule, some require exactly 4489 frames | REUSE TGS parameters/core; REPLACE schedule/file CLI with incremental input |
+| Fusion | M49 TGS can be a neighboring shadow with `tgs_modifies_reference_graph_state=false` | TGS and semantics must influence the current advisory scene, not only viewer composition |
+| Publication | Verified immutable result/publication/recovery exist | KEEP asynchronous final sealing; live results do not await it |
+| Frontend | Setup/job/result surfaces exist, historical M49 viewer links other results | EXTEND common renderer capabilities with current-run layer references and freshness in stage 3 |
+| Recording archive | Source identities and raw evidence must be retained | KEEP; create metadata/index/integrity during acquisition or incrementally, not via preflight full scan |
+
+No changes to the active registry, old queue, source endpoint, UI or canonical
+local server are made by this ADR or the stage-1 validators.
+
+## Versioned handshake and observations
+
+`missioncore.perception-stream-start/v1` is represented by `StreamStart` in
+`src/k1link/perception/realtime_contract.py`. It contains run/source/worker/epoch,
+lease generation, profile/image/effective-config/calibration identities, clock
+domain, input mode and channel declarations. The first profile requires camera,
+point-cloud and pose. Temporary absence of a sample is an explicit modality
+outcome, not removal of a required capability.
+
+It deliberately has no total source bytes, frame count, total duration, whole
+archive, command, environment, complete member inventory or required EOF. The
+initial metadata budget is 64 KiB. Large constant model assets and FOV masks
+belong inside the image; calibration references resolve to bounded metadata.
+
+The stage-2 wire mapping must carry:
+
+- Run/epoch/lease binding on every message; source/channel sequence, capture
+  timestamp and clock domain; independently verifiable payload length/hash.
+- Camera codec initialization and then incremental encoded access units, or
+  bounded fragments thereof. Native decoded representation is BGR8 800×600 KB4.
+  Do not send an entire fMP4 epoch as one observation or turn all frames into PNGs.
+- Current vendor-registered map point increments with point IDs, coordinate
+  frame, units, scalar layout and retained raw attributes; map→sensor pose and
+  calibration identity. The adapter owns vendor decoding; the profile is not a
+  new K1 protocol implementation and cannot command the scanner.
+- Explicit unavailable/gap/end/cancel messages. End is a termination event,
+  never a prerequisite for first inference. Final recording inventory/hash can
+  be sealed after consumption; per-chunk integrity is verified before use.
+- Integer nanoseconds use protobuf integer fields; any JSON projection uses
+  decimal strings for 64-bit times rather than lossy JavaScript numbers.
+
+Maximum payload fragment: 1 MiB; total application inflight bytes: 16 MiB;
+pending decoded camera frames: 2. Fragment reassembly is included in that budget,
+with a deadline and incomplete-chunk accounting. Decoded image/tensor, cloud,
+costmap and archive sinks need their own measured bounds inside the RSS budget.
+Transport flow control alone is not a memory bound or a freshness policy.
+
+gRPC bidirectional streaming is the first transport candidate, not an already
+measured implementation. HTTP control-plane operations remain. The browser does
+not connect directly to model RPCs. One authenticated data-plane endpoint feeds
+the supervised profile; model processes have no arbitrary external network.
+Protocol/codec selection must be measured on actual payload sizes in stage 2;
+changing an archive POST to gRPC without changing execution is not acceptance.
+Existing `missioncore.live-perception-wire/v1` and result wire v2 are migration
+inputs: reuse their semantics and raw-first producer hooks. Their modality count
+caps alone permit more than the proposed 16 MiB aggregate inflight budget; an
+explicit byte bound is required. No duplicate new scanner feed is introduced.
+
+## Time and lifecycle
+
+Open → validate bounded metadata/claim → warm models → Ready → start the replay
+clock → observations and incremental scenes → stop/drain bounded state → close
+and asynchronously seal the immutable receipt. A live source already producing
+data does not queue its entire warmup history: readiness begins at a current
+decodable keyframe, with the skipped interval recorded.
+
+Replay release uses original source intervals, not a convenient fixed 12 Hz and
+not the processing speed. A slow consumer must not stretch source time to make
+inference look real-time. Overload may drop bounded work to preserve liveness,
+but the dropped observations remain in the ledger and fail the strict initial
+performance target. Initial baseline is every camera frame, stride 1. Choosing
+multirate later changes the explicit profile contract; a retained mask is not a
+new inference and cannot receive a fresh capture timestamp.
+
+UTC identifies the run; monotonic clocks measure local durations. Cross-host
+age requires a clock mapping and its uncertainty. No direct subtraction of
+unrelated host monotonic clocks. The common source timeline is separate from
+worker wall-clock time. Causality is bounded by the released-observation
+watermark at decision time, not by whatever later samples exist in the file.
+Each layer retains its own channel/sequence/time; camera anchors and pose/cloud
+association times remain separately visible. Future file look-ahead and offline
+nearest-pose lookup across unreleased data are forbidden.
+
+Lease expiry, cancellation and epoch changes fence the previous producer. No
+new profile may take the GPU until old processes and work have stopped. A
+reconnect creates a new epoch, discards stale backlog, resets temporal/rolling
+state and reacquires codec initialization/keyframe. It never resends a stale
+motor action; this profile has no motor actions at all.
+
+## Outputs and interpretation
+
+Every scene accounts for segmentation, objects, geometry, motion, costmap and
+policy exactly once, with current/held/stale/unavailable state and current-run
+payload identity. Existing source/object/obstacle/temporal/map/threat contracts
+remain the domain vocabulary. `LayerEvidence` adds freshness/completeness checks,
+not a second semantic ontology. In stage 2, layer manifests also carry their full
+input reference sets; a single anchor does not replace camera/cloud/pose lineage.
+
+DDRNet currently crops the center 600×600 to 512×512; outside that ROI is undefined,
+not hard_surface/free. RF-DETR accepts the full 800×600 native raster with its
+pinned valid-FOV handling. Their coordinates must be related explicitly.
+
+RF-DETR's existing filter emits person/cat/dog, but minimum box area 64 px,
+maximum box fraction 0.5, FOV fraction 0.5 and required valid center create known
+small-animal/close-large-object edge cases. They are retained and documented,
+not silently retuned or presented as proven field quality.
+
+Detected-object range is the median camera-Z of owned current support points;
+geometry-only range is nearest Euclidean distance from the sensor. Neither is
+automatically clearance from the physical vehicle body. Missing/ambiguous
+support produces unavailable range, not zero/infinity. Generic static obstacles
+survive absence of a detector class name.
+
+TGS keeps ground/occupied/rejected/unobserved separate. Ground support alone is
+not a traversability or actuation decision. Policy can produce an advisory
+allowed-candidate/high-cost/blocked/unknown outcome with explanation. No planner,
+physical motor command, autonomous driving acceptance or mission configurator
+implementation is introduced.
+
+## Package/dependency decision
+
+The candidate manifest is
+`config/perception/k1-perception-ddrnet39-rfdetr-tgs-prototype-v1.json`.
+It is not installed into the active portable registry and has no invented image
+digest. Model/checkpoint/config identities are pinned; source session IDs and old
+derived output paths in reference experiments are not inherited requirements.
+
+Select process/environment isolation inside one future image: Python 3.12
+supervisor and geometry, RF-DETR TensorRT 11 runtime, existing Python 3.9 / Torch
+1.13.1 cu117 / super-gradients 3.2.0 environment, and CPU C++ TGS. Exchange bounded
+shared-memory/IPC payload references; one supervisor serializes GPU work. This
+avoids a forced dependency upgrade or checkpoint conversion in stage 1. Image
+assembly of the final standalone package and the full schedule pilot remain
+explicit implementation evidence. A temporary common-base image passed separate
+DDRNet, TensorRT, Python 3.12 geometry and C++ TGS execution probes; DDRNet masks
+matched on 64 frames. This proves bounded ABI/execution compatibility in that
+image, not jointly resident models, the supervisor/IPC or a complete profile.
+The probes still mount pinned assets explicitly; they are not standalone proof.
+
+The temporary Triton base lacks Python grpc/protobuf, cv2 and TensorRT bindings;
+native trtexec/server are present. The actual supervisor/transport environment
+must explicitly include and validate its dependencies. Numeric-library thread
+limits are pinned to one after the bounded synthetic local-surface comparison
+(mean 198.0 ms default versus 63.2 ms with limits); algorithm thresholds were not
+changed. Real-cloud timing, output parity and whole-graph scheduling remain gates.
+
+The initial DDRNet image-only probe failed before inference: the installed image
+does not contain the checkpoint at its logical asset path. The bounded baseline
+therefore uses an explicit read-only pinned checkpoint and runner mount. This is
+valid component measurement, but fails the target's standalone packaging claim.
+Super-gradients also needs a writable log directory; a bounded temporary log
+mount was required. The new image must declare scratch areas explicitly without
+requiring a writable host home, code checkout or implicit weight cache.
+
+## Preregistered engineering acceptance
+
+Owner clarification, 2026-09-02 MSK: bounded laboratory overload is an admissible
+experimental outcome. Keep useful profiles that fail the current 4090 target;
+do not block reusable runtime/packaging work solely on that performance result.
+Functional execution, experimental availability, quality and real-time
+qualification are separate dimensions. Qualification belongs to a measured
+profile/config/hardware/source/transport combination, not permanently to a
+model name. More powerful Worker/onboard placement is a future retest target,
+not a claim that existing latency or internal IPC disappears automatically.
+The original budgets and failed measurements below remain unchanged. No slowed
+source clock, hidden drops, stale-as-current evidence or actuation is authorized.
+
+The manifest freezes an initial engineering candidate: p95/p99 full output age
+≤125 ms, required-layer age ≤250 ms, release lag ≤25 ms, first incremental result
+≤1 s after source admission, warmup ≤120 s, stop ≤5 s, VRAM ≤22000 MiB, RSS ≤8192
+MiB. First/last-window backlog growth ≤25 ms and no capacity drops, failures,
+expired selected frames or unaccounted observations. These are prototype targets,
+not physical braking/safety limits or claims of achieved performance.
+
+`ReplayMeasurements`/`realtime_failures` reject full-source preload, EOF-only
+results, slowed replay, incomplete accounting, hidden missing layers and budget
+violations. Declared source gaps must be verified from the input ledger; they
+produce explicit degraded scenes and are not counted as fully fresh perception.
+At least one fully evaluable scene is required. A component FPS figure cannot
+populate this whole-path receipt. A quality comparison or successful export is
+not a substitute for the real-time gate.
+
+Every observation receives a terminal ledger outcome. Track ingress/release,
+selection, decode, inference, completion, emission, drop/expiry/failure and source
+gaps separately. Store startup separately from steady state; record payload bytes,
+queue high-water marks, clock error, RSS/VRAM, image/weights/config/source/equipment
+identities. Export and UI encoding cannot block inference through an unlimited
+queue. A receiver-side render/receipt timestamp, not just Worker completion, ends
+the end-to-end interval.
+
+## Evidence and remaining stage-1 work
+
+See `experiments/perception/PERCEPTION_STREAM_STAGE1_2026-09-01.md` for measured
+component results, bounded validation, Worker maintenance and remaining gates.
+Neither this ADR nor passing synthetic contract tests marks the new runtime ready.