NODEDC_MISSION_CORE/docs/adr/0049-stream-first-perception-profiles.md

# ADR 0049 — Stream-first full perception profiles

Date: 2026-09-01; updated 2026-09-02 16:52 MSK. Status: stage-1 engineering
baseline complete; stage-2 full binary GPU pilot consumes real scoped host facts.
External transport/authentication, native-host GPU inventory, standalone image
and product cutover remain open; local proof is not physical-live acceptance.
Execution order remains the four stages in
[the ExecPlan](../OBSERVATORY_REALTIME_PROFILES_EXECPLAN.md).

## Decision and owner intent

Observatory LABs test the real-time behavior of a complete perception profile,
not just the quality of an offline export. A recording replaces the physical
source, preserving original release timing at 1×. It must not enable a different
batch implementation. The first profile contains DDRNet-39 GOOSE, RF-DETR native,
online LiDAR geometry/ranges, temporal/motion, CPU TRAVEL TGS/costmap and advisory
policy. One dedicated RTX 4090 owns one active full profile.

The first rural prototype accepts existing coarse `hard_surface`, including
asphalt, sidewalk, bikeway and cobble. Existing `static_obstacle`/`static.unknown`
is sufficient. No new bollard classifier, extra segmenter, road/sidewalk taxonomy
or annotation campaign precedes this prototype. Other material rules remain an
explicit effective mission configuration; rural context alone is not permission
for all vegetation or soil. Geometry/unknown/freshness overrides material allowance.

EoMT is preserved as a separate future profile. It is not loaded with DDRNet.
RF-DETR and DDRNet are two required components of the same first profile: one
supervisor and serialized GPU scheduling, not competing profile jobs.

## CURRENT → TARGET / ownership

| Boundary | Current evidence | Decision / stage |
| --- | --- | --- |
| Portable definitions | `portable_run_definitions.py` owns pins, resources and admission | KEEP identity/sealing; EXTEND stream capabilities and qualification in stage 3 |
| Source requirements | Recorded-only requirements include seekability, one media epoch and init digest | REPLACE as live admission criteria with channel/format/calibration/clock requirements; keep init digest as per-stream integrity |
| Job queue | `recorded_jobs.py` has durable claims but global single-active-job scope | KEEP durable queue; EXTEND worker-scoped fencing, not unbounded GPU concurrency |
| Worker agent | Blocking `execute(job)` and final publication | EXTEND incremental lifecycle and cancellation; no per-LAB agent |
| Runtime | `portable_worker_runtime.py:735` materializes before execution | NEW stream runtime in stage 2; never call this materializer in the real-time path |
| Source transport | Whole camera archive download/hash/extraction before use | REPLACE startup barrier with bounded metadata, chunks and incremental integrity |
| Installed LAB V1 | prepare → complete EoMT → complete DDRNet → assemble | Legacy evidence only; not the new profile and not real-time qualified |
| Camera/source replay | `recorded_source.py` has useful pacing and bounded prefetch, but source-specific timeline and paths | REUSE mechanisms; generic unknown-duration source adapter, no route-sized prefetch |
| Existing live ingress | `LivePerceptionIngress`, `LiveIngressEvent.wire_bytes`, `LiveSensorSynchronizer` already provide raw-first modality queues, hashes, epochs and bounded binding | REUSE, not a second acquisition system; extend worker/profile fencing and aggregate byte limits; map existing envelopes to the selected transport |
| Local surface | `RecordedGeometryStore` reads prepared NPZ | REUSE `K1LocalSurfaceShadowEstimator.process` and `lidar_local_surface_geometry`; feed current points/pose |
| Geometry/ranges | Current point association, exclusive ownership and geometry-only clusters exist | REUSE algorithms; remove recorded-store dependency and expose estimator/frame identity |
| Temporal/motion | `BoundedSpatialTemporalProvider`, `ClassIndependentMotionEstimator` | REUSE with current-run inputs and reset on epoch/lease change |
| TGS | Core algorithm reusable; old runners read a full schedule, some require exactly 4489 frames | REUSE TGS parameters/core; REPLACE schedule/file CLI with incremental input |
| Fusion | M49 TGS can be a neighboring shadow with `tgs_modifies_reference_graph_state=false` | TGS and semantics must influence the current advisory scene, not only viewer composition |
| Publication | Verified immutable result/publication/recovery exist | KEEP asynchronous final sealing; live results do not await it |
| Frontend | Setup/job/result surfaces exist, historical M49 viewer links other results | EXTEND common renderer capabilities with current-run layer references and freshness in stage 3 |
| Recording archive | Source identities and raw evidence must be retained | KEEP; create metadata/index/integrity during acquisition or incrementally, not via preflight full scan |

No changes to the active registry, old queue, source endpoint, UI or canonical
local server are made by this ADR or the stage-1 validators.

## Versioned handshake and observations

### Current host observation adapter (2026-09-02 16:52, `61cbdb3`)

`WorkerControlChannel` / `WorkerControlPump` supply the common readiness monitor
with actual GPU name/driver/clocks and Docker image/configured resource limits /
GPU-access inventory from `collect_worker_host.ps1` on the trusted Windows host.
Only the host collector executes fixed read-only Docker/NVIDIA queries. The AI
container has no Docker socket, host setters or network access. Request and reply
directories are separate; the reply mount must be read-only and cannot also be
reachable through the writable output mount. This local filesystem authority is
NOT authentication for external sensor/GCS traffic.

One pending random nonce, activation digest, sequence and a16KiB reply bound
prevent history/queue growth. The collector reads facts AFTER receiving that
challenge. Observation time is the local Linux request-start monotonic value,
not the reply receipt or Windows timestamp. Delayed replies stay old; missing,
malformed or different-nonce replies cannot refresh readiness. A known matching
reply with changed container/image/owner or competitors fences execution.
Owner, effective config and warmup remain local-controller facts, never accepted
from the sensor stream or inferred from utilization. The pump does not renew the
lease; its I/O runs outside lifecycle locks and the heartbeat thread.

Scope is part of the sealed effective configuration: `docker-gpu-access` means
observed Docker GPU-capable containers, NOT all native host/WSL GPU processes.
It cannot satisfy an envelope requiring `host-compute`. Windows reports PID4 /
Insufficient Permissions; native host coverage stays explicitly unproved.
Incomplete inventory or unavailable GPU telemetry cannot be waived even by
`labelled-experiment`; measured low clocks can be logged as overload conditions.

Bootstrap acquires the local lease and attaches readiness before spawning any
model. Warmup telemetry lag blocks further startup without destroying already
loaded children or invalidating an input epoch that has not started. Active
stream lag pauses input, keeps models and local lease, then requires the existing
fresh-keyframe/sensor resynchronization and temporal reset. The bounded pilot's
outer watchdog remains; this is not yet a durable production controller.

Measured normal graph:128/128 exact raw outputs,50 real observations, p95/p99
91.157/97.946ms. Delaying a host reply2200ms:102 completed,26 intentional omissions,
two expired requests, same four resident PIDs/lease; resume seq50, old temporal
stores/cells absent, first scene fresh. Result receipt gap2.719s is NOT latency
from the Windows fault start (clock domains are not subtracted). Both trials
use the temporary authorized stock-clock envelope, not general qualification.
404 focused tests; evidence SHA-256
`aac125bd1b182d164c95d18089793fa54bf9797478bce1afe3de2a1d9093f6ef`.
The first warmup abort and Windows atomic-replacement correction are retained
in the experiment journal. Historical readiness notes below retain their dates.

`missioncore.perception-stream-start/v1` is represented by `StreamStart` in
`src/k1link/perception/realtime_contract.py`. It contains run/source/worker/epoch,
lease generation, profile/image/effective-config/calibration identities, clock
domain, input mode and channel declarations. The first profile requires camera,
point-cloud and pose. Temporary absence of a sample is an explicit modality
outcome, not removal of a required capability.

It deliberately has no total source bytes, frame count, total duration, whole
archive, command, environment, complete member inventory or required EOF. The
initial metadata budget is 64 KiB. Large constant model assets and FOV masks
belong inside the image; calibration references resolve to bounded metadata.

The stage-2 wire mapping must carry:

- Run/epoch/lease binding on every message; source/channel sequence, capture
  timestamp and clock domain; independently verifiable payload length/hash.
- Camera codec initialization and then incremental encoded access units, or
  bounded fragments thereof. Native decoded representation is BGR8 800×600 KB4.
  Do not send an entire fMP4 epoch as one observation or turn all frames into PNGs.
- Current vendor-registered map point increments with point IDs, coordinate
  frame, units, scalar layout and retained raw attributes; map→sensor pose and
  calibration identity. The adapter owns vendor decoding; the profile is not a
  new K1 protocol implementation and cannot command the scanner.
- Explicit unavailable/gap/end/cancel messages. End is a termination event,
  never a prerequisite for first inference. Final recording inventory/hash can
  be sealed after consumption; per-chunk integrity is verified before use.
- Integer nanoseconds use protobuf integer fields; any JSON projection uses
  decimal strings for 64-bit times rather than lossy JavaScript numbers.

Maximum payload fragment: 1 MiB; total application inflight bytes: 16 MiB;
pending decoded camera frames: 2. Fragment reassembly is included in that budget,
with a deadline and incomplete-chunk accounting. Decoded image/tensor, cloud,
costmap and archive sinks need their own measured bounds inside the RSS budget.
Transport flow control alone is not a memory bound or a freshness policy.

gRPC bidirectional streaming is the first transport candidate, not an already
measured implementation. HTTP control-plane operations remain. The browser does
not connect directly to model RPCs. One authenticated data-plane endpoint feeds
the supervised profile; model processes have no arbitrary external network.
Protocol/codec selection must be measured on actual payload sizes in stage 2;
changing an archive POST to gRPC without changing execution is not acceptance.
Existing `missioncore.live-perception-wire/v1` and result wire v2 are migration
inputs: reuse their semantics and raw-first producer hooks. Their modality count
caps alone permit more than the proposed 16 MiB aggregate inflight budget; an
explicit byte bound is required. No duplicate new scanner feed is introduced.

## Time and lifecycle

Stage-2 full-graph recovery increment, 2026-09-02 15:56 (`70927ea`):
`--recover-input` connects lifecycle to PilotController, RecordingSource,
BinaryGraphBridge, GPU/CPU calls and result binding. Source clock mapping is
fixed before ingress; fault windows never restart/slow it. Bounded diagnostic
`--input-gap sequence:milliseconds` is explicit in effective config. Reconnect
polls quiescence without queuing old events; complete accounting distinguishes
skipped from processed input. Historical one-shot defaults remain available.

The receiver waits for an independently decodable fragment and fresh causal
sensor pair. Decoder-local index resets; original source sequence/timestamps
do not. Actual decoder keyframe validation remains. The graph recreates CPU
surface/geometry/temporal/motion/rolling/threat providers using the same configs
and clears cached segmentation. RF-DETR backend, DDRNet/decoder/TGS children
remain. TGS C++ creates an estimator per request; costmap arrays are per-call,
so no native temporal reset command or restart is needed. Rolling input window
resets independently. GPU/CPU/result gates bind the input epoch, not activation.
Obsolete results release their input rather than terminating the profile;
review removes a lingering collector tuple reference to prior input too.

Two normal/faulted full-profile pairs pass on Worker006. Normal raw parity is
128/128; final p95/p99 85.775/89.385ms. Faulted runs preserve four PIDs and clear
nonempty temporal stores, resuming on seq30/110 after gaps at seq16/72. All
post-resume histories and observed costmap timestamps belong to the new epoch;
first scenes have fresh six-layer evidence and new DDRNet inference. No result
was published during pauses. GPU calls were idle at injection; mid-compute
interruption is covered by focused tests, not new real-GPU cancellation proof.

Recovery availability differs from accepted-frame processing latency: 150ms gap
takes1.476s to next complete scene; 2.2s gap takes3.857s, including waiting for
keyframe/sensor alignment. Faulted output is76 processed +52 skipped cameras,
zero unaccounted; none is replayed. Its p95/p99 83.163/89.729ms describes accepted
frames only. Timings use temporary stock clocks2610/10251MHz; whole-route
availability, auto-clock and network timing remain unqualified. Real collector,
authentication, cross-host clock mapping, standalone and onboard safety are open.

383 tests pass. Evidence330 artifacts,236 Worker snapshot hashes/114 current
files verified; `.runtime/perception-stage2-resume-worker-20260902T1545MSK/manifest.json`,
SHA-256 `a9449fa4c012e30ce8a06025790c0e110eeebab7c1859fdebf56d5e785961ea3`.
Initial preflight abort (512MiB threshold below known534MiB idle baseline) is
preserved; no models/clocks ran in it. Auto clocks/four scoped services restored,
empty/retired fixture volumes and containers removed; legacy restart loops not
fixed. The earlier CPU-only increment below is historical, not latest status.
No full-source transfer, new model, UI or actuator path is introduced.

Owner clarification and stage-2 continuity increment, 2026-09-02 15:30
(`c31c46c`): temporary input/network loss must not unload the resident models.
Unknown/stale GPU telemetry is not proof of a competing owner. The desired
response is waiting and resynchronization; verified owner conflict, actual
local lease loss, child/model/decoder failure and explicit cancellation remain
terminal. This supersedes the blanket terminal-readiness TARGET below; previous
terminal fault probes are retained as historical evidence of their opt-in mode.

Two lifetimes are explicit. The resident activation keeps its original
`StreamStart`, local Worker lease and lease generation. Reconnecting input gets
a new `StreamStart.epoch_id`, bound by existing wire v2 on every packet/result;
acquisition session/generation remains independently checked. A network/GCS
heartbeat is not the local ownership heartbeat. The trusted resident controller
continues renewing its own lease while the input waits. No second profile may
claim the GPU during that wait, and telemetry alone cannot renew/release a lease.

`StreamingLifecycle(recover_input=True, source_clock_ns=...)` requires a trusted
source-to-local-clock mapping and, if monitoring is configured, a
`WorkerReadinessMonitor(recoverable=True)`. Both choices belong in the sealed
effective config. Their defaults preserve the historical one-shot diagnostic;
the current full PilotController/BinaryGraphBridge has not been switched. This
compatibility default is not the final product's desired outage policy.

The input phases are active, waiting and synchronizing. Resident GraphState may
remain RUNNING while input is waiting; that means the processes remain loaded,
not that results or actuation are ready. EOF without End, an IPC read deadline,
connection loss or a declared/source-sequence gap pauses the enabled adapter.
Pending input is discarded; active borrowed buffers remain owned until their
callback drains, and its old result cannot pass the epoch/phase gate. A new
connection waits for the old receiver, active callbacks and mailbox-owned work
to drain. No unbounded queue, replay catch-up, future evidence or new model
process is introduced. Explicit End terminates the source; Cancel remains a
real cancellation. Malformed data and actual decoder failure are not relabelled
as a recoverable transport outage. External network error handling remains an
unqualified adapter boundary, not proved by the IPC socket tests.

Resume requires a new init and a successfully decoded random-access H.264 frame,
fresh pose/points from the new epoch, and a temporal-state reset. The measured
decoder resets codec references/DTS inside the same child process; sensor-window
reset retains its bounded allocation but clears rolling state. `ResumeEvidence`
must be derived by the trusted profile adapter, not accepted as a client Boolean.
Every timestamp is at/after reconnect cutoff and no later than the camera;
camera age ≤250 ms, pose/newest-points age ≤100 ms, oldest-points age ≤250 ms and
pose/points skew ≤100 ms. The lifecycle rechecks evidence after reset and cannot
resume if another pause arrived meanwhile. Full graph tracking/motion/TGS/costmap
reset and CPU-consumer old-result disposal are required next, not yet proved by
the diagnostic callback. Replay source time must keep advancing at 1× through
the outage; the CPU fixture proves this only with a synthetic tick source.

Recoverable readiness maps expired/missing inventory, incomplete warmup and
strict-envelope performance unavailability to waiting, not process destruction.
Fresh facts permit beginning resynchronization, not automatic reuse of the old
epoch. Confirmed wrong owner/generation/worker/image/config, known competing GPU
clients and invalid trusted clock/snapshot ordering still fence the activation.
Labelled performance experiments keep their recorded violations and remain
unqualified. The runtime does not manipulate GPU clocks, other services or a
vehicle. Eventual onboard stop/hold, stale-command rejection and motion-resume
policy are independent future safety gates; restored inference is not permission
to move, and GCS heartbeat is not proof of fresh perception.

375 focused tests pass, including repeated reconnects, stale input/results,
pause during reset, telemetry recovery and retained hard-fault fencing. Worker
CPU-only proof: three epochs, 150-ms and 2.2-s outages, unchanged decoder/sentinel
PIDs, identical BGR 3/3, peak input 5,275,461 bytes and complete cleanup. It uses
one real H.264 fragment and synthetic timestamps/pose/points; the sentinel is not
a GPU model, and temporal reset is a diagnostic fixture. No full-graph recovery,
new GPU latency, real inventory collector, external network or standalone-image
qualification follows. Evidence: 85 artifacts, 73 verified code hashes;
`.runtime/perception-stage2-continuity-worker-20260902T1518MSK/manifest.json`,
SHA-256 `fbd327d129c88caa13b7183b1ae2c4767e877a6f5472c0f430875cfd7401ed46`.

Stage-2 controller-readiness increment, 2026-09-02 14:52 (`92625bf`):
`WorkerReadinessMonitor` consumes existing `WorkerOperatingEnvelope` and
`WorkerSnapshot` under the same lifecycle lock as admission. No host I/O occurs
under this lock. The trusted controller, not a sensor payload, provides the
snapshot in a locally comparable Worker monotonic clock domain and seals the
mode/envelope into its effective configuration. Snapshot arrival is not a lease
renewal, and renewing a lease does not refresh inventory. Identity, exclusive
ownership, known inventory and freshness are mandatory in both modes.

`strict-envelope` rejects post-warmup performance conditions outside the
preregistered envelope. `labelled-experiment` permits these conditions, including
unknown performance facts, but records violations and never grants real-time
qualification. It cannot waive unknown/competing GPU clients, stale inventory,
changed worker/image/config/owner or missing warmup. Low clocks during STARTING
do not prohibit model warmup; transition to RUNNING checks warmup explicitly.
Freshness is checked before spawn/admission/compute/result/heartbeat and in the
50-ms watchdog. A late telemetry refresh cannot resurrect expired authority;
old StreamStart updates are rejected without stopping the current owner.

Readiness loss enters STOPPING and stops only owned child groups. A new owner
still requires verified child/thread/callback/input release and a new generation
and epoch. The monitor cannot clear an active durable lease, force recovery,
start a replacement model, change host clocks/quotas or stop other services.
It retains one snapshot and bounded reason sets, not unlimited telemetry history.

The existing diagnostic PilotController has no continuous real collector yet;
unconfigured runtimes expose `worker_readiness.enabled=false` and cannot claim
monitored readiness. No authentication or host-clock translation is implemented
by this in-process method. External inventory collection, control-channel
delivery, production admission/recovery and standalone packaging remain stage 2.
349 focused tests and four sequential Worker Linux CPU-only scenarios pass.
The latter use synthetic GPU facts, zero GPU devices/models and real child
processes: stop 10.676–63.994 ms, zero residual input, released fixture generation
4. These are not new GPU graph timings. Evidence manifest SHA-256:
`4588995d763fe82d896b42689e981f1d00feedaf9d6aab29f4579f771d03da3a`.

Stage-2 raw IPC increment, 2026-09-02 (`da60fef`, `a937400`, `79ce55d`):
`streaming_wire.py` maps existing `LiveIngressEvent` fields into
`missioncore.live-perception-wire/v2`. The v1 producer hooks, class and legacy
serialization remain unchanged. A bounded length-prefixed JSON header carries
decimal uint64 projections; payloads remain binary. This is a controller-supplied
IPC socket candidate, not a selection/replacement of the intended gRPC network
transport. No listener, arbitrary endpoint, scanner feed or command path is added.

Open compares the entire expected StreamStart and acquisition session binding;
every fragment/control carries a digest of that full binding. A hash is integrity,
not authentication. Fragment size is at most 1 MiB, event caps reuse the existing
1 MiB camera / 2 MiB LiDAR/pose limits. Headers are capped at 64 KiB. One
observation assembles at a time with exact offsets, fragment and whole-event
SHA-256, per-channel source identity/sequence/clock checks. End, Cancel and
gap/unavailable notices are explicit. EOF without End, timeout, malformed,
cross-binding or incomplete input fails the bound stream; an unbound wrong Open
is rejected without stopping the current owner. Camera frames require init;
camera gap invalidates init. Actual codec/keyframe/IDR validation is NOT yet here.

`StreamingSender` owns no queue and fails on its bounded write deadline rather
than stretching replay time. `StreamingIngress` is a lifecycle-tracked thread;
idle reads poll the lease and cannot keep accepting an expired owner. Raw/header
and reassembly reservations charge the SAME StreamMailbox 16 MiB input budget as
queued/active bundles. They survive cancellation until the borrowing callback
returns and buffers are dropped. Reservations are count-bounded too. Trusted
decoder adapters must reserve scratch BEFORE allocation and must not retain raw
callback bytes outside that ownership contract. Python metadata object overhead,
socket/kernel buffers, source adapter memory and model tensors additionally need
RSS/OS resource bounds; this application counter alone is not a total-RSS claim.

Worker evidence uses a separate source process, original camera init/fMP4 segments
and existing normalized point/pose increments paced at original 1x times. All
103 events (32 camera + init + 34 LiDAR + 36 pose) match exact bytes/metadata, each
data modality reaches the hash-only consumer before End, and incomplete EOF
delivers no observation. Normal peak input is 553,991 bytes. This validates raw
IPC, NOT decoder output, the full graph on this ingress, model performance,
authenticated network behavior or a standalone image. The next adapter must
feed incremental decode and causal sensors into the same graph without using
the old concatenated camera file or precomputed geometry. Product source queues,
backend routes and the browser are not switched by this increment.

Stage-2 decoder increment, 2026-09-02 (`350366b`, `f102712`):
`media_fragments.py` now owns the existing bounded ISO-BMFF timing parser.
Archive inspection delegates to it and preserves `SessionIntegrityError` at its
boundary. `perception.streaming_decoder.FragmentDecoder` imports neither the
session store nor its materializer. It receives only bounded init/fragment bytes;
source filenames, recording length and EOF are not decoder inputs.

The measured subset is H.264, one 800×600 frame per fragment, one video track,
explicit moof-relative sample size/offset and no composition offset. The first
fragment must be random-access and the decoded frame must be keyframe; DTS must
remain contiguous. One persistent codec context, one slice thread, no flush or
future packet: each call must yield exactly one frame with the same PTS/raster.
Reordered/B-frame streams, unknown layouts, discontinuities and invalid output
fail closed. Other valid camera formats require explicit qualification, not a
hidden decoder fallback. These are current profile bounds, not a new K1 protocol.

PyAV 18.0.0 is pinned to wheel SHA-256
`ae56b40b6f8b067a8ad2dac664fbfbabac7f7a55b9a7bb031eb99289252bc017`.
It is installed only into a private experiment target and mounted read-only;
standalone packaging must include it. The public
[PyAV codec documentation](https://pyav.basswood.io/docs/stable/api/codec.html)
describes persistent packet decoding and thread modes; compatibility claims here
come from the actual 18.0.0 Worker probe, not from documentation version alone.

Sequential CPU-only 128-frame 1× comparisons produced exact BGR parity against
the previous pinned OpenCV decoder. Direct validated sample extraction replaces
opening/demuxing init+fragment on every frame; NumPy loads before readiness.
Measured new-decoder p95 improves 19.927→11.156 ms, p99 20.920→18.675 ms.
This is one bounded component comparison, not a full-profile speedup claim.
The reference full-MP4 path exists only in the separate pixel-comparison process;
neither new decoder path mounts/reads that MP4.

Combined IPC/decoder probe accepts 103 raw events and yields 32 exact BGR frames.
31 frames arrive before the source End marker; the final frame drains before
the receiver handles End. First decoded callback is 3.071 s before the marker.
Input reservations peak at 5,727,751 bytes including conservative decoder scratch
and init/extradata retention; normal and truncated-input cleanup return to zero.
Native codec/DPB allocations are bounded separately by the CPU-only container;
application payload counters alone do not constrain native allocation or execution
time. Before full-graph integration, native decoding must run in a supervised CPU
child with bounded RPC and explicit decoded-bundle ownership transfer.

Point/pose are preserved raw in this probe, not causally fused into a scene;
calibration identity is explicitly non-scene/synthetic. No full graph, GPU model,
network, physical live feed, vehicle control or product cutover is qualified.
Next: preserve the existing causal sensor cut at camera release, connect decoded
bundles to the same full graph, then repeat full-profile parity/timing. The last
full-profile p95/p99 remains 124.98/136.12 ms, not this decoder's timing.

Stage-2 full binary graph increment, 2026-09-02 13:55 MSK (`221e429`):
the preceding decoder-only boundary is now connected to the existing full graph.
The recording adapter alone reads bounded current fMP4 fragments and normalized
map point/pose rows. It emits original timestamps at 1× through the existing
binary socket ABI; the consumer and decoder receive neither source paths nor
recording length. The diagnostic container still mounts source/code/weights and
a private pinned PyAV target: this is not standalone packaging or a network test.

Native decoding runs in a lifecycle-owned CPU child with a 1-GiB address-space
limit. POSIX pipe RPC reserves header scratch, accepts ≤1 MiB input and writes
exactly 1,440,000 BGR bytes into the caller's pre-reserved buffer. A 250-ms request
deadline, malformed reply or lost lease terminates the stream; no implicit retry
across predictive codec state. Process/native memory remains under container
limits, separately from the 16-MiB application payload budget.

`streaming_sensors` owns the existing causal selection rules. Its rolling and
fresh caches are each bounded by 64,000 points/64 increments; camera arrival
freezes the original-time cut before decode, never selecting future pose/points.
The legacy pilot delegates to the same selection rules. A separate CPU oracle
compares the old producer's complete points/rolling/times/pose/lineage bundles.
`admit_reserved` transfers the decoded allocation atomically to the common
mailbox; rejection retains caller ownership until buffers are discarded.
The global two-pending limit and active-input protection remain unchanged.

Worker evidence: 384/384 input events, 128/128 full graph outputs, no drops,
126 outputs before source End; first result 583 ms. BGR, masks, proposals,
tracks, threats, material and raw TGS costmap match the reference on 128/128.
Two `metric_geometry.range_m` values differ by 1.3877787807814457e-17 m
(seq13/observation3 and seq29/observation2): **strict raw parity remains false**.
No rounding/tolerance was added; the numerical cause remains unreproduced.
The geometry distance-estimator definition and all thresholds are unchanged.

Full p95/p99 161.974/191.806 ms FAIL the unchanged 125-ms target. Available
camera/sensor pairs remain 76/128; 74 scenes are fresh at receipt, with seq106/107
additionally expiring. Auto GPU memory clocks fall from 10,251 to 405–810 MHz;
this correlates with longer DDRNet/queue times, but is not a controlled transport
A/B. Peak input 8,883,895 bytes; remaining input/children/lease zero after stop.
VRAM peak 2,367 MiB is not GPU utilization or a compute-capacity percentage.
Normal 32-frame CPU oracles and a synthetic hung-decoder cleanup probe pass.

The full GPU sample preserves its exact measured code snapshot. Subsequent
review bounds finite-check scratch, closes sockets/cache after constructor or
pre-start failure, and removes CPU-harness heartbeat shutdown noise; those three
files pass a separate Worker CPU oracle, not a second GPU performance sample.
271 focused tests, Ruff/format and five-module mypy pass. Four temporarily stopped
Mission Core services are restored; Ollama/Frigate remain exited/restart=no.
Manifest: 161 artifacts, SHA-256
`e224de4e9f2c5fd419fe0fb0b9be6e01ec267628568e104795ae211ec803bce4`,
`.runtime/perception-stage2-binary-graph-worker-20260902T1335MSK/manifest.json`.
Next: explain numerical reproducibility, measure/tune remaining pipeline tails,
then complete controller/network/standalone boundaries within stage 2.
No quality, physical-live, vehicle-control or real-time qualification is granted.

Stage-2 numeric/IPC increment, 2026-09-02 14:22 MSK (`380b6ea`, `f39f1ff`):
the two preceding raw range differences are resolved, not tolerated. A bounded
CPU-only probe on Worker NumPy 1.26.4 reproduces both exact deviations using
identical quaternion values at addresses 8 mod 16. The wire quaternion starts at
byte 24 of a combined pose buffer; its norm changes by one ULP on that layout.
Offsets 0/16/32/48 restore reference norm/rotation/depth, while 8/24/40/56 reproduce
the deviations. The adapter now owns an immutable 32-byte quaternion copy and
checks 16-byte alignment. Input bytes, projection algorithm and tolerances are
unchanged. Other CPU/NumPy combinations still require numerical qualification.

Synchronous local IPC now borrows contiguous image buffers. Decoder and parent
no longer call BGR `tobytes()` for these handoffs; the sender writes framing and
the borrowed body separately instead of concatenating a full message copy.
Partial writes loop with explicit progress validation; noncontiguous buffers
fail before output. The owner retains the image until send completes. This removes
four application-level BGR copies across decoder→parent→DDRNet, not kernel copies,
network bytes, or all intermediate allocations. Framing and lease/deadline checks
are unchanged; a hung-decoder probe still fails and releases all owned resources.

Preregistered sequential A–B–B–A at authorized temporary stock clock locks:
all samples observed SM2610/memory10251 MHz, unchanged 8 CPU/8 GiB/power limit.
Every run completes 128/128 without drops. Control p95/p99 84.702/92.961 and
83.930/91.759 ms; candidate 85.113/91.219 and 82.425/90.530 ms. Both candidates
match reference BGR, masks, objects, geometry/ranges, tracks, threats, materials,
lineage/binding and raw costmap exactly on 128/128. Baselines still reproduce the
two old float deviations. Median decoder RPC-minus-decode falls 1.642–1.825 to
1.100–1.118 ms; DDRNet RPC-minus-component 2.109–2.217 to 1.558–1.613 ms.
This is a modest IPC improvement; full p95 does not show an unambiguous win.
The previous auto-clock 191.806-ms p99 cannot be compared as a copy-only baseline.

Local 125-ms timing passes at this fixed operating envelope, but only 76/128
scenes are fresh because the same 52 input sensor gaps remain. No overall or
auto-clock/network/physical-live qualification is granted. Auto clocks and four
Mission Core services are restored; Ollama/Frigate remain disabled. 279 focused
tests pass. Manifest: 305 artifacts, SHA-256
`9c72a93833a1dda51c4787dca13fb9791a1bf2e31c763b7740e1b21406703707`,
`.runtime/perception-stage2-ipc-worker-20260902T1410MSK/manifest.json`.
Next implementation boundary is the controller/envelope/network integration and
standalone package. Host clock management is not granted implicitly to that controller.

Open → validate bounded metadata/claim → warm models → Ready → start the replay
clock → observations and incremental scenes → stop/drain bounded state → close
and asynchronously seal the immutable receipt. A live source already producing
data does not queue its entire warmup history: readiness begins at a current
decodable keyframe, with the skipped interval recorded.

Replay release uses original source intervals, not a convenient fixed 12 Hz and
not the processing speed. A slow consumer must not stretch source time to make
inference look real-time. Overload may drop bounded work to preserve liveness,
but the dropped observations remain in the ledger and fail the strict initial
performance target. Initial baseline is every camera frame, stride 1. Choosing
multirate later changes the explicit profile contract; a retained mask is not a
new inference and cannot receive a fresh capture timestamp.

UTC identifies the run; monotonic clocks measure local durations. Cross-host
age requires a clock mapping and its uncertainty. No direct subtraction of
unrelated host monotonic clocks. The common source timeline is separate from
worker wall-clock time. Causality is bounded by the released-observation
watermark at decision time, not by whatever later samples exist in the file.
Each layer retains its own channel/sequence/time; camera anchors and pose/cloud
association times remain separately visible. Future file look-ahead and offline
nearest-pose lookup across unreleased data are forbidden.

Lease expiry, cancellation and epoch changes fence the previous producer. No
new profile may take the GPU until old processes and work have stopped. A
reconnect creates a new epoch, discards stale backlog, resets temporal/rolling
state and reacquires codec initialization/keyframe. It never resends a stale
motor action; this profile has no motor actions at all.

## Outputs and interpretation

Every scene accounts for segmentation, objects, geometry, motion, costmap and
policy exactly once, with current/held/stale/unavailable state and current-run
payload identity. Existing source/object/obstacle/temporal/map/threat contracts
remain the domain vocabulary. `LayerEvidence` adds freshness/completeness checks,
not a second semantic ontology. In stage 2, layer manifests also carry their full
input reference sets; a single anchor does not replace camera/cloud/pose lineage.

Stage-1 implementation update, 2026-09-02 (`097e450`):
`missioncore.perception-scene-freshness/v1` in `realtime_scene.py` extends
`LayerEvidence` with oldest-required-input time and checks the six-layer dependency
graph. A derived result cannot refresh its input age. Decimal int64 JSON strings
preserve timestamps/sequence exactly; transport-neutral Python objects use integers.
The pilot validates payload hashes and assesses freshness again at receipt, and
must reassess at later use. Missing/stale output remains inspectable but cannot
retain permissive advisory policy. This envelope is NOT a replacement for
StreamStart identities, complete input lineage or controller lease fencing.

The stage-1 pilot propagated the oldest last-seen permissive TGS cell into the
costmap/policy age and suppressed the entire scene on expiry. Stage-2 increment
`1f8101e` adds `missioncore.costmap-cell-freshness/v1`: bounded (8192 cells),
index-aligned original support timestamps, decimal int64 strings/null, included
in the costmap payload digest together with the guard mode. Unobserved support
is never assigned a timestamp. The pilot now defaults to `per-cell`; the old
`whole-scene` mode remains an explicit comparison control, not a product cutover.

Both publication and receipt remove permission from expired ground cells and
mark them rejected/NO_GO. Occupied cells retain prohibition. The remaining
permissions propagate their original support age AND the mandatory segmentation,
geometry and motion dependencies. A missing/stale mandatory layer still blocks
the whole policy. A derived consumer view has new costmap/policy hashes while
preserving source identity/timestamps; the original published bytes stay immutable.
Reassessment cannot restore a suppressed action or move the observer clock back.
The enclosing StreamStart must bind the grid/profile/epoch/clock; this descriptor
alone does not establish those identities or authenticate a remote producer.

In the measured 128-frame window, raw model/geometry/motion/TGS/material outputs
remain identical to pinned PyTorch. Effective costmap states and permissions
change deliberately on expiry. A fresh six-layer envelope can still contain
rejected/unknown cells; it is not a claim of full free-space coverage, semantic
correctness, vehicle clearance or physical safety. Recheck again at actual use.

`bcacb02` promotes the proven mailbox and serial GPU stage to common perception
modules, with the pilot importing compatibility aliases rather than owning
another scheduler. Pending ingress + completed GPU outputs share two slots;
active inputs share 16 MiB and are released by their owning stage. Cancellation
discards pending work, a timed-out callback retains ownership, and CPU-owned
input is not freed by GPU shutdown. Diagnostic drop history is capped at 256
entries with exact per-reason totals; it is not the durable terminal ledger.
These primitives are NOT the controller supervisor, a cross-process GPU lease,
StreamStart fencing or the binary data plane. Those stage-2 boundaries remain open.

Stage-2 lifecycle increment, 2026-09-02 (`6acf468`, `ac69e3b`):
`StreamingLifecycle` now supplies the subprocess-backed profile supervisor using
the existing `GraphState` and full `StreamStart` identity. `WorkerLease` uses a
stable POSIX file lock on one controller-selected, private directory per Worker,
plus an atomic/fsynced ownership record. All managed profile containers must use
that SAME directory/volume. Neither the directory nor child commands come from
an incoming source/job. This cooperative local fence supplements, not replaces,
the backend claim and the trusted inventory of unmanaged GPU clients.

Open acquires ownership before process spawn/warmup; Ready enables admission.
Renewal cannot resurrect expired ownership. Admission, each GPU/CPU lane and
result publication/receipt validate run/source/worker/epoch/generation and pinned
image/profile/config/calibration/clock identities. A mismatched client is rejected
without cancelling the current owner. Each compute lane remains single-owner.
The watchdog fences an expired owner even without a new frame and terminates
only its dedicated child process groups. Cleanup keeps ownership while callbacks,
tracked threads, pending/active payloads or child groups remain. Clean retirement
persists `released` before unlocking; a higher generation and new epoch may then
activate. A controller crash leaves `active` even when the kernel unlocks, so
the next owner stays quarantined. No force/unverified recovery API exists.

Cross-container contention/clean succession/crash quarantine and full-profile
lease expiry were exercised on Worker 006. The final timed expiry trace had no
receipt at/after the lease deadline, stop requested after 45.74 ms and retirement
after 4109.82 ms. This is bounded evidence, not a hard-real-time scheduler guarantee.
The holder has a 2 s lease and receives local pilot heartbeats every 250 ms;
those settings and the 50 ms watchdog poll are NOT network/vehicle safety limits.

Current integration remains explicitly diagnostic: `pilot_lifecycle.py` acts as
a local controller, legacy GPU services are quiesced by the authorized launcher,
the named volume is an isolated test volume, and the image still has code/model
mounts. Mounted-code hashes remain in the experiment manifest; an image digest
alone cannot seal those overrides. Backend claims/production-wide canonical root,
trusted post-crash resource-release recovery, continuous GPU inventory/envelope
enforcement, network authentication/heartbeat and binary live ingress are NOT
installed or qualified by this increment. Standalone packaging must remove the
developer mounts and retain the same lifecycle boundaries.

`worker_operating_envelope.py` checks a trusted post-warmup snapshot against
preregistered hardware/driver/resource/clock conditions and StreamStart identities,
ownership and client inventory. Unknown or expired facts fail readiness. The
candidate records the previously measured 4090 fixed-clock envelope; a compatible
auto-clock experiment cannot inherit its latency result. The evaluator performs
no I/O, host clock changes, lease acquisition or qualification; collection and
continuous enforcement belong to the common controller/runtime in stage 2.

DDRNet currently crops the center 600×600 to 512×512; outside that ROI is undefined,
not hard_surface/free. RF-DETR accepts the full 800×600 native raster with its
pinned valid-FOV handling. Their coordinates must be related explicitly.

RF-DETR's existing filter emits person/cat/dog, but minimum box area 64 px,
maximum box fraction 0.5, FOV fraction 0.5 and required valid center create known
small-animal/close-large-object edge cases. They are retained and documented,
not silently retuned or presented as proven field quality.

Detected-object range is the median camera-Z of owned current support points;
geometry-only range is nearest Euclidean distance from the sensor. Neither is
automatically clearance from the physical vehicle body. Missing/ambiguous
support produces unavailable range, not zero/infinity. Generic static obstacles
survive absence of a detector class name.

TGS keeps ground/occupied/rejected/unobserved separate. Ground support alone is
not a traversability or actuation decision. Policy can produce an advisory
allowed-candidate/high-cost/blocked/unknown outcome with explanation. No planner,
physical motor command, autonomous driving acceptance or mission configurator
implementation is introduced.

## Package/dependency decision

The candidate manifest is
`config/perception/k1-perception-ddrnet39-rfdetr-tgs-prototype-v1.json`.
It is not installed into the active portable registry and has no invented image
digest. Model/checkpoint/config identities are pinned; source session IDs and old
derived output paths in reference experiments are not inherited requirements.

Select process/environment isolation inside one future image: Python 3.12
supervisor and geometry, RF-DETR TensorRT 11 runtime, existing Python 3.9 / Torch
1.13.1 cu117 / super-gradients 3.2.0 environment, and CPU C++ TGS. Exchange bounded
shared-memory/IPC payload references; one supervisor serializes GPU work. This
avoids a forced dependency upgrade or checkpoint conversion in stage 1. Image
assembly of the final standalone package and the full schedule pilot remain
explicit implementation evidence. A temporary common-base image passed separate
DDRNet, TensorRT, Python 3.12 geometry and C++ TGS execution probes; DDRNet masks
matched on 64 frames. This proves bounded ABI/execution compatibility in that
image, not jointly resident models, the supervisor/IPC or a complete profile.
The probes still mount pinned assets explicitly; they are not standalone proof.

The temporary Triton base lacks Python grpc/protobuf, cv2 and TensorRT bindings;
native trtexec/server are present. The actual supervisor/transport environment
must explicitly include and validate its dependencies. Numeric-library thread
limits are pinned to one after the bounded synthetic local-surface comparison
(mean 198.0 ms default versus 63.2 ms with limits); algorithm thresholds were not
changed. Real-cloud timing, output parity and whole-graph scheduling remain gates.

The initial DDRNet image-only probe failed before inference: the installed image
does not contain the checkpoint at its logical asset path. The bounded baseline
therefore uses an explicit read-only pinned checkpoint and runner mount. This is
valid component measurement, but fails the target's standalone packaging claim.
Super-gradients also needs a writable log directory; a bounded temporary log
mount was required. The new image must declare scratch areas explicitly without
requiring a writable host home, code checkout or implicit weight cache.

## Preregistered engineering acceptance

Owner clarification, 2026-09-02 MSK: bounded laboratory overload is an admissible
experimental outcome. Keep useful profiles that fail the current 4090 target;
do not block reusable runtime/packaging work solely on that performance result.
Functional execution, experimental availability, quality and real-time
qualification are separate dimensions. Qualification belongs to a measured
profile/config/hardware/source/transport combination, not permanently to a
model name. More powerful Worker/onboard placement is a future retest target,
not a claim that existing latency or internal IPC disappears automatically.
The original budgets and failed measurements below remain unchanged. No slowed
source clock, hidden drops, stale-as-current evidence or actuation is authorized.

The manifest freezes an initial engineering candidate: p95/p99 full output age
≤125 ms, required-layer age ≤250 ms, release lag ≤25 ms, first incremental result
≤1 s after source admission, warmup ≤120 s, stop ≤5 s, VRAM ≤22000 MiB, RSS ≤8192
MiB. First/last-window backlog growth ≤25 ms and no capacity drops, failures,
expired selected frames or unaccounted observations. These are prototype targets,
not physical braking/safety limits or claims of achieved performance.

`ReplayMeasurements`/`realtime_failures` reject full-source preload, EOF-only
results, slowed replay, incomplete accounting, hidden missing layers and budget
violations. Declared source gaps must be verified from the input ledger; they
produce explicit degraded scenes and are not counted as fully fresh perception.
At least one fully evaluable scene is required. A component FPS figure cannot
populate this whole-path receipt. A quality comparison or successful export is
not a substitute for the real-time gate.

Every observation receives a terminal ledger outcome. Track ingress/release,
selection, decode, inference, completion, emission, drop/expiry/failure and source
gaps separately. Store startup separately from steady state; record payload bytes,
queue high-water marks, clock error, RSS/VRAM, image/weights/config/source/equipment
identities. Export and UI encoding cannot block inference through an unlimited
queue. A receiver-side render/receipt timestamp, not just Worker completion, ends
the end-to-end interval.

## Evidence and remaining stage-1 work

### Operator telemetry is not runtime authority (2026-09-02)

Reuse System/Worker and the existing Telegraf → MQTT → Timescale route for
operator observability. A profile may export one bounded 8 KiB current snapshot
at 1 Hz to the host collector, independently from controller heartbeat/model
work. Missing or stale observations are unavailable, never fabricated idle,
loaded-model readiness, per-stage timing or real-time qualification. Agent
restart/reconnection and database retention must not block inference. The faster
Worker-local readiness/inventory channel remains separate; MQTT/Timescale/UI
cannot renew a lease or authorize GPU/vehicle activity. This observability
integration does not constitute the external camera/points binary data plane.

See `experiments/perception/PERCEPTION_STREAM_STAGE1_2026-09-01.md` for measured
component results, bounded validation, Worker maintenance and remaining gates.
Neither this ADR nor passing synthetic contract tests marks the new runtime ready.