NODEDC_MISSION_CORE/docs/adr/0049-stream-first-perception-profiles.md

# ADR 0049 — Stream-first full perception profiles

Date: 2026-09-01; updated 2026-09-02 13:55 MSK. Status: stage-1 engineering
baseline complete; stage-2 binary full-graph prototype, not a product runtime cutover.
Execution order remains the four stages in
[the ExecPlan](../OBSERVATORY_REALTIME_PROFILES_EXECPLAN.md).

## Decision and owner intent

Observatory LABs test the real-time behavior of a complete perception profile,
not just the quality of an offline export. A recording replaces the physical
source, preserving original release timing at 1×. It must not enable a different
batch implementation. The first profile contains DDRNet-39 GOOSE, RF-DETR native,
online LiDAR geometry/ranges, temporal/motion, CPU TRAVEL TGS/costmap and advisory
policy. One dedicated RTX 4090 owns one active full profile.

The first rural prototype accepts existing coarse `hard_surface`, including
asphalt, sidewalk, bikeway and cobble. Existing `static_obstacle`/`static.unknown`
is sufficient. No new bollard classifier, extra segmenter, road/sidewalk taxonomy
or annotation campaign precedes this prototype. Other material rules remain an
explicit effective mission configuration; rural context alone is not permission
for all vegetation or soil. Geometry/unknown/freshness overrides material allowance.

EoMT is preserved as a separate future profile. It is not loaded with DDRNet.
RF-DETR and DDRNet are two required components of the same first profile: one
supervisor and serialized GPU scheduling, not competing profile jobs.

## CURRENT → TARGET / ownership

| Boundary | Current evidence | Decision / stage |
| --- | --- | --- |
| Portable definitions | `portable_run_definitions.py` owns pins, resources and admission | KEEP identity/sealing; EXTEND stream capabilities and qualification in stage 3 |
| Source requirements | Recorded-only requirements include seekability, one media epoch and init digest | REPLACE as live admission criteria with channel/format/calibration/clock requirements; keep init digest as per-stream integrity |
| Job queue | `recorded_jobs.py` has durable claims but global single-active-job scope | KEEP durable queue; EXTEND worker-scoped fencing, not unbounded GPU concurrency |
| Worker agent | Blocking `execute(job)` and final publication | EXTEND incremental lifecycle and cancellation; no per-LAB agent |
| Runtime | `portable_worker_runtime.py:735` materializes before execution | NEW stream runtime in stage 2; never call this materializer in the real-time path |
| Source transport | Whole camera archive download/hash/extraction before use | REPLACE startup barrier with bounded metadata, chunks and incremental integrity |
| Installed LAB V1 | prepare → complete EoMT → complete DDRNet → assemble | Legacy evidence only; not the new profile and not real-time qualified |
| Camera/source replay | `recorded_source.py` has useful pacing and bounded prefetch, but source-specific timeline and paths | REUSE mechanisms; generic unknown-duration source adapter, no route-sized prefetch |
| Existing live ingress | `LivePerceptionIngress`, `LiveIngressEvent.wire_bytes`, `LiveSensorSynchronizer` already provide raw-first modality queues, hashes, epochs and bounded binding | REUSE, not a second acquisition system; extend worker/profile fencing and aggregate byte limits; map existing envelopes to the selected transport |
| Local surface | `RecordedGeometryStore` reads prepared NPZ | REUSE `K1LocalSurfaceShadowEstimator.process` and `lidar_local_surface_geometry`; feed current points/pose |
| Geometry/ranges | Current point association, exclusive ownership and geometry-only clusters exist | REUSE algorithms; remove recorded-store dependency and expose estimator/frame identity |
| Temporal/motion | `BoundedSpatialTemporalProvider`, `ClassIndependentMotionEstimator` | REUSE with current-run inputs and reset on epoch/lease change |
| TGS | Core algorithm reusable; old runners read a full schedule, some require exactly 4489 frames | REUSE TGS parameters/core; REPLACE schedule/file CLI with incremental input |
| Fusion | M49 TGS can be a neighboring shadow with `tgs_modifies_reference_graph_state=false` | TGS and semantics must influence the current advisory scene, not only viewer composition |
| Publication | Verified immutable result/publication/recovery exist | KEEP asynchronous final sealing; live results do not await it |
| Frontend | Setup/job/result surfaces exist, historical M49 viewer links other results | EXTEND common renderer capabilities with current-run layer references and freshness in stage 3 |
| Recording archive | Source identities and raw evidence must be retained | KEEP; create metadata/index/integrity during acquisition or incrementally, not via preflight full scan |

No changes to the active registry, old queue, source endpoint, UI or canonical
local server are made by this ADR or the stage-1 validators.

## Versioned handshake and observations

`missioncore.perception-stream-start/v1` is represented by `StreamStart` in
`src/k1link/perception/realtime_contract.py`. It contains run/source/worker/epoch,
lease generation, profile/image/effective-config/calibration identities, clock
domain, input mode and channel declarations. The first profile requires camera,
point-cloud and pose. Temporary absence of a sample is an explicit modality
outcome, not removal of a required capability.

It deliberately has no total source bytes, frame count, total duration, whole
archive, command, environment, complete member inventory or required EOF. The
initial metadata budget is 64 KiB. Large constant model assets and FOV masks
belong inside the image; calibration references resolve to bounded metadata.

The stage-2 wire mapping must carry:

- Run/epoch/lease binding on every message; source/channel sequence, capture
  timestamp and clock domain; independently verifiable payload length/hash.
- Camera codec initialization and then incremental encoded access units, or
  bounded fragments thereof. Native decoded representation is BGR8 800×600 KB4.
  Do not send an entire fMP4 epoch as one observation or turn all frames into PNGs.
- Current vendor-registered map point increments with point IDs, coordinate
  frame, units, scalar layout and retained raw attributes; map→sensor pose and
  calibration identity. The adapter owns vendor decoding; the profile is not a
  new K1 protocol implementation and cannot command the scanner.
- Explicit unavailable/gap/end/cancel messages. End is a termination event,
  never a prerequisite for first inference. Final recording inventory/hash can
  be sealed after consumption; per-chunk integrity is verified before use.
- Integer nanoseconds use protobuf integer fields; any JSON projection uses
  decimal strings for 64-bit times rather than lossy JavaScript numbers.

Maximum payload fragment: 1 MiB; total application inflight bytes: 16 MiB;
pending decoded camera frames: 2. Fragment reassembly is included in that budget,
with a deadline and incomplete-chunk accounting. Decoded image/tensor, cloud,
costmap and archive sinks need their own measured bounds inside the RSS budget.
Transport flow control alone is not a memory bound or a freshness policy.

gRPC bidirectional streaming is the first transport candidate, not an already
measured implementation. HTTP control-plane operations remain. The browser does
not connect directly to model RPCs. One authenticated data-plane endpoint feeds
the supervised profile; model processes have no arbitrary external network.
Protocol/codec selection must be measured on actual payload sizes in stage 2;
changing an archive POST to gRPC without changing execution is not acceptance.
Existing `missioncore.live-perception-wire/v1` and result wire v2 are migration
inputs: reuse their semantics and raw-first producer hooks. Their modality count
caps alone permit more than the proposed 16 MiB aggregate inflight budget; an
explicit byte bound is required. No duplicate new scanner feed is introduced.

## Time and lifecycle

Stage-2 raw IPC increment, 2026-09-02 (`da60fef`, `a937400`, `79ce55d`):
`streaming_wire.py` maps existing `LiveIngressEvent` fields into
`missioncore.live-perception-wire/v2`. The v1 producer hooks, class and legacy
serialization remain unchanged. A bounded length-prefixed JSON header carries
decimal uint64 projections; payloads remain binary. This is a controller-supplied
IPC socket candidate, not a selection/replacement of the intended gRPC network
transport. No listener, arbitrary endpoint, scanner feed or command path is added.

Open compares the entire expected StreamStart and acquisition session binding;
every fragment/control carries a digest of that full binding. A hash is integrity,
not authentication. Fragment size is at most 1 MiB, event caps reuse the existing
1 MiB camera / 2 MiB LiDAR/pose limits. Headers are capped at 64 KiB. One
observation assembles at a time with exact offsets, fragment and whole-event
SHA-256, per-channel source identity/sequence/clock checks. End, Cancel and
gap/unavailable notices are explicit. EOF without End, timeout, malformed,
cross-binding or incomplete input fails the bound stream; an unbound wrong Open
is rejected without stopping the current owner. Camera frames require init;
camera gap invalidates init. Actual codec/keyframe/IDR validation is NOT yet here.

`StreamingSender` owns no queue and fails on its bounded write deadline rather
than stretching replay time. `StreamingIngress` is a lifecycle-tracked thread;
idle reads poll the lease and cannot keep accepting an expired owner. Raw/header
and reassembly reservations charge the SAME StreamMailbox 16 MiB input budget as
queued/active bundles. They survive cancellation until the borrowing callback
returns and buffers are dropped. Reservations are count-bounded too. Trusted
decoder adapters must reserve scratch BEFORE allocation and must not retain raw
callback bytes outside that ownership contract. Python metadata object overhead,
socket/kernel buffers, source adapter memory and model tensors additionally need
RSS/OS resource bounds; this application counter alone is not a total-RSS claim.

Worker evidence uses a separate source process, original camera init/fMP4 segments
and existing normalized point/pose increments paced at original 1x times. All
103 events (32 camera + init + 34 LiDAR + 36 pose) match exact bytes/metadata, each
data modality reaches the hash-only consumer before End, and incomplete EOF
delivers no observation. Normal peak input is 553,991 bytes. This validates raw
IPC, NOT decoder output, the full graph on this ingress, model performance,
authenticated network behavior or a standalone image. The next adapter must
feed incremental decode and causal sensors into the same graph without using
the old concatenated camera file or precomputed geometry. Product source queues,
backend routes and the browser are not switched by this increment.

Stage-2 decoder increment, 2026-09-02 (`350366b`, `f102712`):
`media_fragments.py` now owns the existing bounded ISO-BMFF timing parser.
Archive inspection delegates to it and preserves `SessionIntegrityError` at its
boundary. `perception.streaming_decoder.FragmentDecoder` imports neither the
session store nor its materializer. It receives only bounded init/fragment bytes;
source filenames, recording length and EOF are not decoder inputs.

The measured subset is H.264, one 800×600 frame per fragment, one video track,
explicit moof-relative sample size/offset and no composition offset. The first
fragment must be random-access and the decoded frame must be keyframe; DTS must
remain contiguous. One persistent codec context, one slice thread, no flush or
future packet: each call must yield exactly one frame with the same PTS/raster.
Reordered/B-frame streams, unknown layouts, discontinuities and invalid output
fail closed. Other valid camera formats require explicit qualification, not a
hidden decoder fallback. These are current profile bounds, not a new K1 protocol.

PyAV 18.0.0 is pinned to wheel SHA-256
`ae56b40b6f8b067a8ad2dac664fbfbabac7f7a55b9a7bb031eb99289252bc017`.
It is installed only into a private experiment target and mounted read-only;
standalone packaging must include it. The public
[PyAV codec documentation](https://pyav.basswood.io/docs/stable/api/codec.html)
describes persistent packet decoding and thread modes; compatibility claims here
come from the actual 18.0.0 Worker probe, not from documentation version alone.

Sequential CPU-only 128-frame 1× comparisons produced exact BGR parity against
the previous pinned OpenCV decoder. Direct validated sample extraction replaces
opening/demuxing init+fragment on every frame; NumPy loads before readiness.
Measured new-decoder p95 improves 19.927→11.156 ms, p99 20.920→18.675 ms.
This is one bounded component comparison, not a full-profile speedup claim.
The reference full-MP4 path exists only in the separate pixel-comparison process;
neither new decoder path mounts/reads that MP4.

Combined IPC/decoder probe accepts 103 raw events and yields 32 exact BGR frames.
31 frames arrive before the source End marker; the final frame drains before
the receiver handles End. First decoded callback is 3.071 s before the marker.
Input reservations peak at 5,727,751 bytes including conservative decoder scratch
and init/extradata retention; normal and truncated-input cleanup return to zero.
Native codec/DPB allocations are bounded separately by the CPU-only container;
application payload counters alone do not constrain native allocation or execution
time. Before full-graph integration, native decoding must run in a supervised CPU
child with bounded RPC and explicit decoded-bundle ownership transfer.

Point/pose are preserved raw in this probe, not causally fused into a scene;
calibration identity is explicitly non-scene/synthetic. No full graph, GPU model,
network, physical live feed, vehicle control or product cutover is qualified.
Next: preserve the existing causal sensor cut at camera release, connect decoded
bundles to the same full graph, then repeat full-profile parity/timing. The last
full-profile p95/p99 remains 124.98/136.12 ms, not this decoder's timing.

Stage-2 full binary graph increment, 2026-09-02 13:55 MSK (`221e429`):
the preceding decoder-only boundary is now connected to the existing full graph.
The recording adapter alone reads bounded current fMP4 fragments and normalized
map point/pose rows. It emits original timestamps at 1× through the existing
binary socket ABI; the consumer and decoder receive neither source paths nor
recording length. The diagnostic container still mounts source/code/weights and
a private pinned PyAV target: this is not standalone packaging or a network test.

Native decoding runs in a lifecycle-owned CPU child with a 1-GiB address-space
limit. POSIX pipe RPC reserves header scratch, accepts ≤1 MiB input and writes
exactly 1,440,000 BGR bytes into the caller's pre-reserved buffer. A 250-ms request
deadline, malformed reply or lost lease terminates the stream; no implicit retry
across predictive codec state. Process/native memory remains under container
limits, separately from the 16-MiB application payload budget.

`streaming_sensors` owns the existing causal selection rules. Its rolling and
fresh caches are each bounded by 64,000 points/64 increments; camera arrival
freezes the original-time cut before decode, never selecting future pose/points.
The legacy pilot delegates to the same selection rules. A separate CPU oracle
compares the old producer's complete points/rolling/times/pose/lineage bundles.
`admit_reserved` transfers the decoded allocation atomically to the common
mailbox; rejection retains caller ownership until buffers are discarded.
The global two-pending limit and active-input protection remain unchanged.

Worker evidence: 384/384 input events, 128/128 full graph outputs, no drops,
126 outputs before source End; first result 583 ms. BGR, masks, proposals,
tracks, threats, material and raw TGS costmap match the reference on 128/128.
Two `metric_geometry.range_m` values differ by 1.3877787807814457e-17 m
(seq13/observation3 and seq29/observation2): **strict raw parity remains false**.
No rounding/tolerance was added; the numerical cause remains unreproduced.
The geometry distance-estimator definition and all thresholds are unchanged.

Full p95/p99 161.974/191.806 ms FAIL the unchanged 125-ms target. Available
camera/sensor pairs remain 76/128; 74 scenes are fresh at receipt, with seq106/107
additionally expiring. Auto GPU memory clocks fall from 10,251 to 405–810 MHz;
this correlates with longer DDRNet/queue times, but is not a controlled transport
A/B. Peak input 8,883,895 bytes; remaining input/children/lease zero after stop.
VRAM peak 2,367 MiB is not GPU utilization or a compute-capacity percentage.
Normal 32-frame CPU oracles and a synthetic hung-decoder cleanup probe pass.

The full GPU sample preserves its exact measured code snapshot. Subsequent
review bounds finite-check scratch, closes sockets/cache after constructor or
pre-start failure, and removes CPU-harness heartbeat shutdown noise; those three
files pass a separate Worker CPU oracle, not a second GPU performance sample.
271 focused tests, Ruff/format and five-module mypy pass. Four temporarily stopped
Mission Core services are restored; Ollama/Frigate remain exited/restart=no.
Manifest: 161 artifacts, SHA-256
`e224de4e9f2c5fd419fe0fb0b9be6e01ec267628568e104795ae211ec803bce4`,
`.runtime/perception-stage2-binary-graph-worker-20260902T1335MSK/manifest.json`.
Next: explain numerical reproducibility, measure/tune remaining pipeline tails,
then complete controller/network/standalone boundaries within stage 2.
No quality, physical-live, vehicle-control or real-time qualification is granted.

Open → validate bounded metadata/claim → warm models → Ready → start the replay
clock → observations and incremental scenes → stop/drain bounded state → close
and asynchronously seal the immutable receipt. A live source already producing
data does not queue its entire warmup history: readiness begins at a current
decodable keyframe, with the skipped interval recorded.

Replay release uses original source intervals, not a convenient fixed 12 Hz and
not the processing speed. A slow consumer must not stretch source time to make
inference look real-time. Overload may drop bounded work to preserve liveness,
but the dropped observations remain in the ledger and fail the strict initial
performance target. Initial baseline is every camera frame, stride 1. Choosing
multirate later changes the explicit profile contract; a retained mask is not a
new inference and cannot receive a fresh capture timestamp.

UTC identifies the run; monotonic clocks measure local durations. Cross-host
age requires a clock mapping and its uncertainty. No direct subtraction of
unrelated host monotonic clocks. The common source timeline is separate from
worker wall-clock time. Causality is bounded by the released-observation
watermark at decision time, not by whatever later samples exist in the file.
Each layer retains its own channel/sequence/time; camera anchors and pose/cloud
association times remain separately visible. Future file look-ahead and offline
nearest-pose lookup across unreleased data are forbidden.

Lease expiry, cancellation and epoch changes fence the previous producer. No
new profile may take the GPU until old processes and work have stopped. A
reconnect creates a new epoch, discards stale backlog, resets temporal/rolling
state and reacquires codec initialization/keyframe. It never resends a stale
motor action; this profile has no motor actions at all.

## Outputs and interpretation

Every scene accounts for segmentation, objects, geometry, motion, costmap and
policy exactly once, with current/held/stale/unavailable state and current-run
payload identity. Existing source/object/obstacle/temporal/map/threat contracts
remain the domain vocabulary. `LayerEvidence` adds freshness/completeness checks,
not a second semantic ontology. In stage 2, layer manifests also carry their full
input reference sets; a single anchor does not replace camera/cloud/pose lineage.

Stage-1 implementation update, 2026-09-02 (`097e450`):
`missioncore.perception-scene-freshness/v1` in `realtime_scene.py` extends
`LayerEvidence` with oldest-required-input time and checks the six-layer dependency
graph. A derived result cannot refresh its input age. Decimal int64 JSON strings
preserve timestamps/sequence exactly; transport-neutral Python objects use integers.
The pilot validates payload hashes and assesses freshness again at receipt, and
must reassess at later use. Missing/stale output remains inspectable but cannot
retain permissive advisory policy. This envelope is NOT a replacement for
StreamStart identities, complete input lineage or controller lease fencing.

The stage-1 pilot propagated the oldest last-seen permissive TGS cell into the
costmap/policy age and suppressed the entire scene on expiry. Stage-2 increment
`1f8101e` adds `missioncore.costmap-cell-freshness/v1`: bounded (8192 cells),
index-aligned original support timestamps, decimal int64 strings/null, included
in the costmap payload digest together with the guard mode. Unobserved support
is never assigned a timestamp. The pilot now defaults to `per-cell`; the old
`whole-scene` mode remains an explicit comparison control, not a product cutover.

Both publication and receipt remove permission from expired ground cells and
mark them rejected/NO_GO. Occupied cells retain prohibition. The remaining
permissions propagate their original support age AND the mandatory segmentation,
geometry and motion dependencies. A missing/stale mandatory layer still blocks
the whole policy. A derived consumer view has new costmap/policy hashes while
preserving source identity/timestamps; the original published bytes stay immutable.
Reassessment cannot restore a suppressed action or move the observer clock back.
The enclosing StreamStart must bind the grid/profile/epoch/clock; this descriptor
alone does not establish those identities or authenticate a remote producer.

In the measured 128-frame window, raw model/geometry/motion/TGS/material outputs
remain identical to pinned PyTorch. Effective costmap states and permissions
change deliberately on expiry. A fresh six-layer envelope can still contain
rejected/unknown cells; it is not a claim of full free-space coverage, semantic
correctness, vehicle clearance or physical safety. Recheck again at actual use.

`bcacb02` promotes the proven mailbox and serial GPU stage to common perception
modules, with the pilot importing compatibility aliases rather than owning
another scheduler. Pending ingress + completed GPU outputs share two slots;
active inputs share 16 MiB and are released by their owning stage. Cancellation
discards pending work, a timed-out callback retains ownership, and CPU-owned
input is not freed by GPU shutdown. Diagnostic drop history is capped at 256
entries with exact per-reason totals; it is not the durable terminal ledger.
These primitives are NOT the controller supervisor, a cross-process GPU lease,
StreamStart fencing or the binary data plane. Those stage-2 boundaries remain open.

Stage-2 lifecycle increment, 2026-09-02 (`6acf468`, `ac69e3b`):
`StreamingLifecycle` now supplies the subprocess-backed profile supervisor using
the existing `GraphState` and full `StreamStart` identity. `WorkerLease` uses a
stable POSIX file lock on one controller-selected, private directory per Worker,
plus an atomic/fsynced ownership record. All managed profile containers must use
that SAME directory/volume. Neither the directory nor child commands come from
an incoming source/job. This cooperative local fence supplements, not replaces,
the backend claim and the trusted inventory of unmanaged GPU clients.

Open acquires ownership before process spawn/warmup; Ready enables admission.
Renewal cannot resurrect expired ownership. Admission, each GPU/CPU lane and
result publication/receipt validate run/source/worker/epoch/generation and pinned
image/profile/config/calibration/clock identities. A mismatched client is rejected
without cancelling the current owner. Each compute lane remains single-owner.
The watchdog fences an expired owner even without a new frame and terminates
only its dedicated child process groups. Cleanup keeps ownership while callbacks,
tracked threads, pending/active payloads or child groups remain. Clean retirement
persists `released` before unlocking; a higher generation and new epoch may then
activate. A controller crash leaves `active` even when the kernel unlocks, so
the next owner stays quarantined. No force/unverified recovery API exists.

Cross-container contention/clean succession/crash quarantine and full-profile
lease expiry were exercised on Worker 006. The final timed expiry trace had no
receipt at/after the lease deadline, stop requested after 45.74 ms and retirement
after 4109.82 ms. This is bounded evidence, not a hard-real-time scheduler guarantee.
The holder has a 2 s lease and receives local pilot heartbeats every 250 ms;
those settings and the 50 ms watchdog poll are NOT network/vehicle safety limits.

Current integration remains explicitly diagnostic: `pilot_lifecycle.py` acts as
a local controller, legacy GPU services are quiesced by the authorized launcher,
the named volume is an isolated test volume, and the image still has code/model
mounts. Mounted-code hashes remain in the experiment manifest; an image digest
alone cannot seal those overrides. Backend claims/production-wide canonical root,
trusted post-crash resource-release recovery, continuous GPU inventory/envelope
enforcement, network authentication/heartbeat and binary live ingress are NOT
installed or qualified by this increment. Standalone packaging must remove the
developer mounts and retain the same lifecycle boundaries.

`worker_operating_envelope.py` checks a trusted post-warmup snapshot against
preregistered hardware/driver/resource/clock conditions and StreamStart identities,
ownership and client inventory. Unknown or expired facts fail readiness. The
candidate records the previously measured 4090 fixed-clock envelope; a compatible
auto-clock experiment cannot inherit its latency result. The evaluator performs
no I/O, host clock changes, lease acquisition or qualification; collection and
continuous enforcement belong to the common controller/runtime in stage 2.

DDRNet currently crops the center 600×600 to 512×512; outside that ROI is undefined,
not hard_surface/free. RF-DETR accepts the full 800×600 native raster with its
pinned valid-FOV handling. Their coordinates must be related explicitly.

RF-DETR's existing filter emits person/cat/dog, but minimum box area 64 px,
maximum box fraction 0.5, FOV fraction 0.5 and required valid center create known
small-animal/close-large-object edge cases. They are retained and documented,
not silently retuned or presented as proven field quality.

Detected-object range is the median camera-Z of owned current support points;
geometry-only range is nearest Euclidean distance from the sensor. Neither is
automatically clearance from the physical vehicle body. Missing/ambiguous
support produces unavailable range, not zero/infinity. Generic static obstacles
survive absence of a detector class name.

TGS keeps ground/occupied/rejected/unobserved separate. Ground support alone is
not a traversability or actuation decision. Policy can produce an advisory
allowed-candidate/high-cost/blocked/unknown outcome with explanation. No planner,
physical motor command, autonomous driving acceptance or mission configurator
implementation is introduced.

## Package/dependency decision

The candidate manifest is
`config/perception/k1-perception-ddrnet39-rfdetr-tgs-prototype-v1.json`.
It is not installed into the active portable registry and has no invented image
digest. Model/checkpoint/config identities are pinned; source session IDs and old
derived output paths in reference experiments are not inherited requirements.

Select process/environment isolation inside one future image: Python 3.12
supervisor and geometry, RF-DETR TensorRT 11 runtime, existing Python 3.9 / Torch
1.13.1 cu117 / super-gradients 3.2.0 environment, and CPU C++ TGS. Exchange bounded
shared-memory/IPC payload references; one supervisor serializes GPU work. This
avoids a forced dependency upgrade or checkpoint conversion in stage 1. Image
assembly of the final standalone package and the full schedule pilot remain
explicit implementation evidence. A temporary common-base image passed separate
DDRNet, TensorRT, Python 3.12 geometry and C++ TGS execution probes; DDRNet masks
matched on 64 frames. This proves bounded ABI/execution compatibility in that
image, not jointly resident models, the supervisor/IPC or a complete profile.
The probes still mount pinned assets explicitly; they are not standalone proof.

The temporary Triton base lacks Python grpc/protobuf, cv2 and TensorRT bindings;
native trtexec/server are present. The actual supervisor/transport environment
must explicitly include and validate its dependencies. Numeric-library thread
limits are pinned to one after the bounded synthetic local-surface comparison
(mean 198.0 ms default versus 63.2 ms with limits); algorithm thresholds were not
changed. Real-cloud timing, output parity and whole-graph scheduling remain gates.

The initial DDRNet image-only probe failed before inference: the installed image
does not contain the checkpoint at its logical asset path. The bounded baseline
therefore uses an explicit read-only pinned checkpoint and runner mount. This is
valid component measurement, but fails the target's standalone packaging claim.
Super-gradients also needs a writable log directory; a bounded temporary log
mount was required. The new image must declare scratch areas explicitly without
requiring a writable host home, code checkout or implicit weight cache.

## Preregistered engineering acceptance

Owner clarification, 2026-09-02 MSK: bounded laboratory overload is an admissible
experimental outcome. Keep useful profiles that fail the current 4090 target;
do not block reusable runtime/packaging work solely on that performance result.
Functional execution, experimental availability, quality and real-time
qualification are separate dimensions. Qualification belongs to a measured
profile/config/hardware/source/transport combination, not permanently to a
model name. More powerful Worker/onboard placement is a future retest target,
not a claim that existing latency or internal IPC disappears automatically.
The original budgets and failed measurements below remain unchanged. No slowed
source clock, hidden drops, stale-as-current evidence or actuation is authorized.

The manifest freezes an initial engineering candidate: p95/p99 full output age
≤125 ms, required-layer age ≤250 ms, release lag ≤25 ms, first incremental result
≤1 s after source admission, warmup ≤120 s, stop ≤5 s, VRAM ≤22000 MiB, RSS ≤8192
MiB. First/last-window backlog growth ≤25 ms and no capacity drops, failures,
expired selected frames or unaccounted observations. These are prototype targets,
not physical braking/safety limits or claims of achieved performance.

`ReplayMeasurements`/`realtime_failures` reject full-source preload, EOF-only
results, slowed replay, incomplete accounting, hidden missing layers and budget
violations. Declared source gaps must be verified from the input ledger; they
produce explicit degraded scenes and are not counted as fully fresh perception.
At least one fully evaluable scene is required. A component FPS figure cannot
populate this whole-path receipt. A quality comparison or successful export is
not a substitute for the real-time gate.

Every observation receives a terminal ledger outcome. Track ingress/release,
selection, decode, inference, completion, emission, drop/expiry/failure and source
gaps separately. Store startup separately from steady state; record payload bytes,
queue high-water marks, clock error, RSS/VRAM, image/weights/config/source/equipment
identities. Export and UI encoding cannot block inference through an unlimited
queue. A receiver-side render/receipt timestamp, not just Worker completion, ends
the end-to-end interval.

## Evidence and remaining stage-1 work

See `experiments/perception/PERCEPTION_STREAM_STAGE1_2026-09-01.md` for measured
component results, bounded validation, Worker maintenance and remaining gates.
Neither this ADR nor passing synthetic contract tests marks the new runtime ready.