100 lines
5.3 KiB
Markdown
100 lines
5.3 KiB
Markdown
# ADR 0007: K1 camera preview copy-remux gateway
|
|
|
|
Status: accepted for the local laboratory runtime, 2026-07-16.
|
|
|
|
Supersession note, 2026-07-17: the live copy-remux and lease decisions remain
|
|
active. ADR 0008 extends the same acquisition-owned producer with durable fMP4
|
|
epochs and a generic recorded player. That extension does not weaken the
|
|
single-live-producer or opaque transport boundary defined here.
|
|
|
|
## Context
|
|
|
|
The exact K1 firmware 3.0.2 profile exposes two RTSP/TCP H.264 preview paths.
|
|
Traffic evidence and the direct physical gate establish High Profile Level 4.0,
|
|
800x600, approximately 10 fps, about 6.5 Mbit/s and a one-second GOP. The device
|
|
delivers frames to only one preview consumer/path at a time. Browsers cannot read
|
|
RTSP directly and device addresses must not enter generic UI contracts.
|
|
|
|
The product target will eventually require portable Edge packaging and remote
|
|
delivery. This milestone only needs a low-latency, same-Mac laboratory path that
|
|
does not bake XGRIDS knowledge into the host UI.
|
|
|
|
## Decision
|
|
|
|
1. The XGRIDS plugin owns the RTSP session and accepts only the two reviewed
|
|
`source_id` values. The browser never supplies a host, port, path or RTSP URL.
|
|
2. The target is taken from the active, attested device session and validated as
|
|
private IPv4. Every selection has a monotonically increasing generation.
|
|
3. Switching sources terminates the previous FFmpeg process before publishing
|
|
the new generation. A stale stop cannot close a newer preview.
|
|
4. FFmpeg uses `-c:v copy`: it depacketizes the observed vendor RTP framing and
|
|
copy-remuxes H.264 into fragmented MP4. No video decode, image processing or
|
|
transcode runs in Python.
|
|
5. A background ISO-BMFF parser publishes one `ftyp+moov` init segment followed
|
|
by complete `moof+mdat` media segments. Boxes and segments are bounded; the
|
|
server queue holds at most four segments. A slow consumer causes a clean
|
|
stream stop instead of unbounded latency.
|
|
6. The generic browser renderer uses MSE with a 2 MiB pending queue and retains
|
|
at most about three seconds in `SourceBuffer`. It never drops an arbitrary
|
|
dependent fragment; overflow resets the lease.
|
|
7. Plugin state exposes only an opaque same-origin WebSocket URL, media type,
|
|
generation and generic activation metadata. Device RTSP details remain inside
|
|
the plugin.
|
|
|
|
## Verified gate
|
|
|
|
The exact captured K1 codec sequence was passed in memory through the selected
|
|
copy-remux flags and Chrome MSE. Chrome decoded High L4 at 800x600 with no media
|
|
error and without transcoding. No camera frame fixture was persisted or added to
|
|
Git.
|
|
|
|
The final physical gate was completed on 2026-07-17 with the exact firmware
|
|
3.0.2 profile. In one Mission Core acquisition the operator confirmed:
|
|
|
|
- continuously updating point-cloud geometry in the embedded spatial scene;
|
|
- a visible trajectory matching an approximately 3.2 by 3.35 metre physical
|
|
route (about 1.29 metres of vertical span in the captured poses);
|
|
- live left and right camera selection through the generic source controls;
|
|
- graceful operator-confirmed scanner stop, receiver finalization and media
|
|
process cleanup.
|
|
|
|
The retained out-of-Git evidence records 11,189 point-cloud frames, 11,564 pose
|
|
frames, 33,481,543 published points and zero decode errors. The live preview
|
|
queue reported 2,273 bounded drops under load; this is preview backpressure,
|
|
not loss of the raw-first evidence, and remains a performance follow-up.
|
|
|
|
During the gate the scene initially appeared frozen even though counters and
|
|
captures advanced. The cause was the embedded Rerun time cursor leaving the live
|
|
edge. The accepted implementation now declares the `stream_time` timeline in
|
|
native `Following` state in the backend blueprint; the frontend selects that
|
|
timeline once when the recording opens and does not poll or force the cursor.
|
|
|
|
## Deliberate limits
|
|
|
|
- The current gateway allows one browser consumer because the local UI owns one
|
|
operator preview. Multi-tab fan-out is not claimed.
|
|
- The Homebrew FFmpeg found on this Mac is marked `development-system`. Portable
|
|
releases must supply a pinned repository-local or configured binary.
|
|
- Same-origin local WebSocket delivery is not the future WAN transport. A later
|
|
Edge/Control Station split should use an authenticated WebRTC/WHEP or equivalent
|
|
congestion-aware media plane behind the same descriptor contract.
|
|
- Durable camera recording and host-arrival-best-effort saved-session rewind are
|
|
implemented under ADR 0008, but no retained real K1 session made after that
|
|
archive contract has completed physical recorded-camera acceptance yet.
|
|
- Device-clock synchronization, calibration, frustums, panoramic stitching,
|
|
full-resolution raw imagery and remote multi-consumer delivery remain
|
|
unavailable.
|
|
- A longer physical soak test is still required to characterize sustained
|
|
preview backpressure and rendering cost on target Edge hardware.
|
|
|
|
## Code anchors
|
|
|
|
- `src/k1link/device_plugins/xgrids_k1/camera.py`
|
|
- `src/k1link/web/camera_archive.py`
|
|
- `src/k1link/device_plugins/xgrids_k1/facade.py`
|
|
- `apps/control-station/src/components/MseFmp4WebSocketPlayer.tsx`
|
|
- `apps/control-station/src/components/RecordedFmp4Player.tsx`
|
|
- `apps/control-station/src/core/observation/useObservationLayout.ts`
|
|
- `plugins/xgrids-k1/frontend/src/observationSources.ts`
|
|
- `tests/test_xgrids_camera_gateway.py`
|