SILVER
/
NODEDC_MISSION_CORE
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
NODEDC_MISSION_CORE/docs/adr/0007-k1-camera-preview-copy...

5.3 KiB
Raw Blame History

ADR 0007: K1 camera preview copy-remux gateway

Status: accepted for the local laboratory runtime, 2026-07-16.

Supersession note, 2026-07-17: the live copy-remux and lease decisions remain active. ADR 0008 extends the same acquisition-owned producer with durable fMP4 epochs and a generic recorded player. That extension does not weaken the single-live-producer or opaque transport boundary defined here.

Context

The exact K1 firmware 3.0.2 profile exposes two RTSP/TCP H.264 preview paths. Traffic evidence and the direct physical gate establish High Profile Level 4.0, 800x600, approximately 10 fps, about 6.5 Mbit/s and a one-second GOP. The device delivers frames to only one preview consumer/path at a time. Browsers cannot read RTSP directly and device addresses must not enter generic UI contracts.

The product target will eventually require portable Edge packaging and remote delivery. This milestone only needs a low-latency, same-Mac laboratory path that does not bake XGRIDS knowledge into the host UI.

Decision

  1. The XGRIDS plugin owns the RTSP session and accepts only the two reviewed source_id values. The browser never supplies a host, port, path or RTSP URL.
  2. The target is taken from the active, attested device session and validated as private IPv4. Every selection has a monotonically increasing generation.
  3. Switching sources terminates the previous FFmpeg process before publishing the new generation. A stale stop cannot close a newer preview.
  4. FFmpeg uses -c:v copy: it depacketizes the observed vendor RTP framing and copy-remuxes H.264 into fragmented MP4. No video decode, image processing or transcode runs in Python.
  5. A background ISO-BMFF parser publishes one ftyp+moov init segment followed by complete moof+mdat media segments. Boxes and segments are bounded; the server queue holds at most four segments. A slow consumer causes a clean stream stop instead of unbounded latency.
  6. The generic browser renderer uses MSE with a 2 MiB pending queue and retains at most about three seconds in SourceBuffer. It never drops an arbitrary dependent fragment; overflow resets the lease.
  7. Plugin state exposes only an opaque same-origin WebSocket URL, media type, generation and generic activation metadata. Device RTSP details remain inside the plugin.

Verified gate

The exact captured K1 codec sequence was passed in memory through the selected copy-remux flags and Chrome MSE. Chrome decoded High L4 at 800x600 with no media error and without transcoding. No camera frame fixture was persisted or added to Git.

The final physical gate was completed on 2026-07-17 with the exact firmware 3.0.2 profile. In one Mission Core acquisition the operator confirmed:

  • continuously updating point-cloud geometry in the embedded spatial scene;
  • a visible trajectory matching an approximately 3.2 by 3.35 metre physical route (about 1.29 metres of vertical span in the captured poses);
  • live left and right camera selection through the generic source controls;
  • graceful operator-confirmed scanner stop, receiver finalization and media process cleanup.

The retained out-of-Git evidence records 11,189 point-cloud frames, 11,564 pose frames, 33,481,543 published points and zero decode errors. The live preview queue reported 2,273 bounded drops under load; this is preview backpressure, not loss of the raw-first evidence, and remains a performance follow-up.

During the gate the scene initially appeared frozen even though counters and captures advanced. The cause was the embedded Rerun time cursor leaving the live edge. The accepted implementation now declares the stream_time timeline in native Following state in the backend blueprint; the frontend selects that timeline once when the recording opens and does not poll or force the cursor.

Deliberate limits

  • The current gateway allows one browser consumer because the local UI owns one operator preview. Multi-tab fan-out is not claimed.
  • The Homebrew FFmpeg found on this Mac is marked development-system. Portable releases must supply a pinned repository-local or configured binary.
  • Same-origin local WebSocket delivery is not the future WAN transport. A later Edge/Control Station split should use an authenticated WebRTC/WHEP or equivalent congestion-aware media plane behind the same descriptor contract.
  • Durable camera recording and host-arrival-best-effort saved-session rewind are implemented under ADR 0008, but no retained real K1 session made after that archive contract has completed physical recorded-camera acceptance yet.
  • Device-clock synchronization, calibration, frustums, panoramic stitching, full-resolution raw imagery and remote multi-consumer delivery remain unavailable.
  • A longer physical soak test is still required to characterize sustained preview backpressure and rendering cost on target Edge hardware.

Code anchors

  • src/k1link/web/xgrids_k1_camera.py
  • src/k1link/web/camera_archive.py
  • src/k1link/web/xgrids_k1_facade.py
  • apps/control-station/src/components/MseFmp4WebSocketPlayer.tsx
  • apps/control-station/src/components/RecordedFmp4Player.tsx
  • apps/control-station/src/core/observation/useObservationLayout.ts
  • apps/control-station/src/device-plugins/xgrids-k1/observationSources.ts
  • tests/test_xgrids_camera_gateway.py