SILVER
/
NODEDC_MISSION_CORE
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
NODEDC_MISSION_CORE/docs/adr/0006-vendor-neutral-observa...

4.4 KiB
Raw Blame History

ADR 0006: Vendor-neutral observation sources and live-only timeline

Status: accepted for the experimental Mission Core runtime, 2026-07-16.

Context

The spatial scene and camera workspace must accept one device, a custom rig or a future multi-sensor vehicle without adding vendor-specific branches to the host UI. The currently verified K1 profile exposes one spatial visualization path and two path-labelled H.264 camera preview producers. The direct physical gate showed that only one K1 preview producer can deliver frames at a time. A shared point-cloud/video clock and camera intrinsics/extrinsics remain unavailable.

Hard-coded camera slots, direct RTSP URLs in React components and an apparently seekable timeline would therefore create three false contracts: a fixed sensor count, browser ownership of device transport and replay that does not exist.

Decision

  1. A device plugin normalizes its capabilities into ObservationSourceDescriptor[]. Mission Core consumes only these generic descriptors.
  2. id identifies a runtime source instance; sourceId identifies the producer inside the plugin model; semanticChannelId describes the canonical channel. Transport addresses are not identities.
  3. Provider metadata and device/session/acquisition binding are separate. A new acquisition may change layout context without changing the logical producer.
  4. The host owns visibility, z-order, position, resize and fullscreen. A plugin cannot inject global layout behavior.
  5. Direct device RTSP URLs do not cross into the generic UI. The read-only K1 adapter publishes an opaque, same-origin fMP4/WebSocket delivery lease.
  6. A descriptor may declare a generic activation group, capacity and selection state. The K1 camera producers share one group with maxActive=1; frontend layout enforces it for UX and the plugin remains the backend authority.
  7. Until a bounded buffer and clock mapping exist, the shared timeline is live-only, seekable=false, sessionRecording=false and explicitly marked host-arrival-best-effort.
  8. Directional 3D camera frustums are not rendered until calibration and extrinsics are available and verified. An unresolved camera may be listed as a source, but its orientation must not be invented.

Current K1 mapping

Producer Canonical channel Host representation
sensor.lidar.primary spatial.point-cloud.live primary Rerun viewport
sensor.camera.left camera.preview.live auxiliary media window / camera card
sensor.camera.right camera.preview.live auxiliary media window / camera card

Both camera producers share the proven canonical channel and remain distinct by sourceId. The plugin publishes both catalog rows dynamically, but delivery is present only on the selected producer. Selecting the peer tears down the old RTSP/FFmpeg generation before a new browser lease is issued.

UI consequences

  • The spatial scene source picker renders any number of plugin-published sources.
  • Auxiliary media uses the design-system WorkspaceWindow, constrained to the scene bounds, with controlled drag, resize, close and maximize state.
  • The Cameras workspace filters media modalities and renders 0, 1, 2 or N cards; only its grid scrolls when the source count exceeds available space.
  • Point-cloud focus and camera focus stay inside the application panel rather than taking over the browser viewport.
  • Empty and declared channels remain honest placeholders; no demo frames or synthetic sensor values are substituted.

Follow-up gate

The local laboratory adapter now terminates RTSP/RTP with FFmpeg, copy-remuxes H.264 High L4 without transcoding and publishes complete bounded fMP4 segments to a generic MSE player. Portable FFmpeg packaging, multi-consumer fan-out and remote/WAN delivery are not claimed by this milestone. Synchronized rewind becomes eligible only after persisted buffering and an evidenced mapping between the RTP 90 kHz clock and the spatial stream clock. See ADR 0007.

Code anchors

  • apps/control-station/src/core/runtime/contracts.ts
  • apps/control-station/src/core/observation/useObservationLayout.ts
  • apps/control-station/src/components/ObservationSources.tsx
  • apps/control-station/src/components/ObservationTimeline.tsx
  • apps/control-station/src/workspaces/Workspaces.tsx
  • apps/control-station/src/device-plugins/xgrids-k1/observationSources.ts
  • src/k1link/web/xgrids_k1_camera.py