feat(observation): add dynamic camera sources

2026-07-16 20:56:41 +03:00
parent 05bdab24a5
commit a281faf923
15 changed files with 1592 additions and 52 deletions
@@ -0,0 +1,78 @@
+# ADR 0006: Vendor-neutral observation sources and live-only timeline
+
+Status: accepted for the experimental Mission Core runtime, 2026-07-16.
+
+## Context
+
+The spatial scene and camera workspace must accept one device, a custom rig or a
+future multi-sensor vehicle without adding vendor-specific branches to the host
+UI. The currently verified K1 profile exposes one spatial visualization path and
+two independent H.264 camera preview producers. The browser does not yet have a
+camera delivery adapter, a shared point-cloud/video clock or confirmed camera
+intrinsics/extrinsics.
+
+Hard-coded camera slots, direct RTSP URLs in React components and an apparently
+seekable timeline would therefore create three false contracts: a fixed sensor
+count, browser ownership of device transport and replay that does not exist.
+
+## Decision
+
+1. A device plugin normalizes its capabilities into
+   `ObservationSourceDescriptor[]`. Mission Core consumes only these generic
+   descriptors.
+2. `id` identifies a runtime source instance; `sourceId` identifies the producer
+   inside the plugin model; `semanticChannelId` describes the canonical channel.
+   Transport addresses are not identities.
+3. Provider metadata and device/session/acquisition binding are separate. A new
+   acquisition may change layout context without changing the logical producer.
+4. The host owns visibility, z-order, position, resize and fullscreen. A plugin
+   cannot inject global layout behavior.
+5. Direct device RTSP URLs do not cross into the generic UI. A future read-only
+   camera adapter must publish a browser-decodable local delivery URL.
+6. Until a bounded buffer and clock mapping exist, the shared timeline is
+   `live-only`, `seekable=false`, `sessionRecording=false` and explicitly marked
+   `host-arrival-best-effort`.
+7. Directional 3D camera frustums are not rendered until calibration and
+   extrinsics are available and verified. An unresolved camera may be listed as
+   a source, but its orientation must not be invented.
+
+## Current K1 mapping
+
+| Producer | Canonical channel | Host representation |
+| --- | --- | --- |
+| `sensor.lidar.primary` | `spatial.point-cloud.live` | primary Rerun viewport |
+| `sensor.camera.left` | `camera.preview.live` | auxiliary media window / camera card |
+| `sensor.camera.right` | `camera.preview.live` | auxiliary media window / camera card |
+
+Both camera producers share the proven canonical channel and remain distinct by
+`sourceId`. Their current state is `declared`: the compatibility profile proves
+the channels, but no browser preview URL is advertised yet.
+
+## UI consequences
+
+- The spatial scene source picker renders any number of plugin-published sources.
+- Auxiliary media uses the design-system `WorkspaceWindow`, constrained to the
+  scene bounds, with controlled drag, resize, close and maximize state.
+- The Cameras workspace filters media modalities and renders 0, 1, 2 or N cards;
+  only its grid scrolls when the source count exceeds available space.
+- Point-cloud focus and camera focus stay inside the application panel rather
+  than taking over the browser viewport.
+- Empty and declared channels remain honest placeholders; no demo frames or
+  synthetic sensor values are substituted.
+
+## Follow-up gate
+
+The next implementation step is a plugin-owned, read-only H.264 adapter that
+terminates RTSP/RTP locally and exposes a browser-supported stream plus measured
+per-source metrics. Synchronized rewind becomes eligible only after the adapter
+provides bounded buffering and an evidenced mapping between the RTP 90 kHz clock
+and the spatial stream clock.
+
+## Code anchors
+
+- `apps/control-station/src/core/runtime/contracts.ts`
+- `apps/control-station/src/core/observation/useObservationLayout.ts`
+- `apps/control-station/src/components/ObservationSources.tsx`
+- `apps/control-station/src/components/ObservationTimeline.tsx`
+- `apps/control-station/src/workspaces/Workspaces.tsx`
+- `apps/control-station/src/device-plugins/xgrids-k1/observationSources.ts`