docs(observation): document durable sessions and viewer lifecycle
This commit is contained in:
DCCONSTRUCTIONS
@@ -2,6 +2,11 @@
|
||||
|
||||
Status: accepted for the experimental Mission Core runtime, 2026-07-16.
|
||||
|
||||
Scope update, 2026-07-17: the live device descriptors in this ADR remain
|
||||
live-only. ADR 0008 adds a separate host-owned recorded-source catalog,
|
||||
seekable `session_time` playback and workspace layout; it does not retroactively
|
||||
turn a live RTSP lease into recorded evidence.
|
||||
|
||||
## Context
|
||||
|
||||
The spatial scene and camera workspace must accept one device, a custom rig or a
|
||||
@@ -68,10 +73,12 @@ RTSP/FFmpeg generation before a new browser lease is issued.
|
||||
|
||||
The local laboratory adapter now terminates RTSP/RTP with FFmpeg, copy-remuxes
|
||||
H.264 High L4 without transcoding and publishes complete bounded fMP4 segments
|
||||
to a generic MSE player. Portable FFmpeg packaging, multi-consumer fan-out and
|
||||
remote/WAN delivery are not claimed by this milestone. Synchronized rewind
|
||||
becomes eligible only after persisted buffering and an evidenced mapping between
|
||||
the RTP 90 kHz clock and the spatial stream clock. See ADR 0007.
|
||||
to a generic MSE player. ADR 0008 subsequently added acquisition-owned durable
|
||||
segments, a prepared recorded-media manifest and host-arrival-best-effort rewind
|
||||
on the saved-session timeline. This is not proof of a shared device clock or
|
||||
frame-accurate LiDAR/camera alignment. Portable FFmpeg packaging,
|
||||
multi-consumer fan-out and remote/WAN delivery remain open. See ADR 0007 and
|
||||
ADR 0008.
|
||||
|
||||
## Code anchors
|
||||
|
||||
|
||||
@@ -2,6 +2,11 @@
|
||||
|
||||
Status: accepted for the local laboratory runtime, 2026-07-16.
|
||||
|
||||
Supersession note, 2026-07-17: the live copy-remux and lease decisions remain
|
||||
active. ADR 0008 extends the same acquisition-owned producer with durable fMP4
|
||||
epochs and a generic recorded player. That extension does not weaken the
|
||||
single-live-producer or opaque transport boundary defined here.
|
||||
|
||||
## Context
|
||||
|
||||
The exact K1 firmware 3.0.2 profile exposes two RTSP/TCP H.264 preview paths.
|
||||
@@ -73,16 +78,22 @@ timeline once when the recording opens and does not poll or force the cursor.
|
||||
- Same-origin local WebSocket delivery is not the future WAN transport. A later
|
||||
Edge/Control Station split should use an authenticated WebRTC/WHEP or equivalent
|
||||
congestion-aware media plane behind the same descriptor contract.
|
||||
- Camera recording, synchronized rewind, calibration, frustums, panoramic
|
||||
stitching and full-resolution raw imagery remain unavailable.
|
||||
- Durable camera recording and host-arrival-best-effort saved-session rewind are
|
||||
implemented under ADR 0008, but no retained real K1 session made after that
|
||||
archive contract has completed physical recorded-camera acceptance yet.
|
||||
- Device-clock synchronization, calibration, frustums, panoramic stitching,
|
||||
full-resolution raw imagery and remote multi-consumer delivery remain
|
||||
unavailable.
|
||||
- A longer physical soak test is still required to characterize sustained
|
||||
preview backpressure and rendering cost on target Edge hardware.
|
||||
|
||||
## Code anchors
|
||||
|
||||
- `src/k1link/web/xgrids_k1_camera.py`
|
||||
- `src/k1link/web/camera_archive.py`
|
||||
- `src/k1link/web/xgrids_k1_facade.py`
|
||||
- `apps/control-station/src/components/MseFmp4WebSocketPlayer.tsx`
|
||||
- `apps/control-station/src/components/RecordedFmp4Player.tsx`
|
||||
- `apps/control-station/src/core/observation/useObservationLayout.ts`
|
||||
- `apps/control-station/src/device-plugins/xgrids-k1/observationSources.ts`
|
||||
- `tests/test_xgrids_camera_gateway.py`
|
||||
|
||||
@@ -0,0 +1,230 @@
|
||||
# ADR 0008: durable observation sessions and workspace layout
|
||||
|
||||
- Status: accepted for the laboratory Control Station
|
||||
- Date: 2026-07-17
|
||||
- Scope: host observation storage, recorded spatial playback, camera evidence,
|
||||
and operator workspace layout
|
||||
|
||||
## Context
|
||||
|
||||
Mission Core previously retained K1 live point/pose captures as ignored
|
||||
laboratory directories, while the browser showed only the current process-wide
|
||||
Rerun stream. Camera preview was copy-remuxed for a connected browser but was
|
||||
not part of the durable session. A browser refresh, local service restart or an
|
||||
unexpected process exit therefore made a completed run difficult to discover
|
||||
and made camera history impossible to recover. The disk icon in the spatial
|
||||
workspace also had no stable distinction between saving sensor evidence and
|
||||
saving an operator's layout.
|
||||
|
||||
These are two separate product records:
|
||||
|
||||
1. an **observation session** is evidence from all sources active during one
|
||||
acquisition and must be recorded automatically; and
|
||||
2. a **workspace layout profile** is an operator preference and is saved
|
||||
explicitly.
|
||||
|
||||
Conflating them would allow a UI action to decide whether evidence exists and
|
||||
would make a layout save duplicate or mutate sensor data.
|
||||
|
||||
## Decision
|
||||
|
||||
### Host-owned catalog and private evidence root
|
||||
|
||||
The Control Station owns a private storage root selected by
|
||||
`MISSIONCORE_DATA_DIR`; the repository-local fallback is
|
||||
`.runtime/mission-core/`. The root is outside Git and contains a SQLite catalog
|
||||
plus derived playback artifacts. New source evidence is written to the root
|
||||
selected by `MISSIONCORE_EVIDENCE_DIR`, falling back to
|
||||
`<MISSIONCORE_DATA_DIR>/evidence/sessions`. The current K1 camera gateway still
|
||||
requires that root to remain inside the repository checkout; an external root
|
||||
can retain MQTT evidence but camera archival fails closed. Treat support for an
|
||||
independently attested external camera-evidence root as an open extraction item,
|
||||
not as implemented behavior. SQLite uses foreign keys, WAL journaling and full
|
||||
synchronous durability. Public APIs expose opaque session identifiers and
|
||||
metadata, never filesystem paths.
|
||||
|
||||
Current acquisition writers create only direct children of the private
|
||||
evidence root. Existing repository-level `sessions/*_viewer_live` runs are a
|
||||
legacy import-only source: they are indexed in place, never copied or moved,
|
||||
and are not assigned to a new writer. Their native `.k1mqtt` capture remains
|
||||
the source of record. Import is repeatable and runs in a background reconciler,
|
||||
so completed evidence becomes visible without a restart and HTTP catalog reads
|
||||
never scan the evidence roots. Startup recovery may
|
||||
seal an already incomplete canonical camera epoch under the legacy root; it
|
||||
does not start or append a new acquisition there.
|
||||
|
||||
Before a new session directory is created, the writer atomically creates and
|
||||
locks `<evidence-root>/.current_session`. The marker names the direct child and
|
||||
prevents a second process from writing or replaying the active session. Process
|
||||
exit releases the operating-system lock. On startup, an unlocked stale marker
|
||||
is removed only after no-follow inode revalidation; the interrupted evidence
|
||||
directory is preserved for normal recovery and cataloging.
|
||||
|
||||
If a process exits after raw and metadata creation but before a final summary,
|
||||
the importer accepts only a bounded, structurally valid prefix with aligned
|
||||
metadata and marks the run `interrupted`. It does not invent completion data.
|
||||
When the valid final summary later appears, repeat import promotes the same
|
||||
session to `ready`.
|
||||
|
||||
Native MQTT persistence uses a bounded group commit of at most 0.5 seconds,
|
||||
4 MiB or 32 messages. Raw bytes are fsynced before their aligned metadata rows.
|
||||
The final uncommitted group may be lost, and a raw tail may survive without
|
||||
committed metadata; replay stops at the last validated aligned boundary.
|
||||
|
||||
### Recorded spatial playback
|
||||
|
||||
When a native capture is normally sealed or recovery-sealed, a bounded
|
||||
single-worker reconciler automatically prepares its private derived `.rrd`.
|
||||
Preparation is a backend lifecycle and is never executed by an HTTP replay
|
||||
request. Reopening, seeking, switching tabs or changing display settings reads
|
||||
the already prepared artifact and does not rerun conversion. Export is lossless
|
||||
with respect to every decodable point and pose message in the native capture;
|
||||
it does not pass through the bounded live-preview queue. The RRD uses the
|
||||
recording-local `session_time` duration timeline, while the receive wall clock
|
||||
remains secondary evidence.
|
||||
|
||||
Derived files are published atomically, hashed and accompanied by a cache
|
||||
sidecar bound to the native file identity and digest. A process-owned queue and
|
||||
cross-process file lock prevent overlapping exporters; crash candidates are
|
||||
scavenged before quota accounting. A stale or inconsistent cache is rejected
|
||||
and rebuilt. The browser receives the RRD only through the same-origin opaque
|
||||
session API. It keeps the viewport and timeline hidden until the complete
|
||||
declared recording has been downloaded and decoded, then publishes the scene
|
||||
atomically and starts playback.
|
||||
|
||||
### Camera recording
|
||||
|
||||
Camera acquisition is owned by the device acquisition session, not by a
|
||||
browser window. Preview remains bounded and disposable; archive writes occur
|
||||
before preview fan-out. Closing or refreshing a camera window must not stop the
|
||||
archive while acquisition is active.
|
||||
|
||||
Each codec epoch is stored below the observation session as an initialization
|
||||
segment, independently addressable fMP4 media segments, an append-only JSONL
|
||||
index and an atomic summary. Host arrival epoch and monotonic timestamps are
|
||||
recorded for every segment. The `per-segment-fsync` contract makes `init.mp4`
|
||||
and each complete media segment plus its directory entry durable before its
|
||||
matching index row is committed; the index and interrupted checkpoint are
|
||||
fsynced before append returns. Therefore camera RPO is bounded by the fragment
|
||||
currently being produced, not by an interval or byte batch. Startup recovery
|
||||
retains a contiguous valid prefix, quarantines orphan fragments and marks an
|
||||
unfinalized epoch `interrupted`.
|
||||
|
||||
The background recording-preparation worker also prepares immutable camera
|
||||
descriptors before publishing `ready`. Complete-fragment arrival timestamps are
|
||||
end/availability anchors, so the worker SHA-verifies and parses every fragment,
|
||||
anchors the first decoded sample at `first_arrival - first_duration`, and derives
|
||||
coverage from the exact sum of ISO-BMFF sample durations. It computes finite
|
||||
non-overlapping epoch coverage ends and exact aggregate bytes, then binds both to
|
||||
recorded-media manifest v2's generation digest. Replay v2 additionally requires
|
||||
camera coverage to stay inside the spatial RRD range with a 50 ms tolerance; a
|
||||
future replay v3 will need distinct spatial and session-union ranges.
|
||||
|
||||
The path-free descriptor is atomically persisted in the private derived cache
|
||||
with its checksum and a stat identity covering the native timing origin and
|
||||
every camera summary, index, init and segment file. Restart performs confined
|
||||
stat validation and reuses an unchanged descriptor without reading/hashing media;
|
||||
a missing, corrupt or stale sidecar is rebuilt in the background. An unparseable
|
||||
or ambiguous fragment fails readiness rather than exposing a fake seekable
|
||||
camera. Replay requests consume the prepared descriptor and never repeat media
|
||||
conversion or timing analysis. Launch and manifest aggregate byte counts must
|
||||
agree before the browser admits camera payload downloads.
|
||||
|
||||
The derived RRD writes an actual zero-time anchor at the internal
|
||||
`/__mission_core/session_origin` entity. Keeping it outside `/world` prevents a
|
||||
synthetic visualization layer while ensuring that the decoded RRD timeline,
|
||||
not merely its summary document, begins at session time zero. This changes the
|
||||
derived payload contract: cache v6 rejects v4/v5 sidecars and performs one
|
||||
background rebuild instead of silently reusing an archive without that row.
|
||||
|
||||
Replay v2 exposes two paths to the same pinned generation. `source_url` retains
|
||||
the strict `If-Match: "sha256:…"` contract for Mission Core clients;
|
||||
`viewer_source_url` appends the exact lowercase SHA-256 as a `generation` query
|
||||
for Rerun's native URL loader, which cannot attach custom headers. The server
|
||||
accepts either precondition, rejects a stale query with `412`, and keeps the
|
||||
matching file pinned for the entire immutable/no-transform response lifecycle.
|
||||
|
||||
This is host-arrival synchronization, not proven sensor-clock synchronization.
|
||||
It is sufficient to correlate current K1 point/pose and preview evidence at the
|
||||
known clock boundary, but must not be described as frame-accurate optical/LiDAR
|
||||
alignment.
|
||||
|
||||
Historical runs made before this ADR contain no camera archive. The fact that a
|
||||
camera was visible in the old browser preview does not make video recoverable;
|
||||
the catalog must not manufacture a camera modality for those sessions.
|
||||
|
||||
### Workspace layout
|
||||
|
||||
The first disk action in the spatial workspace saves only the versioned
|
||||
`observation.spatial` layout document:
|
||||
|
||||
- scene display settings;
|
||||
- open tool windows and their z-order;
|
||||
- visible dynamic source identifiers;
|
||||
- active floating source;
|
||||
- source-window rectangles normalized to the observed viewport.
|
||||
|
||||
The document uses optimistic revision control (`ETag` and `If-Match`) and is
|
||||
restored automatically on the next opening. Unknown source identifiers remain
|
||||
desired state so a temporarily disconnected camera can recover its saved
|
||||
window when that source returns. The save action never starts, completes or
|
||||
modifies an observation session.
|
||||
|
||||
### API boundary
|
||||
|
||||
The stable host routes are:
|
||||
|
||||
- `GET /api/v1/observation-sessions` for the recent catalog;
|
||||
- `GET /api/v1/observation-sessions/{id}` for modality/artifact metadata;
|
||||
- `POST /api/v1/observation-sessions/{id}/replay` for a ready launch or a
|
||||
preparation handle;
|
||||
- `GET /api/v1/observation-sessions/{id}/recording-preparation` for exact-job
|
||||
status polling;
|
||||
- `GET /api/v1/observation-sessions/{id}/recording.rrd` for seekable spatial data;
|
||||
- opaque manifest/init/segment routes below
|
||||
`/api/v1/observation-sessions/{id}/media/{artifact}` for recorded cameras;
|
||||
- `GET|PUT /api/v1/workspace-layouts/observation.spatial` for the layout profile.
|
||||
|
||||
The session API precedes the static frontend mount. Recording paths and storage
|
||||
roots are server implementation details.
|
||||
|
||||
## Consequences
|
||||
|
||||
- A completed or interrupted spatial run is discovered and prepared
|
||||
automatically, then can be replayed after browser and service restarts.
|
||||
- A service crash can lose the final uncommitted MQTT group and the camera
|
||||
fragment currently being produced; it cannot make an index point beyond
|
||||
durable camera media or replay a raw tail beyond aligned MQTT metadata.
|
||||
- Raw K1 evidence remains authoritative and derived RRD files can be discarded
|
||||
and regenerated.
|
||||
- Browser performance no longer controls whether camera evidence is retained.
|
||||
- Layout restoration is portable across viewport sizes and dynamic device
|
||||
source catalogs.
|
||||
- Replay builds a separate recorded-source catalog, so an active device camera
|
||||
can never be mixed into the evidence of a saved session. Archived fMP4 codec
|
||||
epochs follow Rerun play/pause/seek through host-arrival best-effort offsets.
|
||||
- A preparation ETag binds polling and cancellation to one exact job; a short
|
||||
launch reservation prevents cache eviction between the launch response and
|
||||
the viewer's first RRD request. Browser session switching never cancels the
|
||||
shared backend preparation.
|
||||
- Storage capacity, retention, quota, encryption-at-rest and export policy are
|
||||
still deployment responsibilities; this ADR does not silently delete source
|
||||
evidence.
|
||||
- Recorded camera playback is confined to canonical archives with a supported
|
||||
fMP4 codec declaration and provable sample duration. Missing historical media,
|
||||
unsupported codec epochs and ambiguous timing fail closed; none is substituted
|
||||
with a live preview.
|
||||
|
||||
## Rejected alternatives
|
||||
|
||||
- **Use browser cache or IndexedDB for sessions.** This loses the evidence when
|
||||
the browser profile is cleared and cannot guarantee acquisition durability.
|
||||
- **Save a session only when the operator presses the disk icon.** This makes
|
||||
safety evidence optional and fails on crashes or forgotten clicks.
|
||||
- **Store only RRD.** This discards the native evidence needed to review future
|
||||
decoders and protocol assumptions.
|
||||
- **Keep camera acquisition coupled to WebSocket viewers.** A UI disconnect
|
||||
would stop recording and recreate the observed data-loss bug.
|
||||
- **Copy every legacy run into the private catalog root.** This doubles large
|
||||
artifacts without improving integrity; confined zero-copy indexing is
|
||||
sufficient for the current laboratory store.
|
||||
Reference in New Issue
Block a user