# ADR 0008: durable observation sessions and workspace layout

- Status: accepted for the laboratory Control Station
- Date: 2026-07-17
- Scope: host observation storage, recorded spatial playback, camera evidence,
  and operator workspace layout

## Context

Mission Core previously retained K1 live point/pose captures as ignored
laboratory directories, while the browser showed only the current process-wide
Rerun stream. Camera preview was copy-remuxed for a connected browser but was
not part of the durable session. A browser refresh, local service restart or an
unexpected process exit therefore made a completed run difficult to discover
and made camera history impossible to recover. The disk icon in the spatial
workspace also had no stable distinction between saving sensor evidence and
saving an operator's layout.

These are two separate product records:

1. an **observation session** is evidence from all sources active during one
   acquisition and must be recorded automatically; and
2. a **workspace layout profile** is an operator preference and is saved
   explicitly.

Conflating them would allow a UI action to decide whether evidence exists and
would make a layout save duplicate or mutate sensor data.

## Decision

### Host-owned catalog and private evidence root

The Control Station owns a private storage root selected by
`MISSIONCORE_DATA_DIR`; the repository-local fallback is
`.runtime/mission-core/`. The root is outside Git and contains a SQLite catalog
plus derived playback artifacts. New source evidence is written to the root
selected by `MISSIONCORE_EVIDENCE_DIR`, falling back to
`<MISSIONCORE_DATA_DIR>/evidence/sessions`. The current K1 camera gateway still
requires that root to remain inside the repository checkout; an external root
can retain MQTT evidence but camera archival fails closed. Treat support for an
independently attested external camera-evidence root as an open extraction item,
not as implemented behavior. SQLite uses foreign keys, WAL journaling and full
synchronous durability. Public APIs expose opaque session identifiers and
metadata, never filesystem paths.

Current acquisition writers create only direct children of the private
evidence root. Existing repository-level `sessions/*_viewer_live` runs are a
legacy import-only source: they are indexed in place, never copied or moved,
and are not assigned to a new writer. Their native `.k1mqtt` capture remains
the source of record. Import is repeatable and runs in a background reconciler,
so completed evidence becomes visible without a restart and HTTP catalog reads
never scan the evidence roots. Startup recovery may
seal an already incomplete canonical camera epoch under the legacy root; it
does not start or append a new acquisition there.

Before a new session directory is created, the writer atomically creates and
locks `<evidence-root>/.current_session`. The marker names the direct child and
prevents a second process from writing or replaying the active session. Process
exit releases the operating-system lock. On startup, an unlocked stale marker
is removed only after no-follow inode revalidation; the interrupted evidence
directory is preserved for normal recovery and cataloging.

If a process exits after raw and metadata creation but before a final summary,
the importer accepts only a bounded, structurally valid prefix with aligned
metadata and marks the run `interrupted`. It does not invent completion data.
When the valid final summary later appears, repeat import promotes the same
session to `ready`.

Native MQTT persistence uses a bounded group commit of at most 0.5 seconds,
4 MiB or 32 messages. Raw bytes are fsynced before their aligned metadata rows.
The final uncommitted group may be lost, and a raw tail may survive without
committed metadata; replay stops at the last validated aligned boundary.

### Recorded spatial playback

When a native capture is normally sealed or recovery-sealed during the current
process lifetime, a bounded single-worker reconciler automatically prepares its
private derived `.rrd`. The reconciler's first scan establishes a historical
baseline and does not enqueue old sessions. Preparation is a backend lifecycle
and is never executed inside an HTTP replay request: an explicit request may
enqueue one cold historical package and returns `202` while the worker owns the
conversion. Reopening, seeking, switching tabs, listing the catalog or changing
display settings reads the already published artifact and does not rerun
conversion. Export is lossless
with respect to every decodable point and pose message in the native capture;
it does not pass through the bounded live-preview queue. The RRD uses the
recording-local `session_time` duration timeline, while the receive wall clock
remains secondary evidence.

Derived files are published atomically, hashed and accompanied by a cache
sidecar bound to the native file identity and digest. A process-owned queue and
cross-process file lock prevent overlapping exporters; crash candidates are
scavenged before quota accounting. A stale or inconsistent cache is rejected
and rebuilt. The browser receives the RRD only through the same-origin opaque
session API. It keeps the viewport and timeline hidden until the complete
declared recording has been downloaded and decoded, then publishes the scene
atomically and starts playback.

### Camera recording

Camera acquisition is owned by the device acquisition session, not by a
browser window. Preview remains bounded and disposable; archive writes occur
before preview fan-out. Closing or refreshing a camera window must not stop the
archive while acquisition is active.

Each codec epoch is stored below the observation session as an initialization
segment, independently addressable fMP4 media segments, an append-only JSONL
index and an atomic summary. Host arrival epoch and monotonic timestamps are
recorded for every segment. The `per-segment-fsync` contract makes `init.mp4`
and each complete media segment plus its directory entry durable before its
matching index row is committed; the index and interrupted checkpoint are
fsynced before append returns. Therefore camera RPO is bounded by the fragment
currently being produced, not by an interval or byte batch. Startup recovery
retains a contiguous valid prefix, quarantines orphan fragments and marks an
unfinalized epoch `interrupted`.

The background recording-preparation worker also prepares immutable camera
descriptors before publishing `ready`. Complete-fragment arrival timestamps are
end/availability anchors, so the worker SHA-verifies and parses every fragment,
anchors the first decoded sample at `first_arrival - first_duration`, and derives
coverage from the exact sum of ISO-BMFF sample durations. It computes finite
non-overlapping epoch coverage ends and exact aggregate bytes, then binds both to
recorded-media manifest v2's generation digest. Replay v2 additionally requires
camera coverage to stay inside the spatial RRD range with a 50 ms tolerance; a
future replay v3 will need distinct spatial and session-union ranges.

The path-free descriptor is atomically persisted in the private derived cache
with its checksum and a stat identity covering the native timing origin and
every camera summary, index, init and segment file. Restart performs confined
stat validation and reuses an unchanged descriptor without reading/hashing media.
A missing, corrupt or stale historical sidecar stays cold until that session is
explicitly opened; startup, catalog reads and completion of another session do
not rebuild it. An unparseable or ambiguous fragment fails readiness rather than
exposing a fake seekable camera. Replay requests consume the prepared descriptor
and never repeat media conversion or timing analysis. Launch and manifest
aggregate byte counts must agree before the browser admits camera payload
downloads.

The derived RRD writes an actual zero-time anchor at the internal
`/__mission_core/session_origin` entity. Keeping it outside `/world` prevents a
synthetic visualization layer while ensuring that the decoded RRD timeline,
not merely its summary document, begins at session time zero. This changes the
derived payload contract. Cache v9 keeps that anchor, binds the owning plugin,
ordered source artifacts and capture-envelope endpoints, and materializes the
sealed completion rows. Older sidecars are incompatible and remain cold until
that session is explicitly opened.

Replay v2 exposes two paths to the same pinned generation. `source_url` retains
the strict `If-Match: "sha256:…"` contract for Mission Core clients;
`viewer_source_url` appends the exact lowercase SHA-256 as a `generation` query
for Rerun's native URL loader, which cannot attach custom headers. The server
accepts either precondition, rejects a stale query with `412`, and keeps the
matching file pinned for the entire immutable/no-transform response lifecycle.

This is host-arrival synchronization, not proven sensor-clock synchronization.
It is sufficient to correlate current K1 point/pose and preview evidence at the
known clock boundary, but must not be described as frame-accurate optical/LiDAR
alignment.

Historical runs made before this ADR contain no camera archive. The fact that a
camera was visible in the old browser preview does not make video recoverable;
the catalog must not manufacture a camera modality for those sessions.

### Workspace layout

The first disk action in the spatial workspace saves only the versioned
`observation.spatial` layout document:

- scene display settings;
- visible dynamic source identifiers;
- active floating source;
- source-window rectangles normalized to the observed viewport.

Schema v2 deliberately excludes open/closed state and z-order of the transient
**Движок / Слои / Отображение** tool windows. They are children of the mounted
spatial workspace, close on every workspace/root/content exit and always start
closed. Startup migration removes the obsolete `tool_windows` object from a
stored schema-v1 row. The document uses optimistic revision control (`ETag` and
`If-Match`) and is restored automatically on the next opening. Unknown source
identifiers remain desired state so a temporarily disconnected camera can
recover its saved window when that source returns. The save action never starts,
completes or modifies an observation session.

### API boundary

The stable host routes are:

- `GET /api/v1/observation-sessions` for the recent catalog;
- `GET /api/v1/observation-sessions/{id}` for modality/artifact metadata;
- `POST /api/v1/observation-sessions/{id}/replay` for a ready launch or a
  preparation handle;
- `GET /api/v1/observation-sessions/{id}/recording-preparation` for exact-job
  status polling;
- `GET /api/v1/observation-sessions/{id}/recording.rrd` for seekable spatial data;
- opaque manifest/init/segment routes below
  `/api/v1/observation-sessions/{id}/media/{artifact}` for recorded cameras;
- `GET|PUT /api/v1/workspace-layouts/observation.spatial` for the layout profile.

The session API precedes the static frontend mount. Recording paths and storage
roots are server implementation details.

## Consequences

- A completed or interrupted spatial run is discovered and prepared
  automatically, then can be replayed after browser and service restarts.
- A service crash can lose the final uncommitted MQTT group and the camera
  fragment currently being produced; it cannot make an index point beyond
  durable camera media or replay a raw tail beyond aligned MQTT metadata.
- Raw K1 evidence remains authoritative and derived RRD files can be discarded
  and regenerated.
- Browser performance no longer controls whether camera evidence is retained.
- Layout restoration is portable across viewport sizes and dynamic device
  source catalogs.
- Replay builds a separate recorded-source catalog, so an active device camera
  can never be mixed into the evidence of a saved session. Archived fMP4 codec
  epochs follow Rerun play/pause/seek through host-arrival best-effort offsets.
- A preparation ETag binds polling and cancellation to one exact job; a short
  launch reservation prevents cache eviction between the launch response and
  the viewer's first RRD request. Browser session switching never cancels the
  shared backend preparation.
- Storage capacity, retention, quota, encryption-at-rest and export policy are
  still deployment responsibilities; this ADR does not silently delete source
  evidence.
- Recorded camera playback is confined to canonical archives with a supported
  fMP4 codec declaration and provable sample duration. Missing historical media,
  unsupported codec epochs and ambiguous timing fail closed; none is substituted
  with a live preview.

## Rejected alternatives

- **Use browser cache or IndexedDB for sessions.** This loses the evidence when
  the browser profile is cleared and cannot guarantee acquisition durability.
- **Save a session only when the operator presses the disk icon.** This makes
  safety evidence optional and fails on crashes or forgotten clicks.
- **Store only RRD.** This discards the native evidence needed to review future
  decoders and protocol assumptions.
- **Keep camera acquisition coupled to WebSocket viewers.** A UI disconnect
  would stop recording and recreate the observed data-loss bug.
- **Copy every legacy run into the private catalog root.** This doubles large
  artifacts without improving integrity; confined zero-copy indexing is
  sufficient for the current laboratory store.
