docs(observation): document durable sessions and viewer lifecycle
This commit is contained in:
DCCONSTRUCTIONS
parent
e94c64eebd
commit
f9ffb7bd1c
69
README.md
69
README.md
|
|
@ -10,9 +10,11 @@ The first proven hardware vertical is the XGRIDS/LixelKity K1 plugin. On firmwar
|
|||
3.0.2 the host provisions the scanner onto an existing LAN without LixelGO,
|
||||
connects to its MQTT broker, persists each raw frame before preview work, decodes
|
||||
point cloud and pose, and renders the real cloud plus trajectory through an
|
||||
embedded self-hosted Rerun Web Viewer. Capture files are `fsync`ed on clean
|
||||
close; per-frame power-loss durability is not claimed. The former Foxglove
|
||||
bridge remains only as a legacy regression module.
|
||||
embedded self-hosted Rerun Web Viewer. Native MQTT persistence uses bounded
|
||||
group commit (at most 0.5 seconds, 4 MiB or 32 messages), and camera archives
|
||||
commit complete fMP4 segments before their index rows. These are explicit
|
||||
crash-RPO bounds, not a zero-loss or disk-replication claim. The former
|
||||
Foxglove bridge remains only as a legacy regression module.
|
||||
|
||||
The repository is intentionally migrating in stages. The current `src/k1link`
|
||||
package is the compatibility implementation of the first plugin path; vendor
|
||||
|
|
@ -124,19 +126,24 @@ revision or content hash. Publishing/vendoring those packages or enforcing an
|
|||
immutable donor revision remains a packaging and CI prerequisite.
|
||||
|
||||
The Observation spatial workspace embeds the open-source Rerun Web Viewer
|
||||
inside the Mission Core shell. It can open a compatible RRD file over HTTP(S) or a
|
||||
Rerun gRPC/proxy source such as `rerun+http://127.0.0.1:9876/proxy`. It does not
|
||||
use an external hosted viewer UI. Dynamic point-cloud and camera source
|
||||
composition, host-owned window layout and the current live-only timeline contract
|
||||
are fixed in [`ADR 0006`](docs/adr/0006-vendor-neutral-observation-sources-and-live-only-timeline.md).
|
||||
inside the Mission Core shell. It can open a compatible RRD over same-origin
|
||||
HTTP or a Rerun gRPC/proxy source such as
|
||||
`rerun+http://127.0.0.1:9876/proxy`; no external hosted viewer UI is used.
|
||||
Dynamic point-cloud/camera composition and the live source contract are fixed in
|
||||
[`ADR 0006`](docs/adr/0006-vendor-neutral-observation-sources-and-live-only-timeline.md).
|
||||
The durable session catalog, recorded `session_time` scrubber and versioned
|
||||
workspace-layout profile are fixed in
|
||||
[`ADR 0008`](docs/adr/0008-durable-observation-sessions-and-workspace-layout.md).
|
||||
|
||||
The first K1 live session or replay in a `k1link serve` process creates one local
|
||||
Rerun `RecordingStream`, starts its gRPC/proxy server on TCP 9876 and publishes
|
||||
the resulting URL through control-plane state. Later sessions reset their
|
||||
session-local scene and metrics and reuse that process-wide stream; this avoids
|
||||
restarting the native listener while the embedded browser remains connected.
|
||||
Unless an operator has entered a manual source, the React application assigns
|
||||
that URL to the embedded viewer. The complete runtime path is K1 MQTT → raw-first
|
||||
The first K1 live session or adapter file-replay (`.k1mqtt`/reviewed TSV) in a
|
||||
`k1link serve` process creates one local Rerun `RecordingStream`, starts its
|
||||
gRPC/proxy server on TCP 9876 and publishes the resulting URL through
|
||||
control-plane state. Later live/file-replay sessions reset their session-local
|
||||
scene and metrics and reuse that process-wide stream. Saved observation sessions
|
||||
do not reuse this listener: they open a private digest-bound RRD generation over
|
||||
same-origin HTTP through Rerun's native incremental receiver. Unless an operator
|
||||
has entered a manual source, the React application assigns the applicable source
|
||||
to the embedded viewer. The complete live runtime path is K1 MQTT → raw-first
|
||||
evidence capture → bounded latest-wins preview queue → explicitly injected K1
|
||||
protobuf/LZ4 normalizer → transport-neutral decoded local views → Rerun
|
||||
`Points3D`, `Transform3D` and `LineStrips3D` → embedded Web Viewer. Rerun does
|
||||
|
|
@ -146,15 +153,21 @@ portable Plugin SDK wire envelopes.
|
|||
The default Rerun blueprint shows a 12-second sliding accumulation of real point
|
||||
frames. Product controls are connected for point size, intensity/height/distance
|
||||
or available RGB coloring, Turbo/Viridis/Plasma/grayscale/custom palettes,
|
||||
point and trajectory visibility, and the scene grid. Projection, custom
|
||||
timeline transport and saved layout remain later product work. No synthetic
|
||||
point cloud, trajectory, camera frame or latency value is generated.
|
||||
point and trajectory visibility, and the scene grid. The first disk action saves
|
||||
and restores the versioned spatial layout without mutating sensor evidence.
|
||||
Saved native point/pose sessions are materialized losslessly into private,
|
||||
digest-bound RRD recordings and can be played, paused and scrubbed on a
|
||||
zero-based `session_time` timeline. No synthetic point cloud, trajectory,
|
||||
camera frame or latency value is generated.
|
||||
|
||||
A powered-device checkpoint passed 80 real MQTT messages through the current
|
||||
Rerun runtime: 38 point-cloud frames, 42 pose frames, 2,775 points in the last
|
||||
cloud and zero decode errors. The later RTSP camera preview is not yet wired into
|
||||
the Rerun/runtime path. Rerun `capture_time` is the Mac receive timestamp, not a
|
||||
proven K1 sensor timestamp or photon-to-screen measurement.
|
||||
cloud and zero decode errors. The later RTSP camera preview is available through
|
||||
the generic floating observation windows and new acquisitions archive its fMP4
|
||||
segments independently of browser delivery. Historical sessions recorded before
|
||||
that archive contract contain no video. Rerun `capture_time` and the camera
|
||||
index use Mac receive/arrival timestamps, not proven K1 sensor timestamps or a
|
||||
photon-to-screen measurement.
|
||||
|
||||
The old Foxglove implementation is retained only in
|
||||
`src/k1link/viewer/foxglove_bridge.py` and its regression tests. The current
|
||||
|
|
@ -162,6 +175,8 @@ live/replay runtime does not start it or use TCP 8765. The
|
|||
[live viewer runbook](docs/06_K1_LIVE_VIEWER.md) records the active Rerun path and
|
||||
its timing/security boundaries; the frontend contract is documented in
|
||||
[`apps/control-station/README.md`](apps/control-station/README.md).
|
||||
The host storage layout, recovery rules, replay API and operator path are in
|
||||
[`docs/09_OBSERVATION_SESSIONS.md`](docs/09_OBSERVATION_SESSIONS.md).
|
||||
|
||||
The FastAPI application and credential endpoint bind to loopback, but the Rerun
|
||||
gRPC server currently binds TCP 9876 on all network interfaces even though its
|
||||
|
|
@ -214,9 +229,15 @@ present.
|
|||
- [Verified MQTT stream profile](docs/05_K1_MQTT_STREAM_PROFILE.md)
|
||||
- [Live console and embedded Rerun runbook](docs/06_K1_LIVE_VIEWER.md)
|
||||
- [Mission Core monorepo and plugin boundary](docs/07_MISSION_CORE_MONOREPO.md)
|
||||
- [Owner-controlled LixelGO/iPhone observation](docs/08_LIXELGO_IPHONE_OBSERVATION.md)
|
||||
- [Observation sessions, playback and workspace layout](docs/09_OBSERVATION_SESSIONS.md)
|
||||
- [Monorepo architecture decision](docs/adr/0002-mission-core-monorepo.md)
|
||||
- [Device plugin UI and runtime boundary](docs/adr/0003-device-plugin-ui-and-runtime-boundary.md)
|
||||
- [Plugin SDK v0alpha2 and experimental device lifecycle](docs/adr/0004-plugin-sdk-v0alpha2-and-experimental-device-lifecycle.md)
|
||||
- [Owner-controlled LixelGO observation decision](docs/adr/0005-owner-controlled-lixelgo-iphone-observation.md)
|
||||
- [Vendor-neutral live observation sources](docs/adr/0006-vendor-neutral-observation-sources-and-live-only-timeline.md)
|
||||
- [K1 camera preview copy-remux gateway](docs/adr/0007-k1-camera-preview-copy-remux-gateway.md)
|
||||
- [Durable observation sessions and workspace layout](docs/adr/0008-durable-observation-sessions-and-workspace-layout.md)
|
||||
- [Redacted live lab report](docs/lab/001_K1_LIVE_MQTT_20260715.redacted.md)
|
||||
- [Session manifest schema](schemas/session-manifest.schema.json)
|
||||
- [Reference input provenance](docs/reference/README.md)
|
||||
|
|
@ -240,6 +261,6 @@ reviewed step. Random writes, fuzzing, brute force, firmware operations,
|
|||
destructive file access and credential guessing remain out of scope.
|
||||
|
||||
Real captures, projects, router metadata, serials, credentials, maps, images,
|
||||
and logs are ignored by normal Git. Redacted manifests and SHA-256 inventories
|
||||
are committed; encrypted artifact storage will be selected only when real data
|
||||
exists.
|
||||
and logs under `.runtime/`, canonical evidence roots and legacy `sessions/` are
|
||||
ignored by normal Git. Redacted manifests and SHA-256 inventories are committed;
|
||||
encrypted/replicated artifact storage remains a deployment decision.
|
||||
|
|
|
|||
|
|
@ -7,9 +7,10 @@ device adapter, но структура интерфейса от него не
|
|||
|
||||
Внутри пространственной рабочей поверхности встроен открытый Rerun Web Viewer.
|
||||
Это self-hosted frontend-компонент из npm-пакета `@rerun-io/web-viewer`, а не
|
||||
переход во внешний облачный интерфейс. При запуске K1 live/replay backend сам
|
||||
создаёт Rerun gRPC/proxy source; ручной адрес нужен только для другого Rerun
|
||||
потока или совместимой RRD-записи.
|
||||
переход во внешний облачный интерфейс. Для K1 live и adapter file-replay backend
|
||||
сам создаёт Rerun gRPC/proxy source. Saved observation sessions открывают
|
||||
проверенную digest-bound RRD generation через same-origin HTTP; ручной адрес
|
||||
нужен только для другого совместимого Rerun source.
|
||||
|
||||
## Текущее состояние
|
||||
|
||||
|
|
@ -20,11 +21,13 @@ device adapter, но структура интерфейса от него не
|
|||
| Локальный control plane | Реализован | React получает состояние и выполняет операции через FastAPI REST и WebSocket на loopback. |
|
||||
| K1 BLE → Wi-Fi | Реализован | Реальный BLE-поиск всех видимых устройств и одна подтверждённая provisioning-запись выбранному устройству. |
|
||||
| K1 live/replay MQTT | Реализован | Read-only приём, raw-first сохранение, декодирование облака точек и позы, реальные метрики. |
|
||||
| Автоматический MQTT → Rerun | Реализован | Первый live/replay поднимает process-wide `RecordingStream` и gRPC/proxy на TCP 9876; следующие сессии сбрасывают сцену и метрики и переиспользуют его. |
|
||||
| Автоматический MQTT → Rerun | Реализован | Первый live/adapter file-replay поднимает process-wide `RecordingStream` и gRPC/proxy на TCP 9876; следующие такие сессии переиспользуют его. Saved observation replay использует отдельный immutable HTTP RRD path. |
|
||||
| Встроенный Rerun Viewer | Реализован | Self-hosted npm-компонент автоматически открывает текущий gRPC source внутри Control Station; внешний viewer не используется. |
|
||||
| Контролы сцены → Rerun | Реализованы для текущей геометрии | Работают размер и видимость точек, атрибут цвета, палитра, окно накопления, траектория и сетка. Проекция, собственный timeline и сохранённые layout-профили ещё не подключены. |
|
||||
| Контролы сцены → Rerun | Реализованы для текущей геометрии | Работают размер и видимость точек, атрибут цвета, палитра, окно накопления, траектория, сетка, host timeline и сохранение/восстановление spatial layout. Проекция и семантические слои ещё не подключены. |
|
||||
| Сохранённые observation sessions | Реализованы для point/pose | Три последние сессии, background preparation, cache v6, generation-bound RRD, atomic admission, autoplay, play/pause/seek и controlled switching больших записей. |
|
||||
| K1 camera preview и archive | Live реализован; recorded contract реализован | Обе RTSP/H.264 камеры физически приняты в live UI. Новые acquisition-owned fMP4 archives не зависят от browser windows; recorded player подключён, но реальная архивная K1 camera-session ещё не прошла physical acceptance. |
|
||||
| Legacy Foxglove module | Только regression | Модуль и тесты сохранены для сравнения декодирования. Текущий live/replay runtime не запускает Foxglove WebSocket и не использует TCP 8765. |
|
||||
| Камеры, карты и миссии | Интерфейс готов | Серверная логика и реальные каналы для этих рабочих поверхностей ещё не подключены. |
|
||||
| Карты и миссии | Интерфейсный каркас | Реальные map/mission backends и vehicle control ещё не подключены. |
|
||||
|
||||
Приложение не генерирует демонстрационное облако, траекторию, кадры или
|
||||
метрики. Если реальных данных нет, область сцены остаётся пустой, а числовые поля
|
||||
|
|
@ -48,9 +51,16 @@ Mission Core Control Station ←→ REST /api/v1/device-plugins/*
|
|||
+ plugin-scoped WebSocket events
|
||||
FastAPI на 127.0.0.1:8000
|
||||
CoreBluetooth + live/replay runtime
|
||||
|
||||
sealed/recovered observation session
|
||||
└── SQLite catalog + bounded background preparation
|
||||
└── atomic RRD cache v6 + recorded-media manifest v2
|
||||
└── generation-bound same-origin HTTP
|
||||
└── aggregate admission
|
||||
└── Rerun native receiver + recorded fMP4 player
|
||||
```
|
||||
|
||||
В момент готовности `RerunBridge` backend публикует адрес вида
|
||||
В момент готовности live/file-replay `RerunBridge` backend публикует адрес вида
|
||||
`rerun+http://127.0.0.1:9876/proxy`. Frontend автоматически назначает его сцене,
|
||||
если оператор не указал ручной source. После остановки приёма URL и встроенный
|
||||
viewer остаются активны, а следующая сессия сбрасывает session-local геометрию,
|
||||
|
|
@ -65,7 +75,7 @@ viewer остаются активны, а следующая сессия сб
|
|||
Shell собран из локальных NODE.DC UI packages и сохраняет одну структуру для
|
||||
всех функциональных модулей:
|
||||
|
||||
1. `AppHeader` — марка NODEDC MISSION CORE, выбор архитектурного раздела и состояние
|
||||
1. `AppHeader` — марка NODE DC, выбор архитектурного раздела и состояние
|
||||
локального backend.
|
||||
2. `AdminNavigationPanel` — контекст аппарата и список рабочих поверхностей
|
||||
выбранного раздела.
|
||||
|
|
@ -78,8 +88,10 @@ Shell собран из локальных NODE.DC UI packages и сохраня
|
|||
blueprint-, selection- и time-панели скрыты, чтобы продуктовые действия жили в
|
||||
Control Station. Размер точек, способ окрашивания и палитра, 12-секундное по
|
||||
умолчанию накопление, видимость облака и траектории и сетка связаны с backend и
|
||||
Rerun Blueprint. Кнопки собственного timeline, смена 2D/3D/карты и сохранение
|
||||
layout пока остаются интерфейсным контрактом.
|
||||
Rerun Blueprint. Host timeline управляет сохранённым `session_time`, а spatial
|
||||
layout сохраняет display settings, tool windows и source-window geometry через
|
||||
revisioned API. Смена 2D/3D/карты и семантические слои пока остаются
|
||||
интерфейсным контрактом.
|
||||
|
||||
## Архитектурные разделы
|
||||
|
||||
|
|
@ -139,10 +151,15 @@ cd apps/control-station
|
|||
npm run dev
|
||||
```
|
||||
|
||||
Vite слушает `http://127.0.0.1:5173` и проксирует весь `/api` (включая WebSocket) на
|
||||
`http://127.0.0.1:8000`. Другой локальный backend можно указать переменной
|
||||
`VITE_API_TARGET`. Preview production-сборки запускается командой
|
||||
`npm run preview` на `http://127.0.0.1:4173`.
|
||||
Для полного live-viewer контура Vite должен открыться на
|
||||
`http://127.0.0.1:5173`: только 5173, production preview 4173 и backend 8000
|
||||
входят в текущий Rerun CORS allowlist. Если Vite сообщает fallback на 5174 или
|
||||
выше, освободите 5173 и перезапустите `npm run dev`; REST/WebSocket proxy на
|
||||
fallback-порту может работать, но прямой browser → Rerun 9876 будет отклонён.
|
||||
Vite проксирует весь `/api` (включая WebSocket) на `http://127.0.0.1:8000`.
|
||||
Другой локальный backend можно указать переменной `VITE_API_TARGET`. Preview
|
||||
production-сборки запускается командой `npm run preview` на
|
||||
`http://127.0.0.1:4173`.
|
||||
|
||||
## Операторский путь для текущего K1 adapter
|
||||
|
||||
|
|
@ -167,6 +184,9 @@ Vite слушает `http://127.0.0.1:5173` и проксирует весь `/a
|
|||
9. Открыть **Наблюдение → Пространственная сцена**. Реальные облако и траектория,
|
||||
частота, число точек, задержка и пропуски preview появятся после прихода
|
||||
сообщений K1.
|
||||
10. После нормального stop или recovery открыть **Сохранённые сессии**. Дождаться
|
||||
состояния **Готово**, выбрать запись и использовать host timeline. Evidence
|
||||
сохраняется автоматически; disk action сохраняет только workspace layout.
|
||||
|
||||
В проверочном live-сеансе через этот путь прошло 80 реальных MQTT-сообщений:
|
||||
38 кадров `lio_pcl`, 42 кадра `lio_pose`, 2 775 точек в последнем облаке и
|
||||
|
|
@ -203,9 +223,12 @@ Plasma, grayscale и custom, накопление `0–120` секунд, а т
|
|||
декодированном формате. Custom/class сейчас означает выбранный сплошной цвет,
|
||||
а не готовую семантическую классификацию.
|
||||
|
||||
Custom timeline, переключатель 2D/3D/карты, семантические слои и сохранение RBL
|
||||
пока не вызывают Blueprint/playback API. Черновик компоновки фиксируется только
|
||||
в памяти текущей страницы и не записывается на диск.
|
||||
Для saved session host timeline вызывает Rerun play/pause/seek только после
|
||||
полного admission. Accumulation и display settings применяются через отдельный
|
||||
маленький blueprint RRD и не заменяют основную запись. Versioned
|
||||
`observation.spatial` layout сохраняется на host с optimistic revision; он не
|
||||
содержит sensor evidence. Переключатель 2D/3D/карты и семантические слои пока не
|
||||
вызывают готовый presentation backend.
|
||||
|
||||
## Контракт локального API
|
||||
|
||||
|
|
@ -216,6 +239,12 @@ Custom timeline, переключатель 2D/3D/карты, семантиче
|
|||
| `GET` | `/api/v1/device-models` | Backend-каталог моделей из валидированных manifests; UI-каталог текущей сборки формируется отдельным static composition root. |
|
||||
| `POST` | `/api/v1/device-plugins/{pluginId}/actions/{actionId}` | Namespaced действие через host allowlist; тело `{ "input": { ... } }`. |
|
||||
| `WS` | `/api/v1/device-plugins/{pluginId}/events` | Plugin-scoped snapshots с `pluginId` и монотонным `sequence`. |
|
||||
| `GET` | `/api/v1/observation-sessions` | Последние cataloged sessions и authoritative preparation state. |
|
||||
| `POST` | `/api/v1/observation-sessions/{id}/replay` | Verified replay launch либо HTTP 202 preparation handle. |
|
||||
| `GET` | `/api/v1/observation-sessions/{id}/recording-preparation` | Exact-job polling с `If-Match`. |
|
||||
| `GET` | `/api/v1/observation-sessions/{id}/recording.rrd` | Immutable generation-bound RRD; arbitrary chunked `send_rrd` не используется. |
|
||||
| `GET` | `/api/v1/observation-sessions/{id}/media/{artifact}/...` | Manifest/init/segments записанных камер по opaque identifiers. |
|
||||
| `GET`, `PUT` | `/api/v1/workspace-layouts/observation.spatial` | Versioned host layout с optimistic revision. |
|
||||
|
||||
Frontend XGRIDS-плагина использует только v1alpha namespaced routes. Старые
|
||||
`/api/state`, `/api/events`, `/api/ble/scan`, `/api/connect`, `/api/session/*` и
|
||||
|
|
@ -257,7 +286,12 @@ facts и старые presentation-поля `phase`, `message`, `devices`,
|
|||
| `src/workspaces/DeviceWorkspace.tsx` | Generic выбор модели и `device.connection` slot. |
|
||||
| `src/device-plugins/xgrids-k1/` | Реальный K1 BLE/Wi-Fi/live/replay UI, client и compatibility mapper. |
|
||||
| `src/workspaces/Workspaces.tsx` | Оперативный обзор, spatial viewport и остальные продуктовые поверхности. |
|
||||
| `src/components/RerunViewport.tsx` | Жизненный цикл встроенного Rerun Web Viewer и selection events. |
|
||||
| `src/components/RerunViewport.tsx` | Live/recorded lifecycle WebViewer, native RRD open, atomic admission, playback и selection events. |
|
||||
| `src/components/ObservationSessionSelect.tsx` | Три последние сессии, состояния `Готово` / `Обработка` / `Ошибка`. |
|
||||
| `src/components/RecordedFmp4Player.tsx` | Проверенный generation-bound MSE playback архивных камер. |
|
||||
| `src/core/observation/sessionArchive.ts` | Строгий wire contract catalog/preparation/replay/media API. |
|
||||
| `src/core/observation/recordedSessionAdmission.ts` | Общий RRD/camera gate до публикации recorded workspace. |
|
||||
| `src/core/observation/workspaceLayout.ts` | Versioned spatial layout и optimistic revision. |
|
||||
| `src/sceneSettings.ts` | Типизированный UI-профиль пространственной сцены. |
|
||||
| `src/presentation.ts` | Vendor-neutral подписи lifecycle и форматирование нормализованных метрик. |
|
||||
| `src/styles.css`, `src/styles/*` | Компоновка shell и рабочих поверхностей. |
|
||||
|
|
@ -277,14 +311,17 @@ facts и старые presentation-поля `phase`, `message`, `devices`,
|
|||
случайные GATT writes и автоматические повторы запрещены.
|
||||
- MQTT live/replay не публикует команды устройству. Запуск и остановка
|
||||
физического сканирования остаются за кнопкой K1.
|
||||
- Live-сессии сначала сохраняют сырые сообщения, затем формируют preview. При
|
||||
перегрузке preview может быть отброшен, raw evidence сохраняется.
|
||||
- Live-сессии сначала сохраняют сырые сообщения и camera segments, затем
|
||||
формируют disposable preview. При перегрузке preview может быть отброшен,
|
||||
native evidence сохраняется. MQTT durability имеет bounded group-commit RPO,
|
||||
camera durability — текущий незавершённый fragment RPO; это не zero-loss claim.
|
||||
- Timeline `capture_time` сохраняет Unix-время приёма сообщения Mac, а окно
|
||||
накопления viewer использует session-local `stream_time`. `capture_time` — не
|
||||
доказанный timestamp сенсора K1 и не photon-to-screen latency.
|
||||
- Панорамный камерный поток в наблюдавшихся MQTT report topics отсутствует;
|
||||
текущая Rerun-сцена содержит только облако точек и позу/траекторию, а
|
||||
операционные метрики отображаются внешней оболочкой Mission Core.
|
||||
- `sessions/` игнорируется Git и может содержать адреса, идентификаторы,
|
||||
траекторию и карту помещения. В репозиторий попадают только redacted manifests
|
||||
и безопасная документация.
|
||||
- Панорамный камерный поток в MQTT report topics отсутствует; отдельные
|
||||
left/right RTSP preview доступны через generic camera windows. LiDAR и camera
|
||||
сейчас синхронизированы только по host arrival, не по доказанным sensor clocks.
|
||||
- `.runtime/`, canonical evidence roots и legacy `sessions/` игнорируются Git и
|
||||
могут содержать адреса, изображения, идентификаторы, траекторию и карту
|
||||
помещения. В репозиторий попадают только код, тесты, redacted manifests и
|
||||
безопасная документация.
|
||||
|
|
|
|||
|
|
@ -13,8 +13,12 @@ All 1,140 captured `lio_pcl` frames decoded as raw-LZ4 protobuf blocks, yielding
|
|||
4,165,862 points. All 1,215 `lio_pose` messages decoded, and the resulting
|
||||
approximately 1.566 m displacement matched the controlled movement. A later
|
||||
owner-operated LixelGO/iPhone capture also observed left/right RTSP/H.264 camera
|
||||
preview and the remote start/stop wire mapping. The remaining camera work is a
|
||||
bounded Mission Core receiver, not transport discovery.
|
||||
preview and the remote start/stop wire mapping. Mission Core now has a bounded
|
||||
read-only receiver, physical live left/right UI acceptance, acquisition-owned
|
||||
fMP4 archival and a generation-bound recorded player. The remaining camera gate
|
||||
is a newly recorded real K1 session played end-to-end with its point/pose
|
||||
timeline. Full-resolution/raw panorama, device-clock synchronization,
|
||||
intrinsics/extrinsics and remote delivery are still unproven.
|
||||
|
||||
The supplied Bible is useful as an OSINT dossier. It is not an executable plan
|
||||
for the actual stand because many experiments assume a phone and LixelGO. The
|
||||
|
|
|
|||
|
|
@ -3,7 +3,7 @@
|
|||
This plan supersedes the app-dependent experiment order in the reference Bible.
|
||||
Each gate produces evidence and an explicit GO, PAUSE or BLOCKED result.
|
||||
|
||||
## Live checkpoint — 2026-07-15
|
||||
## Current checkpoint — 2026-07-17
|
||||
|
||||
| Stage | Result |
|
||||
| --- | --- |
|
||||
|
|
@ -15,8 +15,11 @@ Each gate produces evidence and an explicit GO, PAUSE or BLOCKED result.
|
|||
| Stage 4 artifacts/flows | GO — bounded capture, hashes and negative control |
|
||||
| Stage 5 point cloud | GO — raw-LZ4 protobuf, 1,140 live frames decoded |
|
||||
| Stage 5 pose | GO — 1,215 live frames decoded and motion-correlated |
|
||||
| Stage 5 camera | GO (discovery) — left/right RTSP/H.264 preview observed; runtime adapter pending |
|
||||
| Stage 6 live viewer | GO — React console, Foxglove cloud/path and Mac latency metrics |
|
||||
| Stage 5 camera | GO (live) — left/right RTSP/H.264 preview observed, read-only runtime adapter and physical UI acceptance completed |
|
||||
| Stage 6 live viewer | GO — React Control Station, embedded self-hosted Rerun cloud/trajectory and Mac pipeline metrics |
|
||||
| Stage 7 observation archive | GO (point/pose) — durable catalog, recovery, background RRD preparation, saved-session timeline and atomic playback verified |
|
||||
| Stage 7 recorded cameras | GO (contract), acceptance pending — acquisition-owned fMP4 archive and player are implemented/tested; no real archived K1 camera session exists yet |
|
||||
| Stage 8 product storage | PAUSE — retention, replication, encryption, capacity monitoring and long-run browser/WASM stress remain deployment gates |
|
||||
|
||||
USB project copying remains optional ground truth rather than a blocker for the
|
||||
now-verified network path. Owner-operated LixelGO traffic verifies the MQTT
|
||||
|
|
@ -24,10 +27,14 @@ start/stop mapping and RTSP camera transport. MQTT control publishing remains
|
|||
deliberately deferred because complete request/save/rollback semantics are not
|
||||
yet modeled and the physical button remains a known-safe fallback.
|
||||
|
||||
The Stage 6 alpha uses a bounded raw-first bridge: loss in the visualization
|
||||
queue cannot discard MQTT evidence. Acceptance is replay of the full captured
|
||||
scan followed by a live ideal-LAN run with measured host pipeline latency.
|
||||
Sensor-to-display latency remains a separate clock-correlation test.
|
||||
The Stage 6 live path uses a bounded raw-first bridge: loss in the visualization
|
||||
queue cannot discard MQTT evidence. Stage 7 adds an independent durable
|
||||
observation-session lifecycle. Completed or recovered native captures are
|
||||
prepared once by a bounded backend worker into digest-bound RRD/cache-v6 and
|
||||
camera-manifest generations; a browser replay request never performs conversion.
|
||||
The saved scene, controller and timeline remain hidden until the complete RRD
|
||||
range and every declared camera pass admission. Sensor-to-display latency and
|
||||
camera/LiDAR sensor-clock alignment remain separate correlation tests.
|
||||
|
||||
## Stage 0 — repository and host baseline
|
||||
|
||||
|
|
@ -213,4 +220,6 @@ useful stream is decoded or structurally identified.
|
|||
- automated scan-button electronics;
|
||||
- OpenWrt/monitor-mode infrastructure;
|
||||
- firmware or internal-Linux analysis;
|
||||
- camera branch if no external frame stream is evidenced.
|
||||
- physical end-to-end replay acceptance for newly archived left/right cameras;
|
||||
- long-running large-session WebViewer/WASM memory telemetry;
|
||||
- production retention, replication, encryption and cross-platform packaging.
|
||||
|
|
|
|||
|
|
@ -1,9 +1,11 @@
|
|||
# K1 live console and embedded Rerun bridge
|
||||
|
||||
Status: the Rerun alpha milestone is implemented for the verified firmware-3
|
||||
MQTT streams. Live capture and replay use real K1 point-cloud and pose messages;
|
||||
the application does not generate a placeholder cloud, trajectory, camera frame
|
||||
or latency value.
|
||||
Status: the Rerun live and saved-session milestones are implemented for the
|
||||
verified firmware-3 point/pose streams. Live and adapter file-replay use the
|
||||
process-wide gRPC source; saved observation sessions use private digest-bound
|
||||
RRD over same-origin HTTP. Left/right RTSP preview is delivered separately from
|
||||
the spatial Rerun stream. The application does not generate placeholder cloud,
|
||||
trajectory, camera frame or latency values.
|
||||
|
||||
## Active device-to-scene path
|
||||
|
||||
|
|
@ -36,10 +38,12 @@ React console <-- REST + WebSocket state --> FastAPI on 127.0.0.1:8000
|
|||
```
|
||||
|
||||
The Paho MQTT callback does not decode or render the point cloud. It first
|
||||
writes and flushes the raw frame and metadata, then enqueues a preview message.
|
||||
If visualization cannot keep up, the oldest queued preview is discarded while
|
||||
the raw capture continues. Rerun work stays on the dedicated publisher thread
|
||||
and cannot block raw-first evidence capture.
|
||||
appends the raw frame, flushes it to the operating-system page cache and stages
|
||||
its aligned metadata, then enqueues a preview message. Raw bytes and metadata
|
||||
become a durable pair at the bounded group-commit boundary (at most 0.5 s,
|
||||
4 MiB or 32 messages), not at every callback. If visualization cannot keep up,
|
||||
the oldest queued preview is discarded while capture continues. Rerun work
|
||||
stays on the dedicated publisher thread and cannot block that raw-first path.
|
||||
|
||||
The current live/replay runtime instantiates only `RerunBridge`. The former
|
||||
Foxglove implementation is not a parallel runtime and does not listen on TCP
|
||||
|
|
@ -53,7 +57,7 @@ Prerequisites are the repository-local Python environment and Node.js 20.19+ or
|
|||
```bash
|
||||
uv sync --group dev
|
||||
cd apps/control-station
|
||||
npm install
|
||||
npm ci
|
||||
npm run typecheck
|
||||
npm run build
|
||||
cd ../..
|
||||
|
|
@ -112,18 +116,20 @@ directly and use their sibling metadata receive timestamps when present.
|
|||
9. Double-click K1 again to stop physical scanning, wait for steady green, then
|
||||
stop the local session so captures and summaries are finalized.
|
||||
|
||||
Each live run creates an ignored `sessions/<UTC>_viewer_live/` directory with a
|
||||
redacted manifest, operator notes, raw MQTT frames, per-message metadata and a
|
||||
hash summary. The connector subscribes to the fixed report-topic allowlist and
|
||||
does not publish an application request or modeling command.
|
||||
Each new live run creates a direct child below `MISSIONCORE_EVIDENCE_DIR`, or
|
||||
`.runtime/mission-core/evidence/sessions/` by default, with raw MQTT frames,
|
||||
per-message metadata and a hash summary. Repository-level
|
||||
`sessions/*_viewer_live` is legacy import-only evidence and is never selected by
|
||||
the current writer. The connector subscribes to the fixed report-topic
|
||||
allowlist and does not publish an application request or modeling command.
|
||||
|
||||
## Automatic Rerun source and lifecycle
|
||||
|
||||
On the first live or replay session, `RerunBridge`:
|
||||
On the first live or adapter file-replay session, `RerunBridge`:
|
||||
|
||||
- creates an explicit `RecordingStream("nodedc_mission_core_spatial")`;
|
||||
- installs the default spatial blueprint;
|
||||
- starts the gRPC/proxy server on TCP 9876 with a 512 MiB late-client buffer;
|
||||
- starts the gRPC/proxy server on TCP 9876 with a 32 MiB late-client buffer;
|
||||
- reports `rerun+http://127.0.0.1:9876/proxy` only after the server is ready;
|
||||
- accepts the local development and production browser origins used by this
|
||||
repository;
|
||||
|
|
@ -150,7 +156,7 @@ stream directly from the Rerun gRPC/proxy endpoint.
|
|||
| `/world` | `ViewCoordinates` | right-handed Z-up display convention |
|
||||
| `/world/points` | `Points3D` | decoded metric XYZ with computed or available RGB colors |
|
||||
| `/world/sensor_pose` | `Transform3D` + `TransformAxes3D` | current decoded translation, xyzw quaternion and pose axes |
|
||||
| `/world/trajectory` | `LineStrips3D` | bounded path of up to 20,000 decoded poses |
|
||||
| `/world/trajectory` | `LineStrips3D` | bounded path of up to 2,000 decoded poses |
|
||||
|
||||
Point count, frame rate, queue drops and measured pipeline time remain product
|
||||
metrics in the outer Control Station; they are not logged as Rerun entities and
|
||||
|
|
@ -189,9 +195,34 @@ shows only the current frame. RGB mode falls back to the verified scalar
|
|||
coloring path when the active point format has no RGB fields. The current
|
||||
`class`/custom behavior is a selected solid color, not semantic segmentation.
|
||||
|
||||
Projection switching, a custom playback timeline, semantic object/mask layers,
|
||||
camera frustums and persisted RBL/layout profiles are not wired yet and must not
|
||||
be inferred from the implemented controls above.
|
||||
Projection switching, semantic object/mask layers and camera frustums are not
|
||||
wired yet. The host-owned `observation.spatial` layout profile does persist the
|
||||
implemented scene controls, tool-window state, dynamic source visibility and
|
||||
normalized floating-window geometry. It is separate from Rerun's internal
|
||||
blueprint and from observation evidence.
|
||||
|
||||
## Saved observation sessions
|
||||
|
||||
Every completed native live run is indexed by the local host session store. A
|
||||
valid crash prefix without its final summary is indexed as `interrupted`. The
|
||||
**Сохранённые сессии** control shows the three newest runs and launches an
|
||||
opaque same-origin RRD rather than starting a second live bridge.
|
||||
|
||||
Sealing, recovery or legacy import makes the session eligible for the bounded
|
||||
backend preparation worker. The worker reads every native message once and
|
||||
atomically publishes a private cache-v6 RRD using the zero-based `session_time`
|
||||
duration timeline. A SHA sidecar binds the generation to native evidence.
|
||||
Replay-open never performs conversion: it returns HTTP 202 and a preparation
|
||||
handle or reuses the verified artifact. The embedded viewer opens the exact
|
||||
generation through Rerun's native incremental HTTP receiver. The bottom Control
|
||||
Station timeline then controls play/pause and seek directly; it does not
|
||||
approximate time with a React timer.
|
||||
|
||||
The native `.k1mqtt` remains the evidence master. RRD generation does not use
|
||||
the bounded live-preview queue, so it retains every frame accepted by the
|
||||
reviewed normalizer. See
|
||||
[`09_OBSERVATION_SESSIONS.md`](09_OBSERVATION_SESSIONS.md) for storage,
|
||||
recovery and HTTP Range details.
|
||||
|
||||
## Time and latency semantics
|
||||
|
||||
|
|
@ -228,7 +259,7 @@ and host firewall as appropriate, and never forward it directly to the public
|
|||
Internet or cellular WAN. Remote operation requires an authenticated TLS reverse
|
||||
proxy or another reviewed secure transport before deployment.
|
||||
|
||||
The 512 MiB Rerun server buffer limits retained late-client data but is not an
|
||||
The 32 MiB Rerun server buffer limits retained late-client data but is not an
|
||||
access-control mechanism. The listener and embedded viewer intentionally remain
|
||||
ready between acquisition sessions. Stop `k1link serve` when the network
|
||||
listener and its process memory must be closed unconditionally.
|
||||
|
|
@ -236,8 +267,10 @@ listener and its process memory must be closed unconditionally.
|
|||
## Current boundaries
|
||||
|
||||
- Raw panoramic camera frames were absent from the observed MQTT report topics.
|
||||
This milestone contains point cloud and pose/trajectory; operational metrics
|
||||
are rendered by the outer Control Station.
|
||||
The Rerun spatial recording contains point cloud and pose/trajectory;
|
||||
left/right compressed RTSP preview and acquisition-owned fMP4 archive use a
|
||||
separate generic media path. Historical sessions predating that archive have
|
||||
no recoverable video.
|
||||
- Physical double-click remains the K1 scan start/stop control. Any MQTT command
|
||||
publisher needs a separately reviewed state-changing profile.
|
||||
- No terrain map, elevation model, obstacle segmentation, localization fusion,
|
||||
|
|
|
|||
|
|
@ -17,6 +17,9 @@ already exist. The accepted device-lifecycle decision is recorded in
|
|||
| `src/k1link/protocol/` | Firmware-scoped K1 codecs and explicit vendor normalizer | Plugin-owned protocol adapter |
|
||||
| `src/k1link/data_plane/` | Transport-neutral decoded in-process consumer views | Local projections hydrated from portable SDK envelopes |
|
||||
| `src/k1link/viewer/` | Rerun consumer, visualization runtime and legacy Foxglove regression module | Replaceable canonical scene sink and presentation adapters |
|
||||
| `src/k1link/sessions/` | Host-owned SQLite catalog, evidence discovery/recovery, background preparation and derived-cache lifecycle | Device-neutral observation archive service |
|
||||
| `src/k1link/web/session_api.py` | Opaque saved-session, immutable RRD/media and workspace-layout API | Versioned Control/Edge observation contract |
|
||||
| `apps/control-station/src/core/observation/` | Session selection, replay admission, recorded-source and layout state | Device-neutral observation UI runtime |
|
||||
| `docs/domain-model/` | Versioned experimental Mission Core vocabulary | Promotion source for meanings proven across devices/services |
|
||||
|
||||
## Implemented boundary
|
||||
|
|
@ -33,6 +36,17 @@ raw K1 transport
|
|||
-> embedded local viewer
|
||||
```
|
||||
|
||||
The host-owned recorded path is separate from the disposable live preview:
|
||||
|
||||
```text
|
||||
sealed or recovered native observation session
|
||||
-> SQLite catalog and bounded single-worker preparation
|
||||
-> atomic RRD cache v6 + immutable recorded-media manifest v2
|
||||
-> generation-bound same-origin HTTP
|
||||
-> aggregate RRD/camera admission
|
||||
-> native Rerun HTTP receiver + recorded fMP4 player
|
||||
```
|
||||
|
||||
Rerun does not import K1 protocol code, inspect MQTT topics or receive raw
|
||||
payloads. `VisualizationRuntime` cannot choose a vendor decoder implicitly; its
|
||||
composition owner injects one. The local `Decoded*View` classes are in-process
|
||||
|
|
@ -52,6 +66,8 @@ The local vocabulary distinguishes:
|
|||
```text
|
||||
model != device != transport alias != device session
|
||||
acquisition != operation != source/channel != evidence
|
||||
observation session != preparation job != replay launch != viewer instance
|
||||
workspace layout != sensor evidence
|
||||
acknowledgement != completion
|
||||
raw evidence != decoded data != viewer state
|
||||
```
|
||||
|
|
@ -103,10 +119,14 @@ by physical double-click; no modeling request is published.
|
|||
Left/right camera preview transport, endpoint paths and H.264 framing are now
|
||||
observed under the exact compatibility profile. The local read-only adapter
|
||||
copy-remuxes one selected RTSP producer into bounded fMP4/WebSocket delivery for
|
||||
the generic MSE UI. Portable FFmpeg packaging, fan-out and remote delivery remain
|
||||
open. Device status and heartbeat remain raw observed channels without semantic
|
||||
decoders. Device calibration command and sensor-to-vehicle extrinsics are
|
||||
unavailable.
|
||||
the generic MSE UI. New acquisitions archive selected-camera init/segments/index
|
||||
independently of browser delivery. The generic recorded player, manifest-v2
|
||||
validation and shared `session_time` controls are connected; historical sessions
|
||||
created before this archive contract contain no recoverable video, so physical
|
||||
recorded-camera acceptance remains open. Portable FFmpeg packaging, disk-backed
|
||||
browser buffering, fan-out and remote delivery also remain open. Device status
|
||||
and heartbeat remain raw observed channels without semantic decoders. Device
|
||||
calibration command and sensor-to-vehicle extrinsics are unavailable.
|
||||
|
||||
The 2026-07-17 physical acceptance gate confirmed continuously updating point
|
||||
clouds, a matching live trajectory, and both camera selections in the same
|
||||
|
|
@ -128,14 +148,17 @@ for the measured gate and remaining limits.
|
|||
5. Replace compatibility routes and singleton state with multi-device session
|
||||
routing.
|
||||
6. Split Edge execution from the Control Station behind authenticated transport.
|
||||
7. Package the read-only RTSP/H.264 camera adapter for each target OS and evolve
|
||||
same-host MSE delivery toward an authenticated Edge media plane; keep the
|
||||
modeling-command publisher disabled until its separate safety gate closes.
|
||||
7. Physically accept a newly archived left/right K1 session, then package the
|
||||
read-only RTSP/H.264 adapter for each target OS, add disk-backed sealed media
|
||||
caching and evolve same-host MSE delivery toward an authenticated Edge media
|
||||
plane; keep the modeling-command publisher disabled until its separate safety
|
||||
gate closes.
|
||||
|
||||
## Invariants
|
||||
|
||||
- Raw evidence is persisted before preview work and is never overwritten by a
|
||||
decode result.
|
||||
- Raw bytes are appended and OS-flushed before preview work; aligned raw and
|
||||
metadata become durable together at the bounded group-commit boundary and
|
||||
are never overwritten by a decode result.
|
||||
- Mission Core starts when the XGRIDS manifest is absent. Formal installed
|
||||
plugin quarantine remains a future supervisor feature.
|
||||
- Core code does not branch on XGRIDS IDs, fields, topics or firmware.
|
||||
|
|
@ -160,8 +183,13 @@ for the measured gate and remaining limits.
|
|||
- complete SDK-envelope/EvidenceStore hot-path integration;
|
||||
- remote Edge split, authenticated WAN relay and fleet orchestration;
|
||||
- automatic K1 start/stop/calibration commands;
|
||||
- automatic camera capability discovery, synchronized recording/rewind,
|
||||
calibration, panoramic stitching and multi-consumer delivery.
|
||||
- device-reported camera capability discovery beyond the reviewed K1 profile;
|
||||
- physical acceptance of shared-timeline recorded-camera playback on a newly
|
||||
archived K1 session;
|
||||
- frame-accurate camera/LiDAR calibration, panoramic stitching, disk-backed
|
||||
browser media cache and remote multi-consumer delivery;
|
||||
- production retention, replication, encryption and long-run WebViewer/WASM
|
||||
memory acceptance for large observation sessions.
|
||||
|
||||
## Workspace-repeatable locked architecture gate
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,421 @@
|
|||
# Observation sessions, playback and workspace layout
|
||||
|
||||
Status: implemented for native K1 point/pose evidence and host-side camera
|
||||
archival. Recorded spatial playback is exposed in the Mission Core observation
|
||||
workspace. The camera archive/player contract is implemented and covered by
|
||||
tests, but all retained real K1 sessions predate canonical camera archival.
|
||||
Physical archived-camera playback therefore remains an open acceptance gate;
|
||||
historical sessions contain no recoverable video.
|
||||
|
||||
## Operator path
|
||||
|
||||
1. Start a normal acquisition from **Парк → Локальное устройство**.
|
||||
2. Open **Наблюдение → Пространственная сцена**. Live point cloud, trajectory
|
||||
and the selected camera remain live-only while acquisition is running.
|
||||
3. Stop acquisition normally, or allow the local service to recover an
|
||||
unexpected interruption on its next start. Session recording is automatic;
|
||||
the disk action is not required.
|
||||
4. Open **Сохранённые сессии** in the observation header. The menu shows the
|
||||
three most recent indexed runs, their date, duration, state, modalities and
|
||||
background preparation state.
|
||||
5. Choose a replayable run. Opening a replay never performs conversion in the
|
||||
request. A ready recording opens immediately; otherwise the client receives
|
||||
HTTP 202, keeps the current scene mounted and polls the preparation status.
|
||||
Only after the server returns a verified launch descriptor does Mission Core
|
||||
pause and unload the previous viewer.
|
||||
6. The client then receives and decodes the complete RRD. The new viewport and
|
||||
timeline stay hidden until the declared recording range is fully buffered
|
||||
(`fullyBuffered`). Playback starts once at that point. Partial frames are
|
||||
never shown. If the run contains canonical camera archives, its recorded
|
||||
camera sources replace all live device overlays.
|
||||
7. Use **Воспроизвести / Пауза**, the scrubber, **К началу** and **К концу** on
|
||||
the right half of the bottom timeline. The left half changes point-cloud
|
||||
accumulation. The recorded timeline is zero-based `session_time`.
|
||||
8. Use the first disk button to save the current workspace layout. The next
|
||||
opening restores display settings, tool windows and dynamic source-window
|
||||
positions. This action saves no sensor evidence.
|
||||
|
||||
Display controls have no separate Apply/Reset transaction. Boolean controls
|
||||
commit immediately; sliders, colors and selects commit on release/blur or after
|
||||
a short quiet period, and concurrent commits collapse to the newest value. A
|
||||
layout save flushes and awaits this queue before serializing the confirmed
|
||||
settings. Recorded display updates reuse stable Rerun scene identifiers and do
|
||||
not mutate playback state, so changing accumulation, size, color or visibility
|
||||
does not pause the active replay or replace its camera view.
|
||||
|
||||
Completed and recovered sessions are discovered on startup and by the catalog
|
||||
reconciler, then deduplicated into one bounded, single-worker preparation queue.
|
||||
The worker validates the native source, exports point/pose data, verifies source
|
||||
stability and both SHA-256 digests, finalizes once, and atomically publishes the
|
||||
digest-bound derived RRD plus its cache sidecar. Before the same job becomes
|
||||
`ready`, it also parses and validates every archived camera index and retains
|
||||
the immutable manifest generation in memory. The catalog exposes preparation
|
||||
state and progress while this runs; a recording URL and camera descriptors are
|
||||
returned only after the complete launch generation has passed validation.
|
||||
|
||||
Preparation is a backend lifecycle, not a viewer action: sealing or recovering
|
||||
a native session makes it eligible for the reconciler, which builds the derived
|
||||
RRD once in the background. Reopening, seeking or changing the workspace reads
|
||||
that prepared artifact and never reruns conversion. A cold process may validate
|
||||
an existing cache and rebuild its in-memory camera manifest in the worker, but
|
||||
no replay, status, RRD or manifest request hashes native evidence or parses a
|
||||
camera index synchronously.
|
||||
|
||||
The session menu uses three operator indicators:
|
||||
|
||||
| Indicator | Catalog/preparation state | Meaning |
|
||||
| --- | --- | --- |
|
||||
| Solid green + `Готово` | `ready` | Verified RRD is available. |
|
||||
| Pulsing green + `Обработка` | `queued`, `validating`, `exporting`, `finalizing` | Background preparation is active. |
|
||||
| Dim gray + `Ошибка` | failed, cancelled, non-replayable or invalid session | No launchable recording; inspect the message or retry. |
|
||||
|
||||
Saved-session status never uses yellow or red. Switching sessions aborts only
|
||||
the obsolete browser poll; it does not cancel the process-owned conversion.
|
||||
A queued job may legitimately wait behind another export and therefore uses the
|
||||
overall preparation deadline rather than the active-export heartbeat timeout.
|
||||
|
||||
The first preparation of a long capture can take time because every decodable
|
||||
point/pose message is projected into RRD. Later openings reuse a verified,
|
||||
digest-bound cache. Derived RRD cache v6 is intentionally incompatible with v4
|
||||
and v5 because only v6 guarantees that the payload itself contains a real
|
||||
`session_time = 0` anchor; older generations are rebuilt once in the background.
|
||||
The browser may use the strict `source_url` with `If-Match`, while the embedded
|
||||
Rerun loader uses the canonical `viewer_source_url` whose lowercase SHA-256
|
||||
`generation` query is bound to the same launch descriptor. A missing or stale
|
||||
generation fails with `412`; a matching URL is served with exact length, strong
|
||||
ETag and private immutable/no-transform caching. Every cache pin is released when the
|
||||
HTTP response completes, disconnects or fails, so an interrupted switch cannot
|
||||
make a derived recording permanently non-evictable. A bounded 120-second launch
|
||||
reservation also protects the artifact between the verified launch response and
|
||||
the WebViewer's subsequent RRD request, including a cold WASM startup; it
|
||||
expires automatically if that request never arrives.
|
||||
|
||||
## Storage roots
|
||||
|
||||
By default, both catalog state and new source evidence are private to the
|
||||
checkout:
|
||||
|
||||
```text
|
||||
.runtime/mission-core/
|
||||
├── mission-core.sqlite3
|
||||
├── mission-core.sqlite3-shm
|
||||
├── mission-core.sqlite3-wal
|
||||
├── evidence/
|
||||
│ └── sessions/
|
||||
│ ├── .current_session # present only while a writer owns the root
|
||||
│ └── <UTC>_viewer_live/
|
||||
│ ├── captures/ # native MQTT source evidence
|
||||
│ └── media/ # canonical camera archives
|
||||
└── recordings/
|
||||
├── .export.lock # cross-process conversion/eviction lock
|
||||
└── <opaque-session-id>/
|
||||
├── scene.rrd
|
||||
└── scene.rrd.cache.json
|
||||
```
|
||||
|
||||
`MISSIONCORE_DATA_DIR` relocates the catalog and derived cache. Unless it is
|
||||
overridden separately, new evidence is written to
|
||||
`$MISSIONCORE_DATA_DIR/evidence/sessions`. `MISSIONCORE_EVIDENCE_DIR` can select
|
||||
another absolute MQTT evidence root. In the current K1 integration the camera
|
||||
gateway additionally confines its recording directory to the repository
|
||||
checkout, so a full point-plus-camera acquisition must keep the evidence root
|
||||
inside that checkout. An external evidence volume currently supports MQTT
|
||||
capture/cataloging but camera archival will fail closed until the gateway gets
|
||||
a separately attested storage root:
|
||||
|
||||
```bash
|
||||
export MISSIONCORE_DATA_DIR=/absolute/private/path/mission-core
|
||||
# Full K1 point-plus-camera evidence must currently remain below the checkout.
|
||||
export MISSIONCORE_EVIDENCE_DIR=/absolute/path/to/NODEDC_MISSION_CORE/.runtime/mission-core/evidence/sessions
|
||||
export MISSIONCORE_RRD_CACHE_MAX_BYTES=8589934592
|
||||
export MISSIONCORE_RRD_FREE_SPACE_RESERVE_BYTES=2147483648
|
||||
uv run k1link serve
|
||||
```
|
||||
|
||||
The directories and derived recording cache use owner-only permissions where
|
||||
the host filesystem permits them. A new acquisition writer is assigned only a
|
||||
direct child of the private evidence root; it no longer writes a new run to the
|
||||
repository-level `sessions/` directory.
|
||||
|
||||
Repository-level `sessions/*_viewer_live` runs are a legacy, import-only source
|
||||
for the catalog. They are discovered and confined in place: refresh does not
|
||||
copy, move or migrate their large payloads, and no current writer is assigned
|
||||
that root. Startup may recovery-seal an already existing, incomplete canonical
|
||||
camera epoch there by preserving its valid segment prefix and writing an
|
||||
`interrupted` summary; this is evidence recovery, not a new acquisition write.
|
||||
|
||||
Never add `.runtime/`, `sessions/`, raw captures, RRD files or camera media to
|
||||
Git. They can contain mapped interiors, trajectories and identifiable images.
|
||||
|
||||
## Session source of record
|
||||
|
||||
For the current K1 profile:
|
||||
|
||||
```text
|
||||
MQTT callback
|
||||
├─ durable native .k1mqtt + aligned metadata (source of record)
|
||||
└─ bounded latest-wins Rerun live preview (disposable)
|
||||
|
||||
selected RTSP producer
|
||||
├─ durable init + fMP4 segments + JSONL index (camera evidence)
|
||||
└─ bounded WebSocket/MSE preview (disposable)
|
||||
```
|
||||
|
||||
The live preview is intentionally allowed to drop frames under load. Native
|
||||
point/pose evidence and camera archive writes do not traverse that queue.
|
||||
|
||||
Native `.k1mqtt` bytes with aligned metadata and canonical camera fMP4 archives
|
||||
are the evidence source of truth. The derived RRD contains every decodable point
|
||||
and pose frame from that source, but remains a rebuildable view rather than an
|
||||
evidence master. A cache entry is rebuilt only when native source
|
||||
identity/digests change or an incompatible derived-data export revision is
|
||||
introduced. A UI blueprint or workspace-layout revision never invalidates or
|
||||
rewrites the data RRD. Cache v4/v5 payloads are not reusable as v6 because they
|
||||
do not guarantee the real zero-time anchor.
|
||||
|
||||
Expensive cache misses run through the bounded preparation worker and one global
|
||||
cross-process export gate to cap peak RAM, CPU and temporary-disk use. Crash
|
||||
leftovers from candidates, exporter temporary files and staged replay prefixes
|
||||
are scavenged under that lock before quota accounting. Ready cache hits and
|
||||
active response leases do not wait behind that gate. The derived cache has an
|
||||
8 GiB default quota, preserves a 2 GiB default filesystem reserve and evicts
|
||||
least-recently-used RRDs only; it never deletes native evidence.
|
||||
|
||||
## Camera archive contract
|
||||
|
||||
New acquisitions archive each selected source and codec epoch below the same
|
||||
private evidence session:
|
||||
|
||||
```text
|
||||
<evidence-root>/sessions/<session-id>/media/<stable-source-id>/epoch-1/
|
||||
├── init.mp4
|
||||
├── segments/
|
||||
│ ├── 1.m4s
|
||||
│ └── ...
|
||||
├── index.jsonl
|
||||
└── summary.json
|
||||
```
|
||||
|
||||
Each index row binds a segment sequence, byte length, SHA-256 and host arrival
|
||||
epoch/monotonic timestamps. Camera durability uses a
|
||||
`per-segment-fsync` policy: `init.mp4` and every complete media segment are
|
||||
individually fsynced and their directory entries synchronized before the
|
||||
matching JSONL index row is committed. The index and an atomic `interrupted`
|
||||
checkpoint summary are fsynced before the append returns. The old interval and
|
||||
byte constructor options are compatibility-only and cannot weaken this
|
||||
segment-bound RPO. A power failure may still lose or leave uncommitted the
|
||||
fragment currently being produced; it cannot make a committed index row point
|
||||
past durable media. Camera windows may close or reconnect without terminating
|
||||
archival while the owning acquisition remains active.
|
||||
|
||||
Synchronization is `host-arrival-best-effort`: LiDAR/MQTT and camera segments
|
||||
share the Mac host clock boundary, but K1 sensor exposure time and LiDAR firing
|
||||
time are not proven to use a shared device clock. Do not infer frame-accurate
|
||||
calibration from the playback timeline.
|
||||
|
||||
Finalized media is prepared once by the same process-owned background job that
|
||||
materializes the RRD. The gateway records host time only after a complete
|
||||
`moof+mdat` fragment has arrived, so that timestamp is an availability/end
|
||||
anchor, never a fragment-start timestamp. Preparation reads and SHA-verifies
|
||||
every fragment, parses bounded ISO-BMFF timing tables (`mdhd`, `trex`, `tfhd`,
|
||||
`trun`), anchors the epoch at
|
||||
`max(0, first_arrival - first_fragment_duration)`, and sets its end to that start
|
||||
plus the checked sum of every decoded fragment duration. The declared interval
|
||||
therefore has exactly the duration MSE is expected to expose and is not stretched
|
||||
by host scheduling jitter.
|
||||
|
||||
Epoch arrivals must be strictly monotonic; epoch intervals must be finite,
|
||||
monotonic and non-overlapping. Replay v2 also requires every media interval to
|
||||
fit inside the spatial RRD interval with a 50 ms numeric tolerance; it fails
|
||||
preparation instead of clamping unreachable evidence. A future replay v3 must
|
||||
separate `spatial_range` from a session-wide union range before out-of-RRD camera
|
||||
coverage can be navigated. A missing, ambiguous, oversized or otherwise
|
||||
unparseable timing table fails preparation; the archive remains evidence but is
|
||||
never advertised as seekable media.
|
||||
|
||||
The prepared path-free v2 descriptor and full source stat identity (native raw
|
||||
and timing metadata plus every camera summary, index, init and segment file) are
|
||||
written under the private derived cache with a schema, generation and checksum.
|
||||
Publication uses a private temporary file, file and directory `fsync`, and atomic
|
||||
rename; startup scavenges crash-left temporary files. A restart reuses this
|
||||
sidecar after confined O(n) stat validation, without rereading, hashing or parsing
|
||||
media. A missing, corrupt or stale sidecar is rebuilt only by the background
|
||||
worker. No replay, status, manifest or payload request performs conversion or
|
||||
recalculates these intervals.
|
||||
|
||||
Every derived RRD contains a real `session_time = 0` row at the internal
|
||||
`/__mission_core/session_origin` entity. The anchor is deliberately outside
|
||||
`/world`, so it establishes the actual recording time range without creating a
|
||||
3D layer or drawable scene object. Preparation verifies the derived recording
|
||||
against the declared spatial range rather than relying on summary metadata
|
||||
alone.
|
||||
|
||||
Sessions made before this archive contract have no recoverable video even if a
|
||||
camera preview was visible at the time. In particular, the 2026-07-16 browser
|
||||
camera acceptance run retained point/pose evidence only.
|
||||
|
||||
## Crash and interruption behavior
|
||||
|
||||
- SQLite uses WAL, foreign keys and full synchronous durability.
|
||||
- Before creating a session directory, the acquisition takes an exclusive,
|
||||
cross-process lease by atomically creating and locking
|
||||
`<evidence-root>/.current_session`. The marker contains only the direct child
|
||||
session name. While the lock is held, a second writer fails closed and
|
||||
discovery excludes that in-progress session from replay.
|
||||
- A process exit releases the operating-system lock. On the next service start,
|
||||
stale-marker recovery removes the marker only after it can take the lock and
|
||||
revalidate the marker inode without following symlinks. It never deletes the
|
||||
interrupted session directory; normal catalog recovery then evaluates the
|
||||
durable prefix. A locked/live or suspicious marker is left untouched.
|
||||
- Native MQTT writes use a bounded group commit: at most 0.5 seconds, 4 MiB or
|
||||
32 messages per group. Raw bytes are fsynced before their metadata rows are
|
||||
written and fsynced. This is a bounded RPO, not a zero-loss guarantee: a hard
|
||||
process or power failure may discard the final uncommitted group. Raw bytes
|
||||
from an interrupted commit window may survive beyond the durable metadata;
|
||||
replay uses only the last validated metadata-aligned raw boundary.
|
||||
- A native capture with a missing final summary is accepted only when the raw
|
||||
and metadata prefix is bounded, aligned and structurally valid. It is cataloged
|
||||
as `interrupted`, never silently promoted to `ready`.
|
||||
- A non-newline metadata crash tail can be ignored. Newline-terminated or
|
||||
mid-file corruption fails closed.
|
||||
- Camera recovery retains a contiguous valid segment prefix, quarantines
|
||||
non-contiguous/orphan fragments instead of deleting them, rebuilds the index
|
||||
when necessary and writes an `interrupted` summary. Unindexed bytes are not
|
||||
presented as valid media.
|
||||
- If a valid normal summary appears later, repeat discovery updates the same
|
||||
session to `ready`.
|
||||
- Materialization detects a native capture changing during export and refuses
|
||||
to publish the derived RRD. It reopens the prepared raw source with no symlink
|
||||
following inside the cataloged session roots, and interrupted recordings are
|
||||
materialized from the last validated raw/metadata boundary only.
|
||||
- Internal preparation cancellation is cooperative. A queued job cancels
|
||||
immediately; an active job stops at a safe checkpoint, removes its candidate
|
||||
and never publishes a partial RRD. The operator UI does not cancel automatic
|
||||
preparation when switching sessions or closing a tab: preparation belongs to
|
||||
the application worker, not to an HTTP request.
|
||||
- `failed` and `cancelled` are terminal, retryable states. The API exposes a
|
||||
sanitized error, and an explicit retry creates a fresh job for the current
|
||||
source identity. A stalled poll or failed switch leaves the current scene
|
||||
mounted; the latest operator selection wins over obsolete responses.
|
||||
- A lifecycle shutdown marks only its own interrupted work for automatic
|
||||
reconciliation after restart. An operator cancellation and a genuine failed
|
||||
export remain terminal and are never retried forever by the reconciler.
|
||||
|
||||
These rules provide crash recovery, not replication. A single host disk failure
|
||||
can still destroy local data. Vehicle deployment must add independent onboard
|
||||
and control-station copies, capacity monitoring and a documented retention
|
||||
policy.
|
||||
|
||||
## HTTP API
|
||||
|
||||
All paths are same-origin and expose opaque identifiers only:
|
||||
|
||||
```text
|
||||
GET /api/v1/observation-sessions?limit=3
|
||||
GET /api/v1/observation-sessions/{id}
|
||||
POST /api/v1/observation-sessions/{id}/replay
|
||||
GET /api/v1/observation-sessions/{id}/recording-preparation
|
||||
DELETE /api/v1/observation-sessions/{id}/recording-preparation
|
||||
GET /api/v1/observation-sessions/{id}/recording.rrd
|
||||
POST /api/v1/observation-sessions/{id}/blueprint.rrd
|
||||
GET /api/v1/observation-sessions/{id}/media/{artifact}/manifest
|
||||
GET /api/v1/observation-sessions/{id}/media/{artifact}/epochs/{n}/init.mp4
|
||||
GET /api/v1/observation-sessions/{id}/media/{artifact}/epochs/{n}/segments/{m}.m4s
|
||||
|
||||
GET /api/v1/workspace-layouts/observation.spatial
|
||||
PUT /api/v1/workspace-layouts/observation.spatial
|
||||
```
|
||||
|
||||
The catalog embeds each replayable session's preparation state and progress.
|
||||
`POST .../replay` returns either a verified replay v2 launch document or HTTP
|
||||
202 with the preparation v1 document, `Location`, `Retry-After`, an exact quoted
|
||||
preparation `ETag` and a same-origin status URL. Polling
|
||||
`GET .../recording-preparation` sends that value as `If-Match` and returns HTTP
|
||||
202 while the job is active, HTTP 409 for retryable `failed`/`cancelled` states,
|
||||
and the launch document only when the same job and artifact are ready; every
|
||||
response echoes the same `ETag`. `DELETE` also requires `If-Match`, so a stale
|
||||
tab cannot cancel a replacement job. Conversion is never executed synchronously
|
||||
by a replay-open request. Playback speed and loop are request-local launch
|
||||
policy; they are not stored on or shared through a preparation job.
|
||||
|
||||
Production `recording.rrd` access is bound to the launch generation. The client
|
||||
can use the strict query-free URL with exact strong
|
||||
`If-Match: "sha256:<launch.sha256>"`; a request with neither an `If-Match` nor a
|
||||
generation returns 428. The embedded Rerun receiver instead gets only the
|
||||
canonical `viewer_source_url = source_url + "?generation=<launch.sha256>"`.
|
||||
The server returns 412 for a malformed or replaced generation before opening
|
||||
the body. A successful response echoes the strong ETag, exact `Content-Length`,
|
||||
`application/vnd.rerun.rrd` and immutable private/no-transform cache policy.
|
||||
Rerun's native HTTP receiver owns incremental decoding; `LogChannel.send_rrd`
|
||||
is reserved for independently complete RRD payloads such as the small generated
|
||||
blueprint and must never receive arbitrary HTTP byte slices. The canvas,
|
||||
timeline, controller and autoplay remain closed until the decoded spatial range
|
||||
exactly matches the launch descriptor and every declared camera is ready. API
|
||||
documents never contain local paths. Layout updates require the quoted current
|
||||
revision in `If-Match`; stale writers receive HTTP 412 rather than overwriting
|
||||
another saved profile.
|
||||
|
||||
Recorded media routes expose only opaque catalog identifiers and ordinal codec
|
||||
epochs. The required `missioncore.observation-recorded-media/v2` manifest carries
|
||||
a strong `generation_sha256`, exact JS-safe aggregate `byte_length`, and finite
|
||||
`timeline_start_seconds` / `timeline_end_seconds` for every epoch. Epoch ends
|
||||
participate in the generation digest. Every epoch also declares its init byte
|
||||
length/digest and a complete contiguous segment list with sequence, URL, byte
|
||||
length and digest. The launch source repeats the same aggregate `byte_length`
|
||||
and uses exactly `max(epoch.timeline_end_seconds)` as its end; the spatial RRD
|
||||
end must never pad camera coverage. The browser cross-checks launch, manifest and
|
||||
component lengths. The current browser laboratory policy accepts at most 16
|
||||
camera sources, 128 MiB per source and 512 MiB across the session. This launch
|
||||
preflight finishes before the first camera manifest/init/segment GET; admitted
|
||||
cameras are then downloaded and decode-probed one at a time. Verified raw
|
||||
buffers remain immutable so route changes and player remounts cannot append
|
||||
emptied data. This bounded in-memory strategy is deliberate for the laboratory
|
||||
milestone; an OPFS-backed sealed-generation cache is the next scaling step for
|
||||
larger rigs and must preserve the same launch/manifest/hash admission contract.
|
||||
|
||||
The manifest response ETag is the exact generation, and the manifest GET itself
|
||||
requires that generation in `If-Match`. Init and media GETs likewise require the
|
||||
descriptor's exact SHA `If-Match`, then open through a confined directory file
|
||||
descriptor with no symlink following, verify the declared SHA-256 and serve
|
||||
exact-length bytes with immutable private `no-transform` caching and byte-range
|
||||
support. Physical source ids, RTSP addresses and storage paths never cross the
|
||||
API boundary.
|
||||
|
||||
## Current limit
|
||||
|
||||
The spatial RRD, trajectory and archived fMP4 cameras use the same operator
|
||||
scrubber now. Camera epochs are aligned to zero-based `session_time` from the
|
||||
shared host-arrival monotonic clock and rendered through MSE. The client selects
|
||||
an epoch only inside its declared inclusive interval and verifies that the
|
||||
decoded MSE seekable duration covers that interval. This remains best-effort
|
||||
correlation: codec PTS and K1 sensor exposure time are not proven to share the
|
||||
LiDAR clock. A codec epoch whose init segment does not expose a browser-supported
|
||||
codec or whose timing cannot be proven remains retained evidence and fails
|
||||
closed in the UI/background preparation.
|
||||
Historical sessions with no canonical camera archive honestly show no recorded
|
||||
video. Recorded blueprints can change accumulation, grid visibility, point and
|
||||
trajectory visibility, point radius and a uniform custom point color without
|
||||
rewriting the recording. The client deliberately has no progressive recorded
|
||||
mode: the viewport, timeline and autoplay gate remain closed until the complete
|
||||
declared RRD has been received and decoded. Height, intensity, distance and RGB
|
||||
palettes remain baked into current RRD rows; fully dynamic recoloring requires
|
||||
exporting the corresponding scalar components in a future recording schema.
|
||||
|
||||
## Verification checkpoint — 2026-07-17
|
||||
|
||||
The repository state described above passed the complete local pre-push gate:
|
||||
|
||||
- `uv sync --frozen --group dev` completed against the locked Python graph;
|
||||
- `uv run pytest`: **284 passed**;
|
||||
- `uv run ruff check .`: clean;
|
||||
- project mypy and strict plugin-SDK mypy: clean across 51 and 12 source files;
|
||||
- the XGRIDS K1 profile loader and emitted v0alpha2 JSON schema validated;
|
||||
- clean `npm ci` ran the required Rerun 0.34.1 postinstall patch;
|
||||
- frontend unit suite: **111 passed, 0 failed**;
|
||||
- TypeScript project build and Vite production build completed;
|
||||
- the generated `dist/index.html` and non-empty Rerun WASM artifact were verified.
|
||||
|
||||
Vite still reports its expected large-chunk warning for the embedded Rerun
|
||||
viewer/WASM payload. That is a packaging optimization item, not a failed gate.
|
||||
No retained physical K1 session contains the new canonical camera archive yet,
|
||||
so real recorded-camera playback remains an explicit hardware acceptance test.
|
||||
|
|
@ -2,6 +2,11 @@
|
|||
|
||||
Status: accepted for the experimental Mission Core runtime, 2026-07-16.
|
||||
|
||||
Scope update, 2026-07-17: the live device descriptors in this ADR remain
|
||||
live-only. ADR 0008 adds a separate host-owned recorded-source catalog,
|
||||
seekable `session_time` playback and workspace layout; it does not retroactively
|
||||
turn a live RTSP lease into recorded evidence.
|
||||
|
||||
## Context
|
||||
|
||||
The spatial scene and camera workspace must accept one device, a custom rig or a
|
||||
|
|
@ -68,10 +73,12 @@ RTSP/FFmpeg generation before a new browser lease is issued.
|
|||
|
||||
The local laboratory adapter now terminates RTSP/RTP with FFmpeg, copy-remuxes
|
||||
H.264 High L4 without transcoding and publishes complete bounded fMP4 segments
|
||||
to a generic MSE player. Portable FFmpeg packaging, multi-consumer fan-out and
|
||||
remote/WAN delivery are not claimed by this milestone. Synchronized rewind
|
||||
becomes eligible only after persisted buffering and an evidenced mapping between
|
||||
the RTP 90 kHz clock and the spatial stream clock. See ADR 0007.
|
||||
to a generic MSE player. ADR 0008 subsequently added acquisition-owned durable
|
||||
segments, a prepared recorded-media manifest and host-arrival-best-effort rewind
|
||||
on the saved-session timeline. This is not proof of a shared device clock or
|
||||
frame-accurate LiDAR/camera alignment. Portable FFmpeg packaging,
|
||||
multi-consumer fan-out and remote/WAN delivery remain open. See ADR 0007 and
|
||||
ADR 0008.
|
||||
|
||||
## Code anchors
|
||||
|
||||
|
|
|
|||
|
|
@ -2,6 +2,11 @@
|
|||
|
||||
Status: accepted for the local laboratory runtime, 2026-07-16.
|
||||
|
||||
Supersession note, 2026-07-17: the live copy-remux and lease decisions remain
|
||||
active. ADR 0008 extends the same acquisition-owned producer with durable fMP4
|
||||
epochs and a generic recorded player. That extension does not weaken the
|
||||
single-live-producer or opaque transport boundary defined here.
|
||||
|
||||
## Context
|
||||
|
||||
The exact K1 firmware 3.0.2 profile exposes two RTSP/TCP H.264 preview paths.
|
||||
|
|
@ -73,16 +78,22 @@ timeline once when the recording opens and does not poll or force the cursor.
|
|||
- Same-origin local WebSocket delivery is not the future WAN transport. A later
|
||||
Edge/Control Station split should use an authenticated WebRTC/WHEP or equivalent
|
||||
congestion-aware media plane behind the same descriptor contract.
|
||||
- Camera recording, synchronized rewind, calibration, frustums, panoramic
|
||||
stitching and full-resolution raw imagery remain unavailable.
|
||||
- Durable camera recording and host-arrival-best-effort saved-session rewind are
|
||||
implemented under ADR 0008, but no retained real K1 session made after that
|
||||
archive contract has completed physical recorded-camera acceptance yet.
|
||||
- Device-clock synchronization, calibration, frustums, panoramic stitching,
|
||||
full-resolution raw imagery and remote multi-consumer delivery remain
|
||||
unavailable.
|
||||
- A longer physical soak test is still required to characterize sustained
|
||||
preview backpressure and rendering cost on target Edge hardware.
|
||||
|
||||
## Code anchors
|
||||
|
||||
- `src/k1link/web/xgrids_k1_camera.py`
|
||||
- `src/k1link/web/camera_archive.py`
|
||||
- `src/k1link/web/xgrids_k1_facade.py`
|
||||
- `apps/control-station/src/components/MseFmp4WebSocketPlayer.tsx`
|
||||
- `apps/control-station/src/components/RecordedFmp4Player.tsx`
|
||||
- `apps/control-station/src/core/observation/useObservationLayout.ts`
|
||||
- `apps/control-station/src/device-plugins/xgrids-k1/observationSources.ts`
|
||||
- `tests/test_xgrids_camera_gateway.py`
|
||||
|
|
|
|||
|
|
@ -0,0 +1,230 @@
|
|||
# ADR 0008: durable observation sessions and workspace layout
|
||||
|
||||
- Status: accepted for the laboratory Control Station
|
||||
- Date: 2026-07-17
|
||||
- Scope: host observation storage, recorded spatial playback, camera evidence,
|
||||
and operator workspace layout
|
||||
|
||||
## Context
|
||||
|
||||
Mission Core previously retained K1 live point/pose captures as ignored
|
||||
laboratory directories, while the browser showed only the current process-wide
|
||||
Rerun stream. Camera preview was copy-remuxed for a connected browser but was
|
||||
not part of the durable session. A browser refresh, local service restart or an
|
||||
unexpected process exit therefore made a completed run difficult to discover
|
||||
and made camera history impossible to recover. The disk icon in the spatial
|
||||
workspace also had no stable distinction between saving sensor evidence and
|
||||
saving an operator's layout.
|
||||
|
||||
These are two separate product records:
|
||||
|
||||
1. an **observation session** is evidence from all sources active during one
|
||||
acquisition and must be recorded automatically; and
|
||||
2. a **workspace layout profile** is an operator preference and is saved
|
||||
explicitly.
|
||||
|
||||
Conflating them would allow a UI action to decide whether evidence exists and
|
||||
would make a layout save duplicate or mutate sensor data.
|
||||
|
||||
## Decision
|
||||
|
||||
### Host-owned catalog and private evidence root
|
||||
|
||||
The Control Station owns a private storage root selected by
|
||||
`MISSIONCORE_DATA_DIR`; the repository-local fallback is
|
||||
`.runtime/mission-core/`. The root is outside Git and contains a SQLite catalog
|
||||
plus derived playback artifacts. New source evidence is written to the root
|
||||
selected by `MISSIONCORE_EVIDENCE_DIR`, falling back to
|
||||
`<MISSIONCORE_DATA_DIR>/evidence/sessions`. The current K1 camera gateway still
|
||||
requires that root to remain inside the repository checkout; an external root
|
||||
can retain MQTT evidence but camera archival fails closed. Treat support for an
|
||||
independently attested external camera-evidence root as an open extraction item,
|
||||
not as implemented behavior. SQLite uses foreign keys, WAL journaling and full
|
||||
synchronous durability. Public APIs expose opaque session identifiers and
|
||||
metadata, never filesystem paths.
|
||||
|
||||
Current acquisition writers create only direct children of the private
|
||||
evidence root. Existing repository-level `sessions/*_viewer_live` runs are a
|
||||
legacy import-only source: they are indexed in place, never copied or moved,
|
||||
and are not assigned to a new writer. Their native `.k1mqtt` capture remains
|
||||
the source of record. Import is repeatable and runs in a background reconciler,
|
||||
so completed evidence becomes visible without a restart and HTTP catalog reads
|
||||
never scan the evidence roots. Startup recovery may
|
||||
seal an already incomplete canonical camera epoch under the legacy root; it
|
||||
does not start or append a new acquisition there.
|
||||
|
||||
Before a new session directory is created, the writer atomically creates and
|
||||
locks `<evidence-root>/.current_session`. The marker names the direct child and
|
||||
prevents a second process from writing or replaying the active session. Process
|
||||
exit releases the operating-system lock. On startup, an unlocked stale marker
|
||||
is removed only after no-follow inode revalidation; the interrupted evidence
|
||||
directory is preserved for normal recovery and cataloging.
|
||||
|
||||
If a process exits after raw and metadata creation but before a final summary,
|
||||
the importer accepts only a bounded, structurally valid prefix with aligned
|
||||
metadata and marks the run `interrupted`. It does not invent completion data.
|
||||
When the valid final summary later appears, repeat import promotes the same
|
||||
session to `ready`.
|
||||
|
||||
Native MQTT persistence uses a bounded group commit of at most 0.5 seconds,
|
||||
4 MiB or 32 messages. Raw bytes are fsynced before their aligned metadata rows.
|
||||
The final uncommitted group may be lost, and a raw tail may survive without
|
||||
committed metadata; replay stops at the last validated aligned boundary.
|
||||
|
||||
### Recorded spatial playback
|
||||
|
||||
When a native capture is normally sealed or recovery-sealed, a bounded
|
||||
single-worker reconciler automatically prepares its private derived `.rrd`.
|
||||
Preparation is a backend lifecycle and is never executed by an HTTP replay
|
||||
request. Reopening, seeking, switching tabs or changing display settings reads
|
||||
the already prepared artifact and does not rerun conversion. Export is lossless
|
||||
with respect to every decodable point and pose message in the native capture;
|
||||
it does not pass through the bounded live-preview queue. The RRD uses the
|
||||
recording-local `session_time` duration timeline, while the receive wall clock
|
||||
remains secondary evidence.
|
||||
|
||||
Derived files are published atomically, hashed and accompanied by a cache
|
||||
sidecar bound to the native file identity and digest. A process-owned queue and
|
||||
cross-process file lock prevent overlapping exporters; crash candidates are
|
||||
scavenged before quota accounting. A stale or inconsistent cache is rejected
|
||||
and rebuilt. The browser receives the RRD only through the same-origin opaque
|
||||
session API. It keeps the viewport and timeline hidden until the complete
|
||||
declared recording has been downloaded and decoded, then publishes the scene
|
||||
atomically and starts playback.
|
||||
|
||||
### Camera recording
|
||||
|
||||
Camera acquisition is owned by the device acquisition session, not by a
|
||||
browser window. Preview remains bounded and disposable; archive writes occur
|
||||
before preview fan-out. Closing or refreshing a camera window must not stop the
|
||||
archive while acquisition is active.
|
||||
|
||||
Each codec epoch is stored below the observation session as an initialization
|
||||
segment, independently addressable fMP4 media segments, an append-only JSONL
|
||||
index and an atomic summary. Host arrival epoch and monotonic timestamps are
|
||||
recorded for every segment. The `per-segment-fsync` contract makes `init.mp4`
|
||||
and each complete media segment plus its directory entry durable before its
|
||||
matching index row is committed; the index and interrupted checkpoint are
|
||||
fsynced before append returns. Therefore camera RPO is bounded by the fragment
|
||||
currently being produced, not by an interval or byte batch. Startup recovery
|
||||
retains a contiguous valid prefix, quarantines orphan fragments and marks an
|
||||
unfinalized epoch `interrupted`.
|
||||
|
||||
The background recording-preparation worker also prepares immutable camera
|
||||
descriptors before publishing `ready`. Complete-fragment arrival timestamps are
|
||||
end/availability anchors, so the worker SHA-verifies and parses every fragment,
|
||||
anchors the first decoded sample at `first_arrival - first_duration`, and derives
|
||||
coverage from the exact sum of ISO-BMFF sample durations. It computes finite
|
||||
non-overlapping epoch coverage ends and exact aggregate bytes, then binds both to
|
||||
recorded-media manifest v2's generation digest. Replay v2 additionally requires
|
||||
camera coverage to stay inside the spatial RRD range with a 50 ms tolerance; a
|
||||
future replay v3 will need distinct spatial and session-union ranges.
|
||||
|
||||
The path-free descriptor is atomically persisted in the private derived cache
|
||||
with its checksum and a stat identity covering the native timing origin and
|
||||
every camera summary, index, init and segment file. Restart performs confined
|
||||
stat validation and reuses an unchanged descriptor without reading/hashing media;
|
||||
a missing, corrupt or stale sidecar is rebuilt in the background. An unparseable
|
||||
or ambiguous fragment fails readiness rather than exposing a fake seekable
|
||||
camera. Replay requests consume the prepared descriptor and never repeat media
|
||||
conversion or timing analysis. Launch and manifest aggregate byte counts must
|
||||
agree before the browser admits camera payload downloads.
|
||||
|
||||
The derived RRD writes an actual zero-time anchor at the internal
|
||||
`/__mission_core/session_origin` entity. Keeping it outside `/world` prevents a
|
||||
synthetic visualization layer while ensuring that the decoded RRD timeline,
|
||||
not merely its summary document, begins at session time zero. This changes the
|
||||
derived payload contract: cache v6 rejects v4/v5 sidecars and performs one
|
||||
background rebuild instead of silently reusing an archive without that row.
|
||||
|
||||
Replay v2 exposes two paths to the same pinned generation. `source_url` retains
|
||||
the strict `If-Match: "sha256:…"` contract for Mission Core clients;
|
||||
`viewer_source_url` appends the exact lowercase SHA-256 as a `generation` query
|
||||
for Rerun's native URL loader, which cannot attach custom headers. The server
|
||||
accepts either precondition, rejects a stale query with `412`, and keeps the
|
||||
matching file pinned for the entire immutable/no-transform response lifecycle.
|
||||
|
||||
This is host-arrival synchronization, not proven sensor-clock synchronization.
|
||||
It is sufficient to correlate current K1 point/pose and preview evidence at the
|
||||
known clock boundary, but must not be described as frame-accurate optical/LiDAR
|
||||
alignment.
|
||||
|
||||
Historical runs made before this ADR contain no camera archive. The fact that a
|
||||
camera was visible in the old browser preview does not make video recoverable;
|
||||
the catalog must not manufacture a camera modality for those sessions.
|
||||
|
||||
### Workspace layout
|
||||
|
||||
The first disk action in the spatial workspace saves only the versioned
|
||||
`observation.spatial` layout document:
|
||||
|
||||
- scene display settings;
|
||||
- open tool windows and their z-order;
|
||||
- visible dynamic source identifiers;
|
||||
- active floating source;
|
||||
- source-window rectangles normalized to the observed viewport.
|
||||
|
||||
The document uses optimistic revision control (`ETag` and `If-Match`) and is
|
||||
restored automatically on the next opening. Unknown source identifiers remain
|
||||
desired state so a temporarily disconnected camera can recover its saved
|
||||
window when that source returns. The save action never starts, completes or
|
||||
modifies an observation session.
|
||||
|
||||
### API boundary
|
||||
|
||||
The stable host routes are:
|
||||
|
||||
- `GET /api/v1/observation-sessions` for the recent catalog;
|
||||
- `GET /api/v1/observation-sessions/{id}` for modality/artifact metadata;
|
||||
- `POST /api/v1/observation-sessions/{id}/replay` for a ready launch or a
|
||||
preparation handle;
|
||||
- `GET /api/v1/observation-sessions/{id}/recording-preparation` for exact-job
|
||||
status polling;
|
||||
- `GET /api/v1/observation-sessions/{id}/recording.rrd` for seekable spatial data;
|
||||
- opaque manifest/init/segment routes below
|
||||
`/api/v1/observation-sessions/{id}/media/{artifact}` for recorded cameras;
|
||||
- `GET|PUT /api/v1/workspace-layouts/observation.spatial` for the layout profile.
|
||||
|
||||
The session API precedes the static frontend mount. Recording paths and storage
|
||||
roots are server implementation details.
|
||||
|
||||
## Consequences
|
||||
|
||||
- A completed or interrupted spatial run is discovered and prepared
|
||||
automatically, then can be replayed after browser and service restarts.
|
||||
- A service crash can lose the final uncommitted MQTT group and the camera
|
||||
fragment currently being produced; it cannot make an index point beyond
|
||||
durable camera media or replay a raw tail beyond aligned MQTT metadata.
|
||||
- Raw K1 evidence remains authoritative and derived RRD files can be discarded
|
||||
and regenerated.
|
||||
- Browser performance no longer controls whether camera evidence is retained.
|
||||
- Layout restoration is portable across viewport sizes and dynamic device
|
||||
source catalogs.
|
||||
- Replay builds a separate recorded-source catalog, so an active device camera
|
||||
can never be mixed into the evidence of a saved session. Archived fMP4 codec
|
||||
epochs follow Rerun play/pause/seek through host-arrival best-effort offsets.
|
||||
- A preparation ETag binds polling and cancellation to one exact job; a short
|
||||
launch reservation prevents cache eviction between the launch response and
|
||||
the viewer's first RRD request. Browser session switching never cancels the
|
||||
shared backend preparation.
|
||||
- Storage capacity, retention, quota, encryption-at-rest and export policy are
|
||||
still deployment responsibilities; this ADR does not silently delete source
|
||||
evidence.
|
||||
- Recorded camera playback is confined to canonical archives with a supported
|
||||
fMP4 codec declaration and provable sample duration. Missing historical media,
|
||||
unsupported codec epochs and ambiguous timing fail closed; none is substituted
|
||||
with a live preview.
|
||||
|
||||
## Rejected alternatives
|
||||
|
||||
- **Use browser cache or IndexedDB for sessions.** This loses the evidence when
|
||||
the browser profile is cleared and cannot guarantee acquisition durability.
|
||||
- **Save a session only when the operator presses the disk icon.** This makes
|
||||
safety evidence optional and fails on crashes or forgotten clicks.
|
||||
- **Store only RRD.** This discards the native evidence needed to review future
|
||||
decoders and protocol assumptions.
|
||||
- **Keep camera acquisition coupled to WebSocket viewers.** A UI disconnect
|
||||
would stop recording and recreate the observed data-loss bug.
|
||||
- **Copy every legacy run into the private catalog root.** This doubles large
|
||||
artifacts without improving integrity; confined zero-copy indexing is
|
||||
sufficient for the current laboratory store.
|
||||
|
|
@ -24,13 +24,16 @@ The plugin owns:
|
|||
- firmware-scoped protobuf/LZ4 and legacy codecs;
|
||||
- normalization of K1 point cloud and pose, plus raw-only preservation of the
|
||||
still-undecoded status and heartbeat channels;
|
||||
- exact-profile left/right RTSP producer selection, H.264 copy-remux preview and
|
||||
the handoff of acquisition-owned init/fMP4/index evidence to the host archive;
|
||||
- K1-specific operator instructions and compatibility tests.
|
||||
|
||||
The plugin does not own:
|
||||
|
||||
- Mission Core navigation, fleet, missions, users, roles, or audit;
|
||||
- the generic spatial scene or Rerun Web Viewer;
|
||||
- platform storage, remote transport, or other devices.
|
||||
- the host observation catalog, RRD preparation/cache, recorded-camera player,
|
||||
workspace layout, platform storage, remote transport, or other devices.
|
||||
|
||||
Until extraction is complete, `src/k1link` is the compatibility source of
|
||||
truth and every move into this package must preserve replay and real-device
|
||||
|
|
|
|||
Loading…
Reference in New Issue