diff --git a/README.md b/README.md index a8888a3..e652aed 100644 --- a/README.md +++ b/README.md @@ -10,9 +10,11 @@ The first proven hardware vertical is the XGRIDS/LixelKity K1 plugin. On firmwar 3.0.2 the host provisions the scanner onto an existing LAN without LixelGO, connects to its MQTT broker, persists each raw frame before preview work, decodes point cloud and pose, and renders the real cloud plus trajectory through an -embedded self-hosted Rerun Web Viewer. Capture files are `fsync`ed on clean -close; per-frame power-loss durability is not claimed. The former Foxglove -bridge remains only as a legacy regression module. +embedded self-hosted Rerun Web Viewer. Native MQTT persistence uses bounded +group commit (at most 0.5 seconds, 4 MiB or 32 messages), and camera archives +commit complete fMP4 segments before their index rows. These are explicit +crash-RPO bounds, not a zero-loss or disk-replication claim. The former +Foxglove bridge remains only as a legacy regression module. The repository is intentionally migrating in stages. The current `src/k1link` package is the compatibility implementation of the first plugin path; vendor @@ -124,19 +126,24 @@ revision or content hash. Publishing/vendoring those packages or enforcing an immutable donor revision remains a packaging and CI prerequisite. The Observation spatial workspace embeds the open-source Rerun Web Viewer -inside the Mission Core shell. It can open a compatible RRD file over HTTP(S) or a -Rerun gRPC/proxy source such as `rerun+http://127.0.0.1:9876/proxy`. It does not -use an external hosted viewer UI. Dynamic point-cloud and camera source -composition, host-owned window layout and the current live-only timeline contract -are fixed in [`ADR 0006`](docs/adr/0006-vendor-neutral-observation-sources-and-live-only-timeline.md). +inside the Mission Core shell. It can open a compatible RRD over same-origin +HTTP or a Rerun gRPC/proxy source such as +`rerun+http://127.0.0.1:9876/proxy`; no external hosted viewer UI is used. +Dynamic point-cloud/camera composition and the live source contract are fixed in +[`ADR 0006`](docs/adr/0006-vendor-neutral-observation-sources-and-live-only-timeline.md). +The durable session catalog, recorded `session_time` scrubber and versioned +workspace-layout profile are fixed in +[`ADR 0008`](docs/adr/0008-durable-observation-sessions-and-workspace-layout.md). -The first K1 live session or replay in a `k1link serve` process creates one local -Rerun `RecordingStream`, starts its gRPC/proxy server on TCP 9876 and publishes -the resulting URL through control-plane state. Later sessions reset their -session-local scene and metrics and reuse that process-wide stream; this avoids -restarting the native listener while the embedded browser remains connected. -Unless an operator has entered a manual source, the React application assigns -that URL to the embedded viewer. The complete runtime path is K1 MQTT → raw-first +The first K1 live session or adapter file-replay (`.k1mqtt`/reviewed TSV) in a +`k1link serve` process creates one local Rerun `RecordingStream`, starts its +gRPC/proxy server on TCP 9876 and publishes the resulting URL through +control-plane state. Later live/file-replay sessions reset their session-local +scene and metrics and reuse that process-wide stream. Saved observation sessions +do not reuse this listener: they open a private digest-bound RRD generation over +same-origin HTTP through Rerun's native incremental receiver. Unless an operator +has entered a manual source, the React application assigns the applicable source +to the embedded viewer. The complete live runtime path is K1 MQTT → raw-first evidence capture → bounded latest-wins preview queue → explicitly injected K1 protobuf/LZ4 normalizer → transport-neutral decoded local views → Rerun `Points3D`, `Transform3D` and `LineStrips3D` → embedded Web Viewer. Rerun does @@ -146,15 +153,21 @@ portable Plugin SDK wire envelopes. The default Rerun blueprint shows a 12-second sliding accumulation of real point frames. Product controls are connected for point size, intensity/height/distance or available RGB coloring, Turbo/Viridis/Plasma/grayscale/custom palettes, -point and trajectory visibility, and the scene grid. Projection, custom -timeline transport and saved layout remain later product work. No synthetic -point cloud, trajectory, camera frame or latency value is generated. +point and trajectory visibility, and the scene grid. The first disk action saves +and restores the versioned spatial layout without mutating sensor evidence. +Saved native point/pose sessions are materialized losslessly into private, +digest-bound RRD recordings and can be played, paused and scrubbed on a +zero-based `session_time` timeline. No synthetic point cloud, trajectory, +camera frame or latency value is generated. A powered-device checkpoint passed 80 real MQTT messages through the current Rerun runtime: 38 point-cloud frames, 42 pose frames, 2,775 points in the last -cloud and zero decode errors. The later RTSP camera preview is not yet wired into -the Rerun/runtime path. Rerun `capture_time` is the Mac receive timestamp, not a -proven K1 sensor timestamp or photon-to-screen measurement. +cloud and zero decode errors. The later RTSP camera preview is available through +the generic floating observation windows and new acquisitions archive its fMP4 +segments independently of browser delivery. Historical sessions recorded before +that archive contract contain no video. Rerun `capture_time` and the camera +index use Mac receive/arrival timestamps, not proven K1 sensor timestamps or a +photon-to-screen measurement. The old Foxglove implementation is retained only in `src/k1link/viewer/foxglove_bridge.py` and its regression tests. The current @@ -162,6 +175,8 @@ live/replay runtime does not start it or use TCP 8765. The [live viewer runbook](docs/06_K1_LIVE_VIEWER.md) records the active Rerun path and its timing/security boundaries; the frontend contract is documented in [`apps/control-station/README.md`](apps/control-station/README.md). +The host storage layout, recovery rules, replay API and operator path are in +[`docs/09_OBSERVATION_SESSIONS.md`](docs/09_OBSERVATION_SESSIONS.md). The FastAPI application and credential endpoint bind to loopback, but the Rerun gRPC server currently binds TCP 9876 on all network interfaces even though its @@ -214,9 +229,15 @@ present. - [Verified MQTT stream profile](docs/05_K1_MQTT_STREAM_PROFILE.md) - [Live console and embedded Rerun runbook](docs/06_K1_LIVE_VIEWER.md) - [Mission Core monorepo and plugin boundary](docs/07_MISSION_CORE_MONOREPO.md) +- [Owner-controlled LixelGO/iPhone observation](docs/08_LIXELGO_IPHONE_OBSERVATION.md) +- [Observation sessions, playback and workspace layout](docs/09_OBSERVATION_SESSIONS.md) - [Monorepo architecture decision](docs/adr/0002-mission-core-monorepo.md) - [Device plugin UI and runtime boundary](docs/adr/0003-device-plugin-ui-and-runtime-boundary.md) - [Plugin SDK v0alpha2 and experimental device lifecycle](docs/adr/0004-plugin-sdk-v0alpha2-and-experimental-device-lifecycle.md) +- [Owner-controlled LixelGO observation decision](docs/adr/0005-owner-controlled-lixelgo-iphone-observation.md) +- [Vendor-neutral live observation sources](docs/adr/0006-vendor-neutral-observation-sources-and-live-only-timeline.md) +- [K1 camera preview copy-remux gateway](docs/adr/0007-k1-camera-preview-copy-remux-gateway.md) +- [Durable observation sessions and workspace layout](docs/adr/0008-durable-observation-sessions-and-workspace-layout.md) - [Redacted live lab report](docs/lab/001_K1_LIVE_MQTT_20260715.redacted.md) - [Session manifest schema](schemas/session-manifest.schema.json) - [Reference input provenance](docs/reference/README.md) @@ -240,6 +261,6 @@ reviewed step. Random writes, fuzzing, brute force, firmware operations, destructive file access and credential guessing remain out of scope. Real captures, projects, router metadata, serials, credentials, maps, images, -and logs are ignored by normal Git. Redacted manifests and SHA-256 inventories -are committed; encrypted artifact storage will be selected only when real data -exists. +and logs under `.runtime/`, canonical evidence roots and legacy `sessions/` are +ignored by normal Git. Redacted manifests and SHA-256 inventories are committed; +encrypted/replicated artifact storage remains a deployment decision. diff --git a/apps/control-station/README.md b/apps/control-station/README.md index c8d7e22..a68241f 100644 --- a/apps/control-station/README.md +++ b/apps/control-station/README.md @@ -7,9 +7,10 @@ device adapter, но структура интерфейса от него не Внутри пространственной рабочей поверхности встроен открытый Rerun Web Viewer. Это self-hosted frontend-компонент из npm-пакета `@rerun-io/web-viewer`, а не -переход во внешний облачный интерфейс. При запуске K1 live/replay backend сам -создаёт Rerun gRPC/proxy source; ручной адрес нужен только для другого Rerun -потока или совместимой RRD-записи. +переход во внешний облачный интерфейс. Для K1 live и adapter file-replay backend +сам создаёт Rerun gRPC/proxy source. Saved observation sessions открывают +проверенную digest-bound RRD generation через same-origin HTTP; ручной адрес +нужен только для другого совместимого Rerun source. ## Текущее состояние @@ -20,11 +21,13 @@ device adapter, но структура интерфейса от него не | Локальный control plane | Реализован | React получает состояние и выполняет операции через FastAPI REST и WebSocket на loopback. | | K1 BLE → Wi-Fi | Реализован | Реальный BLE-поиск всех видимых устройств и одна подтверждённая provisioning-запись выбранному устройству. | | K1 live/replay MQTT | Реализован | Read-only приём, raw-first сохранение, декодирование облака точек и позы, реальные метрики. | -| Автоматический MQTT → Rerun | Реализован | Первый live/replay поднимает process-wide `RecordingStream` и gRPC/proxy на TCP 9876; следующие сессии сбрасывают сцену и метрики и переиспользуют его. | +| Автоматический MQTT → Rerun | Реализован | Первый live/adapter file-replay поднимает process-wide `RecordingStream` и gRPC/proxy на TCP 9876; следующие такие сессии переиспользуют его. Saved observation replay использует отдельный immutable HTTP RRD path. | | Встроенный Rerun Viewer | Реализован | Self-hosted npm-компонент автоматически открывает текущий gRPC source внутри Control Station; внешний viewer не используется. | -| Контролы сцены → Rerun | Реализованы для текущей геометрии | Работают размер и видимость точек, атрибут цвета, палитра, окно накопления, траектория и сетка. Проекция, собственный timeline и сохранённые layout-профили ещё не подключены. | +| Контролы сцены → Rerun | Реализованы для текущей геометрии | Работают размер и видимость точек, атрибут цвета, палитра, окно накопления, траектория, сетка, host timeline и сохранение/восстановление spatial layout. Проекция и семантические слои ещё не подключены. | +| Сохранённые observation sessions | Реализованы для point/pose | Три последние сессии, background preparation, cache v6, generation-bound RRD, atomic admission, autoplay, play/pause/seek и controlled switching больших записей. | +| K1 camera preview и archive | Live реализован; recorded contract реализован | Обе RTSP/H.264 камеры физически приняты в live UI. Новые acquisition-owned fMP4 archives не зависят от browser windows; recorded player подключён, но реальная архивная K1 camera-session ещё не прошла physical acceptance. | | Legacy Foxglove module | Только regression | Модуль и тесты сохранены для сравнения декодирования. Текущий live/replay runtime не запускает Foxglove WebSocket и не использует TCP 8765. | -| Камеры, карты и миссии | Интерфейс готов | Серверная логика и реальные каналы для этих рабочих поверхностей ещё не подключены. | +| Карты и миссии | Интерфейсный каркас | Реальные map/mission backends и vehicle control ещё не подключены. | Приложение не генерирует демонстрационное облако, траекторию, кадры или метрики. Если реальных данных нет, область сцены остаётся пустой, а числовые поля @@ -48,9 +51,16 @@ Mission Core Control Station ←→ REST /api/v1/device-plugins/* + plugin-scoped WebSocket events FastAPI на 127.0.0.1:8000 CoreBluetooth + live/replay runtime + +sealed/recovered observation session +└── SQLite catalog + bounded background preparation + └── atomic RRD cache v6 + recorded-media manifest v2 + └── generation-bound same-origin HTTP + └── aggregate admission + └── Rerun native receiver + recorded fMP4 player ``` -В момент готовности `RerunBridge` backend публикует адрес вида +В момент готовности live/file-replay `RerunBridge` backend публикует адрес вида `rerun+http://127.0.0.1:9876/proxy`. Frontend автоматически назначает его сцене, если оператор не указал ручной source. После остановки приёма URL и встроенный viewer остаются активны, а следующая сессия сбрасывает session-local геометрию, @@ -65,7 +75,7 @@ viewer остаются активны, а следующая сессия сб Shell собран из локальных NODE.DC UI packages и сохраняет одну структуру для всех функциональных модулей: -1. `AppHeader` — марка NODEDC MISSION CORE, выбор архитектурного раздела и состояние +1. `AppHeader` — марка NODE DC, выбор архитектурного раздела и состояние локального backend. 2. `AdminNavigationPanel` — контекст аппарата и список рабочих поверхностей выбранного раздела. @@ -78,8 +88,10 @@ Shell собран из локальных NODE.DC UI packages и сохраня blueprint-, selection- и time-панели скрыты, чтобы продуктовые действия жили в Control Station. Размер точек, способ окрашивания и палитра, 12-секундное по умолчанию накопление, видимость облака и траектории и сетка связаны с backend и -Rerun Blueprint. Кнопки собственного timeline, смена 2D/3D/карты и сохранение -layout пока остаются интерфейсным контрактом. +Rerun Blueprint. Host timeline управляет сохранённым `session_time`, а spatial +layout сохраняет display settings, tool windows и source-window geometry через +revisioned API. Смена 2D/3D/карты и семантические слои пока остаются +интерфейсным контрактом. ## Архитектурные разделы @@ -139,10 +151,15 @@ cd apps/control-station npm run dev ``` -Vite слушает `http://127.0.0.1:5173` и проксирует весь `/api` (включая WebSocket) на -`http://127.0.0.1:8000`. Другой локальный backend можно указать переменной -`VITE_API_TARGET`. Preview production-сборки запускается командой -`npm run preview` на `http://127.0.0.1:4173`. +Для полного live-viewer контура Vite должен открыться на +`http://127.0.0.1:5173`: только 5173, production preview 4173 и backend 8000 +входят в текущий Rerun CORS allowlist. Если Vite сообщает fallback на 5174 или +выше, освободите 5173 и перезапустите `npm run dev`; REST/WebSocket proxy на +fallback-порту может работать, но прямой browser → Rerun 9876 будет отклонён. +Vite проксирует весь `/api` (включая WebSocket) на `http://127.0.0.1:8000`. +Другой локальный backend можно указать переменной `VITE_API_TARGET`. Preview +production-сборки запускается командой `npm run preview` на +`http://127.0.0.1:4173`. ## Операторский путь для текущего K1 adapter @@ -167,6 +184,9 @@ Vite слушает `http://127.0.0.1:5173` и проксирует весь `/a 9. Открыть **Наблюдение → Пространственная сцена**. Реальные облако и траектория, частота, число точек, задержка и пропуски preview появятся после прихода сообщений K1. +10. После нормального stop или recovery открыть **Сохранённые сессии**. Дождаться + состояния **Готово**, выбрать запись и использовать host timeline. Evidence + сохраняется автоматически; disk action сохраняет только workspace layout. В проверочном live-сеансе через этот путь прошло 80 реальных MQTT-сообщений: 38 кадров `lio_pcl`, 42 кадра `lio_pose`, 2 775 точек в последнем облаке и @@ -203,9 +223,12 @@ Plasma, grayscale и custom, накопление `0–120` секунд, а т декодированном формате. Custom/class сейчас означает выбранный сплошной цвет, а не готовую семантическую классификацию. -Custom timeline, переключатель 2D/3D/карты, семантические слои и сохранение RBL -пока не вызывают Blueprint/playback API. Черновик компоновки фиксируется только -в памяти текущей страницы и не записывается на диск. +Для saved session host timeline вызывает Rerun play/pause/seek только после +полного admission. Accumulation и display settings применяются через отдельный +маленький blueprint RRD и не заменяют основную запись. Versioned +`observation.spatial` layout сохраняется на host с optimistic revision; он не +содержит sensor evidence. Переключатель 2D/3D/карты и семантические слои пока не +вызывают готовый presentation backend. ## Контракт локального API @@ -216,6 +239,12 @@ Custom timeline, переключатель 2D/3D/карты, семантиче | `GET` | `/api/v1/device-models` | Backend-каталог моделей из валидированных manifests; UI-каталог текущей сборки формируется отдельным static composition root. | | `POST` | `/api/v1/device-plugins/{pluginId}/actions/{actionId}` | Namespaced действие через host allowlist; тело `{ "input": { ... } }`. | | `WS` | `/api/v1/device-plugins/{pluginId}/events` | Plugin-scoped snapshots с `pluginId` и монотонным `sequence`. | +| `GET` | `/api/v1/observation-sessions` | Последние cataloged sessions и authoritative preparation state. | +| `POST` | `/api/v1/observation-sessions/{id}/replay` | Verified replay launch либо HTTP 202 preparation handle. | +| `GET` | `/api/v1/observation-sessions/{id}/recording-preparation` | Exact-job polling с `If-Match`. | +| `GET` | `/api/v1/observation-sessions/{id}/recording.rrd` | Immutable generation-bound RRD; arbitrary chunked `send_rrd` не используется. | +| `GET` | `/api/v1/observation-sessions/{id}/media/{artifact}/...` | Manifest/init/segments записанных камер по opaque identifiers. | +| `GET`, `PUT` | `/api/v1/workspace-layouts/observation.spatial` | Versioned host layout с optimistic revision. | Frontend XGRIDS-плагина использует только v1alpha namespaced routes. Старые `/api/state`, `/api/events`, `/api/ble/scan`, `/api/connect`, `/api/session/*` и @@ -257,7 +286,12 @@ facts и старые presentation-поля `phase`, `message`, `devices`, | `src/workspaces/DeviceWorkspace.tsx` | Generic выбор модели и `device.connection` slot. | | `src/device-plugins/xgrids-k1/` | Реальный K1 BLE/Wi-Fi/live/replay UI, client и compatibility mapper. | | `src/workspaces/Workspaces.tsx` | Оперативный обзор, spatial viewport и остальные продуктовые поверхности. | -| `src/components/RerunViewport.tsx` | Жизненный цикл встроенного Rerun Web Viewer и selection events. | +| `src/components/RerunViewport.tsx` | Live/recorded lifecycle WebViewer, native RRD open, atomic admission, playback и selection events. | +| `src/components/ObservationSessionSelect.tsx` | Три последние сессии, состояния `Готово` / `Обработка` / `Ошибка`. | +| `src/components/RecordedFmp4Player.tsx` | Проверенный generation-bound MSE playback архивных камер. | +| `src/core/observation/sessionArchive.ts` | Строгий wire contract catalog/preparation/replay/media API. | +| `src/core/observation/recordedSessionAdmission.ts` | Общий RRD/camera gate до публикации recorded workspace. | +| `src/core/observation/workspaceLayout.ts` | Versioned spatial layout и optimistic revision. | | `src/sceneSettings.ts` | Типизированный UI-профиль пространственной сцены. | | `src/presentation.ts` | Vendor-neutral подписи lifecycle и форматирование нормализованных метрик. | | `src/styles.css`, `src/styles/*` | Компоновка shell и рабочих поверхностей. | @@ -277,14 +311,17 @@ facts и старые presentation-поля `phase`, `message`, `devices`, случайные GATT writes и автоматические повторы запрещены. - MQTT live/replay не публикует команды устройству. Запуск и остановка физического сканирования остаются за кнопкой K1. -- Live-сессии сначала сохраняют сырые сообщения, затем формируют preview. При - перегрузке preview может быть отброшен, raw evidence сохраняется. +- Live-сессии сначала сохраняют сырые сообщения и camera segments, затем + формируют disposable preview. При перегрузке preview может быть отброшен, + native evidence сохраняется. MQTT durability имеет bounded group-commit RPO, + camera durability — текущий незавершённый fragment RPO; это не zero-loss claim. - Timeline `capture_time` сохраняет Unix-время приёма сообщения Mac, а окно накопления viewer использует session-local `stream_time`. `capture_time` — не доказанный timestamp сенсора K1 и не photon-to-screen latency. -- Панорамный камерный поток в наблюдавшихся MQTT report topics отсутствует; - текущая Rerun-сцена содержит только облако точек и позу/траекторию, а - операционные метрики отображаются внешней оболочкой Mission Core. -- `sessions/` игнорируется Git и может содержать адреса, идентификаторы, - траекторию и карту помещения. В репозиторий попадают только redacted manifests - и безопасная документация. +- Панорамный камерный поток в MQTT report topics отсутствует; отдельные + left/right RTSP preview доступны через generic camera windows. LiDAR и camera + сейчас синхронизированы только по host arrival, не по доказанным sensor clocks. +- `.runtime/`, canonical evidence roots и legacy `sessions/` игнорируются Git и + могут содержать адреса, изображения, идентификаторы, траекторию и карту + помещения. В репозиторий попадают только код, тесты, redacted manifests и + безопасная документация. diff --git a/docs/00_TECHNICAL_AUDIT.md b/docs/00_TECHNICAL_AUDIT.md index 151221f..fe7c71e 100644 --- a/docs/00_TECHNICAL_AUDIT.md +++ b/docs/00_TECHNICAL_AUDIT.md @@ -13,8 +13,12 @@ All 1,140 captured `lio_pcl` frames decoded as raw-LZ4 protobuf blocks, yielding 4,165,862 points. All 1,215 `lio_pose` messages decoded, and the resulting approximately 1.566 m displacement matched the controlled movement. A later owner-operated LixelGO/iPhone capture also observed left/right RTSP/H.264 camera -preview and the remote start/stop wire mapping. The remaining camera work is a -bounded Mission Core receiver, not transport discovery. +preview and the remote start/stop wire mapping. Mission Core now has a bounded +read-only receiver, physical live left/right UI acceptance, acquisition-owned +fMP4 archival and a generation-bound recorded player. The remaining camera gate +is a newly recorded real K1 session played end-to-end with its point/pose +timeline. Full-resolution/raw panorama, device-clock synchronization, +intrinsics/extrinsics and remote delivery are still unproven. The supplied Bible is useful as an OSINT dossier. It is not an executable plan for the actual stand because many experiments assume a phone and LixelGO. The diff --git a/docs/01_IMPLEMENTATION_PLAN.md b/docs/01_IMPLEMENTATION_PLAN.md index 126f1c8..db079e0 100644 --- a/docs/01_IMPLEMENTATION_PLAN.md +++ b/docs/01_IMPLEMENTATION_PLAN.md @@ -3,7 +3,7 @@ This plan supersedes the app-dependent experiment order in the reference Bible. Each gate produces evidence and an explicit GO, PAUSE or BLOCKED result. -## Live checkpoint — 2026-07-15 +## Current checkpoint — 2026-07-17 | Stage | Result | | --- | --- | @@ -15,8 +15,11 @@ Each gate produces evidence and an explicit GO, PAUSE or BLOCKED result. | Stage 4 artifacts/flows | GO — bounded capture, hashes and negative control | | Stage 5 point cloud | GO — raw-LZ4 protobuf, 1,140 live frames decoded | | Stage 5 pose | GO — 1,215 live frames decoded and motion-correlated | -| Stage 5 camera | GO (discovery) — left/right RTSP/H.264 preview observed; runtime adapter pending | -| Stage 6 live viewer | GO — React console, Foxglove cloud/path and Mac latency metrics | +| Stage 5 camera | GO (live) — left/right RTSP/H.264 preview observed, read-only runtime adapter and physical UI acceptance completed | +| Stage 6 live viewer | GO — React Control Station, embedded self-hosted Rerun cloud/trajectory and Mac pipeline metrics | +| Stage 7 observation archive | GO (point/pose) — durable catalog, recovery, background RRD preparation, saved-session timeline and atomic playback verified | +| Stage 7 recorded cameras | GO (contract), acceptance pending — acquisition-owned fMP4 archive and player are implemented/tested; no real archived K1 camera session exists yet | +| Stage 8 product storage | PAUSE — retention, replication, encryption, capacity monitoring and long-run browser/WASM stress remain deployment gates | USB project copying remains optional ground truth rather than a blocker for the now-verified network path. Owner-operated LixelGO traffic verifies the MQTT @@ -24,10 +27,14 @@ start/stop mapping and RTSP camera transport. MQTT control publishing remains deliberately deferred because complete request/save/rollback semantics are not yet modeled and the physical button remains a known-safe fallback. -The Stage 6 alpha uses a bounded raw-first bridge: loss in the visualization -queue cannot discard MQTT evidence. Acceptance is replay of the full captured -scan followed by a live ideal-LAN run with measured host pipeline latency. -Sensor-to-display latency remains a separate clock-correlation test. +The Stage 6 live path uses a bounded raw-first bridge: loss in the visualization +queue cannot discard MQTT evidence. Stage 7 adds an independent durable +observation-session lifecycle. Completed or recovered native captures are +prepared once by a bounded backend worker into digest-bound RRD/cache-v6 and +camera-manifest generations; a browser replay request never performs conversion. +The saved scene, controller and timeline remain hidden until the complete RRD +range and every declared camera pass admission. Sensor-to-display latency and +camera/LiDAR sensor-clock alignment remain separate correlation tests. ## Stage 0 — repository and host baseline @@ -213,4 +220,6 @@ useful stream is decoded or structurally identified. - automated scan-button electronics; - OpenWrt/monitor-mode infrastructure; - firmware or internal-Linux analysis; -- camera branch if no external frame stream is evidenced. +- physical end-to-end replay acceptance for newly archived left/right cameras; +- long-running large-session WebViewer/WASM memory telemetry; +- production retention, replication, encryption and cross-platform packaging. diff --git a/docs/06_K1_LIVE_VIEWER.md b/docs/06_K1_LIVE_VIEWER.md index 59b392b..9001b94 100644 --- a/docs/06_K1_LIVE_VIEWER.md +++ b/docs/06_K1_LIVE_VIEWER.md @@ -1,9 +1,11 @@ # K1 live console and embedded Rerun bridge -Status: the Rerun alpha milestone is implemented for the verified firmware-3 -MQTT streams. Live capture and replay use real K1 point-cloud and pose messages; -the application does not generate a placeholder cloud, trajectory, camera frame -or latency value. +Status: the Rerun live and saved-session milestones are implemented for the +verified firmware-3 point/pose streams. Live and adapter file-replay use the +process-wide gRPC source; saved observation sessions use private digest-bound +RRD over same-origin HTTP. Left/right RTSP preview is delivered separately from +the spatial Rerun stream. The application does not generate placeholder cloud, +trajectory, camera frame or latency values. ## Active device-to-scene path @@ -36,10 +38,12 @@ React console <-- REST + WebSocket state --> FastAPI on 127.0.0.1:8000 ``` The Paho MQTT callback does not decode or render the point cloud. It first -writes and flushes the raw frame and metadata, then enqueues a preview message. -If visualization cannot keep up, the oldest queued preview is discarded while -the raw capture continues. Rerun work stays on the dedicated publisher thread -and cannot block raw-first evidence capture. +appends the raw frame, flushes it to the operating-system page cache and stages +its aligned metadata, then enqueues a preview message. Raw bytes and metadata +become a durable pair at the bounded group-commit boundary (at most 0.5 s, +4 MiB or 32 messages), not at every callback. If visualization cannot keep up, +the oldest queued preview is discarded while capture continues. Rerun work +stays on the dedicated publisher thread and cannot block that raw-first path. The current live/replay runtime instantiates only `RerunBridge`. The former Foxglove implementation is not a parallel runtime and does not listen on TCP @@ -53,7 +57,7 @@ Prerequisites are the repository-local Python environment and Node.js 20.19+ or ```bash uv sync --group dev cd apps/control-station -npm install +npm ci npm run typecheck npm run build cd ../.. @@ -112,18 +116,20 @@ directly and use their sibling metadata receive timestamps when present. 9. Double-click K1 again to stop physical scanning, wait for steady green, then stop the local session so captures and summaries are finalized. -Each live run creates an ignored `sessions/_viewer_live/` directory with a -redacted manifest, operator notes, raw MQTT frames, per-message metadata and a -hash summary. The connector subscribes to the fixed report-topic allowlist and -does not publish an application request or modeling command. +Each new live run creates a direct child below `MISSIONCORE_EVIDENCE_DIR`, or +`.runtime/mission-core/evidence/sessions/` by default, with raw MQTT frames, +per-message metadata and a hash summary. Repository-level +`sessions/*_viewer_live` is legacy import-only evidence and is never selected by +the current writer. The connector subscribes to the fixed report-topic +allowlist and does not publish an application request or modeling command. ## Automatic Rerun source and lifecycle -On the first live or replay session, `RerunBridge`: +On the first live or adapter file-replay session, `RerunBridge`: - creates an explicit `RecordingStream("nodedc_mission_core_spatial")`; - installs the default spatial blueprint; -- starts the gRPC/proxy server on TCP 9876 with a 512 MiB late-client buffer; +- starts the gRPC/proxy server on TCP 9876 with a 32 MiB late-client buffer; - reports `rerun+http://127.0.0.1:9876/proxy` only after the server is ready; - accepts the local development and production browser origins used by this repository; @@ -150,7 +156,7 @@ stream directly from the Rerun gRPC/proxy endpoint. | `/world` | `ViewCoordinates` | right-handed Z-up display convention | | `/world/points` | `Points3D` | decoded metric XYZ with computed or available RGB colors | | `/world/sensor_pose` | `Transform3D` + `TransformAxes3D` | current decoded translation, xyzw quaternion and pose axes | -| `/world/trajectory` | `LineStrips3D` | bounded path of up to 20,000 decoded poses | +| `/world/trajectory` | `LineStrips3D` | bounded path of up to 2,000 decoded poses | Point count, frame rate, queue drops and measured pipeline time remain product metrics in the outer Control Station; they are not logged as Rerun entities and @@ -189,9 +195,34 @@ shows only the current frame. RGB mode falls back to the verified scalar coloring path when the active point format has no RGB fields. The current `class`/custom behavior is a selected solid color, not semantic segmentation. -Projection switching, a custom playback timeline, semantic object/mask layers, -camera frustums and persisted RBL/layout profiles are not wired yet and must not -be inferred from the implemented controls above. +Projection switching, semantic object/mask layers and camera frustums are not +wired yet. The host-owned `observation.spatial` layout profile does persist the +implemented scene controls, tool-window state, dynamic source visibility and +normalized floating-window geometry. It is separate from Rerun's internal +blueprint and from observation evidence. + +## Saved observation sessions + +Every completed native live run is indexed by the local host session store. A +valid crash prefix without its final summary is indexed as `interrupted`. The +**Сохранённые сессии** control shows the three newest runs and launches an +opaque same-origin RRD rather than starting a second live bridge. + +Sealing, recovery or legacy import makes the session eligible for the bounded +backend preparation worker. The worker reads every native message once and +atomically publishes a private cache-v6 RRD using the zero-based `session_time` +duration timeline. A SHA sidecar binds the generation to native evidence. +Replay-open never performs conversion: it returns HTTP 202 and a preparation +handle or reuses the verified artifact. The embedded viewer opens the exact +generation through Rerun's native incremental HTTP receiver. The bottom Control +Station timeline then controls play/pause and seek directly; it does not +approximate time with a React timer. + +The native `.k1mqtt` remains the evidence master. RRD generation does not use +the bounded live-preview queue, so it retains every frame accepted by the +reviewed normalizer. See +[`09_OBSERVATION_SESSIONS.md`](09_OBSERVATION_SESSIONS.md) for storage, +recovery and HTTP Range details. ## Time and latency semantics @@ -228,7 +259,7 @@ and host firewall as appropriate, and never forward it directly to the public Internet or cellular WAN. Remote operation requires an authenticated TLS reverse proxy or another reviewed secure transport before deployment. -The 512 MiB Rerun server buffer limits retained late-client data but is not an +The 32 MiB Rerun server buffer limits retained late-client data but is not an access-control mechanism. The listener and embedded viewer intentionally remain ready between acquisition sessions. Stop `k1link serve` when the network listener and its process memory must be closed unconditionally. @@ -236,8 +267,10 @@ listener and its process memory must be closed unconditionally. ## Current boundaries - Raw panoramic camera frames were absent from the observed MQTT report topics. - This milestone contains point cloud and pose/trajectory; operational metrics - are rendered by the outer Control Station. + The Rerun spatial recording contains point cloud and pose/trajectory; + left/right compressed RTSP preview and acquisition-owned fMP4 archive use a + separate generic media path. Historical sessions predating that archive have + no recoverable video. - Physical double-click remains the K1 scan start/stop control. Any MQTT command publisher needs a separately reviewed state-changing profile. - No terrain map, elevation model, obstacle segmentation, localization fusion, diff --git a/docs/07_MISSION_CORE_MONOREPO.md b/docs/07_MISSION_CORE_MONOREPO.md index 2bea885..a87cb1b 100644 --- a/docs/07_MISSION_CORE_MONOREPO.md +++ b/docs/07_MISSION_CORE_MONOREPO.md @@ -17,6 +17,9 @@ already exist. The accepted device-lifecycle decision is recorded in | `src/k1link/protocol/` | Firmware-scoped K1 codecs and explicit vendor normalizer | Plugin-owned protocol adapter | | `src/k1link/data_plane/` | Transport-neutral decoded in-process consumer views | Local projections hydrated from portable SDK envelopes | | `src/k1link/viewer/` | Rerun consumer, visualization runtime and legacy Foxglove regression module | Replaceable canonical scene sink and presentation adapters | +| `src/k1link/sessions/` | Host-owned SQLite catalog, evidence discovery/recovery, background preparation and derived-cache lifecycle | Device-neutral observation archive service | +| `src/k1link/web/session_api.py` | Opaque saved-session, immutable RRD/media and workspace-layout API | Versioned Control/Edge observation contract | +| `apps/control-station/src/core/observation/` | Session selection, replay admission, recorded-source and layout state | Device-neutral observation UI runtime | | `docs/domain-model/` | Versioned experimental Mission Core vocabulary | Promotion source for meanings proven across devices/services | ## Implemented boundary @@ -33,6 +36,17 @@ raw K1 transport -> embedded local viewer ``` +The host-owned recorded path is separate from the disposable live preview: + +```text +sealed or recovered native observation session + -> SQLite catalog and bounded single-worker preparation + -> atomic RRD cache v6 + immutable recorded-media manifest v2 + -> generation-bound same-origin HTTP + -> aggregate RRD/camera admission + -> native Rerun HTTP receiver + recorded fMP4 player +``` + Rerun does not import K1 protocol code, inspect MQTT topics or receive raw payloads. `VisualizationRuntime` cannot choose a vendor decoder implicitly; its composition owner injects one. The local `Decoded*View` classes are in-process @@ -52,6 +66,8 @@ The local vocabulary distinguishes: ```text model != device != transport alias != device session acquisition != operation != source/channel != evidence +observation session != preparation job != replay launch != viewer instance +workspace layout != sensor evidence acknowledgement != completion raw evidence != decoded data != viewer state ``` @@ -103,10 +119,14 @@ by physical double-click; no modeling request is published. Left/right camera preview transport, endpoint paths and H.264 framing are now observed under the exact compatibility profile. The local read-only adapter copy-remuxes one selected RTSP producer into bounded fMP4/WebSocket delivery for -the generic MSE UI. Portable FFmpeg packaging, fan-out and remote delivery remain -open. Device status and heartbeat remain raw observed channels without semantic -decoders. Device calibration command and sensor-to-vehicle extrinsics are -unavailable. +the generic MSE UI. New acquisitions archive selected-camera init/segments/index +independently of browser delivery. The generic recorded player, manifest-v2 +validation and shared `session_time` controls are connected; historical sessions +created before this archive contract contain no recoverable video, so physical +recorded-camera acceptance remains open. Portable FFmpeg packaging, disk-backed +browser buffering, fan-out and remote delivery also remain open. Device status +and heartbeat remain raw observed channels without semantic decoders. Device +calibration command and sensor-to-vehicle extrinsics are unavailable. The 2026-07-17 physical acceptance gate confirmed continuously updating point clouds, a matching live trajectory, and both camera selections in the same @@ -128,14 +148,17 @@ for the measured gate and remaining limits. 5. Replace compatibility routes and singleton state with multi-device session routing. 6. Split Edge execution from the Control Station behind authenticated transport. -7. Package the read-only RTSP/H.264 camera adapter for each target OS and evolve - same-host MSE delivery toward an authenticated Edge media plane; keep the - modeling-command publisher disabled until its separate safety gate closes. +7. Physically accept a newly archived left/right K1 session, then package the + read-only RTSP/H.264 adapter for each target OS, add disk-backed sealed media + caching and evolve same-host MSE delivery toward an authenticated Edge media + plane; keep the modeling-command publisher disabled until its separate safety + gate closes. ## Invariants -- Raw evidence is persisted before preview work and is never overwritten by a - decode result. +- Raw bytes are appended and OS-flushed before preview work; aligned raw and + metadata become durable together at the bounded group-commit boundary and + are never overwritten by a decode result. - Mission Core starts when the XGRIDS manifest is absent. Formal installed plugin quarantine remains a future supervisor feature. - Core code does not branch on XGRIDS IDs, fields, topics or firmware. @@ -160,8 +183,13 @@ for the measured gate and remaining limits. - complete SDK-envelope/EvidenceStore hot-path integration; - remote Edge split, authenticated WAN relay and fleet orchestration; - automatic K1 start/stop/calibration commands; -- automatic camera capability discovery, synchronized recording/rewind, - calibration, panoramic stitching and multi-consumer delivery. +- device-reported camera capability discovery beyond the reviewed K1 profile; +- physical acceptance of shared-timeline recorded-camera playback on a newly + archived K1 session; +- frame-accurate camera/LiDAR calibration, panoramic stitching, disk-backed + browser media cache and remote multi-consumer delivery; +- production retention, replication, encryption and long-run WebViewer/WASM + memory acceptance for large observation sessions. ## Workspace-repeatable locked architecture gate diff --git a/docs/09_OBSERVATION_SESSIONS.md b/docs/09_OBSERVATION_SESSIONS.md new file mode 100644 index 0000000..469e847 --- /dev/null +++ b/docs/09_OBSERVATION_SESSIONS.md @@ -0,0 +1,421 @@ +# Observation sessions, playback and workspace layout + +Status: implemented for native K1 point/pose evidence and host-side camera +archival. Recorded spatial playback is exposed in the Mission Core observation +workspace. The camera archive/player contract is implemented and covered by +tests, but all retained real K1 sessions predate canonical camera archival. +Physical archived-camera playback therefore remains an open acceptance gate; +historical sessions contain no recoverable video. + +## Operator path + +1. Start a normal acquisition from **Парк → Локальное устройство**. +2. Open **Наблюдение → Пространственная сцена**. Live point cloud, trajectory + and the selected camera remain live-only while acquisition is running. +3. Stop acquisition normally, or allow the local service to recover an + unexpected interruption on its next start. Session recording is automatic; + the disk action is not required. +4. Open **Сохранённые сессии** in the observation header. The menu shows the + three most recent indexed runs, their date, duration, state, modalities and + background preparation state. +5. Choose a replayable run. Opening a replay never performs conversion in the + request. A ready recording opens immediately; otherwise the client receives + HTTP 202, keeps the current scene mounted and polls the preparation status. + Only after the server returns a verified launch descriptor does Mission Core + pause and unload the previous viewer. +6. The client then receives and decodes the complete RRD. The new viewport and + timeline stay hidden until the declared recording range is fully buffered + (`fullyBuffered`). Playback starts once at that point. Partial frames are + never shown. If the run contains canonical camera archives, its recorded + camera sources replace all live device overlays. +7. Use **Воспроизвести / Пауза**, the scrubber, **К началу** and **К концу** on + the right half of the bottom timeline. The left half changes point-cloud + accumulation. The recorded timeline is zero-based `session_time`. +8. Use the first disk button to save the current workspace layout. The next + opening restores display settings, tool windows and dynamic source-window + positions. This action saves no sensor evidence. + +Display controls have no separate Apply/Reset transaction. Boolean controls +commit immediately; sliders, colors and selects commit on release/blur or after +a short quiet period, and concurrent commits collapse to the newest value. A +layout save flushes and awaits this queue before serializing the confirmed +settings. Recorded display updates reuse stable Rerun scene identifiers and do +not mutate playback state, so changing accumulation, size, color or visibility +does not pause the active replay or replace its camera view. + +Completed and recovered sessions are discovered on startup and by the catalog +reconciler, then deduplicated into one bounded, single-worker preparation queue. +The worker validates the native source, exports point/pose data, verifies source +stability and both SHA-256 digests, finalizes once, and atomically publishes the +digest-bound derived RRD plus its cache sidecar. Before the same job becomes +`ready`, it also parses and validates every archived camera index and retains +the immutable manifest generation in memory. The catalog exposes preparation +state and progress while this runs; a recording URL and camera descriptors are +returned only after the complete launch generation has passed validation. + +Preparation is a backend lifecycle, not a viewer action: sealing or recovering +a native session makes it eligible for the reconciler, which builds the derived +RRD once in the background. Reopening, seeking or changing the workspace reads +that prepared artifact and never reruns conversion. A cold process may validate +an existing cache and rebuild its in-memory camera manifest in the worker, but +no replay, status, RRD or manifest request hashes native evidence or parses a +camera index synchronously. + +The session menu uses three operator indicators: + +| Indicator | Catalog/preparation state | Meaning | +| --- | --- | --- | +| Solid green + `Готово` | `ready` | Verified RRD is available. | +| Pulsing green + `Обработка` | `queued`, `validating`, `exporting`, `finalizing` | Background preparation is active. | +| Dim gray + `Ошибка` | failed, cancelled, non-replayable or invalid session | No launchable recording; inspect the message or retry. | + +Saved-session status never uses yellow or red. Switching sessions aborts only +the obsolete browser poll; it does not cancel the process-owned conversion. +A queued job may legitimately wait behind another export and therefore uses the +overall preparation deadline rather than the active-export heartbeat timeout. + +The first preparation of a long capture can take time because every decodable +point/pose message is projected into RRD. Later openings reuse a verified, +digest-bound cache. Derived RRD cache v6 is intentionally incompatible with v4 +and v5 because only v6 guarantees that the payload itself contains a real +`session_time = 0` anchor; older generations are rebuilt once in the background. +The browser may use the strict `source_url` with `If-Match`, while the embedded +Rerun loader uses the canonical `viewer_source_url` whose lowercase SHA-256 +`generation` query is bound to the same launch descriptor. A missing or stale +generation fails with `412`; a matching URL is served with exact length, strong +ETag and private immutable/no-transform caching. Every cache pin is released when the +HTTP response completes, disconnects or fails, so an interrupted switch cannot +make a derived recording permanently non-evictable. A bounded 120-second launch +reservation also protects the artifact between the verified launch response and +the WebViewer's subsequent RRD request, including a cold WASM startup; it +expires automatically if that request never arrives. + +## Storage roots + +By default, both catalog state and new source evidence are private to the +checkout: + +```text +.runtime/mission-core/ +├── mission-core.sqlite3 +├── mission-core.sqlite3-shm +├── mission-core.sqlite3-wal +├── evidence/ +│ └── sessions/ +│ ├── .current_session # present only while a writer owns the root +│ └── _viewer_live/ +│ ├── captures/ # native MQTT source evidence +│ └── media/ # canonical camera archives +└── recordings/ + ├── .export.lock # cross-process conversion/eviction lock + └── / + ├── scene.rrd + └── scene.rrd.cache.json +``` + +`MISSIONCORE_DATA_DIR` relocates the catalog and derived cache. Unless it is +overridden separately, new evidence is written to +`$MISSIONCORE_DATA_DIR/evidence/sessions`. `MISSIONCORE_EVIDENCE_DIR` can select +another absolute MQTT evidence root. In the current K1 integration the camera +gateway additionally confines its recording directory to the repository +checkout, so a full point-plus-camera acquisition must keep the evidence root +inside that checkout. An external evidence volume currently supports MQTT +capture/cataloging but camera archival will fail closed until the gateway gets +a separately attested storage root: + +```bash +export MISSIONCORE_DATA_DIR=/absolute/private/path/mission-core +# Full K1 point-plus-camera evidence must currently remain below the checkout. +export MISSIONCORE_EVIDENCE_DIR=/absolute/path/to/NODEDC_MISSION_CORE/.runtime/mission-core/evidence/sessions +export MISSIONCORE_RRD_CACHE_MAX_BYTES=8589934592 +export MISSIONCORE_RRD_FREE_SPACE_RESERVE_BYTES=2147483648 +uv run k1link serve +``` + +The directories and derived recording cache use owner-only permissions where +the host filesystem permits them. A new acquisition writer is assigned only a +direct child of the private evidence root; it no longer writes a new run to the +repository-level `sessions/` directory. + +Repository-level `sessions/*_viewer_live` runs are a legacy, import-only source +for the catalog. They are discovered and confined in place: refresh does not +copy, move or migrate their large payloads, and no current writer is assigned +that root. Startup may recovery-seal an already existing, incomplete canonical +camera epoch there by preserving its valid segment prefix and writing an +`interrupted` summary; this is evidence recovery, not a new acquisition write. + +Never add `.runtime/`, `sessions/`, raw captures, RRD files or camera media to +Git. They can contain mapped interiors, trajectories and identifiable images. + +## Session source of record + +For the current K1 profile: + +```text +MQTT callback + ├─ durable native .k1mqtt + aligned metadata (source of record) + └─ bounded latest-wins Rerun live preview (disposable) + +selected RTSP producer + ├─ durable init + fMP4 segments + JSONL index (camera evidence) + └─ bounded WebSocket/MSE preview (disposable) +``` + +The live preview is intentionally allowed to drop frames under load. Native +point/pose evidence and camera archive writes do not traverse that queue. + +Native `.k1mqtt` bytes with aligned metadata and canonical camera fMP4 archives +are the evidence source of truth. The derived RRD contains every decodable point +and pose frame from that source, but remains a rebuildable view rather than an +evidence master. A cache entry is rebuilt only when native source +identity/digests change or an incompatible derived-data export revision is +introduced. A UI blueprint or workspace-layout revision never invalidates or +rewrites the data RRD. Cache v4/v5 payloads are not reusable as v6 because they +do not guarantee the real zero-time anchor. + +Expensive cache misses run through the bounded preparation worker and one global +cross-process export gate to cap peak RAM, CPU and temporary-disk use. Crash +leftovers from candidates, exporter temporary files and staged replay prefixes +are scavenged under that lock before quota accounting. Ready cache hits and +active response leases do not wait behind that gate. The derived cache has an +8 GiB default quota, preserves a 2 GiB default filesystem reserve and evicts +least-recently-used RRDs only; it never deletes native evidence. + +## Camera archive contract + +New acquisitions archive each selected source and codec epoch below the same +private evidence session: + +```text +/sessions//media//epoch-1/ +├── init.mp4 +├── segments/ +│ ├── 1.m4s +│ └── ... +├── index.jsonl +└── summary.json +``` + +Each index row binds a segment sequence, byte length, SHA-256 and host arrival +epoch/monotonic timestamps. Camera durability uses a +`per-segment-fsync` policy: `init.mp4` and every complete media segment are +individually fsynced and their directory entries synchronized before the +matching JSONL index row is committed. The index and an atomic `interrupted` +checkpoint summary are fsynced before the append returns. The old interval and +byte constructor options are compatibility-only and cannot weaken this +segment-bound RPO. A power failure may still lose or leave uncommitted the +fragment currently being produced; it cannot make a committed index row point +past durable media. Camera windows may close or reconnect without terminating +archival while the owning acquisition remains active. + +Synchronization is `host-arrival-best-effort`: LiDAR/MQTT and camera segments +share the Mac host clock boundary, but K1 sensor exposure time and LiDAR firing +time are not proven to use a shared device clock. Do not infer frame-accurate +calibration from the playback timeline. + +Finalized media is prepared once by the same process-owned background job that +materializes the RRD. The gateway records host time only after a complete +`moof+mdat` fragment has arrived, so that timestamp is an availability/end +anchor, never a fragment-start timestamp. Preparation reads and SHA-verifies +every fragment, parses bounded ISO-BMFF timing tables (`mdhd`, `trex`, `tfhd`, +`trun`), anchors the epoch at +`max(0, first_arrival - first_fragment_duration)`, and sets its end to that start +plus the checked sum of every decoded fragment duration. The declared interval +therefore has exactly the duration MSE is expected to expose and is not stretched +by host scheduling jitter. + +Epoch arrivals must be strictly monotonic; epoch intervals must be finite, +monotonic and non-overlapping. Replay v2 also requires every media interval to +fit inside the spatial RRD interval with a 50 ms numeric tolerance; it fails +preparation instead of clamping unreachable evidence. A future replay v3 must +separate `spatial_range` from a session-wide union range before out-of-RRD camera +coverage can be navigated. A missing, ambiguous, oversized or otherwise +unparseable timing table fails preparation; the archive remains evidence but is +never advertised as seekable media. + +The prepared path-free v2 descriptor and full source stat identity (native raw +and timing metadata plus every camera summary, index, init and segment file) are +written under the private derived cache with a schema, generation and checksum. +Publication uses a private temporary file, file and directory `fsync`, and atomic +rename; startup scavenges crash-left temporary files. A restart reuses this +sidecar after confined O(n) stat validation, without rereading, hashing or parsing +media. A missing, corrupt or stale sidecar is rebuilt only by the background +worker. No replay, status, manifest or payload request performs conversion or +recalculates these intervals. + +Every derived RRD contains a real `session_time = 0` row at the internal +`/__mission_core/session_origin` entity. The anchor is deliberately outside +`/world`, so it establishes the actual recording time range without creating a +3D layer or drawable scene object. Preparation verifies the derived recording +against the declared spatial range rather than relying on summary metadata +alone. + +Sessions made before this archive contract have no recoverable video even if a +camera preview was visible at the time. In particular, the 2026-07-16 browser +camera acceptance run retained point/pose evidence only. + +## Crash and interruption behavior + +- SQLite uses WAL, foreign keys and full synchronous durability. +- Before creating a session directory, the acquisition takes an exclusive, + cross-process lease by atomically creating and locking + `/.current_session`. The marker contains only the direct child + session name. While the lock is held, a second writer fails closed and + discovery excludes that in-progress session from replay. +- A process exit releases the operating-system lock. On the next service start, + stale-marker recovery removes the marker only after it can take the lock and + revalidate the marker inode without following symlinks. It never deletes the + interrupted session directory; normal catalog recovery then evaluates the + durable prefix. A locked/live or suspicious marker is left untouched. +- Native MQTT writes use a bounded group commit: at most 0.5 seconds, 4 MiB or + 32 messages per group. Raw bytes are fsynced before their metadata rows are + written and fsynced. This is a bounded RPO, not a zero-loss guarantee: a hard + process or power failure may discard the final uncommitted group. Raw bytes + from an interrupted commit window may survive beyond the durable metadata; + replay uses only the last validated metadata-aligned raw boundary. +- A native capture with a missing final summary is accepted only when the raw + and metadata prefix is bounded, aligned and structurally valid. It is cataloged + as `interrupted`, never silently promoted to `ready`. +- A non-newline metadata crash tail can be ignored. Newline-terminated or + mid-file corruption fails closed. +- Camera recovery retains a contiguous valid segment prefix, quarantines + non-contiguous/orphan fragments instead of deleting them, rebuilds the index + when necessary and writes an `interrupted` summary. Unindexed bytes are not + presented as valid media. +- If a valid normal summary appears later, repeat discovery updates the same + session to `ready`. +- Materialization detects a native capture changing during export and refuses + to publish the derived RRD. It reopens the prepared raw source with no symlink + following inside the cataloged session roots, and interrupted recordings are + materialized from the last validated raw/metadata boundary only. +- Internal preparation cancellation is cooperative. A queued job cancels + immediately; an active job stops at a safe checkpoint, removes its candidate + and never publishes a partial RRD. The operator UI does not cancel automatic + preparation when switching sessions or closing a tab: preparation belongs to + the application worker, not to an HTTP request. +- `failed` and `cancelled` are terminal, retryable states. The API exposes a + sanitized error, and an explicit retry creates a fresh job for the current + source identity. A stalled poll or failed switch leaves the current scene + mounted; the latest operator selection wins over obsolete responses. +- A lifecycle shutdown marks only its own interrupted work for automatic + reconciliation after restart. An operator cancellation and a genuine failed + export remain terminal and are never retried forever by the reconciler. + +These rules provide crash recovery, not replication. A single host disk failure +can still destroy local data. Vehicle deployment must add independent onboard +and control-station copies, capacity monitoring and a documented retention +policy. + +## HTTP API + +All paths are same-origin and expose opaque identifiers only: + +```text +GET /api/v1/observation-sessions?limit=3 +GET /api/v1/observation-sessions/{id} +POST /api/v1/observation-sessions/{id}/replay +GET /api/v1/observation-sessions/{id}/recording-preparation +DELETE /api/v1/observation-sessions/{id}/recording-preparation +GET /api/v1/observation-sessions/{id}/recording.rrd +POST /api/v1/observation-sessions/{id}/blueprint.rrd +GET /api/v1/observation-sessions/{id}/media/{artifact}/manifest +GET /api/v1/observation-sessions/{id}/media/{artifact}/epochs/{n}/init.mp4 +GET /api/v1/observation-sessions/{id}/media/{artifact}/epochs/{n}/segments/{m}.m4s + +GET /api/v1/workspace-layouts/observation.spatial +PUT /api/v1/workspace-layouts/observation.spatial +``` + +The catalog embeds each replayable session's preparation state and progress. +`POST .../replay` returns either a verified replay v2 launch document or HTTP +202 with the preparation v1 document, `Location`, `Retry-After`, an exact quoted +preparation `ETag` and a same-origin status URL. Polling +`GET .../recording-preparation` sends that value as `If-Match` and returns HTTP +202 while the job is active, HTTP 409 for retryable `failed`/`cancelled` states, +and the launch document only when the same job and artifact are ready; every +response echoes the same `ETag`. `DELETE` also requires `If-Match`, so a stale +tab cannot cancel a replacement job. Conversion is never executed synchronously +by a replay-open request. Playback speed and loop are request-local launch +policy; they are not stored on or shared through a preparation job. + +Production `recording.rrd` access is bound to the launch generation. The client +can use the strict query-free URL with exact strong +`If-Match: "sha256:"`; a request with neither an `If-Match` nor a +generation returns 428. The embedded Rerun receiver instead gets only the +canonical `viewer_source_url = source_url + "?generation="`. +The server returns 412 for a malformed or replaced generation before opening +the body. A successful response echoes the strong ETag, exact `Content-Length`, +`application/vnd.rerun.rrd` and immutable private/no-transform cache policy. +Rerun's native HTTP receiver owns incremental decoding; `LogChannel.send_rrd` +is reserved for independently complete RRD payloads such as the small generated +blueprint and must never receive arbitrary HTTP byte slices. The canvas, +timeline, controller and autoplay remain closed until the decoded spatial range +exactly matches the launch descriptor and every declared camera is ready. API +documents never contain local paths. Layout updates require the quoted current +revision in `If-Match`; stale writers receive HTTP 412 rather than overwriting +another saved profile. + +Recorded media routes expose only opaque catalog identifiers and ordinal codec +epochs. The required `missioncore.observation-recorded-media/v2` manifest carries +a strong `generation_sha256`, exact JS-safe aggregate `byte_length`, and finite +`timeline_start_seconds` / `timeline_end_seconds` for every epoch. Epoch ends +participate in the generation digest. Every epoch also declares its init byte +length/digest and a complete contiguous segment list with sequence, URL, byte +length and digest. The launch source repeats the same aggregate `byte_length` +and uses exactly `max(epoch.timeline_end_seconds)` as its end; the spatial RRD +end must never pad camera coverage. The browser cross-checks launch, manifest and +component lengths. The current browser laboratory policy accepts at most 16 +camera sources, 128 MiB per source and 512 MiB across the session. This launch +preflight finishes before the first camera manifest/init/segment GET; admitted +cameras are then downloaded and decode-probed one at a time. Verified raw +buffers remain immutable so route changes and player remounts cannot append +emptied data. This bounded in-memory strategy is deliberate for the laboratory +milestone; an OPFS-backed sealed-generation cache is the next scaling step for +larger rigs and must preserve the same launch/manifest/hash admission contract. + +The manifest response ETag is the exact generation, and the manifest GET itself +requires that generation in `If-Match`. Init and media GETs likewise require the +descriptor's exact SHA `If-Match`, then open through a confined directory file +descriptor with no symlink following, verify the declared SHA-256 and serve +exact-length bytes with immutable private `no-transform` caching and byte-range +support. Physical source ids, RTSP addresses and storage paths never cross the +API boundary. + +## Current limit + +The spatial RRD, trajectory and archived fMP4 cameras use the same operator +scrubber now. Camera epochs are aligned to zero-based `session_time` from the +shared host-arrival monotonic clock and rendered through MSE. The client selects +an epoch only inside its declared inclusive interval and verifies that the +decoded MSE seekable duration covers that interval. This remains best-effort +correlation: codec PTS and K1 sensor exposure time are not proven to share the +LiDAR clock. A codec epoch whose init segment does not expose a browser-supported +codec or whose timing cannot be proven remains retained evidence and fails +closed in the UI/background preparation. +Historical sessions with no canonical camera archive honestly show no recorded +video. Recorded blueprints can change accumulation, grid visibility, point and +trajectory visibility, point radius and a uniform custom point color without +rewriting the recording. The client deliberately has no progressive recorded +mode: the viewport, timeline and autoplay gate remain closed until the complete +declared RRD has been received and decoded. Height, intensity, distance and RGB +palettes remain baked into current RRD rows; fully dynamic recoloring requires +exporting the corresponding scalar components in a future recording schema. + +## Verification checkpoint — 2026-07-17 + +The repository state described above passed the complete local pre-push gate: + +- `uv sync --frozen --group dev` completed against the locked Python graph; +- `uv run pytest`: **284 passed**; +- `uv run ruff check .`: clean; +- project mypy and strict plugin-SDK mypy: clean across 51 and 12 source files; +- the XGRIDS K1 profile loader and emitted v0alpha2 JSON schema validated; +- clean `npm ci` ran the required Rerun 0.34.1 postinstall patch; +- frontend unit suite: **111 passed, 0 failed**; +- TypeScript project build and Vite production build completed; +- the generated `dist/index.html` and non-empty Rerun WASM artifact were verified. + +Vite still reports its expected large-chunk warning for the embedded Rerun +viewer/WASM payload. That is a packaging optimization item, not a failed gate. +No retained physical K1 session contains the new canonical camera archive yet, +so real recorded-camera playback remains an explicit hardware acceptance test. diff --git a/docs/adr/0006-vendor-neutral-observation-sources-and-live-only-timeline.md b/docs/adr/0006-vendor-neutral-observation-sources-and-live-only-timeline.md index adb0d04..62e35a8 100644 --- a/docs/adr/0006-vendor-neutral-observation-sources-and-live-only-timeline.md +++ b/docs/adr/0006-vendor-neutral-observation-sources-and-live-only-timeline.md @@ -2,6 +2,11 @@ Status: accepted for the experimental Mission Core runtime, 2026-07-16. +Scope update, 2026-07-17: the live device descriptors in this ADR remain +live-only. ADR 0008 adds a separate host-owned recorded-source catalog, +seekable `session_time` playback and workspace layout; it does not retroactively +turn a live RTSP lease into recorded evidence. + ## Context The spatial scene and camera workspace must accept one device, a custom rig or a @@ -68,10 +73,12 @@ RTSP/FFmpeg generation before a new browser lease is issued. The local laboratory adapter now terminates RTSP/RTP with FFmpeg, copy-remuxes H.264 High L4 without transcoding and publishes complete bounded fMP4 segments -to a generic MSE player. Portable FFmpeg packaging, multi-consumer fan-out and -remote/WAN delivery are not claimed by this milestone. Synchronized rewind -becomes eligible only after persisted buffering and an evidenced mapping between -the RTP 90 kHz clock and the spatial stream clock. See ADR 0007. +to a generic MSE player. ADR 0008 subsequently added acquisition-owned durable +segments, a prepared recorded-media manifest and host-arrival-best-effort rewind +on the saved-session timeline. This is not proof of a shared device clock or +frame-accurate LiDAR/camera alignment. Portable FFmpeg packaging, +multi-consumer fan-out and remote/WAN delivery remain open. See ADR 0007 and +ADR 0008. ## Code anchors diff --git a/docs/adr/0007-k1-camera-preview-copy-remux-gateway.md b/docs/adr/0007-k1-camera-preview-copy-remux-gateway.md index 206ca73..9808204 100644 --- a/docs/adr/0007-k1-camera-preview-copy-remux-gateway.md +++ b/docs/adr/0007-k1-camera-preview-copy-remux-gateway.md @@ -2,6 +2,11 @@ Status: accepted for the local laboratory runtime, 2026-07-16. +Supersession note, 2026-07-17: the live copy-remux and lease decisions remain +active. ADR 0008 extends the same acquisition-owned producer with durable fMP4 +epochs and a generic recorded player. That extension does not weaken the +single-live-producer or opaque transport boundary defined here. + ## Context The exact K1 firmware 3.0.2 profile exposes two RTSP/TCP H.264 preview paths. @@ -73,16 +78,22 @@ timeline once when the recording opens and does not poll or force the cursor. - Same-origin local WebSocket delivery is not the future WAN transport. A later Edge/Control Station split should use an authenticated WebRTC/WHEP or equivalent congestion-aware media plane behind the same descriptor contract. -- Camera recording, synchronized rewind, calibration, frustums, panoramic - stitching and full-resolution raw imagery remain unavailable. +- Durable camera recording and host-arrival-best-effort saved-session rewind are + implemented under ADR 0008, but no retained real K1 session made after that + archive contract has completed physical recorded-camera acceptance yet. +- Device-clock synchronization, calibration, frustums, panoramic stitching, + full-resolution raw imagery and remote multi-consumer delivery remain + unavailable. - A longer physical soak test is still required to characterize sustained preview backpressure and rendering cost on target Edge hardware. ## Code anchors - `src/k1link/web/xgrids_k1_camera.py` +- `src/k1link/web/camera_archive.py` - `src/k1link/web/xgrids_k1_facade.py` - `apps/control-station/src/components/MseFmp4WebSocketPlayer.tsx` +- `apps/control-station/src/components/RecordedFmp4Player.tsx` - `apps/control-station/src/core/observation/useObservationLayout.ts` - `apps/control-station/src/device-plugins/xgrids-k1/observationSources.ts` - `tests/test_xgrids_camera_gateway.py` diff --git a/docs/adr/0008-durable-observation-sessions-and-workspace-layout.md b/docs/adr/0008-durable-observation-sessions-and-workspace-layout.md new file mode 100644 index 0000000..9168b17 --- /dev/null +++ b/docs/adr/0008-durable-observation-sessions-and-workspace-layout.md @@ -0,0 +1,230 @@ +# ADR 0008: durable observation sessions and workspace layout + +- Status: accepted for the laboratory Control Station +- Date: 2026-07-17 +- Scope: host observation storage, recorded spatial playback, camera evidence, + and operator workspace layout + +## Context + +Mission Core previously retained K1 live point/pose captures as ignored +laboratory directories, while the browser showed only the current process-wide +Rerun stream. Camera preview was copy-remuxed for a connected browser but was +not part of the durable session. A browser refresh, local service restart or an +unexpected process exit therefore made a completed run difficult to discover +and made camera history impossible to recover. The disk icon in the spatial +workspace also had no stable distinction between saving sensor evidence and +saving an operator's layout. + +These are two separate product records: + +1. an **observation session** is evidence from all sources active during one + acquisition and must be recorded automatically; and +2. a **workspace layout profile** is an operator preference and is saved + explicitly. + +Conflating them would allow a UI action to decide whether evidence exists and +would make a layout save duplicate or mutate sensor data. + +## Decision + +### Host-owned catalog and private evidence root + +The Control Station owns a private storage root selected by +`MISSIONCORE_DATA_DIR`; the repository-local fallback is +`.runtime/mission-core/`. The root is outside Git and contains a SQLite catalog +plus derived playback artifacts. New source evidence is written to the root +selected by `MISSIONCORE_EVIDENCE_DIR`, falling back to +`/evidence/sessions`. The current K1 camera gateway still +requires that root to remain inside the repository checkout; an external root +can retain MQTT evidence but camera archival fails closed. Treat support for an +independently attested external camera-evidence root as an open extraction item, +not as implemented behavior. SQLite uses foreign keys, WAL journaling and full +synchronous durability. Public APIs expose opaque session identifiers and +metadata, never filesystem paths. + +Current acquisition writers create only direct children of the private +evidence root. Existing repository-level `sessions/*_viewer_live` runs are a +legacy import-only source: they are indexed in place, never copied or moved, +and are not assigned to a new writer. Their native `.k1mqtt` capture remains +the source of record. Import is repeatable and runs in a background reconciler, +so completed evidence becomes visible without a restart and HTTP catalog reads +never scan the evidence roots. Startup recovery may +seal an already incomplete canonical camera epoch under the legacy root; it +does not start or append a new acquisition there. + +Before a new session directory is created, the writer atomically creates and +locks `/.current_session`. The marker names the direct child and +prevents a second process from writing or replaying the active session. Process +exit releases the operating-system lock. On startup, an unlocked stale marker +is removed only after no-follow inode revalidation; the interrupted evidence +directory is preserved for normal recovery and cataloging. + +If a process exits after raw and metadata creation but before a final summary, +the importer accepts only a bounded, structurally valid prefix with aligned +metadata and marks the run `interrupted`. It does not invent completion data. +When the valid final summary later appears, repeat import promotes the same +session to `ready`. + +Native MQTT persistence uses a bounded group commit of at most 0.5 seconds, +4 MiB or 32 messages. Raw bytes are fsynced before their aligned metadata rows. +The final uncommitted group may be lost, and a raw tail may survive without +committed metadata; replay stops at the last validated aligned boundary. + +### Recorded spatial playback + +When a native capture is normally sealed or recovery-sealed, a bounded +single-worker reconciler automatically prepares its private derived `.rrd`. +Preparation is a backend lifecycle and is never executed by an HTTP replay +request. Reopening, seeking, switching tabs or changing display settings reads +the already prepared artifact and does not rerun conversion. Export is lossless +with respect to every decodable point and pose message in the native capture; +it does not pass through the bounded live-preview queue. The RRD uses the +recording-local `session_time` duration timeline, while the receive wall clock +remains secondary evidence. + +Derived files are published atomically, hashed and accompanied by a cache +sidecar bound to the native file identity and digest. A process-owned queue and +cross-process file lock prevent overlapping exporters; crash candidates are +scavenged before quota accounting. A stale or inconsistent cache is rejected +and rebuilt. The browser receives the RRD only through the same-origin opaque +session API. It keeps the viewport and timeline hidden until the complete +declared recording has been downloaded and decoded, then publishes the scene +atomically and starts playback. + +### Camera recording + +Camera acquisition is owned by the device acquisition session, not by a +browser window. Preview remains bounded and disposable; archive writes occur +before preview fan-out. Closing or refreshing a camera window must not stop the +archive while acquisition is active. + +Each codec epoch is stored below the observation session as an initialization +segment, independently addressable fMP4 media segments, an append-only JSONL +index and an atomic summary. Host arrival epoch and monotonic timestamps are +recorded for every segment. The `per-segment-fsync` contract makes `init.mp4` +and each complete media segment plus its directory entry durable before its +matching index row is committed; the index and interrupted checkpoint are +fsynced before append returns. Therefore camera RPO is bounded by the fragment +currently being produced, not by an interval or byte batch. Startup recovery +retains a contiguous valid prefix, quarantines orphan fragments and marks an +unfinalized epoch `interrupted`. + +The background recording-preparation worker also prepares immutable camera +descriptors before publishing `ready`. Complete-fragment arrival timestamps are +end/availability anchors, so the worker SHA-verifies and parses every fragment, +anchors the first decoded sample at `first_arrival - first_duration`, and derives +coverage from the exact sum of ISO-BMFF sample durations. It computes finite +non-overlapping epoch coverage ends and exact aggregate bytes, then binds both to +recorded-media manifest v2's generation digest. Replay v2 additionally requires +camera coverage to stay inside the spatial RRD range with a 50 ms tolerance; a +future replay v3 will need distinct spatial and session-union ranges. + +The path-free descriptor is atomically persisted in the private derived cache +with its checksum and a stat identity covering the native timing origin and +every camera summary, index, init and segment file. Restart performs confined +stat validation and reuses an unchanged descriptor without reading/hashing media; +a missing, corrupt or stale sidecar is rebuilt in the background. An unparseable +or ambiguous fragment fails readiness rather than exposing a fake seekable +camera. Replay requests consume the prepared descriptor and never repeat media +conversion or timing analysis. Launch and manifest aggregate byte counts must +agree before the browser admits camera payload downloads. + +The derived RRD writes an actual zero-time anchor at the internal +`/__mission_core/session_origin` entity. Keeping it outside `/world` prevents a +synthetic visualization layer while ensuring that the decoded RRD timeline, +not merely its summary document, begins at session time zero. This changes the +derived payload contract: cache v6 rejects v4/v5 sidecars and performs one +background rebuild instead of silently reusing an archive without that row. + +Replay v2 exposes two paths to the same pinned generation. `source_url` retains +the strict `If-Match: "sha256:…"` contract for Mission Core clients; +`viewer_source_url` appends the exact lowercase SHA-256 as a `generation` query +for Rerun's native URL loader, which cannot attach custom headers. The server +accepts either precondition, rejects a stale query with `412`, and keeps the +matching file pinned for the entire immutable/no-transform response lifecycle. + +This is host-arrival synchronization, not proven sensor-clock synchronization. +It is sufficient to correlate current K1 point/pose and preview evidence at the +known clock boundary, but must not be described as frame-accurate optical/LiDAR +alignment. + +Historical runs made before this ADR contain no camera archive. The fact that a +camera was visible in the old browser preview does not make video recoverable; +the catalog must not manufacture a camera modality for those sessions. + +### Workspace layout + +The first disk action in the spatial workspace saves only the versioned +`observation.spatial` layout document: + +- scene display settings; +- open tool windows and their z-order; +- visible dynamic source identifiers; +- active floating source; +- source-window rectangles normalized to the observed viewport. + +The document uses optimistic revision control (`ETag` and `If-Match`) and is +restored automatically on the next opening. Unknown source identifiers remain +desired state so a temporarily disconnected camera can recover its saved +window when that source returns. The save action never starts, completes or +modifies an observation session. + +### API boundary + +The stable host routes are: + +- `GET /api/v1/observation-sessions` for the recent catalog; +- `GET /api/v1/observation-sessions/{id}` for modality/artifact metadata; +- `POST /api/v1/observation-sessions/{id}/replay` for a ready launch or a + preparation handle; +- `GET /api/v1/observation-sessions/{id}/recording-preparation` for exact-job + status polling; +- `GET /api/v1/observation-sessions/{id}/recording.rrd` for seekable spatial data; +- opaque manifest/init/segment routes below + `/api/v1/observation-sessions/{id}/media/{artifact}` for recorded cameras; +- `GET|PUT /api/v1/workspace-layouts/observation.spatial` for the layout profile. + +The session API precedes the static frontend mount. Recording paths and storage +roots are server implementation details. + +## Consequences + +- A completed or interrupted spatial run is discovered and prepared + automatically, then can be replayed after browser and service restarts. +- A service crash can lose the final uncommitted MQTT group and the camera + fragment currently being produced; it cannot make an index point beyond + durable camera media or replay a raw tail beyond aligned MQTT metadata. +- Raw K1 evidence remains authoritative and derived RRD files can be discarded + and regenerated. +- Browser performance no longer controls whether camera evidence is retained. +- Layout restoration is portable across viewport sizes and dynamic device + source catalogs. +- Replay builds a separate recorded-source catalog, so an active device camera + can never be mixed into the evidence of a saved session. Archived fMP4 codec + epochs follow Rerun play/pause/seek through host-arrival best-effort offsets. +- A preparation ETag binds polling and cancellation to one exact job; a short + launch reservation prevents cache eviction between the launch response and + the viewer's first RRD request. Browser session switching never cancels the + shared backend preparation. +- Storage capacity, retention, quota, encryption-at-rest and export policy are + still deployment responsibilities; this ADR does not silently delete source + evidence. +- Recorded camera playback is confined to canonical archives with a supported + fMP4 codec declaration and provable sample duration. Missing historical media, + unsupported codec epochs and ambiguous timing fail closed; none is substituted + with a live preview. + +## Rejected alternatives + +- **Use browser cache or IndexedDB for sessions.** This loses the evidence when + the browser profile is cleared and cannot guarantee acquisition durability. +- **Save a session only when the operator presses the disk icon.** This makes + safety evidence optional and fails on crashes or forgotten clicks. +- **Store only RRD.** This discards the native evidence needed to review future + decoders and protocol assumptions. +- **Keep camera acquisition coupled to WebSocket viewers.** A UI disconnect + would stop recording and recreate the observed data-loss bug. +- **Copy every legacy run into the private catalog root.** This doubles large + artifacts without improving integrity; confined zero-copy indexing is + sufficient for the current laboratory store. diff --git a/plugins/xgrids-k1/README.md b/plugins/xgrids-k1/README.md index dae671f..7363ede 100644 --- a/plugins/xgrids-k1/README.md +++ b/plugins/xgrids-k1/README.md @@ -24,13 +24,16 @@ The plugin owns: - firmware-scoped protobuf/LZ4 and legacy codecs; - normalization of K1 point cloud and pose, plus raw-only preservation of the still-undecoded status and heartbeat channels; +- exact-profile left/right RTSP producer selection, H.264 copy-remux preview and + the handoff of acquisition-owned init/fMP4/index evidence to the host archive; - K1-specific operator instructions and compatibility tests. The plugin does not own: - Mission Core navigation, fleet, missions, users, roles, or audit; - the generic spatial scene or Rerun Web Viewer; -- platform storage, remote transport, or other devices. +- the host observation catalog, RRD preparation/cache, recorded-camera player, + workspace layout, platform storage, remote transport, or other devices. Until extraction is complete, `src/k1link` is the compatibility source of truth and every move into this package must preserve replay and real-device