SILVER/NODEDC_MISSION_CORE
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

feat: activate self-hosted lidar spatial scene

Browse Source
This commit is contained in:
DCCONSTRUCTIONS
2026-07-16 10:36:51 +03:00
parent 6402601d6d
commit 904c6ea13a
27 changed files with 2009 additions and 646 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/06_K1_LIVE_VIEWER.md
+209 -83
View File
@@ -1,9 +1,11 @@
# K1 live console and Foxglove bridge
# K1 live console and embedded Rerun bridge
Status: alpha implementation for the verified firmware-3 MQTT streams. It uses
real K1 data and does not generate a placeholder cloud, pose or latency value.
Status: the Rerun alpha milestone is implemented for the verified firmware-3
MQTT streams. Live capture and replay use real K1 point-cloud and pose messages;
the application does not generate a placeholder cloud, trajectory, camera frame
or latency value.
## Architecture
## Active device-to-scene path
```text
K1 lio_pcl / lio_pose
@@ -11,139 +13,263 @@ K1 lio_pcl / lio_pose
v
read-only MQTT subscription on TCP 1883
|
+--> raw .k1mqtt + JSONL + SHA-256 summary (first)
+--> raw .k1mqtt + JSONL + SHA-256 summary (written first)
|
v
bounded latest-wins preview queue (32 messages)
|
v
raw-LZ4/protobuf decoder --> foxglove.PointCloud / Pose / SceneUpdate
reviewed raw-LZ4/protobuf decoders
|
v
ws://127.0.0.1:8765 --> Foxglove 3D
Rerun Points3D + Transform3D + LineStrips3D
|
v
Rerun gRPC/proxy on TCP 9876
|
+--> rerun_grpc_url in REST/WebSocket state
|
v
self-hosted @rerun-io/web-viewer inside NODE.DC Control Station
React console <-- REST + WebSocket state --> FastAPI on 127.0.0.1:8000
```
The Paho MQTT callback never decodes the point cloud. It writes and flushes the
raw frame and metadata, then enqueues a preview reference. If visualization cannot
keep up, the oldest queued preview is discarded while the raw capture continues.
The Paho MQTT callback does not decode or render the point cloud. It first
writes and flushes the raw frame and metadata, then enqueues a preview message.
If visualization cannot keep up, the oldest queued preview is discarded while
the raw capture continues. Rerun work stays on the dedicated publisher thread
and cannot block raw-first evidence capture.
The current live/replay runtime instantiates only `RerunBridge`. The former
Foxglove implementation is not a parallel runtime and does not listen on TCP
8765; it is retained only as a legacy regression module and test oracle.
## Start the local application
Prerequisites are the repository-local Python environment and Node.js 20 or
newer. No global Python package or system component is installed.
Prerequisites are the repository-local Python environment and Node.js 20.19+ or
22.12+. No global Python package or system component is installed.
```bash
uv sync --group dev
cd apps/k1-viewer
npm install
npm run typecheck
npm run build
cd ../..
uv run k1link serve
```
Open `http://127.0.0.1:8000`. `k1link serve` intentionally exposes no LAN bind
option because its provisioning endpoint temporarily receives a Wi-Fi password.
The password is accepted only in the POST body, is never logged or persisted by
the connector, and is cleared from the React form after success.
Open `http://127.0.0.1:8000`. The static Control Station, REST API, WebSocket
state channel and credential endpoint bind to loopback. The Rerun gRPC server is
a separate listener with a different network boundary described under
[Network and security boundary](#network-and-security-boundary).
## Replay the captured proof
## Replay a reviewed capture
1. In **Session**, select **Replay capture**.
2. Enter the repository-relative path:
1. In **Парк → Локальное устройство**, choose the replay mode.
2. Enter a native `mqtt.raw.k1mqtt` capture or a reviewed four-column TSV,
for example:
```text
sessions/20260715T122850Z_live_power_cycle/captures/mqtt_scan_full_payloads_03.tsv
```
3. Use speed `1` and enable loop for initial viewer setup.
4. Start replay, then select **Open Foxglove 3D**.
5. In a Foxglove 3D panel, enable `/k1/points`, `/k1/pose` and
`/k1/trajectory`.
6. For the cloud, choose `intensity` as the color field and a Turbo/Rainbow
colormap or custom two-color gradient. Foxglove can also color by `x`, `y`,
`z` and its derived `<distance>`.
3. Select replay speed and optional looping, then start the session.
4. The runtime starts the process-wide Rerun `RecordingStream` and its
gRPC/proxy server on TCP 9876 before the first replay. Later sessions reuse it.
5. Open **Наблюдение → Пространственная сцена**. With the manual source field
empty, the embedded viewer automatically uses:
The reviewed TSV reader accepts exactly four columns: receive timestamp, topic,
declared payload length and hex payload. It verifies timestamp, UTF-8 topic,
declared length, hex encoding and allocation bounds. Native
`mqtt.raw.k1mqtt` captures are supported directly and use their sibling metadata
timestamps when present.
```text
rerun+http://127.0.0.1:9876/proxy
```
6. Use **Отображение** and **Слои** to adjust the real cloud and trajectory.
7. Stop the session when finished. Acquisition and session metrics stop, while
the embedded viewer and `rerun_grpc_url` remain ready. A later session resets
scene-local state and reuses the same listener without reloading the page.
The TSV reader accepts exactly receive timestamp, topic, declared payload length
and hexadecimal payload. It verifies timestamp, UTF-8 topic, declared length,
hex encoding and allocation bounds. Native `.k1mqtt` captures are supported
directly and use their sibling metadata receive timestamps when present.
## Connect and stream live
1. Power K1 to its normal steady-green standby state.
2. Confirm the manual power checklist in **Connect**.
3. Run the real six-second BLE scan and select the K1 candidate.
4. Enter the existing router SSID/password and press **Provision Wi-Fi &
connect**. That button is the explicit authorization for one reviewed 99-byte
provisioning write; the backend never retries automatically.
5. When K1 reports a non-AP private address, press **Start live stream**.
6. Open Foxglove before scanning if convenient.
7. Double-click the physical K1 button to start scanning. Double-click again to
stop, wait for the LED to return to steady green, then stop the local session.
2. Confirm the manual power checklist in **Парк → Локальное устройство**.
3. Run the real six-second BLE scan and select the intended device from the
complete visible-device list.
4. Enter the existing router SSID/password and explicitly authorize the reviewed
provisioning write. The backend does not retry the write automatically.
5. When K1 reports a non-AP private address, start live reception.
6. Wait until the UI reports that the local Rerun bridge is ready. No manual
viewer URL is needed.
7. Open **Наблюдение → Пространственная сцена**.
8. Double-click the physical K1 button to start scanning. Real point frames and
pose/trajectory updates then appear in the embedded viewport.
9. Double-click K1 again to stop physical scanning, wait for steady green, then
stop the local session so captures and summaries are finalized.
Each live run creates an ignored `sessions/<UTC>_viewer_live/` directory with a
redacted manifest, operator notes, raw MQTT frames, per-message metadata and a
hash summary. No application request topic is published.
hash summary. The connector subscribes to the fixed report-topic allowlist and
does not publish an application request or modeling command.
## Published topics
## Automatic Rerun source and lifecycle
| Topic | Foxglove schema | Meaning |
On the first live or replay session, `RerunBridge`:
- creates an explicit `RecordingStream("nodedc_device_spatial")`;
- installs the default spatial blueprint;
- starts the gRPC/proxy server on TCP 9876 with a 512 MiB late-client buffer;
- reports `rerun+http://127.0.0.1:9876/proxy` only after the server is ready;
- accepts the local development and production browser origins used by this
repository;
- keeps the recording and URL alive until `k1link serve` shuts down.
At every session start it resets the trajectory, current point count, metrics,
blueprint and session-local visible time, then feeds the new source through the
same recording. This process-wide lifecycle is intentional: the Rerun 0.34.1
browser receiver can remain connected after canvas teardown, so restarting the
native listener on the same port is not a reliable session boundary.
The frontend prefers an operator-entered source when one is present; otherwise
it automatically uses `rerun_grpc_url` from backend state. A manual field can
still open another compatible `rerun+http://.../proxy` source or an RRD file
served over HTTP(S). No externally hosted viewer UI is involved:
`@rerun-io/web-viewer` and its WebAssembly runtime are bundled with the local
application. FastAPI carries control state only; the browser reads the point
stream directly from the Rerun gRPC/proxy endpoint.
## Rerun entities
| Entity | Rerun archetype | Meaning |
| --- | --- | --- |
| `/k1/points` | `foxglove.PointCloud` | metric XYZ float32 + uint8 intensity, stride 16 |
| `/k1/pose` | `foxglove.PoseInFrame` | current decoded K1 pose in the `map` frame |
| `/k1/trajectory` | `foxglove.SceneUpdate` | bounded cyan line strip, throttled while growing |
| `/k1/metrics` | JSON schema | latency, decode/publish time, FPS, points and drops |
| `/world` | `ViewCoordinates` | right-handed Z-up display convention |
| `/world/points` | `Points3D` | decoded metric XYZ with computed or available RGB colors |
| `/world/sensor_pose` | `Transform3D` + `TransformAxes3D` | current decoded translation, xyzw quaternion and pose axes |
| `/world/trajectory` | `LineStrips3D` | bounded path of up to 20,000 decoded poses |
Point coordinates remain `(x/scaler, y/scaler, z/scaler)` exactly as verified.
No axis swap, quaternion normalization or vehicle extrinsic is silently applied.
Only the low byte of `rgbi` is labeled intensity; interpreting the upper bytes
as RGB remains unverified.
Point count, frame rate, queue drops and measured pipeline time remain product
metrics in the outer Control Station; they are not logged as Rerun entities and
therefore cannot create additional automatic viewer panes.
## Latency semantics
Point coordinates remain `(x/scaler, y/scaler, z/scaler)` exactly as decoded.
No axis swap, quaternion normalization, scanner-to-vehicle extrinsic or SLAM
post-processing is silently applied. The point entity is in `/world` rather
than under the pose transform, so an already map-frame cloud is not transformed
twice. The physical K1 axes and a scanner-to-vehicle transform still require
calibration before vehicle mounting.
`pipeline_ms` / `mqtt_to_publish_ms` is measured with the host monotonic clock
from MQTT callback receipt through raw disk write, bounded queue wait, decode,
packing and Foxglove channel publish. Rolling p50/p95 values use the last 512
published decoded messages.
Only the low byte of firmware-3 `rgbi` has been verified as intensity. The
upper bytes are not labeled RGB. Legacy frames use RGB only when those decoded
fields are actually present.
This is the exact Mac pipeline latency, not yet sensor-photon-to-screen latency.
The K1 header timestamp epoch has not been proven, Foxglove rendering time is
outside the Python publisher, and display latency is not observable without a
clock-correlated device timestamp or high-speed-camera experiment. Foxglove
message time therefore uses host receive Unix time while durations use
`monotonic_ns`.
## Scene controls
The current product controls call `POST /api/viewer/settings`; backend state is
authoritative and the bridge applies the updated settings to subsequent
messages.
| Control | Implemented behavior |
| --- | --- |
| Point size | Rerun UI-point radius from 0.5 to 12.0; default 2.5 |
| Color attribute | intensity, height Z, distance from the point-coordinate origin, available RGB, or solid class/custom color |
| Palette | Turbo, Viridis, Plasma, grayscale or operator-selected custom color |
| Accumulation | sliding session-local `stream_time` window from 0 to 120 seconds; default 12 seconds |
| Point layer | shows or clears `/world/points` |
| Trajectory | shows or clears the decoded path; publication is throttled while it grows |
| Grid | updates visibility of the Rerun 3D line grid through the active blueprint |
The default 12-second accumulation is a viewer time range over real point
frames; it does not duplicate or alter the raw capture. An accumulation of zero
shows only the current frame. RGB mode falls back to the verified scalar
coloring path when the active point format has no RGB fields. The current
`class`/custom behavior is a selected solid color, not semantic segmentation.
Projection switching, a custom playback timeline, semantic object/mask layers,
camera frustums and persisted RBL/layout profiles are not wired yet and must not
be inferred from the implemented controls above.
## Time and latency semantics
The Rerun `capture_time` timeline uses
`StreamMessage.received_at_epoch_ns`: Unix time when the Mac received the MQTT
message. The K1 header timestamp epoch is not proven, so `capture_time` is not
a sensor acquisition timestamp. The viewer accumulation window uses
`stream_time`, assigned when the current live/replay run publishes a message;
therefore replayed historical timestamps do not mix two sequential sessions.
For live MQTT only, `pipeline_ms` / `mqtt_to_publish_ms` uses the Mac monotonic
clock from MQTT callback receipt through raw disk write, bounded queue wait,
decode, Rerun packing and the SDK log call. Replay and idle states expose no
latency value. Browser fetch, WebAssembly ingestion, GPU rendering and display
scanout are outside this measurement.
The displayed value is therefore Mac pipeline latency, not sensor-photon-to-
screen latency. End-to-end display latency requires a proven device clock or a
separate clock-correlated/high-speed-camera experiment.
## Network and security boundary
There are two different listeners:
- FastAPI, the WebSocket control plane and Wi-Fi credential endpoint bind only
to `127.0.0.1:8000`;
- Rerun gRPC/proxy on TCP 9876 currently binds **all network interfaces**, even
though the URL returned to the local browser contains `127.0.0.1`.
The Rerun endpoint has no connector-level authentication and no TLS. It may
expose point clouds, trajectory and mapped interiors to any host that can reach
the port. Use it only on a trusted laboratory LAN, block TCP 9876 at the router
and host firewall as appropriate, and never forward it directly to the public
Internet or cellular WAN. Remote operation requires an authenticated TLS reverse
proxy or another reviewed secure transport before deployment.
The 512 MiB Rerun server buffer limits retained late-client data but is not an
access-control mechanism. The listener and embedded viewer intentionally remain
ready between acquisition sessions. Stop `k1link serve` when the network
listener and its process memory must be closed unconditionally.
## Current boundaries
- Raw panoramic camera frames were not present on the observed MQTT report
topics; this milestone intentionally ships point cloud plus trajectory only.
- Physical double-click remains the start/stop control. A future MQTT modeling
- Raw panoramic camera frames were absent from the observed MQTT report topics.
This milestone contains point cloud and pose/trajectory; operational metrics
are rendered by the outer Control Station.
- Physical double-click remains the K1 scan start/stop control. Any MQTT command
publisher needs a separately reviewed state-changing profile.
- The React application opens the official Foxglove viewer over a local
WebSocket. It does not embed or fork the proprietary modern viewer; Foxglove
account/seat terms apply to that viewer independently of this connector.
- Exact path axes and a scanner-to-vehicle transform must be calibrated before
mounting on the unmanned platform.
- No terrain map, elevation model, obstacle segmentation, localization fusion,
mission planner or vehicle control is implemented by this viewer milestone.
- Exact coordinate axes and the scanner-to-vehicle transform remain a mounting
calibration task.
- A single local runtime owns TCP 9876. A port collision causes a visible bridge
startup error rather than silently selecting another source. The Rerun native
listener lives for the lifetime of `k1link serve` and is reused by later
sessions in that process; shutdown closes it explicitly.
## Verification checkpoint — 2026-07-15
## Verification checkpoint — 2026-07-16
The full captured 180-second scan was passed through the production Foxglove
bridge without a viewer-side substitute:
A powered K1 capture was passed through the active MQTT → decoder → Rerun gRPC →
embedded Web Viewer path without a viewer-side substitute:
| Check | Result |
| --- | ---: |
| MQTT messages read | 2,836 |
| `lio_pcl` frames published | 1,140 |
| `lio_pose` frames published | 1,215 |
| points packed and published | 4,165,862 |
| real MQTT messages processed | 80 |
| `lio_pcl` frames published | 38 |
| `lio_pose` frames published | 42 |
| points in the last cloud | 2,775 |
| decoder errors | 0 |
| final trajectory poses | 1,215 |
The same replay was then started through the FastAPI/React contract at `10x`.
The API exposed the loopback Foxglove URL, live point/pose metrics and non-zero
preview-drop accounting under deliberate acceleration, and released TCP 8765
after the stop request. This validates replay, overload behavior and lifecycle;
the next checkpoint is the powered K1 ideal-LAN live run.
The 38 point frames and 42 pose frames account for all 80 observed messages.
This proves current real-device decoding, bridge publication and embedded-viewer
delivery for point cloud plus trajectory. It does not prove a camera channel or
sensor-to-screen latency.
## Legacy Foxglove regression module
`src/k1link/viewer/foxglove_bridge.py` and its tests remain in the repository
to compare decoder/packing behavior and preserve earlier evidence. They are not
instantiated by `VisualizationRuntime`, are not the Control Station source, and
do not make TCP 8765 part of the current operator path.