199 lines
9.5 KiB
Markdown
199 lines
9.5 KiB
Markdown
# NDC XGRIDS K1 Connector
|
|
|
|
Pre-production research project for connecting an owner-controlled
|
|
XGRIDS/LixelKity K1 to a Mac without LixelGO, firmware changes, device opening,
|
|
or speculative writes.
|
|
|
|
Current status: live proof completed on firmware 3.0.2. The Mac provisioned the
|
|
K1 onto an existing LAN without LixelGO, connected to its MQTT broker, captured
|
|
the scan-correlated point-cloud and pose streams, and decoded both successfully.
|
|
The repository also contains a local React control console, an automatic Rerun
|
|
gRPC bridge for the verified point-cloud and trajectory streams, and a
|
|
self-hosted Rerun Web Viewer embedded in that console. The former Foxglove
|
|
bridge remains only as a legacy regression module.
|
|
|
|
The repository now contains one narrowly gated state-changing command:
|
|
`ble wifi-configure`. It accepts only the reviewed firmware-3 provisioning
|
|
profile and requires explicit `--confirm-write`; the Wi-Fi password is collected
|
|
through a hidden local macOS dialog. MQTT capture and decoding are read-only.
|
|
Nothing changes router settings, firmware or global Python packages.
|
|
|
|
## Available stand
|
|
|
|
- one XGRIDS/LixelKity K1;
|
|
- one Apple Silicon MacBook running macOS;
|
|
- one ordinary TP-Link Deco/mesh network used by other devices;
|
|
- no LixelGO, phone, Linux host, dedicated AP, OpenWrt, or vendor SDK.
|
|
|
|
The ordinary router is sufficient for the first gates. We first observe the
|
|
existing LAN without changing it. A Guest/IoT SSID is optional and may be
|
|
counterproductive if Deco isolates clients; Mac and K1 must ultimately be able
|
|
to reach each other.
|
|
|
|
## What is actually being proved
|
|
|
|
The project has three independent gates:
|
|
|
|
1. K1 is operational and can record a project autonomously.
|
|
2. Mac can discover and inspect the K1 BLE/GATT surface safely.
|
|
3. Without LixelGO, K1 can be associated with Wi-Fi and a proprietary data
|
|
session can be opened.
|
|
|
|
All three gates are now proven on the tested unit. The external stream is plain
|
|
MQTT 3.1.1 on TCP 1883. Firmware-3 `lio_pcl` is protobuf wrapped in a raw LZ4
|
|
block, and `lio_pose` is an uncompressed protobuf. Raw panoramic camera access is
|
|
still unproven and is not implied by point-cloud success.
|
|
|
|
## Local environment
|
|
|
|
The project uses Python 3.12 in a repository-local `.venv` managed by `uv`.
|
|
This does not install Python packages globally and does not modify neighboring
|
|
repositories.
|
|
|
|
```bash
|
|
cd /Users/dcconstructions/Downloads/mnt/NODEDC/NDC_xgrids-k1-connector
|
|
uv sync --group dev
|
|
uv run k1link doctor
|
|
uv run pytest
|
|
```
|
|
|
|
## NODE.DC Control Station and visualization adapters
|
|
|
|
The browser application is now a universal NODE.DC Control Station rather than
|
|
a K1-specific Foxglove launcher. Its fixed shell contains six architectural
|
|
sections — Center, Fleet, Observation, Missions, Data and System — while the K1
|
|
BLE/Wi-Fi/live workflow remains isolated as the first real device adapter.
|
|
|
|
Install, type-check, build and serve the complete local application from the
|
|
repository root:
|
|
|
|
```bash
|
|
uv sync --group dev
|
|
cd apps/k1-viewer
|
|
npm install
|
|
npm run typecheck
|
|
npm run build
|
|
cd ../..
|
|
uv run k1link serve
|
|
```
|
|
|
|
Open `http://127.0.0.1:8000`. The static application, REST/WebSocket control
|
|
plane and credential endpoint bind to loopback only. The current K1 adapter
|
|
still provides real CoreBluetooth discovery, one operator-triggered reviewed
|
|
BLE Wi-Fi provisioning write, read-only MQTT live capture, native `.k1mqtt` and
|
|
reviewed-TSV replay, raw-first evidence storage and measured preview metrics.
|
|
Physical K1 scanning is still started and stopped by the verified double-click;
|
|
the connector publishes no modeling command.
|
|
|
|
The Observation spatial workspace embeds the open-source Rerun Web Viewer
|
|
inside the NODE.DC shell. It can open a compatible RRD file over HTTP(S) or a
|
|
Rerun gRPC/proxy source such as `rerun+http://127.0.0.1:9876/proxy`. It does not
|
|
use an external hosted viewer UI.
|
|
|
|
The first K1 live session or replay in a `k1link serve` process creates one local
|
|
Rerun `RecordingStream`, starts its gRPC/proxy server on TCP 9876 and publishes
|
|
the resulting URL through control-plane state. Later sessions reset their
|
|
session-local scene and metrics and reuse that process-wide stream; this avoids
|
|
restarting the native listener while the embedded browser remains connected.
|
|
Unless an operator has entered a manual source, the React application assigns
|
|
that URL to the embedded viewer. The complete runtime path is K1 MQTT → raw-first evidence
|
|
capture → bounded latest-wins preview queue → reviewed protobuf/LZ4 decoders →
|
|
Rerun `Points3D`, `Transform3D` and `LineStrips3D` → embedded Web Viewer.
|
|
|
|
The default Rerun blueprint shows a 12-second sliding accumulation of real point
|
|
frames. Product controls are connected for point size, intensity/height/distance
|
|
or available RGB coloring, Turbo/Viridis/Plasma/grayscale/custom palettes,
|
|
point and trajectory visibility, and the scene grid. Projection, custom
|
|
timeline transport and saved layout remain later product work. No synthetic
|
|
point cloud, trajectory, camera frame or latency value is generated.
|
|
|
|
A powered-device checkpoint passed 80 real MQTT messages through the current
|
|
Rerun runtime: 38 point-cloud frames, 42 pose frames, 2,775 points in the last
|
|
cloud and zero decode errors. Raw panoramic camera frames remain absent. Rerun
|
|
`capture_time` is the Mac receive timestamp, not a proven K1 sensor timestamp or
|
|
photon-to-screen measurement.
|
|
|
|
The old Foxglove implementation is retained only in
|
|
`src/k1link/viewer/foxglove_bridge.py` and its regression tests. The current
|
|
live/replay runtime does not start it or use TCP 8765. The
|
|
[live viewer runbook](docs/06_K1_LIVE_VIEWER.md) records the active Rerun path and
|
|
its timing/security boundaries; the frontend contract is documented in
|
|
[`apps/k1-viewer/README.md`](apps/k1-viewer/README.md).
|
|
|
|
The FastAPI application and credential endpoint bind to loopback, but the Rerun
|
|
gRPC server currently binds TCP 9876 on all network interfaces even though its
|
|
reported source URL contains `127.0.0.1`. It has no connector-level
|
|
authentication or TLS. Use it only on a trusted laboratory LAN, do not expose
|
|
9876 to the public Internet or a cellular WAN, and add an authenticated secure
|
|
proxy before any remote deployment. Stopping acquisition keeps the local scene
|
|
server and its URL available for the next session. Stop `k1link serve` to close
|
|
the listener and release its retained memory.
|
|
|
|
`doctor` is intentionally non-invasive. It checks the local Python environment
|
|
and reports external tools; it does not request Bluetooth permission, scan the
|
|
LAN, touch the K1, alter Homebrew, or change capture permissions.
|
|
|
|
The implemented laboratory commands include:
|
|
|
|
```bash
|
|
uv run k1link ble scan --duration 30 --out sessions/<id>/captures/ble.json
|
|
uv run k1link ble gatt-dump --device <corebluetooth-uuid> \
|
|
--out sessions/<id>/captures/gatt.json
|
|
uv run k1link ble wifi-configure --device <corebluetooth-uuid> \
|
|
--profile xgrids-k1-fw3-wifi-v1 --write-mode with_response \
|
|
--confirm-write --out sessions/<id>/captures/wifi.sensitive.json
|
|
uv run k1link net snapshot --out sessions/<id>/captures/network.json
|
|
uv run k1link net mqtt-capture --host <confirmed-private-k1-ip> \
|
|
--confirm-owned-device --duration 180 \
|
|
--out sessions/<id>/captures/mqtt-run
|
|
uv run k1link analyze mqtt-streams \
|
|
--capture sessions/<id>/captures/mqtt-run/mqtt.raw.k1mqtt \
|
|
--out sessions/<id>/analysis/mqtt-streams.summary.json
|
|
```
|
|
|
|
On macOS the BLE scan is active CoreBluetooth discovery, but it does not connect
|
|
to or modify devices. `gatt-dump` connects and performs service discovery only.
|
|
`mqtt-capture` accepts only a literal RFC1918 target, uses a fixed report-topic
|
|
allowlist, never publishes and never reconnects. It writes a length-framed raw
|
|
file, JSONL metadata and an integrity summary with mode `0600`.
|
|
|
|
Session output is sensitive and ignored by Git. It can contain device identity,
|
|
trajectory, mapped interiors and local addressing even when no credentials are
|
|
present.
|
|
|
|
## Documentation
|
|
|
|
- [Technical audit](docs/00_TECHNICAL_AUDIT.md)
|
|
- [Implementation gates](docs/01_IMPLEMENTATION_PLAN.md)
|
|
- [First lab runbook](docs/02_FIRST_LAB_RUNBOOK.md)
|
|
- [Artifact and secret policy](docs/03_ARTIFACT_POLICY.md)
|
|
- [Reviewed BLE Wi-Fi profile](docs/04_K1_WIFI_PROVISIONING_PROFILE.md)
|
|
- [Verified MQTT stream profile](docs/05_K1_MQTT_STREAM_PROFILE.md)
|
|
- [Live console and embedded Rerun runbook](docs/06_K1_LIVE_VIEWER.md)
|
|
- [Redacted live lab report](docs/lab/001_K1_LIVE_MQTT_20260715.redacted.md)
|
|
- [Session manifest schema](schemas/session-manifest.schema.json)
|
|
- [Reference input provenance](docs/reference/README.md)
|
|
|
|
The two supplied source documents are retained unchanged under
|
|
`docs/reference/`. Corrections and decisions are recorded separately so their
|
|
provenance remains clear.
|
|
|
|
## Safety boundary
|
|
|
|
Allowed initial work is non-mutating discovery, standard device-information reads,
|
|
controlled notification listening, autonomous button operation, targeted
|
|
capture of traffic to or from the confirmed K1 address, and offline analysis of
|
|
owned artifacts.
|
|
|
|
The reviewed provisioning write requires its named profile and explicit operator
|
|
confirmation. Application command publishing remains disabled: physical
|
|
double-click is the verified start/stop mechanism. Any future MQTT publisher,
|
|
router configuration change or new BLE write requires its own evidence and
|
|
reviewed step. Random writes, fuzzing, brute force, firmware operations,
|
|
destructive file access and credential guessing remain out of scope.
|
|
|
|
Real captures, projects, router metadata, serials, credentials, maps, images,
|
|
and logs are ignored by normal Git. Redacted manifests and SHA-256 inventories
|
|
are committed; encrypted artifact storage will be selected only when real data
|
|
exists.
|