feat(perception): integrate calibrated operator pipeline
Add calibrated K1 projection, recorded and near-live perception qualification, unified Rerun operator layers, bounded replay admission, audited viewer controls, worker experiments, and lab evidence.
This commit is contained in:
@@ -0,0 +1,627 @@
|
||||
# Mission Core handoff: AI worker, external perception and K1 Quick Connect
|
||||
|
||||
Date of factual refresh: 2026-07-19 21:15 MSK.
|
||||
|
||||
> 2026-07-20 superseding update: official firmware analysis proved that the
|
||||
> exact K1 `3.0.2` AP credential is firmware-constant and that the SSID follows
|
||||
> an identity/MAC rule. A prepared Mac completed AP-ready, CoreWLAN association
|
||||
> and the normal control lifecycle. This did **not** solve clean-host bootstrap:
|
||||
> the provider had been separately imported from the firmware archive. The
|
||||
> owner closed this product branch, rejected automatic firmware download,
|
||||
> iPhone extraction and hard-coding, retained the working UI/Keychain state and
|
||||
> selected Bridge/direct-LAN as the continuing route. Canonical details:
|
||||
> `docs/lab/004_K1_FW302_AP_CREDENTIAL_PROVIDER_20260720.redacted.md`.
|
||||
|
||||
This document is a transition context for a new engineering chat. It separates
|
||||
committed/accepted work from current uncommitted K1 work and from future plans.
|
||||
It contains operational addresses, ports, repository paths and public
|
||||
fingerprints, but no password, private key, token, device secret, raw capture or
|
||||
camera image.
|
||||
|
||||
## Executive state
|
||||
|
||||
Mission Core currently has two independently meaningful physical results:
|
||||
|
||||
1. The Mac/K1 control and archive path is accepted for one owner-controlled
|
||||
XGRIDS LixelKity K1, firmware 3.0.2, Bridge/direct-LAN, handheld, GNSS without
|
||||
RTK. Mission Core can run the reviewed MQTT bootstrap, START acquisition,
|
||||
display live points, send explicit STOP, observe final READY/unbound and open
|
||||
the durable Rerun/camera archive.
|
||||
2. A replaceable Windows RTX 4090 worker is accepted for one recorded,
|
||||
camera-only perception job. The Mac publishes a bounded immutable
|
||||
`missioncore.compute-job/v1`; Windows verifies it, runs pinned YOLOX-S through
|
||||
pinned Triton, publishes `missioncore.compute-result/v1`; Mission Core
|
||||
revalidates and projects the optional result into the matching saved Rerun
|
||||
recording.
|
||||
|
||||
This is not yet a live autonomous-vehicle stack. The worker has no K1 authority,
|
||||
Triton is not exposed to the LAN, the current transfer is SSH/SCP laboratory
|
||||
bootstrap, Tailscale is not installed, and live fan-out/fusion/tracking/free
|
||||
space/control are not accepted.
|
||||
|
||||
The current K1 Quick Connect implementation is a separate uncommitted worktree.
|
||||
It proved BLE AP activation, exact AP-ready observation and prepared-host native
|
||||
macOS association/control. It remains laboratory-only because a clean host has
|
||||
no autonomous evidence-backed credential source. The implementation and current
|
||||
machine state are retained, but Bridge/direct-LAN is the product route.
|
||||
|
||||
## Canonical topology and authority
|
||||
|
||||
```text
|
||||
XGRIDS K1
|
||||
| BLE / MQTT / RTSP
|
||||
v
|
||||
Mission Core Edge on Mac
|
||||
- sole K1 command authority
|
||||
- sole acquisition lifecycle owner
|
||||
- authoritative raw .k1mqtt + camera fMP4 archive
|
||||
- durable session catalog and Rerun projection
|
||||
|
|
||||
| bounded immutable missioncore.compute-job/v1
|
||||
| current lab transport: SSH/SCP
|
||||
v
|
||||
Mission Core AI Server Worker on Windows / RTX 4090
|
||||
- verifies the complete job and every digest
|
||||
- reconstructs and decodes only the admitted camera epoch
|
||||
- calls local pinned Triton / pinned model
|
||||
- atomically publishes missioncore.compute-result/v1
|
||||
|
|
||||
v
|
||||
Mission Core Edge validates result and creates optional derived Rerun layer
|
||||
```
|
||||
|
||||
The Windows machine is an executor, not a second Mission Core. It does not know
|
||||
K1 credentials, IP, MQTT topics or RTSP endpoints and cannot discover,
|
||||
provision, START or STOP the scanner. Derived inference never replaces or
|
||||
rewrites the Mac source archive.
|
||||
|
||||
## Why the original architecture proposal was narrowed
|
||||
|
||||
The supplied architecture review was accepted as a boundary, not as a mandate
|
||||
to install every named technology immediately. The following principles were
|
||||
kept:
|
||||
|
||||
- one vendor owner for K1 control and source streams;
|
||||
- immutable raw-first evidence and separately versioned derived data;
|
||||
- replaceable external compute;
|
||||
- local mission/safety authority rather than a remote GPU dependency;
|
||||
- explicit separation of command/state, real-time flow and archive transport;
|
||||
- declarative, reproducible deployment built mostly from mature components.
|
||||
|
||||
The following were deliberately deferred because they would precede the first
|
||||
vertical evidence:
|
||||
|
||||
- custom universal Mission Core Node Runtime;
|
||||
- custom cross-platform installer, node console, OTA, model package format and
|
||||
generic scheduler;
|
||||
- ROS 2 before a real onboard computer, robot state and actuator/autopilot
|
||||
contract exist;
|
||||
- BehaviorTree.CPP before a local mission executor is needed;
|
||||
- Zenoh until direct/routed transport can be compared on identical data;
|
||||
- MCAP as a replacement for the accepted K1 raw source;
|
||||
- dora-rs as a control/safety foundation;
|
||||
- LM Studio for the camera/LiDAR detector vertical.
|
||||
|
||||
Triton was selected for the first executor because it already provides pinned
|
||||
model repositories, HTTP/gRPC, health, metrics and replaceable inference
|
||||
backends. Mission Core continues to own its northern job/result contracts.
|
||||
|
||||
## Windows host identity and hardware
|
||||
|
||||
Live read-only inventory at the refresh time:
|
||||
|
||||
- host: `<worker-host>`;
|
||||
- interactive/SSH account: `<worker-host>\<worker-user>`;
|
||||
- OS: Windows 11 Pro x64, version 10.0.26200, build 26200;
|
||||
- last boot: 2026-07-19 15:38:18 MSK;
|
||||
- PowerShell: 5.1.26100.7705;
|
||||
- CPU: Intel Core i9-13900KF, 24 cores / 32 logical processors;
|
||||
- RAM: 95.85 GiB;
|
||||
- GPU: NVIDIA GeForce RTX 4090, 24,564 MiB VRAM, compute capability 8.9;
|
||||
- NVIDIA driver: Studio 610.47 WHQL;
|
||||
- live GPU snapshot: about 10,229 MiB used, 13,910 MiB free, 41 C and 68.68 W;
|
||||
- disk `C:`: NTFS, 1,861.99 GiB total, 1,700.86 GiB free;
|
||||
- disk `D:`: NTFS label `DC`, 1,863.00 GiB total, 411.44 GiB free;
|
||||
- Git: 2.53.0.windows.1;
|
||||
- Codex CLI: 0.142.1;
|
||||
- FFmpeg: 8.0.1 essentials build;
|
||||
- native host Python: not installed; Windows Store aliases exist but do not
|
||||
resolve to a Python runtime;
|
||||
- Tailscale: not installed.
|
||||
|
||||
The driver was upgraded from 591.86 to 610.47 after the first Triton startup
|
||||
showed CUDA Minor Version Compatibility mode. The official NVIDIA installer was
|
||||
978,481,008 bytes, SHA-256
|
||||
`59AC4A1659664AAD0A6FC525E5DF99B3FA76887BDE663F9E36E0E7EBB5DBA937`,
|
||||
had a valid NVIDIA Corporation Authenticode signature and returned exit code 0.
|
||||
After reboot, the compatibility warning disappeared and the full compute smoke
|
||||
passed.
|
||||
|
||||
## WSL and Docker substrate
|
||||
|
||||
- WSL: 2.7.8.0;
|
||||
- default distribution: Ubuntu 24.04, WSL version 2;
|
||||
- kernel: 6.18.33.1-microsoft-standard-WSL2;
|
||||
- Docker Desktop distribution: WSL2 and currently running;
|
||||
- Docker Desktop: 4.78.0;
|
||||
- Docker Engine/client: 29.5.3;
|
||||
- Docker Compose: 5.1.4;
|
||||
- Docker Linux VM: 32 CPUs and approximately 50.43 GB memory;
|
||||
- storage driver: overlayfs;
|
||||
- NVIDIA container runtime is registered; default runtime remains `runc` and
|
||||
the Compose service explicitly requests all GPUs.
|
||||
|
||||
Docker Desktop's engine-owned image/layer VHDX remains on `C:`. Mission Core
|
||||
application state is on `D:`. Moving Docker's VHDX was intentionally not made a
|
||||
pilot prerequisite.
|
||||
|
||||
## Isolated Windows directory layout
|
||||
|
||||
Root: `D:\NDC_MISSIONCORE`.
|
||||
|
||||
```text
|
||||
D:\NDC_MISSIONCORE\
|
||||
README.md
|
||||
workspace\
|
||||
mission-core-compute\ versioned deployment and worker repository
|
||||
runtime\
|
||||
cache\ disposable downloads/build cache
|
||||
derived\ decoded intermediates and immutable results
|
||||
docker-cli\ isolated Docker CLI state for SSH automation
|
||||
jobs\ bounded transferred compute jobs
|
||||
logs\ service/task transcripts
|
||||
models\ Triton model repository and weights
|
||||
tmp\ disposable temporary data
|
||||
secrets\ machine-local credentials, never committed
|
||||
```
|
||||
|
||||
The root itself is not a Git repository. Only
|
||||
`D:\NDC_MISSIONCORE\workspace\mission-core-compute` is versioned. Runtime
|
||||
weights, jobs, results, caches, logs and secrets are outside Git.
|
||||
|
||||
## Windows worker repository
|
||||
|
||||
Repository:
|
||||
`D:\NDC_MISSIONCORE\workspace\mission-core-compute`.
|
||||
|
||||
Current state:
|
||||
|
||||
- branch: `main`;
|
||||
- HEAD: `a3c36c3 feat(perception): prove recorded Triton vertical`;
|
||||
- previous commits:
|
||||
- `3c62815 guard native CUDA runtime compatibility`;
|
||||
- `d1043d0 qualify CUDA 13.3 driver path`;
|
||||
- `8edd003 bootstrap Windows GPU compute node`;
|
||||
- working tree: clean;
|
||||
- Git remote: none configured.
|
||||
|
||||
Versioned content includes `compose.yaml`, `.env.example`, model config,
|
||||
baseline/decision/acceptance docs, bounded PowerShell start/stop/test/handoff
|
||||
scripts and `worker/run_yolox_recorded.py`.
|
||||
|
||||
No custom Windows service, host Python environment, LM Studio integration,
|
||||
universal model loader or second Mission Core instance was installed.
|
||||
|
||||
## Active Triton deployment
|
||||
|
||||
Container: `mission-core-triton`, Compose project `mission-core-compute`.
|
||||
|
||||
Pinned image:
|
||||
|
||||
`nvcr.io/nvidia/tritonserver:26.06-py3@sha256:58df7489c3f2276f9591d500a012dee03e23d35543ce3c390b4c001e6bf90794`
|
||||
|
||||
The local image reports Triton 2.70.0 and a roughly 22.7 GB image size.
|
||||
|
||||
Effective deployment properties:
|
||||
|
||||
- `gpus: all`;
|
||||
- shared memory: 2 GiB;
|
||||
- model control: explicit;
|
||||
- automatic model config completion: disabled;
|
||||
- strict readiness: enabled;
|
||||
- server does not exit for one model error;
|
||||
- HTTP 8000, gRPC 8001 and metrics 8002 enabled;
|
||||
- every host binding is loopback-only: `127.0.0.1:8000-8002`;
|
||||
- `D:\NDC_MISSIONCORE\runtime\models` is mounted read-only at `/models`;
|
||||
- `no-new-privileges:true`;
|
||||
- container restart policy: `no`;
|
||||
- health check: HTTP `/v2/health/ready` every 5 seconds;
|
||||
- current state: running and healthy;
|
||||
- live, ready and metrics checks: HTTP 200;
|
||||
- GPU metrics present;
|
||||
- CUDA compatibility warning absent;
|
||||
- deprecated model-config warning absent.
|
||||
|
||||
The committed `.env` contains only the loopback bind and pinned Triton image.
|
||||
No credential is required for the current loopback-only service.
|
||||
|
||||
## Startup, shutdown and task bridge
|
||||
|
||||
The normal operator scripts are:
|
||||
|
||||
```powershell
|
||||
Set-Location D:\NDC_MISSIONCORE\workspace\mission-core-compute
|
||||
.\scripts\Start-MissionCoreCompute.ps1
|
||||
.\scripts\Test-MissionCoreCompute.ps1
|
||||
.\scripts\Stop-MissionCoreCompute.ps1
|
||||
```
|
||||
|
||||
`Start-MissionCoreCompute.ps1` uses the node-local Docker CLI directory,
|
||||
targets Docker Desktop's Linux engine pipe, creates loopback-only `.env` from
|
||||
the example when absent, starts Docker Desktop when needed, performs Compose
|
||||
pull/up and waits for Triton readiness.
|
||||
|
||||
Docker Desktop's Windows credential helper rejects the OpenSSH network-logon
|
||||
token even for public image pulls. The reviewed workaround is not a second
|
||||
deployment path: `Invoke-StartAsInteractiveUser.ps1` creates/starts the
|
||||
on-demand `MissionCore-StartCompute` task in the logged-on user's limited
|
||||
interactive token. It executes the same committed start script and writes
|
||||
`D:\NDC_MISSIONCORE\runtime\logs\start-compute-latest.log`.
|
||||
|
||||
Scheduled task state:
|
||||
|
||||
- `MissionCore-DockerDesktop`: enabled logon trigger for the interactive user,
|
||||
limited privilege, currently Running;
|
||||
- `MissionCore-StartCompute`: no trigger, on-demand only, last result 0,
|
||||
currently Ready;
|
||||
- Triton Compose restart policy is `no`.
|
||||
|
||||
Therefore Docker Desktop starts at user logon, but Triton is not a fully
|
||||
autonomous reboot-surviving service. It is started explicitly/on demand.
|
||||
|
||||
## Model repository
|
||||
|
||||
Accepted model: official YOLOX-S ONNX release 0.1.1rc0, Apache-2.0.
|
||||
|
||||
- model path: `runtime\models\yolox_s\1\model.onnx`;
|
||||
- byte length: 35,858,002;
|
||||
- SHA-256:
|
||||
`C5C2D13E59AE883E6AF3B45DAEA64AF4833A4951C92D116EC270D9DDBE998063`;
|
||||
- Triton platform: ONNX Runtime GPU;
|
||||
- model version: 1 only;
|
||||
- input: FP32 `[1,3,640,640]`, name `images`;
|
||||
- output: FP32 `[1,8400,85]`, name `output`;
|
||||
- one GPU instance;
|
||||
- preprocessing: bilinear resize, top-left letterbox, BGR, pad 114;
|
||||
- postprocessing: official YOLOX grid/stride decode, score 0.25, NMS 0.45,
|
||||
COCO-80.
|
||||
|
||||
The model is useful only as a contract/timing smoke. It is not accepted for
|
||||
navigation, obstacle avoidance, free-space or safety decisions.
|
||||
|
||||
## Accepted recorded job and result
|
||||
|
||||
Source Mission Core session:
|
||||
`20260718T201659Z_viewer_live` / display name `TEST007`.
|
||||
|
||||
Accepted input:
|
||||
|
||||
- source: `sensor.camera.left`, physical codec epoch 1;
|
||||
- job: `recorded-camera-fae25b92ecf645c2c9765dd4`;
|
||||
- schema: `missioncore.compute-job/v1`;
|
||||
- full input SHA-256:
|
||||
`fae25b92ecf645c2c9765dd4b4963f23cf8c101f22fd5a2e14a104f0b447140d`;
|
||||
- 59 digest-bound files, 4,567,569 package bytes;
|
||||
- reconstructed H.264 stream: 4,549,458 bytes;
|
||||
- stream SHA-256:
|
||||
`bceca577f762fdb79c4e8901e5c2a2330fd07d24dc091b1b703c5af4010d8a56`;
|
||||
- decode: H.264 High, 800x600, 56 frames, 5.497 seconds;
|
||||
- synchronization: `host-arrival-best-effort`;
|
||||
- frame timestamps are made strictly increasing from best-effort decode time;
|
||||
this is not calibrated camera/LiDAR sensor time.
|
||||
|
||||
Accepted result:
|
||||
|
||||
- result:
|
||||
`result-5484a72e81192b19f4e1da2a2dcfd10c876c277ff3e96c6d60cbc9d917b9f604`;
|
||||
- schema: `missioncore.compute-result/v1`;
|
||||
- detection schema: `missioncore.object-detections/v1`;
|
||||
- 56/56 frames processed;
|
||||
- 54 frames with detections;
|
||||
- 56 generic COCO `person` detections;
|
||||
- detection artifact: 23,802 bytes;
|
||||
- detection SHA-256:
|
||||
`71b291616e87bbdd2b7a295b1e00dac2ff3e60c627e63b72a27ba802c32a3ae0`;
|
||||
- client-observed inference latency: mean 15.611 ms, p50 15.001 ms,
|
||||
p95 23.728 ms, max 31.099 ms.
|
||||
|
||||
Triton currently reports 168 successful inference requests, 0 failed. Three
|
||||
56-frame executions account for that count. The final exact repeated command
|
||||
reused the already published content-addressed result and did not increase the
|
||||
counter, proving idempotent reuse.
|
||||
|
||||
The first runner attempt completed inference but failed its post-inference EOF
|
||||
guard. No partial result was published. The runner was corrected before the
|
||||
accepted immutable result was produced.
|
||||
|
||||
## Rerun result projection on the Mac
|
||||
|
||||
Mission Core revalidates exact session/job/result binding, all digests,
|
||||
timestamps, dimensions and frame count before projection. It builds one complete
|
||||
RRF2 derived overlay, approximately 62.55 MB, cached privately by
|
||||
result/recording identity.
|
||||
|
||||
The browser accepts the overlay on an isolated Rerun channel. Missing or invalid
|
||||
perception returns no optional layer and never replaces the base point-cloud
|
||||
recording. In TEST007 the `Распознавание` switch showed the real recorded left
|
||||
camera frame with `person · 34%` Boxes2D; `Облако точек` returned to the base 3D
|
||||
view.
|
||||
|
||||
Committed Mac work:
|
||||
|
||||
- `648d5bc feat(compute): add bounded camera job contract`;
|
||||
- `31fc4f6 docs(perception): record external worker acceptance`;
|
||||
- `2b53168 feat(perception): project recorded results into Rerun`;
|
||||
- `ada2a55 docs(perception): record Rerun projection acceptance`.
|
||||
|
||||
Mac `main` is currently at `ada2a55`; `origin/main` remains at `75d2e5d`, so the
|
||||
four perception commits are local and not pushed.
|
||||
|
||||
## Other active workloads on the RTX host
|
||||
|
||||
The GPU workstation is shared. These containers are not part of Mission Core:
|
||||
|
||||
- `sentinel-frigate`, image `ghcr.io/blakeblackshear/frigate:0.17.2-tensorrt`,
|
||||
healthy, publishes host ports 5000, 8554, 8555 TCP/UDP and 8971;
|
||||
- `sentinel-ollama`, image `ollama/ollama:latest`, publishes host port 11434 and
|
||||
has no configured container health check.
|
||||
|
||||
Their current resource use contributes to the roughly 10 GiB observed GPU
|
||||
allocation. Mission Core currently has no resource reservation, scheduler or
|
||||
admission policy against these workloads. Long-epoch and live performance tests
|
||||
must record this co-tenancy or isolate it deliberately.
|
||||
|
||||
## SSH access from the Mac
|
||||
|
||||
Windows OpenSSH was installed through official winget package
|
||||
`Microsoft.OpenSSH.Preview` 10.0.0.0 because Windows Feature-on-Demand failed:
|
||||
|
||||
- `Add-WindowsCapability`: Access denied;
|
||||
- DISM with `/LimitAccess`: `0x800f0912`, source files not found;
|
||||
- `wuauserv` startup/ACL changes were denied by host policy.
|
||||
|
||||
Current Windows SSH facts:
|
||||
|
||||
- service: `sshd`, Running, Automatic;
|
||||
- binary: `C:\Program Files\OpenSSH\sshd.exe`, version 10.0.0.0;
|
||||
- listen: `0.0.0.0:22` and `[::]:22`;
|
||||
- ED25519 host fingerprint:
|
||||
`SHA256:LgvblT9h+5KJFtkDq6JWGUyvgpiFyeL6cztVVMY0ZQM`;
|
||||
- public-key authentication: enabled;
|
||||
- password authentication: also enabled;
|
||||
- strict modes: enabled;
|
||||
- max authentication attempts: 6.
|
||||
|
||||
The Mac connection uses a key and strict pinning, but the server itself is not
|
||||
yet key-only. Disabling server password authentication is an explicit future
|
||||
hardening action, not a completed fact.
|
||||
|
||||
Mac SSH alias in `~/.ssh/config`:
|
||||
|
||||
```sshconfig
|
||||
Host mission-gpu
|
||||
HostName <worker-host>.local
|
||||
User <worker-user>
|
||||
IdentityFile ~/.ssh/id_ed25519
|
||||
IdentitiesOnly yes
|
||||
HostKeyAlias mission-gpu
|
||||
UserKnownHostsFile ~/.ssh/known_hosts_mission_core
|
||||
StrictHostKeyChecking yes
|
||||
ServerAliveInterval 15
|
||||
ServerAliveCountMax 3
|
||||
```
|
||||
|
||||
Connection and safe read-only smoke:
|
||||
|
||||
```bash
|
||||
ssh -o BatchMode=yes mission-gpu whoami
|
||||
ssh -o BatchMode=yes mission-gpu \
|
||||
'powershell.exe -NoProfile -NonInteractive -Command "$env:COMPUTERNAME"'
|
||||
```
|
||||
|
||||
The dedicated known-hosts entry matches the Windows ED25519 fingerprint above.
|
||||
No private key or password is stored in this repository or Ops.
|
||||
|
||||
## Windows network and firewall
|
||||
|
||||
Current physical Wi-Fi:
|
||||
|
||||
- SSID/profile: `<owner-lan-ssid>`;
|
||||
- interface: `Беспроводная сеть`;
|
||||
- category: Private;
|
||||
- IPv4: `<worker-lan-ip>/<owner-lan-prefix>`;
|
||||
- IPv4 connectivity: Internet.
|
||||
|
||||
VPN profile:
|
||||
|
||||
- name: `Сеть 5`;
|
||||
- interface: `hidemy.name VPN OpenVPN Adapter`;
|
||||
- category: Public;
|
||||
- IPv4: `<vpn-assigned-ip>/<vpn-prefix>`.
|
||||
|
||||
Windows Firewall is enabled for Private and Public profiles. The active custom
|
||||
SSH rule is:
|
||||
|
||||
- name: `MissionCore-SSH-From-Mac`;
|
||||
- display name: `MISSION CORE SSH from Mac`;
|
||||
- enabled, inbound allow, Private only;
|
||||
- interface: `Беспроводная сеть` only;
|
||||
- protocol/local port: TCP/22;
|
||||
- remote address: `LocalSubnet`;
|
||||
- local address: Any.
|
||||
|
||||
The broad MSI-created `OpenSSH SSH Server Preview (sshd)` inbound rule is
|
||||
disabled. The current custom rule is broader than the original single Mac IP
|
||||
because it admits the local subnet, but it remains confined to the Private Wi-Fi
|
||||
interface and does not apply to the Public VPN profile.
|
||||
|
||||
Triton ports 8000-8002 listen only on 127.0.0.1. No Windows firewall rule exposes
|
||||
them to the LAN. Consequently the Mac cannot yet call Triton directly; the
|
||||
accepted recorded path transfers bounded artifacts through SSH/SCP and executes
|
||||
the runner on Windows.
|
||||
|
||||
The current worker IPv4 `<worker-lan-ip>` previously appeared as a K1 target in
|
||||
historical TEST007 evidence. Before concurrent K1/worker LAN exposure, reserve
|
||||
non-conflicting addresses or prove topology separation. Do not expose Triton on
|
||||
this address until that conflict and the source-restricted firewall contract are
|
||||
reviewed.
|
||||
|
||||
## Current K1 Quick Connect state
|
||||
|
||||
The K1 plugin exposes three explicit local connection directions:
|
||||
|
||||
- Bridge/direct-LAN: K1 joins an existing shared network; physically accepted;
|
||||
- Direct Connect/controller-hotspot: K1 joins the controller's network;
|
||||
implemented/offline-verified, not physically accepted;
|
||||
- Quick Connect/device-AP: K1 becomes the AP and a prepared Mac joins it;
|
||||
physically accepted on the current host, retained as laboratory-only.
|
||||
|
||||
The reviewed LixelGO flow is:
|
||||
|
||||
1. keep one BLE connection open;
|
||||
2. write one reviewed 100-byte AP-enable frame (99 zero bytes plus final 1);
|
||||
3. poll `7f02` until the live status reports `WIFI_AP`, baseline address
|
||||
`192.168.56.1` and the AP-ready byte changes from 0 to nonzero;
|
||||
4. while BLE remains alive, use the native OS Wi-Fi API to scan exact SSID and
|
||||
associate;
|
||||
5. continue to K1 network/control only after association.
|
||||
|
||||
The earlier evidence session
|
||||
`.runtime/mission-core/evidence/sessions/20260719T173132Z_viewer_k1_ap_association`
|
||||
proved AP activation, AP-ready and native SSID discovery but stopped at the
|
||||
secure prompt. A later prepared-host session
|
||||
`.runtime/mission-core/evidence/sessions/20260719T220850Z_viewer_k1_ap_association`
|
||||
completed one CoreWLAN association and admitted `192.168.56.1`; the normal UI
|
||||
then completed control/acquisition. The operator disconnected afterward only to
|
||||
restore the external chat route.
|
||||
|
||||
Final credential findings:
|
||||
|
||||
- LixelGO persists `WiFiAP_SSID` and `WiFiAP_Password` in a per-device profile;
|
||||
- the Unity flow later passes those profile values to the native join path;
|
||||
- no password derivation from DeviceInfo, serial, BLE UUID or AP SSID was found;
|
||||
- official K1 `3.0.2` firmware contains one firmware-constant AP credential and
|
||||
constructs the SSID from device identity with a MAC fallback;
|
||||
- the reviewed BLE exchange enables the AP and reports readiness but exposes no
|
||||
observed credential response;
|
||||
- LixelGo login and the scanner AP are separate layers. The owner performed no
|
||||
explicit hardware-to-account pairing, which is consistent with the local
|
||||
firmware-defined credential, but does not prove universal account access;
|
||||
- the successful Mission Core run depended on a separate firmware import into
|
||||
this Mac's Keychain. A copied build on a clean Mac would fail before BLE.
|
||||
|
||||
The owner closed this branch. Do not download firmware during connection, seek
|
||||
the value from iPhone, hard-code it, or present machine-local Keychain state as
|
||||
portable automation. Keep the existing UI, implementation, Keychain/provider
|
||||
state and evidence intact. Continue through Bridge/direct-LAN.
|
||||
|
||||
## Current Mac repository and runtime state
|
||||
|
||||
Repository:
|
||||
`<mission-core-repository>`.
|
||||
|
||||
- branch: `main`;
|
||||
- HEAD: `ada2a55`;
|
||||
- `origin/main`: `75d2e5d`;
|
||||
- local Mission Core HTTP service: `127.0.0.1:8000`;
|
||||
- current health: service OK, one of one plugin runtimes ready;
|
||||
- current local server PID at 2026-07-20 closure refresh: 97887;
|
||||
- worker/perception commits are clean and local;
|
||||
- K1 Quick Connect changes are uncommitted and make the worktree dirty.
|
||||
|
||||
The dirty set spans K1 docs/profile/frontend, native CoreWLAN helper,
|
||||
BLE AP activation, host-network adapter and tests. It must not be reset or
|
||||
cleaned because it contains the current physical Quick Connect implementation
|
||||
and evidence-aligned fixes.
|
||||
|
||||
Latest validation of the current dirty tree:
|
||||
|
||||
- complete backend suite: passed;
|
||||
- Ruff: clean;
|
||||
- mypy: clean across 75 source files;
|
||||
- frontend: 140 unit tests passed;
|
||||
- TypeScript typecheck: clean;
|
||||
- production Vite build: clean;
|
||||
- Swift helper parse: clean;
|
||||
- `git diff --check`: clean.
|
||||
|
||||
## What is accepted, partial and not started
|
||||
|
||||
Accepted:
|
||||
|
||||
- physical K1 Bridge control lifecycle START/live/STOP/READY;
|
||||
- durable session catalog, point cloud and camera playback;
|
||||
- bounded content-addressed recorded camera job;
|
||||
- Windows digest verification and deterministic 56-frame decode;
|
||||
- pinned Triton/YOLOX-S inference on RTX 4090;
|
||||
- immutable content-addressed result and exact-repeat reuse;
|
||||
- validated optional Rerun camera/Boxes2D projection;
|
||||
- strict SSH host-key pinning and local-lab access path;
|
||||
- reproducible Windows worker repository and operator scripts.
|
||||
|
||||
Partial/laboratory-only:
|
||||
|
||||
- Quick Connect is physically accepted on one prepared Mac but has no clean-host
|
||||
credential bootstrap and is closed as a product route;
|
||||
- Windows is a reproducible lab worker but Triton is loopback-only and manually
|
||||
started;
|
||||
- SSH is source-restricted by firewall and the Mac uses a key, but sshd still
|
||||
permits passwords;
|
||||
- the worker result is recorded-only and camera-only;
|
||||
- GPU resource use is shared with unrelated Frigate/Ollama workloads.
|
||||
|
||||
Not accepted/not implemented:
|
||||
|
||||
- Direct Connect physical test;
|
||||
- second K1 or firmware portability matrix;
|
||||
- RTK, UGV, UAV, calibrated sensor time or camera/LiDAR extrinsics;
|
||||
- long TEST007 epoch throughput/resource qualification;
|
||||
- bounded live inference, queue/drop policy and recovery testing;
|
||||
- direct authenticated LAN worker transport;
|
||||
- Tailscale/routed worker transport and deny-by-default grants;
|
||||
- point-cloud inference, camera/LiDAR fusion, tracking, segmentation or free
|
||||
space;
|
||||
- actuator/autopilot integration, mission executor or safety authority;
|
||||
- production process supervision, operation journal, retention, encryption,
|
||||
replication and resource scheduler;
|
||||
- custom installer/node console/model marketplace, intentionally deferred.
|
||||
|
||||
## Recommended next gates in exact order
|
||||
|
||||
1. Continue K1 operation through the accepted Bridge/direct-LAN path. Preserve
|
||||
the prepared Quick Connect implementation and evidence without extending it.
|
||||
2. Commit the retained K1 mechanism/evidence/docs in logical commits when the
|
||||
owner chooses; do not claim clean-host portability.
|
||||
3. Qualify the 206-second TEST007 camera epoch on the same immutable model
|
||||
profile, recording transfer, decode, inference, GPU memory, queue depth and
|
||||
co-tenant effects.
|
||||
4. Define a direct authenticated worker endpoint that preserves the current
|
||||
`compute-job/result` schemas; reserve a non-conflicting worker address and add
|
||||
a source-restricted firewall rule before any LAN port exposure.
|
||||
5. Add bounded live fan-out from the sole Mac RTSP owner with explicit sampling,
|
||||
latest-wins/drop counters, deadlines, stale state and failure isolation.
|
||||
6. Kill/restart Triton, power off the worker and break transport while proving
|
||||
K1 control/raw archive continue and the UI reports degraded perception.
|
||||
7. Repeat the same endpoint contract through Tailscale; do not expose public
|
||||
Triton ports.
|
||||
8. Only after this evaluate tracking, segmentation/free-space and a LiDAR CPU
|
||||
processor. Keep every result versioned and non-safety until calibrated and
|
||||
separately accepted.
|
||||
9. Evaluate ROS 2/BehaviorTree only when an actual onboard computer,
|
||||
robot/autopilot and actuator/safety contract exist. Evaluate Zenoh only
|
||||
against the measured direct transport.
|
||||
|
||||
## Canonical local sources for the next chat
|
||||
|
||||
- `docs/10_EXTERNAL_PERCEPTION_WORKER.md` — job/result and Rerun contract;
|
||||
- `docs/adr/0014-bounded-external-perception-worker.md` — accepted architecture;
|
||||
- `docs/lab/005_FIRST_RECORDED_PERCEPTION_20260719.redacted.md` — physical
|
||||
recorded evidence;
|
||||
- `src/k1link/compute/jobs.py` — compute job creation/validation;
|
||||
- `src/k1link/compute/results.py` — result validation/Rerun projection inputs;
|
||||
- `docs/adr/0013-k1-local-connection-matrix.md` — K1 connection directions;
|
||||
- `src/k1link/device_plugins/xgrids_k1/ble/ap_activation.py` — reviewed AP
|
||||
activation session;
|
||||
- `src/k1link/host_network/wifi.py` — OS Wi-Fi profile boundary;
|
||||
- `plugins/xgrids-k1/macos/associate_wifi.swift` — CoreWLAN and Keychain helper;
|
||||
- this handoff — exact current machine/runtime/SSH/partial-acceptance snapshot.
|
||||
Reference in New Issue
Block a user