fix(k1): restore low-latency live recovery path
This commit is contained in:
DCCONSTRUCTIONS
@@ -0,0 +1,213 @@
|
||||
# K1 ↔ Mission Core internal live baseline — 2026-08-22
|
||||
|
||||
Status: accepted internal engineering reference
|
||||
|
||||
Scope: one owner-controlled XGRIDS/LixelKity K1, exact firmware 3.0.2
|
||||
|
||||
Connection: Mission Core Bridge/direct-LAN path
|
||||
|
||||
Operator path: canonical Mission Core UI only
|
||||
|
||||
## Classification
|
||||
|
||||
This is the fast **Mission Core ↔ K1 internal reference**, not a vendor/LixelGO
|
||||
reference, not a production SLA and not proof for arbitrary Wi-Fi conditions,
|
||||
another K1, another firmware, Quick Connect, Direct Connect, LTE, a tunnel or a
|
||||
vehicle installation.
|
||||
|
||||
The reference exists because the same visible regressions repeatedly returned:
|
||||
slow START/STOP, delayed point-cloud admission, a camera that disappeared and
|
||||
reopened, browser decoder errors and a discrete trajectory. It records both the
|
||||
fastest clean internal run and the accepted recovery-capable checkpoint that
|
||||
followed it. Raw MQTT, video and device evidence remain outside Git; only
|
||||
redacted summaries, hashes and engineering conclusions are retained here.
|
||||
|
||||
No point count, image quality, bitrate, source FPS or K1 protocol setting was
|
||||
reduced to obtain either result.
|
||||
|
||||
## Baseline A — fast ideal-path reference
|
||||
|
||||
- Code: `eaad9de` (`fix(k1): stabilize live recovery and media admission`).
|
||||
- Evidence session: `20260822T105904Z_viewer_live`.
|
||||
- Local redacted manifest SHA-256:
|
||||
`990b9433d40965b2b712df155f5f96d38770d53fddde804852c11eca90d96b3e`.
|
||||
- Local MQTT summary SHA-256:
|
||||
`9438cc8947c7a868ff8ffcfc163233dfea40a95b7aae34050806bb072b2a7750`.
|
||||
- Local camera summary SHA-256:
|
||||
`bc333e1dd1e7800204b242eecd01a83d1c91c388ab2202a238e8d2fc4a43aa14`.
|
||||
- Final phase: `idle`; stop reason: `external_stop`; rejected MQTT messages: 0.
|
||||
|
||||
Operator-observed UI/physical timing:
|
||||
|
||||
| Boundary | Observed time |
|
||||
| --- | ---: |
|
||||
| START click → K1 calibration onset | < 5 s |
|
||||
| K1 calibration | 21 s |
|
||||
| calibration end → visible point cloud | 2 s |
|
||||
| calibration end → visible right camera | 4 s |
|
||||
| STOP click → physical stop onset | 1 s |
|
||||
|
||||
Recorded Mission Core facts:
|
||||
|
||||
| Metric | Value |
|
||||
| --- | ---: |
|
||||
| PCL frames / pose frames | 121 / 132 |
|
||||
| points published | 412,175 |
|
||||
| point decode errors | 0 |
|
||||
| preview dropped | 0 |
|
||||
| MQTT callback → publish p50 / p95 | 23.839 / 41.541 ms |
|
||||
| camera archive | complete, 70 segments, 5,645,411 bytes |
|
||||
| camera failure code | none |
|
||||
| offline H.264 decode errors | 0 |
|
||||
|
||||
This run used a fresh, short physical-command history: the terminal ledger was
|
||||
2,989 bytes at revision 15 with no archive segments. That distinction is
|
||||
material. The timings are a low-latency reference, not evidence that a mature
|
||||
recovery ledger can be cryptographically reread several times per UI poll for
|
||||
free.
|
||||
|
||||
## Regression that exposed the shared bottleneck
|
||||
|
||||
The recovery-capable working line later produced session
|
||||
`20260822T124524Z_viewer_live`:
|
||||
|
||||
- START click → calibration onset: 26 s;
|
||||
- calibration end → cloud: about 2 s;
|
||||
- calibration end → camera: 19 s with repeated disappearance and large lag;
|
||||
- STOP click → physical stop onset: 12 s;
|
||||
- preview dropped: 393;
|
||||
- callback → publish p50/p95: 110.331/309.434 ms;
|
||||
- the retained fMP4 archive contained real H.264 macroblock/bytestream decode
|
||||
errors, while the ideal reference did not.
|
||||
|
||||
The camera UI resets were therefore a symptom, not the primary cause. Increasing
|
||||
preview queues or reducing media/point quality would only hide overload.
|
||||
|
||||
Profiling found repeated full parsing, validation and archive-hash traversal of
|
||||
the physical START/STOP ledger inside passive `state.read`, nested public state
|
||||
projections and the first-PCL camera path. The mature ledger was 18,378 bytes at
|
||||
revision 1,219 plus five immutable archive segments / 318,027 bytes. One passive
|
||||
state request took 240–680 ms while the UI requested state four times per second.
|
||||
That contention delayed MQTT publication and FFmpeg stdout consumption together.
|
||||
|
||||
## Baseline B — accepted recovery-capable checkpoint
|
||||
|
||||
Evidence session: `20260822T130323Z_viewer_live`.
|
||||
|
||||
- Local redacted manifest SHA-256:
|
||||
`f870bedd9036a3f03993344d405eab4ef668d1f0fdae55e16adf8ecdcb51f48a`.
|
||||
- Local MQTT summary SHA-256:
|
||||
`2d42d31c14d98d77bcefc8a1b9983e102cace2375ada9e5f89f021a33920a55d`.
|
||||
- Local camera summary SHA-256:
|
||||
`1386ea16615a9716b699d6ba35733ef26d367b6a5257d572cde11509e546cdda`.
|
||||
- Final phase: `idle`; stop reason: `external_stop`; rejected MQTT messages: 0.
|
||||
|
||||
Operator-observed UI/physical timing:
|
||||
|
||||
| Boundary | Observed time |
|
||||
| --- | ---: |
|
||||
| START click → K1 calibration onset | 14 s |
|
||||
| K1 calibration | 22 s |
|
||||
| calibration end → visible point cloud | about 1 s |
|
||||
| calibration end → visible right camera | 8 s |
|
||||
| STOP click → physical stop onset | 7 s |
|
||||
|
||||
Recorded Mission Core facts:
|
||||
|
||||
| Metric | Value |
|
||||
| --- | ---: |
|
||||
| PCL frames / pose frames | 369 / 400 |
|
||||
| points published | 1,318,018 |
|
||||
| point decode errors | 0 |
|
||||
| preview dropped | 70 |
|
||||
| callback → publish p50 / p95 | 83.934 / 223.589 ms |
|
||||
| first-PCL camera admission | 65 ms |
|
||||
| camera authority / FFmpeg / post-spawn commit | 5 / 16 / 2 ms |
|
||||
| total backend camera activation | 23 ms |
|
||||
| backend activation → browser playing | 5.383 s |
|
||||
| browser camera restarts / slow-consumer retirements | 0 / 0 |
|
||||
| camera archive | complete, 342 segments, 27,469,046 bytes |
|
||||
| camera failure code | none |
|
||||
| offline H.264 decode errors | 0 |
|
||||
|
||||
The accepted STOP was emitted once. Mission Core received an exactly correlated
|
||||
successful K1 application response after about 2.35 s, later observed fresh
|
||||
`READY` and resolved the durable physical command as
|
||||
`stop-standby-observed`. No automatic physical retry was introduced.
|
||||
|
||||
After the bounded read-path change, passive `state.read` measured 23–43 ms on
|
||||
the same mature ledger instead of 240–680 ms.
|
||||
|
||||
## What restored the usable path
|
||||
|
||||
1. A single verified physical-ledger snapshot is shared across one public state
|
||||
projection instead of being recursively reloaded by nested projections.
|
||||
2. Process-local control facts can be read without implicitly traversing the
|
||||
durable physical ledger again.
|
||||
3. Passive snapshots reuse an already cryptographically verified ledger only
|
||||
while the ledger directory, main file, archive directory and every bounded
|
||||
archive entry retain the exact device/inode/mode/owner/link-count/size/
|
||||
`mtime_ns`/`ctime_ns` fingerprint. Every mutation and proof still performs a
|
||||
complete reload. Cross-process atomic writes and same-size archive tampering
|
||||
invalidate the cache and fail closed.
|
||||
4. Camera authority is fully proved before FFmpeg `Popen`; after spawn only a
|
||||
bounded in-memory compare-and-swap may commit the producer. Reader threads
|
||||
therefore start without a second durable-ledger pause that can back up the
|
||||
fMP4 pipe.
|
||||
5. First-authoritative-PCL camera admission no longer recursively rereads the
|
||||
same physical ledger on the 10 Hz ingress callback.
|
||||
6. Structured timings were added for first-PCL admission, camera activation and
|
||||
STOP dispatch. These are diagnostics only; they do not add a K1 command,
|
||||
status, retry or product window.
|
||||
|
||||
## Fragile boundaries that remain open
|
||||
|
||||
- Baseline B is usable but does not equal Baseline A: 70 preview drops and
|
||||
83.934/223.589 ms p50/p95 remain above the clean reference of 0 drops and
|
||||
23.839/41.541 ms.
|
||||
- START and physical STOP onset remain materially slower than the fast reference.
|
||||
The current evidence separates local validation, correlated application
|
||||
response and later `READY`, but does not yet expose a device-side calibration
|
||||
onset or physical motor/LED timestamp on the same clock.
|
||||
- Camera presentation still depends on RTSP startup, the next usable H.264
|
||||
parameter-set/IDR boundary and browser MSE admission. The accepted archive has
|
||||
non-monotonic-DTS warnings but no H.264 decode corruption.
|
||||
- A successful application response is necessary but not sufficient to dismiss
|
||||
the interface. Mission Core must keep the active scene until authoritative K1
|
||||
state proves the requested physical transition.
|
||||
- Wi-Fi loss, controller-process loss, K1 power/battery loss and return must
|
||||
retain the current no-automatic-START/STOP rule and exact lineage. They need
|
||||
separate UI acceptance runs; this baseline must not be used to infer them.
|
||||
- Bridge remains the accepted product path. Quick Connect/Bridge switching must
|
||||
not reset the whole reactive application or reuse the other mode's reconnect
|
||||
affordance, and must not perturb the Bridge protocol to improve Quick Connect.
|
||||
- The original LixelGO packet captures remain manufacturer-app evidence. They
|
||||
are not interchangeable with this Mission Core internal baseline.
|
||||
|
||||
## Regression rule
|
||||
|
||||
Future K1 work starts from one named baseline and changes one bounded invariant.
|
||||
The canonical acceptance path is the Mission Core UI. A run is rejected if it
|
||||
introduces an archived-camera H.264 decode error, browser `MEDIA_ERR_DECODE`,
|
||||
missing point-cloud admission, missing terminal archive, lost control/acquisition
|
||||
lineage, duplicate physical command or a UI state not backed by authoritative K1
|
||||
evidence. Performance must not be recovered by silently reducing source data.
|
||||
|
||||
## Delivery validation
|
||||
|
||||
- Canonical UI physical run: accepted as Baseline B above.
|
||||
- Focused backend regression suite for the touched viewer diagnostics,
|
||||
acquisition lifecycle, application session, camera gateway and physical
|
||||
ledger: 800/800 passed.
|
||||
- Ruff: passed for every changed Python source and test file.
|
||||
- mypy: passed for all five changed source modules.
|
||||
- `git diff --check`: passed.
|
||||
- Offline decode of the accepted 50-second camera window: no H.264 decode
|
||||
corruption; only non-monotonic-DTS warnings were observed.
|
||||
- Canonical Mission Core service remained available on `127.0.0.1:8000` after
|
||||
validation.
|
||||
|
||||
The next performance increment should be driven by a generated run comparison
|
||||
under ADR 0042. Kafka, Redis or ClickHouse are not a substitute for fixing the
|
||||
local control/media hot path and are not admitted without a measured requirement
|
||||
beyond the existing MQTT/Timescale telemetry contour.
|
||||
Reference in New Issue
Block a user