NODEDC_MISSION_CORE/docs/audits/2026-09-20-reference-preparation-and-recovery.md

Reference preparation and recovery before the long field route

Scope and decision

Owner-authorized follow-up: useful, evidence-based preparation before recording a 1–2 km reference. Preserve dense-start-first v6, all eligible route regions, ambiguity rejection, source freshness and independent recording ownership. No new field capture, K1 command, threshold relaxation or vehicle authority. The current worktree already contained substantial unrelated and earlier work; no commit, reset or unrelated cleanup was performed.

Implementation

• missions/sources.py: prepare one verified private source snapshot for the tiles of one atlas, instead of copying and hashing the raw recording again for every tile. Verify original source identity and digests before staging and after assembly, before returning the atlas. Staging is removed on success, corruption, extraction failure and cancellation. Cancellation from the shared recording copier is translated to InterruptedError, so it remains a cancelled planning preparation rather than a product error. Single-submap API remains compatible; numerical and presentation maps remain separate derivatives.
• missions/reference_map.py: assemble inside that verified snapshot context. Retain the original tile order, 40 m tile geometry and first-point-per-voxel deduplication semantics. Lightweight archive/test adapters without a prepared snapshot capability retain the existing submap interface.
• missions/reference_window.py: an immutable, run-owned spatial index stores source indices grouped in 10 m cells. Exact spherical filtering retains all eligible points, duplicates, boundary points and original source order. No point-count acceptance gate, thinning, expanded acceptance radius or changed transform convention. A reference-identity mismatch fails explicitly.
• missions/stationary_live.py: build the index once for the run and reuse it for fresh validation windows. The index is local to the live worker and is discarded with it. Lookup mode and examined-point count are retained in each window's evidence alongside the unchanged target-array hash.
• missions/route_relocalization.py: preprocess the exact local target once per candidate place, sharing its GICP tree across that place's three yaw seeds. Seed list, ranking, complete-search obligation, deadlines, ambiguity and fresh confirmation are unchanged. This is not a new global retrieval algorithm and does not qualify arbitrary kilometre-scale cold starts.
• scripts/planning_archive_source.py, check_planning_live_bootstrap.py: explicit receipt-drop interval for the private archive adapter, preserving surviving source payloads, identity and receipt intervals. Capture ownership, recovery and source integrity are checked independently of recognition success.
• scripts/check_reference_windows.py, check_reference_preparation.py: bounded real-data exactness and preparation checks. They do not use a live singleton, connect to hardware, modify physical reports or generate synthetic kilometres.

Exact geometry and measured preparation

Private physical source: existing reference JA-SADOVAYA-001 and independent physical repeat 018, with source digests from the previous sealed acceptance.

qualification-20260920-reference-index-018.json and its -r2 successor show the index's development measurements. All 21 saved fresh-fit input windows matched both the full-scan result and original frozen registration target bit-for-bit. Initial default sorting was slower; stable integer sorting retained exact source order with lower overhead. These are retained measurements, not a claim that the short reference became faster.

Final preparation evidence: .runtime/qualification-20260920-reference-preparation-full/report.json. The harness uses the real PlanningSources verification/staging/export/map path with a private, hash-bound archive catalog adapter; no production catalog or application state is changed.

Measured reference	Selected physical interval	Complete saved trajectory
Recorded length	120.073 m	487.611 m
Preparation tiles	4	13
Numerical points	172,954	390,668
Numerical array bytes	4,150,896	9,376,032
Preparation wall time	8.671 s	28.109 s
Index construction	0.081 s	0.187 s
Index integer-array bytes, excluding dictionary overhead	1,383,632	3,125,344
Median full-scan window	3.076 ms	5.682 ms
Median indexed window	3.508 ms	5.109 ms
Median examined points	76,985	114,760

The selected interval's reconstructed map is bit-identical to the original physical report. For each map, 21 real query windows matched the corresponding full-scan implementation exactly. The complete map contains additional nearby geometry, so its targets are not claimed identical to targets on the smaller map. This is preparation/lookup evidence, not a 488 m independent traversal or recognition acceptance on the expanded map. Timings are one bounded pass per map, not a latency guarantee or a synthetic load test.

Trajectory export took 0.353 s for 4,959 poses; its private cache is 868,331 bytes. All input/code digest checks and staging-cleanup checks passed.

Real archive baseline and injected loss

.runtime/qualification-20260920-map-preparation-018-baseline replays complete 018 spatial receipts at 1x through the production ingress/decoder/planner, numerical child and in-process 10 Hz scene-delta endpoint. All 13 acceptance checks passed. Tracking began at 45.84 s; 22/22 disjoint fresh checks passed over 85.023 m, with no recovery before admitted STOP. Both spatial queues published 1,439 and consumed 1,435 receipts, with zero overflow; four per modality remained queued at STOP, not lost from raw recording. Maximum depths: lidar 10/32, pose 9/32. Maximum publisher lag 0.048 s; 1,261 scene requests. Compared with the previous 52.40 s replay this acquired earlier, but the comparison is not a controlled performance guarantee and scheduling changes fresh-window boundaries.

.runtime/qualification-20260920-map-preparation-018-gap uses the same archive with only derived pose/lidar receipts from [70, 80) seconds omitted. Surviving timestamps are not compressed and raw files are unchanged. Exactly 198 receipts were omitted (99 each). All 17 checks passed: tracking established before the fault; authority cleared at about 74.25 s when the last accepted result expired; recovery began with capture still active; ordinary STOP completed the study; no previous transform was resurrected as live authority.

The recorded operator continues walking after this artificial loss and stops for less than the required 10-second stationary prefix at the end. Recovery therefore remains unconfirmed. The recorded 270 recovery_attempt increments are repeated motion-interrupted collection attempts, not 270 expensive route searches. No new successful numerical recovery or stationary behaviour was invented. Both queues again had zero overflow (1,340 published / 1,337 consumed per modality); three remained queued at STOP. This proves continued recovery ownership and fail-closed behaviour on moving real data, not successful physical stop-and-reacquire. That success remains a separate field check; focused tests cover the orchestration with a new stationary prefix and three fresh checks.

Both replays sealed unchanged executed code and original inputs. The subsequent production change was only cancellation-exception translation in sources.py; these archive replays supply already-prepared geometry and do not exercise that method. The later real preparation probe and final focused suite exercise the final sources.py. No executed replay source was edited during its run.

Tests and runtime

Final focused suite: 152 tests, zero failures/errors/skips, retained in .runtime/qualification-20260920-reference-preparation-tests.xml. Coverage includes exact indexed lookup, source mismatch, one snapshot per atlas, mutation after assembly, extraction failure, cancellation during copying and between tiles, unchanged single-submap API, survivor clocks/payloads, complete ranked search, exhausted budget, ambiguity, late jobs, actual loss/recovery, STOP ownership, route limits, recording separation and lease cleanup. Ruff and whitespace checks passed; existing TestClient/httpx deprecation warning remains. All heavy checks ran sequentially; no Mac synthetic load/stress job.

The canonical 8000 LaunchAgent was reloaded using its reviewed plan/apply tool, after proving no acquisition and resolved/inactive device-command state. Current/desired plist SHA remained 5d7a58122c016f8021fede27d1b21891ee20238fc38c281e5b33574ff8204e3f; backup and health acceptance succeeded. This is a local code reload, not a NAS artifact deployment, data-root migration or scanner command. The UI and 80 m presentation ceiling were not changed.

Post-reload acceptance: operational health, idle acquisition/control, resolved inactive physical command, preserved historical completed planning result and 80 m ceiling. PID 6431 was the sole backend listener on 8000; none on 8765 and no temporary replay/numerical/test/Docker workload remained. Swap decreased from 5,947.50 to 5,931.50 MiB across the sequential checks. A concise result was appended to the existing MISSIONCOR-81 card via ops-context; private identifiers, paths and raw operational evidence were kept local.

Remaining scaling work and next field gate

The preparation still decodes selected frames after two sequential archive passes per tile and merges voxel keys over the accumulated map. These costs still grow with route size. Shared staging removes repeated copying/hashing, not all route-length-dependent work. The atlas remains in memory; it is not a disk-paged large-area map. A shared seekable frame index / streaming tile builder is the next preparation optimisation if the long capture's measurements require it.

Protective implementation bounds still exist: export/extraction wall budgets, trajectory cache byte/pose count bounds, per-tile frame/point budgets, queue capacity and numerical freshness deadlines. None is evidence of a 100 m maximum. They were identified, not blindly removed. The original 100,000-point local-map rejection remains absent; the separate 100,000 trajectory pose bound remains. The current 4,959-pose reference does not qualify that bound at long dwell times.

Cold route-wide search still verifies every eligible 5 m anchor with three precise seeds within its existing budget. Around 2 km gives 401 anchors; sharing target preprocessing does not prove this exhaustive stage will finish on time. Do not weaken ambiguity checks or accept an incomplete search to hide this.

Morning sequence remains: ordinary recording of one approximately 2 km reference (line or convenient loop); verify complete capture, export/map preparation, memory and source integrity; then a separate repeat from a known start. Cold mid-route restart on the long map and physical stop-and-reacquire are separate gates. No 2 km, seasonal, absolute-position or autonomous-driving acceptance is claimed by the present increment.