NODEDC_MISSION_CORE/docs/audits/2026-09-20-reference-preparation-and-recovery.md

# Reference preparation and recovery before the long field route

## Scope and decision

Owner-authorized follow-up: useful, evidence-based preparation before recording
a 1–2 km reference. Preserve dense-start-first v6, all eligible route regions,
ambiguity rejection, source freshness and independent recording ownership.
No new field capture, K1 command, threshold relaxation or vehicle authority.
The current worktree already contained substantial unrelated and earlier work;
no commit, reset or unrelated cleanup was performed.

## Implementation

- `missions/sources.py`: prepare one verified private source snapshot for the
  tiles of one atlas, instead of copying and hashing the raw recording again
  for every tile. Verify original source identity and digests before staging
  and after assembly, before returning the atlas. Staging is removed on success,
  corruption, extraction failure and cancellation. Cancellation from the shared
  recording copier is translated to `InterruptedError`, so it remains a cancelled
  planning preparation rather than a product error. Single-submap API remains
  compatible; numerical and presentation maps remain separate derivatives.
- `missions/reference_map.py`: assemble inside that verified snapshot context.
  Retain the original tile order, 40 m tile geometry and first-point-per-voxel
  deduplication semantics. Lightweight archive/test adapters without a prepared
  snapshot capability retain the existing submap interface.
- `missions/reference_window.py`: an immutable, run-owned spatial index stores
  source indices grouped in 10 m cells. Exact spherical filtering retains all
  eligible points, duplicates, boundary points and original source order.
  No point-count acceptance gate, thinning, expanded acceptance radius or changed
  transform convention. A reference-identity mismatch fails explicitly.
- `missions/stationary_live.py`: build the index once for the run and reuse it
  for fresh validation windows. The index is local to the live worker and is
  discarded with it. Lookup mode and examined-point count are retained in each
  window's evidence alongside the unchanged target-array hash.
- `missions/route_relocalization.py`: preprocess the exact local target once
  per candidate place, sharing its GICP tree across that place's three yaw
  seeds. Seed list, ranking, complete-search obligation, deadlines, ambiguity
  and fresh confirmation are unchanged. This is not a new global retrieval
  algorithm and does not qualify arbitrary kilometre-scale cold starts.
- `scripts/planning_archive_source.py`, `check_planning_live_bootstrap.py`:
  explicit receipt-drop interval for the private archive adapter, preserving
  surviving source payloads, identity and receipt intervals. Capture ownership,
  recovery and source integrity are checked independently of recognition success.
- `scripts/check_reference_windows.py`, `check_reference_preparation.py`: bounded
  real-data exactness and preparation checks. They do not use a live singleton,
  connect to hardware, modify physical reports or generate synthetic kilometres.

## Exact geometry and measured preparation

Private physical source: existing reference JA-SADOVAYA-001 and independent
physical repeat 018, with source digests from the previous sealed acceptance.

`qualification-20260920-reference-index-018.json` and its `-r2` successor show
the index's development measurements. All 21 saved fresh-fit input windows
matched both the full-scan result and original frozen registration target
bit-for-bit. Initial default sorting was slower; stable integer sorting retained
exact source order with lower overhead. These are retained measurements, not
a claim that the short reference became faster.

Final preparation evidence:
`.runtime/qualification-20260920-reference-preparation-full/report.json`.
The harness uses the real `PlanningSources` verification/staging/export/map
path with a private, hash-bound archive catalog adapter; no production catalog
or application state is changed.

| Measured reference | Selected physical interval | Complete saved trajectory |
|---|---:|---:|
| Recorded length | 120.073 m | 487.611 m |
| Preparation tiles | 4 | 13 |
| Numerical points | 172,954 | 390,668 |
| Numerical array bytes | 4,150,896 | 9,376,032 |
| Preparation wall time | 8.671 s | 28.109 s |
| Index construction | 0.081 s | 0.187 s |
| Index integer-array bytes, excluding dictionary overhead | 1,383,632 | 3,125,344 |
| Median full-scan window | 3.076 ms | 5.682 ms |
| Median indexed window | 3.508 ms | 5.109 ms |
| Median examined points | 76,985 | 114,760 |

The selected interval's reconstructed map is bit-identical to the original
physical report. For each map, 21 real query windows matched the corresponding
full-scan implementation exactly. The complete map contains additional nearby
geometry, so its targets are not claimed identical to targets on the smaller
map. This is preparation/lookup evidence, not a 488 m independent traversal or
recognition acceptance on the expanded map. Timings are one bounded pass per
map, not a latency guarantee or a synthetic load test.

Trajectory export took 0.353 s for 4,959 poses; its private cache is 868,331 bytes.
All input/code digest checks and staging-cleanup checks passed.

## Real archive baseline and injected loss

`.runtime/qualification-20260920-map-preparation-018-baseline` replays complete
018 spatial receipts at 1x through the production ingress/decoder/planner,
numerical child and in-process 10 Hz scene-delta endpoint. All 13 acceptance
checks passed. Tracking began at 45.84 s; 22/22 disjoint fresh checks passed over
85.023 m, with no recovery before admitted STOP. Both spatial queues published
1,439 and consumed 1,435 receipts, with zero overflow; four per modality remained
queued at STOP, not lost from raw recording. Maximum depths: lidar 10/32, pose
9/32. Maximum publisher lag 0.048 s; 1,261 scene requests. Compared with the
previous 52.40 s replay this acquired earlier, but the comparison is not a
controlled performance guarantee and scheduling changes fresh-window boundaries.

`.runtime/qualification-20260920-map-preparation-018-gap` uses the same archive
with only derived pose/lidar receipts from [70, 80) seconds omitted. Surviving
timestamps are not compressed and raw files are unchanged. Exactly 198 receipts
were omitted (99 each). All 17 checks passed: tracking established before the
fault; authority cleared at about 74.25 s when the last accepted result expired;
recovery began with capture still active; ordinary STOP completed the study;
no previous transform was resurrected as live authority.

The recorded operator continues walking after this artificial loss and stops
for less than the required 10-second stationary prefix at the end. Recovery
therefore remains unconfirmed. The recorded 270 `recovery_attempt` increments
are repeated motion-interrupted collection attempts, **not 270 expensive route
searches**. No new successful numerical recovery or stationary behaviour was
invented. Both queues again had zero overflow (1,340 published / 1,337 consumed
per modality); three remained queued at STOP. This proves continued recovery
ownership and fail-closed behaviour on moving real data, not successful physical
stop-and-reacquire. That success remains a separate field check; focused tests
cover the orchestration with a new stationary prefix and three fresh checks.

Both replays sealed unchanged executed code and original inputs. The subsequent
production change was only cancellation-exception translation in `sources.py`;
these archive replays supply already-prepared geometry and do not exercise that
method. The later real preparation probe and final focused suite exercise the
final `sources.py`. No executed replay source was edited during its run.

## Tests and runtime

Final focused suite: 152 tests, zero failures/errors/skips, retained in
`.runtime/qualification-20260920-reference-preparation-tests.xml`.
Coverage includes exact indexed lookup, source mismatch, one snapshot per atlas,
mutation after assembly, extraction failure, cancellation during copying and
between tiles, unchanged single-submap API, survivor clocks/payloads, complete
ranked search, exhausted budget, ambiguity, late jobs, actual loss/recovery,
STOP ownership, route limits, recording separation and lease cleanup.
Ruff and whitespace checks passed; existing TestClient/httpx deprecation warning
remains. All heavy checks ran sequentially; no Mac synthetic load/stress job.

The canonical 8000 LaunchAgent was reloaded using its reviewed plan/apply tool,
after proving no acquisition and resolved/inactive device-command state.
Current/desired plist SHA remained
`5d7a58122c016f8021fede27d1b21891ee20238fc38c281e5b33574ff8204e3f`;
backup and health acceptance succeeded. This is a local code reload, not a NAS
artifact deployment, data-root migration or scanner command. The UI and 80 m
presentation ceiling were not changed.

Post-reload acceptance: operational health, idle acquisition/control, resolved
inactive physical command, preserved historical completed planning result and
80 m ceiling. PID 6431 was the sole backend listener on 8000; none on 8765 and
no temporary replay/numerical/test/Docker workload remained. Swap decreased
from 5,947.50 to 5,931.50 MiB across the sequential checks. A concise result was
appended to the existing MISSIONCOR-81 card via ops-context; private identifiers,
paths and raw operational evidence were kept local.

## Remaining scaling work and next field gate

The preparation still decodes selected frames after two sequential archive
passes per tile and merges voxel keys over the accumulated map. These costs
still grow with route size. Shared staging removes repeated copying/hashing,
not all route-length-dependent work. The atlas remains in memory; it is not a
disk-paged large-area map. A shared seekable frame index / streaming tile builder
is the next preparation optimisation if the long capture's measurements require it.

Protective implementation bounds still exist: export/extraction wall budgets,
trajectory cache byte/pose count bounds, per-tile frame/point budgets, queue
capacity and numerical freshness deadlines. None is evidence of a 100 m maximum.
They were identified, not blindly removed. The original 100,000-point local-map
rejection remains absent; the separate 100,000 **trajectory pose** bound remains.
The current 4,959-pose reference does not qualify that bound at long dwell times.

Cold route-wide search still verifies every eligible 5 m anchor with three
precise seeds within its existing budget. Around 2 km gives 401 anchors; sharing
target preprocessing does not prove this exhaustive stage will finish on time.
Do not weaken ambiguity checks or accept an incomplete search to hide this.

Morning sequence remains: ordinary recording of one approximately 2 km reference
(line or convenient loop); verify complete capture, export/map preparation,
memory and source integrity; then a separate repeat from a known start. Cold
mid-route restart on the long map and physical stop-and-reacquire are separate
gates. No 2 km, seasonal, absolute-position or autonomous-driving acceptance is
claimed by the present increment.