docs(observatory): record cache proof and pending activation

experiments/perception/OBSERVATORY_PUBLISHED_CACHE_2026-09-03.md

@@ -0,0 +1,92 @@
+# Observatory published-cache increment — 2026-09-03
+
+## Status and scope
+
+Implemented and contract-tested; **not activated on canonical8000 yet**.
+Implementation commit: `1e4ddc2` on `codex/m5-1-observatory` (no push).
+This is stage1 source/provenance/cache work, not model or recorded-run evidence.
+No Worker containers, model profiles, raw recordings, result history, capture,
+network settings or physical controls were changed. No Synology deployment.
+
+## Contract
+
+- Exact key: admitted source snapshot + immutable RunDefinition (including
+  executor image/release, models, adapter, resource and result contracts).
+- Candidate: successful queue job with durable `publication_state=published`.
+  Legacy `succeeded/not-required`, pending or failed publication is not a hit.
+- Verify full job/source/definition/profile provenance, Session binding, sealed
+  package and CAS manifest identity, exact artifact inventory and file hashes.
+- Return the newest usable exact result with `existing / open-existing`,
+  `submission_allowed=false` and path-free evidence identities. Different source,
+  changed snapshot or profile version remains a distinct computation.
+- Existing catalog/preflight/viewer contracts are reused. No per-LAB UI branch.
+  Full selector filtering and progress are deliberately the next product stage.
+- Cached preflight does not prepare raw data or call a Worker; read-only result
+  review is independent of current source compatibility/dispatch availability.
+  Source snapshot comparison reads the catalog, not all source media.
+- Corrupt/missing/symlinked result objects do not suppress a fresh calculation.
+  History is retained. Cached review is not a navigation/safety acceptance.
+
+## IO and concurrency
+
+First use or changed metadata streams result objects in 1MiB chunks. Content
+checks use a bounded4096-entry fingerprint memo; each lookup rechecks file
+device/inode/size/mtime_ns/ctime_ns. Contracts/manifests remain independently
+parsed and checked. Warm refresh does not reread unchanged large result objects.
+Cold verification can cost result-store IO and is not advertised as constant-time.
+
+Two additive queue indexes support exact published-source and computation lookup.
+Publication/INSERT races are fenced in the existing transaction. Its verifier
+does not re-enter the queue DB or hash large files under the write lock. A cold,
+concurrent or changed verification requests Refresh with HTTP409, never silently
+creates another job. Confirmed corruption permits retry with a new key; the
+original idempotency key always refers to the original job.
+
+## Verification
+
+- 120 focused backend tests PASS: portable publisher/cache, setup API/projector,
+  queue binding, recorded queue, Worker API, publication retry/reconciliation.
+- 23 focused frontend tests PASS: portable catalog/decoder and application
+  architecture, sequential Node runner with192MiB heap cap. Peak RSS~108MiB.
+- Focused strict TypeScript check of the portable decoder/fetch contract PASS,
+ 192MiB heap cap; peak process RSS~366MiB including compiler/native memory.
+- Ruff on all changed Python files PASS; mypy five changed core/API modules PASS.
+- Tests use synthetic filesystem/SQLite packages, not historical LAB outputs as
+  an inference substitute. They cover complete publish→cache→view, missing and
+  same-size corruption with restored mtime, symlink substitution, changed central
+  manifest/provenance/source/version, publication race, idempotent retry and no
+  source preparation/model submission when an exact result is available.
+- First fixture run correctly rejected legacy `succeed()` as no publication
+  outbox. Fixture now exercises `complete_for_publication()` followed by actual
+  publisher and durable `mark_published()`; the production gate was not relaxed.
+- Existing FastAPI/httpx test deprecation warning remains; dependencies unchanged.
+- Legacy SQLite migration is covered: indexes are created after publication
+  columns are added, preserving the pre-existing job and idempotency identity.
+
+Reproduce backend tests with `.venv/bin/pytest -o addopts='' -q` and these files:
+`tests/test_observatory_portable_result_publisher.py`,
+`tests/test_observatory_portable_setup_api.py`,
+`tests/test_observatory_portable_setup_projection.py`,
+`tests/test_observatory_portable_queue_binding.py`,
+`tests/test_observatory_recorded_jobs.py`, `tests/test_observatory_worker_api.py`,
+`tests/test_observatory_publication_retry_api.py`,
+`tests/test_observatory_publication_reconciler.py`.
+
+Frontend, from `apps/control-station`:
+`NODE_OPTIONS=--max-old-space-size=192 node --test --test-concurrency=1 test/observatoryLaboratorySetups.test.mjs test/applicationArchitecture.test.mjs`.
+
+## Activation still required
+
+Mac pressure rose from1 to2 during the turn, swap~7.6–7.8GiB; no temporary heavy job
+remains. Existing operator Docker services were inspected, not stopped. The
+resource gate in `mission-core-product-ui` defers full build and browser QA.
+Canonical8000 remains PID57796 on the previous matched release, `/api/health`
+HTTP200. No listeners on8765/4173; unrelated launcher5173 is not this project.
+Do not claim that the new cache is already live in the operator interface.
+
+After memory pressure normalizes: sequential production typecheck/tests/build,
+activate the matching backend/frontend on canonical8000, verify health/catalog
+and browser decoder behavior without creating inference jobs, then update this
+status. Do not start a second backend or replay process. Stage2 then implements
+the selector/progress/full recorded calculation and sequential M4.9T5/LAB V1
+acceptance. Neither this report nor a cached viewer FPS proves onboard realtime.