docs(observatory): record cache activation and Docker memory recovery

This commit is contained in:
DCCONSTRUCTIONS
2026-09-03 09:44:02 +03:00
parent d777f3198b
commit c2710b55a7
3 changed files with 95 additions and 6 deletions
+23 -2
View File
@@ -70,10 +70,31 @@ QA **пока не выполнены**: Mac memory-pressure=2 при swap~7.6
backend/frontend release, health200; новый cache ещё не активирован. Worker,
модели и живые очереди не изменялись. Полный LAB-run не запускался.
[Проверки и следующий шаг](../experiments/perception/OBSERVATORY_PUBLISHED_CACHE_2026-09-03.md).
Этап1 остаётся открытым до сборки/активации согласованного release и проверки
каталога; затем этап2 — селектор, прогресс и полный цикл на записях. Не повторять
Первоначальный ресурсный blocker выше снят следующим инкрементом. Не повторять
уже закрытый ремонт claim/v3 и не возвращаться к сетевым latency canary.
**Активация cache-инкремента, 2026-09-03.** По разрешению владельца перезапущена
старая Docker VM: footprint14GB→1565MB, pressure2→1, swap7722→2746MiB. Браузеры и
Little Snitch не трогали. Три прежних Mission Core контейнера healthy;12 Plane
restart overrides возвращены, Plane выключен. Состояния45 non-Core контейнеров,
image/mount identities сохранены; лимиты Docker и данные не менялись.
Полный typecheck,714 frontend tests и production build PASS, последовательно.
Согласованная версия на8000/PID67747, health200; UI `/assets/index-DSuuMkq4.js`.
Каталоги004TREE/00 совместимы с двумя profiles;01 заблокирован отсутствующим
capture attestation. Точных опубликованных cache hits пока нет; legacy
succeeded/not-required не принят за новый результат. Все6 queue-table row hashes
сохранены. Worker/model jobs не создавались. Browser: каталог, выбор, Refresh,
normal/expanded и Escape проверены, console warnings/errors0.
Этап1 пользовательски ещё не закрыт: текущий селектор смешивает архивные entries
и portable definitions, а source/setup-only выбор последней job показывает
старое «Расчёт завершён» рядом с новым «Рассчитать». Это не текущий cache hit.
Следующий шаг на стыке этапов12: exact-definition queue/status binding,
селектор только нерассчитанных profiles, прогресс и полный цикл на записях.
Положительный publish→cache→view пока доказан synthetic tests, не новым LAB-run.
Подробности/rollback в том же отчёте выше; повторять сборку без изменения кода
или снова освобождать память не требуется.
- Сверить каталог, очередь, публикацию и просмотр для текущих M4.9T5 и LAB V1.
- Связать готовность с точной исходной записью и immutable RunDefinition,
включающей image/model/config/adapter/result-contract identities.
+9 -2
View File
@@ -83,8 +83,11 @@ See the [repair evidence and rollback boundary](../../experiments/perception/OBS
## Third increment: exact published-result cache (2026-09-03)
Implemented in source; activation of the matching backend/frontend release is
pending the local resource gate. This does not claim a completed recorded run.
Implemented and activated as a matching backend/frontend release on canonical8000
on 2026-09-03 after an owner-approved Docker VM restart recovered memory headroom.
Full typecheck,714 frontend tests and production build passed sequentially;
API catalogs and in-app UI were checked without creating inference jobs.
All queue row hashes are preserved. This does not claim a completed recorded run.
The portable catalog now joins successful **published** queue receipts to the
current per-source catalog snapshot, immutable definition, Session publication
@@ -112,6 +115,10 @@ return the original job. Verified corruption permits a new request.
The frontend's existing portable decoder now accepts only bound, internally
consistent cached results. No per-LAB page, renderer or selector implementation
was introduced. Full selector/progress/product acceptance remains stage 2.
Browser QA confirms that source/setup-only job selection can still display a
legacy completion status against a current definition. Exact-version status
binding and separating archives from executable choices remain required;
the activated backend cache does not prove those UI changes complete.
See [verification and activation status](../../experiments/perception/OBSERVATORY_PUBLISHED_CACHE_2026-09-03.md).
## Boundaries
@@ -2,7 +2,8 @@
## Status and scope
Implemented and contract-tested; **not activated on canonical8000 yet**.
Implemented, contract-tested and **activated on canonical8000** on 2026-09-03.
Full product acceptance remains open; see the activation evidence below.
Implementation commit: `1e4ddc2` on `codex/m5-1-observatory` (no push).
This is stage1 source/provenance/cache work, not model or recorded-run evidence.
No Worker containers, model profiles, raw recordings, result history, capture,
@@ -75,7 +76,7 @@ Reproduce backend tests with `.venv/bin/pytest -o addopts='' -q` and these files
Frontend, from `apps/control-station`:
`NODE_OPTIONS=--max-old-space-size=192 node --test --test-concurrency=1 test/observatoryLaboratorySetups.test.mjs test/applicationArchitecture.test.mjs`.
## Activation still required
## Initial activation hold — historical, resolved below
Mac pressure rose from1 to2 during the turn, swap~7.67.8GiB; no temporary heavy job
remains. Existing operator Docker services were inspected, not stopped. The
@@ -90,3 +91,63 @@ and browser decoder behavior without creating inference jobs, then update this
status. Do not start a second backend or replay process. Stage2 then implements
the selector/progress/full recorded calculation and sequential M4.9T5/LAB V1
acceptance. Neither this report nor a cached viewer FPS proves onboard realtime.
## Resource recovery and activation — 2026-09-03
The earlier memory gate was real, but browser RSS was not a sufficient cause
diagnosis. After the owner stopped unrelated containers, the23-day-old Docker
VM retained a14GB footprint (about13GB accounted swapped/compressed), while
its guest had about12.5GiB available. This was not14GB resident physical RAM
or a model workload. Chrome, Arc and Little Snitch were not touched.
With explicit owner approval, saved all48 container identities/states/policies,
temporarily changed the12 stopped Plane `always` policies to `no`, and restarted
Docker Desktop gracefully. VM PID9403→66297; footprint14GB→1565MB, pressure2→1,
used swap7722.12→2746.19MiB. Only the same3 Mission Core telemetry containers
resumed, all healthy. Restored Plane policies to `always` without starting Plane.
All45 non-Core containers retain their state/StartedAt. Images and mount
identities are unchanged (Docker reordered some Mounts arrays). No containers,
volumes or data deleted; Docker MemoryMiB14336/SwapMiB4096 unchanged. Plane can
start on a future daemon restart: this was only a temporary restart barrier.
Sequential validation under `mission-core-product-ui`:
- Architecture4/4; full typecheck; frontend714/714 unit tests; production build
PASS. Typecheck5.47s/peak RSS902152192bytes, tests18.76s/242204672bytes,
build15.38s/2255650816bytes. Node heap caps1024/512/2048MiB respectively.
Pressure remained1; swap did not grow. Existing large-chunk warnings remain.
- Built in a fresh ignored staging directory; preserved the old dist for
rollback, then replaced the matching frontend/backend via the existing
LaunchAgent. Backend PID67747, UI `/assets/index-DSuuMkq4.js`, index SHA256
`eb8c47d373a2ce1077557d348e097aa354556ddf2a529e3d6c07b4d5fe91922f`.
Health200/reconciler ready. No alternate backend8765 or preview4173.
- Portable catalogs HTTP200:004TREE/00 compatible with both current profiles;
01 remains blocked by missing capture attestation, not a cache failure.
No exact published cache hits in current real data. Both new indexes exist.
- In-app browser: actual new build, source/evidence catalog, portable selection,
Refresh, normal/expanded window, dropdown Escape PASS; console warnings/errors0.
Initial navigation hit the short backend startup gap; a fresh app tab worked.
No heavy replay/model run. Automatic retention of the QA tab was declined by
browser action review; no alternate retention mechanism was used.
- All6 queue-table canonical row hashes unchanged before/after activation and
UI checks:11 jobs,50000 legacy claims,1 reconciliation,0 live leases/v3 grants/
preemptions. The sole succeeded job is still `not-required`, not a new cache
hit. Positive publish→cache→view remains synthetic contract-test evidence.
Private snapshots/logs and previous frontend: `.runtime/docker-vm-recovery-z1636o/`.
No raw recording content was collected into this diagnostic evidence.
## Remaining product acceptance
The cache increment is active, but the complete workflow is not accepted.
Browser QA reproduced the next-stage UI mismatch: archived entries and portable
definitions are mixed in the selector; current M4.9T5 can show an old completion
label next to Calculate. `useObservatoryRecordedJobs` selects by source/setup,
then `jobs[0]`, without an exact definition filter; `ObservatoryWorkspace`
renders `succeeded/not-required` as completed. That is not a current cache hit.
Next: exact-version queue/status binding, selector filtering, publication
progress, complete recorded calculation and sequential M4.9T5/LAB V1 acceptance.
Stage1 user-facing reconciliation and the full stage2 workflow remain open.
Do not repeat the memory recovery or claim/v3 repair, or treat the old realtime
FAIL as a new gate. No per-LAB UI branch or automatic inference on Refresh.