From c2710b55a703758503bf0678e28e4f2a6fcbc9c4 Mon Sep 17 00:00:00 2001 From: DCCONSTRUCTIONS Date: Thu, 3 Sep 2026 09:44:02 +0300 Subject: [PATCH] docs(observatory): record cache activation and Docker memory recovery --- .../OBSERVATORY_REALTIME_PROFILES_EXECPLAN.md | 25 ++++++- docs/adr/0050-recorded-observatory-first.md | 11 +++- .../OBSERVATORY_PUBLISHED_CACHE_2026-09-03.md | 65 ++++++++++++++++++- 3 files changed, 95 insertions(+), 6 deletions(-) diff --git a/docs/OBSERVATORY_REALTIME_PROFILES_EXECPLAN.md b/docs/OBSERVATORY_REALTIME_PROFILES_EXECPLAN.md index ff9424a..5cfc252 100644 --- a/docs/OBSERVATORY_REALTIME_PROFILES_EXECPLAN.md +++ b/docs/OBSERVATORY_REALTIME_PROFILES_EXECPLAN.md @@ -70,10 +70,31 @@ QA **пока не выполнены**: Mac memory-pressure=2 при swap~7.6 backend/frontend release, health200; новый cache ещё не активирован. Worker, модели и живые очереди не изменялись. Полный LAB-run не запускался. [Проверки и следующий шаг](../experiments/perception/OBSERVATORY_PUBLISHED_CACHE_2026-09-03.md). -Этап1 остаётся открытым до сборки/активации согласованного release и проверки -каталога; затем этап2 — селектор, прогресс и полный цикл на записях. Не повторять +Первоначальный ресурсный blocker выше снят следующим инкрементом. Не повторять уже закрытый ремонт claim/v3 и не возвращаться к сетевым latency canary. +**Активация cache-инкремента, 2026-09-03.** По разрешению владельца перезапущена +старая Docker VM: footprint14GB→1565MB, pressure2→1, swap7722→2746MiB. Браузеры и +Little Snitch не трогали. Три прежних Mission Core контейнера healthy;12 Plane +restart overrides возвращены, Plane выключен. Состояния45 non-Core контейнеров, +image/mount identities сохранены; лимиты Docker и данные не менялись. +Полный typecheck,714 frontend tests и production build PASS, последовательно. +Согласованная версия на8000/PID67747, health200; UI `/assets/index-DSuuMkq4.js`. +Каталоги004TREE/00 совместимы с двумя profiles;01 заблокирован отсутствующим +capture attestation. Точных опубликованных cache hits пока нет; legacy +succeeded/not-required не принят за новый результат. Все6 queue-table row hashes +сохранены. Worker/model jobs не создавались. Browser: каталог, выбор, Refresh, +normal/expanded и Escape проверены, console warnings/errors0. + +Этап1 пользовательски ещё не закрыт: текущий селектор смешивает архивные entries +и portable definitions, а source/setup-only выбор последней job показывает +старое «Расчёт завершён» рядом с новым «Рассчитать». Это не текущий cache hit. +Следующий шаг на стыке этапов1–2: exact-definition queue/status binding, +селектор только нерассчитанных profiles, прогресс и полный цикл на записях. +Положительный publish→cache→view пока доказан synthetic tests, не новым LAB-run. +Подробности/rollback в том же отчёте выше; повторять сборку без изменения кода +или снова освобождать память не требуется. + - Сверить каталог, очередь, публикацию и просмотр для текущих M4.9T5 и LAB V1. - Связать готовность с точной исходной записью и immutable RunDefinition, включающей image/model/config/adapter/result-contract identities. diff --git a/docs/adr/0050-recorded-observatory-first.md b/docs/adr/0050-recorded-observatory-first.md index 695f436..9dd8c0a 100644 --- a/docs/adr/0050-recorded-observatory-first.md +++ b/docs/adr/0050-recorded-observatory-first.md @@ -83,8 +83,11 @@ See the [repair evidence and rollback boundary](../../experiments/perception/OBS ## Third increment: exact published-result cache (2026-09-03) -Implemented in source; activation of the matching backend/frontend release is -pending the local resource gate. This does not claim a completed recorded run. +Implemented and activated as a matching backend/frontend release on canonical8000 +on 2026-09-03 after an owner-approved Docker VM restart recovered memory headroom. +Full typecheck,714 frontend tests and production build passed sequentially; +API catalogs and in-app UI were checked without creating inference jobs. +All queue row hashes are preserved. This does not claim a completed recorded run. The portable catalog now joins successful **published** queue receipts to the current per-source catalog snapshot, immutable definition, Session publication @@ -112,6 +115,10 @@ return the original job. Verified corruption permits a new request. The frontend's existing portable decoder now accepts only bound, internally consistent cached results. No per-LAB page, renderer or selector implementation was introduced. Full selector/progress/product acceptance remains stage 2. +Browser QA confirms that source/setup-only job selection can still display a +legacy completion status against a current definition. Exact-version status +binding and separating archives from executable choices remain required; +the activated backend cache does not prove those UI changes complete. See [verification and activation status](../../experiments/perception/OBSERVATORY_PUBLISHED_CACHE_2026-09-03.md). ## Boundaries diff --git a/experiments/perception/OBSERVATORY_PUBLISHED_CACHE_2026-09-03.md b/experiments/perception/OBSERVATORY_PUBLISHED_CACHE_2026-09-03.md index ac6026f..b9a32bc 100644 --- a/experiments/perception/OBSERVATORY_PUBLISHED_CACHE_2026-09-03.md +++ b/experiments/perception/OBSERVATORY_PUBLISHED_CACHE_2026-09-03.md @@ -2,7 +2,8 @@ ## Status and scope -Implemented and contract-tested; **not activated on canonical8000 yet**. +Implemented, contract-tested and **activated on canonical8000** on 2026-09-03. +Full product acceptance remains open; see the activation evidence below. Implementation commit: `1e4ddc2` on `codex/m5-1-observatory` (no push). This is stage1 source/provenance/cache work, not model or recorded-run evidence. No Worker containers, model profiles, raw recordings, result history, capture, @@ -75,7 +76,7 @@ Reproduce backend tests with `.venv/bin/pytest -o addopts='' -q` and these files Frontend, from `apps/control-station`: `NODE_OPTIONS=--max-old-space-size=192 node --test --test-concurrency=1 test/observatoryLaboratorySetups.test.mjs test/applicationArchitecture.test.mjs`. -## Activation still required +## Initial activation hold — historical, resolved below Mac pressure rose from1 to2 during the turn, swap~7.6–7.8GiB; no temporary heavy job remains. Existing operator Docker services were inspected, not stopped. The @@ -90,3 +91,63 @@ and browser decoder behavior without creating inference jobs, then update this status. Do not start a second backend or replay process. Stage2 then implements the selector/progress/full recorded calculation and sequential M4.9T5/LAB V1 acceptance. Neither this report nor a cached viewer FPS proves onboard realtime. + +## Resource recovery and activation — 2026-09-03 + +The earlier memory gate was real, but browser RSS was not a sufficient cause +diagnosis. After the owner stopped unrelated containers, the23-day-old Docker +VM retained a14GB footprint (about13GB accounted swapped/compressed), while +its guest had about12.5GiB available. This was not14GB resident physical RAM +or a model workload. Chrome, Arc and Little Snitch were not touched. + +With explicit owner approval, saved all48 container identities/states/policies, +temporarily changed the12 stopped Plane `always` policies to `no`, and restarted +Docker Desktop gracefully. VM PID9403→66297; footprint14GB→1565MB, pressure2→1, +used swap7722.12→2746.19MiB. Only the same3 Mission Core telemetry containers +resumed, all healthy. Restored Plane policies to `always` without starting Plane. +All45 non-Core containers retain their state/StartedAt. Images and mount +identities are unchanged (Docker reordered some Mounts arrays). No containers, +volumes or data deleted; Docker MemoryMiB14336/SwapMiB4096 unchanged. Plane can +start on a future daemon restart: this was only a temporary restart barrier. + +Sequential validation under `mission-core-product-ui`: + +- Architecture4/4; full typecheck; frontend714/714 unit tests; production build + PASS. Typecheck5.47s/peak RSS902152192bytes, tests18.76s/242204672bytes, + build15.38s/2255650816bytes. Node heap caps1024/512/2048MiB respectively. + Pressure remained1; swap did not grow. Existing large-chunk warnings remain. +- Built in a fresh ignored staging directory; preserved the old dist for + rollback, then replaced the matching frontend/backend via the existing + LaunchAgent. Backend PID67747, UI `/assets/index-DSuuMkq4.js`, index SHA256 + `eb8c47d373a2ce1077557d348e097aa354556ddf2a529e3d6c07b4d5fe91922f`. + Health200/reconciler ready. No alternate backend8765 or preview4173. +- Portable catalogs HTTP200:004TREE/00 compatible with both current profiles; + 01 remains blocked by missing capture attestation, not a cache failure. + No exact published cache hits in current real data. Both new indexes exist. +- In-app browser: actual new build, source/evidence catalog, portable selection, + Refresh, normal/expanded window, dropdown Escape PASS; console warnings/errors0. + Initial navigation hit the short backend startup gap; a fresh app tab worked. + No heavy replay/model run. Automatic retention of the QA tab was declined by + browser action review; no alternate retention mechanism was used. +- All6 queue-table canonical row hashes unchanged before/after activation and + UI checks:11 jobs,50000 legacy claims,1 reconciliation,0 live leases/v3 grants/ + preemptions. The sole succeeded job is still `not-required`, not a new cache + hit. Positive publish→cache→view remains synthetic contract-test evidence. + +Private snapshots/logs and previous frontend: `.runtime/docker-vm-recovery-z1636o/`. +No raw recording content was collected into this diagnostic evidence. + +## Remaining product acceptance + +The cache increment is active, but the complete workflow is not accepted. +Browser QA reproduced the next-stage UI mismatch: archived entries and portable +definitions are mixed in the selector; current M4.9T5 can show an old completion +label next to Calculate. `useObservatoryRecordedJobs` selects by source/setup, +then `jobs[0]`, without an exact definition filter; `ObservatoryWorkspace` +renders `succeeded/not-required` as completed. That is not a current cache hit. + +Next: exact-version queue/status binding, selector filtering, publication +progress, complete recorded calculation and sequential M4.9T5/LAB V1 acceptance. +Stage1 user-facing reconciliation and the full stage2 workflow remain open. +Do not repeat the memory recovery or claim/v3 repair, or treat the old realtime +FAIL as a new gate. No per-LAB UI branch or automatic inference on Refresh.