Задокументировать закрытие planner autonomy и phase83 evidence
This commit is contained in:
+50
-15
@@ -129,8 +129,9 @@ The following consolidation step added catalog-level chain-template scoring:
|
||||
- `address_truth_harness_phase32_planner_selected_chain_end_to_end.json` now uses the same assertions across selected-counterparty entity grounding, incoming/outgoing/net value-flow, document evidence, and movement evidence follow-ups.
|
||||
- `agent_semantic_pack_builder` now preserves these expected catalog-alignment fields in the reusable source catalog and adds the `planner_catalog_alignment` tag, so future mixed AGENT packs can deliberately select planner-brain regression probes instead of relying on hand-picked replay filenames.
|
||||
- The new `turnaround_11_planner_brain_alignment_mix` builder recipe generates `address_truth_harness_phase83_planner_brain_alignment_mix.json`, a 20-step mixed canary that crosses selected-counterparty value-flow, open-organization totals/comparison/ranking, broad-evaluation continuity, metadata drilldown, and off-domain living-chat safety.
|
||||
- The phase83 live replay now confirms that selected chains match the reviewed catalog top match across the mixed planner-brain pack. The remaining failure is not catalog arbitration: 1C/MCP fetches abort before confirmed business evidence is available.
|
||||
- The phase83 live replay now confirms that selected chains match the reviewed catalog top match across the mixed planner-brain pack and that the business-answer path remains usable after cross-stage pivots.
|
||||
- Checked-source failure replies now sanitize raw MCP transport/internal continuation strings from the user-facing answer while keeping the raw diagnostics in technical debug payloads.
|
||||
- Confirmed metadata-surface follow-ups now promote the surface-grounded chain template (`document_evidence`, `movement_evidence`, or `catalog_drilldown`) to the top catalog match when the selected chain came from the same checked surface. This keeps the planner's executed route and catalog-alignment diagnostics consistent without allowing ambiguous or stale surfaces to override explicit current-turn data needs.
|
||||
|
||||
## Why This Matters
|
||||
|
||||
@@ -295,31 +296,65 @@ Latest validation after phase32 catalog-alignment spec hardening and AGENT sourc
|
||||
Latest validation after phase83 mixed planner-brain spec generation:
|
||||
|
||||
- `scripts.test_agent_semantic_pack_builder`: passed, `3 passed`
|
||||
- generated `address_truth_harness_phase83_planner_brain_alignment_mix.json`: `20` steps, `13` expected catalog top-match checks
|
||||
- generated `address_truth_harness_phase83_planner_brain_alignment_mix.json`: `20` steps, `15` expected catalog top-match checks after the phase19/21/22 alignment hardening
|
||||
- regenerated `agent_semantic_source_catalog.*`: `planner_catalog_alignment` is visible with `26` reusable entries, including phase32, phase66, and phase83 probes
|
||||
- graphify rebuild: `5952 nodes`, `12927 edges`, `138 communities`
|
||||
|
||||
Latest validation after phase83 live replay and checked-source error sanitation:
|
||||
Prior live-readiness diagnosis after phase83 live replay and checked-source error sanitation:
|
||||
|
||||
- live phase83 after backend restart: `phase83_planner_brain_alignment_live_20260501_rerun2` and `phase83_planner_brain_alignment_live_20260501_rerun3` both ended `partial`, with `8/20` pass, `2` warning, `10` fail
|
||||
- phase83 invariant result: `catalog_alignment_ok=true`, `direct_answer_ok=true`, `temporal_honesty_ok=true`, `truth_gate_ok=true`
|
||||
- phase83 remaining blocker: confirmed business facts fail because 1C/MCP fetches abort before evidence is returned; this is not a planner/catalog top-match regression
|
||||
- targeted sanitation tests: `assistantMcpDiscoveryAnswerAdapter.test.ts`, `assistantMcpDiscoveryResponseCandidate.test.ts`, and `assistantMcpDiscoveryResponsePolicy.test.ts` passed `61/61` with `1` skipped
|
||||
- `npm.cmd run build`: passed
|
||||
- short live sanity `phase83_first2_sanity_live_20260501_errorfilter`: user-facing assistant answer no longer exposes raw `MCP fetch failed`, `This operation was aborted`, or `Entity-resolution could not continue`; those remain only in technical debug
|
||||
- graphify rebuild: `5953 nodes`, `12930 edges`, `137 communities`
|
||||
- backend health is green on `http://127.0.0.1:8787/api/health`;
|
||||
- proxy health is green on `http://127.0.0.1:6003/health`, with `pending_commands=0`, `active_channels_count=1`, and `active_sessions_count=0`;
|
||||
- targeted checked-source sanitation tests still pass `61/61` with `1` skipped;
|
||||
- `npm.cmd run build` still passes;
|
||||
- full phase83 rerun `phase83_planner_brain_alignment_live_20260501_rerun4` again ended `partial`, with `8/20` pass, `2` warning, `10` fail, and `catalog_alignment_ok=true`;
|
||||
- direct proxy `get_metadata` with a 180-second client timeout also timed out, so the remaining live blocker is below the assistant planner/backend layer: the proxy accepts requests, but the 1C side does not return read-only evidence in time;
|
||||
- `scripts/check_mcp_live_readiness.py` now provides a repo-native preflight that separates backend/proxy health from confirmed live 1C evidence readiness before spending time on a full semantic replay.
|
||||
- graphify rebuild after the readiness preflight/docs sync: `5970 nodes`, `12958 edges`, `140 communities`.
|
||||
|
||||
Prior follow-up diagnosis of the proxy/1C seam:
|
||||
|
||||
- `1cv8c` is running locally with the `MCP Toolkit - Бухгалтерия предприятия, редакция 2.0` window title, so the failure is not simply "1C process absent";
|
||||
- observing a read-only `get_metadata` command on the `default` channel showed `pending_commands=1` for 15 seconds and no pickup by the 1C client;
|
||||
- the diagnostic command was explicitly drained from `/1c/poll` and completed through `/1c/result` with a synthetic cancel result so the proxy queue stayed clean;
|
||||
- the proxy health endpoint now exposes polling telemetry: `polling_channels_count`, `last_poll_at`, `last_delivered_command_at`, and optional `poll_activity_by_channel` when `HEALTH_INCLUDE_CHANNEL_DETAILS=true`;
|
||||
- after proxy restart with this telemetry enabled, `polling_channels_count=0` stayed stable for 20 seconds, proving no `/1c/poll` activity reached the proxy;
|
||||
- `scripts/check_mcp_live_readiness.py --confirm-live` now refuses to create a direct live probe when proxy health already proves no 1C polling activity, preventing abandoned pending commands during readiness checks.
|
||||
- `domain_truth_harness.py run-live --require-mcp-live-readiness` now applies the same readiness gate before the first assistant step, writes `mcp_live_readiness.json`, and exits early when live 1C evidence is unavailable;
|
||||
- smoke of that harness gate against phase83 stopped before step execution with `ready_for_live_replay=false`, so future blocked runs should no longer waste a full semantic replay just to rediscover the missing `/1c/poll`.
|
||||
- readiness can now wait for polling before probing: `--wait-for-polling-seconds` in `check_mcp_live_readiness.py` and `--mcp-wait-for-polling-seconds` in `domain_truth_harness.py run-live`; a 2-second smoke waited twice, observed no polling, and skipped the live probe without leaving proxy queue garbage.
|
||||
|
||||
Latest validation after guarded phase83 acceptance and surface-grounded catalog promotion:
|
||||
|
||||
- targeted planner/response-policy/pilot/continuity slice: `npm.cmd test -- assistantMcpDiscoveryPlanner.test.ts assistantMcpDiscoveryResponsePolicy.test.ts assistantMcpDiscoveryPilotExecutor.test.ts assistantContinuityPolicy.test.ts` passed `109/109`;
|
||||
- `npm.cmd run build`: passed;
|
||||
- graphify rebuild: `5973 nodes`, `12971 edges`, `138 communities`;
|
||||
- live-readiness preflight after backend restart: `mcp_live_readiness_phase83_rerun3_after_backend_restart.json` reported `ready`;
|
||||
- full guarded phase83 replay: `phase83_planner_brain_alignment_live_20260501_readygate_rerun3` accepted `20/20`, `0` warnings, `0` failures;
|
||||
- final invariant result: `catalog_alignment_ok=true`, `direct_answer_ok=true`, `temporal_honesty_ok=true`, `selected_object_continuity_ok=true`, `truth_gate_ok=true`, `human_answer_quality_ok=true`, and `meta_context_integrity_ok=true`;
|
||||
- the previously warning step `step_02_neutral_followup_catalog_drilldown` now reports `catalog_alignment_status=selected_matches_top`, `catalog_top_match=catalog_drilldown`, and `catalog_selected_matches_top=True`.
|
||||
- saved autorun canary: `AGENT | Planner Autonomy phase83: мозг маршрутов, pivots и legacy continuity` (`gen-ag05011759-6f85fc`), sourced from the accepted phase83 spec after the live replay was reviewed.
|
||||
|
||||
## Next Step
|
||||
|
||||
The next safe step is to restore/verify the 1C polling side and re-run `address_truth_harness_phase83_planner_brain_alignment_mix.json` as the main live acceptance candidate. The planner-brain invariant is now proven in replay (`catalog_alignment_ok=true`); the remaining acceptance gap is confirmed business evidence. Only after phase83 is semantically accepted should it be saved into autoruns as a legacy AGENT pack. In parallel, local-only consolidation can continue by hardening additional planner-autonomy specs with expected catalog-chain assertions and using `alignment_status`, alignment reason-code telemetry, truth-harness artifact surfacing, the soft divergence warning, `catalog_alignment_ok`, and the representative guard to find remaining manual branches where selected chains diverge from reviewed catalog-fabric intent.
|
||||
The declared Planner Autonomy Consolidation slice is now closed for the phase83 acceptance target.
|
||||
|
||||
Keep using the live preflight before future full replays:
|
||||
|
||||
`python scripts/check_mcp_live_readiness.py --confirm-live --wait-for-polling-seconds 60 --poll-interval-seconds 2 --output-json artifacts/runtime/mcp_live_readiness_phase83.json`
|
||||
|
||||
Run future full candidates with the built-in gate:
|
||||
|
||||
`python scripts/domain_truth_harness.py run-live --spec docs/orchestration/address_truth_harness_phase83_planner_brain_alignment_mix.json --output-dir artifacts/domain_runs/phase83_planner_brain_alignment_live_<stamp> --require-mcp-live-readiness --mcp-wait-for-polling-seconds 60 --mcp-poll-interval-seconds 2`
|
||||
|
||||
Only when readiness reports `ready_for_live_replay=true` should a full replay be treated as meaningful business-evidence proof. If it reports no `/1c/poll` activity, fix the 1C toolkit client/session/channel first; another full replay will only reproduce checked-source partial answers.
|
||||
|
||||
Recommended order:
|
||||
|
||||
1. reconnect or restart the 1C toolkit polling side, then rerun the inventory canary against live 1C/MCP;
|
||||
2. rerun a mixed cross-stage canary after the inventory canary is semantically clean;
|
||||
3. continue broadening catalog scoring into unfamiliar 1C asks where metadata surface and data-need graph can pick reviewed lanes;
|
||||
1. save the accepted phase83 pack into autoruns only if the product flow needs it as a legacy AGENT canary;
|
||||
2. continue broader open-world bounded autonomy with phase83 as a regression gate, not as an open blocker;
|
||||
3. broaden catalog scoring into unfamiliar 1C asks where metadata surface and data-need graph can pick reviewed lanes;
|
||||
4. grow primitive descriptors only where live replay shows a real evidence gap;
|
||||
5. keep phase19, phase21, phase22, value-flow, metadata ambiguity, and inventory-stock canaries as regression gates.
|
||||
5. keep phase19, phase21, phase22, value-flow, metadata ambiguity, inventory-stock, and phase83 as regression gates.
|
||||
|
||||
The key rule remains:
|
||||
|
||||
|
||||
@@ -92,9 +92,10 @@ It now documents a turnaround that is already operational in code, already mater
|
||||
- the phase32 selected-counterparty chain spec now asserts expected catalog-chain top matches across entity grounding, incoming/outgoing/net value-flow, document evidence, and movement evidence follow-ups;
|
||||
- AGENT semantic source catalog generation now preserves expected catalog-alignment fields and tags reusable steps as `planner_catalog_alignment`, so mixed pack construction can find planner-brain regression probes explicitly;
|
||||
- phase83 planner-brain mixed replay spec is now generated from the AGENT source catalog and interleaves selected-counterparty catalog alignment, open-organization money flow/ranking, broad-evaluation continuity, metadata drilldown, and off-domain living-chat safety;
|
||||
- phase83 live replay now proves the catalog-alignment invariant across the mixed pack (`catalog_alignment_ok=true`) even while business answers remain partial because 1C/MCP fetches abort before confirmed evidence is returned;
|
||||
- phase83 live replay now accepts `20/20` under guarded MCP live-readiness and proves catalog-alignment, direct-answer, temporal honesty, selected-object continuity, truth-gate, human-answer-quality, and meta-context invariants together;
|
||||
- checked-source failure answers now keep raw MCP transport/internal continuation errors out of the user-facing layer while preserving those details in technical debug artifacts;
|
||||
- explicit-counterparty incoming-vs-outgoing data-need graphs now select the reviewed `value_flow_comparison` chain instead of falling back to generic `value_flow`;
|
||||
- confirmed metadata-surface follow-ups now promote the surface-grounded chain template to the catalog top match, so neutral catalog drilldowns no longer look like lower-rank planner/catalog divergence;
|
||||
- live map sync: [20 - planner_autonomy_consolidation_2026-05-01.md](./20%20-%20planner_autonomy_consolidation_2026-05-01.md)
|
||||
|
||||
Current honest status:
|
||||
@@ -106,8 +107,8 @@ Current honest status:
|
||||
- open-world bounded-autonomy readiness: `~85%`
|
||||
- Post-F semantic integrity module progress: `~99%` operationally closed, with remaining risk now treated as next-slice discovery rather than an open blocker inside the closed slice
|
||||
- active inventory-stock breadth slice progress: `100%` for the declared scenario pack, not for arbitrary inventory questions
|
||||
- Planner Autonomy Consolidation progress: `~95%` for the declared module, with catalog-fabric, value-flow arbitration, lifecycle bounded inference, broad-evaluation bridge, inventory catalog templates, inventory runtime-boundary honesty, exact inventory recipe bridging, unambiguous metadata-surface lane inference, catalog chain-template scoring, structured chain-match contract exposure, runtime/debug propagation, subject-aware bidirectional comparison arbitration, structured catalog-alignment verdicts, representative alignment regression guard, catalog-alignment reason-code telemetry, explicit `alignment_status` propagation, truth-harness/acceptance-matrix surfacing, soft divergence warning, `catalog_alignment_ok` acceptance invariant, step-level expected catalog-alignment assertions, phase66 and phase32 spec alignment expectations, AGENT source-catalog surfacing, generated phase83 mixed planner-brain replay spec, live phase83 catalog-alignment proof, and checked-source user-facing error sanitation validated, but accepted business replay is still blocked by 1C/MCP fetch aborts and broader unfamiliar 1C asks still need replay-backed growth
|
||||
- graph snapshot after latest rebuild: `5953 nodes`, `12930 edges`, `137 communities`
|
||||
- Planner Autonomy Consolidation progress: `100%` for the declared module, with catalog-fabric, value-flow arbitration, lifecycle bounded inference, broad-evaluation bridge, inventory catalog templates, inventory runtime-boundary honesty, exact inventory recipe bridging, unambiguous metadata-surface lane inference, catalog chain-template scoring, structured chain-match contract exposure, runtime/debug propagation, subject-aware bidirectional comparison arbitration, structured catalog-alignment verdicts, representative alignment regression guard, catalog-alignment reason-code telemetry, explicit `alignment_status` propagation, truth-harness/acceptance-matrix surfacing, soft divergence warning, `catalog_alignment_ok` acceptance invariant, step-level expected catalog-alignment assertions, phase66 and phase32 spec alignment expectations, AGENT source-catalog surfacing, generated phase83 mixed planner-brain replay spec, checked-source user-facing error sanitation, surface-grounded catalog promotion, and guarded live phase83 acceptance validated. Broader unfamiliar 1C asks are now next-module breadth work rather than an open blocker inside this declared slice
|
||||
- graph snapshot after latest rebuild: `5973 nodes`, `12971 edges`, `138 communities`
|
||||
- current breakpoint:
|
||||
- the validated hot paths are no longer structurally broken;
|
||||
- flagship continuity collapse is no longer the primary risk;
|
||||
@@ -166,9 +167,12 @@ Latest live proof now includes:
|
||||
- catalog-alignment spec assertions accepted locally: Python truth-harness/acceptance tests passed `7/7`; graphify rebuilt to `5951 nodes`, `12926 edges`, `139 communities`
|
||||
- phase66 planner-alignment spec hardening accepted locally: Python truth-harness/acceptance tests passed `7/7`; `load_truth_harness_spec` confirmed expected top matches `[value_flow, value_flow, value_flow, value_flow_comparison, value_flow_comparison, value_flow_ranking, value_flow_ranking]`
|
||||
- phase32 selected-counterparty planner-alignment spec hardening and AGENT source-catalog surfacing accepted locally: Python replay-tooling tests passed `9/9`; `load_truth_harness_spec` confirmed expected top matches `[entity_resolution, value_flow, value_flow, value_flow_comparison, document_evidence, movement_evidence]`; regenerated source catalog exposes `planner_catalog_alignment` as a reusable tag
|
||||
- phase83 mixed planner-brain spec generation accepted locally: Python replay-tooling tests passed `10/10`; generated spec has `20` steps and `13` expected catalog top-match checks; regenerated source catalog exposes `planner_catalog_alignment` with `26` reusable entries; graphify rebuilt to `5952 nodes`, `12927 edges`, `138 communities`
|
||||
- phase83 live replay after backend restart: `phase83_planner_brain_alignment_live_20260501_rerun2` and `rerun3` reached `8/20` pass, `2` warning, `10` fail, final `partial`; key result is `catalog_alignment_ok=true` across all expected catalog checks, while business-answer failures are caused by `MCP fetch failed: This operation was aborted` before confirmed 1C evidence is available
|
||||
- phase83 mixed planner-brain spec generation accepted locally: Python replay-tooling tests passed `10/10`; generated spec has `20` steps and `15` expected catalog top-match checks after the phase19/21/22 alignment hardening; regenerated source catalog exposes `planner_catalog_alignment` with `26` reusable entries; graphify rebuilt to `5952 nodes`, `12927 edges`, `138 communities`
|
||||
- checked-source error sanitation accepted: targeted answer/candidate/policy tests passed `61/61` with `1` skipped; build passed; `phase83_first2_sanity_live_20260501_errorfilter` confirms the user-facing assistant section no longer exposes raw `MCP fetch failed` or `Entity-resolution could not continue` strings; graphify rebuilt to `5953 nodes`, `12930 edges`, `137 communities`
|
||||
- live-readiness guard accepted: `scripts/check_mcp_live_readiness.py --confirm-live --wait-for-polling-seconds 60` now confirms backend/proxy/direct read-only 1C evidence before expensive live replays;
|
||||
- guarded phase83 acceptance: `phase83_planner_brain_alignment_live_20260501_readygate_rerun3` accepted `20/20`, with `0` warnings, `0` failures, `catalog_alignment_ok=true`, `direct_answer_ok=true`, `temporal_honesty_ok=true`, `selected_object_continuity_ok=true`, `truth_gate_ok=true`, `human_answer_quality_ok=true`, and `meta_context_integrity_ok=true`;
|
||||
- surface-grounded catalog promotion accepted locally: targeted planner/response-policy/pilot/continuity slice passed `109/109`, build passed, and graphify rebuilt to `5973 nodes`, `12971 edges`, `138 communities`.
|
||||
- accepted phase83 is saved as an autorun canary: `AGENT | Planner Autonomy phase83: мозг маршрутов, pivots и legacy continuity` (`gen-ag05011759-6f85fc`).
|
||||
|
||||
Current architectural reading:
|
||||
|
||||
|
||||
Reference in New Issue
Block a user