АРЧ АП11 - Архитектура после регресса: + Архитектура: восстановить bridge от provenance выбранной позиции к VAT-периоду и закрыть phase10 replay
This commit is contained in:
+18
@@ -193,6 +193,24 @@ Still open after the accepted phase9 replay:
|
||||
- business answers are now semantically correct on this path, but some inventory list formatting still feels heavier and more mechanical than the target human style;
|
||||
- the next architecture slice should keep expanding saved-session proof across additional real user chains, while separately tightening answer presentation so exact routes do not feel template-driven even when the truth path is already correct.
|
||||
|
||||
Latest phase10 bridge-and-aggregate evidence after the manual replay recovery:
|
||||
|
||||
- live replay `address_truth_harness_phase10_manual_bridge_and_aggregate_mix_live_20260418_rerun8` is accepted end-to-end with `9/9` steps green;
|
||||
- the previously broken bridge `selected item purchase provenance -> VAT on purchase date` is now explicit instead of implicit:
|
||||
- the continuity layer derives the purchase month from the grounded provenance evidence;
|
||||
- the same session keeps `selected object` continuity instead of collapsing into generic root-only VAT arbitration;
|
||||
- the runtime now routes this follow-up as `vat_liability_confirmed_for_tax_period`, not as `forecast`, `unknown`, or generic clarification;
|
||||
- the same replay also proves that the neighboring aggregate fixes are live on the real assistant path:
|
||||
- top-customer-all-time now returns a direct business answer first;
|
||||
- top-year aggregate now returns a direct business answer first;
|
||||
- very-old-stock now prefers `inventory_aging_by_purchase_date` over a generic inventory snapshot;
|
||||
- this matters architecturally because the seam that used to exist only as ambient monolith behavior is now protected as an explicit carryover contract plus replay-backed acceptance path.
|
||||
|
||||
Still open after the accepted phase10 replay:
|
||||
|
||||
- the user-facing VAT explanation block is now correct and grounded, but some long exact answers still feel heavier than the target human product tone;
|
||||
- the next architecture slice should keep moving from repaired bridge authority into answer-shaping cleanup and broader saved-session replay coverage, not back into isolated wording tweaks.
|
||||
|
||||
## Ready Signal
|
||||
|
||||
The project can leave the current breakpoint when:
|
||||
|
||||
+240
@@ -0,0 +1,240 @@
|
||||
# 12 - Manual Run System Analysis `assistant-stage1-3NilqwT1G2` (2026-04-18)
|
||||
|
||||
## Purpose
|
||||
|
||||
This note analyzes the manual saved-session run `assistant-stage1-3NilqwT1G2`.
|
||||
|
||||
The goal is not to review isolated prompts.
|
||||
|
||||
The goal is to determine:
|
||||
|
||||
- which failures should improve as orchestration / continuity authority is repaired;
|
||||
- which failures are separate route or capability gaps;
|
||||
- which failures come from answer shaping and presentation rather than from routing.
|
||||
|
||||
## Evidence
|
||||
|
||||
Primary artifacts:
|
||||
|
||||
- `llm_normalizer/reports/assistant-stage1-3NilqwT1G2.md`
|
||||
- `llm_normalizer/reports/assistant-stage1-3NilqwT1G2.json`
|
||||
- `llm_normalizer/data/assistant_sessions/assistant-stage1-3NilqwT1G2-SAVED-001.json`
|
||||
|
||||
Relevant architecture context:
|
||||
|
||||
- `10 - regression_breakpoint_analysis_2026-04-17.md`
|
||||
- `11 - continuity_stabilization_plan_2026-04-17.md`
|
||||
- `docs/orchestration/active_domain_contract.json`
|
||||
|
||||
## High-Level Verdict
|
||||
|
||||
This run is bad for more than one reason.
|
||||
|
||||
It is not a single orchestration bug.
|
||||
|
||||
It is also not a pure capability-gap run.
|
||||
|
||||
The failure pattern is mixed:
|
||||
|
||||
1. Some turns fail because the right contour already exists, but orchestration or intent selection does not reach it.
|
||||
2. Some turns fail because extracted neighboring entities no longer share an explicit bridge contract.
|
||||
3. Some turns fail because the needed business capability family is only partially modeled.
|
||||
4. Some turns are technically routed but still feel bad because answer shape and presentation are too template-like.
|
||||
|
||||
## What Still Works
|
||||
|
||||
The following contours are still alive in this run:
|
||||
|
||||
- inventory root snapshot and historical date carryover;
|
||||
- selected-item supplier provenance;
|
||||
- selected-item sale trace;
|
||||
- confirmed receivables snapshot on a date;
|
||||
- VAT confirmed for the carried tax period after a debt-date question;
|
||||
- documents by counterparty;
|
||||
- inventory root return on later turns.
|
||||
|
||||
This matters because it means the runtime foundation is not dead.
|
||||
|
||||
The project did not lose all exact routes.
|
||||
|
||||
The main problem is that the system no longer enters the correct contour reliably under mixed real-session wording.
|
||||
|
||||
## Failure Families
|
||||
|
||||
### A. Contour exists, but the runtime does not enter it
|
||||
|
||||
These are the clearest orchestration / intent-resolution failures.
|
||||
|
||||
#### `Q12` - `прикинь какой ндс нам надо заплатить на февраль 2017`
|
||||
|
||||
- Actual behavior: fell into `living_chat`, `living_chat_response_source = llm_chat`.
|
||||
- Why this is bad: the system already has `vat_liability_confirmed_for_tax_period` as a real contour.
|
||||
- Why this is architecture, not wording only: the right family exists, but the lane authority does not protect it under colloquial phrasing.
|
||||
|
||||
#### `Q16` - `мы должны комуто денег на сегодня?`
|
||||
|
||||
- Actual behavior: `detected_intent = unknown`, partial refusal.
|
||||
- Why this is bad: `payables_confirmed_as_of_date` exists in recipe, capability, filter, compose, and navigation layers.
|
||||
- Why this is architecture: the contour is present, but the resolver / route arbitration fails to bind the phrase to the payables family.
|
||||
|
||||
#### `Q29` - `Есть ли остатки товара, которые закупались очень давно`
|
||||
|
||||
- Actual behavior: routed as `inventory_on_hand_as_of_date`, returned a generic current stock snapshot.
|
||||
- Why this is bad: the active domain contract explicitly contains the old-stock / aging path, and the runtime has `inventory_aging_by_purchase_date`.
|
||||
- Why this is architecture: generic inventory root wins over the more specific aging branch because precedence is wrong.
|
||||
|
||||
### B. Cross-domain bridge disappeared after extraction
|
||||
|
||||
These are the most important "neighbor entities used to live together" failures.
|
||||
|
||||
#### `Q10` - `ндс можешь прикинуть на дату покупки рабочей станции?`
|
||||
|
||||
- Actual behavior: `detected_intent = unknown`, no route, organization only, selected-item / purchase-date context dropped.
|
||||
- What this should have used: selected item -> purchase provenance bundle -> purchase date / period -> VAT contour.
|
||||
- Why this is architecture: the system can answer selected-item provenance and can answer VAT periods, but the bridge between those adjacent contours is not explicit.
|
||||
|
||||
#### `Q17` - `а нам?`
|
||||
|
||||
- Actual behavior: rewritten into a generic "our company / our base" question and lost the debt polarity follow-up.
|
||||
- What this should have used: `payables -> receivables` role-swap follow-up over the same date scope.
|
||||
- Why this is architecture: short polarity pivots need an explicit follow-up contract, not free-form reinterpretation.
|
||||
|
||||
#### `Q23` - `а по свк`
|
||||
|
||||
- Actual behavior: stayed inside `list_documents_by_counterparty`, but the short alias retarget is semantically weak and the answer quality is poor.
|
||||
- Why this is architecture: short retargets over an existing counterparty lane need a dedicated alias / retarget seam.
|
||||
|
||||
### C. Business family partially exists, but not at the right grain
|
||||
|
||||
These are not fixed by continuity alone.
|
||||
|
||||
#### `Q13` - `кто у нас самый доходный клиент за все время`
|
||||
|
||||
- Actual behavior: `detected_intent = unknown`, partial refusal.
|
||||
- What exists nearby: `customer_revenue_and_payments`.
|
||||
- What is missing in practice: stable ranking / all-time aggregate handling for company-level customer revenue questions under colloquial wording.
|
||||
|
||||
#### `Q18` - `какой у нас самый доходный год`
|
||||
|
||||
#### `Q19` - `а за 2017 мы скок заработали?`
|
||||
|
||||
#### `Q20` - `сколько вообще денег мы заработали за все время?`
|
||||
|
||||
- Actual behavior: broad refusal or partial refusal.
|
||||
- Why this matters: these are not random questions; they are business analytics asks at company/year/all-time aggregate grain.
|
||||
- Why this is not only orchestration: current nearby revenue contour is not enough to honestly cover company-year and all-time aggregate semantics.
|
||||
|
||||
### D. Route exists nearby, but wording-to-intent binding regressed
|
||||
|
||||
#### `Q11` - `а какой ндс мы должны сгрузить на март 2020?`
|
||||
|
||||
- Actual behavior: `detected_intent = unknown`.
|
||||
- What exists nearby: VAT confirmed liability contour.
|
||||
- Why this matters: colloquial accounting wording like `сгрузить НДС` should not fall out of contour if the product claims live accountant dialogue.
|
||||
|
||||
#### `Q25` - `что нам отгружать чепурнов? какой товар или услугу?`
|
||||
|
||||
- Actual behavior: partial refusal.
|
||||
- What exists nearby: counterparty documents and related shipment evidence.
|
||||
- Why this matters: the system has adjacent evidence contours but lacks a stable "counterparty -> shipped items/services" interpretation.
|
||||
|
||||
### E. User-facing quality defects even when routing is not fully broken
|
||||
|
||||
#### `Q2` - `расскажи что можешь интересного`
|
||||
|
||||
- The answer is cleaner than before, but still catalog-like and mechanically enumerated.
|
||||
|
||||
#### `Q14` / `Q15`
|
||||
|
||||
- These turns route correctly, but the answer shape still exposes internal "block" thinking and feels less human than the target product tone.
|
||||
|
||||
#### `Q28`
|
||||
|
||||
- The route works, but the result relies on fallback reasoning (`confirmed_balance_unavailable_fallback_to_heuristic_candidates` / `open_items_account_query_override_to_movements`), so business confidence should stay explicit.
|
||||
|
||||
## System Classification
|
||||
|
||||
This run should be read as:
|
||||
|
||||
- `orchestration / continuity authority failures`: `Q10`, `Q12`, `Q16`, `Q17`, `Q23`, `Q29`
|
||||
- `resolver / wording-to-intent failures`: `Q11`, `Q13`, `Q25`
|
||||
- `aggregate business contour gaps`: `Q18`, `Q19`, `Q20`
|
||||
- `answer-shape / presentation defects`: `Q2`, `Q14`, `Q15`, `Q28`
|
||||
|
||||
The important conclusion:
|
||||
|
||||
- a large share of the bad run is still architecture and orchestration;
|
||||
- but not all of it will heal automatically just by continuing continuity fixes.
|
||||
|
||||
## What The Old Monolith Likely Had
|
||||
|
||||
Before extraction, several neighboring decisions lived in one ambient runtime region:
|
||||
|
||||
- intent guess;
|
||||
- follow-up carryover;
|
||||
- selected-object reuse;
|
||||
- organization/date reuse;
|
||||
- recipe selection;
|
||||
- answer packaging.
|
||||
|
||||
That old shape was fragile, but it allowed adjacent capabilities to "borrow" context from one another without explicit contracts.
|
||||
|
||||
After extraction, those same borrow paths became seams.
|
||||
|
||||
The system now needs those seams to be first-class contracts.
|
||||
|
||||
Without that, optimization does not produce a stronger stone.
|
||||
|
||||
It produces split pieces whose joints are filled by ad hoc glue.
|
||||
|
||||
## Architecture Requirement
|
||||
|
||||
The extraction path is still the right direction only if every broken adjacency becomes explicit.
|
||||
|
||||
The missing explicit seams now visible in this run are:
|
||||
|
||||
- `inventory root <-> inventory aging`
|
||||
- `selected item / provenance bundle <-> VAT period ask`
|
||||
- `payables <-> receivables polarity flip`
|
||||
- `counterparty documents <-> short alias retarget`
|
||||
- `counterparty evidence <-> shipped goods/services summary`
|
||||
- `organization scope <-> company analytics aggregates`
|
||||
|
||||
Each seam needs:
|
||||
|
||||
1. an explicit transition / carryover contract;
|
||||
2. one clear intent-family arbitration rule;
|
||||
3. answer-shape expectations for the user-facing reply;
|
||||
4. replay coverage inside a saved-session scenario, not only in isolated unit cases.
|
||||
|
||||
## Final Conclusion
|
||||
|
||||
Yes, the architecture can become better than the old monolith.
|
||||
|
||||
But only under one condition:
|
||||
|
||||
- every extracted neighbor relationship that used to work implicitly must now be reintroduced as an explicit contract and guarded by scenario-level acceptance.
|
||||
|
||||
If that is done, the stone becomes better:
|
||||
|
||||
- less magical;
|
||||
- more inspectable;
|
||||
- safer to expand across domains.
|
||||
|
||||
If that is not done, the stone does not become better.
|
||||
|
||||
It becomes fragmented and cosmetically re-glued.
|
||||
|
||||
## Next Recommended Slice
|
||||
|
||||
The next phase should not be a random patch sweep.
|
||||
|
||||
It should be one focused replay pack built from this exact saved session with priority order:
|
||||
|
||||
1. `inventory -> selected item -> VAT purchase-date bridge`
|
||||
2. `direct VAT colloquial liability wording`
|
||||
3. `payables / receivables polarity swap`
|
||||
4. `inventory aging vs generic inventory precedence`
|
||||
5. `company analytics aggregates`
|
||||
|
||||
That pack should become the next acceptance gate before wider domain expansion.
|
||||
Reference in New Issue
Block a user