ARCH: зафиксировать план восстановления semantic dialog authority
This commit is contained in:
parent
1b9dbebe47
commit
4e7974acc7
|
|
@ -0,0 +1,489 @@
|
|||
# 14 - Semantic Dialog Authority Recovery Plan (2026-04-19)
|
||||
|
||||
## Purpose
|
||||
|
||||
This note defines the next large architecture move after the current turnaround `11` stabilization wave.
|
||||
|
||||
It is not a generic "improve the assistant" wishlist.
|
||||
|
||||
It is the focused recovery plan for one product-critical defect class:
|
||||
|
||||
- a new human user can still feel that the assistant is glitchy;
|
||||
- the system can still miss the asked meaning on already-supported contours;
|
||||
- the system can still replay stale context instead of reacting to the current question;
|
||||
- the product can still look like a brittle trained script instead of a live stateful assistant.
|
||||
|
||||
This note treats that defect class as an architecture problem, not as prompt polish.
|
||||
|
||||
## Why This Is Now The Main Problem
|
||||
|
||||
Turnaround `11` already repaired the original acute continuity collapse on the validated replay families.
|
||||
|
||||
That is no longer the main blocker.
|
||||
|
||||
The current blocker is this:
|
||||
|
||||
- route truth on repaired packs is materially stronger;
|
||||
- continuity on repaired packs is materially stronger;
|
||||
- but the assistant can still feel semantically wrong in live wording;
|
||||
- and a new user will judge the product through that feeling first.
|
||||
|
||||
The product risk is no longer:
|
||||
|
||||
- "the route obviously collapsed"
|
||||
|
||||
but:
|
||||
|
||||
- "the route may exist, but the assistant still feels like it did not understand me."
|
||||
|
||||
## Trigger Evidence
|
||||
|
||||
The saved session:
|
||||
|
||||
- [assistant-stage1-OFwMSAX_-e.md](/x:/1C/NDC_1C/llm_normalizer/reports/assistant-stage1-OFwMSAX_-e.md:1)
|
||||
- [assistant-stage1-OFwMSAX_-e-SAVED-001.json](/x:/1C/NDC_1C/llm_normalizer/data/assistant_sessions/assistant-stage1-OFwMSAX_-e-SAVED-001.json:1)
|
||||
|
||||
shows the exact defect class this plan targets.
|
||||
|
||||
The most important failures in that run are not "missing route only".
|
||||
|
||||
They are:
|
||||
|
||||
1. supported root meaning is still fragile under light typo/noise:
|
||||
- `кто намс должен денег на сегодня` falls into limited/unsupported behavior;
|
||||
- the neighboring clean wording `кто нам должен денег на сегодня` works;
|
||||
- therefore the business contour exists, but the current-turn meaning is too brittle.
|
||||
|
||||
2. stale carryover can still beat the new explicit question:
|
||||
- after `покажи документы по чепурнову`
|
||||
- the turn `какой оборот был свк`
|
||||
- incorrectly replays the previous `list_documents_by_counterparty` result for `Чепурнов П.Д.`
|
||||
- instead of switching to the new object and new value/turnover intent family.
|
||||
|
||||
3. continuity and organization authority still feel inconsistent across contours:
|
||||
- some business turns answer directly;
|
||||
- neighboring turns fall back to clarification;
|
||||
- this may be technically honest, but still feels unstable to a new user.
|
||||
|
||||
The key conclusion is:
|
||||
|
||||
- the system still lacks one authoritative runtime object for "what the user means on this turn";
|
||||
- and carryover still behaves too much like a co-author of meaning instead of a helper for missing context.
|
||||
|
||||
## Architectural Reading
|
||||
|
||||
The current stack already has several strong owner seams:
|
||||
|
||||
- [assistantContinuityPolicy.ts](/x:/1C/NDC_1C/llm_normalizer/backend/src/services/assistantContinuityPolicy.ts:927)
|
||||
- [assistantRoutePolicy.ts](/x:/1C/NDC_1C/llm_normalizer/backend/src/services/assistantRoutePolicy.ts:382)
|
||||
- [assistantTransitionPolicy.ts](/x:/1C/NDC_1C/llm_normalizer/backend/src/services/assistantTransitionPolicy.ts:365)
|
||||
- [addressIntentResolver.ts](/x:/1C/NDC_1C/llm_normalizer/backend/src/services/addressIntentResolver.ts:1903)
|
||||
- [addressCounterpartyIntentSignals.ts](/x:/1C/NDC_1C/llm_normalizer/backend/src/services/addressCounterpartyIntentSignals.ts:145)
|
||||
- [composeStage.ts](/x:/1C/NDC_1C/llm_normalizer/backend/src/services/address_runtime/composeStage.ts:1)
|
||||
- [answerComposer.ts](/x:/1C/NDC_1C/llm_normalizer/backend/src/services/answerComposer.ts:1)
|
||||
|
||||
The current architectural weakness is not absence of seams.
|
||||
|
||||
It is that meaning is still reconstructed in parallel by several seams:
|
||||
|
||||
- intent resolver;
|
||||
- route policy;
|
||||
- transition/carryover policy;
|
||||
- continuity snapshot readers;
|
||||
- answer-shaping behavior.
|
||||
|
||||
That creates exactly the observed defects:
|
||||
|
||||
- supported meaning can fail to survive noisy wording;
|
||||
- new explicit meaning can fail to override stale carryover;
|
||||
- answer packaging can reflect the old route family instead of the asked question.
|
||||
|
||||
## Target Runtime Rule
|
||||
|
||||
Before any of the following decisions happen:
|
||||
|
||||
- route arbitration;
|
||||
- clarification arbitration;
|
||||
- carryover depth selection;
|
||||
- selected-object reuse;
|
||||
- answer top-block shaping;
|
||||
|
||||
the runtime must first resolve one explicit current-turn meaning authority.
|
||||
|
||||
That object should then constrain every downstream owner.
|
||||
|
||||
In practical terms:
|
||||
|
||||
- continuity may help fill missing context;
|
||||
- continuity may not redefine the current explicit question;
|
||||
- answer wording may clarify the result;
|
||||
- answer wording may not pretend the current question meant the previous route.
|
||||
|
||||
## Big Block 1. Current-Turn Meaning Authority
|
||||
|
||||
### Goal
|
||||
|
||||
Create one first-class runtime object that answers:
|
||||
|
||||
- what the user is asking on this turn;
|
||||
- how strong that meaning is;
|
||||
- what parts are explicit versus inherited;
|
||||
- whether stale replay is forbidden.
|
||||
|
||||
### Target Object
|
||||
|
||||
`assistant_turn_meaning_v1`
|
||||
|
||||
### Minimum Fields
|
||||
|
||||
- `asked_domain_family`
|
||||
- `asked_action_family`
|
||||
- `explicit_intent_candidate`
|
||||
- `explicit_entity_candidates`
|
||||
- `explicit_organization_scope`
|
||||
- `explicit_date_scope`
|
||||
- `meaning_confidence`
|
||||
- `meaning_source_mix`
|
||||
- `intent_override_strength`
|
||||
- `carryover_budget`
|
||||
- `clarification_need`
|
||||
- `unsupported_but_understood_family`
|
||||
- `stale_replay_forbidden`
|
||||
|
||||
### Architectural Role
|
||||
|
||||
This object should become the shared input for:
|
||||
|
||||
- route policy;
|
||||
- transition policy;
|
||||
- clarification policy;
|
||||
- answer top-block shaping.
|
||||
|
||||
### Stack Mapping
|
||||
|
||||
Primary new owner:
|
||||
|
||||
- `llm_normalizer/backend/src/services/assistantTurnMeaningPolicy.ts`
|
||||
|
||||
Primary consumers:
|
||||
|
||||
- [assistantRoutePolicy.ts](/x:/1C/NDC_1C/llm_normalizer/backend/src/services/assistantRoutePolicy.ts:382)
|
||||
- [assistantTransitionPolicy.ts](/x:/1C/NDC_1C/llm_normalizer/backend/src/services/assistantTransitionPolicy.ts:365)
|
||||
- [assistantAddressOrchestrationRuntimeAdapter.ts](/x:/1C/NDC_1C/llm_normalizer/backend/src/services/assistantAddressOrchestrationRuntimeAdapter.ts:1)
|
||||
- [answerComposer.ts](/x:/1C/NDC_1C/llm_normalizer/backend/src/services/answerComposer.ts:1)
|
||||
|
||||
### What Must Move Out Of Ambient Glue
|
||||
|
||||
The following logic should stop being reconstructed ad hoc in multiple places:
|
||||
|
||||
- repaired raw wording vs canonical wording precedence;
|
||||
- current-turn explicit intent candidate;
|
||||
- current-turn explicit entity switch;
|
||||
- current-turn explicit action switch;
|
||||
- current-turn semantic confidence;
|
||||
- "this looks like a new supported business ask, not a follow-up replay."
|
||||
|
||||
### Done Criteria
|
||||
|
||||
This block is complete only when:
|
||||
|
||||
- route and transition owners consume one shared turn-meaning object;
|
||||
- a new explicit entity/action question can be detected before carryover arbitration begins;
|
||||
- unsupported-but-understood meaning is explicitly representable without replaying the previous exact route.
|
||||
|
||||
## Big Block 2. Continuity Authority Must Become Subordinate To Current Meaning
|
||||
|
||||
### Goal
|
||||
|
||||
Keep `assistant_session_continuity_v1` as the owner of remembered context, but stop letting it behave like the owner of the new turn's meaning.
|
||||
|
||||
### Architectural Rule
|
||||
|
||||
The resolution order must become:
|
||||
|
||||
1. current-turn meaning;
|
||||
2. continuity snapshot;
|
||||
3. explicit arbitration between them.
|
||||
|
||||
Not:
|
||||
|
||||
1. continuity snapshot;
|
||||
2. local heuristics;
|
||||
3. hope that the new wording wins.
|
||||
|
||||
### Core Override Policy
|
||||
|
||||
The runtime must implement these hard rules:
|
||||
|
||||
1. New explicit entity beats stale entity carryover.
|
||||
2. New explicit action family beats stale previous intent.
|
||||
3. New explicit supported meaning may reuse root scope, but may not silently replay the previous object route.
|
||||
4. New explicit unsupported-but-understood meaning must degrade to truthful clarification or bounded unsupported reply, not previous exact replay.
|
||||
5. Carryover may enrich missing filters only after the current-turn meaning has been accepted as compatible.
|
||||
|
||||
### Critical Anti-Defect Targets
|
||||
|
||||
This block specifically targets failures like:
|
||||
|
||||
- `какой оборот был свк` returning old documents for `Чепурнов`;
|
||||
- short retarget turns being absorbed by the previous route;
|
||||
- clarification state surviving longer than the resumed business question;
|
||||
- root-only context and object context leaking into each other.
|
||||
|
||||
### Stack Mapping
|
||||
|
||||
Primary owner:
|
||||
|
||||
- [assistantTransitionPolicy.ts](/x:/1C/NDC_1C/llm_normalizer/backend/src/services/assistantTransitionPolicy.ts:365)
|
||||
|
||||
Shared authority helpers:
|
||||
|
||||
- [assistantContinuityPolicy.ts](/x:/1C/NDC_1C/llm_normalizer/backend/src/services/assistantContinuityPolicy.ts:471)
|
||||
- [assistantContinuityPolicy.ts](/x:/1C/NDC_1C/llm_normalizer/backend/src/services/assistantContinuityPolicy.ts:927)
|
||||
|
||||
Main refactoring target inside the stack:
|
||||
|
||||
- move from "carry previous intent unless something obviously blocks it"
|
||||
- to "carry previous context only when the current-turn meaning explicitly allows it."
|
||||
|
||||
### Done Criteria
|
||||
|
||||
This block is complete only when:
|
||||
|
||||
- stale replay of a previous exact answer becomes structurally impossible when a new explicit meaning is present;
|
||||
- root context, object context, and clarification state are all interpreted through the same authority order;
|
||||
- follow-up policy can explain why carryover was allowed or denied through machine-readable arbitration output.
|
||||
|
||||
## Big Block 3. Semantic Robustness For Supported Business Questions
|
||||
|
||||
### Goal
|
||||
|
||||
Make already-supported business contours resilient to the way real users speak:
|
||||
|
||||
- typos;
|
||||
- compressed colloquial wording;
|
||||
- short retarget turns;
|
||||
- natural business shorthand.
|
||||
|
||||
### Architectural Rule
|
||||
|
||||
Semantic robustness is not a post-processing UX lane.
|
||||
|
||||
It is part of the runtime contract for entering the correct contour at all.
|
||||
|
||||
### Required Shift
|
||||
|
||||
The intent layer must stop behaving like a direct capability picker under brittle wording.
|
||||
|
||||
It should first resolve a stable business family such as:
|
||||
|
||||
- receivables snapshot
|
||||
- payables snapshot
|
||||
- counterparty documents
|
||||
- counterparty turnover/value
|
||||
- inventory root
|
||||
- inventory selected-object drilldown
|
||||
- VAT liability / VAT as-of
|
||||
|
||||
Only after that should the runtime bind to the concrete capability.
|
||||
|
||||
### Why This Matters
|
||||
|
||||
For `кто намс должен денег на сегодня`, the system should still recognize:
|
||||
|
||||
- the user clearly asks for receivables snapshot;
|
||||
- the typo does not create a new business meaning;
|
||||
- therefore the route may proceed into the supported contour.
|
||||
|
||||
For `какой оборот был свк`, the system should still recognize:
|
||||
|
||||
- this is a new counterparty value/turnover ask;
|
||||
- this is not the same action family as `show documents`;
|
||||
- even if the exact capability is incomplete, stale document replay is forbidden.
|
||||
|
||||
### Stack Mapping
|
||||
|
||||
Primary owner seams:
|
||||
|
||||
- [addressIntentResolver.ts](/x:/1C/NDC_1C/llm_normalizer/backend/src/services/addressIntentResolver.ts:1903)
|
||||
- [addressCounterpartyIntentSignals.ts](/x:/1C/NDC_1C/llm_normalizer/backend/src/services/addressCounterpartyIntentSignals.ts:145)
|
||||
|
||||
Primary design changes:
|
||||
|
||||
- reduce direct route/capability dependence on brittle wording branches;
|
||||
- expand family-level semantic matching over normalized lexical evidence;
|
||||
- explicitly separate:
|
||||
- `supported_and_bindable`
|
||||
- `supported_family_but_needs_clarification`
|
||||
- `understood_but_capability_not_ready`
|
||||
- `truly_non_domain`
|
||||
|
||||
### Done Criteria
|
||||
|
||||
This block is complete only when:
|
||||
|
||||
- supported root questions survive light typo/noise without route collapse;
|
||||
- short new object retargets are recognized as new meaning rather than stale follow-up echoes;
|
||||
- family-level meaning can remain stable even when exact capability binding still requires clarification.
|
||||
|
||||
## Big Block 4. Answer Authority Must Reflect Current Meaning
|
||||
|
||||
### Goal
|
||||
|
||||
Make the user-facing answer prove that the system understood the current question, not merely that it found some nearby route.
|
||||
|
||||
### Architectural Rule
|
||||
|
||||
Top-of-answer semantics should be constrained by:
|
||||
|
||||
- current-turn meaning authority;
|
||||
- truth gate result;
|
||||
- continuity compatibility result.
|
||||
|
||||
Not by:
|
||||
|
||||
- whatever historical route happened to stay alive.
|
||||
|
||||
### Product-Level Requirement
|
||||
|
||||
The first screen of the answer must make a new user feel:
|
||||
|
||||
- the system understood the question;
|
||||
- the answer belongs to this question;
|
||||
- the system is not replaying stale context by mistake.
|
||||
|
||||
### Stack Mapping
|
||||
|
||||
Primary owner seams:
|
||||
|
||||
- [composeStage.ts](/x:/1C/NDC_1C/llm_normalizer/backend/src/services/address_runtime/composeStage.ts:1)
|
||||
- [answerComposer.ts](/x:/1C/NDC_1C/llm_normalizer/backend/src/services/answerComposer.ts:1)
|
||||
- [assistantTruthAnswerPolicyRuntimeAdapter.ts](/x:/1C/NDC_1C/llm_normalizer/backend/src/services/assistantTruthAnswerPolicyRuntimeAdapter.ts:1)
|
||||
|
||||
Required change:
|
||||
|
||||
- answer shaping should read current-turn meaning authority as an explicit input;
|
||||
- user-facing top blocks should not open with a previous route family when the current turn changed object or action.
|
||||
|
||||
### Done Criteria
|
||||
|
||||
This block is complete only when:
|
||||
|
||||
- answer top blocks reliably match the current asked action and object;
|
||||
- truthful limited answers do not look like stale replay;
|
||||
- human answer quality becomes a structural acceptance dimension, not a soft preference.
|
||||
|
||||
## Concrete Stack Plan
|
||||
|
||||
This problem should be addressed in the current stack through four large architecture blocks, not through many micro-passes.
|
||||
|
||||
### Stack Block A. Turn Meaning Layer
|
||||
|
||||
Introduce a new owner for current-turn semantic authority:
|
||||
|
||||
- `assistantTurnMeaningPolicy.ts`
|
||||
|
||||
Responsibilities:
|
||||
|
||||
- combine raw wording, repaired wording, LLM predecompose hints, semantic extraction contract, and lexical family signals;
|
||||
- emit one authoritative `assistant_turn_meaning_v1` object;
|
||||
- expose explicit signals for:
|
||||
- new entity switch
|
||||
- new action switch
|
||||
- current-turn family confidence
|
||||
- stale replay forbidden
|
||||
|
||||
### Stack Block B. Route + Transition Arbitration Layer
|
||||
|
||||
Rewire the hot path so both route and follow-up logic read the same turn meaning authority first.
|
||||
|
||||
Primary files:
|
||||
|
||||
- [assistantRoutePolicy.ts](/x:/1C/NDC_1C/llm_normalizer/backend/src/services/assistantRoutePolicy.ts:382)
|
||||
- [assistantTransitionPolicy.ts](/x:/1C/NDC_1C/llm_normalizer/backend/src/services/assistantTransitionPolicy.ts:365)
|
||||
- [assistantContinuityPolicy.ts](/x:/1C/NDC_1C/llm_normalizer/backend/src/services/assistantContinuityPolicy.ts:927)
|
||||
|
||||
Required result:
|
||||
|
||||
- carryover becomes conditional on semantic compatibility;
|
||||
- new explicit meaning becomes authoritative by design;
|
||||
- organization clarification becomes resumable context, not route-dominating noise.
|
||||
|
||||
### Stack Block C. Intent-Family Hardening Layer
|
||||
|
||||
Strengthen the business family layer before final capability binding.
|
||||
|
||||
Primary files:
|
||||
|
||||
- [addressIntentResolver.ts](/x:/1C/NDC_1C/llm_normalizer/backend/src/services/addressIntentResolver.ts:1903)
|
||||
- [addressCounterpartyIntentSignals.ts](/x:/1C/NDC_1C/llm_normalizer/backend/src/services/addressCounterpartyIntentSignals.ts:145)
|
||||
|
||||
Required result:
|
||||
|
||||
- supported contours remain stable under colloquial and typo-noisy wording;
|
||||
- counterparty value/turnover families do not collapse back into `documents_by_counterparty`;
|
||||
- unresolved family-level meaning degrades truthfully instead of replaying stale exact output.
|
||||
|
||||
### Stack Block D. Answer And Acceptance Layer
|
||||
|
||||
Make user-facing correctness part of the architecture gate.
|
||||
|
||||
Primary files and artifacts:
|
||||
|
||||
- [composeStage.ts](/x:/1C/NDC_1C/llm_normalizer/backend/src/services/address_runtime/composeStage.ts:1)
|
||||
- [answerComposer.ts](/x:/1C/NDC_1C/llm_normalizer/backend/src/services/answerComposer.ts:1)
|
||||
- [domain_truth_harness.py](/x:/1C/NDC_1C/scripts/domain_truth_harness.py:1)
|
||||
- [scenario_acceptance_policy.py](/x:/1C/NDC_1C/scripts/scenario_acceptance_policy.py:1)
|
||||
|
||||
Required result:
|
||||
|
||||
- live replay acceptance starts from semantic human review first;
|
||||
- top-block answer correctness becomes part of acceptance;
|
||||
- "route technically matched" no longer overrules semantic mismatch.
|
||||
|
||||
## Required Acceptance Invariants
|
||||
|
||||
The architecture should not be considered corrected until the following invariants are green:
|
||||
|
||||
1. `supported_root_question_survives_light_typo_noise`
|
||||
2. `explicit_new_entity_overrides_stale_entity_carryover`
|
||||
3. `explicit_new_action_overrides_previous_route_family`
|
||||
4. `unsupported_new_meaning_does_not_replay_previous_exact_answer`
|
||||
5. `organization_clarification_resumes_pending_business_question`
|
||||
6. `short_followup_retains_dialog_stem_without_glitch_replay`
|
||||
7. `answer_top_block_matches_current_user_intent`
|
||||
8. `meta_interrupt_does_not_corrupt_business_thread`
|
||||
|
||||
## Execution Rule
|
||||
|
||||
Do not implement this plan as:
|
||||
|
||||
- scattered regex patching;
|
||||
- isolated answer-wording cleanup;
|
||||
- one more local fix inside `assistantService.ts`;
|
||||
- a stealth rewrite toward a new monolith.
|
||||
|
||||
Implement it as:
|
||||
|
||||
- one shared current-turn meaning authority;
|
||||
- one explicit arbitration rule between new meaning and continuity;
|
||||
- stronger family-level semantic robustness for supported contours;
|
||||
- answer and replay gates that prove the assistant now feels alive to a human user.
|
||||
|
||||
## Bottom Line
|
||||
|
||||
The assistant does not currently fail mainly because it lacks "knowledge".
|
||||
|
||||
It fails because it still lacks a stable architecture for:
|
||||
|
||||
- recognizing the meaning of the current turn;
|
||||
- subordinating continuity to that meaning;
|
||||
- and reflecting that meaning in the final user-visible answer.
|
||||
|
||||
That is the next large architecture block.
|
||||
|
||||
Not a cosmetic one.
|
||||
|
||||
Not a prompt one.
|
||||
|
||||
The assistant will start feeling alive only when current-turn meaning becomes a first-class runtime authority across the whole stack.
|
||||
|
|
@ -31,6 +31,7 @@ This package answers the next question:
|
|||
11. [11 - continuity_stabilization_plan_2026-04-17.md](./11%20-%20continuity_stabilization_plan_2026-04-17.md)
|
||||
12. [12 - manual_run_system_analysis_3NilqwT1G2_2026-04-18.md](./12%20-%20manual_run_system_analysis_3NilqwT1G2_2026-04-18.md)
|
||||
13. [13 - pre_multidomain_readiness_audit_2026-04-18.md](./13%20-%20pre_multidomain_readiness_audit_2026-04-18.md)
|
||||
14. [14 - semantic_dialog_authority_recovery_plan_2026-04-19.md](./14%20-%20semantic_dialog_authority_recovery_plan_2026-04-19.md)
|
||||
|
||||
## Current Status Snapshot (2026-04-19)
|
||||
|
||||
|
|
@ -87,6 +88,7 @@ For the detailed audit, current percentages, and remaining debt, read:
|
|||
- [11 - continuity_stabilization_plan_2026-04-17.md](./11%20-%20continuity_stabilization_plan_2026-04-17.md)
|
||||
- [12 - manual_run_system_analysis_3NilqwT1G2_2026-04-18.md](./12%20-%20manual_run_system_analysis_3NilqwT1G2_2026-04-18.md)
|
||||
- [13 - pre_multidomain_readiness_audit_2026-04-18.md](./13%20-%20pre_multidomain_readiness_audit_2026-04-18.md)
|
||||
- [14 - semantic_dialog_authority_recovery_plan_2026-04-19.md](./14%20-%20semantic_dialog_authority_recovery_plan_2026-04-19.md)
|
||||
|
||||
## Architectural Objects Of Planning
|
||||
|
||||
|
|
@ -118,6 +120,7 @@ Read in this order:
|
|||
12. `11 - continuity_stabilization_plan_2026-04-17.md`
|
||||
13. `12 - manual_run_system_analysis_3NilqwT1G2_2026-04-18.md`
|
||||
14. `13 - pre_multidomain_readiness_audit_2026-04-18.md`
|
||||
15. `14 - semantic_dialog_authority_recovery_plan_2026-04-19.md`
|
||||
|
||||
## Planning Rules
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue